Internet Interventions 1 (2014) 216–224
Contents lists available at ScienceDirect
Internet Interventions journal homepage: www.invent-journal.com/
The use of automated assessments in internet-based CBT: The computer will be with you shortly Elizabeth C. Mason ⁎, Gavin Andrews Clinical Research Unit for Anxiety and Depression, School of Psychiatry, University of New South Wales, St Vincent's Hospital, Level 4, O'Brien Centre, 394-404 Victoria Street, Darlinghurst, NSW 2010, Australia
a r t i c l e
i n f o
Article history: Received 11 July 2014 Received in revised form 30 October 2014 Accepted 30 October 2014 Available online 5 November 2014 Keywords: iCBT Etherapy Anxiety Depression Cognitive behavioral therapy Assessment
a b s t r a c t There is evidence from randomized control trials that internet-based cognitive behavioral therapy (iCBT) is efficacious in the treatment of anxiety and depression, and recent research demonstrates the effectiveness of iCBT in routine clinical care. The aims of this study were to implement and evaluate a new pathway by which patients could access online treatment by completing an automated assessment, rather than seeing a specialist health professional. We compared iCBT treatment outcomes in patients who received an automated pre-treatment questionnaire assessment with patients who were assessed by a specialist psychiatrist prior to treatment. Participants were treated as part of routine clinical care and were therefore not randomized. The results showed that symptoms of anxiety and depression decreased significantly with iCBT, and that the mode of assessment did not affect outcome. That is, a pre-treatment assessment by a psychiatrist conferred no additional treatment benefits over an automated assessment. These findings suggest that iCBT is effective in routine care and may be implemented with an automated assessment. By providing wider access to evidence-based interventions and reducing waiting times, the use of iCBT within a stepped-care model is a cost-effective way to reduce the burden of disease caused by these common mental disorders. © 2014 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
1. Introduction Anxiety and depressive disorders are the most common mental health disorders (Kessler et al., 2005; Andrews et al., 2001). They can be treated effectively using cognitive behavioral therapy (CBT; Roy-Byrne and Cowley, 2007; Barlow, 2008) as well as medication (e.g., selective serotonin reuptake inhibitors; Montgomery, 2006; Van Der Linden et al., 2000; Gorman and Kent, 1999). Despite the success of these interventions, there are significant barriers to treatment access and long-term treatment outcomes. Specifically, many patients are reluctant to take medication and relapse rates are elevated once medication is terminated (e.g., Gould et al., 1995, 1997). Although the effects of CBT outlast the treatment itself, it is widely recognized that there are simply not enough trained clinicians to cater for the number of patients who could benefit from it (Lovell and Richards, 2000; Williams and Martinez, 2008). This discrepancy between supply and demand has led to lengthy waiting lists; indeed, waiting times from between 6 and 12 months are not unusual (Lovell and Richards, 2000; Shapiro et al., 2003). Internet-based treatments, which greatly facilitate the dissemination of evidence-based interventions, provide a valuable tool for
⁎ Corresponding author. Tel.: +61 2 8382 1400; fax: +61 2 8382 1402. E-mail address:
[email protected] (E.C. Mason).
overcoming this difficulty and it is clear that the use of internet-based treatments has increased dramatically over recent years. Internet-based cognitive behavioral therapy (iCBT) has been shown to be efficacious in the treatment of anxiety and depression. Specifically, the efficacy of iCBT has been demonstrated in the treatment of Generalized Anxiety Disorder (Robinson et al., 2010*, Cuijpers et al., 2009),1Social Anxiety Disorder (Titov et al., 2008b*, Titov et al., 2009a*, Titov et al., 2009b*, Carlbring et al., 2009), Panic Disorder with and without Agoraphobia (Wims et al., 2010*, Carlbring et al., 2006), Depression (Andersson and Cuijpers, 2009; Perini et al., 2009*), and mixed anxiety and depression (Titov et al., 2011*, Newby et al., 2013*). A recent metaanalysis of 22 studies demonstrated that iCBT had a mean effect size of 0.88 (Number needed to treat = 2.13; Andrews et al., 2010). Moreover, internet treatment has been shown to be as effective as face-to-face treatment (for review, see Andrews et al., 2010). Further, iCBT is highly acceptable to patients and adherence rates are high (Andrews et al., 2010). As one would imagine, iCBT is also more cost-effective than face-to-face treatment. There is now also evidence which demonstrates the effectiveness of iCBT in clinical practice (Williams and Andrews, 2013; Mewton et al.,
1 *References marked with an asterisk are published efficacy trials of the courses used in the current study.
http://dx.doi.org/10.1016/j.invent.2014.10.003 2214-7829/© 2014 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
E.C. Mason, G. Andrews / Internet Interventions 1 (2014) 216–224
2012; Hilvert-Bruce et al., 2012; Sunderland et al., 2012; Newby et al., 2013, 2014; Hedman et al., 2013, 2014; Bergstrom et al., 2009; Ruwaard et al., 2012). We currently have two routes by which patients can access iCBT in routine clinical care — (1) Patients can obtain a referral for a specialist face-to-face assessment at our clinic and then receive supervision through the course by one of the clinicians working at the clinic; or (2) Patients can receive a “prescription” for the iCBT course by their general practitioner (GP) who is registered with This Way Up Clinic. In that pathway, the “prescribing” GP also supervises them through the course. There are barriers to the uptake of treatment in both of these pathways. In the specialist pathway, patients are required to attend a face-to-face clinical interview with a specialist psychiatrist, which is both costly to the health system and time-consuming for patients. In the primary care pathway, doctors may feel that they lack the time or expertise to supervise patients through the course and thus be less inclined to prescribe an iCBT course for a patient presenting with an anxiety or depression disorder. Both of these factors limit the extent to which evidence-based iCBT treatments are used. In this study, we sought to implement and evaluate a new pathway in order to overcome these potential barriers to the uptake of iCBT. Specifically, we designed a pathway by which patients could access online treatment by completing an automated assessment, rather than seeing a specialist health professional, and receive supervision from a clinician at our clinic rather than from their GP. In this “automated” pathway, the patient's GP in the community referred the patient for an online assessment to determine suitability for iCBT and course recommendation. Patients in that pathway received supervision through the course by a clinician at our clinic, rather than their referring GP. In order to evaluate this new pathway, treatment outcomes from this group of patients, who completed an automated assessment, were compared to treatment outcomes for patients who were seen for a face-to-face assessment. 2. Method
217
distress, fear, avoidance, and impairment in functioning. This scale ranges from 0 to 28, and a score of 9 or higher indicates a score in the clinical range (Shear et al., 2001). It has very good internal consistency (Cronbach's α = 0.92) and test–retest reliability (Houck et al., 2002). 2.2.3. Social Anxiety Disorder Social Anxiety Disorder was assessed by the Mini-Social Phobia Inventory (Mini-SPIN; Connor et al., 2001). This is a 3-item questionnaire, in which items are rated on a 5-point scale. Scores range from 0 to 12 and clinical Social Anxiety is indicated by a score of 6 or above (Weeks et al., 2007). The scale has been shown to have sound psychometric properties, with good internal consistency (Cronbach's α = 0.85) and reasonable convergent and discriminant validity (Weeks et al., 2007). 2.2.4. Depression Depression was assessed by the Patient Health Questionnaire (PHQ-9; Kroenke et al., 2001). This is a 9-item self-report scale in which participants rate the frequency of symptoms over the past fortnight. Scores range from 0 to 27, and a score of 10 or higher indicates clinically significant levels of Depression (Zuithoff et al., 2010). The PHQ-9 has been shown to have good sensitivity and specificity (Wittkampf et al., 2007) and excellent reliability (Cronbach's α = 0.86–0.89; test–retest reliability = 0.84) and validity (Kroenke et al., 2001). 2.2.5. General levels of psychological distress General levels of psychological distress were assessed by the Kessler Psychological Distress Scale (K10; Kessler et al., 2002). This 10-item questionnaire measures non-specific psychological distress over the past two weeks. Scores range from 10 to 50. Individuals who score less than 20 are likely to be well; scores between 20 and 24 indicate a mild level of distress; scores between 25 and 29 indicate a moderate level of distress; and scores above 30 indicate severe levels of distress. The K10 has excellent psychometric properties, with high internal consistency (Cronbach's α = 0.93; Kessler et al., 2002).
2.1. Participants Participants were 308 patients who enrolled in an online CRUfADclinic (now known as This Way Up Clinic) iCBT course between October 2010 and September 2011. 135 of these patients came through the specialist assessment pathway and the remaining 173 patients came through the automated assessment pathway. Note that participants were not randomized but rather this was a naturalistic study of these two modes of assessment that already existed in the clinic. Patients with psychosis, bipolar disorder, or substance abuse/dependence, as well as patients who were taking benzodiazepines or were actively suicidal were excluded from treatment. See Fig. 1 for information on patient flow, including patient inclusion/exclusion and drop-outs, over the course of the study. 2.2. Measures 2.2.1. Generalized Anxiety Disorder (GAD) Generalized Anxiety Disorder (GAD) was assessed by the Generalized Anxiety Disorder Scale-7 (GAD-7; Spitzer et al., 2006). The GAD-7 is a 7-item measure of the symptoms and severity of GAD (Lowe et al., 2008). Participants rate each item on a 4-point scale, based on their symptoms over the past two weeks. Scores range from 0 to 21 and a score of 10 or higher indicates clinical levels of GAD. The scale has been shown to have good internal consistency (Cronbach's α = 0.92) and test–retest reliability (0.83), as well as criterion and construct validity for the diagnosis of GAD (Kroenke et al., 2007; Spitzer et al., 2006). 2.2.2. Panic Disorder with or without Agoraphobia Panic Disorder with or without Agoraphobia was assessed by the 7item Panic Disorder Symptom Scale, Self Report form (PDSS-SR; Houck et al., 2002). Items assess the frequency of panic attacks as well as
2.2.6. Disability Disability was assessed by the World Health Organization Disability Assessment II (WHODAS-II; Andrews et al., 2009). This is a 12-item scale which assesses functional impairment or activity limitation that is associated with physical illness, mental illness or a combination of both. Scores range from 0 to 48, with higher scores indicating higher functional impairment/activity limitation. A score of 10 or above is thought to indicate clinically significant disability (Andrews et al., 2009). The WHODAS-II has been shown to possess excellent psychometric properties, with good internal consistency (Cronbach's α = 0.86) and test–retest reliability (0.98) as well as good concurrent validity (Borkovec et al., 2004). 2.2.7. Suicidality Suicidality was assessed in the specialist assessment pathway by the intake psychiatrist. Suicidality in the automated assessment pathway was assessed by the following screening question — In the last month, have you thought that your life is not worth living? If patients answered yes to that question, they were asked the following three questions — (1) In the last month, have you thought about ending your life?; (2) In the last month, have you made a plan as to how you might do this?; and (3) Have you ever in your life made a suicide attempt? Patients who answered yes to any of the latter three questions were excluded from treatment. 2.3. Procedure 2.3.1. Assessment 2.3.1.1. Automated assessment group. Participants were referred to CRUfADclinic (www.crufadclinic.org), now known as This Way Up Clinic
218
E.C. Mason, G. Andrews / Internet Interventions 1 (2014) 216–224
Fig. 1. Participant flow.
(www.thiswayupclinic.org), by their doctor (General Practitioner, Psychiatrist, or Primary Health Care Physician). These participants were provided with a “Referral” code to allow them to access the system. Once participants logged on to CRUfADclinic, they completed a battery of self-report questionnaires (see Measures). Based on their scores on the questionnaires, participants were recommended to do one of the five iCBT courses (see treatment section below for details). A simple algorithm was used to determine which course would be recommended to the patient by the online system — If the patient scored above the clinical threshold on only one of the symptom questionnaires (PHQ-9, GAD-7, Mini-SPIN, or PDSS-SR), they were recommended the corresponding course. For example, if the patient scored above threshold on PHQ-9 only, they were recommended the Depression course. If patients scored above the clinical threshold on more than one measure or below threshold on all measures, they were recommended the Transdiagnostic mixed anxiety and Depression course. 2.3.1.2. Specialist assessment group. Participants were referred by their doctor (General Practitioner, Psychiatrist, or Primary Health Care Physician) for assessment at CRUfAD at St Vincent's Hospital. Patients arrived at the clinic and completed the questionnaire battery on a computer. They were then seen by a psychiatrist for a clinical interview to
determine their diagnosis and treatment options. If iCBT was deemed to be appropriate for the patient, they were provided with a “Prescription” for the online course, which indicated the prescribed course and a password in order to access the course. Patients were assigned to one of the following five courses based on their primary diagnosis. The K10, WHODAS-II, and course-specific measures were again administered immediately post-treatment for patients who did iCBT. That is, in addition to the K10 and the WHODAS-II, patients who completed the Social Anxiety course also completed the Mini-SPIN post-treatment, patients who completed the Panic disorder course also completed the PDSS-SR, etc., and patients who completed the Transdiagnostic mixed anxiety and Depression course also completed the Mini-SPIN, GAD-7, PDSS-SR, and the PHQ-9. Questionnaires were re-administered online immediately following the completion of the final lesson, but prior to the download of the homework for that lesson, to ensure that responses to these questionnaires would be obtained. This study was approved as part of the quality assurance activities undertaken by the Patient Safety and Quality Unit at St. Vincent's Hospital, Sydney, with whom a copy of this manuscript has been lodged. Prior to enrolment in any of the treatment programs, all individuals were informed that data will be collected and used as per the following: “By participating in This Way Up clinic, you acknowledge that your data will
E.C. Mason, G. Andrews / Internet Interventions 1 (2014) 216–224
219
be pooled, analysed and periodically published in scientific articles to enhance scientific knowledge in anxiety and depression. In any publication, information will be provided in such a way that you cannot be identified”. Patients could opt out of the use of their data for these purposes via email. All patients provided electronic informed consent that their pooled data could be used for these purposes. 2.3.2. Treatment Patients who were accepted for iCBT were able to enroll in the course immediately. Patients in both groups could choose a course other than the recommended/prescribed one, though this was not encouraged. There were five courses available to patients — Depression; Panic Disorder; Generalized Anxiety Disorder; Social Anxiety Disorder; and the Transdiagnostic course for mixed anxiety and depression. The course cost patients $44AUD for 90 day access. Courses varied in length, consisting of 5 or 6 lessons. Each lesson consists of a comic involving a character with the relevant diagnosis who learns about their disorder and the CBT skills required to manage their symptoms. Patients then download a homework summary, which consists of psychoeducation and various exercises such as cognitive restructuring and developing an exposure hierarchy. Patients were encouraged to spend between 3 and 4 h a week on the course. Patients logged on to complete each lesson. They were able to complete lessons at their own pace with the limitations that the first two lessons needed to be completed within 30 days of registering and the entire course needed to be completed within 90 days, although extensions were granted in some cases to allow patients who were slow to complete the lessons to finish the course (see Results). During the course, all patients received email contact from their clinician (a clinical psychologist or psychiatry registrar) working at the Anxiety Disorders Clinic at CRUfAD. Patients could contact their clinician by email or phone throughout treatment, though email was the most frequently used form of contact. At the very least, clinicians contacted their patients via email after the first, second, and last lessons. Clinicians also responded to any patient contact made. Prior to each lesson, patients completed the K10 questionnaire to assess their level of emotional distress. Clinicians contacted patients who exhibited an increase of 5 points or an increase to a score of 30 or above on this measure between lessons. Clinicians primarily contacted patients to encourage them to complete the lessons and to respond to questions clarifying the content of the course. 2.4. Data analysis Cohen's d within-group effect sizes were calculated based on the pooled standard deviations, and adjusted for the correlation between the means. When reporting independent t-tests, in cases where the assumption of equal variances between groups was violated (as indicated by Levine's Test for Equality of Variances), the nominal degrees of freedom and adjusted p-value are reported. These cases are indicated by the subscript UV (i.e., Unequal Variances), and appear as pUV.
Fig. 3. Percentage of patients in each group allocated to each course.
3. Results 3.1. Demographics 51.9% of the specialist assessment group were female (n = 70) and 72.3% of the automated assessment group were female (n = 125). The mean age of patients in the specialist assessment group was 36 years (SD = 12) and the mean age in the automated assessment group was 40 years (SD = 14). 3.2. Pre-treatment severity Mean scores across measures at baseline are shown in Fig. 2. Patients in the specialist assessment group reported more distress than patients in the automated assessment group [t(271) = 2.56, pUV = 0.015]. Patients in the specialist group also reported more severe symptoms of GAD [t(272) = 2.19, p = 0.03] and Panic Disorder [t(276) = 4.53, pUV b 0.001] than patients in the automated group. There were no significant differences between groups on measures of Depression [t(271) = 1.91, pUV = 0.07], Social Anxiety [t(263) = 1.75, pUV = 0.09], or Disability [t(302) = 0.56, p = 0.57]. 3.3. Course allocation The percentage of patients in each course is shown in Fig. 3. 3.4. Adherence and length of course 67.4% of patients in the specialist assessment group completed their course and 61.3% of patients in the automated assessment pathway completed their course. This difference between groups was not significant (χ2 = 1.24, p = 0.27).
Fig. 2. Mean pre-treatment scores for patients in the specialist assessment and automated assessment groups: (a) Symptom scores; (b) distress and disability. N.B. Error bars in all graphs represent the standard error of the mean (SEM). * indicates a significant difference between groups.
220
E.C. Mason, G. Andrews / Internet Interventions 1 (2014) 216–224
Fig. 4. Pre- and post-treatment (a) Distress across courses; (b) disability across courses.
31.9% of specialist assessment patients and 17.3% of automated assessment patients were provided with extensions to allow them to complete the course. This was a significant difference between groups (χ2 = 8.83, p = 0.003). When the analyses were restricted to patients who were not given course length extensions, 60.9% of patients in the specialist assessment group completed their course and 59.4% of patients in the automated assessment group completed their course, suggesting that the difference between groups in the number of extensions given did not have a meaningful impact on adherence rates.
3.5. Post-treatment symptom scores Mean post-treatment symptom scores for treatment completers are shown in Figs. 4 and 5. Relative to pre-treatment scores, at posttreatment, both groups reported significant reductions in distress, [F(1, 173) = 187.24, p b 0.001] and disability [F(1, 195) = 84.69, p b 0.001]. There was also a time-by-group interaction on the measure of distress [F(1, 173) = 5.77, p = 0.02], reflecting a difference between groups at baseline [t(173) = 2.24, p = 0.03] but not at post-treatment [t(195) = 0.16, p = 0.88]. There was no time-by-group interaction on the measure of disability [F(1, 195) = 1.46, p = 0.23] and no main effect of group on either of these measures [largest F(1, 173) = 1.71, p = 0.19]. Collapsing across groups, the within-group effect size for reductions in distress over treatment was 1.05 (95% confidence interval, CI: 0.83–1.25) and the effect size for reductions in disability was 0.54 (95% CI: 0.34–0.84). At the end of treatment, 66.0% of participants scored in the “well” range on the K10, 17.3% scored in the mild range, and the remaining 16.8% scored in the moderate or severe range. At the end of treatment, 57.9% of participants scored in the range indicating no clinically significant disability on the WHODAS. Patients in both groups who completed the Depression course reported significant reductions in their Depression severity [F(1, 14) = 21.33, p b 0.001]. There was also a significant group-by-time interaction [F(1, 14) = 5.35, p = 0.04], reflecting a difference between groups at baseline2 [t(14) = 3.01, p = 0.009] but not at post-treatment on this measure [t(16) = 0.47, p = 0.64]. There was no main effect of group [F(1, 14) = 1.07, p = 0.32]. Collapsing across groups, the within-group effect size (Cohen's d) for reductions in Depression over treatment in patients who completed the Depression course was 1.10 (95% CI: 0.40–1.80). At the end of treatment, 72.2% of participants who completed the Depression course scored in the non-clinical range on the PHQ-9. Patients in both groups who completed the GAD course reported significant reductions in their GAD severity [F(1, 45) = 68.09, p b 0.001]. There was no interaction [F b 1] and no main effect of group [F(1, 45) = 2.62, p = 0.11]. Collapsing across groups, the within-group effect size (Cohen's d) for reductions in GAD over treatment in patients who completed the GAD course was 1.14 (95% CI: 2 Note that this difference between groups wasn't present in the pre-treatment scores which included all patients. That is, this difference was only apparent in the subset of patients who completed the Depression course.
0.73–1.55). At the end of treatment, 84.3% of participants who completed the GAD course scored in the non-clinical range on the GAD-7. Patients in both groups who completed the Panic course reported significant reductions in their Panic Disorder severity [F(1, 31) = 51.24, p b 0.001]. There was no interaction [F(1, 31) = 2.14, p = 0.15] and no main effect of group [F b 1]. Collapsing across groups, the within-group effect size (Cohen's d) for reductions in Panic symptoms over treatment in patients who completed the Panic course was 1.46 (95% CI: 0.97–1.95). At the end of treatment, 69.2% of participants who completed the Panic course scored in the non-clinical range on the PDSS-SR. Patients in both groups who completed the Social Anxiety course reported significant reductions in their Social Anxiety severity [F(1, 34) = 57.27, p b 0.001]. There was no interaction and no main effect of group [both Fs b 1]. Collapsing across groups, the within-group effect size (Cohen's d) for reductions in Social anxiety over treatment in patients who completed the Social course was 1.33 (95% CI: 0.87–1.79). At the end of treatment, 48.8% of participants who completed the Social Anxiety course scored in the non-clinical range on the Mini-SPIN. Patients in both groups who completed the Transdiagnostic mixed anxiety and Depression course reported significant reductions in symptoms of Depression [F(1, 38) = 20.75, p b 0.001], (d = 0.86; 95% CI: 0.42–1.30), GAD [F(1, 38) = 20.09, p b 0.001], (d = 0.94; 95% CI: 0.50–1.38), Social Anxiety [F(1, 37) = 8.76, p = 0.005], (d = 0.47; 95% CI: 0.03–0.91), and Panic Disorder [F(1, 38) = 6.13, p = 0.018], (d = 0.55; 95% CI: 0.11–0.99). There were no significant time-bygroup interactions [largest F(1, 38) = 1.06, p = 0.31] and no main effects of group across measures [all Fs b 1]. At the end of treatment, amongst patients who completed the Transdiagnostic anxiety and Depression course, 78.3% scored in the non-clinical range on the PHQ9, 84.8% scored in the non-clinical range on the GAD-7, 73.9% scored in the non-clinical range on the PDSS-SR, and 73.9% scored in the nonclinical range on the Mini-SPIN. 3.6. Completers and non-completers — change in distress across lessons Lesson-by-lesson K10 scores for course completers and noncompleters can be found in Table 1. Collapsed across specialist and automated groups, there were no differences between completers' and non-completers' distress levels at any lesson [largest t(264), lesson 4, = 0.87, p = 0.39]. 4. Discussion This study demonstrated that internet-based cognitive behavioral therapy is effective in routine clinical care regardless of whether patients came via a pathway that involved a specialized assessment by a psychiatrist or via a pathway that involved an automated online assessment. Prior to treatment, there were minor differences between groups in terms of symptom severity. Specifically, patients in the specialist assessment group were more distressed, and also reported more symptoms of GAD and Panic Disorder than patients in the automated assessed group. Nevertheless, distress, disability and symptom
E.C. Mason, G. Andrews / Internet Interventions 1 (2014) 216–224
221
Fig. 5. Pre- and post-treatment (a) Depression severity scores for patients who completed the Depression course; (b) GAD severity scores for patients who completed the GAD course; (c) Panic Disorder severity scores for patients who completed the Panic course; (d) Social Anxiety severity scores for patients who completed the Social Anxiety course; (e) symptom scores for patients who completed the Transdiagnostic course.
severity across measures of anxiety and depression decreased significantly for both groups over treatment, and there were no differences between groups at post-treatment. 4.1. Adherence Adherence rates, defined as the completion of all lessons, for the specialist assessment group were the same as in the automated assessment group. It is of interest, however, that more extensions were granted to patients in the specialist assessment group than to patients in the automated assessment group. This may be due to the fact that at the start of treatment, the patients in the specialist assessment group were often met in person by their supervising clinician, whereas patients in the online assessment group were never met in person by their supervising clinician. This might have led the supervising clinicians to feel more obliged to give extensions to the patients they had actually met and
may have also resulted in the specialist assessed patients feeling more comfortable to request extensions than automated assessed patients. Whatever the reason for the difference between groups in the number of extensions provided, it is worth noting that the provision of extensions did not have a meaningful impact on course adherence rates. When the analyses were restricted only to patients who did not receive an extension, both groups' adherence rates were similar, and very close to those obtained when patients who were given extensions were included in the analyses. The overall adherence rate across groups in this study was 64%. Across the 22 randomized control trials of iCBT included in Andrews et al.'s (2010) meta-analysis of iCBT, the median rate of adherence was 80%. Therefore, the adherence rate in the current study is lower than those achieved in research iCBT trials, which is to be expected given that this was conducted in a routine care, where adherence has been shown to be lower than in formal clinical trials. Indeed, adherence
222
E.C. Mason, G. Andrews / Internet Interventions 1 (2014) 216–224
Table 1 K10 (distress) scores for completers and non-completers across lessons. Lesson Mean K10 score (SEM; n)
Six lesson courses
Completers
Specialist Automated
Non-completers
Specialist Automated
Five lesson courses
Completers
Specialist Automated
Non-completers
Specialist Automated
1
2
3
4
5
6
25.6 (0.7; 80) 24.1 (0.7; 90) 25.4 (1.3; 36) 25.3 (0.9; 58) 22.9 (2.6; 11) 24.5 (1.7; 16) 26.7 (4.2; 7) 23.0 (4.0; 6)
22.9 (0.7; 80) 22.5 (0.7; 90) 23.8 (1.4; 35) 23.2 (1.0; 51) 20.5 (2.0; 11) 23.1 (2.0; 16) 23.3 (4.6; 7) 18.8 (1.8; 5)
20.6 (0.7; 80) 20.7 (0.6; 90) 21.1 (1.5; 31) 21.2 (1.0; 42) 17.8 (1.7; 11) 21.3 (2.0; 16) 22.8 (6.7; 5) 16.0 (3.0; 4)
19.8 (0.8; 80) 19.5 (0.7; 90) 22.1 (1.8; 28) 19.4 (1.1; 34) 16.7 (1.5; 11) 20.1 (2.0; 16) 18.3 (5.0; 4) 17.3 (3.7; 3)
19.5 (0.8; 80) 18.9 (0.6; 90) 21.7 (2.5; 19) 18.0 (1.4; 22) 14.4 (1.3; 11) 19.6 (1.9)
18.1 (0.7; 80) 17.4 (0.6; 90) 18.0 (1.9; 13) 15.3 (2.3; 3)
in the current study was higher than has been observed in other effectiveness trials conducted in routine clinical care (Williams and Andrews, 2013; ~55% adherence, Newby et al., 2014; ~50% adherence). Recent work has examined ways in which to improve adherence rates for iCBT in routine care (Hilvert-Bruce et al., 2012). In particular, Hilvert-Bruce et al. (2012) found that the inclusion of automated patient reminders to complete lessons and a course fee, as well as the provision of course choice and the inclusion of a calendar system in which patients nominated when they would next complete a lesson, increased adherence rates to some 60%. Those strategies were included in the current study and therefore, it is clear that there is still room for improvement in adherence rates for iCBT used in routine clinical care. This field would benefit from further research to identify additional factors to improve adherence rates. As can be seen in Table 1, whether courses are five or six lessons does not appear to affect outcome. Importantly, what is also clear is that both non-completers and completers reported a reduction in distress across lessons completed. The finding that non-completers still derive some benefit from the online CBT course is consistent with what has been observed previously (Hilvert-Bruce et al., 2012). Although, the reasons for non-completion are not known, there appears to be subset of patients who do not complete treatment who have made good progress in reducing their distress and may therefore have a positive reason for concluding early. 4.2. Course allocation It is worth noting that there were differences between groups in terms of their course allocation. This might reflect a greater number of patients with Depression coming through the automated assessment route, as compared to the specialized assessment group. It also likely reflects the fact that the automated assessment was biased towards recommending the Transdiagnostic mixed anxiety and depression course, given that any comorbity led to the Transdiagnostic course being recommended. In contrast, patients who came through the specialist assessment pathway did the course recommended by the psychiatrist who assessed them, who may have been more likely to recommend the course that corresponded to the patient's primary diagnosis, with the assumption that any comorbid condition, that was perhaps secondary to the primary condition, would remit with effective treatment of the primary disorder. 4.2.1. Clinical implications This research has important implications for clinical practice in the treatment of anxiety and depression. One, it suggests that internet-
15.3 (4.1; 3) 14.0 (–; 1)
based CBT courses for anxiety and depressive disorders have utility in routine care. Two, this research suggests that specialist assessments and initial face-to-face contact do not influence treatment outcome, and that patients do just as well with an automated assessment. The use of automated assessments has clear benefits related to cost and waiting times. When reflecting on this finding, one might be tempted to wonder whether clinicians are required at all in the treatment of patients using online CBT. Although this study suggests that having a clinician at the assessment stage does not influence treatment outcome, there is reason to believe that having someone (though not necessarily a specialized clinician) to support patients through online treatment does affect the outcome of treatment. That is, guided iCBT has been shown to lead to better outcomes than non-guided iCBT. For example, in a trial examining iCBT for Social Anxiety Disorder, Titov and colleagues found that 77% of patients in the clinician-assisted group completed all of the lessons and only 33% of patients in the selfguided condition completed all lessons (Titov et al., 2008a). Despite the importance of guidance through iCBT, it appears as though the person providing the guidance does not need to be a specialized clinician. For example, Robinson et al. (2010) and Titov et al. (2010) found that regardless of whether a clinician or technician supervised patients through the online course, treatment outcome was the same, and was comparable to that associated with face-to-face treatment. One might argue that a limitation of using an automated assessment is that this approach does not afford the opportunity to provide patients who may not benefit from iCBT for anxiety and/or depression with alternative treatment recommendations, for example, if they meet criteria for another disorder or might benefit more from a different form of therapy. However, if this was a major problem in that automated assessment led to more inappropriate patients starting an iCBT course than did specialist assessment, then one would expect to have observed poorer treatment outcomes in the automated assessment group in the current study, and/or that adherence rates would have been lower in the automated group relative to the specialist group. Clearly, this was not shown to be the case in the present study. Further, a fairly simplistic automated assessment and algorithm for course recommendations were used in the current study and there is certainly scope to develop more sophisticated online assessment algorithms for treatment recommendations in this area. Indeed, a more sophisticated algorithm might better be able to distinguish between primary and secondary diagnoses and make superior treatment recommendations based on this. In addition, the use of online structured diagnostic tools might increase diagnostic reliability. It should also be noted that a standardized clinical interview such as the SCID was not used in the current
E.C. Mason, G. Andrews / Internet Interventions 1 (2014) 216–224
study and therefore it is possible that diagnosis reliability could also have been enhanced in the specialist group. Finally, it should also be pointed out that a fairly conservative approach to suicide assessment was used in the automated group and given that our group have since shown that suicidal ideation can be reduced with iCBT (Watts et al., 2012), it is likely these criteria could be less restrictive in the future. More than half a century ago, Meehl (1954) argued that statistical prediction consistently outperforms clinical judgment, for example in relation to diagnosing and planning optimal treatment. In 1965, he reviewed 51 studies that almost all confirmed this earlier conclusion (Meehl, 1965). As noted by Dawes et al. (1989), “Optimal planning and care often hinge on the consultant's judgmental accuracy” (p. 1668). Nevertheless, it is clear that many professionals, including psychiatrists and psychologists, tend to rely on their own judgment over indications provided by actuarial methods. Indeed, we predicted that patients who were assessed by a specialist would do better in treatment than those assessed by a computer because we expected that the specialists would be better at diagnosing and selecting patients who would most benefit from iCBT, as well as matching patients to the most appropriate iCBT course. The fact that both groups of patients did as well as each other in treatment suggests that clinical judgment in these circumstances provides no additional benefit over the use of an automated assessment. With the current state of knowledge in the field, however, we don't possess the information to predict accurately who will benefit from CBT, whether delivered in the traditional face-to-face format or over the internet. It is likely that once more research is conducted that provides information on factors that accurately predict who will benefit from CBT, both modes of assessment (automated/actuarial and specialist) will benefit and improve in their predictive ability to select patients who are most likely to benefit from treatment so that we can truly optimize treatment for patients with these common mental disorders. It is important to note that a key limitation of the current study was that participants were not randomized to the two groups and therefore it is possible that sample differences may play a role in these findings. The current findings would be strengthened by future research in which participants were randomized into the two forms of assessment and compared. 4.3. What about patients who don't benefit from iCBT? Of course, as with all treatments, a subset of patients drop out of treatment and even of those who do adhere, a subset will not achieve significant gains. In the clinic in which this study was conducted, iCBT is used within a stepped-care model, by which patients are first treated with iCBT and then if they require further treatment at the end of iCBT, they go on to be treated with face-to-face therapy. Although this model has intuitive practical appeal, it is unclear whether patients who do not benefit from iCBT will benefit from face-to-face CBT. This is an empirical question, which would be an interesting avenue for future research. 4.4. Conclusions In the treatment of mental disorders, it has long been believed that seeing people face-to-face is important. However, with the rapidly emerging field of internet-based interventions for mental disorders, it is now becoming increasingly accepted that therapy over the internet leads to significant therapeutic gains. Nevertheless, a specialized assessment in person or over the phone continues to be the norm in this field. In this study, we showed that not only can patients in real-world settings benefit significantly from online cognitive behavioral therapy for anxiety and depression, but that the pre-treatment assessment can also be completed online, without any therapist involvement, without compromising treatment outcome. These findings have implications for the treatment of anxiety and depressive disorders in routine clinical practice. In particular, they suggest that specialized mental health clinicians may not be essential for the online treatment of anxiety and
223
depression. Using an automated assessment and online treatment can greatly lessen the burden of disease related to anxiety and depression by reducing waiting times and significantly increasing access to evidence-based interventions. Future work should continue to examine further factors that influence treatment outcome, and ways in which to increase adherence rates in real-world settings.
References Andersson, G., Cuijpers, P., 2009. Internet-based and other computerized psychological treatments for adult depression: a meta-analysis. Cogn. Behav. Ther. 38, 196–205. Andrews, G., Henderson, S., Hall, W., 2001. Prevalence, comorbidity, disability and service utilisation: overview of the Australian National Mental Health Survey. Br. J. Psychiatry 178, 145–153. Andrews, G., Kemp, A., Sunderland, M., Von Korff, M., Ustun, T.B., 2009. Normative data for the 12 Item WHO Disability Assessment Schedule 2.0. PLoS ONE 4, e8343. Andrews, G., Cuijpers, P., Craske, M.G., McEvoy, P., Titov, N., 2010. Computer therapy for the anxiety and depressive disorders is effective, acceptable and practical health care: a meta-analysis. PLoS ONE 5, e13196. Barlow, D.H., 2008. Clinical Handbook of Psychological Disorders: A Step-by-step Treatment Manual, 4th ed. Guilford Press, New York, NY, US. Bergstrom, J., Andersson, G., Karlsson, A., Andreewitch, S., Ruck, C., Carlbring, P., Lindefors, N., 2009. An open study of the effectiveness of internet treatment for panic disorder delivered in a psychiatric setting. Nord. J. Psychiatry 63, 44–50. Borkovec, T.D., Alcaine, O., Behar, E., 2004. Avoidance theory of worry and generalized anxiety disorder. Generalized Anxiety Disorder: Advances in Research and Practice. Guilford Press; US, New York, NY. Carlbring, P., Bohman, S., Brunt, S., Buhrman, M., Westling, B.E., Ekselius, L., Andersson, G., 2006. Remote treatment of panic disorder: a randomized trial of internet-based cognitive behavior therapy supplemented with telephone calls. Am. J. Psychiatry 163, 2119–2125. Carlbring, P., Nordgren, L.B., Furmark, T., Andersson, G., 2009. Long-term outcome of internet-delivered cognitive-behavioural therapy for social phobia: a 30-month follow-up. Behav. Res. Ther. 47, 848–850. Connor, K.M., Kobak, K.A., Churchill, L.E., Katzelnick, D., Davidson, J.R.T., 2001. Mini-SPIN: a brief screening assessment for generalized social anxiety disorder. Depression Anxiety 14, 137–140. Cuijpers, P., Marks, I.M., Van Straten, A., Cavanagh, K., Gega, L., Andersson, G., 2009. Computer-aided psychotherapy for anxiety disorders: a meta-analytic review. Cogn. Behav. Ther. 38, 66–82. Dawes, R.M., Faust, D., Meehl, P.E., 1989. Clinical versus actuarial judgment. Science 243, 1668–1674. Gorman, J.M., Kent, J.M., 1999. SSRIs and SNRIs: broad spectrum of efficacy beyond major depression. J. Clin. Psychiatry 60, 33–39. Gould, R.A., Otto, M.W., Pollack, M.H., 1995. A meta-analysis of treatment outcome for panic disorder. Clin. Psychol. Rev. 15, 819–844. Gould, R.A., Otto, M.W., Pollack, M.H., Yap, L., 1997. Cognitive behavioral and pharmacological treatment of generalized anxiety disorder: a preliminary meta-analysis. Behav. Ther. 28, 285–305. Hedman, E., Ljotsson, B., Ruck, C., Bergstrom, J., Andersson, G., Kaldo, V., Jansson, L., Andersson, E., Blom, K., El Alaoui, S., Falk, L., Ivarsson, J., Nasri, B., Rydh, S., Lindefors, N., 2013. Effectiveness of Internet-based cognitive behaviour therapy for panic disorder in routine psychiatric care. Acta Psychiatr. Scand. 128, 457–467. Hedman, E., Ljótsson, B., Kaldo, V., Hesser, H., El Alaoui, S., Kraepelien, M., Andersson, E., Rück, C., Svanborg, C., Andersson, G., Lindefors, N., 2014. Effectiveness of internetbased cognitive behaviour therapy for depression in routine psychiatric care. J. Affect. Disord. 155, 49–58. Hilvert-Bruce, Z., Rossouw, P.J., Wong, N., Sunderland, M., Andrews, G., 2012. Adherence as a determinant of effectiveness of internet cognitive behavioural therapy for anxiety and depressive disorders. Behav. Res. Ther. 50, 463–468. Houck, P.R., Speigel, D.A., Shear, M., Rucci, P., 2002. Reliability of the self-report version of the panic disorder severity scale. Depression Anxiety 15, 183–185. Kessler, R.C., Andrews, G., Colpe, L.J., Hiripi, E., Mroczek, D.K., Normand, S.L.T., Walters, E.E. , Zaslavsky, A.M., 2002. Short screening scales to monitor population prevalences and trends in non-specific psychological distress. Psychol. Med. J. Res. Psychiatry Allied Sci. 32, 959–976. Kessler, R.C., Chiu, W.T., Demler, O., Walters, E.E., 2005. Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication. Arch. Gen. Psychiatry 62, 617–627. Kroenke, K., Spitzer, R.L., Williams, J.B.W., 2001. The PHQ-9: validity of a brief depression severity measure. J. Gen. Intern. Med. 16, 606–613. Kroenke, K., Spitzer, R.L., Williams, J.B.W., Monahan, P.O., Lowe, B., 2007. Anxiety disorders in primary care: prevalence, impairment, comorbidity, and detection. Ann. Intern. Med. 146, 317–325. Lovell, K., Richards, D., 2000. Multiple access points and levels of entry (MAPLE): ensuring choice, accessibility and equity for CBT services. Behav. Cogn. Psychother. 28, 379–391. Lowe, B., Decker, O., Muller, S., Brahler, E., Schellberg, D., Herzog, W., Herzberg, P.Y., 2008. Validation and standardization of the generalized anxiety disorder screener (GAD-7) in the general population. Med. Care 46, 266–274. Meehl, P.E., 1954. Clinical Versus Statistical Prediction: A Theoretical Analysis and a Review of the Evidence. Jason Aronson, Lanham, MD, US. Meehl, P.E., 1965. Seer over sign: the first good example. J. Exp. Res. Pers. 1, 27–32.
224
E.C. Mason, G. Andrews / Internet Interventions 1 (2014) 216–224
Mewton, L., Wong, N., Andrews, G., 2012. The effectiveness of internet cognitive behavioural therapy for generalized anxiety disorder in clinical practice. Depression Anxiety 29, 843–849. Montgomery, S.A., 2006. Guidelines in major depressive disorder, and their limitations. Int. J. Psychiatry Clin. Pract. 10, 3–9. Newby, J.M., Mackenzie, A., Williams, A.D., McIntyre, K., Watts, S., Wong, N., Andrews, G., 2013. Internet cognitive behavioural therapy for mixed anxiety and depression: a randomized controlled trial and evidence of effectiveness in primary care. Psychol. Med. 43, 1–14. Newby, J.M., Mewton, L., Williams, A.D., Andrews, G., 2014. Effectiveness of transdiagnostic internet cognitive behavioural treatment for mixed anxiety and depression in primary care. J. Affect. Disord. 165, 45–52. Perini, S., Titov, N., Andrews, G., 2009. Clinician-assisted Internet-based treatment is effective for depression: randomized controlled trial. Aust. N. Z. J. Psychiatry 43, 571–578. Robinson, E., Titov, N., Andrews, G., McIntyre, K., Schwencke, G., Solley, K., 2010. Internet treatment for generalized anxiety disorder: a randomized controlled trial comparing clinician vs. technician assistance. PLoS ONE 5, e10942. Roy-Byrne, P.P., Cowley, D.S., 2007. Pharmacological Treatments for Panic Disorder, Generalized Anxiety Disorder, Specific Phobia, and Social Anxiety Disorder. A guide to treatments that work, 3rd ed. Oxford University Press, New York, NY, US. Ruwaard, J., Lange, A., Schrieken, B., Dolan, C.V., Emmelkamp, P., 2012. The effectiveness of online cognitive behavioral treatment in routine clinical practice. PLoS ONE 7. Shapiro, D.A., Cavanagh, K., Lomas, H., 2003. Geographic inequity in the availability of cognitive behavioural therapy in England and Wales. Behav. Cogn. Psychother. 31, 185–192. Shear, M., Rucci, P., Williams, J., Frank, E., Grochocinski, V., Vander Bilt, J., Houck, P., Wang, T., 2001. Reliability and validity of the Panic Disorder Severity Scale: replication and extension. J. Psychiatr. Res. 35, 293–296. Spitzer, R.L., Kroenke, K., Williams, J.B.W., Löwe, B., 2006. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch. Intern. Med. 166, 1092–1097. Sunderland, M., Wong, N., Hilvert-Bruce, Z., Andrews, G., 2012. Investigating trajectories of change in psychological distress amongst patients with depression and generalised anxiety disorder treated with internet cognitive behavioural therapy. Behav. Res. Ther. 50, 374–380. Titov, N., Andrews, G., Choi, I., Schwencke, G., Mahoney, A., 2008a. Shyness 3: randomized controlled trial of guided versus unguided internet-based CBT for social phobia. Aust. N. Z. J. Psychiatry 42, 1030–1040.
Titov, N., Andrews, G., Schwencke, G., Drobny, J., Einstein, D., 2008b. Shyness 1: distance treatment of social phobia over the internet. Aust. N. Z. J. Psychiatry 42, 585–594. Titov, N., Andrews, G., Johnston, L., Schwencke, G., Choi, I., 2009a. Shyness programme: longer term benefits, cost-effectiveness, and acceptability. Aust. N. Z. J. Psychiatry 43, 36–44. Titov, N., Andrews, G., Schwencke, G., Solley, K., Johnston, L., Robinson, E., 2009b. An RCT comparing effect of two types of support on severity of symptoms for people completing internet-based cognitive behaviour therapy for social phobia. Aust. N. Z. J. Psychiatry 43, 920–926. Titov, N., Andrews, G., Davies, M., McIntyre, K., Robinson, E., Solley, K., 2010. Internet treatment for depression: a randomized controlled trial comparing clinician vs. technician assistance. PLoS ONE 5. Titov, N., Dear, B.F., Schwencke, G., Andrews, G., Johnston, L., Craske, M.G., McEvoy, P., 2011. Transdiagnostic internet treatment for anxiety and depression: a randomised controlled trial. Behav. Res. Ther. 49, 441–452. Van Der Linden, G.J., Stein, D.J., Van Balkom, A.J., 2000. The efficacy of the selective serotonin reuptake inhibitors for social anxiety disorder (social phobia): a metaanalysis of randomized controlled trials. Int. Clin. Psychopharmacol. 15, S15–S23. Watts, S., Newby, J.M., Mewton, L., Andrews, G., 2012. A clinical audit of changes in suicide ideas with internet treatment for depression. BMJ Open 2 (2012-001558). Weeks, J.W., Spokas, M.E., Heimberg, R.G., 2007. Psychometric evaluation of the MiniSocial Phobia Inventory (Mini-SPIN) in a treatment-seeking sample. Depression Anxiety 24, 382–391. Williams, A.D., Andrews, G., 2013. The effectiveness of internet cognitive behavioural therapy (iCBT) for depression in primary care: a quality assurance study. PLoS ONE 8, e57447. Williams, C., Martinez, R., 2008. Increasing access to CBT: stepped care and CBT self-help models in practice. Behav. Cogn. Psychother. 36, 675–683. Wims, E., Titov, N., Andrews, G., Choi, I., 2010. Clinician-assisted Internet-based treatment is effective for panic: a randomized controlled trial. Aust. N. Z. J. Psychiatry 44, 599–607. Wittkampf, K.A., Naeije, L., Schene, A.H., Huyser, J., Van Weert, H.C., 2007. Diagnostic accuracy of the mood module of the Patient Health Questionnaire: a systematic review. Gen. Hosp. Psychiatry 29, 388–395. Zuithoff, N., Vergouwe, Y., King, M., Nazareth, I., Van Wezep, M., Moons, K., Geerlings, M., 2010. The Patient Health Questionnaire-9 for detection of major depressive disorder in primary care: consequences of current thresholds in a crosssectional study. BMC Fam. Pract. 11, 98.
