session impact and confidence aspect of therapeutic alliance more highly than ... Jr, Department of Psychology, Miami University, Oxford, Ohio 45056, USA.
Counselling and Psychotherapy Research, September 2006; 6(3): 164 168
ORIGINAL ARTICLE
An investigation of session impact and alliance in internet based psychotherapy: Preliminary results D’ARCY J. REYNOLDS, JR1, WILLIAM B. STILES1, & JOHN M. GROHOL2 1
Department of Psychology, Miami University, Ohio, USA and 2PsychCentral.com, Massachusetts, USA
Abstract Although psychotherapy has been and continues to be a face-to-face activity primarily, a growing minority of therapists are conducting text-based (i.e. e-mail) psychotherapy over the Internet. This study compared the session impact (measured by the Session Evaluation Questionnaire, SEQ; Stiles, Gordon, & Lani, 2002) and the client-therapist alliance (measured by the Agnew Relationship Measure, ARM; Agnew-Davies, Stiles, Hardy, Barkham, & Shapiro, 1998) of the exchanges between clients and therapists who are engaged in e-mail therapy with previously published results on face-to-face therapy. According to preliminary results, the online clients provided similar session impact and therapeutic alliance ratings compared to face-to-face clients. Although online therapists followed this general trend, they evaluated the depth, smoothness, and positivity aspects of session impact and confidence aspect of therapeutic alliance more highly than face-to-face therapists.
Keywords: Online therapy, session impact, therapeutic alliance, psychotherapy process research, online data collection
Introduction Currently, the most common mode of delivery for Internet psychotherapy is asynchronous text-based email (Grohol, 1998). This mode of delivery would be expected to be less stimulus-rich than conventional face-to-face therapy (i.e., to lack the nonverbal cues), but it may not necessarily impair interaction quality (Grohol, 1999). Indeed, anecdotal reports have indicated that clients and therapists may perceive text therapy as similar to traditional therapy. For example, Fenichel and his colleagues (2002) commented on the ‘‘. . . similarity between a text-based transcript and a comparable office session’’ and more specifically on ‘‘. . . the expressiveness and depth of the text-based communication’’ (’. 26). A recent review of studies that have evaluated the outcome of online therapy, provides preliminary evidence that online modes of therapy can be effective in reducing clients’ presenting problems (Mallen, Vogel, Rochlen, & Day, 2005). For example, Robinson and Serfaty (2001, 2003) conducted a three-month pre- and post-comparison of 23 British eating disordered females randomly allocated to one of two experienced consultant psychiatrists practicing cognitive behavioural e-mail therapy. They found a significant reduction in clients’ bingeing behavioural and bulimic and depression symptoms and concluded that e-mail therapy may provide a useful treatment for bulimic disorders. However, critics contend there is
no clear understanding of the therapeutic effectiveness or even of the appropriateness of using e-mail as a therapeutic endeavour (e.g., Maheau & Gordon, 2000). While some studies have started to examine the outcome of online therapy, even fewer have attempted to measure process variables; there is a need for the direct examination of differences across important process elements between online therapy and face-to-face therapy (Mallen et al., 2005). We compared these different modes of therapy with respect to the participants’ evaluations of their relationship (alliance) and their sessions (impact). The alliance is widely regarded as a vehicle for conveying therapy’s active ingredients and perhaps a key active ingredient itself (Hartley & Strupp, 1983). Session impact encompasses participants’ evaluations of their session but also participants’ postsession affective state (Stiles et al., 1994). Among the few studies that have attempted to compare the evaluations of internet and face-to-face therapy, Cohen and Kerr’s (1998) therapy analogue study examined a single meeting between face-toface psychotherapy and text chat online psychotherapy for anxiety. Participants (24 undergraduate volunteers as clients and six male counselling graduate students as therapists) scored higher on the Session Evaluation Questionnaire (SEQ) post-session arousal after face-to-face sessions than after online sessions, but not on any other SEQ scales.
Correspondence: D’Arcy J. Reynolds, Jr, Department of Psychology, Miami University, Oxford, Ohio 45056, USA. E-mail: reynold1@ muohio.edu 1473-3145 (print)/1746-1405 (online) – 2006 British Association for Counselling and Psychotherapy DOI: 10.1080/14733140600853617
Psychotherapy process evaluation of internet therapy 165 What does this study explore? j
Whether e-mail counselling produces session impact and client therapist alliance equivalent to face-to-face therapy
Cook and Doyle (2002) compared Working Alliance Inventory (WAI) scores after the third session between 15 predominantly female therapy consumers mainly using e-mail and a representative archival sample of 25 traditional face-to-face therapy clients. They found that Internet therapy consumers’ scores on the goal subscale and composite score of the WAI were significantly higher than those of face-to-face therapy clients. Leibert, Archer, Munson, and York’s (2006) study surveyed 52 primarily young Caucasian female clients mainly using e-mail and text chat at various points in their treatment process and compared their Working Alliance Inventory-Short Form (WAI-S) scores with a corresponding archival sample of 46 face-to-face therapy clients. Clients established weaker working alliances when working with counsellors online. However clients working alliance scores were significantly associated with greater satisfaction of their online counselling. Design We report preliminary results regarding the comparability of alliance and session impact ratings of online therapy conduced via asynchronous text with previously published results from ratings of face-to-face psychotherapy sessions. We expected that participants using e-mail therapy would evaluate their relationships and session impacts similarly to participants in face-to-face therapy. This expectation was based on the common factors notion that similar outcomes are found for the various theoretical orientations in face-to-face therapy (e.g., Wampold, 2001) because it suggests that similar processes (which mediate outcome) would be found for various modes of conducting psychotherapy. Method
Participants Therapists (N /16) each saw from 1 to 10 clients (Mdn /1) and rated a total of 178 sessions. Clients (N /17) rated a total of 205 sessions. Therapists and clients could participate independently, however, both members of the dyad rated 49 of the sessions, thus accounting for 98 of the 383 sets of ratings. The 16 therapists were aged 23 62 years (Mdn / 47), 62% female, 81% Caucasian, and 56% married/ partnered. They were predominately qualified to practice in the United States (n /10 therapists) and a plurality worked from a Cognitive/Behavioural perspective (44%).
The 17 clients were aged from 22 55 (Mdn /39), 71% female, 82% Caucasian, and 53% married/ partnered. All except one client had completed high school. Their most common self-reported presenting problems (given the option to report multiple problems) were depression (8 clients), stress and anxiety (4), and childhood abuse (2).
Measures Demographic questionnaire. The demographic questionnaire requested information on the respondent’s email address, date of birth, marital status, gender, ethnicity, highest year of education, and either client’s presenting problem or therapist’s full name, geographical location qualified to practice of face-to-face therapy licensure, and theoretical orientation. Session Evaluation Questionnaire (SEQ). The SEQ (Form 5; Stiles, Gordon, & Lani, 2002) consists of 21 seven point bipolar adjective items on which respondents were instructed to ‘‘Please select the appropriate number to show how you feel about your therapeutic exchanges this week:’’ The stem ‘‘The therapeutic exchanges this week were:’’ precede the first eleven session evaluation items. There is one global item (i.e., bad-good). Five items comprise the depth scale (i.e., valuable-worthless, deep-shallow, full-empty, powerful-weak, and special-ordinary), and five items comprise the smoothness scale (i.e., easy-difficult, relaxed-tense, pleasant-unpleasant, smooth-rough, and comfortable-uncomfortable). The stem ‘‘Based on the therapeutic exchanges this week, I feel:’’ precedes the second 10 postsession mood items, aggregated into two additional scales. Five items comprise the positivity scale (i.e., happysad, pleased-angry, definite-uncertain, confidentafraid, and friendly-unfriendly) whereas five items comprise the arousal scale (i.e., moving-still, excitedcalm, fast-slow, energetic-peaceful, and arousedquiet). Agnew Relationship Measure (ARM). The ARM version used in this study was a 12-item short form (Stiles, Hardy, Cahill, Barkham, & Agnew-Davies, 2003) of the 28-item ARM (Agnew-Davies et al., 1998). Both versions are rated on parallel forms by clients and therapists using a seven-point scale anchored ‘strongly disagree’ to ‘strongly agree.’ Instructions on the online short form read, ‘‘Thinking about your therapeutic exchanges this week, please indicate how strongly you agreed or disagreed with each statement by clicking on the appropriate response.’’ The short form ARM includes four factorbased scales, named bond (e.g., ‘‘I feel friendly towards my therapist’’), partnership (e.g., ‘‘My therapist and I agree about how to work together’’), confidence (e.g., ‘‘I have confidence in my therapist and his/her techniques’’) and openness (e.g., ‘‘I feel I
166
D. J. Reynolds et al.
can only express my thoughts and feelings to my therapist’’).
Procedure An online system developed for this study presented the demographic questionnaire and the SEQ and ARM measures. Therapist-client pairs were recruited online from private practitioner sites, online clinics (e.g., HelpHorizons.com and LetsTalkCounseling.com), and mental health related discussion boards with the assistance of an undergraduate research team and two external investigators. The directors of several online clinics were approached concerning their therapist’s participation. Interested therapists had the option to invite current clients to take part. However, either the therapists or the clients could participate alone. On their initial visit to the study’s online site, participants completed a consent form and then the demographic forms. They were then taken to the status screen, where they could choose to enter their weekly data immediately or log out to complete their ratings later. At the status screen, therapists could request an e-mail invitation to register be sent to their clients. A weekly e-mail with an embedded link to the login screen was sent on Mondays to all participants who had yet to complete the forms for the previous week. When they logged in, participants were presented with the status screen, which had links for the prior two weeks. When providing their ratings for a given
week of exchanges, they entered the number of sent and received e-mails, completed the ARM and SEQ, and returned to an updated status screen. When participants logged out, they were reminded to visit again next week. Results Feasibility was demonstrated by therapists and clients from various log in locations registering with the online database and completing the online process measures consistently over most of their therapeutic relationship.
Descriptive statistics Table I indicates the means of almost all the online ARM and SEQ scales were above the midpoint of 4.0, indicating that, on average, dyads considered their alliance and their sessions as positive. The alpha coefficients showed that the five-item SEQ indexes were generally reliable whereas the three-item ARM indexes were somewhat less reliable. We decided to combine the bond and partnership indices because their alpha coefficients were low and their intercorrelations were high. Online therapy session impact ratings were within the range of the reported averages for face-to-face therapy from the previous studies with a few exceptions (see Table I). Therapists’ online SEQ depth, smoothness, and positivity ratings and online ARM confidence ratings were above the range of the
Table I. Clients’ and therapists’ internal consistency reliabilities (coefficient alpha), means, standard deviations, and previous studies’ range of means for Session Evaluation Questionnaire (SEQ) indexes and the Agnew Relationship Measure (ARM) indexes. Index
Previous range of meansa
No. of items
Alpha
M
SD
Client ratings SEQ Depth Smoothness Positivity Arousal
5 5 5 5
.96 .93 .93 .63
4.78 4.28 4.39 4.03
1.74 1.43 1.68 .79
4.65 4.13 4.38 3.82
ARM Bond and Partnership Confidence Openness
6 3 3
.87 .85 .60
5.97 6.19 5.27
1.26 1.24 1.42
5.88 to 6.43 5.74 to 6.29 5.39 to 5.76
Therapist ratings SEQ Depth Smoothness Positivity Arousal
5 5 5 5
.87 .82 .80 .67
5.23 4.80 5.39 4.14
1.00 1.23 .95 .92
4.25 3.86 4.38 3.95
ARM Bond and Partnership Confidence Openness
6 3 3
.66 .74 .75
5.72 5.76 4.73
.94 .99 1.55
5.24 to 5.89 4.85 to 5.31 4.47 to 5.70
to to to to
to to to to
5.85 5.28 4.88 4.40
5.10 4.52 5.16 4.58
Note. n /178 sessions for therapists and n /205 sessions for clients. Indexes were calculated as the mean of therapists’ and clients’ ratings on constituent items. SEQ and ARM items could range from 1 to 7. Alpha/internal consistency measured by coefficient alpha. a The means are from prior studies of the SEQ (Cummings, Slemon, & Hallberg, 1993; Dill-Standiford, Stiles, & Rorer, 1988; Kivlighan, Angelone, & Swafford, 1991; Nocita & Stiles, 1986; Reynolds et al., 1996; Stiles et al., 1994; Stiles, Shapiro, & Firth-Cozens, 1988; Stiles & Snow, 1984; Tryon, 1990) and ARM (Agnew-Davies et al., 1998; Stiles, Agnew-Davies, et al., 2002; Stiles et al., 2003).
Psychotherapy process evaluation of internet therapy 167 What do we learn from this study? j
j
Online clients provided session impact and therapeutic alliance ratings similar to those of face-to-face clients Online therapists evaluated the depth, smoothness, and positivity aspects of session impact, and the confidence aspect of the therapeutic alliance, more highly than face-to face therapists
means from the prior studies, whereas the clients’ online ARM openness mean was below the range of means from the prior studies. Discussion Our results suggested that alliance and impact were as strong in these online therapies as in face-to-face therapies studied previously. Thirteen of the 14 mean alliance and session impact index comparisons indicated that online therapy participants rated their exchanges the same as or higher than their face-toface therapy counterparts. Online therapists, as compared to face-to-face therapists, tended to rate their online exchanges as somewhat deeper and smoother, and they felt more positive and more confident about their therapeutic relationships. These findings are preliminary, with a limited number of participants representing narrow ranges of demographics (predominantly female Caucasian adults) and presenting problems (mostly depression and stress/anxiety issues), and possible self-selection biases, insofar as our participants volunteered to participate in this study. The higher session impact ratings by online therapists’ may stem from the asynchronous form of communication, which allows for a "zone of reflection" (Suler, 1999, ’ 8). Therapists can edit unclear phrasing which might help them to consciously to convey main points and avoid the awkward fumbling for words that is common in face-to-face interactions, which increases the perceived depth and smoothness of their online exchanges, respectively. The therapists’ higher ARM confidence responses relative to previous studies might reflect the reduced-cues environment and client characteristics. Perhaps using e-mails gave them a heightened sense of control over the impression they conveyed, or perhaps they felt more powerful because clients who receive information via the computer would tend to believe it more (Cohen & Kerr, 1998). Of course, we had no index of treatment outcome, so we cannot say whether their confidence was justified. Our clients’ lower ARM openness ratings could have reflected their ability to present themselves selectively online, avoiding material that face-to-face therapists detect through nonverbal behaviour (King & Moreggi, 1998). For example, one of our therapists
frustratingly discovered after several weeks that his client who had been medically advised to quit cigarettes was actually smoking during their text exchanges. During the conceptualization of this study, several ethical issues were considered and addressed in the course of securing approval from our institutional review board and funding from the state mental health agency. An important concern was the procedure for online therapists recruiting their own clients to participate in the study, particularly insofar as the therapists were given compensation for participation. This raised the question about the possibility that clients would be subtly coerced to participate in the study. In addition to believing that therapists would consider that the ethical treatment of clients is always their first responsibility, we did not believe that online clients would be susceptible to such coercion for a number of reasons. First, therapists were not told whether their clients were participating, and the online clients could terminate their participation in the study at any time without compromising their current or future therapeutic services. Second, the participating online therapists were all qualified to practice face-to-face therapy and assumedly had the proper grounding in ethics to appreciate the nuances of soliciting their clients into our study. Third, the modest sum of money that we paid the online therapists would not cause them potentially to jeopardize the more substantial sum of money to be gained from subsequent sessions with their client by coercing them to do something contrary to their wishes or interests. Fourth, therapists agreed to an online consent form that brought this issue to their attention by asking them to not influence their client’s decision to participate. Ultimately, we trusted that the online therapists would avoid coercion in soliciting their clients’ participation into our study. In practice, our experience with a prior face-to-face psychotherapy process study confirmed that therapists were extremely cautious in recruiting the participation of their clients (Reynolds, 2004). Our preliminary results offer a qualified encouragement for current face-to-face practitioners and potential clients who are considering using online therapy. Session evaluations and therapeutic relationships were apparently as strong in online therapy as in face-to-face therapy. Provided that further data collection supports this tentative conclusion, we will be able to more clearly answer the question, "Does e-mail therapy yield alliances and impacts that are within the range of those encountered in face-to-face therapy?" in the affirmative. That is, we will be able to suggest that online therapists are able to connect meaningfully with their clients during potent sessions without the assistance of nonverbal cues. Given that these potential common factors of alliance and session impact can be achieved in online therapy, further efficacy research may focus on how these critical elements relate to outcome in online therapy.
168
D. J. Reynolds et al.
This would contribute to answering the question of whether the processes and outcome of online therapy are equivalent to face-to-face psychotherapy (Barnett, 2005). In any case, this investigation of session processes suggests that online e-mail exchanges hold promise as a manner of conducting psychotherapy. Acknowledgements This research was supported by grant 05.1207 from the Ohio Department of Mental Health. We thank Chelsi Day, Elizabeth Lawton, Erin Hoffman, Beth Linkhart, Carlyn Porter, and Emily Verkamp for data collection; Britt Carr and Christian Ratterman for help with the website conceptualization, implementation, and testing; and Art Cooksey and Jason Zack for recruitment assistance. References Agnew-Davies, R., Stiles, W. B., Hardy, G. E., Barkham, M., & Shapiro, D. A. (1998). Alliance structure assessed by the Agnew Relationship Measure (ARM). British Journal of Clinical Psychology, 37, 155 172. Barnett, J. E. (2005). Online counseling: New entity, new challenges. The Counseling Psychologist, 33, 872 880. Cohen, G. E., & Kerr, B. A. (1998). Computer-mediated counseling: An empirical study of a new mental health treatment. Computers in human services, 15, 13 26. Cook, J. E., & Doyle, C. (2002). Working alliance in online therapy as compared to face-to-face therapy: Preliminary Results. CyberPsychology & Behavior, 5, 95 105. Cummings, A. L., Slemon, A. G., & Hallberg, E. T. (1993). Session evaluation and recall of important events as a function of counselor experience. Journal of Counseling Psychology, 40, 156 165. Dill-Standiford, T. J., Stiles, W. B., & Rorer, L. G. (1988). Counselorclient agreement on session impact. Journal of Counseling Psychology, 35, 47 55. Fenichel, M., Suler, J., Barak, A., Zelvin, E., Jones, G., et al. (2002). Myths and Realities of Online Clinical Work. Retrieved from: http://www.fenichel.com/myths/ Grohol, J. M. (1998). Future clinical directions: Professional development, pathology, and psychotherapy on-line. In J. Gackenbach (Ed.), Psychology and the Internet: Intrapersonal, interpersonal, and transpersonal implications (pp. 111 140). San Diego, CA: Academic Press. Grohol, J. M. (1999). Too much time online: Internet addiction or healthy social interactions? CyberPsychology & Behavior, 2, 395 401. Hartley, D. E., & Strupp, H. H. (1983). The therapeutic alliance: Its relationship to outcome in brief psychotherapy. In J. Masling (Ed.), Empirical studies in Psychoanalytic theories (Vol. 1, pp. 1 37). Hillsdale, NJ: Erlbaum. King, S. A., & Moreggi, D. (1998). Internet therapy and self help groups-The pros and cons. Retrieved from: http://webpages. charter.net/stormking/Chapter5/ Kivlighan, D. M., Jr., Angelone, E. O., & Swafford, K. G. (1991). Live supervision in individual psychotherapy: Effects of therapist’s
intention use and client’s evaluation of session effect and working alliance. Professional Psychology: Research and Practice, 22, 489 495. Leibert, T., Archer, J., Jr., Munson, J., & York, G. (2006). An exploratory study of client perceptions of Internet counseling and the therapeutic alliance. Journal of Mental Health Counseling, 28, 69 83. Maheu, M. M., & Gordon, B. L. (2000). Counseling and therapy on the Internet. Professional Psychology: Research and Practice, 31, 484 489. Mallen, M. J., Vogel, D. L., Rochlen, A. B., & Day, S. X. (2005). Online counseling: Reviewing the literature from a counseling psychology framework. The Counseling Psychologist, 33, 819 871. Nocita, A., & Stiles, W. B. (1986). Client introversion and counseling session impact. Journal of Counseling Psychology, 33, 235 241. Reynolds, D. J., Jr. (2004). Online data collection for psychotherapy process research: Session impact and alliance evaluations. Unpublished masters thesis, Miami University, Oxford, Ohio. Reynolds, S., Stiles, W. B., Barkham, M., Shapiro, D. A., Hardy, G. E., & Rees, A. (1996). Acceleration of changes in session impact during contrasting time-limited psychotherapies. Journal of Consulting and Clinical Psychology, 64, 577 586. Robinson, P. H., & Serfaty, M. A. (2001). The use of e-mail in the identification of bulimia nervosa and its treatment. European Eating Disorders Review, 9, 182 193. Robinson, P., & Serfaty, M. (2003). Computers, e-mail and therapy in eating disorders. European Eating Disorders Review, 11, 210 221. Stiles, W. B., Agnew-Davies, R., Barkham, M., Culverwell, A., Goldfried, M. R., Halstead, J., et al. (2002). Convergent validity of the Agnew Relationship Measure and the Working Alliance Inventory. Psychological Assessment, 14, 209 220. Stiles, W. B., Gordon, L. E., & Lani, J. A. (2002). Session evaluation and the session evaluation questionnaire. In G. S. Tryon (Ed.), Counseling based on process research: Applying what we know (pp. 325 343). Boston, MA: Allyn and Bacon. Stiles, W. B., Hardy, G. E., Cahill, J., Barkham, M., & Agnew-Davies, R. (2003, November). The Short ARM (A short form of the Agnew Relationship Measure). Paper presented at the annual meeting of the North American Society for Psychotherapy Research, Newport, RI. Stiles, W. B., Reynolds, S., Hardy, G. E., Rees, A., Barkham, M., & Shapiro, D. A. (1994). Evaluation and description of psychotherapy sessions by clients using the session evaluation questionnaire and the session impacts scale. Journal of Counseling Psychology, 41, 175 185. Stiles, W. B., Shapiro, D. A., & Firth-Cozens, J. A. (1988). Do sessions of different treatments have different impacts? Journal of Counseling Psychology, 35, 391 396. Stiles, W. B., & Snow, J. S. (1984). Counseling session impact as viewed by novice counselors and their clients. Journal of Counseling Psychology, 31, 3 12. Suler, J. (1999). The basic psychological features of e-mail communication. Retrieved from: www.shpm.com/articles/Internet/features.html Tryon, G. S. (1990). Session depth and smoothness in relation to the concept of engagement in counseling. Journal of Counseling Psychology, 37, 248 253. Wampold, B. E. (2001). The great psychotherapy debate: Models, methods, and findings. Mahwah, NJ: Lawrence Erlbaum Associates.
