Journal of Consulting and Clinical Psychology 2008, Vol. 76, No. 3, 449 – 458
Copyright 2008 by the American Psychological Association 0022-006X/08/$12.00 DOI: 10.1037/0022-006X.76.3.449
A Randomized Control Trial Examining the Effect of Acceptance and Commitment Training on Clinician Willingness to Use Evidence-Based Pharmacotherapy Alethea A. Varra
Steven C. Hayes, Nancy Roget, and Gary Fisher
Veterans Affairs Puget Sound Healthcare System
University of Nevada, Reno
This study evaluated the effectiveness of acceptance and commitment training (ACT) for increasing drug and alcohol counselors’ willingness to use evidence-based agonist and antagonist pharmacotherapy. Fifty-nine drug and alcohol counselors were randomly assigned to either a 1-day ACT workshop or a 1-day educational control workshop. Both groups then attended a 2-day workshop on empirically supported treatments for substance abuse. Measures were taken at pre- and posttraining and 3-month follow-up on reported use of pharmacotherapy, willingness to use pharmacotherapy, perceived barriers to implementing new treatments, and general acceptance. As compared with those in the education alone condition, participants in the ACT condition showed significantly higher rates of referrals to pharmacotherapy at follow-up, rated barriers to learning new treatments as less believable at posttraining and follow-up, and showed greater psychological flexibility at posttraining and follow-up. Mediational analyses indicated that reduced believability of barriers and greater psychological flexibility mediated the impact of the intervention. Results support the idea that acceptance-based interventions may be helpful in addressing the psychological factors related to poor adoption of evidence-based treatments. Keywords: dissemination, pharmacotherapy, substance abuse, acceptance and commitment therapy
tional support (McLellan, Carise, & Kleber, 2003). New methods are needed if evidence-based care is to integrate more fully into clinical practice. The present study presents preliminary evidence on a fundamentally different tactic: addressing the psychological barriers to adoption on the part of trainees through an acceptancebased training. If effective, these techniques could be easily integrated into already established training in evidence-based treatments with little to no additional institutional or clinician investment in terms of either support or time. Learning to use new treatments is a psychological and behavioral challenge. Experienced practitioners are suddenly thrust into a situation in which they are neophytes facing issues they may not have faced for some time—issues such as a lack of confidence, clinical uncertainty, and lack of skill (Addis, Wade, & Hatgis, 1999). The impact of this uncertainty along with the impact of other psychologically based factors (e.g., social pressures, stigma, and values; Addis & Krasnow, 2002; Addis et al., 1999; Forman, Bovasso, & Woody, 2001) provides an understudied and potentially rewarding area for targeted interventions. This seems particularly important to consider given basic research in psychology showing that emotions are of fundamental importance for learning and memory (Lewis & Haviland-Jones, 2004). The impact of emotions in educational contexts is pervasive across age groups, genders, and cultures (Shweder & Haidt, 2004), and emotions have been shown to impact learning and performance by changing brain dopamine levels affecting long-term memory (Ashby, Isen, & Turken, 1999), directing attentional processes and the use of cognitive resources (Meinhardt & Pekrun, 2003), inducing and sustaining student interest in learning material (Ainley, Corrigan, & Richardson, 2005; Krapp, 2005), triggering different modes of information processing and problem solving
In many areas of clinical practice there is still a large gap between what has been shown empirically to be helpful and what is performed by practitioners (Hodgson, 1994; Miller et al., 1995; Plante, Andersen, & Boccaccini, 1999). Adoption of evidencebased treatments is largely fostered through journal publications, manuals, and workshops (Millman, Samet, & Shaw, 1990). However, the impact of these interventions on clinical practice has been shown to be limited (Beisser, 1976; Knudsen, Ducharme, Roman, & Link, 2005; Norris & Larsen, 1976). A wide variety of more intensive methods have been tried, including follow-up supervision (Calhoun, Moras, & Pilkonis, 1998; Rounsaville, O’Malley, Foley, & Weissman, 1988) and giving regular feedback to clinicians on the adoption of practices (Shea, DuMouchel, & Bahamonde, 1996). However, these can be difficult to integrate into systems of care, particularly in areas where the delivery system is underfunded and unable to maintain good standards of organiza-
Alethea A. Varra, Mental Health Service, Veterans Affairs Puget Sound Healthcare System, Seattle, Washington; Steven C. Hayes, Nancy Roget, and Gary Fisher, Department of Psychology, University of Nevada, Reno. Portions of this article were presented at the annual convention of the Association for Cognitive & Behavioral Therapies, held in Chicago in November 2006. Funding for this project was provided to the Nevada Practice Improvement Collaborative by a grant from the Center for Substance Abuse Treatment, Substance Abuse and Mental Health Services Administration, Grantee TI12899 (Nancy Roget, principal investigator; Steven C. Hayes and Gary Fisher, coinvestigators). Correspondence concerning this article should be addressed to Alethea A. Varra, Veterans Affairs Puget Sound Healthcare System, Mental Health Service (116), 1660 S. Columbian Way, Seattle, WA 98108. E-mail:
[email protected] 449
450
VARRA, HAYES, ROGET, AND FISHER
(Isen, 1999), and facilitating or impeding students’ self-regulation of learning and performance (Pekrun, Goetz, Titz, & Perry, 2002). Specifically, emotions such as anxiety (Zeidner, 1998), shame (Turner & Schallert, 2001), and boredom (Pekrun et al., 2002) significantly add to the prediction of learning and achievement, over and above the predictive value of cognitive and motivational constructs. Given these findings, it seems important to consider an intervention that targets emotional and psychological factors related to poor adoption of evidence-based treatments. Substance abuse treatments in general and pharmacotherapy in particular provide a particularly apt area to examine these issues. First, a number of evidence-based treatments reduce the use of substances and their associated adverse health, legal, and employment consequences (Meyer, Mirin, & Sachon, 1979; O’Brien & McKey, 2002; O’Connor, Oliveto, & Shi, 1996). Second, despite the presence of frequent training concerning these evidence-based treatments, such treatments are greatly underused by drug and alcohol counselors (Forman, Bovasso, & Woody, 2001; Sloboda & Schildhaus, 2002; Thomas, Wallack, Lee, McCarty, & Swift, 2003). Finally, counselors often have stigmatizing attitudes toward medications in particular, resulting in low adoption (Hayes, Masuda, Bissett, Luoma, & Guerrero, 2006; Ling & Smith, 2002; Millman et al., 1990; Thomas et al., 2003). In the case of other evidence-based technologies, the barriers may be more practical than attitudinal (Henggeler et al., 2007). Although counselors are not the ones who prescribe medications, they play a key role in patient care, and without their endorsement these treatments are generally not a viable option, regardless of their known efficacy. Recent research has suggested that methods based on acceptance and mindfulness can be particularly effective in overcoming the behavioral barriers created by difficult emotions and cognitions, both in clients (Grossman, Niemann, Schmidt, & Walach, 2004; Hayes, Luoma, Bond, Masuda, & Lillis, 2006) and in clinicians (Hayes, Bissett, et al., 2004; Luoma et al., 2007). These methods focus on at least three key processes that may serve as barriers to the use of continuing education in evidence-based treatments. Providers may fail to adopt evidence-based treatments because of resistance to experiencing difficult emotions and thoughts occasioned by using new treatments (e.g., uncertainty, anxiety, or lack of confidence); because of emotion-based cognitive rigidity (e.g., stigma of using medications to treat drug problems); and because of passivity, inaction, and a lack of flexible values-based action that is exacerbated by emotional avoidance and cognitive entanglement. Acceptance and mindfulness methods such as acceptance and commitment therapy (Hayes, Strosahl, & Wilson, 1999) have been shown to undermine all of these processes and to produce better outcomes because of those changes (Hayes, Luoma, et al., 2006). While a full description of this approach as well as how it might differ from a more traditional cognitive therapy approach (e.g., Beck, 1993) is beyond the scope of this article, it is important to note that acceptance-based interventions seek to alter the impact of thoughts and emotions rather than to alter the content, frequency, or intensity of thoughts and emotions. The present study examined whether acceptance and commitment training (ACT) directed toward providers receiving continuing education about evidence-based pharmacotherapy in substance abuse would increase practitioners’ willingness to incorporate the
use of pharmacotherapy with their clients. Specifically, we hypothesized that clinicians in the ACT condition would report being more willing to use pharmacotherapy and would report significantly higher levels of referrals for pharmacotherapy than would those in the educational control condition. In addition, we hypothesized that participants in the ACT condition would report that barriers encountered in the work setting are less believable as actual deterrents to learning and implementing new and evidencebased treatments and would endorse higher levels of general acceptance of and willingness to use these treatments. Finally, we predicted that changes in general acceptance and believability of barriers would mediate changes in primary outcomes.
Method Participants Participants were 59 drug and alcohol counselors recruited randomly from preregistered attendees of a 4-day continuing education conference on substance abuse treatment. Using the results of previous ACT interventions as a guide, we recruited 60 participants, assuming a moderate effect of the intervention. Participants were compensated for the additional day required by this study with free continuing education credits at the conference. Fiftyeight percent were women; 34% were minorities (25% African American, 7% Hispanic, 2% Native American); and mean age was 53.68 years (SD ⫽ 11.70). In terms of level of training and experience, 87% had a bachelor’s degree or higher, 81% of the sample reported being licensed alcohol and drug counselors, 10% of the sample reported being licensed clinical social workers, and 3% reported being licensed psychologists. Participants had a mean of 22 hr a week in contact with drug and alcohol clients (SD ⫽ 15). Participants reported on the types of therapies they were likely to use with their current clients. Ninety-five percent reported at least occasionally using or referring their clients to relapse prevention, 70.2% the matrix model, 77.2% supportive-expressive psychotherapy, 94.7% individualized drug counseling, 89.5% motivational enhancement therapy, 93% behavior therapy, 71.9% multidimensional family therapy, 87.7% 12-step abstinence-based group therapy, 66.7% community reinforcement approach, 57.9% voucher-based reinforcement therapy, and 39% agonist or antagonist pharmacotherapy.
Administration of Assessments Informed consent was acquired before the pretraining measures were given. Participants were assessed at the beginning (pre) and at the end (post) of 3 days of training and at 3-month follow-up. All measures were given in each assessment period, and data were entered for analysis with 100% double entry verification. This research was approved by the University of Nevada’s Social Behavioral Institutional Review Board, and APA ethical standards were followed in the conduct of this study.
Measures Current Use and Willingness to Use Empirically Supported Pharmacotherapy Treatments (EST-Pharmacotherapy). The primary outcome variables consisted of four items that assessed counselors’ current use and willingness to use agonist and antag-
EVIDENCE-BASED TREATMENTS
onist pharmacotherapy with their clients. Specifically, counselors were asked, “How often do you use this treatment with your clients now?” and, “Would you be willing to use this treatment with your clients in the future?” for both antagonist and agonist pharmacotherapies. These questions were asked in the context of other methods that could be employed, and therefore they were clarified with “For these treatments please answer as if your role in the therapy is referring the client to the appropriate provider.” A separate validation study (Varra & Hayes, 2007) found that these counselor self-reports correlated significantly with independent patient reports of medication referrals, r(208) ⫽ .42, p ⬍ .01. More descriptively, 86% of the participants receiving medication for substance use had counselors who said that they frequently referred for pharmacotherapy (above a score of 3 on a 5-point scale), while only 17% of those not receiving medication had counselors with self-reported use that high. On the basis of this independent validation, in the present study counselors with a score above 3 on this measure were considered to be likely referrers to agonist or antagonist medications with their patients. Current Use of Empirically Supported Psychotherapy (ESTPsychotherapy). Twenty items assessing current use of and willingness to use 10 specific evidence-based psychotherapeutic approaches for the treatment of substance abuse were crafted in the same way as with agonist and antagonist medications. Since these methods were not specifically taught in the present study, this outcome measure was used to see if general encouragement to use evidence-based treatments was enough to effect change, even without specific training in these methods. Acknowledgement and Believability of Barriers to Using Empirically Supported Treatments (EST-Barriers). Twenty-eight items measured therapists’ perceived barriers to implementing new treatment approaches. These items were taken from the Texas Christian University Organizational Readiness for Change— Treatment Staff Version instrument (Lehman, Greener, & Simpson, 2002; Simpson, 2002), a 129-item self-report assessment of organizational functioning with good reliability and validity. Items were modified to fit the current purpose and to lessen the assessment burden. Counselors were asked to report on a 5-point scale whether they experienced a particular barrier in their current workplace. Sample items were “Other staff members will not support new treatments” or “The workload and pressures at my job keep motivation for new training low.” In addition, since ACT specifically targets the believability of barriers, participants were then asked to rate on a 5-point scale for each item, “How believable is it that this prevents you from learning or using new treatments?” It is important to note that we were not measuring the believability of the barrier’s existence but rather the believability of its impact. These two scores (acknowledgement of barriers and believability of barriers) were used as process measures. The 10 most commonly experienced barriers at pretesting are shown in Table 1. Acceptance and Action Questionnaire (AAQ; Hayes, Strosahl, Bunting, Twohig, & Wilson, 2004). The AAQ assesses participants’ willingness to defuse undesirable thoughts and feelings and accept new ones while acting in a way that is consistent with their values. Two general terms for the AAQ have been used in the literature: experiential avoidance and psychological flexibility. The latter will be used here because while most items on the AAQ address experiential avoidance, others assess cognitive fusion
451
Table 1 Top 10 Barriers Endorsed by Substance Abuse Counselors as Measured by Reported Incidence in Their Current Place of Employment 1. 2. 3. 4. 5. 6. 7. 8.
I already know a treatment that works just fine. Some staff members resist any type of change. I am not able to spend enough time with clients. My offices, equipment, and materials are inadequate. 12-step (AA/NA) is followed by many of my coworkers. Other staff aren’t trying to change or improve their treatment. Staff members are given too many rules. The workload and pressures at my job keep motivation for new training low. 9. There is little mutual cooperation or trust among staff. 10. Other staff members will not support new treatments. Note.
AA/NA ⫽ Alcoholics Anonymous/Narcotics Anonymous.
(treating thoughts literally) and absence of values-based action. The 16-item version (Bond & Bunce, 2003) of this process measure was used. Higher scores indicate greater psychological acceptance and behavioral flexibility. The AAQ has been shown in several studies to have adequate reliability and validity (Bond & Bunce, 2003; Hayes, Strosahl, et al., 2004; see Hayes, Luoma, et al., 2006, for a meta-analysis of 32 AAQ studies with several thousand participants).
Procedures All participants attended the same 2-day training on pharmacotherapy and evidence-based treatment. The day prior, one half of the participants were randomly assigned to attend a daylong presentation (total contact time of 6 hr) of either ACT (n ⫽ 30) or an educational control training (n ⫽ 30). Participants were randomized by study support staff using a list randomizer to randomize subject IDs with sequence concealed until the interventions were assigned. No staff members were blind to the intervention, as study measures were self-reports and study investigators were also group leaders. One participant in the control condition failed to show up for the training. During the 3-day training, 2 participants (3%), one in each condition, left and did not complete postassessments. There were no significant differences between these individuals and those who completed postassessments on preassessment measures or demographic variables. At the 3-month follow-up, 8 additional participants (13.6%), 3 in the control condition and 5 in the ACT condition, were unavailable because they had left their employment positions and their employer either did not know how to contact them or did not feel comfortable sharing current contact information. There were no significant differences between these individuals and those who completed postassessments on preassessment measures or demographic variables. See Appendix for a flow chart of participants’ progress through the study. Adherence strategy. Adherence was assured in each workshop by producing detailed PowerPoint slides covering all points that had to be covered in sequential order. Expert trainers who had themselves been involved in creating the slide presentations were used. The slides were deliberately extensive so that presenters had to stick closely to the content of the slides in order to complete their task. For example, a total of 76 slides were used in the pharmacotherapy workshop.
452
VARRA, HAYES, ROGET, AND FISHER
Acceptance and commitment training (ACT). The ACT condition was based on acceptance and commitment therapy (Hayes et al., 1999), but as with previous research of this kind (Hayes, Bissett, et al., 2004) it is termed acceptance and commitment training here in order to distinguish the protocol from similar protocols used for psychotherapy purposes. This approach has been shown to help therapists reduce stress and burnout by alleviating entanglement with difficult thoughts and feelings related to work (Hayes, Bissett, et al., 2004). An experienced ACT clinician (Alethea A. Varra) and the originator of ACT (Steven C. Hayes) served as trainers. The trainers asked participants to discuss what barriers to learning new and evidence-based treatments they encounter in their current place of employment. The group was then introduced to principles of cognitive defusion. The arbitrary nature of evaluative reactions (i.e., terms that come up automatically about barriers to learning or implementing new treatments and about oneself when working with difficult clients) was explored with the help of defusion techniques drawn from Hayes et al. (1999). To be clear, the validity of these thoughts and evaluations were not challenged as either valid or not valid; nor were they categorized as functional or dysfunctional. Rather, the relationship between participants and their thoughts was explored (e.g., whether thoughts are experienced as causes of behavior or as thoughts separate from behavior). The training then addressed the issue of avoidance and control of emotions and thoughts. The negative impact of trying to control or avoid emotions and thoughts was explored. Acceptance skills were taught as methods of dealing with the difficult thoughts and feelings that come from learning and using new treatments. Finally, the ACT trainers asked participants to identify their values as a therapist. They participated in a small group commitment exercise in which they identified what is important to them as counselors. They were asked to identify what they might have been doing instead, what that might have cost their clients, and what their commitment was for future treatment and clients. Educational control condition. The educational control condition was designed to control for attention, education, and encouragement to learn. Three experienced professional trainers who regularly gave workshops in these areas were used. Participants in the educational control condition first attended a 3.5-hr presentation on prevention curriculums within employee assistance programs. This included two presentations: (1) a discussion of current research findings in the area of prevention and (2) a discussion of strategies to prevent addiction. Participants then attended a 3.5-hr presentation titled “Strength-Based Leadership Styles,” which discussed positive strength– based leadership styles of management perspectives. The content of the workshop was designed to help participants understand the importance of learning and practicing good leadership perspectives when working with both coworkers and clients. Pharmacotherapy/evidence-based treatment training. All participants then attended a 2-day workshop (14 hr of training) on evidence-based treatment and medications used in substance use disorders. These workshops were part of the larger conference, and thus workshop participants in this phase consisted of individuals in the ACT group, the educational control group, and general conference attendees. Two experienced psychiatrists who regularly gave workshops in these areas were used. Training focused on the symptoms of withdrawal, detoxification principles, adjunctive
medications for withdrawal from different classes of substances, and research concerning the use of agonist and antagonist medications for the treatment of substance abuse. It also included discussion of the philosophy of evidence-based treatment, barriers to adoption, a list of current treatments that are considered to be evidence-based, and how manualized treatments fit within an evidence-based treatment model. A total of 103 slides were used, 73% of which focused on pharmacotherapy, 22% on the philosophy and importance of empirically supported practice, and 5% on which psychotherapies are considered evidence-based for substance abuse treatment.
Results Means, standard deviations, and within-group effect sizes for all measures and conditions are presented in Table 2. All statistical comparisons were not interpreted unless a p value of .05 was reached. In line with Cohen (1988), effect sizes were termed small, medium, or large if their d values were .2, .5, or .8, respectively. Analysis of variance or chi-square techniques were used to test whether the experimental groups were equivalent on demographic and dependent variables at baseline. No differences between groups were found on any of the demographic variables collected at baseline ( p ⬎ .05). No differences were found at post or follow-up on outcome or process measures when comparing gender, age, or race as well. These supplementary analyses are voluminous and can be found elsewhere (Varra, 2007).
Outcome Analyses Each of the three outcome measures (Willingness to Use Pharmacotherapy; Reported Use of Pharmacotherapy; Willingness to Use Empirically Supported Psychotherapy) was assessed by a repeated-measures analysis of covariance (ANCOVA). The pretreatment value was the covariate, treatment condition (acceptance, control) was the between-subjects variable, and time (post, follow-up) was the within-subjects variable. Missing data were analyzed using maximum likelihood estimation, and missing values were replaced using the Missing Values Analysis/Estimation Methods module of SPSS Version 13. Willingness to Use Pharmacotherapy. There was a statistically significant and large effect for condition on the willingness to use pharmacotherapy, F(1, 56) ⫽ 11.46, p ⫽ .001, 2p ⫽ .17, but not for phase, F(1, 56) ⫽ 0.56, p ⫽ .51, 2p ⫽ .008, or the group by phase interaction, F(1, 56) ⫽ 0.59, p ⫽ .45, 2p ⫽ .010. Participants with acceptance at pretraining showed significantly higher willingness at post and follow-up, adjusted for prescores, than did those with the educational control pretraining. Reported Use of Pharmacotherapy. The most important outcome variable was reported use of agonist and antagonist pharmacotherapy. At baseline, there was no significant difference in reporting between groups, 2(1, N ⫽ 56) ⫽ 3.01, p ⫽ .083, Cohen’s d ⫽ 0.16, with 10% of those in the acceptance pretraining condition likely referrers as compared with 23% in the comparison condition. At follow-up, 54% of those in the acceptance pretraining condition were likely referrers, as compared with 13% in the comparison condition—a significant and large effect, 2(1, N ⫽ 56) ⫽ 10.29, p ⫽ .001, Cohen’s d ⫽ 1.03.
EVIDENCE-BASED TREATMENTS
453
Table 2 Means, Standard Deviations, Within-Group Cohen’s d, and Between-Groups Cohen’s d of All Measures Pre Measure and group Willingness to Use Pharmacotherapy Control ACT Between-groups d Reported Use of Pharmacotherapy Control ACT Between-groups d Willingness to Use Empirically Supported Psychotherapy Control ACT Between-groups d Barriers to Use of New Treatments Control ACT Between-groups d Believability of Barriers to Use of New Treatments Control ACT Between-groups d Psychological flexibility (AAQ) Control ACT Between-groups d
Post
Follow-Up
M
SD
M
SD
Within d
M
SD
Within d
2.67 2.32
1.47 1.10 –0.27
2.58 3.25
1.18 1.11 0.58*
–0.07 0.84*
2.46 3.46
1.25 1.11 0.85*
–0.15 1.03*
2.50 2.12
1.34 1.05 –0.32
— —
— — —
— —
2.32 3.31
1.12 1.20 0.85*
–0.15 1.06*
3.62 3.12
0.68 0.81 –0.67
3.83 3.66
0.60 0.61 –0.28
0.33 0.76*
3.54 3.36
0.79 0.62 –0.25
–0.11 0.33
69.45 69.73
13.75 13.10 0.02
71.81 82.58
13.39 13.12 0.81*
0.17 0.98*
68.47 73.13
11.63 11.07 0.41
–0.08 0.28
66.03 69.80
19.76 17.46 –0.2
66.64 53.55
19.79 17.08 0.71*
–0.03 0.94*
65.94 52.26
19.92 13.83 0.80*
0.01 1.11*
83.24 80.67
8.85 9.30 –0.28
81.19 85.23
7.89 7.52 0.52*
–0.24 0.54*
84.86 89.43
8.79 8.23 0.54*
0.18 1.00*
Note. Within d signifies the within-group Cohen’s d for differences between pre and post and pre and follow-up measures. Dashes indicate that this variable was not measured during the postassessment due to the fact that there was no opportunity to use these treatments between pre and post. Pre ⫽ pretraining; Post ⫽ posttraining; ACT ⫽ acceptance and commitment training; AAQ ⫽ Acceptance and Action Questionnaire (Hayes, Strosahl, Bunting, Twohig, & Wilson, 2004). * p ⬍ .05 (as shown by t test).
5 Mean Reported Use of Pharmacotherapy
A one-way ANCOVA using prescores as the covariate found a significant impact of treatment, F(1, 56) ⫽ 10.51, p ⫽ .002, 2p ⫽ .16, and group by phase interaction, F(1, 56) ⫽ 10.33, p ⫽ .002, 2p ⫽ .15, with acceptance-pretrained participants reporting more referrals for medications in their practice than did those in the control condition. These effects are shown in Figure 1. Willingness to Use Empirically Supported Psychotherapy. This measure assessed whether extensive training in the content area was needed to see improved willingness to use empirically supported psychotherapy. The repeated measures ANCOVA showed no effect for phase, F(1, 56) ⫽ 0.37, p ⫽ .54, 2p ⫽ .007, the group by phase interaction, F(1, 56) ⫽ 0.28, p ⫽ .60, 2p ⫽ .005, or condition, F(1, 56) ⫽ 0.01, p ⫽ .92, 2p ⫽ .000, suggesting that the effects seen in the primary outcome analysis required both acceptance pretraining and specific and extensive education in the actual methods, not just attitudinal appeals.
4.5 4
Control ACT
3.5 3 2.5 2 1.5
Process Variables Acknowledgement of and believability of barriers. The repeated measures ANCOVA showed no effect for phase, F(1, 56) ⫽ 1.02, p ⫽ .32, 2p ⫽ .018. However, there was a significant effect for the group by phase interaction, F(1, 56) ⫽ 5.49, p ⫽ .023, 2p ⫽ .089, and large effect for condition on the perceived barriers to using new evidence-based treatments, F(1, 56) ⫽ 10.02, p ⫽ .003, 2p ⫽ .152. This effect was accounted for by those with
1 Pre
Follow-up Assessment Time
Figure 1. Mean reported use of pharmacotherapy for ACT and control groups at pretraining and follow-up. The difference between groups is significant ( p ⬍ .01) at follow-up. ACT ⫽ acceptance and commitment training; pre ⫽ pretreatment.
VARRA, HAYES, ROGET, AND FISHER
454
acceptance pretraining admitting to more barriers than did the educational control group, particularly at post. The analysis of the believability of these same barriers showed a different effect, however. A repeated measures ANCOVA showed no effect for either phase, F(1, 56) ⫽ 0.61, p ⫽ .44, 2p ⫽ .013, or the group by phase interaction, F(1, 56) ⫽ 0.04, p ⫽ .84, 2p ⫽ .001, but there was a statistically significant and large effect for condition on the believability of barriers, F(1, 56) ⫽ 24.36, p ⫽ .000, 2p ⫽ .30, with those within the acceptance pretraining condition reporting that it was less believable, both at post and follow-up, that these barriers were actually getting in the way of learning new and evidence-based treatments. In other words, acceptance participants acknowledged barriers more but believed them less. Psychological flexibility. The repeated measures ANCOVA showed no effect for either phase, F(1, 56) ⫽ 0.79, p ⫽ .38, 2p ⫽ .014, or the group by phase interaction, F(1, 56) ⫽ 0.03, p ⫽ .87, 2p ⫽ .000, but there was a statistically significant and large effect for condition on psychological flexibility as measured by the AAQ, F(1, 56) ⫽ 13.40, p ⫽ .001, 2p ⫽ .193. This effect was accounted for by those with acceptance pretraining reporting greater flexibility than did the control group at post and follow-up, adjusted for prescores on this measure.
in which kappa samples of the original size are taken from the obtained data (with replacement after each specific number is selected), and mediational effects are calculated in each sample. In the present set of analyses, parameter estimates were based on 5,000 bootstrap samples. The point estimate of the indirect cross product (see Table 3) is the mean for these 5,000 samples; the bias-corrected and -accelerated 95% confidence intervals are similar to the 2.5 and 97.5 percentile scores of the obtained distribution over the samples, but with z score– based corrections for bias due to the underlying distribution (Preacher & Hayes, 2004, 2006). If the confidence intervals do not contain zero, the point estimate is significant at the level indicated. The mediators examined in the present study were increased psychological flexibility and reduced believability of barriers. Changes in willingness to refer to pharmacotherapy at follow-up (see Figure 2 and Table 3) were mediated by changes in flexibility and believability at post, both in total and for each mediator, controlling for the other. Changes in reported actual referrals at follow-up were mediated by both working together, but when examined individually only change in believability of barriers functioned as an independent mediator. Changes in psychological flexibility increased the indirect effect beyond change in believability alone, however (see Table 3).
Mediational Analysis
Discussion
Mediation was tested using the cross product of the coefficients for the independent variable to mediator relation and the mediator to outcome relation controlling for treatment. A cross product test is recognized as perhaps the best all-around available method to test mediation (MacKinnon, Lockwood, Hoffman, West, & Sheets, 2002) because it directly tests the significance of the difference between the direct and indirect, or mediated, effect (Sobel, 1982) rather than doing it only by inference, as is the case in a causal steps approach (Baron & Kenny, 1986). There is also a wellaccepted logic (Bollen, 1989) for applying this method to detecting the indirect effects of multiple mediators, which was desirable in this case. The Sobel (1982) test is the best known cross product method, but it assumes a normal distribution of the cross product, which is generally incorrect (Preacher & Hayes, 2004). The nonparametric method used in the current study (Preacher & Hayes, 2004, 2006) solves the distribution problem through bootstrapping,
To our knowledge, this is the first study designed to improve the dissemination of evidence-based practice by increasing the psychological flexibility of providers before training. Previous research has shown that traditional educational approaches to dissemination of evidence-based care have often been unsuccessful (Beisser, 1976; Knudsen et al., 2005; Norris & Larsen, 1976; Rounsaville, O’Malley, Foley, & Weissman, 1988). Results of the current study replicate this finding. Two entire days of education on the importance of empirically based treatment and on the science and practice of pharmacotherapy, as well as an additional day of pretraining on prevention of substance abuse and the importance of demonstrating leadership in treatment settings, had absolutely no positive impact on drug and alcohol counselors’ willingness to use or the actual use of pharmacotherapy. In fact, participants in the control condition’s willingness to use pharmacotherapy actually drifted lower at post than before their training in
Table 3 Bootstrapped Point Estimates and Bias-Corrected and -Accelerated (BCa) Confidence Intervals (CIs) for the Total and Specific Indirect Effects on Primary Outcomes Bootstrapping BCa 95% CI
Product of coefficients Variable Willingness to use pharmacotherapy at follow-up Change in psychological flexibility Change in believability Total Reported use of pharmacotherapy at follow-up Change in psychological flexibility Change in believability Total
Point estimate
SE
z
p
Lower
Upper
.24 .43 .67
.133 .178 .221
1.83 2.42 3.05
⬍.05 ⬍.05 ⬍.005
0.08 0.20 0.37
0.49 0.75 1.06
.14 .35 .48
.117 .172 .205
1.16 2.03 2.36
⬍.25 ⬍.05 ⬍.05
–0.01 0.12 0.22
0.35 0.65 0.81
EVIDENCE-BASED TREATMENTS
Total Effects
1.00*
Willingness to Use Pharmacotherapy
0.98*
Use of Pharmacotherapy at Follow-up
Treatment Group
Indirect Effects
Change in Psychological Flexibility
.04*
5.76*
.02
Willingness to Use Pharmacotherapy
.33 Treatment Group
455
Direct Effects .50 16.07*
Change in Believability of Barriers
Use of Pharmacotherapy at Follow-up
.03* .03*
Figure 2. Unstandardized regression weights indicating total, direct, and mediated effects of treatment condition on the primary outcomes of the experiment. *p ⬍ .05.
evidence-based practice (this finding was not significant, and it is not clear if this drift was due to random error, iatrogenic effects of the educational control, or even an aspect of the evidence-based training itself). However, when counselors’ avoidance of difficult feelings, entanglement with difficult thoughts, and absence of values-based action were targeted through ACT, a large and persistent change in adoption followed continuing education in evidence-based pharmacotherapy methods. Mediational analyses suggest that this occurred because ACT reduced the believability of barriers to adoption and increased counselors’ ability to be more flexible in their actions while experiencing difficult thoughts and feelings. The fact that no changes were seen in willingness to use empirically supported psychotherapy, which was not extensively targeted in the continuing education module, suggests that the content of continuing education was important and necessary to the effects seen, but it was not sufficient in isolation to effect change. The finding of this study considerably extends the findings of a recent study (Luoma et al., 2007), which found that a supervision group based on ACT and on relapse prevention (Marlatt & Gordon, 1985) increased the adoption of group drug counseling (GDC; Daley, Mercer, & Carpenter, 2002) when ACT was presented after a 1-day continuing education workshop on GDC. As with the current study, Luoma et al. (2007) was based on the idea that adoption is a psychologically challenging event that requires acceptance of discomfort and behavioral persistence, but it used a method that is much more difficult to implement: 8 weeks of acceptance-based supervision (90 min per session). The study was also less well controlled, using a no-contact control and adoption measures that had not been validated against client report or other records. Nevertheless, the outcomes of Luoma et al. (2007) lend additional support to those found in the present study, which used a method that could be readily applied to a wide variety of clinical methods and continuing education efforts. The types of barriers to adoption reported in this study are consistent with previous studies that indicate that counselors have
a number of social, psychological, and stigma-related reasons for failing to adopt new treatments (Addis et al., 1999), not just intellectual (Addis & Krasnow, 2002) and practical (Addis et al., 1999; Forman, Bovasso, & Woody, 2001) reasons. Of the 10 most highly endorsed barriers for counselors in this study, only 3 were practical. Only 1 barrier was intellectual, which may help explain why the educational alone condition was so inert: It targeted only a small part of what participants see as the barriers to adoption. Most barriers were linked to fears of the views of coworkers; the stigma of pharmacotherapy; and other emotional, social, and psychological barriers. There are a number of evidence-based psychosocial treatments that also raise attitudinal or other psychological barriers for clinicians (e.g., McGovern, Fox, & Xie, 2004), and thus it would be an important next step to see if this same approach will be helpful in these situations. Some of the current outcomes, such as an increase in acknowledgement of barriers but a decrease in their believability, may be unexpected outside of an acceptance model, but they are typical in that literature. For example, in one study psychotic patients who receive acceptance and commitment therapy admitted to more hallucinations and delusions than did a control group, but rehospitalization was particularly low among those who did so because these symptoms were less believable (Bach & Hayes, 2002). A replication study by a different research team has found that for those receiving acceptance and commitment therapy, hallucinations were less distressing for the same reason (Gaudiano & Herbert, 2004). It seems ethically important that these methods remain focused on empowering the values of individual practitioners and that they be linked to the best available science. In the present study, workshop leaders did not tell participants what to value or what to do with the continuing education that was to follow. The scientific evidence for agonist and antagonist treatments is relatively clear, but many other forms of pharmacotherapy are embroiled in controversy, and ACT should not provide cover for those difficulties or for any similar problems that may be involved with other
VARRA, HAYES, ROGET, AND FISHER
456
methods. It seems unlikely that ACT, when properly done, could do so, since it seeks to support individual values and empower participants to be more active and flexible recipients of information. Bond and Bunce (2000), for example, found that worksitebased ACT programs for stress lead workers to confront supervisors to make needed changes, not just accept work stressors, even without direct suggestions in the ACT program to make such changes. Nevertheless, the effects found in the present study raise an important issue: how to ensure that increased psychological flexibility remains an open process that cannot be misused by manipulative interests. Within the ACT research and practice community, this issue is being addressed by commitment to an open, transparent, nonhierarchical process of development and dissemination of ACT, but continuous dialogue on this issue seems important. There are several limitations to the current study. Counselors initially reported being quite unwilling to refer for these particular types of therapies, with an average reported use just above 2 on a 5-point scale. Even in the acceptance condition, willingness and referral scores did not exceed 3.5 at follow-up, indicating that there is still room for improvement. Only replication will reveal whether these results apply to a broader population of treatment providers (e.g., psychologists vs. social workers; drug and alcohol vs. mental health; demographic subgroups). There was no control for the quality of trainers other than using experts in all conditions. While the outcomes measure was validated against patient report, there was no actual direct behavioral measure of referrals to pharmacotherapy. There was no measurement of exactly how much participants learned in the pharmacotherapy workshop, so the role of knowledge in adoption cannot be assessed. Further, the findings are limited to specific types of pharmacotherapy, not to all empirically supported practices. Although the findings of Luoma et al. (2007) provide some reassurance on that topic, it will be important to more directly measure the impact of the present approach on other forms of evidence-based practice, including psychotherapeutic interventions and combination or enhanced treatments (e.g., contingency management to enhance compliance with naltrexone for alcohol dependence). Finally, there was no measurement of client outcomes, so it is not known if they improved as a result of these clinicians’ receiving training. The present findings open up a new avenue worthy of exploration in dissemination research. These data suggest that attending to learners’ psychological challenges may create greater impact for continuing education in evidence-based treatments than will be likely with continuing education alone. Furthermore, the relative ease of integrating such methods into existing training protocols without requiring additional institutional support or clinician time allows for the possibility of greatly impacting effective dissemination within already overburdened health care systems. An important next step in this research will be to examine the effectiveness of an integrated ACT/evidence-based training and compare this approach with more burdensome supervision and continued contact models of dissemination.
References Addis, M. E., & Krasnow, A. D. (2002). A national survey of practicing psychologists’ attitudes toward psychotherapy treatment manuals. Journal of Consulting and Clinical Psychology, 68, 331–339.
Addis, M. E., Wade, W. A., & Hatgis, C. (1999). Barriers to dissemination of evidence-based practices: Addressing practitioners’ concerns about manual-based psychotherapies. Clinical Psychology: Science and Practice, 6, 430 – 441. Ainley, M., Corrigan, M., & Richardson, N. (2005). Students, tasks, and emotions: Identifying the contribution of emotions to students’ reading of popular culture and popular science texts. Learning and Instruction, 15, 433– 447. Ashby, F. G., Isen, A. M., & Turken, A. U. (1999). A neuropsychological theory of positive affect and its influence on cognition. Psychological Review, 106, 529 –550. Bach, P., & Hayes, S. C. (2002). The use of acceptance and commitment therapy to prevent rehospitalization of psychotic patients: A randomized controlled trial. Journal of Consulting and Clinical Psychology, 70, 1129 –1139. Baron, R., & Kenny, D. A. (1986). The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51, 1173–1182. Beck, A. T. (1993). Cognitive therapy: Past, present, and future. Journal of Consulting and Clinical Psychology, 61, 194 –198. Beisser, A. R. (1976). Continuing education for community mental health: Person, system, and task. Los Angeles: Health Training Center. Bollen, K. A. (1989). Structural equations with latent variables. New York: Wiley. Bond, F. W., & Bunce, D. (2000). Mediators of change in emotion-focused and problem-focused worksite stress management interventions. Journal of Occupational Health Psychology, 5, 156 –163. Bond, F. W., & Bunce, D. (2003). The role of acceptance and job control in mental health, job satisfaction, and work performance. Journal of Applied Psychology, 88, 1057–1067. Calhoun, K. S., Moras, K., & Pilkonis, P. A. (1998). Empirically supported treatments: Implications for training. Journal of Consulting and Clinical Psychology, 66, 151–162. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum. Daley, D. C., Mercer, D., Carpenter, G. (2002). Drug counseling for cocaine addiction: The Collaborative Cocaine Treatment Study model (NIH Publication No. 02– 4381). Washington, DC: U.S. Department of Health and Human Services. Forman, R. F., Bovasso, G., & Woody, G. (2001). Staff beliefs about addiction treatment. Journal of Substance Abuse Treatment, 21, 1–9. Gaudiano, B. A., & Herbert, J. D. (2004). Believability of hallucinations as a potential mediator of their frequency and associated distress in psychotic inpatients. Behavioral and Cognitive Psychotherapy, 34, 497– 502. Grossman, P., Niemann, L., Schmidt, S., & Walach, H. (2004). Mindfulness-based stress reduction and health benefits: A meta-analysis. Journal of Psychosomatic Research, 57, 35– 43. Hayes, S. C., Bissett, R., Roget, N., Padilla, M., Kohlenberg, B. S., & Fisher, G. (2004). The impact of acceptance and commitment training and multicultural training on the stigmatizing attitudes and professional burnout of substance abuse counselors. Behavior Therapy, 35, 821– 836. Hayes, S. C., Luoma, J., Bond, F., Masuda, A., & Lillis, J. (2006). Acceptance and commitment therapy: Model, processes, and outcomes. Behavior Research and Therapy, 44, 1–25. Hayes, S. C., Masuda, A., Bissett, R., Luoma, J., & Guerrero, L. F. (2006). DBT, FAP, and ACT: How empirically oriented are the new behavior therapy technologies? Behavior Therapy, 35, 35–54. Hayes, S. C., Strosahl, K. D., Bunting, K., Twohig, M. P., & Wilson, K. G. (2004). What is acceptance and commitment therapy? In S. C. Hayes & K. D. Strosahl (Eds.), A practical guide to acceptance and commitment therapy. New York: Plenum/Kluwer. Hayes, S. C., Strosahl, K., & Wilson, K. G. (1999). Acceptance and
EVIDENCE-BASED TREATMENTS commitment therapy: An experiential approach to behavior change. New York: Guilford Press. Henggeler, S. W., Chapman, J. E., Rowland, M. D., Halliday-Boykins, C. A., Randall, J., Shackelford, J., & Schoenwald, S. K. (2007). If you build it, they will come: Statewide practitioner interest in contingency management for youths. Journal of Substance Abuse Treatment, 32, 121–131. Hodgson, R. (1994). Treatment of alcohol problems. Addiction, 89, 1529 – 1534. Isen, A. M. (1999). Positive affect. In T. Dalgleish & M. Power (Eds.), Handbook of cognition and emotion (pp. 521–539). New York: Wiley. Knudsen, H. K., Ducharme, L. J., Roman, P. M., & Link, T. (2005). Buprenorphine diffusion: The attitudes of substance abuse treatment counselors. Journal of Substance Abuse Treatment, 29, 95–106. Krapp, A. (2005). Basic needs and the development of interest and intrinsic motivational orientations. Learning and Instruction, 15, 381–395. Lehman, W. K., Greener, J. M., & Simpson, D. (2002). Assessing organizational readiness for change. Journal of Substance Abuse Treatment, 22, 197–209. Lewis, M., & Haviland-Jones, J. M. (Eds.). (2004). Handbook of emotions (2nd ed.). New York: Guilford Press. Ling, W., & Smith, D. (2002). Buprenorphine: Blending practice and research. Journal of Substance Abuse Treatment, 23, 87–92. Luoma, J. B., Hayes, S. C., Roget, N., Fisher, G., Padilla, M., Bissett, R., et al. (2007). Augmenting continuing education with psychologicallyfocused group consultation: Effects on adoption of group drug counseling. Psychotherapy Theory, Research, Practice, Training, 44, 463– 469. MacKinnon, D. P., Lockwood, C. M., Hoffman, J. M., West, S. G., & Sheets, V. (2002). A comparison of methods to test mediation and other intervening variable effects. Psychological Methods, 7, 83–104. Marlatt, G. A., & Gordon, J. R. (1985). Relapse prevention: Maintenance strategies in the treatment of addictive behaviors. New York: Guilford Press. McGovern, M. P., Fox, T. S., & Xie, H. (2004). A survey of clinical practices and readiness to adopt evidence-based practices: Dissemination research in an addiction treatment system. Journal of Substance Abuse Treatment, 26, 305–312. McLellan, A. T., Carise, D., & Kleber, H. D. (2003). Can the national addiction treatment infrastructure support the public’s demand for quality care? Journal of Substance Abuse Treatment, 25, 117–121. Meinhardt, J., & Pekrun, R. (2003). Attentional resource allocation to emotional events: An ERP study. Cognition and Emotion, 17, 477–500. Meyer, R. E., Mirin, S. M., & Sachon, F. (1979). Community outcome on narcotic antagonists. In R. E. Meyer & S. Mirin (Eds.), The heroin stimulus: Implications for a theory of addiction. New York: Plenum. Miller, W. R., Brown, J. M., Simpson, T. L., Handmaker, N. S., Bien, T. H., Luckie, L. F., et al. (1995). What works? A methodological analysis of the alcohol treatment outcome literature. In R. K. Hester & W. R. Miller (Eds.), Handbook of alcoholism treatment approaches: Effective alternatives (pp. 12– 44). Needham Heights, MA: Simon & Schuster. Millman, J., Samet, S., & Shaw, J. (1990). The dissemination of psychological research. American Psychologist, 45, 668 – 669. Norris, E. L., & Larsen, J. K. (1976). Critical issues in mental health service delivery: What are the priorities? Hospital and Community Psychiatry, 27, 561–566.
457
O’Brien, C. P., & McKey, J. (2002). Pharmacological treatments for substance use disorders. In P. Nathan & J. Gorman (Eds.) A guide to treatments that work (2nd ed.). New York: Guilford Press. O’Connor, P. G., Oliveto, A. H., & Shi, J. M. (1996). A pilot study of primary-care– based buprenorphine maintenance for heroin dependence. American Journal of Drug & Alcohol Abuse, 22, 523–531. Pekrun, R., Goetz, T., Titz, W., & Perry, R. P. (2002). Academic emotions in students’ self-regulated learning and achievement: A program of quantitative and qualitative research. Educational Psychologist, 37, 91– 106. Plante, T. G., Andersen, E. N., & Boccaccini, M. T. (1999). Empirically supported treatments and related contemporary changes in psychotherapy practice: What do clinical ABPPs think? Clinical Psychologist, 52, 23–31. Preacher, K. J., & Hayes, A. F. (2004). SPSS and SAS procedures for estimating indirect effects in simple mediation models. Behavior Research Methods, Instruments, and Computers, 36, 717–731. Preacher, K. J., & Hayes, A. F. (2006). Asymptotic and resampling strategies for assessing and comparing indirect effects in simple and multiple mediator models. Unpublished manuscript. Retrieved May 1, 2006, from http://www.quantpsy.org. Rounsaville, B. J., O’Malley, S., Foley, S., & Weissman, M. M. (1988). Role of manual-guided training in the conduct and efficacy of interpersonal psychotherapy for depression. Journal of Consulting and Clinical Psychology, 56, 681– 688. Shea, S., DuMouchel, W., & Bahamonde, L. (1996). A meta-analysis of 16 randomized controlled trials to evaluate computer-based clinical reminder systems for preventive care in the ambulatory setting. Journal of the American Medical Informatics Association, 3, 399 – 409. Shweder, R. A., & Haidt, J. (2004). The cultural psychology of the emotions: Ancient and new. In M. Lewis & J. M. Haviland-Jones (Eds.), Handbook of emotions (pp. 397– 414). New York: Guilford Press. Simpson, D. D. (2002). A conceptual framework for transferring research to practice. Journal of Substance Abuse Treatment, 22, 171–182. Sloboda, Z., & Schildhaus, S. (2002). A discussion of the concept of technology transfer of research-based drug abuse prevention and treatment interventions. Substance Use & Misuse, 37, 1079 –1087. Sobel, M. E. (1982). Asymptotic confidence intervals for indirect effects in structural equations models. In S. Leinhart (Ed.), Sociological methodology (pp. 290 –312). San Francisco: Jossey-Bass. Thomas, C. P., Wallack, S. S., Lee, S., McCarty, D., & Swift, R. (2003). Research to practice: Adoption of naltrexone in alcoholism treatment. Journal of Substance Abuse Treatment, 24, 1–11. Turner, J. E., & Schallert, D. L. (2001). Expectancy-value relationships of shame reactions and shame resiliency. Journal of Educational Psychology, 93, 320 –329. Varra, A. A. (2007). The effect of acceptance and commitment training on clinician willingness to use empirically-supported pharmacotherapy for drug and alcohol abuse. Unpublished doctoral dissertation, University of Nevada, Reno. Varra, A. A., & Hayes, S. C. (2007). Assessing referrals for pharmacotherapy: A comparison of therapist and client report. Journal of Substance Abuse Treatment, 32, 411– 413. Zeidner, M. (1998). Test anxiety: The state of the art. New York: Plenum.
(Appendix follows)
VARRA, HAYES, ROGET, AND FISHER
458
Appendix Flow Diagram of Subject Progress Through the Phases of the ACT Intervention
Randomized to study (n = 60)
Allocated to ACT intervention (n = 30) Received ACT intervention (n = 30) Did not receive ACT intervention (n = 0)
Allocated to control intervention (n = 30) Received control intervention (n = 29) Did not receive control intervention (n = 1) Ð 1 randomized individual did not attend the conference for unknown reasons
Lost to post (n = 0) Discontinued intervention (n = 1) Ð Individual left the training, stating that he/she did not want to continue
Lost to post (n = 0) Discontinued intervention (n = 1) Ð Individual left the training, stating that he/she did not want to continue
Lost to follow-up (n = 5) Ð Study personnel were unable to contact these individuals for follow-up due to job or address change Discontinued intervention (n = 0)
Analyzed (n = 24) Excluded from analysis (n = 0)
Lost to follow-up (n = 3) Ð Study personnel were unable to contact these individuals for follow-up due to job or address change Discontinued intervention (n = 0)
Analyzed (n = 25) Excluded from analysis (n = 0)
Figure A1. Flow chart of participants’ progress through the phases of the acceptance and commitment training (ACT) intervention. p ⬍ .05.
Received September 25, 2007 Revision received February 7, 2008 Accepted February 8, 2008 䡲
