A randomized evaluation of consumer versus ... - Springer Link

Community Mental Health Journal, Vol. 31, No. 3, June 1995

A Randomized Evaluation of Consumer Versus Nonconsumer Training of State Mental Health Service Providers Judith A. Cook, Ph.D. Jessica A. Jonikas, M.A. Lisa Razzano, Ph.D.

ABSTRACT: P r e l i m i n a r y evidence suggests t h a t m e n t a l h e a l t h consumers can successfully serve as peer companions, case m a n a g e m e n t aides, case m a n a g e r s , job coaches, and drop-in center staff. However, few empirical investigations have addressed t h e use of consumers to t r a i n m e n t a l h e a l t h professionals. This project employed a r a n d o m i z e d design to test t h e effects of using consumers as t r a i n e r s for m e n t a l h e a l t h service providers. Fifty-seven state m e n t a l h e a l t h professionals p a r t i c i p a t e d in a twod a y t r a i n i n g designed to a c q u a i n t t r a i n e e s with the a t t i t u d e s and knowledge necessary for d e l i v e r i n g a s s e r t i v e case m a n a g e m e n t services. P a r t i c i p a n t s were r a n d o m l y assigned to one of two conditions: one in which t h e y received the second day of t r a i n i n g from a consumer and t h e other involving t r a i n i n g by a nonconsumer. A n a l y s e s r e v e a l e d t h a t p o s t - t r a i n i n g a t t i t u d e s were significantly more positive for those p a r t i c i p a n t s t r a i n e d by the consumer. Subjective e v a l u a t i o n s also reflected positive reactions to the use of consumers as t r a i n e r s . Implications for f u r t h e r use of m e n t a l h e a l t h consumers as t r a i n e r s are explored.

Funded, in part, by contract #MH19359 and #C151038092 from the Illinois Department of Mental Health and Developmental Disabilities, Springfield, Illinois. Also supported by the Center for Mental Health Services, Substance Abuse and Mental Health Services Administration, and the U.S. Department of Education, National Institute on Disability and Rehabilitation Research (Cooperative agreement #H133B00011). The opinions expressed herein do not reflect the position or policy of any agency and no official endorsement should be inferred. The authors gratefully acknowledge Elise Brooks, Mark Gervain, Eleanor Guzzio, Karen Lee, and Mardi Solomon, as well as participants in the training, for their valuable contributions to this research project. Dr. Cook is the Director of the Thresholds National Research and Training Center (TNRTC), 2001 N. Clybourn Avenue, Suite 302, Chicago, IL 60614. Ms. Jonikas is the Director of Dissemination for the TNRTC. Dr. Razzano is the Assistant Director of the TNRTC.

9'29

9 1995 Human Sciences Press, Inc.

230

Community Mental Health Journal

The use of mental health consumers as service providers has gained increasing attention in recent years. Because of their first-hand acquaintance with the mental health system, consumer service providers may possess unique insights, practical skills for manipulating the system, and an enhanced ability to empathize with client experiences. Consumers in this role are potential examples to professionals that recovery is possible, providing motivation and inspiration for working with persons who have mental illness. Several studies have indicated the positive effects of services delivered by mental health consumers as peer companions, case management aides, case managers, job coaches, and drop-in center staff (Cook, Jonikas & Solomon, 1992; McGill & Patterson, 1990; Mowbray, Chamberlain, Jennings, & Reed, 1988; Nikkel, Smith & Edwards, 1992; Sherman & Porter, 1991). This research has indicated that utilizing consumers as service deliverers is a cost-effective strategy (Mowbray et al., 1988), and provides program participants with beneficial levels of social support, acceptance, and problem solving assistance (Cook, Jonikas, & Solomon, 1992; McGill & Patterson, 1990; Mowbray & Tan, 1992; Sherman & Porter, 1991). Aside from these findings, there are few rigorous empirical evaluations of the impact of consumer service delivery on program participants or on other professionals in the field. The use of consumers to train mental health professionals has been advocated by those who point out the value of direct exposure to client populations in roles emphasizing strengths rather than deficits (Cook & Hoffschmidt, 1993). Consumer trainers can serve as examples of the degree of recovery possible with effective service delivery (Cook, Kozlowski Graham & Razzano, 1993). Moreover, they can directly address questions related to consumer perspectives on treatment and the value of consumer empowerment. Most importantly, consumer trainers can assume an ~expert" role that directly contradicts commonly-held stereotypes about persons with mental illness. Given these possibilities, it is hypothesized that receiving training from consumers will result in more positive attitudes toward people with mental illness t h a n receiving training solely from nonconsumers. To address this hypothesis, the present study employed a randomized design to test the effects of using consumers as trainers of mental health service providers. Trainees were state-funded community mental health staff who participated in a two-day course on the principles and practice of assertive community treatment (ACT) (Stein & Test, 1980; Test & Stein, 1978). Participants received the second day of training from either a consumer or a nonconsumer trainer and com-

Judith A. Cook, Ph.D., Jessica A. Jonikas, M.A., and Lisa Razzano, Ph.D.

pleted pretests and postests measuring with mental illness.

231

their attitudes toward persons

METHOD Subjects Fifty-seven state mental health professionals (34% male, 66% female) were the participants in this study. Eighty percent were Caucasian, 13 percent African American, five percent Hispanic, and two percent Asian. Trainees' ages ranged from 21 to 58 years with a mean of 34.5 years. They had spent an average of 3.5 years (ranging from one month to 10 years) at their current jobs and 8.4 years (ranging from four months to 35 years) working in the mental health field.

Instruments Training Curriculum. A two-day curriculum (Brooks, Jonikas, Cook, Engstrom, & Witheridge, 1991) was designed to acquaint mental health professionals with the basic concepts and techniques involved in delivering assertive community treatment services based on the ACT model (Stein & Test, 1980; Test & Stein, 1978). The curriculum was prepared by staff at the Thresholds National Research and Training Center in Chicago, Illinois under contract to the Illinois Department of Mental Health and Developmental Disabilities (DMH-DD). The first day covered qualities needed in an outreach worker, services comprising ACT, features of mental illness and substance use disorders, health issues, and financial entitlements. The second day addressed establishing a therapeutic alliance, using a total team approach, developing collateral linkages with community service providers, processing interactions with clients, establishing cultural sensitivity, and impacting public policy. Pedagogic methods included didactic sessions, question and answer segments, discussions, interactive problem solving using case examples, role plays, and worksheet completion. Training sessions, also funded by DMH-DD, were conducted at five locations in Northern and Central Illinois; two trainings were held in metropolitan areas and three in rural or small town settings. All trainees were state-funded mental health service providers who voluntarily attended during working hours. Pretest/Posttest Measures. The pretest/posttest instrument was designed to measure attitudes toward persons with mental illness, particularly in the roles of service recipient, service deliverer, and trainer. This 30-item instrument was compiled from several scales used in prior research: the Robert Wood Johnson Foundation survey of Public Attitudes Toward People with Mental Illness (Yankelovich, 1990); the Mental Health Opinion Statements scale (Olmsted & Smith, 1980); the Scale of Attitudes Toward Disabled Persons (Anotak, 1982); and items derived from focus group interviews with consumer service deliverers. Trainees rated their level of agreement or disagreement on a Likert scale ranging from 1 (Strongly Disagree) to 6 (Strongly Agree). Also included were: a one-page Background Information sheet requesting demographic information; and an open-ended training satisfaction questionnaire. The first step in the analysis was to create a series of outcome measures assessing trainees' positive attitudes toward mental health consumers. Attitudinal outcome mea-

232


sures were created from a factor analysis of the Likert scaled items described above, after removing four items with low intercorrelations. Using an iterative principal axis solution and squared multiple correlations as prior estimates of communalities, the presence of three factors was indicated. This conclusion was based primarily on preliminary eigenvalues that exceeded 2.0. These factors were rotated to a Varimax solution; following this, all items with low factor loadings (.47 or below) were eliminated from their respective factors. Items loading on each factor were weighted equally and summed (following reversal of negatively worded items) to produce an index of the degree to which trainees agreed with each set of statements. The first factor, CONSUMER PROVIDER, consisted of 11 items concerning the abilities of consumers as service deliverers, trainers, and as persons who can be relied upon. Trainees who scored high on this factor agreed more strongly with items such as: "Mental health consumers have much to teach us about how to work with mentally ill clients" and "Using mental health consumers to provide training to professionals is a good idea;" and disagreed more strongly with items such as ~Consumer service deliverers may have difficulty separating their own experiences from those of their mentally ill clients" and "I would feel uncomfortable ~venting' my feelings about situations I encounter in my case work if there was a consumer service deliverer present." Loadings for the CONSUMER PROVIDER factor ranged from .74 to .48. The second factor, labeled RECOVERY, consisted of eight items regarding the likelihood that mental health consumers could live a normal life, return to work, and become productive citizens. Trainees who scored high on this factor agreed more strongly with items such as "Most people with serious mental illness can, with treatment, get well and return to productive lives" and "Many of the people who go to mental hospitals are able to return to work in society again." Loadings for the RECOVERY factor ranged from .69 to .49. The third factor, termed NONSTIGMATIZING, consisted of four items addressing ~typical" stigmatizing attitudes about people with mental illness regarding their potential for violence, their personal appearance, and the likelihood of their being the kind of clients that are personally satisfying to work with. Trainees who scored high on this factor disagreed more strongly with items such as "You can tell a person who is mentally ill from his appearance" and "People with chronic mental illnesses are, by far, more dangerous than the general population," and agreed more strongly that "I get quite a bit of person satisfaction from delivering services to persons who have severe mental illness." Loadings for this factor ranged from .56 to .50. Next, it was necessary to ascertain whether the attitudinal measures were internally consistent and appropriate for use as outcome measures. Reliability analyses were conducted for each of the three factors as well as a total score, termed PROCONSUMER, comprised of all 23 items. Reliability coefficients (Cronbach's alpha) indicated excellent reliability for PROCONSUMER (alpha = .83), and CONSUMER PROVIDER (alpha = .83), adequate reliability for RECOVERY (alpha = .73) and modest reliability for NONSTIGMATIZING (alpha = .58). These four outcome variables were used as dependent variables in the analyses.

Procedures Immediately preceding the first day of training, trainees completed the pretest and numbered it with a unique identifier of their own choosing. Following pretesting, all trainees received the first day of training from the nonconsumer. Although it would have been ideal to deliver consumer training on both days of training, project financial limitations and the consumer's own preferences led to the decision to limit consumer


233

instruction to the second day. At the beginning of the second day, trainees were randomly assigned to receive training from either the consumer or the nonconsumer trainer. Random assignment involved alphabetizing trainees' surnames and assigning every other name to an experimental (E) or control (C) condition. At the end of the second day of training, participants completed the posttest along with their unique identifier, allowing matching of the instruments while maintaining subject anonymity. T-test analysis of the experimental and control groups revealed no significant differences by age, education, number of years in current position, or number of years in the mental health field. There were significant differences on gender (more women in the experimental [consumer training] group) and ethnicity (more minorities in the experimental condition). Subsequent statistical analyses were run with controls for all demographic features. In order to avoid differential response biases due to the fact that only one individual was used in each of the training conditions, the consumer and nonconsumer trainer were matched on the following characteristics: gender (female), ethnicity (Caucasian), education level (master's degree), and profession (mental health service provider). In order to match the use of personal examples in the two conditions, the nonconsumer trainer talked about her experiences as a family member of a relative with severe psychiatric disturbance while the consumer talked about her own experiences with mental illness. Another difference was that the role of the nonconsumer trainer in coauthoring the curriculum presumably gave her the advantage of knowing it more thoroughly than the consumer trainer. Finally, the consumer trainer had been diagnosed with a bipolar disorder 15 years earlier, and had a recurrent history of psychiatric hospitalizations and psychotropic medication usage.

R E S UL TS To t e s t t h e h y p o t h e s i s t h a t t h e r e w o u l d be s i g n i f i c a n t l y m o r e p o s i t i v e p o s t t e s t a t t i t u d e s a m o n g t h o s e t r a i n e d by t h e c o n s u m e r t r a i n e r t h a n t h e n o n c o n s u m e r t r a i n e r , OLS r e g r e s s i o n w a s u s e d in m o d e l s consisti n g of p r e t e s t score, e x p e r i m e n t a l c o n d i t i o n ( c o n s u m e r v e r s u s noncons u m e r t r a i n e r ) , t r a i n e e s ' g e n d e r , age, m i n o r i t y s t a t u s , a n d n u m b e r of y e a r s e m p l o y e d in t h e m e n t a l h e a l t h field. As s h o w n i n T a b l e 1, c o n t r o l l i n g for all v a r i a b l e s , t h o s e t r a i n e d b y t h e c o n s u m e r h a d s i g n i f i c a n t l y m o r e positive p o s t t e s t a t t i t u d e s on t h e C O N S U M E R P R O V I D E R subscale. H e r e , p r e t e s t scores a n d experim e n t a l condition were the only significant variables. Thus, even after c o n t r o l l i n g for i n i t i a l a t t i t u d e s t o w a r d c o n s u m e r s a n d i n d i v i d u a l t r a i n e e c h a r a c t e r i s t i c s , t h o s e t r a i n e d b y t h e c o n s u m e r felt m o r e posi t i v e l y a b o u t c o n s u m e r s as service providers. T u r n i n g n e x t to t h e R E C O V E R Y o u t c o m e , o n l y p r e t e s t scores w e r e s i g n i f i c a n t in t h e model. All o t h e r v a r i a b l e s , i n c l u d i n g t h e e x p e r i m e n t a l c o n d i t i o n , w e r e n o n s i g n i f i c a n t . T h u s , t r a i n i n g by a c o n s u m e r did n o t r e s u l t in s i g n i f i c a n t l y m o r e positive a t t i t u d e s a b o u t t h e r e c o v e r y potent i a l of people w i t h severe m e n t a l illness.

234

Community M e n t a l H e a l t h Journal

TABLE 1 R e g r e s s i o n Models (with Control Variables) Predicting Posttest Attitudes T o w a r d C o n s u m e r s b y Pretest Scores a n d Trainer Status a Experimental Condition

Pretest Score

Consumer Provider Recovery Nonstigmatizing Proconsumer

ba

Beta & Sig. a

b~

B e t a & Sig. ~

R 2a

Sig. a

.58 .77 .52 .66

.67**** .78**** .61"*** .70****

3.32 0.82 2.04 6.73

.23* .10 .40*** .31"*

.60 .65 .55 .62

.00001 .00002 .0000s .00004

*p < .05 **p < .01 ***p < .001 ****p < .0001 aEstimates under the complete model, controlling for gender, minority status, age, and job tenure in the m e n t a l h e a l t h field. ldf = (6,40) 2df = (6,42) ~df = (6,41) 4df = (6,38)

In the model predicting NONSTIGMATIZING, both the pretest score and the experimental condition were significant. As in prior models, none of the respondent background variables were significant. Even controlling for trainees' baseline attitudes and other characteristics, those trained by the consumer trainer had significantly more nonstigmatizing attitudes following the training. Finally, in the model predicting total PROCONSUMER attitudes, pretest score and experimental condition were the only significant variables in the equation. Here, controlling for all trainee demographic and work history variables as well as initial PROCONSUMER attitudes, those trained by the consumer had significantly more positive attitudes following training. As hypothesized, after controlling for baseline attitudinal levels, those trained by the consumer trainer showed more positive attitudes overall, felt more positively about consumers as service providers and trainers, and expressed more nonstigmatizing attitudes than did those trained by the nonconsumer. The only outcome that was not signifi-


235

cantly affected by the consumer trainer concerned attitudes about the recovery potential of people with mental illness.

Qualitative Reactions to the Training In the training satisfaction questionnaires, many participants commented on the value of receiving training from the consumer trainer. Several themes were identified in these comments. First, trainees felt that the consumer trainer was as professional and effective as the nonconsumer trainer. The following quotes were typical: [ ] did a great job facilitating. She was well informed, personable, had good anecdotes, and tried to accommodate the needs of the group. The last day we were instructed by [ ] and she was a fantastic facilitator. [She was] organized, focused, and interesting.

No one expressed negative opinions about the consumer trainers' ability to convey the material, generate discussion, or clarify information. Another theme concerned the impact on trainees of the consumer trainer's sharing of both personal and professional experiences. This ability to provide a double perspective captured trainees' attention and offered a more integrative learning experience. As two trainees put it: It was very helpful to have [ ]'s perspective both as a consumer and a provider. I appreciate the generosity with which she shared her experiences with us. [ ] clearly has had experiences both personally and professionally t h a t were invaluable as training tools.

A third theme revealed a degree of consciousness-raising in addition to increased knowledge levels. Phrases such as ~'new ideas and insights," ~'thought-provoking," and ~'eye opening" were used to describe the effects of receiving training from a mental health consumer. Enjoyed t h e . . , consumer involvement. H a v e broadened my understanding and awareness in this a r e a - M a n y ideas to think a b o u t - v e r y thought-provoking and

helpful. Thank you!

For many participants, the consumer trainer was the highest functioning person with mental illness they had ever encountered. This led them to confront some of their own stereotypes about what is possible for persons with this disability, recognizing for the first time the strengths and wellness of persons with mental illness.

236


I found being trained by [ ] helped me appreciate the consumers' views more. Our clients are not as functional in the community and meeting her has reinforced t h a t some of m y clients can live better lives. The entire group should have had the opportunity to meet with the consumer trainer. We are (at least I am) used to seeing lower-functioning consumers a n d . . , the experience was interesting, stimulating, and eye-opening.

Finally, a telling indicator of the value of this type of training was the fact that participants urged that future trainings include consumers. Comments suggested that we ~'use consumers more and more in the training;" and continue to ~have consumer case managers come share their perspectives." As one participant put it: Our last day of training was by far the best. Having a consumer trainer (service provider/professional and consumer) gave us the ability to see a more holistic picture of psychosocial rehabilitation. Use [ ] as much as you can for [training] psychosocial service providers and all professionals working in this field.

DISCUSSION The results of our analysis revealed that state-funded mental health service providers educated by a consumer trainer had more positive attitudes toward people with mental illness following the training t h a n did those trained by a nonconsumer trainer. Even controlling for pretest attitudes and respondent background characteristics, those in the experimental condition showed more positive posttest attitudes overall, felt more positively about consumers as service providers and trainers, and expressed more nonstigmatizing attitudes t h a n those trained solely by the nonconsumer. Comments from the trainees indicated several reasons for the consumer trainer's advantage. While in a trainer role invested with competence and authority, the consumer impressed her students with her training skills. For some, the consumer trainer was a rare example of a high-achieving person with severe psychiatric disability. Others were inspired by her unique insights as both a consumer and a service deliverer. The only outcome t h a t was not significantly affected by the consumer trainer concerned attitudes about the recovery potential of people with mental illness. Perhaps training effects are weakest for this type of outcome. Since both groups were trained by the nonconsumer on the first day, this may have further decreased the degree of difference


237

between the two conditions. It also is possible that a larger sample size would have resulted in a significant difference on this outcome as well. In addition to the power limitations of the sample, there are several caveats to these findings. First, the results may simply be due to the novelty of having a new trainer on the second day of the curriculum. It is possible that any new individual personally knowledgeable about the system might achieve better results than use of the same first-day trainer. However, the evaluation comments made by the trainees suggest that it was the nature of the new trainer as a c o n s u m e r that led to their unique insights and a positive learning experience. Another caveat concerns limitations on external validity stemming from the fact that only one consumer was used in the experimental condition. On the basis of these results alone, findings cannot be generalized to all consumers who might serve as trainers. Future studies should include a pool of consumer trainers in order to test for effects as well as to control for individual differences. Additional research also should address the effects of training that is mandatory, unlike the present study's training, for which providers volunteered. Also noteworthy were the support needs of the consumer in areas such as learning the material, deciding how much personal information to disclose, and feeling comfortable as a trainer. While the nonconsumer also required help and support, the larger share of assistance was directed to the consumer trainer. Indeed, the training's success depended to a large extent on the willingness of the nonconsumer trainer to act as support source and logistics coordinator. Such a partnership model of co-training worked particularly well in this instance and may be worthy of replication. However, studies also should investigate the effects of training delivered solely by consumers. Overall, the use of a consumer trainer achieved positive results and enhanced the effectiveness of the training. If the goal is to make mental health service delivery training an insightful and integrative experience, this study provides preliminary evidence that such an approach may be a viable option. For mental health consumers, the trainer role expands the range of employment opportunities and draws upon their familiarity with the mental health system and service recipient perspective. For trainees, instruction by a consumer is inspiring and thought-provoking. Hopefully, future research will investigate consumer training using more rigorous designs along with multiple trainers in each condition. Also important will be a better understanding of whether or not the use of consumer trainers has benefits for the recipients of services as well.

238


REFERENCES Anotak, R.F. (1982). Development and psychometric analysis of the scale of attitudes toward disabled persons. Journal of Applied Rehabilitation Counseling, 13(2), 22-29. Brooks, E.B., Jonikas, J.A., Cook, J.A., Engstrom, K., & Witheridge T. (1991). Assertive Case Management in the Community. In K. Lee, (Ed.), Psychosocial Rehabilitation Treatment in the Community: A Comprehensive Curriculum for Mental Health Personnel Illinois Department of Mental Health and Developmental Disabilities. Cook, J.A. & Hoffschmidt, S.J. (1993). Comprehensive Models of Psychosocial Rehabilitation, in R.W. Flexer & P. Solomon (eds.), Psychiatric Rehabilitation in Practice. (pp. 81-97), New York: Butterworth-Heinemann. Cook, J.A., Jonikas, J.A. & Solomon, M.L. (1992). Models of vocational rehabilitation for youths and adults with severe mental illness. American Rehabilitatior~ 18(3), 6-11. Cook, J.A., Kozlowski Graham, K. & Razzano, L. (1993). Psychosocial rehabilitation of deaf persons with severe mental illness: A multivariate model of residential outcomes. Rehabilitation Psychology, 38(4), 261-274. Daniel Yankelovich Group. (1990). Public attitudes toward people with chronic mental illness: final report. Boston: Robert Wood Johnson Foundation, Program on Chronic Mental Illness. McGill, C.W. & Patterson, C.J. (1990). Former patients as peer counselors on locked psychiatric inpatient units. Hospital and Community Psychiatry, 41(9), 1017-1019. Mowbray, C.T., Chamberlain, P., Jennings, M., & Reed, C. (1988). Consumer-run mental health service: Results from five demonstration projects. Community Mental Health Journal, 24(2), 151-156. Mowbray, C.T. & Tan, C. (1992). Evaluation of an innovative consumer-run service model: The drop-in center. Innovations and Research, 1(2), 19-23. Mowbray, C.T., Wellwood, R. & Chamberlain, P. (1988). Project Stay: A consumer-run support service. Psychosocial Rehabilitation Journal, 12(1), 33-42. Nikkel, R.E., Smith, G. & Edwards, D. (1992). A consumer-operated case management project. Hospital and Community Psychiatry, 43(6), 577-579. Olmsted, D.W. & Smith, D.L. (1980). The socialization of youth into the American mental health belief system. Journal of Health and Social Behavior, 21, 181-184. Sherman, P.S. & Porter, R. (1991). Mental health consumers as case management aides. Hospital and Community Psychiatry, 42(5), 494-498. Stein, L.I. & Test, M.A. (1980). Alternative to mental hospital treatment. Archives of General Psychiatry, 37, 392-397. Test, M.A. & Stein, L.I. (1978). Community treatment of the chronic patient: research overview. Schizophrenia Bulletin 4(3), 350-364.