Community Ment Health J (2008) 44:213–224 DOI 10.1007/s10597-007-9109-4 ORIGINAL PAPER
Evidence-based Practice Implementation Strategies: Results of a Qualitative Study Charles A. Rapp Æ Diane Etzel-Wise Æ Doug Marty Æ Melinda Coffman Æ Linda Carlson Æ Dianne Asher Æ Jennifer Callaghan Æ Rob Whitley
Received: 30 November 2006 / Accepted: 21 September 2007 / Published online: 1 November 2007 Ó Springer Science+Business Media, LLC 2007
Abstract This study reports on the strategies used to implement the evidencebased practices of supported employment and integrated dual diagnosis treatment. Using qualitative research methods, the study uncovered eight strategies that contributed to successful implementation in six sites. Keywords Evidence-based practice Implementation strategies Mental health administration
Introduction Evidence is overwhelming that people with severe mental illness infrequently receive effective services in community mental health centers (CMHCs) (U.S. Department of Health and Human Services 1999; President’s New Freedom Commission on Mental Health 2003; Lehman et al. 1998; West et al. 2003; Tashjian et al. 1989). Torrey and Gorman (2005) note that lack of knowledge regarding successful implementation of proven interventions in CMHCs inhibits the development of effective service provision. The National Implementing EvidenceBased Practice (EBP) Project was designed to explore the facilitating conditions, barriers, and strategies that affected implementation of EBPs in CMHCs (McHugo et al. in press). The project involved 53 sites in eight states with each CMHC implementing one of the following EBPs: Supported employment (SE), integrated dual diagnosis treatment (IDDT), family psychoeducation, illness self-management, or assertive community treatment. The implementation intervention lasted 2 years with the first year being devoted to implementation and the second to sustaining. C. A. Rapp (&) D. EtzelWise D. Marty M. Coffman L. Carlson D. Asher J. Callaghan R. Whitley School of Social Welfare, The University of Kansas, Twente Hall, Lawrence, KS 66044, USA e-mail:
[email protected]
123
214
Community Ment Health J (2008) 44:213–224
A previous report described the barriers to implementation of IDDT and SE in five CMHCs in Kansas (Rapp et al. in press). Three significant barriers emerged: (1) Deficits in skills and role performance by front-line supervisors; (2) resistance by front-line practitioners (3) failure of other agency personnel (e.g., medical staff) to adequately fulfill new responsibilities. This article focuses on strategies employed to overcome these barriers and achieve successful implementation outcomes in the five Kansas CMHCs involved in the project.
Methods Sample The Kansas’ Department of Social and Rehabilitation Services (SRS), the umbrella agency for the state mental health authority, issued a request for applications from CMHCs interested in implementing either IDDT or SE. The SRS Commissioner of Mental Health selected five CMHCs according to the guidelines presented by the National Project oversight committee (one site offered both EBPs). This included a mix of rural and urban sites and commitment of agency leadership. Each CMHC created a leadership team consisting of the CMHC executive director, Community Support Services director, program leader, consumers, families, and a state representative. On the ground, the implementation team consisted of a program leader (PL) and 3–6 direct service staff. This team was assisted by a Consultant and Trainer (CAT) assigned to the site as the principle support for implementation.
Data Collection Implementation data were collected over 2 years by independent implementation monitors (IMs) during monthly site visits where trainings, leadership meetings, team meetings, and everyday activities were systematically observed. The IM also conducted structured interviews with the PL and CAT every 6 months. The CAT also made written observations of the implementation process on a monthly basis. IMs and CATs met regularly to share observations, notes and impressions. In order to enhance trustworthiness and credibility (Lincoln and Guba 1985), IMs participated in monthly conference calls with the National Coordinating Centre (Dartmouth Psychiatric Research Centre) and attended annual meetings for training and feedback. Similarly, IMs submitted examples of collected data to the National Centre on a monthly basis for review of appropriate scope and quality. Formal fidelity reviews were conducted at 6-monthly intervals from baseline to 24 months by the IM and CAT. These reviews assessed the CMHCs degree of adherence to the practice model of the EBP. This practice model was given to the leadership team of each CMHC as a gold standard to which they should aspire to reach (Bond et al. 2000). Detailed protocols directed the reviewers inquiry. Fidelity scores were based on interviews with consumers, staff and administrators, observation of practice and team meetings, and reviews of a sample of case
123
Community Ment Health J (2008) 44:213–224
215
records. This information was then used to rate the individual fidelity items on a five-point scale. The CAT and the IM independently rated levels of practice fidelity before a composite score was calculated.
Data Analysis Firstly the raw data for each site was organized into three categories: Facilitating conditions, strategies, and barriers. Facilitators referred to pre-intervention conditions assisting EBP implementation. Strategies referred to intentional actions assisting EBP implementation. Barriers referred to activity inhibiting EBP implementation. Secondly, themes were developed within 26 a priori deductively formulated categories such as ‘‘staffing,’’ ‘‘attitude,’’ and ‘‘engagement.’’ A theme was defined as a thread of activity or condition that was related to the category and considered ‘‘salient, prominent, conspicuous, or non-ignorable.’’ Key stakeholders involved with the theme were also identified. These coding processes occurred within Atlas-ti qualitative software (Atlas.ti 2002). On completion of the intervention (24 months), the IM wrote a comprehensive site reports describing the process and outcome of implementation (24 months). For this paper, the six site reports were analyzed inductively by three investigators, who were blinded to the titles of the 24 a priori themes described in the reports. The goal was to develop a set or category of emergent strategies that adequately organized and accounted for the data in its local context. This was done iteratively as perceived flaws and inadequacies in a category led to ongoing reformulations of the emergent schemata (Strauss and Corbin 1990). Once a provisional coding guide was established, one site was selected and two analysts independently re-coded data relevant to strategies. In conjunction with the primary investigator, the results were compared. Through ongoing discussion, codes were modified and a coding guide established. This was then applied to the data of all six sites. Part of this process involved returning to the raw data or the IMs/CATs in order to clarify, inform, and elaborate on ambiguous data.
Results Five strategies emerged from the data (see in Table 1). These are described separately in this section.
Instituting Expectations Evidence-based practices impose a new set of expectations on front-line staff. These include increased documentation, the acquisition, and use of new skills (e.g., motivational interviewing), and more sophisticated time-management. There were varying degrees of acceptance/resistance among staff to these new expectations, ranging from enthusiasm to overt hostility. However positive actions by front-line
123
216
Community Ment Health J (2008) 44:213–224
Table 1 Strategies employed
Strategy 1
Instituting expectations
2
Unique role of upper management Championing the EBP Changing structure and policy
3
Creating intra-agency synergy Integrative structure: Leadership teams Integrative structure: EBP team meetings
4
Managing information: Fidelity measurement
5
Training and consultation
supervisors and upper-management tended to overcome hostility to new expectations. In the beginning, supervisors in five of the six sites resisted significant pieces of the EBP and appeared to be weak leaders. In each case, expectations of front-line staff were not set and/or not monitored and enforced. Implementation was sluggish until new supervisors were hired. In contrast, one site had a tradition of expectationsetting and concomitant staff enthusiasm to meet these expectations. This site had quicker and more successful implementation outcomes. In cases of practitioner resistance to new EBP expectations, a combination of education and persuasion were used first to engender acceptance. PLs and CATs provided information, research results, and case vignettes to practitioners. They met individually with practitioners to address concerns. In conjunction with upper administration, they would convey enthusiasm about the EBP. This was generally effective with those who were simply skeptical about the EBP. However, this strategy was ineffective on those who were opposed outright to the EBP. Progress was only made with such recalcitrant individuals when the supervisor and/or upper management stated ‘‘get in line or take another position.’’ Some practitioners quit, others transferred to other positions in the agency (often quite successfully), others were counseled to find other employment and some were fired. New hires were people who accepted the expectations and were judged to have the necessary skills to develop in the post. Without upper management intervention as the ‘‘ultimate enforcer,’’ implementation of the EBP’s would likely have been slower and less successful. The clarity and conscientious monitoring of expectations seemed to go together. PLs who set behavioral standards for practice (e.g., % of time in community) and outcomes (e.g., number of consumer employed) were also most likely to closely monitor performance through the use of data, field supervision, review of documentation and team meetings. They shared performance information, provided feedback, and strategized avenues of improvement. They also sought to alter wider barriers to performance like agency productivity policies. Similarly, they induced cooperation from others in the agency, for example getting psychiatrists more closely involved with IDDT. In general, things moved slower for IDDT than SE, as the supervisor was often learning the practice at the same time as the front line staff.
123
Community Ment Health J (2008) 44:213–224
217
Unique Role of Upper Management: Championing The Ebp In each site, a champion of the EBP was present. In two sites, the principal champion was the executive director of the agency. One of these (SE) was located in a large urban CMHC, the other (IDDT) a small, rural one. The urban center’s executive championed the practice by frequently expressing commitment to high fidelity implementation. The executive communicated confidence that the CMHC would succeed and her unwillingness to accept second-best, asking pertinent questions and ensuring relevant answers (e.g., ‘‘Why can’t we do this?,’’ ‘‘How are we going to do it?’’). The director modeled the ‘‘can-do’’ approach by making supportive decisions, for example transferring, upgrading or adding positions. When agency personnel became distracted by other demands, she would re-focus the team back to the EBP. The rural site executive drove the entire implementation. This executive attended and was active in the Leadership Team meetings, frequently expressing commitment to the EBP, confidence in staff, and expectations that implementation would be a success. The executive also made supportive decisions such as reallocation of resources and adjustment of policies. When interest in IDDT waned among staff, the executive reestablished expectations and took measures to boost staff confidence, for example attending staff training as a participant. Additionally, the Executive Director, the Community Support Services (CSS) Director and the Substance Abuse Director all visited an out-of-state recommended IDDT program along with the CAT to learn more about successful implementation. At one point, the executive director presented the results of a fidelity review to the Board of County Commissioners in order to sustain support for implementation. In three sites (2-SE; 1-IDDT), the executive director was only occasionally involved but authority for implementation was delegated to the CSS Director, who became the principle champion. Each of these sites had rather weak project supervisors and some front-line practitioners who were hostile toward implementation. In two cases, the CSS Directors instigated or supported the transfer or firing of recalcitrant workers, as well as the replacement of supervisors with more able and committed people. In one site, the CSS director was instrumental in making many positive changes, but was unwilling to assertively intervene with an underperforming supervisor. This project attained the lowest fidelity score of the six sites. In all cases, EBP champions allocated necessary resources, set expectations, and helped remove obstacles to implementation. They became ‘‘partners’’ with the CAT to ensure successful implementation.
Unique Role of Upper Management: Structural and Policy Change Two categories of structure/policy were targeted by the EBP Project. First both IDDT and SE fidelity ratings contained items that were structural in nature, in contrast to service or clinical items. In general, structural items refer to the organization of service delivery for the EBP practice. Examples for SE would include fidelity items concerning caseload size, integration with mental health
123
218
Community Ment Health J (2008) 44:213–224
services, and specification of a vocational specialist’s job. For IDDT, examples would include the composition of the multidisciplinary team and the expectation that services are delivered in an outreach mode. IDDT contains three structural items and SE contains six. Second, sites confronted obstacles to effective EBP services that emanated from their agency, catchment area or state system. In some cases, structural/policy remedies were pursued. The Kansas SE-EBP sites started the project already high in fidelity to the six structural items, with scores of 3.6, 3.8, 4.0 (Max = 5, Min = 1). These scores further increased during the 2 years of the project to 4.6; 4.6; 5.0. Biggest gains occurred during the initial 6 months. These changes were reflective of upper management’s commitment to implementation, as only they had the authority to make the necessary changes. In fact, once notified of fidelity criteria, all three sets of upper management initiated the necessary changes. For example, in all three SE sites, a decision was quickly made that employment specialists should be assigned to specific case management teams. Their caseload was simultaneously limited to consumers from that team, and they were required to attend all case management team meetings. Other facilitative policy changes were: Allowing indefinite followalong supports, discontinuing pre-vocational service and entry to SE based solely on expressed desire to work. The three IDDT structural items at baseline averaged 4.0, 2.25, 3.5. The scores increased over the 2 years of the project to: 5.0, 4.25, 4.0. For one site, the biggest improvement occurred during the initial 6 months. This was the only site with an executive director who was integrally involved in EBP implementation from day one. The other two sites saw most increase between 6 and 12 months. The most notable improvement in both concerned the structuring of a multidisciplinary team that included an integrated substance abuse specialist. Other facilitative policy changes were: Additional supervisory meetings that included the psychiatrist and were focused on DD consumer, removal of limits on substance abuse counselors’ time to serve consumers, the assignment of substance abuse specialist to team meetings and reduced caseloads to 15:1. The fact that each agency already had an SE program where no agency had an IDDT program at baseline may explain the time differential in making structural changes by EBP. Two-thirds of the sites adjusted their standards for billable hours by lowering them for the EBP personnel. This allowed employment specialists to increase their time for job development and for IDDT workers to attend on-going training to develop IDDT skills.
No Program is an Island: Creating Intra-Agency Synergy For a given EBP to reach high fidelity, agency personnel, beyond the staff targeted for EBP implementation, are required to practice in certain ways. Three strategies were used by the six EBP sites to ensure other personnel fulfilled this requirement. The first was education and training. This involved activities such as general orientation to the practice and training specific to the behaviors necessary for successful EBP implementation. For IDDT, this included training medical staff on best-practice prescribing and follow-up consultations for dually diagnosed
123
Community Ment Health J (2008) 44:213–224
219
consumers, and educating substance abuse staff on IDDT philosophy and methods. For SE this included showing how work can be effective ‘‘therapy’’ for many consumers (a special training was designed for case managers) and educating therapists on the benefits of work and the research that work does not necessarily increase stress for consumers. Often the education and training occurred through the joint activity of CATs, PLs, and upper management; they made positive comments about the EBP and shared relevant information widely. The second strategy was consultation and modeling. Here the CAT or PL would intervene in a particular situation where the EBP practitioner was having difficulty influencing other agency personnel to practice in a way consistent with the EBP. For example, specific case managers, medical staff or therapists who would not support a consumer’s desire to work were targeted for intervention. On occasion, the CAT would model complex skill sets. For example, in SE employment specialists were each assigned to a specific case management team to enhance integration. While the specialists dutifully attended, they were often passive. The employment specialists, the CM’s, and CM supervisor were not clear on their role and contribution in the meeting. The CAT or supervisor attended the case management team meetings and modeled appropriate behaviors; for example intimating ideas about work being a solution to a particular consumer problem or asking if consumers had ever talked about work. The IDDT CATs ability to problem-solve during case reviews made a strong impression on many practitioners and aided in the acceptance of IDDT. The third strategy involved structural changes to enhance on-going participation of agency personnel. In many cases, they were made members of the leadership team. In IDDT, the integrated treatment team was created (or altered) to involve psychiatrists, nurses, psychosocial staff, substance abuse counselors, and SE workers. In SE, one site integrated the benefits’ specialists in team meeting. At times, this involved changing meeting times or altering the structure so that participation could occur. As participation increased, knowledge and skills increased, and more consistent agency-wide practice occurred.
Integrative Structure: Leadership Teams The National EBP Project recommended the creation of a steering committee at each site. In Kansas, they were called ‘‘leadership teams.’’ These teams brought a variety of key actors together to improve EBP implementation. Membership at all sites included the agency executive director (or designee), CSS director, team supervisor, a representative of the direct service workers, consumer representative, state field support service person and CAT. At IDDT sites, a substance abuse counselor was part of this team as was the Director of Substance Abuse services. In one IDDT site, the psychiatrist was a member. For two of these sites, the Director of SA services also served as co-program leader of the implementation team. The SE sites all had the Vocational Rehabilitation Chief from the region on the team. In one site, all members of the SE team were members of the leadership team. In no case, were the sites successful in recruiting a family member for regular involvement.
123
220
Community Ment Health J (2008) 44:213–224
Leadership teams oversaw EBP implementation. The CAT led the meetings for the first year, then an agency person took over. As such, they monitored implementation through frequent reports by the PL and CAT, and fidelity reviews. They made decisions concerning training, ensuring the necessary EBP structural features were adopted such as changes regarding job duties, caseload size and documentation. They also ensured adequate resources were available for implementation, as well as aligning other agency practices to be consistent and supportive of the EBP. Finally, they formulated solutions to obstacles as they arose. For 12 months, the leadership team met monthly, before moving to bi-monthly or quarterly meetings. There seemed to be a common evolution to the leadership teams. Teams were constituted 4–6 months before implementation kick-off. The first phase served as a principal mechanism by which the CAT could educate staff about the EBP, thereby boosting the agency’s commitment to change. The second phase focused on planning and sponsoring the kick off events, and training of staff. The third phase was dominated by structural and personnel changes. The fourth phase focused on overcoming barriers to implementation. The final phase was preoccupied with developing the mechanisms for sustaining the EBP. These ‘‘phases’’ should not be conceptualized as homogenous time-periods, but rather points of emphasis during these times. The leadership teams were a critical component of successful implementation. As all the key stakeholders were involved in the team and the team was the prime decision-making mechanism, decisions were easily supported and defended agencywide. Decisions made were decisions executed, in a timely and efficient manner. This stands in contrast to the more typical pattern of a recommendation being passed up various levels of the organization whereby each level has de facto veto power, leading to costly and time-consuming revision and indecision.
Integrative Structure: Ebp Team Meetings While the leadership team was the core mechanism for administrative decisionmaking, team meetings in conjunction with group supervision (Rapp and Goscha 2006) were the key integrative mechanism at the service delivery level. The central task of these meetings was the generation of promising ideas to work more effectively with clients through the skilled use of EBP practices. This was done through reviews of case situations where progress has not been occurring. Typically each case situation consumes 20–30 min and 2–4 cases can be covered during a 2-h session. Group supervision was a critical vehicle for improving EBP fidelity. It was the setting where supervisors and CAT’s helped direct services workers apply the clinical skills to the myriad of idiosyncratic case situations. All supervisors were trained and tutored in leading group supervision and the CAT modeled the practice. The consistent presence of the CATS in team meetings was vitally important in this regard. Two SE supervisors (the two that replaced earlier ones) quickly became proficient in providing the necessary structure and expectations for group
123
Community Ment Health J (2008) 44:213–224
221
supervision. By 24 months, each EBP supervisor was operating well-run sessions, though hitches were encountered on the way at some sites. One SE supervisor took the better part of the 24-months to develop sufficient discipline. For example, case reviews often did not follow the guidelines and therefore evolved into unfocused unproductive discussions. In other cases, group supervision time would be consumed discussing administrative matters. This was in a large part due to the team leader’s inability to provide expectations and structure for the team meeting.
Managing Information: Fidelity Measurement ‘‘Managing information’’ refers to the design and use of information (data) for monitoring, goal-setting, and evaluation, with the aim of instigating management action and program improvement. Fidelity measures developed for SE and IDDT were the principle means for assessing the implementation of the key elements of the service. As previously stated, fidelity was assessed by the CAT and IM every 6 months from baseline to 24 months. The CAT would write a report after each assessment based on this data that included fidelity ratings, supporting evidence and recommendations for next steps. This was then presented to the leadership team and staff. The ratings were trusted by the agencies as accurate, were used to set goals, celebrate successes, and make plans for improving the scores. The fidelity measures required information that was not previously collected at the CMHC. For IDDT, examples would include consumers in the action or relapse prevention stage receiving substance abuse counseling, and those participating in self-help groups. For SE, examples would include employment specialist time in community, number of job development contacts, date of first employer contact, and type and location of employment. In this way, the fidelity measures provoked systematic data collection on critical service variables not previously collected. There is evidence that the collection of data, under certain conditions, can generate energy around the activity being measured (Nadler 1977; Taylor et al. 1984; Taylor 1987). A few supervisors were adept at using this information to monitor components, giving appropriate feedback to staff to improve performance.
Training and Consultation Training and consultation was a core component of the implementation intervention package. The CATS devoted 2 years to working with each EBP site, visiting for consultation bi-monthly during the first year. These diminished to monthly visits during the second year. Training employed lectures, power-point slides, roleplaying, discussion, and application of methods to specific case situations. In IDDT, emphasis was placed on assessing stages of change, motivational interviewing skills, substance abuse counseling skills, and other IDDT related assessments. The CAT also supported study groups where IDDT practitioners could practice skills. In team meetings, the CAT would help practitioners present case situations using the stages of change model. The CAT began field mentoring when practitioners
123
222
Community Ment Health J (2008) 44:213–224
conducted IDDT case management. The CAT would observe, give feedback, and at times model the IDDT skills. In SE, emphasis was placed on using the vocational profile, follow-along supports, individualized job search and job development. Training was provided on using motivational interviewing as a means of engagement. Job development, the work one does with an employer to help a consumer obtain a job, was new and uncomfortable for many practitioners. The CAT arranged for a special workshop by a renowned expert in the area. Even with this extra training and consultation, job development skills remained difficult for some practitioners to master. The CAT began to do mentoring where she would go with practitioners as they did job development with employers in the field. The CAT would observe, give feedback, and model the job development skills. It became clear that the CAT nor the supervisor had a clear or complete picture of the practitioners job development work in the field. The field mentoring was a major breakthrough in teaching skills and increasing the comfort level of employment specialists. Although naturally reluctant at first to have their practice observed, after a short period of time most perceived of it as helpful. After our training in job development, our supervisor set the expectation that we should be doing job development and set a certain number of contacts that we should be having with employers each month. I didn’t really do it. I didn’t like to do job development and did not feel comfortable doing it. My supervisor said she wanted to help us be more effective with our job development and would be going out regularly with us to talk with employers. At first, I was really upset and did not want to do it. After a while, though, I realized that I was feeling more comfortable with talking with employers and that my supervisor was really helping me do it better. I really liked it when I started getting clients jobs through my contact with employers. I realize now how helpful it was and how much better I am at job development. For both SE and IDDT, CATs devoted time to instruction in proper documentation. This included work on goal-setting and treatment planning and doing quality assessments. Part of the impetus concerning good practice in this regard was to ensure that the EBP service would meet Medicaid documentation standards and thereby be reimbursable. The CATs also used visits to ‘‘model’’ programs in other states. One SE site was convinced that the impoverished area they served prevented consumers from obtaining employment. Representatives from the site visited an SE program in Hartford, Connecticut which helped dissuade them from their perspective. For IDDT, representatives of all three sites visited an exemplary program in Ohio. Supervisors attended all of the practitioner training and a 2-day supervisor training. The supervisor training covered the following topics: (a) Managing by consumer outcomes (b) how to use data to help supervise (c) methods for enhancing fidelity (d) giving feedback (e) group supervision methods for team meetings (f) creating a reward—Based environment (g) transfer of training & learning to actual practice. The CATS did a lot of modeling and job coaching in running a team meeting, practicing skills with front-line staff and conducting effective case
123
Community Ment Health J (2008) 44:213–224
223
reviews. Each EBP had quarterly project leader meetings (Supervisor Forum) with the CAT to review programs, share innovations and problem-solve.
Discussion The analysis uncovered specific tools (e.g., fidelity measurement), mechanisms (e.g., leadership teams, focused team meetings) and methods (e.g., field mentoring, modeling, and group supervision) that contributed to successful implementation of the EBPs in Kansas. The findings provide further specificity in the behaviors necessary. For example, rather than ‘‘leadership,’’ expectations could include instituting and enforcing expectations, willingness to transfer staff, making EBPspecific structural changes, etc. The overall impression from the data was that it was the confluence of tools, methods, mechanisms that produced successful EBP implementation rather than any one factor. This analysis and a previous one on barriers (Rapp et al. in press) also underscores the critical role of quality supervision that seems to transcend specific methods or tools. Although the role of program leaders (front-line supervisors) is rarely studied in the implementation literature, the context and requirements of EBP implementation may be somewhat unique. Both IDDT and SE have clinical elements that direct service staff are to carry out with skill. This direct service staff are virtually all B.A. level with limited experience with people with psychiatric disabilities and who receive very little training prior to assuming their responsibilities. In this situation, the onus of implementation of clinical skills falls on the supervisor who teaches, guides, monitors, and enforces skill development and use. Within this context, group supervision and field mentoring become important mechanisms for discharging this responsibility. The positive influence of fidelity measurement had on implementation is consistent with the decades of research demonstrating that the collection and feedback of information affects human behavior and system change in desired directions (Alvero et al. 2001; Balcazar et al. 1985; Nadler 1977; Taylor et al. 1984). In this project, the fidelity reviews were the basis for agencies setting goals. As Locke et al. (1981) state: ‘‘The beneficial effect of goal setting on task performance is one of the most robust and replicable findings in the psychological literature’’ (p. 145). The findings suggest a framework for future EBP implementation research. The organizational schema would specify the people and behaviors necessary for implementation success. It would also include the role of ‘‘auxiliary’’ agency staff such as case managers, therapists, and psychiatrists, as well as staff directly responsible for implementation. Measurement would focus on the degree to which these people meet behavioral expectations. This study is exploratory and limited to one state’s implementation efforts with only two EBPs. It is likely that other states and other practices may suggest a different array of tools, mechanisms, and methods that contribute to success. Acknowledgments The authors would like to thank Professor Edward Canda from The University of Kansas School of Social Welfare and Greg McHugo from the Dartmouth College Psychiatric Research
123
224
Community Ment Health J (2008) 44:213–224
Center for their assistance with this paper. Developed by The University of Kansas School of Social Welfare Office of Mental Health Research and Training through a contract with the Kansas Department of Social and Rehabilitation Services.
References Alvero, A. M., Bucklin, B. R., & Austin, J. (2001). An objective review of the effectiveness and essential characteristics of performance feedback in organizational settings (1985–1998). Journal of Organizational Behavior Management, 21(1), 3–29. Atlas.ti (2002). Atlas (version 2.0) [computer software] Berlin, Germany: GmbH. Balcazar, F. E., Hopkins, B. L., & Suarez, Y. (1985). A critical objective review of performance feedback. Journal of Organizational Behavior Management, 7, 65–89. Bond, G. R., Evans, L., Salyers, M. P., Williams, J., & Kim, H. K. (2000). Measurement of fidelity in psychiatric rehabilitation. Mental Health Service Research, 2(2), 75–87. Lehman, A. F., Steinwachs, D. M., & co-investigators. (1998). Patterns of usual care for schizophrenia: Initial results from the schizophrenia patient outcomes research team (PORT) client survey. Schizophrenia Bulletin, 24, 11–20. Lincoln, S. L., & Guba, E. G. (1985). Naturalistic inquiry. Newbury Park: Sage Publications. Locke, E., Shaw, K., Saari, L., & Latham, G. (1981). Goal setting and task performance: 1969–1980. Psychological Bulletin, 90(1), 125–152. McHugo, G. J., Drake, R. E., Whitley, R., Bond, G. R., Campbell, K., Rapp, C. A., Goldman, H. H., Lutz, W., & Finnerty, M. (in press). Fidelity outcomes in the National Evidence-Based Practice Project. Psychiatric Services. Nadler, D. A. (1977). Feedback and organization development. Using data based methods. Reading, MA: Addison-Wesley. President’s New Freedom Commission on Mental Health. (2003). Achieving the promise: Transforming mental health care in America. DHHS Publication No. SMA-03-3832, Rockville, MD. Rapp, C. A., & Goscha, R. (2006). The strengths model: Case management with people with psychiatric disabilities (2nd ed.). New York: Oxford University Press. Rapp, C. A., Etzel-Wise, D., Marty, D., Coffman, M., Carlson, L., Asher, D., Callaghan, J., & Holter, M. (in press). Barriers to Evidence-Based Practice Implementation: Results of a Qualitative Study. Community Mental Health Journal. Strauss, S., & Corbin, J. (1990). Basis of qualitative research. Newbury Park, CA: Sage. Tashjian, M. D., Hayward, B. J., Stoddard, S., & Kraus, L. (1989). Best practice study of vocational rehabilitation services to severely mentally ill persons. Washington, DC: Washington Rehabilitation Services Administration, US Department of Education. Taylor, M. S. (1987). The effects of feedback on the behavior of organizational personnel. Administration in Social Work, 11(3/4) Fall/Winter, 191–203. Taylor, M. S., Fisher, C. D., & Ilgen, D. R. (1984). Individuals’ reaction to performance feedback in organizations: Control theory perspective. In K. M. Rowland, & G. R. Ferris (Eds.), Research in personnel and human resource management (Vol. 2, p. 810124). Greenwich, CT: JAI. Torrey, W. C., & Gorman, P. G. (2005). Closing the gap between what services are and what they could be. In R. E. Drake, M. R. Merrans, & D. W. Lynde (Eds.), Evidence-based mental health practice (pp. 167–188). New York: W.W. Norton. U.S. Department of Health, Human Services (DHHS). (1999). A report of the surgeon general. Washington, DC: Author, Wang, P. S., Demler, O., & Kessler, R. C. (2002). West, J., Olfson, M., Narrow, W. E., Rae, D., Wilk Marcus, S., & Reier, D. A. (2003). The treatment in routine psychiatric practice. Paper presented at the Academy of Health Annual Meeting, Nashville, TN.
123
