Examining Inside the Black Box of Implementation Facilitation: Process and Effects on Program Quality Louise E. Parker, PhD Mona J. Ritchie, MSW, PhD Candidate Laura Bonner, PhD JoAnn E. Kirchner, MD VA HSR&D QUERI-funded SDP 08-316 (PI: JoAnn Kirchner, MD)
Purpose • Based on the quantitative findings just described, facilitation appears to: Help organizations overcome organizational barriers to implementation Increase the adoption and improve the reach of programs
• So that we could better understand this effective method, we examined inside the black box of facilitation to described its process • In this section, we describe our major findings from that portion of the study
Research Questions • Implementation scientists propose that facilitators can both do things for sites and enable sites to act for themselves.*
Are unique sets of facilitation activities associated with each of these two purposes? • How does the implementation facilitation (IF) process change over time and in response to context? • What do participants value most about IF? • Are sites that had access to IF able to implement higher quality programs than those that did not? (*Dogherty et al. 2012, Stetler et al. 2006, Lombarts et al. 2005, Caldwell 2006)
Methods • Over a two and half year period, we followed: The national expert external facilitator (EF) Both internal regional facilitators (IRFs)
•
We conducted: Monthly debriefings with the facilitators Semi-structured interviews midway through, and at the end of, the intervention with the facilitators, clinicians, and managers We conducted a content analysis of all the above data
•
We asked experts to rate PC-MHI program quality at the eight intervention sites and eight matched comparisons
Results: Doing versus enabling? • Scholars have observed that facilitation activities fall into two broad categories: Doing activities for the benefit of others Enabling others to act
• Many but not all activities fell predominately into one category or the other
Results: Doing versus enabling? Both Regions: 100% Doing
Both Regions: 100% or Mostly Doing
Baseline data collection Ongoing data collection Administrative tasks
Task orientation Providing support Updates and Feedback Managing team processes Clinical education Engaging stakeholders Technical assistance non IT
Both Regions: 100% Enabling
Both Regions: 100% or Mostly Enabling
Developing shared vision/consensus building Adapt to local context Organizational change (cultural & unspecified)
Networking/peers Networking/experts Problem-solving Action/implementation plans Organizational change (structural) Problem identification
Results: Doing versus enabling? Context Dependent Activity
Region 1
Region 2
Overcoming Resistance
Mostly Doing
Mostly Enabling
Goal Setting
Mostly Enabling
Mostly Doing
Working with other Departments
Mostly Enabling
50/50
Strategy Policy Development
50/50
Mostly Enabling
Technical Support (IT)
50/50
Mostly Enabling
Marketing
50/50
Mostly Doing
Change Skills Education
Mostly Doing
None
Marketing Education
50/50
None
Hiring new PC-MHI providers
50/50
None
Results: Doing versus enabling? Summary of Findings • Although some activities tended to be either “doing” or “enabling” activities, many did not necessarily fit into one facilitation category or the other
For example: Both possible to set goals (i.e., do goal setting) and to help others to set their own goals (i.e., enable goal setting) Context affected the extent to which facilitators “did” or “enabled” an activity - Region 1 staff members were generally more open to the idea of PC-MHI than were Region 2 - Thus goal setting was more of an enabling activity in Region 1 than Region 2
Should also note that facilitators slowly backed away from active engagement into a more consultative and supportive role (enabling) over time, until sites no longer needed them when they disengaged entirely
Results: Do particular activities occur during particular implementation periods? • A few occurred predominantly during particular periods Occurred primarily at the beginning of implementation Baseline Data Collection Strategy/Policy Development Planning
Reoccurred if original plan did not fit or with staff changes
Marketing Overcoming Resistance
Concentrated at beginning but longer at some sites
Administrative Tasks
Highest at beginning but continues throughout
Occurred primarily at the middle of implementation Managing Team Process Networking with Peers
Both dropped off more rapidly in Region 2 than 1; this may be due to Region 1’s higher level of buy-in
Results: Do particular activities occur during particular implementation periods?
For most activities evolving context, rather than implementation phase, dictated the presence and intensity (e.g., low buy-in, type of staffing, patient needs, staff changes)
This included activities that would expect to occur primarily at beginning Activities expected at to be concentrated at beginning but timing varied by site Engaging Stakeholders
Marketing and Education
Task Orientation
Goal and Priority Setting
Developing Shared Vision
Adapting to Local Context
Problem Identification
No expected pattern Problem Solving
Organizational Structure and Cultural Change
Networking with Experts
Technical Assistance
Working with Other Departments
Results: What do facilitators and stakeholders value most about IF? • Across the board, the most valued activities were support, consultation, and management of people and organizations • Region 2 also placed relatively high value on facilitators involvement in organizational change and sustaining programs Possible explanation: Region 1 sites generally had greater leadership support for their programs than did Region 2 sites
• Interestingly, although staff members from both regions placed high value on promoting buy-in and technical assistance, neither IRF did
Results: Does facilitation improve the quality of programs implemented? • Midway through the study: Seven intervention but only three comparison sites had implemented a program Experts rated the quality of the intervention site programs most highly
Results: Does facilitation improve the quality of programs implemented? Time One Expert Rating Site
Overall Quality
Evidence
Sustainability
Level of Improvement
Intervention - V1
3.33 4.33 4.67 4.00
3.33 4.67 5.00 4.00
3.33 4.00 5.00 3.67
3.33 4.33 4.33 4.00
Intervention - V2
4.00 1.33 6.33 No Program
3.33 1.33 6.33 No Program
3.67 2.00 6.00 No Program
4.44 2.00 6.33 No Program
Comparison - V3
No Data 2.67 2.33 1.33
No Data 2.67 2.33 1.33
No Data 5.00 2.00 2.00
No Data 3.67 2.33 2.00
Comparison - V4
No data No Program No Program No Program
No data No Program No Program No Program
No data No Program No Program No Program
No data No Program No Program No Program
Results: Does facilitation improve the quality of programs implemented? • At the end of the study, all intervention but only five comparison sites had programs. All but one of the intervention sites had a higher rated program than its comparison.
Results: Does facilitation improve the quality of programs implemented? Time Two Expert Rating Site
Overall Quality
Evidence
Sustainability
Level of Improvement
Intervention - V1
5.00 5.33 5.33 5.00
4.33 5.33 5.67 5.00
5.33 5.00 5.33 5.00
5.00 5.33 5.67 6.00
Intervention - V2
1.67 3.00 6.00 3.67
1.67 3.00 5.67 3.33
2.00 3.00 4.67 4.67
2.00 2.67 6.33 3.67
Comparison - V3
2.00 3.33 3.67 4.33
2.33 3.33 3.33 4.00
3.33 3.33 5.00 4.33
3.33 4.00 3.67 4.33
Comparison - V4
4.33 No Program No Program No Program
4.33 No Program No Program No Program
4.33 No Program No Program No Program
4.00 No Program No Program No Program
Discussion • IF can foster adoption and implementation of high quality and evidenced based new practices. • Facilitation activities did not occur according to a defined series of stages as many implementation scientists would have predicted.* Rather, as a number of organizational change scholars theorized, change occurred in response to local group interests and in the context of organizational politics and power through non-linear, multi-level, and incremental processes.** *Dogherty et al. 2012, Hayden et al. 2001 **Caldwell 2006
Discussion • We also observed regional differences in the level and timing of activities. We cannot determine if these differences were due to IRF’s skills and characteristics, needs of their regions, or a combination of the two. Implementation experts assume, however, that certain characteristics and skills foster an individual’s ability to be a successful facilitator.
•
The question then arises: To what extent are facilitators “born” versus “made” or in other words:
Is it possible that coaching and mentoring can enable would be facilitators to learn the requisite skills to foster organizational and practice change?
•
The next section of this panel addresses methods for transferring Implementation Knowledge and Skills
For More Information Louise E. Parker, PhD Professor of Practice, Healthcare Management and Policy College of Management, University of Massachusetts, Boston and Affiliated Researcher, VA Mental Health QUERI
Email:
[email protected]
