Understanding Staff Perspectives on Collaborative Quality Improvement in the ICU: A Qualitative Exploration
by
Katie Naismith Dainty-Chuk
A thesis submitted in conformity with the requirements for the degree of Doctor of Philosophy Institute of Medical Science University of Toronto
© Copyright by Katie Naismith Dainty-Chuk 2011
Understanding Staff Perspectives on Collaborative Quality Improvement in the ICU: A Qualitative Exploration Katie N. Dainty-Chuk Doctor of Philosophy Institute of Medical Science University of Toronto 2011
Abstract Despite the ongoing initiatives of quality improvement collaboratives in healthcare which reflect various multifaceted intervention packages, clear evidence of the effectiveness of the model itself is lacking. Little is known about the true impact of the collaborative approach on improvement outcomes or how specific components are actually implemented within participating organizations.
This dissertation reports on empirical qualitative research undertaken to investigate “how” healthcare providers and management describe the experience of being involved in a collaborative network for quality improvement. Using a process evaluation of a case study collaborative, this research reveals that frontline staff do not feel the need to conform or be identical to their peer organizations; rather they feel that by participating with them that their high level of care is finally recognized. In addition, the existing communication structure is ineffective for staff engagement and a “QI bubble” seems to exist in terms of knowledge transfer and the idea of collaboration bears out more internally in increased intra-team cooperation than externally between organizations or units. Selected theoretical perspectives from the fields of sociology and organizational behaviour are used as an analytic framework from which the author posits that based on the findings from this case study that in fact collaboratives may not actually ii
function by any of the commonly held assumptions of legitimization, communication and collaboration. A conceptual framework for how these constructs are related in terms of QI collaborative design is proposed for future testing.
With further work and on-the-ground testing of this model and relational hypotheses, this research can help the QI community develop a more functional theory of collaborative improvement and use mixed methods evaluation to better understand complex QI implementation.
iii
Acknowledgments It is said we are not truly alive until we are made aware of our blessings. - Thornton Wilder
The process of completing my PhD has made me very aware of my blessings. There are not enough pages in any thesis for me to properly acknowledge and thank all of the people who have helped and supported me through the last five years, however there are a few very special people I would like to mention. Dr. William Sibbald, without who’s “gentle” encouragement I wouldn’t be here today and who sits on my shoulder to this day guiding me in everything I do. Dr. Merrick Zwarenstein, who through his quiet leadership, undying support & passion for health services research has shown me what it means to be a truly great scientist. Drs. Laurie Morrison, Jan Barnsley and Brian Golden – my Progress Advisory Committee – their invaluable critiques, constant support, academic guidance and most importantly time, has been instrumental in getting this dissertation to where it is today. Thank you so much to each of you. Michael Chuk, my husband and truly my life partner, whose unwavering support and encouragement on my worst days made it possible for me to keep seeing the light at the end of the tunnel. Thank you Honey, I love you. My parents, Bill & Sharon Dainty, and all of my extended family and family-in-law, who still have no idea what I do for a living but whose love and support throughout my life have made me who I am today. Dr. Damon Scales (Primary Investigator) and the study team of the Ontario ICU Best Practices Collaborative (Randomized Control Trial), thank you for allowing me to study your quality improvement collaborative and be involved in such an incredible provincial initiative. All of the healthcare providers at the various hospitals across Ontario and the Scientists at the University of Toronto who I have had the pleasure of interacting with during my research – I am continuously amazed and inspired by your dedication to improving the health care system for all Canadians and I hope that I have the pleasure of collaborating with all of you for many years to come.
iv
TABLE OF CONTENTS Abstract…………………………………………………………………………………………...ii Acknowledgments.......................................................................................................................... iv List of Tables ................................................................................................................................ vii List of Figures .............................................................................................................................. viii List of Appendices ......................................................................................................................... ix Chapter 1 – Introduction and Thesis Outline ............................................................................ 1 1.1 Introduction............................................................................................................................... 1 1.2 Why study Quality Improvement Collaboratives – they must work, everbody is doing them..3 1.3 Intensive Care Quality Improvement………………………………………………………….5 1.4 Thesis Outline ........................................................................................................................... 7 1.6 Summary ................................................................................................................................. 11 Chapter 2 – Background and Rationale ................................................................................... 13 2.1 The Quality Improvement Movement – before Healthcare.................................................... 13 2.2 Quality Improvement meets Healthcare ................................................................................. 14 2.3 Alliances, Networks, Collaboratives and the concept of Collaboration ................................. 17 2.6 The Collaborative Approach to Quality Improvement in Healthcare..................................... 19 2.7 The Existing Model for a QI Collaborative Approach ........................................................... 21 2.8 Evaluation of Collaborative QI Interventions in Health Care ................................................ 24 2.9 Summary ................................................................................................................................. 29 Chapter 3 – Building an Analytic Framework......................................................................... 31 3.1 Introduction............................................................................................................................. 31 3.2 Defining the term “QI Collaborative”..................................................................................... 33 3.3 Building an Analytic Framework from Theory in other Domains ......................................... 36 3.3.1 Why Healthcare Staff think it is important for units to belong to QI Collaboratives? .. 37 3.3.1a Institutional Theory............................................................................................ 38 3.3.2 How do staff engagement & communication occur within a QI collaborative?............ 40 3.3.2a Theory of Interorganizational Relationships ..................................................... 41 3.3.2b Social Movements Theory .................................................................................. 42 3.3.3 Does collaboration actually occur within a QI collaborative program? ........................ 43 3.3.3a Theory of Collaborative Advantage................................................................... 44 3.4 Discussion ............................................................................................................................... 47 Chapter 4 - Research Context: Description Of The Sample Collaborative .......................... 48 4.1 Introduction............................................................................................................................. 48 4.2 The Ontario ICU Best Practices Collaborative....................................................................... 49 4.2.1 Intended Study Purpose and Design .............................................................................. 50 4.2.2 Participants..................................................................................................................... 50 4.2.3 Baseline Assessment...................................................................................................... 50 4.2.4 Selection of Clinical Best Practices ............................................................................... 51 4.2.5 Knowledge Translation & Behaviour Change Strategies .............................................. 52 4.2.6 Data Collection .............................................................................................................. 53 4.2.7 Outcome Measures......................................................................................................... 54 v
4.2.8 Data Analysis ................................................................................................................ 56 4.4 Summary ................................................................................................................................. 57 Chapter 5 – Research Methods ................................................................................................. 59 5.1 Introduction............................................................................................................................. 59 5.1.1 Applying a Process Evaluation Framework to a Case Study Collaborative.................. 61 5.2 Study Design........................................................................................................................... 62 5.2.1 Research Questions........................................................................................................ 63 5.2.2 Participant Sampling...................................................................................................... 63 5.3 Analytic Methods.................................................................................................................... 66 5.4 Summary ................................................................................................................................. 68 Chapter 6 –Empirical Qualitative Findings ............................................................................ 69 6.1 Introduction............................................................................................................................. 69 6.2 Findings................................................................................................................................... 70 6.2.1 Why healthcare staff think its important for units to be part of QI Collaboratives ....... 70 6.2.2 Staff Engagement........................................................................................................... 72 6.2.3 Communication during the Project between Management and Frontline Staff............. 74 6.2.4 Inter-organizational Communication ............................................................................. 77 6.2.5 Intra-unit Communication.............................................................................................. 78 6.2.6 Competition in Collaborative Clothing.......................................................................... 79 6.2.7 Multiple QI Projects and Resource Scarcity.................................................................. 80 6.3 Discussion ............................................................................................................................... 82 Chapter 7 - Discussion ................................................................................................................ 84 7.1 Summary of Key Exploratory Themes ................................................................................... 84 7.2 Advancing the Theoretical Discussion ................................................................................... 86 7.2.1 Why Healthcare Staff feel it is Important for Units to belong to QI Collaboratives ..... 87 7.2.2 Staff Engagement and Communication within the Collaborative ................................. 89 7.2.3 Collaboration in the Collaborative................................................................................. 92 7.3 An Emerging Conceptual Framework .................................................................................... 94 7.4 Summary ................................................................................................................................. 97 Chapter 8 - Conclusions, Implications & Future Directions .................................................. 98 8.1 Review .................................................................................................................................... 98 8.2 Implications of this Research……………………………………………………………….100 8.3 Limitations………………………………………………………………………………….101 8.4 Future Research Directions…………………………………………………………………102 8.5 Concluding Thoughts……………………………………………………………………….104 References……………………………………………………………………………………...107 Copyright Permissions……….………………………………………………………………..119
vi
List of Tables Table 1 – Published Controlled Study Evaluations of QI Collaboratives…………………p.27 Table 2 – Best Practice Site Selection Statistics…………………………………………..p.52 Table 3 – Best Practices and Associated Process of Care Indicators……………………...p.55
vii
List of Figures
Figure 1 – Napkin drawing of IHI Breakthrough Series concept…………………………p.22 Figure 2 – A proposed conceptual framework ……………………………………………p.96
viii
List of Appendices
Appendix A – Copy of the Semi-Structured Interview Guide Appendix B – Open Coding List Appendix C - Summary of the ICU Collaborative Trial Results
ix
- CHAPTER 1 INTRODUCTION & THESIS OUTLINE "Just because everything is different doesn't mean anything has changed." - Irene Peter, Author
Chapter Overview The purpose of this chapter is to introduce the reader to the topic of this thesis and provide the background for the approach taken to its development. Here I start the discussion of topics such as quality improvement (QI) in healthcare, the collaborative approach to quality improvement and the choice of QI in intensive care medicine as a case study for my research. Following this brief orientation, I explain the origin of my interest and perspective in the area of quality improvement research and provide a general outline of how this thesis is put together.
1.1 Introduction The term quality improvement (QI) collaborative is typically used to describe a program which provides different multifaceted packages that in general focus on accelerating better outcomes to a group of health professionals, units or organizations (Schouten et al., 2008). QI collaboratives are used in different clinical areas and organizational contexts and have been adopted by large and small, acute and long-term healthcare systems. These initiatives represent substantial investments of time, effort, and funding, although estimates of the total investment and applications of most collaboratives are not available. The theoretical cornerstones of the collaborative approach for QI are the involvement of frontline staff, efficient use of experts, peer communication, empowerment of cross-disciplinary teams and the exchange of best practices to facilitate and guide improvement (Berwick, 1989; Blumenthal & Kilo, 1998).
The quality improvement movement has increasingly become a priority in healthcare since the early 1960’s when leaders in the field began to investigate the application of quality tools & methods from industrial areas to healthcare organizations and systems (Kenney, 2008). More recently, the US Institute of Medicine (IOM) report “To err is Human” (Kohn, Corrigan, & Donaldson, M.S. (Institute of Medicine), 2000) identified a serious deficiency in the quality of the health care being provided in the United States. Once this report became mainstream the issue of quality and quality improvement in health care have received enormous attention on several international fronts (Altman, Clancy, & Blendon, 2004; Leape & Berwick, 2005).
Several methods or approaches to quality improvement have been developed over the years and include tools such as the Plan-Do-Check-Act (PDCA) cycle, process mapping, Total Quality Management (TQM), Continuous Quality Improvement (CQI), Six Sigma, Lean Methodology and several others. All of these represent methods and techniques for identifying, measuring, prioritizing and improving processes, all critical components of a quality improvement program.
One of the most popular quality improvement approaches used in healthcare is the multiorganizational collaborative or network approach. Teams participating in such groups are typically located in healthcare facilities in different geographic areas and may even work for different units within the same organization. The collaborative engages the teams in implementing a set of identified best practices in their respective settings. The collaborative also aim to facilitate active sharing of strategies and ideas for improvement among participating teams, so that teams learn from each other and can quickly benefit from successful changes implemented by other teams. The main selling feature of the collaborative approach is predicated on the idea of learning from others and not re-inventing the wheel; that sharing of 2
information between institutions somehow increases the achievability of improvements through a virtual group effort (Kilo 1998; Wilson, Berwick, and Cleary 2003; Ayers 2005).
Despite a plethora of research in the area of how to do quality improvement in terms of effecting quantifiable improvement in patient outcomes, relatively little has been produced on understanding the process of collaborative initiatives. The field has yet to develop comprehensive evidence that any particular factor related to collaborative efforts can be responsible for increased QI success. Such unexamined factors include: fundamental evidence of the extent to which healthcare staff are active in externally driven collaborative improvement initiatives; the role of communication patterns within the collaborative framework or how knowledge transfer strategies are used in such collaboratives. Fore example, we do not know exactly what is exchanged among teams, whether it is simply social support or actual information, innovation or detailed advice on care improvement methods. More broadly, the question becomes what benefits are gained by conducting QI efforts via collaborative arrangements over single organization or department level QI efforts, which might be less expensive, more relevant, and a less time-consuming method of knowledge transfer.
1.2 Why study Quality Improvement Collaboratives – they must work, everybody is doing it? Writing of organizational behaviour, Johnson and Duberley (Johnson & Duberley, 2000) point out that certain ways of thinking become so embedded in our language and culture that it can seem simply a matter of common sense and, as such, natural and taken for granted. I will argue that this is one of the main reasons there has been a lack of development of a formal theoretical framework for and widespread uptake of the fundamentally unevaluated collaborative approach
3
to quality improvement. To date, the collaborative QI model has been applied in a non-reflexive fashion and in a time when successful quality improvement is so important in healthcare. It seems irresponsible not make a stronger attempt to understand and improve such an accepted and resource intensive approach.
My personal experience in implementing critical care quality improvement initiatives has led me to be intensely curious about the real world operations of QI at the front lines of healthcare, and specifically how the processes of a collaborative relate to its intended results. Current published evidence for the effectiveness of the collaborative approach consists mainly of descriptions and commentary pieces from proponents of the model (Greenhalgh, Bate, & Kyriakidou, 2007)and reporting of quantitative quality indicator data (Fennell & Warnecke, 1988; Grilli & Lomas, 1994; Meyer & Goes, 1988). The tendency to default to quantitative measurement can most likely be
attributed to the fact that it is difficult for medicine and healthcare to break away from what it knows best: a quantitative data-driven, show me the randomized trial, ignore the impact of social complexity, model of evaluation – a very positivist paradigm. It seems appealing to think of health care organizations like machines, like the human body, receiving inputs, transforming them, and producing outputs. This machine metaphor leads to stereotypical beliefs on how the “system” can be studied and how the “system” can be improved. Examine the parts separately, look at the mechanics and if the system is not working as planned, then identify the broken part and replace it. This mechanistic approach along with a historical reliance on quantitative evaluation ignores the experience of the 'participant' in the constitution of the reality of how these programs actually operate. This is not to say that there is not a need for quantitative measurement of improvement indicators – however, we need to begin to place equivalent value on complimentary methods for understanding complex social interventions like quality
4
improvement in healthcare to be able to further increase our impact.
Through my doctoral training and exposure to various paradigms and perspectives, I have developed an active interest in studying the model of quality improvement collaboratives using a more critical theoretical approach. My understanding of how to use social science oriented qualitative methodologies to answer such “how” type research questions has increased exponentially and brought this thesis to where it is today. I hope more than anything that the research included in
this thesis can show the value of these types of methodologies and be part of a move forward for quality improvement and implementation science in healthcare.
1.3 Intensive Care Quality Improvement One of the areas where the collaborative approach to quality improvement has been applied in several instances is the speciality field of critical care medicine. Critical care is “a multidisciplinary field concerned with patients who have sustained, or are at risk of sustaining, life-threatening single or multiple organ systems failure due to disease or injury” (The Royal College of Physicians and Surgeons of Canada, 1996). Also known as intensive care, it includes a broad spectrum of patients, from chronically ill elderly patients who develop acute illnesses, to cancer patients sustaining life-threatening complications of potentially curative surgery and chemotherapy, to young victims of trauma and patients who develop complex clinical syndromes such as sepsis. Due to the intensive care required for critically ill patients, staff must be specially trained in critical care medicine and this level of care is generally confined to a specific geographic area in the hospital known as the intensive care or critical care unit (ICU). For the purposes of this thesis, “ICU” will describe these types of units or hospital area where patients
5
typically receive aggressive and invasive life-saving treatments and technologies, such as mechanical ventilation and pulmonary artery catheters.
ICU patients account for approximately 10% of all inpatient acute care beds in the United States (Groeger et al., 1992; Needham et al., 2005)and roughly the same numbers in Canada. Intensive care resources and beds are in scarce supply in Canada (Bell & Robinson, 2005)and the system demands are growing exponentially due to an aging population (Halpern, Pastores, & Greenstein, 2004; Halpern, Bettes, & Greenstein, 1994) and the introduction of new lifesaving technologies. It is also expensive care; the ICU direct costs per day for survivors have been estimated at between six and seven times those for non-ICU care (Norris, Jacobs, Rapoport, & Hamilton, 1995)
Health care resources, especially those in high intensity environments such as the ICU are critically taxed and although the need to implement solutions that improve efficiency and quality, while simultaneously improving access has never been more important, wasting these resources on inefficient QI approaches is unacceptable. By using this particular field as a case study for this research, I hope to be able to provide a better understanding of how the collaborative QI process functions so that we can shed some light on possible redesigns that would increase its effectiveness in such a resource depleted system. Using a single example case for this research is an appropriate method as the research question is exploratory, contemporary and significant events or variable cannot be manipulated experimentally (Yin, 1989). In addition this approach is appropriate given that the research is meant to give a descriptive rather than analytical view of the collaborative approach to QI.
6
1.4 Thesis Outline This dissertation reports on empirical qualitative research undertaken to better understand how QI collaboratives function at the frontlines of healthcare. The overarching research question under investigation is: how do healthcare providers and management describe the experience of being involved in a collaborative network for quality improvement? Integral to approaching this overarching question are more specific secondary questions: (a) why do healthcare managers and frontline staff think it is important to belong to a QI collaborative; (b) how does engagement and communication occur within the collaborative program and (c) does collaboration actually take place within the collaborative.
This thesis extends research and theorization on non-profit collaboration in the healthcare sector in three social process-oriented dimensions. First, it is among the first attempts to develop a systemic understanding of why healthcare organizations or units decide to join collaborations for the purposes of quality improvement. Although an emerging body of literature has focused on collaborations among non-profit organizations, most efforts to date have either discussed collaboration in general terms or considered a specific form of collaboration (e.g., networks or strategic alliances) in an isolated manner. By analyzing motivations and decision-making processes within healthcare organizations, this research provides interesting insight into the spectrum of motivations and contextual circumstances under which these organizations and units believe the collaborative approach to be beneficial. Second, in light of the unique features of the interorganizational relationships within QI in the healthcare sector, I use the data in my interviews with key frontline stakeholders to begin to build on the theoretical thinking in this area. The information gathered also helped me gain an understanding of the nature of collaboration which may or may not occur and therefore begin to advance or criticize the theory 7
of collaborative advantage as it pertains to healthcare QI. Each of these concepts could be a large empirical investigation of its own and the potential mediation of each on QI collaborative effectiveness will require extensive empirical testing in several contexts. However, given the paucity of research or contextual theory advancement in this area to date, it is my hope that I can draw some interesting conclusions through the empirical work of this thesis as a starting point for further scientific work in this area.
I have chosen to use a process evaluation methodology in order to frame my investigational research on a large provincial quality improvement collaborative aimed at ICU care in Ontario. This methodology is typically used in a more quantitative fashion in industrial quality improvement and documents and analyzes the early development and actual implementation of a strategy or program, assessing whether strategies were implemented as planned and whether expected output was actually produced. More than 30 years ago, Avedis Donabedian proposed a model for assessing health care quality based on structures, processes and outcomes (Donabedian, 1966). He defined structure as the environment in which health care is provided, process as the method by which health care is provided, and outcome as the consequence of the health care provided. As a result, process management is limited, and often temporary, when the structure isn't also improved. The methodology has recently been described specifically for QI evaluation by Hulsher et al and they assert it “sheds light on the mechanisms actually responsible for the result obtained in the intervention group” and allows evaluators to include the experience of those exposed to the intervention (Hulscher, Laurant, & Grol, 2003). Applying this framework to my dissertation and using the collaborative approach as the intervention, I present 1) an outline of the detailed design and evaluation of the Ontario ICU Best Practices Project; 2) a summary of the quantitative process outcomes measured by the project and 3) an empirical 8
investigation of the attitudes of frontline healthcare workers towards involvement in this quality improvement collaborative and the activities undertaken with other organizations. In this thesis I will draw on sociologic and organizational behaviour theory to argue that quality improvement collaboratives may not work the way we currently think they do in terms of why staff think they are important, communication and staff engagement and the very concept of collaboration.
The foundation of this thesis begins in Chapter 2 with a comprehensive literature review of the history of quality improvement in healthcare and the “collaborative” approach as well as an overview of the evaluation of quality improvement to date. This chapter largely provides a background synopsis of the influential topics that relate to research on QI collaboratives and an adequate foundation for those readers who may not be familiar with the current landscape of quality improvement in healthcare.
In Chapter 3, I review a selection of related theoretical perspectives from other fields, which may be useful in supporting the model for a collaborative approach to QI in healthcare. I will use the core tenets from these common theories in Chapters 5 and 6 as a starting point into the analysis of my data from the case study collaborative and examine whether it supports or negates the existing assumptions.
In Chapter 4, I focus attention on the description of the setting and context of my research. This chapter provides an in-depth account of the design of a two-year provincial QI collaborative known as the Ontario Ministry of Health and Long Term Care (MOHLTC) ICU Best Practices Project. I have selected to highlight this particular collaborative due to its deliberate multifaceted design and inclusion of a voluntary group of heterogeneous ICUs. Also, I have been 9
intimately involved in the design and implementation of this collaborative as a co-investigator and it is through this experience that I came to realize that the conceptual model of this intervention may have been flawed in its assumptions. This led to my interested in the exploring the relationship of such assumptions to the improvement results (or lack thereof). In order to begin such an exploration I will outline the intended intervention and the outcomes that were measured as part of the original study.
Chapter 5 provides the detailed qualitative methodology and analysis that I used for my independent doctoral research. This research is based on a grounded theory analysis of a large interview data set from key informant nurses, physicians, respiratory therapists and unit management working in the sixteen participating ICUs. Chapter 6 follows with a summary of the emerging themes and findings of the study including supporting sample quotes from the data.
Chapter 7 consists of a review and discussion of how the research findings relate to the proposed theoretical assumptions from Chapter 3 and inform the development of a modified conceptual framework for multi-organizational quality improvement. And finally Chapter 8 provides a summary of the research and situates it within the current context of knowledge in the area of quality improvement. I will discuss how the findings can contribute improved insight into antecedents of and strategies for fostering more effective use of multi-organizational QI efforts and propose new directions for the design and development of such efforts. I will identify limitations and implications of my research which in turn present several future research directions and provide some final thoughts and conclusions on my work.
10
1.6 Summary Despite the ongoing initiatives of the quality improvement collaboratives reflecting different multifaceted intervention packages, clear evidence of the effectiveness of the model is lacking. There are a growing number of published papers, apparent face validity of the model, and facilitators who claim that healthcare professionals appreciate taking part in a collaborative for both professional and organizational development. However, little is known about the true impact of the collaborative approach on improvement outcomes or which specific components that enhance the effectiveness of such collaboratives. Furthermore, there is no empirical information about the perceptions of frontline healthcare workers of participating in QI collaboratives and its various components. More broadly, the question becomes what benefits are gained by conducting QI efforts via collaborative arrangements over single organization or department level QI efforts, which might be less expensive, more relevant, and a less timeconsuming method of knowledge transfer.
This dissertation reports on empirical qualitative research undertaken to investigate “how” healthcare providers and management describe the experience of being involved in a collaborative network for quality improvement including why they join, how they communicate and how they collaborate. I have applied a process evaluation (PE) methodology to present my research on a large provincial quality improvement collaborative aimed at ICU care in Ontario. Using the PE framework, I present 1) an outline of the detailed design and evaluation of the Ontario ICU Best Practices Project; 2) a summary of the quantitative outcomes of the project and 3) a report of the findings of my in-depth empirical exploration of the experience of frontline healthcare workers towards involvement in this quality improvement collaborative and the activities that occurred. 11
The findings of this research will hopefully contribute improved insight into components of and strategies for fostering more effective use of collaborative efforts at the frontlines of health care and provide information to support new perspectives on the design and development of such efforts.
12
- CHAPTER 2 -
BACKGROUND & RATIONALE “…we’re really good at collaborating…we’ve been cuddling together for 10 years, but we’ve made little actual progress” - A director, interprofessional research collaboration from Managing to Collaborate by Huxam & Vangen
Chapter Overview The purpose of this chapter is to provide an overview of the current knowledge and scholarship in the area of quality improvement, quality improvement in healthcare and the collaborative model, and current approaches to evaluation of QI efforts. This background, while comprehensive will also highlight where gaps in current knowledge exist and provide a rationale for the research to follow.
2.1 The Quality Improvement Movement – before Healthcare The roots of total quality management can be traced to the early 1920’s production quality control ideas. However the most common work we know today mainly stems from notable concepts developed in Japan in the late 1940’s and 1950’s and pioneered by American gurus such as W. Edwards Deming, Joseph Juran and others in the 1980’s.
W. Edwards Deming was an American statistician, and is considered the father of modern quality movement (Gabor 1992) Many international manufacturers consider him a guru of quality and his ideas revolutionized Japan's auto industry in the 1950s, but did not make a substantial impact in the United States until the 1980’s (The W. Edwards Deming Institute, ).
13
Increasingly today, service organizations, from hotels to public utility companies to healthcare, are experimenting with his principles and statistical methods for analyzing quality.
Joseph Juran is widely credited for adding the human dimension to quality management. He pushed for the education and training of managers and expanding the Pareto principle (80% of a problem is caused by 20% of the causes) by applying it to quality issues. In his terms, “cultural resistance” was the root cause of quality issues His late 1960’s work in Japan also brought the concept of “quality circles” to the United States (Society of Quality Control Engineers, 1967). A quality circle is a volunteer group composed of workers, usually under the leadership of their supervisor, who are trained to identify, analyze and solve work-related problems and present their solutions to management in order to improve the performance of the organization, and motivate and enrich the work of employees.
The work of these men and others produced classic tools such as the Plan-Do-Check-Act cycle (Deming, 1986), Pareto analysis, cause and effect diagrams, run charts, scatter plots, process mapping, Total Quality Management (TQM), Continuous Quality Improvement (CQI), Six Sigma, Lean Methodology and several others. All of these represent tools and techniques for identifying, measuring, prioritizing and improving processes, and are all critical components of a quality improvement paradigm.
2.2 Quality Improvement meets Healthcare The classic work of the major figures mentioned above directly influenced the application of the modern quality improvement movement to healthcare. In the early 1980’s several independent
14
physician leaders in the United States simultaneously discovered the concept of “quality improvement” from the work of experts like Deming and Juran in other domains (Kenney, 2008). Drs. Donald Berwick, Paul Batalden and Lucian Leape are just three of the instrumental leaders credited with introducing industrial improvement tools and methods to healthcare in the US over the last two decades (Berwick & Leape, 1999; Batalden & Stoltz, 1993; Brennan & Leape,1991).
Quality improvement in the healthcare arena is now a broad term that encompasses everything from reducing wait times in an emergency department to preventing central-line infections in an intensive care unit to improving inter-professional communication across an organization, all of which are extremely complex undertakings. Since the 1960’s healthcare resources has become increasingly limited and there was and is an ever-growing need to implement solutions that would increase patient safety and quality of care, while simultaneously improving access to services and improving efficiency. Quality improvement as a management method seeks to develop a culture within an organization so that, in an orderly and planned fashion, "everyone at all levels can play an active role in understanding problems and the processes of work that underlie them, collecting and analyzing data on those processes, generating and testing hypotheses about the causes of flaws, and designing, implementing, and testing remedies" (Berwick, p. 47). From 1987-1991, Don Berwick came to the forefront as co-founder and CoPrincipal Investigator for the National Demonstration Project on Quality Improvement in Health Care, designed to explore opportunities for quality improvement in health care. Since then Berwick has been best known for his work in the management of health care systems, with emphasis on using scientific methods and evidence-based medicine and comparative effectiveness research to improve the tradeoff among quality, safety and costs (Adams, 2009) 15
Despite three decades of work in this area, in 1999 and 2001 the Institute of Medicine in the Unites States, one of the three bodies that make up the U.S. National Academy of Sciences, published two landmark reports entitled “To Err is Human” (Kohn et al., 2000) and “Crossing the Quality Chasm” (Institute of Medicine Committee on Quality and Health Care in America, 2001) which brought critical attention to serious deficiencies that continued to exist in the American healthcare system in the area of safety and quality. “To Err is Human” was primarily a patient safety report which highlighted the gross medical errors that occur in healthcare and “reframed medical error as a chronic threat to public health, as lethal as breast cancer, motor vehicle accidents, or AIDS” (Berwick, 2003). “Crossing the Quality Chasm” focused more on quality of care, calling for improvements in six dimensions of health care performance: safety, effectiveness, patient-centeredness, timeliness, efficiency, and equity; and provided a rationale and a framework for the redesign of the U.S. health care system. These reports were preceded by the “Quality in Australian Healthcare study” in 1995 (Wilson et al., 1995) and in 2004, Canada responded with an equally damning study of adverse events in Canadian hospitals (Baker et al., 2004). Many consider all of these reports, which resonated with health systems the world over, to be a major launching pad for a new and persistent focus on improving the quality of healthcare.
Since that time, immense energy, resources and funding have been devoted globally to improving quality and patient safety in healthcare. Healthcare quality improvement is now virtually its own field of research in both the health sciences and management disciplines. Scholars in several areas have created a vast amount of knowledge around “what works and what doesn’t work”, both through peer-reviewed publications and to an even greater extent in the “grey” literature (reports, expert opinion, etc). We know from several systematic reviews of the literature that active approaches to behaviour change are more likely to be effective. 16
Interventions such as audit and feedback, and use of local opinion leaders have variable effectiveness, educational outreach and reminders are considered generally effective and that multifaceted interventions based on assessment of potential barriers to change are more likely to be effective than single interventions (Grimshaw et al., 2006). Cross-sector methodologies such as PDSA cycles, Lean methodology, root cause analysis and others have been shown to be very effective in helping organizations approach QI from a systems perspective (Shojania, McDonald, Wachter, & et al., 2004). The very popular quality improvement collaborative or network approach, which is based on collaborative learning and exchange of ideas, is an example of a QI method that has been accepted and employed across the globe with great enthusiasm. From here I would like to explore this particular QI methodology in greater detail.
2.3 Alliances, Networks, Collaboratives and the concept of Collaboration In recent decades, there has been unprecedented growth in the private and industrial sectors of corporate partnering and reliance on external collaboration (Gulati, 1995). These various types of interorganizational alliances take on many forms, ranging from partnerships to joint ventures to collaborative manufacturing to complex co-marketing arrangements. The most common rationales offered for this upsurge in collaboration involve some combination of risk sharing, obtaining access to new markets and technologies, speeding products to market, and pooling complementary skills (Powell, Koput, & Smith-Doerr, 1996). Empirical evidence shows that the number of inter-organizational alliances prior to 1980 was small (Gulati & Gagiulo, 1999) but it appears that strategic alliances in all sectors have become necessary in increasingly uncertain market places.
17
With the increase in the use of networks and collaboration has come an increase in the research into these relationships within the business and management literature. However, the concept and outcomes of collaboration remains not well understood (Alter & Hage, 1993; Morrison, 1996; Morrison, 1996; O'Looney, 1994; Reilly, 2001). The promotion of collaboration may have roots in its value as a symbol of rationality, efficiency, legitimacy, and social responsibility (Morrison, 1996; Reitan, 1998). Walter and Petr (Walter & Petr, 2000) observed that collaboration is commonly understood as "working together" (p. 5). Winer and Ray (Winer & Ray, 2000) maintain that the terms cooperation, coordination, and collaboration are often used interchangeably and have offered distinctions among these concepts. However, attempting to standardize the term ‘collaboration’ is difficult as there does not appear to be a unified understanding of the concept (Alter & Hage, 1993; Reilly, 2001).
While the increased in interorganizational relationships (IORs) has led to numerous studies and theoretical advancement, the majority of research has focused on their formation, the first phase in IOR development (Child & Faulkner, 1998; Dyer, 2000; Eden & Huxham, 2001; Gray, 1989). Some research has also addressed IOR management (Frisby, Thibault, & Kikulis, 2004; Huxham & Vangen, 2000a; Huxham & Vangen, 2000b; Hodge & Greve, 2005) and has shown that inadequate structures and processes can plague ongoing partner relations. Despite the increased attention on partnership formation and management, the outcomes of partnership relationships have rarely been empirically examined, in part because of the substantial difficulties associated with identifying and assessing the multiple and diverse interests involved (Callahan & Kloby, 2007; Human & Provan, 2000). In their framework to evaluate IORs, Provan and Milward (Provan & Milward, 2001) noted the challenges of assessing the effectiveness of a group of collaborating organizations ‘‘because key stakeholders and their interests are so diverse’’ (p. 18
422) however the authors argued that evaluating the effectiveness of IORs ‘‘is critical from the perspectives of those organizations that make up the network, those who are served by the network, and those whose policy and funding actions affect the network’’ (p. 422). They concluded that ‘‘what has been lacking in most of this work . . . is an examination of the relationship between interorganizational network structures and activities and measures of effectiveness’’ (p. 414). The ongoing challenge with applying the concept of IORs to other sectors is that empirical research on IOR effectiveness has received very little attention in the academic literature and so the factors of success or benefit have yet to be quantified or described.
2.6 The Collaborative Approach to Quality Improvement in Healthcare For many of the same conceptual benefits that private sector counterparts see in collaboration, healthcare organizations have recently taken to creating and joining collaboratives, networks, working groups, etc dedicated to quality improvement (Schouten, Hulscher, van Everdingen, Huijsman, & Grol, 2008). This has also been stimulated by the concerted, ongoing efforts of various international platforms which have highlighted the need for a greater emphasis on quality of care and shared knowledge translation. Multidisciplinary teams from various healthcare departments or organizations join forces for several months or years to apparently work in a collaborative way to improve their provision of care. Healthcare QI collaboratives are used regularly in the United States, Canada, Australia, and several European countries. In northern European countries such as the United Kingdom and the Netherlands health authorities have supported nationwide quality programs based on this strategy (Reerink, 1990; United Kingdom National Health Service, 2010).
19
The earliest well documented activities of QI collaboratives are those of the Northern New England Cardiovascular Disease Study Group, established in 1986 (The Northern New England Cardiovascular Disease Study Group, 2009), and the Vermont Oxford Network, established in 1988 (The Vermont Oxford Network, 2010). The Vermont Oxford Network is a good example of a long-standing non-profit voluntary collaboration of health care professionals dedicated to quality improvement, in this case as it relates to the medical care for newborn infants and their families. They also maintain an extensive database including information about the care and outcomes of high-risk newborn infants for use in quality management, process improvement, internal audit and peer review. VON offers member organizations the opportunity to participate in multi-center quality improvement collaboratives through their NIC/Q program, which allows for more intensive, face-to-face collaboration and their iNICQ program, which allows centers to participate with others over the internet via web-based conferences.
A system-wide approach to collaborative quality improvement is evidenced in the US in the work by the Institute for Healthcare Improvement or IHI (www.ihi.org). In 1991, Dr. Donald Berwick took his quality improvement efforts a step further and created the IHI as an independent not-for-profit national organization that is now leading the improvement of health care throughout the world at the system level through a campaign approach to “save lives” (referred to as “The 100,000 Lives Campaign”). Their approach to more specific topic areas is addressed in their Breakthrough Series program. Developed in 1995, the Breakthrough Series is a short-term (6- to 15-month) learning system that brings together a large number of teams from hospitals or clinics to seek improvement in a focused topic area such as physician prescribing patterns, efficiencies in intensive care, asthma care and reducing delays and wait times (Institute for Healthcare Improvement, 2003). 20
2.7 The Existing Model for a QI Collaborative Approach As discussed previously, the pioneer group to market the collaborative approach to quality improvement in healthcare was the Institute for Healthcare Improvement with their Breakthrough Series program. The model of the Breakthrough Series was conceptualized in late 1994 when one of IHI’s founders, Dr. Paul Batalden, sketched the model (Figure 1) on a napkin at a meeting of IHI’s Group Practice Improvement Network and handed it to IHI’s CEO, Dr. Don Berwick. Batalden and Berwick were seeking ways to accelerate improvement in health care beyond what IHI had achieved using traditional educational approaches.
The key to their thinking was to combine subject matter experts in specific clinical areas with application experts who could help organizations select, test, and implement changes on the front lines of care. They felt that breakthrough change couldn’t happen in a traditional didactic setting; instead, organizations would commit to working over a period of 6 to 15 months, alternating between Learning Sessions in which teams from all participating organizations would come together to learn about the chosen topic and to plan changes, and Action Periods in which the teams would return to their organizations and test those changes in clinical settings, leveraging the creation of inter-organizational relationships to learn from each other (Institute for Healthcare Improvement, 2003).
Since 1994, various iterations of the Breakthrough Series collaborative have emerged but they all seem to maintain the same components as the initial rendering – essentially engagement of organizations to work in structured, multi-faceted ways to improve one or more aspects of the quality of their service and that supports collaborative, interorganizational communication and knowledge exchange. More than 2000 teams from 1000 organizations in 7 countries have 21
participated in the IHI’s Breakthrough Series Collaboratives (Institute for Healthcare Improvement, 2003) and collaboratives now exist across the world in countries such as Australia, Cambodia, Ecuador, France, Honduras, the Netherlands, Nicaragua, Niger, Norway, Peru, Russia, Rwanda, Sweden, Tanzania and the United Kingdom (Wilson, Berwick, & Cleary, 2003).
Figure 1 – Napkin Sketch of the IHI Breakthrough Series Model by P. Batalden (1994) Source: The Breakthrough Series: IHI’s Collaborative Model for Achieving Breakthrough Improvement. IHI Innovation Series white paper. Boston: Institute for Healthcare Improvement; 2003. (Available on www.IHI.org)
22
Using the IHI as a model, individuals and organizations across Canada have also come together formally since 2005 to develop a national approach to promoting improvements in quality and patient safety called SaferHealthCareNow! It is described as “a grassroots campaign” to enlist Canadian healthcare organizations in implementing six targeted interventions in patient care (The Canadian Patient Safety Institute, 2010) and involves extensive data collection to support an audit & feedback framework.
Specifically related to ICU care, SaferHealthcareNow! has recently partnered with another national collaborative-based organization known as The Canadian ICU Collaborative (CICUC). The CICUC provides clinical support to the three ICU-related interventions (Rapid Response Teams, Central Line Infections, and Ventilator Associated Pneumonia) and is based directly on the IHI Breakthrough Series model. Their focus is on the use of the Plan- Do-Study-Act (PDSA) cycle to implement improvement initiatives and they leverage the collaborative use of several national experts in their selected topic areas.
The above are just a few examples of collaborative QI programs in operation at the national level in North America; however, thousands of similar programs exist around the world. Most operate in a similar template fashion – hospitals or clinics join the collaborative as an organization or a particular unit joins a specific initiative designed for their patient population (i.e. ICU, cardiac surgery, etc). Compliance or “participation” is generally measured by data submission related to clinical outcome measures and an integrated quality improvement culture is assumed to exist when improvement in a set of indicators is reported. Typical elements of collaborative-style QI programs involve multidisciplinary teams from participating organizations who participate in face-to-face meetings, electronic listserves, telephone conferences, group 23
training in improvement methods, submit data for pre-determined targets for improvement, development of disease-specific content, and follow-up support from other organizations and collaborative faculty.
2.8 Evaluation of Collaborative QI Interventions in Health Care To date, collaborative or multi-organizational QI networks have achieved mixed but promising results with varying success across participating organizations (Cretin, Shortell, & Keeler, 2004; Kilo, 1998; Mittman, 2004; Ovretveit et al., 2002; Shortell, Bennett, & Byck, 1998). A 2006 non-systematic review concluded that the collaborative methodology has important potential to improve outcomes for patients and to facilitate sustainability of quality improvement (Newton, Davidson, Halcomb, Denniss, & Westgarth, 2006). However, this introduction to the collaborative methodology neither considers whether the evaluation of effectiveness is based on a controlled or an uncontrolled evaluation design nor makes clear on what collaborative expectations the conclusions are based. A more recent systematic review by Schouten et al (Schouten et al., 2008) concluded that evidence of the impact of quality improvement collaboratives is positive but limited and that quality improvement collaboratives “seem” (emphasis added) to play a key part in current strategies focused on accelerating improvement, represent substantial investments of time and funding, but may have only modest effects on outcomes at best. Of the 72 published articles reviewed by Schouten et al, 60 were reports of uncontrolled studies which relied almost entirely on post measurement, used before-after studies which did not control for secular trends, made use of self-report data rather than chart review, included only anecdotal information or only reported data from self selected sites.
Of the
remaining 12 studies reviewed, only two were randomized controlled trials, one was an
24
interrupted time series and nine were controlled before-after studies. These twelve controlled studies are summarized in Table 1. Due to methodological flaws, highly heterogeneous intervention designs and irrelevant outcome measures to test collaborative effectiveness, there remains no certainty that the quality improvement collaborative was responsible for any of the positive effects published to date.
Despite the recognized success of the IHI program and others, to date a testable theoretical framework for the collaborative model has not been posited nor has anyone empirically evaluated how collaboratives work in relation to the outcomes they are said to achieve. This is a common problem in the evaluation of quality improvement interventions and Wachter and Pronovost actually raised it in relation to the IHI 100,000 Lives Campaign, stating that although this massive intervention succeeded in catalyzing efforts to improve safety and quality in hospitals, evaluation concerns make it difficult to interpret how much of the benefit can be attributed to the campaign itself (Wachter & Pronovost, 2006). This type of knowledge is becoming crucially important in light of modest quantitative effects reported in most quality improvement collaborative projects, serious sustainability issues and resulting calls for better evidence for this somewhat costly and time intensive method.
Clinicians who are asked to participate in quality improvement programs in healthcare organizations are often heard to ask for evidence that they work (Walshe, 2007). By that, they often mean they want randomized controlled trials, which show that PDSA cycles, Lean methodology, continuous quality improvement, collaboratives or whatever approach is being proposed, can cause meaningful and worthwhile improvements in the quality of care (Grimshaw et al., 2003). However, rarely are QI interventions designed as randomized clinical trials or any 25
other recognizable empirical design and so there is an important challenge now confronting stakeholders in synthesizing the results of the quality improvement interventions to answer the call for evidence. The heterogeneity of the evaluation methods used to determine the effectiveness of these typically multi-faceted interventions is problematic and is reflected in the lack of a formal Cochrane Library systematic review for quality improvement interventions. The Newton and Schouten publications discussed above which have attempted to synthesize the current literature in this area are based on the loose meta-analysis of quantitative results or extrapolations made on the broad accumulation of generally positive results and have not addressed the issue of the effectiveness of the actual implementation. The evaluation of the collaborative approach to quality improvement remains a classic example of a “missing link”. While a number of collaborative programs have published results of what are referred to as ‘evaluations’ based on the quantitative results of the clinically relevant outcome improvement indicators (Mittman, 2004), to date no published reports of such programs have properly evaluated the effectiveness of the collaborative approach itself in producing those results. Questions such as how teams interact as a result of collaborative vehicles for information exchange, how organizations learn or improve from participating in a network or collaborative and how participants experience the collaborative approach remain largely unanswered. In most cases an assumption is made that if positive improvements are seen that the collaborative approach must have had a hand in it. Rarely, when modest or negative results are found is there an investigation into the barriers or the failure of the collaborative approach.
26
Table 1 – Published Controlled Study Evaluations of QI Collaboratives (listed by study design) Primary Author
Title
(Horbar et al., 2004)
Vermont Oxford Network - Neonatal ICU (Surfactant treatment in preterm infants) (Homer et al., Breakthrough 2005) Series Collaborative; Chronic Care Model (Asthma in Children) (PierceBreakthrough Bulger & Series embedded in Nighswander, longitudinal QI 2001) activities (infant mortality in the community) (Horbar et Vermont Oxford al., 2001) Network - Neonatal ICU (Infection or chronic lung disease in preterm infants) (Rogowski et Vermont Oxford al., 2001) Network (Infection or chronic lung disease in preterm infants) (Baier et al., Based on 2004) Breakthrough Series (Benedetti, Flock, Pedersen, & Ahern, 2004)
Breakthrough Series Collaborative; Chronic Care Model
(Landon et al., 2004)
Breakthrough Series (HIV treatment)
Study Design (Study Sample Size) Cluster randomized control trial (114 units)
Method of Analysis
Primary Outcomes
Database analysis of clinical outcomes
Timing of surfactant treatment, mortality pneumothorax
Randomized Control Trial (22 primary care practices)
Telephone interviews
Written asthma management plan, daily use of inhaled steroids, daily use of controller drugs
Interrupted time series (1 medical centre; no control group)
Chart review
Annual average days between deaths
Controlled prepost study (76 units)
Database analysis of clinical outcomes
Rates of infection, rates of oxygen supplementation, death
Controlled prepost study (19 units)
Database analysis of clinical outcomes
Median cost per infant
Controlled prepost study (87 nursing home units) Controlled prepost study (30 providers)
Minimum dataset analysis
Prevalence of patients with pain
Method unclear
Controlled prepost study (69 clinics)
Review of medical records
End organ surveillance, glycaemic control, dyslipidaemia therapy and hypertension control Antiretroviral therapy, screening & prophylaxis and access to care
27
In 2003, Wilson and colleagues reported that data from fifteen participants in collaborative-style QI initiatives worldwide, indicated that many believed in the importance of both supportive interactions with peers from other organizations and competition among organizations, uniting staff in a race with other collaborative teams (T. Wilson et al., 2003); however actual communication activity has never been prospectively measured. Recent work by Marsteller et al (Marsteller et al., 2007) produced an informative assessment of team interaction between teams in three QI collaboratives. Using telephone surveys of the collaborative contact persons and four performance measures to assess the usefulness of ties to other teams, they found that teams were generally satisfied with their contacts, and that tapping another team’s “knowledge, experience or expertise” was the most common primary reason that teams chose for contacts with other teams.
In addition, Nemhard et al (Nembhard, 2007) used secondary data collected in four IHI Breakthrough series’ combined with participant telephone interviews to understand participants’ views on the relative helpfulness of various common features of a collaborative approach. Data collected from 53 teams over three years showed that participants viewed collaborative faculty, solicitation of their staff’s ideas, change packages, Plan-Do-Study-Act cycles, learning session interactions, collaborative support and project management skills as the six features most helpful for advancing their improvement efforts and knowledge acquisition overall.
These empirical findings begin to provide some support for the largely unsubstantiated theories about why interorganizational collaboratives could work for large-scale quality improvement. They represent research that comes from a more interpretivist paradigm in that they aim to understand collaboratives from the point of view of the participants, rather than an explanation of 28
the world based on external factors (Sandelowski, 1993). However these few studies are still not enough to give us a full understanding of the effectiveness of the collaborative approach. The only related study is that by Mills et al (Mills, Weeks, & Surott-Kimberly, 2003) which examined whether innovations spread within organizations from participating units to other units in the same hospital or other regional hospitals. Unfortunately, they found that there is little diffusion outside participating teams even though most participating teams showed improvement and reported a positive experience.
2.9 Summary There is no question that the quality movement has had a major impact on the improvement and safety of healthcare systems around the world. The collaborative approach to QI specifically is an example of a promising methodology, however enthusiasm has taken the place of evidence to date and we are lacking an empirical understanding of how the approach and specifically knowledge translation within collaboratives actually works in real world circumstances. The published evidence on the effectiveness of QI collaboratives in healthcare, as discussed above, is almost exclusively based on the quantitative results of the clinical improvement indicators – rarely if at all are measures of the components of the collaborative approach (i.e. communication analysis, use of KT strategies, resource uptake, etc) reported or discussed. Further knowledge of the effectiveness of the basic components or strategies of collaboratives, cost effectiveness, variability within collaboratives, and reproducible success factors is crucial for truly determining their value. Like any new paradigm or program, we must be careful not to accept the success of the collaborative approach on face validity alone, especially given its expense in terms of resources and human effort. The apparent inconsistency between the widespread belief in and use of
29
quality improvement collaboratives and the available evidence heightens the importance of a deeper understanding of the relative strength of this intervention for healthcare (Schouten et al., 2008). When interventions succeed or fail in achieving significant outcomes, it is important to understand which components of the intervention contributed to the success or failure. This is especially true when large sums of tax money or research funding is at stake for large-scale quality improvement initiatives, such as is the case in healthcare.
The research presented in this thesis is focused specifically on beginning to piece together this deeper understanding of the participant’s experience of the collaborative approach using process evaluation methodology in a sample collaborative. A key starting point for an exploration of any complex intervention such as this is a review of the developed theory from adjacent fields of knowledge that may help to direct and inform the investigation of the research questions of interest. In the next chapter I will review relevant theories from organizational behaviour, communication and collaborative theory to help inform this particular study.
30
- CHAPTER 3 BUILDING AN ANALYTIC FRAMEWORK
Chapter Overview Despite the widespread uptake and use of collaborative models for quality improvement, there remains little published discussion of the theoretical underpinnings of this approach. A range of models and theoretical perspectives focused on related areas are available however they have not been utilized in relation to the collaborative as it is used in healthcare QI. In this chapter I provide an overview of selected perspectives from multiple disciplines that may be useful as a an analytic framework for the investigation of specific components of collaborative quality improvement in healthcare.
3.1 Introduction Collaborative activity underpins a great majority of human problem solving effort (Elliott, 2007). Interorganizational collaboration is also the foundation of the recently popularized quality improvement “collaboratives” in healthcare. Beginning with the Institute for Healthcare Improvement series of breakthrough collaboratives in the early 1990’s, this model is based on the idea of local providers and organizations working together to achieve breakthrough results for common health care challenges such as asthma care, reducing infection rates and end of life care. As discussed in the previous chapter, despite the widespread uptake and use of this collaborative model throughout healthcare, there remains little published discussion of the theoretical underpinnings of this approach. This is becoming increasingly detrimental to the advancement of such models, as it does not allow for generalizable evaluation or useful theory development. Common network-level theories typically draw on many of the behaviour, 31
process, and structure ideas and measures developed by organization-level researchers. The focus of interest in the case of QI collaboratives is not only about the individual organizations but on explaining properties and characteristics of how a collaborative functions as a whole and how these functions might increase the effectiveness of approaching QI this way.
A range of models and theoretical perspectives from disciplines outside healthcare have previously addressed related concepts of collaboration and interorganizational relationships (IORs) and so it is important to address them here as an analytic lens for the empirical work presented in this thesis. For example, the strategy literature and theory emphasizes collaboration’s role in helping organizations acquire resources and skills that cannot be produced internally (Dyer, 2000; Gulati, Nohria, & Zaheer, 2000; Hamel, 1991). A similar approach is found in domain theory, in which theorists argue that collaboration helps to pool resources and produce solutions to social problems (Gray, 1989). Learning and innovation theorists argue that collaboration can facilitate the creation of new knowledge and not just the transfer of existing knowledge (Gulati & Gagiulo, 1999; Powell et al., 1996; Larsson, Bengtsson, Henriksson, & Sparks, 1998). And finally, the literature on networks, social capital and political aspects suggests that collaboration can affect the structure of interorganizational relationships, making some organizations more central and increasing their influence or power over others (Burt, 1992; Gulati, 1998; Tsai, 2001; Wasserman & Galaskiewicz, 1994). Despite advancements made within these individual streams, the lack of cross-reference between models has not allowed for thinking towards a common theoretical base for results in the absence of any one integrated theory of collaboration or collaborative activity.
32
3.2 Defining the term “QI Collaborative” Definitions are key to developing a relevant and testable framework. Because of the broad use of the term “collaborative”, it is important to discuss the limitations of current definitions and define the term as it is used for the purposes of this dissertation. Recently healthcare “networks” have been the object of study and theorization (Morrissey et al., 1994; Provan & Milward, 2001;Provan, Nakama, Veazie, Teufel-Shone, & Huddleston, 2003). The definition of networks in these cases are most often describing care “networks” which involve improving sequential interorganizational care of the same patients versus collaborations for QI and knowledge exchange. This highlights a very important point about the nomenclature in this area. In researching the theoretical literature related to interorganizational relationships I became aware of the variation in the use of the terms network, collaborative, alliance etc. A collaborative network is typically viewed as a physically decentralized social network made up of individuals who form a community but are not members of the same formal organization (Howard, 2002). These organizational networks may also be referred to as several other things, for example ‘communities of practice’ in sociology or ‘knowledge networks’ in management or ‘collaboratives’ in healthcare. A range of methodological approaches has been used to study these social phenomena, but each approach is built upon a set of models used to explain or describe individual relevant components rather than addressing the process as a whole. In order to be clear for the forthcoming theoretical discussion I will now present the definition of a quality improvement collaborative that will be used for the purposes of this thesis.
33
To begin to construct a useful definition, I first looked to the definitions of the word “collaboration” in and of itself. A countless number of definitions have been used to define collaboration in the organizational and management literature, a few of which include:
Gray’s definition of collaboration as “a process through which parties who see different aspects of a problem can constructively explore their differences and search for solutions that go beyond their own limited vision of what is possible” (Gray, 1989) Roberts and Bradley, borrowing some elements of Gray and others construct the definition as “an interactive process having a shared transmutational purpose and characterized by explicit voluntary membership, joint decision making, agreed upon rules and a temporary structure” (Roberts & Bradley, 1991) Selsky does not define collaboration specifically but defines the development of a collaborative venture as a “medium to long-term systematic capacity for addressing shared problems or for achieving shared goals at the interorganizational and community levels (Selsky, 1991)
Wood and Gray synthesized these various definitions and attempted to create a broader definition that was less implicit, more precise and included all of the elements that appeared consistently in the other definitions found in the literature. They created the following general definition (Gray & Wood, 1991):
Collaboration occurs when a group of autonomous stakeholders of a problem domain engage in an interactive process using shared rules, norms and structures to act on or decide issues related to that domain.
Quality improvement collaboratives have also been diversely defined, which in itself has led to evaluation challenges. A sample of some of the definitions found in the current literature on QI collaboratives include:
The IHI defines their Breakthrough Series Collaborative as “a short-term …learning system that brings together a large number of teams from hospitals or clinics to seek improvement in a focused topic area” (Institute for Healthcare Improvement, 2003). Ovretveit et al (2002) - defined a collaborative as “an initiative that brings together groups of practitioners from different healthcare organizations to work in a structured way to improve one aspect of the quality of their service” (Ovretveit et al., 2002).
34
Mittman (2004) - described the QI collaborative method as “bringing together a group of participating health care delivery organizations and guides them in studying a specific health care quality problem, designing and implementing specific solutions, evaluating and refining these solutions, and disseminating findings to other organizations”(Mittman, 2004). Schouten et al (2008) - defined a QI collaborative in very vague terms as “a program which provides different multifaceted packages that in general focus on accelerating better outcomes to a group of health professionals, units or organizations”(Schouten et al., 2008)
It is of note that none of the QI definitions given to date include explicit terminology that address the action of “collaborating”. The wording most often used is more indicative of the type of membership than the actual act of collaborating or even communication between groups towards a common goal. The often-referenced successful components of most QI collaboratives, such as peer communication, collaborative learning and knowledge exchange (Schouten et al., 2008) as discussed in Chapter 2 are excluded from the existing definitions.
By combining the work done by Wood and Gray on a general definition of collaboration with the most comprehensive definition of a QI collaborative from Ovretveit and adding the missing elements of learning and knowledge exchange, I have created a more functional definition of the quality improvement collaborative for the purposes of this dissertation:
An approach that centrally coordinates voluntary groups of autonomous healthcare organizations with a common purpose to work in structured, multi-faceted ways to improve one or more aspects of the quality of clinical care and supports collaborative problem-solving, interorganizational communication and knowledge exchange at all levels.
This rather ideological definition is the foundation of this thesis and links to the most recognized quality improvement collaborative design. Other network or strategic alliance relationships described throughout the organizational literature do not necessarily intend collaboration but 35
rather the improved coordination of the provision of services or products and therefore it is important to be clear about what type of program is under study here.
3.3 Building an Analytic Framework from Theory in other Domains While interorganizational collaboration is receiving an increasing amount of attention from a wide range of research fields (such as art, science, industry, business, education, technology and medicine), disciplinary silos often impede the capacity to discover and share theorizing and research about collaboration and various collaborative models. This makes it particularly challenging to develop a generalizable theoretical framework that could inform practitioners who wish to utilize collaboration as a problem solving strategy or theorize its application in diverse contexts. The work to date also falls short of generalizing the unique features of collaborations within the non-profit sector (Guo & Acar, 2005). Rather than attempt to build a single metatheory about collaborative functioning, I have chosen to focus on three specific components that could benefit from theory development and inform a more full understanding of this phenomenon in healthcare.
My first area of interest is the motivation behind why organizations enter into collaborative relationships for the purposes of quality improvement. This is an important starting point as it may prove to have an effect on improvement results at the organizational level and therefore on the effectiveness of the collaborative as a whole. The second element is levels of communication that occur within a QI collaborative project. This is important as the nature of both intraorganizational and inter-organizational communications may have an impact on the identity of the collaborative as having true collaborative activity. The final area of interest is the notion of collaboration itself and whether it actually occurs in the context of a quality improvement
36
collaborative, as defined above. Theorization of this concept is central to the discussion of what contributes to the effectiveness of collaborative QI if actual collaboration is not the predominant activity.
Focusing on these three concepts, I have drawn on four streams of theory and/or research – organizational behaviour, network theory, social theory, and the nascent collaboration theory – to help build an analytic framework for how they may relate to the idea of collaborative quality improvement and to each other. In this case, I hope that the use of multiple theoretical perspectives to guide the analysis will be more useful than one overarching theory and will provide a unique lens for looking to for describing QI collaboratives, given their complex and multifaceted structures.
3.3.1 Why Healthcare Staff think it is important for units to belong to QI Collaboratives? While a shared goal or pursuit is generally the desired outcome of cooperation or collaboration, a collective effort is not always characterized by shared objectives and motivations on the part of each of its individual participants. Participants may have many varying motivations and objectives for collaborating around a particular cause. Theory about a healthcare organization’s or unit’s initial motivations in joining one or more QI collaboratives may be an important factor which can inform our understanding of the more micro-level functioning of a collaborative and the resultant outcomes for each organization.
The development of interorganizational relationships has roots in several of the foundational organizational behaviour theories; however organizational sociologists have typically viewed
37
network formation as driven by external factors such as economic or political pressures (Pfeffer & Salancik, 1978). In the last two decades the literature concerning interorganizational relationships, particularly strategic alliances, joint ventures, and social networks, has expanded dramatically and brought with it extensive theory development. A well-known theory from the field of organizational behaviour is Institutional Theory may be relevant to the question of why organizations feel it is important to belong to inter-organizational projects in the first place.
3.3.1a Institutional Theory Primarily developed out of post-WWII upheavals, such as the collapse of Communism, which led to uncertainty about the relation of institutions to political and economic change, institutional theory focuses on organizational management as it relates to control of environmental factors experienced by an organization. These include “external or societal norms, rules, and requirements that an organization must conform to, in order to receive legitimacy and support” (Scott, 1987). The most basic principle of institutional theory is conformity and conformity is considered the metric that is used to determine the legitimacy of an organization. Conformity is realized in how organizations aim to resemble one another or resemble successful organizations in their environment because they are faced with the same social and political pressures and wish to be perceived as similar to their lucrative counterparts. This is more specifically referred to in the organizational literature as isomorphism. DiMaggio & Powell (DiMaggio & Powell, ) identified three mechanisms by which it typically occurs: 1) coercive isomorphism which stems from political influence and the issue of legitimacy, such as in the case of government mandated performance benchmarks for hospitals; 2) mimetic isomorphism resulting from standard responses to uncertainty; such as when best practice processes or quality goals are poorly understood and hospitals look to other organizations for direction; and 3) normative isomorphism
38
which is associated with professionalization or the collective struggle of members of an occupation to define the conditions and methods of their work; such as when management & frontline healthcare staff look to what staff at other healthcare organizations do to achieve success and model it in hopes of similar improvement outcomes. This idea of isomorphism and homogenization of healthcare organizations is important in the context of quality improvement collaboratives as the perception of being more like successful organizations by being part of the same collaborative or having access to their improvement processes may be moving us in a direction that makes organizations similar without necessarily making them more efficient.
The drive for resource stability and isomorphism leads organizations to seek legitimation, which is achieved by the organization becoming embedded in political, legal, organizational, and cultural relationships that confer legitimacy (Suchman, 1995). DiMaggio & Powell, among other institutional theorists, emphasize the dominance of the motive of organizational legitimation over the motive of organizational efficiency. According to this theory, when forced to choose, organizations will select options, which preserve and enhance organizational legitimation. Legitimation leads to continuation of the resource stream upon which the organization depends, more so than does organizational effectiveness. Provan and Milward (Provan & Milward, 2001) and Reitan (Reitan, 1998) suggest that those organizations that claim participation in “collaborations or collaborative efforts” enhance their legitimacy within the community or environments in which they operate.
In recent years there has been mounting pressure on healthcare organizations to legitimize their focus on quality improvement by meeting government benchmarks and being seen to be participating in provincial and national quality improvement initiatives. While hospitals in 39
Canada do not face market pressures per se, they still compete for staff, funding, accreditation and government endorsement. Therefore the acquisition of resources and tools to provide a higher level of quality and perceived relationships with successful organizations still has a “competitive” advantage. In recent years there has also been an upsurge of quality improvement related collaboratives and networks for hospitals and units to join and despite increasingly constrained resources, organizations are choosing to join multiple programs thus validating the theory that organizations pursue increased legitimization over organizational efficiency. This may prove to explain the overall mediocre healthcare improvement results to date and the difficulties in achieving sustainable organizational change and improvement culture. I expand on the ideas of isomorphism and legitimization over efficiency in my analysis of the empirical work described in Chapter 7.
3.3.2 How do staff engagement & communication occur within a QI collaborative? Once healthcare organizations have joined a quality improvement collaborative or network, it is then the actions they take as participants in that collaborative that provide us with insight into the functioning of such groups. The early network effectiveness literature often focused on the predetermined organizational goals or network outcomes with minimal concern for the process, integration or diversity of views (Provan & Milward, 1995) that might impact effectiveness. An alternative approach has been to consider the view of an organization’s multiple constituencies or key stakeholders (Zammuto, 1984). This takes the thinking about effectiveness to the microlevel and begins to shift the focus to the process and structural issues previously ignored.
A key feature of the collaborative approach is the development of lines of communication both with external and internal stakeholders that contribute to knowledge exchange and the creation of 40
a collective identity. Quality improvement collaboratives have been described as complex social interventions, for which high levels of variance in context, content and application are inherent characteristics of the initiative (Walshe, 2007; Waterman et al., 2007). The level of participation of different healthcare organizations or units in the activities of a collaborative QI initiative will vary significantly based on various communication structures and engagement within each organization. This is actually a desirable situation for the purposes of innovative knowledge exchange however, it does not necessarily allow for clear theorization in any one direction. In order to look more generally at the constructs of staff engagement and communication within collaboratives I have chosen to draw on selected aspects of the theory of interorganizational relationships (Nohria & Eccles, 1992; Granovetter, 1985; Nohria & Eccles, 1992) and social movement theory, a subset of social theory, (Bate, 1995; Bate, Robert, & Bevan, 2004).
3.3.2a Theory of Interorganizational Relationships Theory of interorganizational relationships posits that IORs have the potential to add value to organizations in two ways: 1) they provide the possibility for shared innovation and enhancement, and 2) they offer participants the chance to discuss current professional practices with others in related fields, which may enable employees to integrate best practices more readily (Kraatz, 1998). The literature concerning learning processes in IORs suggests that communication and knowledge exchange occurs on both micro and macro levels (Knight, 2002). At the micro level of analysis, inter-personal links generally offer individuals the opportunity to share and learn skills that will improve their personal lives or work. At the macro level of analysis, IORs are thought to provide a forum for organizations to share and receive knowledge which may result in improving their organizations competitiveness and profitability. For example, Lorenzoni & Lipparini (Lorenzoni & Lipparini, 1999) found that the capacity of 41
interaction with network members was positively associated with a firm’s growth and innovation.
Quality improvement collaboratives are generally designed to increase communication and knowledge transfer at several levels and encourage a collective culture of improvement. At the organizational level, collaboratives are thought to provide an opportunity for hospitals or care units to communicate with their peers and thus exchange knowledge and collaborative solve common issues. At the individual level, collaboratives are theoretically designed to provide the synergy and resources to mobilize local action and engage frontline healthcare staff in the quality improvement movement. Enter social movement theory.
3.3.2b Social Movements Theory Social movement theory postulates that innovations are only likely to ‘‘catch hold’’ on the ground, or even be recognized as viable possibilities, if they are communicated and developed with stakeholders and consistent with local customs, habits, aspirations, and passions (S. P. Bate, 1995). The core issue in social movements theory is ‘‘local mobilization’’ or grass roots change. At the individual level, mobilization refers to “the concrete actions taken by a person in the direction of change while, at the organizational level, mobilization refers to the process of rallying staff within the organization to undertake joint action and to realize common change goals” (Huy, 1999). Broadly, the purist social movement theory perspective would advocate that improvement strategies move beyond a centre-led program approach to embrace a concept of citizen-led change that draws upon largely self-organizing local phenomena (P. Bate et al., 2004). For example, existing improvement programs in the United Kingdom’s National Health Service (such as the Cancer Services Collaborative) explicitly focus on improving patient 42
outcomes and experiences and the close involvement of clinicians in objective setting is proving more successful in terms of engaging clinicians than programs focusing solely on improving access to services. From this perspective, effective and sustainable QI through the collaborative approach is very much a social issue—a question of clear communication, stakeholder engagement and collective goal setting.
Frontline staff are the key players in the achievement of the quality improvement goals and it is their collective buy-in to the organizational approaches that gives any intervention, including the QI collaborative, life. Decisions to join a collaborative project or program are generally made at the organizational level, however active participation and related outcomes are controlled at the individual practitioner or group level. This dichotomy and the nature of communications which mobilizes stakeholders within the participating organizations has never been addressed in either the theoretical or empirical literature but is highly important for proper evaluation of the effectiveness of the collaborative approach. If communication within a collaborative is poor, the engagement and collective identity of staff will be poor and the expected organizational and network outcomes will be affected. However if evaluation of staff engagement and embeddedness of the collaborative intervention is not included as a mediating factor in the evaluation of the network effectiveness then the source of any positive outcomes is questionable.
3.3.3 Does collaboration actually occur within a QI collaborative program? Generally speaking, it is assumed that the “theoretical” reasons for why multi-organizational collaboration has an advantage over individual approaches to quality improvement are based on the concepts of shared work, potential for innovation and “many hands make light work” style
43
thinking. While the idea of sharing resources and working together to solve complex problems makes intuitive sense, the theoretical discussions about collaboration define it as a much more formal interaction. Due to the lack of a generalized theory of collaboration, I have chosen to draw from a more recent theory – the theory of collaborative advantage – to guide my research in this area.
3.3.3a Theory of Collaborative Advantage Although some researchers have expressed the need for a ‘general theory of collaboration’ others have argued that this is not a useful endeavour and to date no specific field of research has attempted such a formulation. Despite the word “collaboration” being widely used in several varying contexts such as education, science, art and business, very little research has been carried out to determine the properties of it as a process and no theory predicting its direction of action.
A theory of “collaborative advantage” has recently been introduced by Huxham and MacDonald (Huxham & MacDonald, 1992) and highlights the concept of synergy development between organizations towards the achievement of common goals. They state that collaborative advantage is achieved when something unusually creative is produced synergistically or an objective is met that no organization could have produced on its own and when each organization through the collaboration is able to achieve its own, objectives better than it could have alone (Huxham, 1993). A key term in this proposition is “synergistically”. Applied to a wide range of phenomenon, synergy has been broadly defined as ‘the combined or cooperative effects produced by the relationships among various forces, particles, elements, parts, or individuals in a given context—effects that are not otherwise possible’ (Corning, 2003). In addition, synergy provides one of collaboration’s most important effects—the capacity for the 44
collective creative output to be greater and/or different than if the individuals were working alone. Using a business example, the synergy involved in a corporate organization allows for greater and different results than if each employee were doing business individually. In the healthcare QI example, this would see hospitals or units working together be able to achieve better outcomes and faster results in quality improvement initiatives than if they were working on the same initiatives alone.
In describing the broadest distinguishing characteristics and theory of collaborative activity, it is also useful to contrast the term against others, which are also attributed to collective activity. Specifically, I have found that conceptualizing collaboration in relation to cooperation adds further dimension to the investigation of whether in fact it is the proper term for what happens in multi-organizational quality improvement programs. Although ‘collaboration’ may be referred to as an object in developing theories and definitions that describe it as a phenomenon, it is important to realize that collaboration is also a process and the actual act(s) of collaborating is what is of greatest interest here.
As a starting point for distinguishing “collaboration” from the often-interchanged term “cooperation”, the Oxford definition confirms that there is a considerable difference. Cooperation, defined as ‘working together towards the same end, purpose, or effect’ varies little to that of collaboration, except that the latter stipulates ‘especially in literary, artistic, or scientific work’ (Oxford english dictionary1989). This subtle yet important difference is reflected in its first usages which were in reference to literary collaboration (Elliott, 2007). The early usages of cooperation on the other hand are true to the above definition in that they refer to collective activity aimed at a shared pursuit without a creative component. The main distinction 45
between these two terms is therefore the addition of the concept of something being co- created in the case of collaboration.
While creativity provides a concise point of distinction between collaboration and cooperation, the theorizing of creativity is itself a large and diverse field, which it is well beyond the scope of this thesis. Applied to collective problem solving activities, collective knowledge production relates to cooperative processes where participants comply with common procedures, but are not engaged in the generation of alternative processes, objectives or ideas. Conversely, collaboration is characterized by divergent knowledge production where participants are required to generate and develop multiple solutions, one of which is then selected and applied by the collective as a whole. Contrasting the terms cooperation and collaboration provides insight into the processes and relationships regarding collective activity. Collaboration goes beyond but also includes cooperation as it relies upon compliance with standard procedures and processes but is distinguished from the ‘shared pursuit’ of cooperation by the inclusion of collective creation of a shared product. Arguably, the interorganizational synergy expected within a QI collaborative should be able to create that capacity for the collective creative output to be greater and or different than if the individuals were working alone however the concept of divergent knowledge production is questionable. I will specifically look at the collective or collaborative activities within the collaborative QI model in this thesis in order to begin to theorize what synergies exist and what is actually shared between healthcare organizations in a QI collaborative, which might increase their improvement success.
46
3.4 Discussion Given the complex nature of the collaborative approach to QI in the healthcare context, components of several multi-disciplinary theories are useful in beginning to think about how and why such an approach would be taken up by health organizations and why it would be successful in helping those organizations attain quality improvement goals. As mentioned above, many of the theories cited in this review have been developed in the context of for-profit business or economics and within a hierarchy-power framework. The context of interorganizational collaborative efforts in the non-profit healthcare sector, tend to come from a very different orientation, therefore negating the application of any one complete theory posited to date. Many network researchers have strongly criticized economic and sociological theories that attempt to explain organizations' practices and outcomes as solely the result of their individualistic striving for profit, autonomy, and/or stability within an environment of resources and competitors (Granovetter, 1985; Gulati, 1995; Nohria & Eccles, 1992). They have sought to augment or replace these views by mapping out patterns of ongoing social relationships among organizations and examining how social processes that occur within them may shape organizational participation and outcomes (Kraatz, 1998).
47
- CHAPTER 4 RESEARCH CONTEXT: DESCRIPTION OF THE SAMPLE COLLABORATIVE
Chapter Overview This chapter details in-depth account of the design of the sample quality improvement collaborative under study. This was a two-year provincial QI program known as the Ontario Ministry of Health and Long Term Care (MOHLTC) ICU Best Practices Project.
4.1 Introduction The demand for intensive care is increasing because of an aging population and the introduction of new life-sustaining technologies (Needham et al., 2005). This care is expensive and the necessary resources are limited (Groeger et al., 1992; Halpern et al., 1994; Halpern et al., 2004). Despite advances in critical care delivery, mortality remains high (Angus et al., 2001; Rubenfeld et al., 2005). It is thus imperative that eligible patients receive interventions which improve outcomes or decrease intensive care unit (ICU) length of stay (Pronovost et al., 2004). Delays between demonstration of effectiveness and the widespread use of such critical care evidencebased ‘best practices’ (Ilan et al., 2007; Rubenfeld, Caldwell, & Hudson, 2001) constitute errors of omission and jeopardize patient outcomes (Institute of Medicine Committee on Quality and Health Care in America, 2001; McGlynn et al., 2003). These delays in implementation of clinical best practices may be more extreme in non-academic hospitals, with heavier individual clinician workloads and fewer personnel to engage in collaborative continuing educational activities. This general problem is compounded in the province of Ontario, Canada because ICUs are widely dispersed geographically and no formal quality improvement program exists (Ontario Critical Care LHIN Leadership Table, 2007). 48
Changing clinical behaviour in the ICU can be challenging (Rubenfeld, Cooper, Carter, Thompson, & Hudson, 2004; Sinuff, Cook, Giacomini, Heyland, & Dodek, 2007). It is a very busy 24-7 hospital unit where patients are quite variable and could be there for trauma, cardiovascular, neurologic, oncologic or geriatric reasons or a combination of each. Patients are very sick and require intensive monitoring and care by a large multi-disciplinary team and time is at a premium for education and other non-patient care related tasks. In the non-ICU setting, multifaceted interventions targeting different barriers to change have proven to be somewhat more effective than single interventions (Grimshaw et al., 2001). Promising strategies include educational outreach, audit and feedback, and reminders (Grimshaw et al., 2006).
4.2 The Ontario ICU Best Practices Collaborative 1 In the fall of 2006, a multifaceted knowledge translation project was implemented in sixteen Ontario ICUs using quality improvement collaborative methodology. The aim of the collaborative was to increase the adoption of 6 evidence-based ICU clinical best practices that have been shown in high quality studies to improve patient care. The existing Ontario-wide videoconferencing telemedicine system was leveraged to allow all unit staff to have the ability to communicate in real-time with each other and with the coordinating academic hospital.
This study was registered at www.clinicaltrials.gov [ID #: NCT00332982] and was approved by the Research Ethics Boards of all 16 participating hospitals, each of which waived the requirement for obtaining individual patient consent. 1
The protocol for this study has been published as a peer-reviewed article of which I am a co-author and has been reproduced in part here with permission of the co-authors (Scales DC, Dainty KN, Hales BH, Pinto R , et al. An innovative telemedicine knowledge translation program to improve quality of care in intensive care units: protocol for a cluster randomized pragmatic trial. Implement Sci. 2009 Feb 16;4:5)
49
4.2.1 Intended Study Purpose and Design The primary objective of the case study collaborative was to determine whether a collaborative network delivering a multifaceted knowledge translation intervention (KT) including interorganizational communication, education, reminders and audit and feedback could improve the uptake of evidence based best practices in geographically separate ICUs. From a methodological perspective, this study was also designed to test a template for creating ‘quality improvement clusters’ of hospitals across regions, facilitating system-wide sharing of information, new knowledge and strategies for improving best practice use in various contexts
4.2.2 Participants The collaborative involved 15 individual intensive care units in 15 Ontario community hospitals, with units representing various geographic locations and ICU sizes (Figure 1). The collaborative was hosted by Sunnybrook Health Sciences Centre, where the medical-surgical-trauma ICU of this academic hospital was used as a pilot site for the KT interventions and data collection approaches. Because this ICU already had a well-developed educational and quality improvement infrastructure, data collected from this academic ICU was not considered in the primary analyses but was included in secondary analyses.
A central coordinating office
conducted the knowledge translation interventions, disseminated educational and promotional materials, arranged videoconferences and collaborative meetings, and analyzed all quantitative data.
4.2.3 Baseline Assessment A baseline survey of the directors and nurse managers of participating ICUs was conducted to understand their organizational structure, quality improvement culture and to obtain estimates of 50
patient characteristics. Dedicated intensivists supervise the daily care of admitted patients (‘closed’ model) in 7 (47%) ICUs, 1 (7%) ICU has intensivists available for consultation on admitted patients (‘mixed’ unit), and 7 (47%) ICUs are staffed by generalists (‘open’ ICUs). Many ICUs conduct multidisciplinary rounds involving physicians (9, 60%) nurses (11, 73%), respiratory therapists (9, 60%), pharmacists (9, 60%), and dieticians (7, 47%). The estimated mean number of patients admitted annually to participating ICUs is 824 (range 307 to 1700), and the median number of daily staffed and occupied beds is 10 (range 4 to 19). Mechanical ventilation is provided to an estimated 42% (range 10 to 65%) of patients.
4.2.4 Selection of Clinical Best Practices The following criteria were used to select best practices: potential to improve clinical outcomes (based on existing evidence); applicable to most patients; feasible to implement and measure process of care indicators or patient outcomes; not already consistently applied. An expert advisory panel generated 15 candidate best practices believed to satisfy these criteria. ICU directors of the participating sites (n=15) were asked to rate these in the form of a baseline survey. The following 6 best practices were chosen for this study because they received the highest ratings for relevance (4 or 5 on a 5-point Likert scale indicating ‘very relevant’ to ‘extremely relevant’) and mean estimated proportion of eligible patients: (1) prevention of venous thromboembolism; (2) prevention of ventilator-associated pneumonia; (3) prevention of catheter-related bloodstream infections; (4) daily use of spontaneous breathing trials for mechanically ventilated patients; (5) provision of early enteral nutrition; and (6) prevention of decubitus (pressure) ulcers.
51
Table 2 – Best Practice Site Selection Statistics BEST PRACTICE
% of sites who selected “very to extremely relevant”
% of patients eligible in participating sites
prevention of venous thromboembolism
87%
72%
prevention of ventilator-associated pneumonia
80%
50%
prevention of catheter-related bloodstream infections
93%
76%
daily use of spontaneous breathing trials for mechanically ventilated patients
80%
48%
provision of early enteral nutrition
87%
70%
prevention of decubitus (pressure) ulcers
93%
68%
4.2.5 Knowledge Translation & Behaviour Change Strategies During the course of the study, a multifaceted plan of behaviour change strategies was employed. This included well evidenced tools such as educational outreach, audit and feedback, and behavioural reminders (Grimshaw et al., 2006).
Educational Outreach For each best practice, a bibliography of relevant literature was generated and nationally endorsed guidelines were summarized in easy to read bulletins. A recognized content expert provided an interactive educational session using the videoconferencing network for each of the best practices. These presentations were made available on a website for later viewing by those staff not on shift during the conference. Each site was also encouraged to provide in-services and conduct their own educational activities during the course of the study.
52
Audit and Feedback Process of care indicators were audited for each best practice (Table 3) on a daily basis and feedback reports were disseminated to participating ICUs on a monthly basis. Each ICU was able to determine the identity of their own hospital on these reports, but performance data from other hospitals was presented as aggregate data. This enabled each ICU to perform anonymous inter-site comparisons to monitor their own progress throughout the project, and to provide feedback for educational and motivational purposes to their staff.
Reminders Behavioural reminders were introduced for each best practice where appropriate. Examples of these reminders included: promotional items (posters, bulletins, pins, pens, stamps, pocket cards), pre-printed order sets, and checklists.
The Use of Telemedicine The Ontario Telemedicine Network (OTN) videoconferencing infrastructure, which allows for real-time and simultaneous interactive video discussions involving participants at multiple sites was used extensively for collaborative communication. The OTN was used to coordinate study activities, provide interactive educational sessions from content experts, conduct monthly network meetings among ICUs, and host training sessions for data collectors and site educators.
4.2.6 Data Collection Collection of demographic information and pre-specified process of care indicators were performed in each ICU by data collectors at times distinct from the usual multidisciplinary patient care rounds. The central coordinating office trained each data collector in a standard way
53
to ensure data quality. A handheld electronic device (Palm Lifedrive™, Sunnyvale, CA, USA) was used to collect data in 15 ICUs each day from Monday to Friday, and in some cases on weekends and holidays. The central coordinating office also conducted regular site visits and intermittent audits of data collection processes. All data entered into the handheld electronic devices was automatically encrypted (128-bit) and wirelessly connected for real time uploading to the central encrypted database.
Each best practice was associated with at least one process of care indicator and criteria for determining eligibility for the best practice (Table 2). If a process of care indicator was recorded as being present, it was assumed that the best practice has been delivered to that patient for that entire day. Once data collection for the processes of care had commenced for a specific best practice, these data were collected in both study arms for the remainder of the study. However, the targeted campaign to improve any given best practice only took place over 3 to 4 month periods.
4.2.7 Outcome Measures The primary endpoint for the program was the difference in the rate of change in proportion of patients receiving each best practice in the actively targeted ICUs compared to the same practice in control ICUs during each 4 month study phase. This was described quantitatively as the ratio of odds ratios for improvement over time for eligible patients receiving each best practice in the actively targeted ICUs compared to the same practice in control ICUs (odds ratio [actively targeted] / odds ratio [control]) and adjusted for clustering within centres. The unit of analysis for this endpoint was the individual patient.
54
Table 3 – Best Practices and Associated Process of Care Indicators Best Practice Prevention of ventilator-associated pneumonia
Process of Care Indicators
Main Other Measurements Measurement Semirecumbent positioning - Number of - Number of eligible patient eligible patientdays associated with Orotracheal intubation days with head orotracheal (vs nasotracheal) of bed ≥30° intubation
Prophylaxis against deep vein thrombosis
Administration of anticoagulant prophylaxis
Daily spontaneous breathing trials
Spontaneous breathing trial or extubation within previous 24 hours
- Number of eligible patientdays on which spontaneous breathing trial (or extubation) was performed
Prevention of catheter- 7-point checklist for sterile insertion completed related bloodstream fulfilment of all 7 criteria infections listed on checklist anatomic site of catheter insertion
- Number of central venous catheters inserted using all 7 criteria on checklist
- Number of - Number of eligible patient eligible patients days associated with receipt receiving of anticoagulant prophylaxis Use of antiembolic appropriate - Ineligible days associated stockings if anticoagulant anticoagulant with use of antiembolic prophylaxis contraindicated prophylaxis stockings within 48 hours
- Number of central venous catheters inserted at the subclavian site (vs jugular or femoral sites)
Early enteral feeding
Initiation of enteral feeds within 48 hours of ICU admission
- Number of - Number of eligible patients eligible patients achieving 50% of their target receiving early caloric goal via the enteral enteral feeding route by 72 hours within 48 hours of ICU admission
Decubitus ulcer prevention
Completion of the Braden index at least daily
- Number of patient days with Braden index completed
55
Secondary endpoints also included the rate of improvement over time in actively targeted ICUs and the rate of improvement over time in control ICUs; the proportion of eligible patients receiving each best practice during the final month of the study phase in active ICUs versus control ICUs; the proportion of eligible patients receiving each best practice in actively targeted ICUs one year after the initial intervention. Information on ICU length of stay and ICU mortality were collected for descriptive purposes only. Limited clinical outcome measures were measured for some best practices (for example, rate of catheter-related bloodstream infection) but were not considered a primary outcome of the study.
4.2.8 Data Analysis All data was analyzed using SAS (version 9.1, Cary, NC). For descriptive statistics, mean and standard deviation or median and interquartile range for continuous variables and proportions for dichotomous variables was reported. The student t-test or Mann Whitney U test was used, where appropriate, for comparisons of continuous variables and the Chi square or Fisher exact test for comparisons of proportions.
The odds ratio for receiving a particular best practice, identified using process of care indicators performed in eligible patients, was calculated in both groups using generalized linear mixed methods (GLMM) to account for the hierarchical nature (clustering within centres) of the data (Murray, 1998). The primary dichotomous outcome - the rate of change in proportion of patients receiving each best practice - was analyzed by testing for the effects of group (targeted intervention versus control), time (during 4 months of intervention), and the interaction between group and time (the ratio of the odds ratio of improving over time in the targeted group versus the odds ratio in the control group). For secondary analyses, a before-after comparison was 56
performed in all hospitals to calculate the odds ratio for receiving each best practice following the active intervention phase. For hospitals that have already been assigned to receive a particular best practice prior to the crossover period, declining use of the best practice following the crossover point was monitored and reported as ratio of odds ratios for receiving each best practice, after versus before.
4.4 Summary To our knowledge, this was one of the few randomized controlled trials of a QI collaborative targeting adult ICU quality improvement. Much has been written about the effectiveness of various knowledge translation strategies in the outpatient setting (Foy et al., 2005; Foy et al., 2007; Grimshaw & Russell, 1993; Grimshaw et al., 2004; Grimshaw, Eccles, & Tetroe, 2004; Grimshaw et al., 2006), but less is known about the ICU environment (Kahn & Fuchs, 2007; McMillan & Hyzy, 2007).
The organizational structure of an ICU can pose unique challenges to quality improvement because of the multidisciplinary approach to care, heterogeneous patient populations, and the focus on patients defined by geographical location in the hospital rather than by a particular disease (Curtis et al., 2006; Garland, 2005a; Garland, 2005b). A frequent shortcoming of many previous knowledge translation studies has been the use of non-randomized designs (Zwarenstein & Reeves, 2006) the lack of integrated qualitative evaluation to explore causal mechanisms and a focus on generalization which may be impossible to achieve. This innovative 6-in-1 trial of best practices implemented across a group of heterogeneous ICUs enabled us to evaluate a collaborative, multi-faceted network intervention at the level of individual ICUs and at the level of the entire system. 57
The design of the quantitative piece of this study incorporates several unique features that merit attention. First, the cluster-randomized approach enabled inter- and intra-ICU comparisons of performance, and enable adjustment for unit-level factors that might affect utilization of best practices. Second, the active control arm ensured that all ICUs are engaged in quality improvement activities during each study phase (each ICU simultaneously functions as an intervention unit and a control unit), and avoids the perceptions of unfairness that would arise from randomizing individual ICUs to no quality improvement. Finally, the design allowed for longitudinal before-after comparisons for units that are originally assigned to receive the control phase for a given best practice, and for assessment of decay in the use of a best practice for ICUs initially assigned to the intervention phase. This unique approach to evaluation should be appealing to policy makers and funding bodies interested in studying future system-level initiatives in the ICU and in other areas of healthcare as it takes a collaborative system level approach to standardization of quality care provision in a complex environment.
By providing the details of the program implementation here I do not intend to offer it up for questioning but rather to provide a detailed context for the in-depth qualitative investigation which is the focus of this dissertation. The structure-process-outcome frame, of which the context description is the “structure”, is one of the strengths of the process evaluation approach. Appreciating the context of this investigation will allow readers to better understand the findings of my independent qualitative investigation of the collaborative approach.
58
- CHAPTER 5 RESEARCH METHODS
Chapter Summary This chapter provides with a detailed description of the design, execution and analytic methods I used in the empirical qualitative investigation of frontline healthcare staff experiences within the sample collaborative.
5.1 Introduction Avedis Donadbedian said “Measurement in the classical sense - implying precision in quantification - cannot reasonably be expected for such a complex and abstract object as quality.” (Donabedian, 1980). Yet most quality improvement initiatives today are evaluated solely on quantitative outcome indicators selected for clinical relevance i.e. infection rates, length of stay or mortality rates and do not account for the evaluation of the actual program design or process improvements targeted by such programs. This point of view marginalizes the impact of socio-cultural and contextual factors in the evaluation of quality improvement success and overestimates the ability of experimental and largely quantitative methods in assessing and understanding what is actually a complex social intervention. In particular, more attention needs to be paid to understanding the contexts in which interventions are used, the process by which they are applied, the perceptions of participants involved in the intervention and the nature of the results or outcomes (Walshe, 2007).
Scholars in the field of public and community health, education and law have studied the implementation of complex evidence-based behaviour change strategies for many years,
59
however the science of doing so in real-world clinical health service settings is quite nascent. This research has recently been referred to as, among other things, ‘implementation science’. A defining feature of implementation science is the close attention it pays to the socio-cultural environments or systems in which health services interventions must be fit. Current quantitative evaluations of professional behaviour change strategies provide little insight into the causal mechanisms through which the interventions lead to change or do not lead to change and how these are moderated by different knowledge translation barriers and enablers (Michie, Fixsen, Grimshaw, & Eccles, 2009).
Huschler et al more recently proposed the use of process evaluation methodology for describing and understanding large scale quality improvement (Hulscher et al., 2003). Process evaluation is not a new concept and was described as early as the 1960’s when it appeared in a widely used textbook on program evaluation (Suchman, 1995), although Suchman does not label it “process evaluation” per se (Steckler, 1992). The contemporary process evaluation theory, its methods and their application to healthcare related interventions started in the mid to late 1980’s with a key public health publication by Basch et al entitled “Avoiding Type III Errors in Health Education Program Evaluations: A Case Study” (Basch, 1985). This article argues that measuring program implementation is critical in avoiding “type III error” – correctly rejecting the null hypothesis for the wrong reason (Mosteller, 1948) or in this case, evaluating a program that has not been adequately implemented (Dobson & Cook, 1980; Rezmovic, 1982) - and thus drawing incorrect conclusions about the effectiveness of a given intervention. Huschler et al propose applying the components of this methodology to quality improvement evaluation and reporting in order to draw out pertinent information about how the program was designed, how it was actually implemented and how it was received or perceived by the participants. 60
This approach shows promise in helping to understand the impact of implementation on the outcomes of QI collaboratives – a highly complex social intervention. In addition to establishing the core components of the collaborative as an intervention, progress in determining truly effective strategies requires not only an understanding of how the intervention works but also identifying a clear link between the intervention and its theoretical mechanisms of change (Michie et al., 2009). Theory informed process evaluations collect data on theoretical constructs alongside randomised trials to explore possible causal mechanisms and effect modifiers. This is akin to measuring intermediate endpoints in clinical trials to further understand the biological basis of any observed effects (Grimshaw et al., 2007). This type of approach is crucial in order to make the leap from a literature peppered with moderately effective interventions to a true understanding of the mechanisms of improvement.
5.1.1 Applying a Process Evaluation Framework to a Case Study Collaborative As discussed in previous chapters, the purpose of this dissertation is to move towards a more thorough understanding of how a quality improvement collaborative actually works on the ground. As such, the use of a process evaluation framework (as described above) provides a logical way to outline the intended and actual implementation of a QI collaborative that will inform my analysis of the specific components of the program both from a quantitative and qualitative perspective. The first step in applying this methodology is a detailed outline of the intended design of the case study QI collaborative. The case study I have chosen to use for my empirical work is the Ontario Intensive Care Unit Best Practices Collaborative. The description of this collaborative program involves a detailed explanation of the intended study design, participants, interventions and outcome measures. From there I present a summary of the quantitative improvement results based on the primary and secondary clinical outcomes of the 61
study. The final component of the process evaluation, and the main scientific focus of this thesis, is the presentation of my in-depth qualitative analysis of the opinions and experiences of the frontline participants elicited through one-on-one semi-structured key informant interviews. I will also examine three specific theoretical concepts as they relate to collaborative quality improvement approach - 1) why organizations join or agree to participate in collaborative efforts 2) how communications operate within a collaborative and 3) the concept of “collaboration” itself and if it actually occurred within this collaborative.
To my knowledge, this is one of the first prospective, theory-informed process evaluations of a large quality improvement collaborative with a focus on in-depth qualitative evaluation. It is important to note that the sample collaborative was not a mixed methods design – the collaborative itself was designed as a quantitative, cluster randomized trial. The research presented here is a post-hoc qualitative study which I conceptualized and carried out to help explain how the “collaborative” intervention was experienced by the participants and perhaps shed some light on the why the quantitative outcome results of the trial were somewhat modest. In doing so, I hope to uncover some of the mediating factors that influence collaborative effectiveness and contribute to both theoretical and methodological advancement of the collaborative as an effective QI strategy.
5.2 Study Design One-on-one key informant telephone interviews were conducted with staff members from a sample of the ICU’s who participated in the Ontario ICU Best Practice Collaborative between 2006 and 2008. The interviews themselves were conducted between 2008 and 2009. Personal key informant interview methods provide the most thorough way of capturing the participant 62
perspective and learning about the social processes of participation in such a project, communication activity and how collaboration may manifest itself, from the emic viewpoint.
5.2.1 Research Questions The overarching research question guiding this investigation is: how do healthcare providers and management describe the experience of being involved in a collaborative network for quality improvement? More specific questions which I will explore through the participant experiences include: (a) why do healthcare organizations choose to join collaborative QI projects; and (b) how does engagement and communication occur within a QI collaborative and (c) does collaboration actually occur within a QI collaborative.
As the next sections will illustrate, the nature of these questions determined the qualitative data collection and analysis methodology. The research questions provided a framework for the development of the interview guide and a starting point for the analysis to explore certain concepts that emerged. The interview guide was not informed by the theories discussed in Chapter 3 as these were intended to be used as more of an analytic framework. As with any inquiry, various approaches could have been adopted for this analysis. It is the specific concepts of purpose of joining, engagement and communication and collaboration that I explored in depth.
5.2.2 Participant Sampling A purposive sampling strategy (Denzin & Lincoln, 2000) was used to select the participants for the interviews. As it was impossible to interview all staff in each of the 16 participating hospitals, it was decided that a cohort of staff which represented the various hospital sizes, locations and staff groups involved in the project would provide a comprehensive data set. An 63
invitation email including the letter of information and consent-to-interview forms were sent to the project contact in each of the participating ICUs. This contact was generally the unit manager and so was considered a good starting point for communications. The contact was asked to invite other managers, frontline nursing staff, allied health professionals and physicians to participate. A minimum of three respondents from each site was expected. The only inclusion criterion was that the interviewees had to have worked as a nurse, allied healthcare professional, physician or manager in one of the sixteen participating ICUs during the time that the ICU Best Practices Project was active (January 2005 to October 2007).
The data collection for the
qualitative study reported here was done between October 2007 and April 2008. Ethics approval for this qualitative study was obtained from the Research Ethics Review Boards of all sixteen participating hospitals.
Those participants who were interested in participating were asked to fax a consent form with their telephone number and convenient call times back to the interviewer. Once consent was received, participants were contacted by telephone by the interviewer and the purpose and structure of the interview was reviewed. It was also confirmed through verbal consent that they knew they would be audio taped. Interviews lasted on average approximately 40 minutes.
Each interview was conducted one-on-one by telephone with only the interviewer and the interviewee present. It was decided that due to my intimate involvement with the implementation and evaluation of the ICU Best Practices Project as a co-investigator, it would be more suitable for an independent research consultant to conduct the interviews for this empirical work. A total of 32 interviews were completed representing 12 of the 16 sites that participated in the ICU Best Practices Project. 64
At the remaining four sites no response was received to the initial or follow-up email to the project contact. The non-participating sites did were not differ in any significant way from significantly different than the participating sites. The thirty-two participants were a mix of male (4/32) and female (28/32) and included 23 staff nurses, 2 unit managers, 3 respiratory therapists, 1 dietician and 3 physicians. Participants experience in the ICU setting ranged from 1 to 32 years (mean = 15.7 years)
At the beginning of each interview the interviewer introduced herself as an independent research consultant hired by the project team to perform the interviews. She then described the nature of the project as further research that would help the team understand areas for improvement and inform the design of the next collaborative project. This description was scripted and identical for each interview. Participants were encouraged to be open and honest and the interviewer underscored the anonymous nature of the interview and that only the evaluation team would have access to the de-identified data.
Following a semi-structured interview guide (see Appendix A), each interview then began with broad questions regarding the respondent’s role in the intensive care unit, how long they had worked in that unit and critical care in general and then moved to more specific questions about their involvement and understanding of the ICU Best Practices Project. The interview then typically moved naturally between specific questions about certain aspects of the project and collaborative quality improvement to more probing questions of certain topics as they were raised by the respondent. Questions were phrased in a very open-ended fashion and clarifications were only made in the event that there seemed to be some confusion about which project or concept we were asking about. 65
All interviews were audio taped and transcribed by a professional transcriptionist. The transcribed files were then uploaded and managed in N-Vivo Qualitative Analysis software (NVivo 2009). In-depth data analysis began following completion of all of the interviews.
5.3 Analytic Methods The interviews were initially analyzed in the order in which they were completed so that I could get a sense of how the questions and narratives changed over time. When approximately half of the interviews were completed, the interviewer and I met to discuss the discourse that was emerging in general and areas for further exploration were discussed. I completed all of the analyses independently and then collaborated with an experienced qualitative colleague who provided a secondary review of a subset of interviews (6 interviews) starting from my coding framework and adding as she saw necessary to ensure an unbiased analysis.
I chose to use a grounded theory approach for this research because of the following salient features: a) it emphasizes the need for the researchers to be immersed in data, and b) it does not require a researcher to suspend or ignore all pre-existing theoretical knowledge, but instead encourages the development/enrichment of inductively derived theories by drawing upon (though not driven by) broad theoretical approaches that are not in the same substantive area (Glaser 1978); c) it involves both inductive and deductive thinking (Corbin & Strauss 2008); and d) it synthesizes subjective sampling and analysis techniques with systematic coding procedures (Corbin & Strauss 2008), thus allowing the researcher to be flexible and creative, yet rigorous. Although not part of the grounded theory rhetoric, grounded theorists are largely influenced by emic understandings of the world: they use categories drawn from respondents themselves and 66
tend to focus on making implicit belief systems explicit. The narratives from the interviews provided the data for the analysis used to reconstruct the perspectives of the individuals working in the organizations being studied. My analytic process consisted of three comparative steps – (1) comparison within each single interview, (2) comparison with other interviews and (3) comparison of interviews from other staff groups. During the analysis I also looked for ‘deviant’ or ‘negative’ cases, examples of experiences that ran counter to emerging propositions.
Open descriptive codes were attached to segments of the text in each transcript (Corbin & Strauss, 2008) The aim of this internal comparison in the context of the open coding process was to develop categories and to label them with the most appropriate codes. These first open codes were informed but not constrained by the theoretical influences proposed in Chapter 3. In this way it was possible to formulate the core message of the interview and to understand the interview as it related to the concepts of interest including any highlights or inconsistencies. This helped to interpret the parts of the interview in the context of the entire story from the interviewee’s perspective (Boeije, 2002). These were then used to suggest ways in which to organize the open codes.
Axial coding was then used to compare interviews from others in the data set. The open codes were grouped in to broad topic-oriented categories using the concepts of institutional theory, social movement theory, the theory of interorganizational relationships and collaborative advantage as a possible guide. All text segments belonging to the same category were then compared (Corbin & Strauss, 2008). Ultimately the topic oriented categories became further refined and formulated into fewer analytic categories through an inductive, iterative process of
67
going back and forth between the data and the analytic framework of the study (see Appendix B). Here I began to hypothesize about patterns and theoretical implications.
5.4 Summary The framework and analytic choices for this dissertation have been both pragmatic and purposeful in that they allow for study and explanation of the lived experience of a quality improvement collaborative and gave me the opportunity to learn more about the reality of quality improvement implementation than just the numbers were telling us. From here I will present the results of the empirical qualitative work in detail and relate the findings to the theoretical propositions discussed in Chapter 3.
68
- CHAPTER 6 EMPIRICAL QUALITATIVE FINDINGS
Chapter Overview This chapter is a report of the empirical findings of the qualitative research study undertaken for this dissertation. My analysis aims to outline some of the main themes arising from the key informant interview data as analyzed within the theoretical frameworks presented in Chapter 3.
6.1 Introduction By focusing on the experience of a group of people or of a particular event, qualitative research uses the development of generally insightful concepts to reveal the essence of social interaction or environmental conditions. They represent an analyst’s impressionistic understandings of what is being described in the experiences, spoken word, actions, interactions and issues expressed by participants (Corbin & Strauss, 2008). The science of qualitative research is grounding the concepts in the data and the art is knowing what concepts to pursue, how far to develop an idea and when it doesn’t fit at all (Eakin, 2010).
By using the theoretical propositions discussed in Chapter 3 as specific port holes through which I could begin to look at the sea of data, I was able to conceptualize larger thematic story lines which linked back to the theoretical assumptions but at the same time drew their own path in providing insight into my research questions. Here I outline those major concepts as well as some related minor themes which emerged from the grounded theory analysis and tie them back to the data through selected illustrative quotes from the interviews.
69
6.2 Findings In general the respondents spoke positively about their unit’s participation in the ICU Best practices project – when asked directly they made vague but positive comments like “it was great”, “I think it’s important” and “I was sad to see it end”. From there, more nuanced responses were elicited when participants were asked detailed questions about why they feel it is important to participate, how they communicated within the project and their perception on the concept of collaboration.
6.2.1 Why healthcare staff think its important for units to be part of QI Collaboratives This research question was purposely not asked directly during any of the interviews however respondents raised the topic unprompted as having significance. The following quotes demonstrate participant’s sense that their involvement in a QI collaborative was linked to their reputation, profile, and ability to be seen as competitive with other organizations.
CBP7 (Nurse Educator) -“…Well, they want to know that they are giving equal or better care than everywhere else in Ontario, that’s important to them, the people take pride in what they do” [coded as external recognition] CBP22 (Physician) - “…not just this project but a lot of the projects we’ve been involved within the unit, while they’ve been a lot of work and they’ve been great for patient care, in many ways they have been great for us because because we have participated, they’ve also been great for our profile with senior leadership and the board….all that kind of stuff just keeps your profile up there, in a good way.” [coded as internal recognition] CBP11 (Nurse): I think – and I’m going to be frank with you – I think there is an expectation that you will participate, because it is a Ministry initiative and that if you’re not seen to be participating then that will have an impact on your reputation. [coded as external reputation] CBP30 (Staff Nurse)“…I think anybody who cares about their work and their status in the community and whether we should be a considered a reliable ICU where people would want to have their family members come if they needed to, we need to care about how we are compared to other hospitals. Especially being a community hospital we want 70
to know how we stack up to teaching centres and that information is important. Not that I, you know, care what St. Mike’s or UHN really do, but it’s just nice to know if we’re comparing with our colleagues.” [coded as external recognition and comparison] The interview segments around this topic initially appeared to represent different things or at least different perspectives (initial codes used shown in square brackets). However after having initially coded them in different ways I started to compare data between interviews and noticed that several codes seemed to fit conceptually under a larger umbrella concept. Most participants talked about the benefits of participation at a more organizational level, more about what it meant for their unit in particular versus themselves personally. As I looked closely at the type of language the participants used and the nature of these comments it struck me that they seemed to be talking about a need for recognition and describing the impact of the organization’s reputation on their own identity. Staff at these hospitals, most of which are small community health centres, have great pride in what they do and their ability to provide good quality care despite not having the same level of resources (human and otherwise) as larger teaching centres. By participating in a multi-organizational group like a QI collaborative, they are able to use that as validation and recognition of their organization’s reputation. Their pride is not expressed as personal pride in their personal ability to provide good quality care for their patients but pride in their unit to provide high quality care. Participating in a multi-hospital group project is not legitimizing for themselves as clinicians but for their organization. This is very different from some respondents’ descriptions earlier in the interviews regarding how they participate in the project, how information is disseminated and what the data meant to them, which tended to be more individualistic and about what they got out of it personally.
71
It was also of note that a number of respondents would start their responses in this part of the interview with “…I’m going to be frank with you…” (as in the third example quote above) or “To be quite honest….”, indicating that they are about say something that is perhaps considered taboo or that they are enlightening the interviewer on some unacknowledged truth about how things really are.
6.2.2 Staff Engagement Tied to the respondent’s perceptions about the benefit of being part of a QI collaborative project was the nature of the communication they received about the project and its results. It is generally assumed that in the collaborative approach there is open bi-directional communication that occurs regularly, such that everyone from a participating organization or unit is aware and informed about the activities of the collaborative they are involved in and the improvement strategies being implemented. It assumes a connection at the individual level to the collaborative or network. It did not take long in the interview process to see that this was definitely not the situation in this particular case study collaborative. Participants discussed severe communication blockages and lack of stakeholder engagement quite frankly and revealed some very interesting perspectives. The main streams of discourse in this theme were about their thoughts on the staff engagement in the project, intra-organizational communication issues between staff and management with regard to QI work and their view of inter-organizational communication or lack thereof.
Because frontline staff are the key caregivers in hospitals, they significantly influence the quality of care provided and, ultimately, treatment and patient outcomes. Consequently, a healthcare organization’s pursuit of high-quality patient care is thought to be dependent, at least in part, on 72
their ability to engage and change culture within unit staff effectively. Despite this integral role, there seemed to be a complete lack of deliberate engagement of staff in taking on the project or discussion about the purpose and objective of participating. CBP04 (Staff Nurse) - “No one asked me per se. Sometimes in our unit things are…it’s whoever’s around and if you are not there on that particular day then you don’t hear about things until it’s already started” IN:Right, okay. And were you involved in the ICU Clinical Best Practices Project from the start? CBP04 - Not really involved, I just... I mean, I was there when they said, “This is what we’re doing and this is our focus and these are the kinds of things we’re going to be watching for.” CBP16 (Staff Nurse) - “You shouldn’t assume that people wouldn’t be interested. Instead of handpicking a couple of people that happen to be standing there, you know, maybe post something and say “This is coming around. Would you like to be involved”…” CBP03 (Staff Nurse) - I’m not really sure who all was involved but that was, for me personally as a staff nurse, that wasn’t something I was asked and I think that would have tweaked my interest earlier because then I would have been looking for it, right? CBP05 (Nurse Educator) - “Our committee would have fairly regular meetings and we would have input into the different information. Usually once we decided what we were going to do we would get a copy of different forms and then chose one and give it to the staff.”
The informal communication methods, assumptions and use of discourse such as “they said, “This is what we’re doing and this is our focus and these are the kinds of things we’re going to be watching for.” And “we would …then chose one and give it to the staff” supports the notion that frontline staff were not formally engaged in the process or involved in the decision making related to this QI initiative. Therefore, the lived experiences of staff involved in this particular
73
QI collaborative did not support the use of widespread and meaningful staff engagement, which is often considered a key success factor for quality improvement initiatives.
6.2.3 Communication during the Project between Management and Frontline Staff Although the goal is to substantively engage all staff in quality improvement activities, there is considerable variation in the degree to which this occurs. Respondents reported that often a disproportionate share of the responsibility and knowledge is held by nursing management or a select few individuals. To date the social process by which information is communicated and how responsibility for information is allotted has not been explored in any depth empirically. However, in this study the issue seemed to manifest itself as severe communication issues between management and frontline or bedside staff. I began to code this type of data as “the QI Bubble” because it seemed that there was a certain level of people involved in QI efforts in the individual units but there was a clear lack of dissemination of information and engagement throughout and across the team. The “bubble” wasn’t impermeable but it was not naturally inclusive either. It was perceived as more exclusive than inclusive by most bedside staff participants and therefore operates to prevent equitable staff engagement in the QI project, as the following quotes demonstrate: CBP34 (Chief Nursing Officer): “Most of our initiatives are led by management. Don’t get me wrong, I am management, and I think we have done a great job. But, if we are able to figure out a way to involve the front-line staff sooner, we would be able to improve our performance—not just improve our performance, but improve performance faster.” CBP03 (Nurse) - “Gosh…honestly I didn’t know a whole lot of details. There was only really discussion about it amongst our managers and some staff. Like when we found out we were going to be participating in the project there was some talk about it and then there was the board (referring to an information bulletin board in the unit),
74
but that was about it. So I guess I didn’t get enough information to really make me that interested in it.” CBP04 (Nurse) - “I mean, I kept doing the best practices because I figured that was something that was there, but I thought the project was over long before it was apparently, because I just stopped hearing anything about it.” CBP05 (Respiratory Therapist): “I suppose sometimes input from some…you know, a little more input from maybe a little more larger group of people [than just our committee] might have been good so that maybe they’d have a better understanding of the project itself. These descriptions also convey a reliance on very passive forms of information communication, such as “the board” the CBP03 Nurse mentions, rather than on more interactive forms. Strategies such as unit communication books or mass emails to disseminate project information and data results seem to contribute to the communication barriers as experienced by frontline staff. This point is more fully developed in the following accounts: CBP05 (Staff Nurse) - “We have a communication book that the manager writes important information in for staff and people are supposed to be reading it on, like, you know, every few days or every week or two, like, just to see what is new (referring to how project information was generally shared with staff).” CBP03 (Staff Nurse/Project Lead) - “Well we had to do multiple things. You know, we had some staff that don’t check their emails very much, you know, then you have some that, you know, maybe aren’t real interested in the board, and so I made a point of posting all of our results on that board. So people seemed to be interested in the results so that took them to the board where there is other information…and I just kept it really simple, you know” (referring to how they communicated project information with their staff)
In many instances, participants referred to communications as happening “by accident” which is an interesting finding given that this was a knowledge transfer study. This description suggests an apparent lack of planning or clear communication structures within units and introduces the
75
notion that perhaps the management felt it was more appropriate for them to make choices about what they felt needed to be shared with the bedside staff.
By contrast, respondents who were in leadership roles provided detailed, lengthy and sometimes celebratory accounts of activity and engagement on the part of their unit of which other members of which the same staff were often largely unaware. In many cases it seems the videoconference meeting or educational sessions were attended by only one or two individuals. Information about education and knowledge translation opportunities was known by only a small number of project and clinical leaders who then selected bits of information to share with the rest of the team in very brief sessions at the nursing desk or bedside. Although this study generated less data from allied health workers than other groups their responses very clearly suggested that their participation in quality improvement was often marginal. Remarkably, this remained true even when the intervention involved implementing guidelines specific to their areas of practice, such as spontaneous breathing trials for respiratory therapists or early enteral feeding for dieticians.
More than half of the respondents also commented on the knowledge translation difficulties between the various shifts, in particular night shift versus day shift. Often communication and QI information transfer is done solely to day shift clinicians because that is when the unit manager or project coordinator is working. CBP14 (Charge Nurse): Yeah, because that way even the night shift could do it. There is a lot of people that work permanent nights, by choice, and they miss out on a lot of these in- services. IN: So what happens when they have to have essential training? How do you do it? CBP14: They have to come in on their day off. They don’t have a choice... it’s mandated.
76
CBP16 (Respiratory Therapist): …to me that’s similar to presenting it on e-mail (referring to posting information for all shifts). Do they read it? They read it, they delete it. You can put posters up, ask them to sign it. Some do, some don’t. It’s very difficult, on night shift, to get education across, there’s no one to do it and we struggle with that, you know, not only involving this practice but certainly other areas within our department. CBP06 (Staff Nurse): “I see my manager five times in six weeks, so… because the way my schedule works I work a lot of nights, so I don’t really ever get to see her, and there’s always this “Oh ya, do this,” or “You have to read this,” and not everyone gets to read it or discuss it.” This further feeds into this concept of the “QI bubble” in that there is not only a discrimination about which individuals are directly involved with QI projects but further communication “shiftism” which disadvantages those who work night shift. This must have direct consequences on process of care and patient outcomes given that night shift staff are responsible for one third of a patients care in a 24 hour period. There seems to be the feeling of a 9am-5pm approach to quality improvement and knowledge translation in general. While there are not a vast amount of staff who work only nights there is a “different attitude on nights” which could definitely impact the improvement work done during the day and play a huge role in the success or failure of project interventions.
6.2.4 Inter-organizational Communication An important aspect of data analysis in qualitative research is being equally sensitive to both what participants say and what is absent from their accounts. This was particularly relevant in my attempt to explore the notion of inter-organizational communication.
One of the foundations of the collaborative approach to quality improvement is the principle that by creating open communication opportunities between similar organizations or units,
77
improvement efforts (and results) will come easier, faster or more efficiently. In only one interview of 32 did a participant speak about the benefits of inter-unit communication and the actual act of collaborating with another unit within this QI collaborative. In a few instances respondents mentioned the benefit of hearing what other organizations were doing through organized events but there was no indication of ongoing self-perpetuated communication between units or staff. And even at that, the nature of the interest in sharing was not for the purposes of collaborating for group improvement but rather for gaining access to information and strategies for their own use.
6.2.5 Intra-unit Communication Conversely, despite the lack of data supporting inter-unit communication as beneficial, surprisingly several participants described an increase in intra-unit communication and support that occurred because of their participation in the project. CBP22 (Physician Director) - “Yeah. And so it allowed us as a group to go there as a team and to understand what the whole project was about and then come back as a team and be fired up. We made sure that we had key people there (names people), and so it gave us the sense that it was something that we were doing together as a team.” (referring to the usefulness of in-person collaborative meetings) CBP05 (Nurse) – “I think with a lot things it was more the increase in staff-to-staff conversation that went hand in hand with the data collecting that helped.” CBP23 (Physician): “I think it’s more from an internal motivation and encouragement point of view, to feed back to the positive culture, get the nurses empowered to speak at the same table as physicians, but insofar as the actual comparison data, no, I think it’s more the internal motivation of our own data.” (referring to the usefulness of comparing data between hospitals)
78
This represents what may be a very important and under-appreciated benefit for QI collaborative participants. It certainly provides food for thought for those designing collaborative-based QI projects in that perhaps the leverage of participating in a quality improvement project occurs mainly for the intra unit team and that communication and collaboration would be more effectively fostered here rather than between inter-organizational teams, units or hospitals.
6.2.6 Competition in Collaborative Clothing One of the most fascinating findings in this data was the fact that these frontline stakeholders did not refer to collaboration or truly collaborative efforts in any real form. Inter-organizational activity did not seem to be of primary importance to the participants however a rather opposing concept of the creation of “friendly competition” was commonly discussed.
CBP7 (Nurse) - “Oh yes…especially if we were behind other hospitals, we’d say “You know what?” There’s a sense of competition…”Oh, we’ve got to get our act together. Look, we’re below these other hospitals, so….” (referring to the importance of seeing the interhospital comparative data) CBP9 (Nurse) - “…we knew it wasn’t just within our organization, like, we actually represented our organization in a bigger group, so it kind of instilled a little bit of competitiveness – and that helps, that never hurts…when we saw improvement or that we were doing better than others it allowed us to advocate for some practices and resources that we didn’t have and so in the end it was successful.” CBP4 (Staff Nurse) – “Well I don’t think it would have changed my practice but it might have been kind of nice to compare ourselves and see whether we were better or not, because then, you know, you get that competitive spirit going.” The respondents did not discuss looking for high performing hospitals and connecting or “collaborating” with them to find out how they improved, they spoke more to the comparative data as the driver for improvement. This idea of “friendly competition” was not discussed in a 79
negative context but rather was used to describe what motivated them. It was operationalized through the comparative quantitative process outcome data that was provided to each of the randomized groups at each phase of the project. The respondents talked about how important this data was both for “friendly competition” and for “proof of progress” (their words). The apparent benefit of creating motivational competition rather than focusing on collaboration may be a more productive leverage strategy for multi-organizational QI projects.
6.2.7 Multiple QI Projects and Resource Scarcity It is common for healthcare organizations and even individual units to participate in several quality improvement initiatives at the same time, some internally driven but many externally driven and often in the form of collaboratives. In several instances (about a fifth of the interviews) it took our interviewer a few minutes to orient the key informant to the specific QI project we were interested in discussing that day. IN: Right. Sorry, you know what? I’m just wondering, you sound like you’re talking about [name of another collaborative project]. CBP15 (Nursing Director): No, actually. IN: Okay, sorry, because people are sometimes involved in multiple projects. CBP15: Now, my question – maybe I am getting confused here – I’m talking about the performance improvement, or, like, the coaching teams, or are you talking about the performance improvement collaborative that used to be OCCTKN.... IN: Yes, this is a group... well, the leads are based at [this] Hospital... it’s the ICU Clinical Best Practices Demonstration Project. CBP15: Okay. IN: Right? And you were... CBP15: So that used to be the OCCTKN, that’s the one you’re talking about now. IN: Right, so all this we’ve just been discussing has... CBP15: Was the Ministry of Health coaching teams. IN: That’s something else. CBP15: Yeah, I think so…
80
Discussion of the impact of working on multiple projects simultaneously often led to the mention of the lack of resources, human and otherwise and also seemed to tie back to the “QI bubble” concept in terms of delegation of responsibility and information awareness. CBP05 (Nurse): “…they were implementing these different new programs and I think she (referring to the Unit Manager) just picked certain people that would be good at their input and good at teaching people rather than burden everyone…we only have so many staff to get this all done.” CBP21 (Unit Manager): “Well, the chief nursing officer at that time got us engaged in these programs and she was the one going to the meetings, and then as her role changed and the amalgamation took place she needed to download to someone else and so it was in the possession of the person that was the Telehealth coordinator, and then when I was given the intensive care unit as part of my management responsibilities it was given to me.”
In most cases resource limitations require that responsibility for QI work must be written into a job description in order for people in those roles to be accountable for it. For others it becomes an unpaid add-on responsibility which does not promote engagement or investment and projects lose significant momentum once that particular staff member leaves. Finally, there is evidence of a mismatch of expectations between leadership and front-line staff in relation to QI activity. Leaders or those involved in QI seem to make decisions about what needs to be transferred to frontline staff due to lack of resources. However, most staff would at least like to be informed and do not find it overly burdensome if it is the best for their patient. It becomes about getting the task or job done or just enough investment to make the benchmark as opposed to creating a common purpose and motivation which supports a culture of change and sustainability.
81
6.3 Discussion The perceptions of those sampled who were associated with ICU Best Practices project are somewhat against the grain of the positive promotional quotes generally seen about participating in a QI collaborative, however they may be suggestive of what actually occurs in over taxed healthcare units which feel the pressure to “do QI” and “share with others” but do not truly know how to or necessarily want to do that. In terms of process evaluation, the findings also expose gaps between the intended implementation process and what actually was disseminated and able to be taken up by frontline staff. In the next chapter I will discuss how these findings support or refute the commonly held theories of interorganizational relationships and what they mean for future evaluations of collaborative quality improvement.
Despite the lack of overwhelming quantitative improvement results as described in Chapter 4, the respondents used very positive discourse in discussing their general experience within the project. Staff openly discussed several reasons for why they felt it was important that their unit participate in such collaborative quality improvement projects, the most common of which was that it offers an opportunity to increase recognition of the high level of intensive care they provide. Commonly held foundations of collaborative QI such as sharing of resources and collaboration were not as evident and this may in fact be underpinned by the highly passive and fragmented communication systems in relation to quality improvement and participation in multi-organizational projects. This communication deficit seemed to operate at three levels; at initial engagement of staff in the project, of information and purpose during the project and at a hierarchical level which leads to the development of what I have termed a “QI bubble”. Further to this, staff felt that the purpose of information sharing was more useful as a method of inciting
82
“friendly competition” rather than a collaborative advantage and the use of comparative data to feed the motivation for self- improvement versus improvement of the group as a whole. These patterns in the data now need to be discussed in relation to the theoretical assumptions presented in Chapter 3 and how their support or negation impacts theory building in the area of multiorganizational healthcare QI.
83
- CHAPTER 7 DISCUSSION If you want to understand what a science is, you should look not in the first instance at its theories or findings, and certainly not what its apologists say about it: You should look at what the practitioners of it do - Geertz 1973: 5
Chapter Overview In this chapter, I will synthesize the findings of the qualitative investigation and discuss how they further our understanding of how collaborative QI may work at the frontlines of healthcare and interact with the prevailing theories of organizational behaviour, social movement and collaboration as discussed in Chapter 3.
7.1 Summary of Key Exploratory Themes The data set from the 32 key informant interviews was rich in several areas. However three major concepts and several supporting themes emerged during my analysis Using the related theoretical assumptions discussed in Chapter 3 as a way of looking into the data, I was able to begin to explore possible explanations for the research questions posed in Chapter 2. The overarching research question under investigation is: how do healthcare providers and management describe the experience of being involved in a collaborative network for quality improvement? Integral to approaching this overarching question are more specific secondary questions: (a) why do healthcare managers and frontline staff think it is important to belong to a QI collaborative; (b) how does engagement and communication occur within the collaborative program and (c) does collaboration actually take place within the collaborative.
84
The findings of this study are somewhat against the grain of the positive promotional quotes generally seen about participating in a QI collaborative quality improvement projects. The most interesting of these is that this collaborative was seen to provide a forum for recognition amongst their community peers of their ability to provide high level of intensive care to their patients versus to provide improvement goals from better institutions. Ideologic foundations of collaborative QI such as sharing of resources and collaboration were not as evident and this may in fact be underpinned by the existence of highly passive and fragmented communication systems in relation to quality improvement and participation in multi-organizational projects. The communication deficit seemed to operate at three levels; 1) at initial engagement of front line staff in the project, 2) of ongoing transfer of information and progress during the project to front line staff and 3) at an institutional leadership level which leads to the development of what I have labelled as a “QI bubble”. Further to this, staff felt that the purpose of information sharing seemed to be more useful as a method of inciting “friendly competition” rather than a collaborative advantage and the use of comparative data fed motivation for self- improvement versus improvement of the group as a whole.
These patterns in the data now need to be discussed in relation to the theoretical lenses presented in Chapter 3. Specifically, how their support or negation of existing theories impacts theory building in the area of how multi-organizational quality improvement actually happens in over taxed healthcare units which feel the pressure to “do QI” and “share with others” but do not truly know how to incorporate that within their daily workload. In this chapter I will interpret and discuss how these findings support or refute the commonly held theories of interorganizational relationships and what they mean for future evaluations of collaborative quality improvement.
85
7.2 Advancing the Theoretical Discussion Explanation is essential to theory and practice (Pentland, 1999). Unfortunately to date there has been little explanatory work and even less theory informed evaluation, done in the area of quality improvement (Walshe 2007; Eccles & ICEBERG, 2006) and more specifically none about the popular QI collaborative. To progress further it is becoming more and more important that we move from isolated quantitative measures of effectiveness to including an explanation of the underlying structures and processes which give rise to the improvement results and can help guide future implementation strategies.
By insisting on more detailed contextual evaluation in the future we will be able to better understand how consideration of social constructs should influence the design and theoretical underpinnings of QI initiatives. The qualitative findings presented in Chapter 6 clearly show that the participant’s description of their experience in the case study collaborative did not match well with the assumptions or logic used in its design and implementation. This doesn’t mean that as designers and implementers of the collaborative we were wrong, or that participants just didn’t participate, but rather that we did not put enough emphasis on evaluating the actual process by which the collaborative was functioning, designing target interventions early in the course of the project to address these shortcomings and the differences in underlying assumptions. Using my empirical findings as a base, I will now discuss each of the major concepts in relation to the existing theory and how it may inform the development of a more accurate theory of multiorganizational quality improvement.
86
7.2.1 Why Healthcare Staff feel it is Important for Units to belong to QI Collaboratives As mentioned in Chapter 3, the concept of legitimacy and its associated metric of conformity is the most basic principle of the long standing institutional theory of organizational behaviour. This idea of legitimization from the organizational perspective and in particular how organizations operate in multi-organizational relationships is not a new one, although it has not been applied directly in healthcare QI to my knowledge.
The metric of conformity is realized
in how organizations aim to resemble one another or resemble successful organizations in their environment because they are faced with the same social and political pressures and wish to be perceived as like their successful counterparts. In recent years there has been mounting pressure on healthcare organizations to legitimize their focus on quality of care by meeting government benchmarks and being seen to be participating in provincial and national quality improvement initiatives. However, the results of my research seems to take this idea of legitimacy in a different direction - frontline staff do not feel the need for their unit or organization to conform or be like their peer institutions, they feel that by being in the same QI program as other recognized hospitals, their current high level care will be recognized, and essentially it is their reputation that is “legitimized”. Their membership is simply a conduit for recognition rather then aimed at improvement benefits or the opportunity of learning from the best organizations.
But what does the concept of “legitimacy” mean here? The Oxford dictionary definition of “legitimate” refers to notions of being in compliance with the law; being lawful or being in accordance with established or accepted patterns and standards (Oxford English Dictionary, 1989). From the examples shown in Chapter 5 and others throughout the data, the staff appear to perceive it is more than just being in accordance with best practices, but exceeding those 87
standards to give exceptionally high quality care and receiving recognition for the quality of work they feel they routinely provide. I did not get the impression that the staff in these units want to be recognized for just complying; they want to be recognized for doing the absolute “best for their patients” (their words). Obviously they feel there is an underlying lack of recognition by others and that perhaps because of their hospital size and in some cases remote location, they are automatically marginalized as not possibly being able to give as high quality care as a large hospital in a major city. But by having the opportunity to collect validated data on their practices and be measured in the same way as others, they feel they can be recognized to be providing equally as good care as larger teaching centres. However, they are not looking for “equality” on all fronts - they actually seem to take pride in the fact that they are different and seek recognition for being good at what they do, despite their inequalities. This concept of reputational recognition rather than legitimization, including properties of acceptance and acknowledgement of their efforts externally seems to be a more accurate explanation of what they see as the benefit of their participation in multi-organizational QI.
Lawrence (1998) in his work in the area of legitimacy versus reputation states that legitimacy indicates that one is qualified for a particular profession or skill whereas reputation in contrast encompasses the concept of organizational standing vis-à-vis its counterparts. Reputation “differentiate(s) between the qualified [i.e. legitimate] and the “outstanding” (Lawrence, 1998). This seems to be more in line with what respondents described as the importance of belonging to the project. However, gaining a favourable reputation also implies that differentiation is necessary; therefore the other central concept of institutional theory, isomorphism, may have a different outcome on reputation than it would have on legitimacy. The idea of homogenization
88
of healthcare organizations and the assumption that participant organizations want to search out and emulate more successful organizations does not appear to actually be at play. Proposition 1 – Healthcare organizations join multi-organizational quality improvement programs for reputational recognition as equal to, or better than their counterparts in terms of quality rather than to aim to be like stronger peer organizations.
7.2.2 Staff Engagement and Communication within the Collaborative Operating at the same organizational level of analysis, the inter-organizational relationships afforded by belonging to such collaboratives are theoretically thought to provide a forum for organizations to share and receive knowledge which may result in improving their competitiveness and profitability. However, the inter-organizational links that have been described as beneficial within other inter-organizational relationships do not appear to exist in the QI collaborative sampled in this study or be of importance from the perspective of those who participated in the interviews. What was notable to them was the increase in inter-professional links and communication within their own units. The lack of inter-organizational links was not necessarily purposeful, they just didn’t seem to desire it or have the time to build such relationships. Or perhaps the lack of solid intra-organizational relationship between the leadership and the frontline staff as manifested in the breakdown in communication about the ICU Best Practices project and about other QI initiatives in the same unit, fails to provide a stable platform to value and take advantage of interorganizational communication and sharing across a network. It follows that staff may not want to share when they feel insecure or poorly informed on what is going on in their own institution. The evidence from the field of social network analysis and diffusion of innovation theory tells us that information is more likely diffused through geographically close, trusted ties (Rogers 1983) which would support this finding. 89
Despite the fact that the inter-organizational links are not as one would expect, the concept of units receiving inter-organizational knowledge to improve their competitiveness or standing is actually somewhat in line with the findings from this case study collaborative. Frontline respondents seem to feel that having access to knowledge about other organizations leveled the playing field and comparative data was useful for the purposes of leveraging “friendly competition” as motivation rather then for not-for-profit style collaboration. They did not speak of the benefit of getting information from the other organizations directly but more about having access to what others were doing through the central coordinating office. This rather undermines the collaborative purpose and provides insight into the issue of whether collaboration actually exists within a collaborative which is discussed later in this section.
Proposition 2 – Building inter-organizational relationships is not seen as an advantage of multi-organizational quality improvement projects for frontline participants, but what is useful is access to information about what others are doing. If we now move to look at the issues of communication and engagement at that more micro level of analysis, social movements theory tells us that it is about ‘‘local mobilization’’, referring to the process of rallying staff within the organization to undertake joint action and to realize common change goals (Huy, 1999). Again the case study collaborative seemed to depart from this approach; project knowledge and activity seemed to end up in a hierarchical “bubble” involving the upper management which did not promote widespread grass roots engagement or open communication relationships. This exclusivity, both intended and unintended is completely counter to the concept of grass roots mobilization proposed by social movements theory. The reasons for the bubble are likely deeply cultural and complex to unpack but the data identify a 90
number of functional contributing factors. First, existing roles as they are currently defined within the ICU team seem to subconsciously designate certain individuals as central and others as peripheral in relation to engagement with QI initiatives. Nurse educators, special projects staff, and team leaders are typically more central in QI communication circles while bedside staff nurses, allied health professionals and in many cases physicians are positioned only as consumers of information and on a need-to-know basis.
Second, reliance on existing passive communication practices does little to support broader and more meaningful engagement with quality improvement. As long as active discussion of the issues is limited to meetings attended mainly by those in leadership roles while others depend on notice boards and email for information, it is difficult to see how this will change. Moreover, simple structural barriers such as e-mail distribution lists that do not include all members of the multi-disciplinary team or information meetings held only during dayshift hours are clear impediments to effective and engaging communication. This is not unique to QI initiatives. Communication books/logs and end or start of shift group learning or point of care educators have been developed to encourage bidirectional dialogue on normal practice in each unit employing staff in shifts. It is unclear if any of these methods of communication to the front line staff are more effective than mass email and notice boards, teleconferences, group learning and central support by phone to research staff employed in this case study collaborative. A more personal approach to transfer of information (face to face meetings) requires adherence to union requirements (salary support, overtime, travel expenses) concurrent staffing such that ongoing care continues, all of which are costly for health care institutes struggling with budgetary constraints.
91
These findings suggest a very different theoretical standpoint – there are no inter-organizational relationships and collaborative quality improvement is clearly not operating as a social movement - therefore it does not subscribe to the theoretical assumptions of either network or social movements theory. Quality improvement in this sample collaborative appeared to operate in a rather uncoordinated task-oriented fashion, communicated randomly through a socially constructed hierarchy and not effectively resourced such that everyone in the unit could benefit. For these reasons, one could argue that the intervention was not truly “embedded” in these organizations who were meant to be working together to improve results. Related work by May et al on embeddedness of complex interventions proposes that embedding of a complex intervention is dependent on work to operationalize it in practice. The work of embedding (implementing and integrating) a complex intervention is shaped by factors that promote or inhibit participants enacting it and that the production and reproduction of a complex intervention requires that participants collectively invest effort in it (May & Finch, 2009). These concepts and effects of operationalization appear to be lacking in this collaborative model and may require much more serious attention and study in future QI collaboratives.
Proposition 3 – Lack of organizational communication and poor staff engagement may directly impact improvement results and embeddedness of quality improvement interventions.
7.2.3 Collaboration in the Collaborative Clearly, the above discussion of engagement and communication also has a direct influence on the notion of collaboration within a collaborative. Remembering back to Chapter 3, the theory of collaborative advantage tells us that the advantage is achieved when something unusually creative is produced synergistically or an objective is met that no organization could have
92
produced on its own and when each organization through the collaboration is able to achieve its own objectives better than it could have alone (Huxham, 1993). The latter is essentially the cornerstone of why it is believed that collaborative QI is an effective approach - hospitals or units working together must be able to achieve better outcomes and faster results in quality improvement initiatives than if they were working on the same initiatives alone.
The concept of collaboration for quality improvement may actually be a misnomer, that in fact organizations are not actually interested in collaboration as much as they are in co-operation or even competition. As discussed in Chapter 3, it is important to distinguish between the terms collaboration and co-operation because the subtle difference between these words describes a lot about where they are versus where they need to be vis-à-vis multi-organizational approaches to quality improvement. Admittedly, dictionary definitions of the two words can be quite similar however the root definition of collaboration actually focuses on the act of working together whereas cooperation stresses the product of the work. Cooperative learning in education for example is defined by a set of processes which help people interact together in order to accomplish a specific goal or develop an end product which is usually content specific. This is very different from the alternative of collaborative learning which there is a sharing of authority and acceptance of responsibility among group members for the action of the group (Panintz, 2003). The underlying premise of collaborative learning is based upon consensus building through cooperation by group members. In the case of multi-organizational healthcare quality improvement it is appears that the benefit, as experienced by the participants in this collaborative, was accessing help through defined or proven processes to accomplish a goal – in contrast to building consensus or sharing of responsibility for the actions of the group as a whole.
93
The education dichotomy actually maps very nicely to the QI collaborative case study and further underscores the need to reconsider what the expected effect is with this approach and how that effect can be responsible for increased improvement results. Theory in this area should not move in the direction of explaining collaboration, which effectively did not exist in this case study collaborative, but rather towards a theory of co-operative activity, the result of which drives competitive improvement in participating organizations. While it may not be as socially pleasing as the idea of collaboration, there is nothing wrong with competition as an expected effect. What is important is that the expected effect must be understood from the beginning for all involved in order to optimize the impact and leverage the process to maximize improvement results, which is the ultimate goal.
Proposition 4 - Healthcare organizations benefit from more co-operative and competitive style advantages rather than collaborative advantages in multiorganizational QI programs In summary, it appears that in comparison to other sectors, healthcare collaboratives operate at a level of institutional individualism that precludes actual collaboration.
7.3 An Emerging Conceptual Framework Academia defines a theory as a statement of relationships between units observed or approximated in the empirical world (Bacharach, 1989). A theoretical model of QI collaboratives would therefore be an explanatory statement of the relationships between the structure, processes and outcome of the collaborative which ideally would reflect input from the participating organizations. Despite the fact that the data presented for this sample collaborative does not appear to support existing theoretical assumptions about inter-organizational networks 94
in other domains, social movements or collaborative advantage, by uncovering the misalignment in this single study suggests a an alternative explanatory thinking as it pertains to the multiorganizational co-operative approach to quality improvement. Based on the findings of this case study, perhaps some QI initiatives are better described as multi-organizational quality improvement rather then quality improvement collaboratives.
The three constructs explored from the participant perspective in this collaborative may actually be related in what could form the beginning of a conceptual framework for the process of cooperative quality improvement (Figure 2). The reigning approach of organizations joining a “collaborative” for political reasons or without an articulated organizational purpose may in fact lie at the core of unfulfilled improvement outcomes; a common and clearly communicated organizational goal or purpose for participation is important and can also impact the embeddedness of the overall intervention. The idea of a commonly understood purpose is of course rooted in a solid internal communication structure which not only facilitates its determination as an engaged and invested group but of the continued dissemination of the organizational objectives and results. With the above in place, healthcare organizations can then benefit from belonging to multi-organizational projects where there is co-operation through a central access point for knowledge and comparative data which does not assume collaboration. In figure 2, the connecting arrows between the organizations and the central repository are bidirectional to indicate that something is given and taken from each but purposefully do not intersect between organizations thus subtracting the expectation of inter-organizational collaboration from the equation. The organizations are also lined up vertically on purpose to indicate ‘standing’, to reflect the importance of reputational legitimacy discussed above. Based on the findings presented here, it seems that recognition of these more accurate types of 95
relationships could lead to greater success for multi-organizational quality improvement initiatives.
This nascent conceptual framework is certainly not complete but is a starting point and can be used to discuss and test several initial hypotheses related to the concepts of organizational goals of participation, the impact of staff engagement, intra- and inter-organizational communication and competition, all of which remain unstudied in relation to multi-organizational quality improvement.
Figure 2 – A conceptual framework proposal for multi-organizational co-operative QI.
96
7.4 Summary Although a collaborative may be used as a noun, in developing theories and definitions which describe it as a phenomenon it is important to remember that collaboration is a process. Studying a process involves greater complexity than that of an object, as its existence is momentary, conditional and contextual. In focusing on collaboration as a process, aspects such as engagement, shared understandings and functional assumptions become important features. The findings discussed here are informative both for recognizing the importance of implementation gaps and the impact of pre-existing socio-cultural influences on complex implementations and ultimately outcomes.
In the sample collaborative, the lack of significant quantitative improvement may be attributable to the deficiencies in staff engagement, communication and inter-organizational collaboration identified in the parallel qualitative research presented here. Given the disparity in fundamental assumptions like these, one could argue whether there was an intervention at all in this project and this may in fact be the case in several other unresponsive trials of quality improvement collaboratives. Despite unique cultural conditions in individual organizations, these are patterns of activity which can be used to move beyond description to explanation in the initial stages of theory building. The difficulty in developing a theory in this area is that the process of quality improvement is not linear – organizations do not join a QI collaborative, receive the intervention and produce the outcome, nor do two organizations ever participate in the process the same way to get the same results. Providing probabilities to the explanations through pattern recognition and introducing connections between constructs may provide the starting point for developing a testable model of observed relationships to explain how the process occurs in reality and build more targeted interventions to complement the model and may provide more significant change. 97
- CHAPTER 8 CONCLUSIONS, IMPLICATIONS & FUTURE DIRECTIONS
Chapter Overview The purpose of this chapter is to provide a summary of the research presented, discuss how the findings can contribute improved insight into antecedents of and strategies for fostering more effective use of multi-organizational QI efforts and propose new directions for the design and development of such efforts. This chapter will also include the limitations and implications of the research which in turn present several future research directions and provide some final thoughts and conclusions on this body of work.
8.1 Review The aim of this dissertation was to begin to understand “how” a the popular collaborative approach to quality improvement actually functions from the perspective of the frontline staff using a provincial ICU collaborative as a sample. Using grounded theory qualitative research methods, I set out to explore the overarching question of how healthcare providers and management describe the experience of being involved in a collaborative network for critical care quality improvement. In the absence of any guiding theory specific to this area, three related concepts from other domains were used as a analytic framework for looking at an explanation of the process of collaborative quality improvement. These concepts were (a) why healthcare providers and managers feel it is important to be part of a QI collaborative; (b) how staff engagement and communication occur within the collaborative and (c) whether there actually collaboration occurring within the collaborative.
98
The sample collaborative was selected with a view to exploring the story behind its quantitative results in order to learn how to improve for the next initiative.
In order to present and
incorporate all of the information concerning the case study a process evaluation framework to the presentation of this set of research was applied. This included a detailed description of the sample collaborative design, a brief summary of the quantitative improvement results and the presentation of the empirical findings of in-depth qualitative research which is the focus of the dissertation and a true process evaluation. Using selected organizational and sociologic theories as starting points, 32 key informant interviews were analyzed revealing three major patterns which shed light on concepts of interest and the overarching research question. These findings then led to the development of a working conceptual model; the “multi-organizational cooperative quality improvement”.
The prevailing approach to the evaluation of quality improvement in healthcare tends to a focus on purely quantitative measurement – the observable are measured and reported (i.e. infection rates, timing of antibiotics, hand washing) with little regard for the impact of the unobservable, the social contexts, and variation in organizational behaviour. Ovretveit et al in 2002 identified four research questions about QI collaboratives that seem to follow a more critical theoretical paradigm including questions around whether improvements spread more quickly in collaborative programs, if the resulting improvements are larger in magnitude; if the results last longer and if the best practices are spread more widely (Ovretveit et al., 2002). Despite an ever growing literature on the results of various quality improvement collaboratives, the questions of “whether it is the collaborative that works”, along with questions of “how” these collaborative programs are perceived at the frontlines are unfortunately still largely unanswered in 2010. The findings and theoretical discussion presented in this thesis will help to move the field in the 99
direction of answering these types of foundational questions. This type of research is not easy but it is important if we are to improve the science of healthcare quality improvement
8.2 Implications of this Research Recently a follow-up IOM report (Institute of Medicine Board on Health Care Services, 2010) pointed out that quality & quality measurement efforts in healthcare have become disparate and disorganized, and that more coordination and cooperation is desperately needed. Moreover, all of these varied attempts have created duplicative, resource intensive and ever-changing requirements for individual hospitals. This thesis, based on a single sample collaborative study suggests a similar limitation of our current approach to QI collaboratives. The results of this inquiry are not meant to condemn multi-organizational QI programs or to warn organizations against participating, but rather to move the science of improvement forward such that we understand its actual processes and underlying assumptions and develop a reliable and sustainable methodology for its study. With further work and on-the-ground testing of models and relational hypothesis, we may be able to get to a place where we can eventually use mixed methods research and theory to better prepare organizations to maximize their quality improvement and co-operative activity.
Secondly, the socio-cultural findings reported in this thesis have the potential to make a contribution to policy making which would address what is an increasing problematic reliance on traditional, positivist assumptions of organizational change.
The need to effectively
operationalize quality improvement in healthcare has never been greater as regulators, funders, politicians and the public increasingly demand transparent and accountable mechanisms for ensuring safe and high-quality patient care and services. Only by examining all perspectives of
100
quality improvement, and most importantly that of frontline healthcare staff, can we understand its complex nature, the impact of that complexity on results and ultimately how true collective outcomes (i.e. system wide) might be generated.
Thirdly, the findings related to legitimization and communication can have implications for the field of QI research as a whole. These early concepts should be built upon such that some day we can have a solid pragmatic theory/model base for knowledge translation and quality improvement initiatives, including the collaborative/cooperative approach. Process Evaluation methodology as well as other theoretical models such as Normalization Process Theory (May & Finch, 2009) show great promise in providing guiding frameworks for maximizing mixed methods research in the evaluation of the implementation of complex interventions.
8.3 Limitations All empirical research despite being designed with the best intentions must admit to certain limitations. There are three important limitations to the work presented here. First and most obviously, this is a single case study of one collaborative in a specific clinical area. In this situation, the single case study approach provided the best opportunity to pursue the exploratory nature of this work, especially considering the lack of a pre-existing hypothesis or theory in any one particular direction. To study such complex social concepts in any larger a sample would have been unmanageable in these early stages. I think it is fair to say that this particular sample collaborative can be considered representative of an average topic-based quality improvement collaborative based on its design, methods and metrics and therefore is appropriate to be used to explore the initial concepts discussed in this dissertation. There was nothing unusual in its
101
development and expression that sets it apart from other similar collaboratives suggesting generalizeability, however external validation through explorations with other collaboratives will of course be required.
Secondly, I chose to focus my analytic discussion on three particular concepts. Arguably, the analysis of each could in fact be a full dissertation. However by beginning to explore some of these larger concepts and their relationships to each other here it allows the focus to be broad yet neither too narrow (i.e. looking at one in isolation) nor too extensive as to not provide any depth of interpretation. The realized goal was to provide insight into new perspectives on the measurement of effectiveness of the collaborative approach and may serve to fuel other future or ongoing research agendas.
And lastly, strict causality can of course not be inferred. It is possible that an entirely different set of constructs was responsible for the lack of large quantitative outcome gains in this particular collaborative program. However, a group of patterns in both inter and intraorganizational processes across sixteen units were revealed through this exploratory study that may explain the level of the engagement in the project as a whole and may contribute to an explanation of the resultant outcomes. Again, how these patterns play out in other collaboratives or even outside the ICU environment should be the subject of future research.
8.4 Future Research Directions The effectiveness of many quality improvement interventions, including the collaborative approach, has been studied for more than a decade now, and research suggests that most have
102
highly variable effects which depend heavily on the context in which they are used and the way they are implemented. It is time that our community be reflexive about what has been done in the past in order to be truly innovative in the future. Schutz and Weber (Schutz, 1967) define reflexivity as the ability to periodically suspend our natural attitude and notice the taken-forgranted ways in which our communities of knowledge are constructed and interpreted. This can open novel possibilities for changing them and suggests three important future research directions. Firstly, it means that the approach and results of quality improvement used in an organization probably matters less than how and by whom it is used. Further inquiry into the true “embeddedness” of quality improvement interventions, coherence, collective identity and action and other concepts of normalization may provide the basis for potential future research. There is tremendous potential for participatory action research in this area and it may well be that rigorous evaluation on a single organizational experience can provide useful information that complements other work through multi-organizational collaboratives and large randomized trials in terms of furthering our knowledge about the complexities of healthcare improvement in various settings. Secondly, the concepts explored in this thesis may be the tip of the iceberg and qualitative data such as this case study and others can and should be analyzed with regard to multiple other perspectives including focal actors, power, governance, discourse and others to unlock further influential findings. In particular the discourse of quality improvement, multi-organizational and otherwise, is a fascinating topic which highlights potential barriers to integration that may ultimately lead to mislabelled improvement failure. I think inter-disciplinary research with fields such as organizational behaviour, sociology and health services will be extremely useful for these types of questions and yield the most productive investigations.
103
And thirdly, I think we must also look into the cost effectiveness of various quality improvement approaches in combination with their outcome results and sociologic impact. The hidden costs of implementing a large scale, multi-organizational collaborative are high – not just direct financial cost but also indirect costs such as time, human resources and productivity (or loss thereof) are largely unknown. It is has often been assumed that QI should be implemented as part of a healthcare organizations mandate and leverage the investment of goodwill, but in fact there are real costs that if ignored may in fact be part of what jeopardizes successful implementation and sustainability.
8.5 Concluding Thoughts Along with the empirical qualitative interpretation presented in this dissertation, it is important to make a few “take home messages” about what can be learned from this research.
#1 – Healthcare QI collaboratives may work, just not the way they are thought to The purpose of this thesis was never to show whether or not the collaborative approach to quality improvement is effective or not but rather to explore the constructs which may help us better understand why they may or may not be as effective as possible. Using existing theory from other interorganizational relationships as a starting point, it has been possible to shine a light on how engagement, communication and collaboration are socially constructed in a way that can impact the effectiveness of a collaborative. Such discoveries may change the way we think about how collaboratives work and what areas we should leverage to increase effectiveness in the future.
104
#2 – Communication infrastructures and accountability for quality improvement within healthcare organizations are not clear and may silently be affecting improvement results in a negative way. The use of the term “silently” in this statement was quite purposeful. The impact of socially mediated constructs such as communication and accountability are only silent as long as we ignore them and allow them to remain unexamined. Giving such concepts voice through unique social research methods such as institutional ethnography and discourse analysis will enable the body of work in quality initiatives to move beyond the superficial study of effectiveness of complex quality improvement intervention.
#3 – We cannot assume that collaboration, by definition, is what healthcare organizations see as the main benefit of participating in a quality improvement collaborative. This is an interesting new perspective on processes of change and one that deserves much further empirical study in order to understand what is happening at the “lived border” and how those experiences can be leveraged to increase effectiveness and benefit for participants. The “lived border” refers to the actual locations within the world where those things, events and circumstances that people experience are meaningfully conveyed and described. A qualitative inquiry at the lived border assumes a distinctive vantage point from which to observe the relationship between how actors “in the world” both participate in and re-present their experiences (Gubrium & Holstein, 1997, p. 101). The work presented here is not saying that healthcare organizations shouldn’t work together to get better improvement results faster, but understanding how they participate and experience working together is what is the essential knowledge. The research presented here used an independent post-hoc qualitative inquiry to help understand and explain quantitative improvement results in a sample QI collaborative. This method has
105
been very enlightening, but going forward truly mixed methods research which integrates quantitative and qualitative evaluations from the beginning should be the standard for studying quality improvement initiatives in healthcare. This partnered approach will not only provide a much more informative summative evaluation but can also provide invaluable formative data which can inform implementation in situ and support maximum success of an intervention. In concluding this dissertation I draw inspiration from an elegant editorial by Dr. Kieran Walshe of the University of Manchester. Like Walshe, “I do not argue in any way that there is no place for quantitative research methods in testing the effectiveness of quality improvement interventions. I fully acknowledge the power of positivist experiments in quantifying effect of interventions, but I also assert the need for parallel research approaches to understanding their nature as complex social processes and the effects of that nature, and the greater explanatory power of a more contingent and contextually sensitive approach to research” (Walshe 2007, p. 2).
Our literature is filled with statements about constructs that claim to offer an explanation about what has occurred, e.g. “such and such regression model explains 30% of the variance in Y”. But the explanation lies in the story of the relationship between X and Y and the context in which they exist. Knowing that the relationship between X and Y in healthcare is mediated by complex social and organizational processes that one cannot directly observe or measure is not unique to quality improvement but one that needs much more concerted attention if we hope to move to more influential interventions and strategies.
In the case of collaborative quality
improvement, the explanation and understanding of these relationships, their associated assumptions and their correlation to improvement outcomes are the DNA of the seeds of future QI success.
106
References Adams, K.T. (2009) Rethinking Comparative Effectiveness Research. Interview with Donald Berwick. Biotechnol Healthc. 6(2); 35-36, 38 Alter, C., & Hage, J. (1993). Organizations working together. Newbury Park, CA: Sage Publications. Altman, D. E., Clancy, C., & Blendon, R. J. (2004). Improving patient safety - five years after the IOM report. New England Journal of Medicine, 351, 2041. Angus, D. C., Linde-Zwirble, W. T., Lidicker, J., Clermont, G., Carcillo, J., & Pinsky, M. R. (2001). Epidemiology of severe sepsis in the united states: Analysis of incidence, outcome, and associated costs of care. Critical Care Medicine, 29(7), 1303-1310. Ayers LR, Beyea SC, Godfrey MM, Harper DC, Nelson EC, Batalden PB. Quality Improvement Learning Collabroatives. (2005) Quality Management in Health Care, 14, 234-247.). Bacharach, S. B. (1989). Organizational theories: Some criteria for evaluation. Academy of Management Review, 14(4), 496. Baier, R. R., Gifford, D. R., Patry, G., Banks, S. M., Rochon, T., DeSilva, D., et al. (2004). Ameliorating pain in nursing homes: A collaborative quality-improvement project. Journal of the American Geriatrics Society, 52(12), 1988-1995. Baker, G. R., Norton, P. G., Flintoft, V., Blais, R., Brown, A., Cox, J., et al. (2004). The canadian adverse events study: The incidence of adverse events among hospital patients in canada. CMAJ : Canadian Medical Association Journal = Journal De l'Association Medicale Canadienne, 170(11), 1678-1686. Basch, C. E. (1985). Avoiding type III errors in health education program evaluations: A case study. Health Education Quarterly, 12(4), 315. Batalden PB, Stoltz PK. A framework for the continual improvement of health care: building and applying professional and improvement knowledge to test changes in daily work. Jt Comm J Qual Improv. 1993 Oct;19(10):424-47 Bate, S. P. (1995). Strategies for cultural change. Oxford, UK: Butterworth Heinemann. Bate, P., Robert, G., & Bevan, H. (2004). The next phase of healthcare improvement: What can we learn from social movements? Quality and Safety in Health Care, 13, 62. Bell, R., & Robinson, L. (2005). Final report of the ontario critical care steering committeeOntario Ministry of Health and Long-Term Care.
107
Benedetti, R., Flock, B., Pedersen, S., & Ahern, M. (2004). Improved clinical outcomes for feefor-service physician practices participating in a diabetes care collaborative. Joint Commission Journal on Quality and Safety, 30(4), 187-194. Brennan TA, Leape LL. Adverse events, negligence in hospitalized patients: Results from the Harvard Medical Practice Study. Perspect Healthc Risk Manage 1991; 11: 2-8. Berwick, D. M. (1989). Continuous improvement as an ideal in health care. The New England Journal of Medicine, 320(1), 53-56. Berwick DM, Leape LL. Reducing errors in medicine. Br Med J 1999; 319: 136-137 Berwick, D. M. (2003). Disseminating innovations in health care. Journal of the American Medical Association, 289(15), 1969. Blumenthal, D., & Kilo, C. M. (1998). A report card on continuous quality improvement. The Milbank Quarterly, 76(4), 625-48, 511. Boeije, H. (2002). A purposeful approach to the constant comparative method in the analysis of qualitative interviews. Quality and Quantity, 36, 391. Burt, R. (1992). Structural holes: The social structure of competition. Cambridge, MA: Harvard University Press. Callahan, K., & Kloby, K. (2007). Collaboration meets the performance measurement challenge. The Public Manager, 36(2), 9. Child, J., & Faulkner, D. (1998). Strategies of cooperation. London, UK: Oxford Press. Corbin, J. M., & Strauss, A. (2008). Basics of qualitative research: Techniques and procedures for developing grounded theory (3rd Rev Ed. ed.). UK: Sage Publications. Corning, P. (2003). Nature’s magic: Synergy in evolution and the fate of humankind. Cambridge, UK: Cambridge University Press. Cretin, S., Shortell, S. M., & Keeler, E. B. (2004). An evaluation of collaborative interventions to improve chronic illness care. framework and study design. Evaluation Review, 28(1), 28-51. Curtis, J. R., Cook, D. J., Wall, R. J., Angus, D. C., Bion, J., Kacmarek, R., et al. (2006). Intensive care unit quality improvement: A "how-to" guide for the interdisciplinary team. Critical Care Medicine, 34(1), 211-218. Deming, W. Edwards (1986). Out of the Crisis. MIT Center for Advanced Engineering Study. Denzin, N. K., & Lincoln, Y. S. (2000). Handbook of qualitative research (2nd Ed ed.) Sage Publications.
108
DiMaggio, P. J., & Powell, W. W. The iron cage revisited: Institutional isomorphism and collective rationality in organizational fields. American Sociological Review, 48, 147. Dobson, D., & Cook, T. J. (1980). Avoiding type III errors in program evaluation: Results from a field experiment. Evaluation and Program Planning, 3, 269. Donabedian, A. (1966). Evaluating the quality of medical care. Milbank Memorial Fund Quarterly, 44(1), 166. Donabedian, A. (1980). Explorations in quality assessment and monitoring. Ann Arbour, MI: Health Administration Press. Dyer, J. H. (2000). Collaborative advantage. winning through extended enterprise supplier networks. New York, NY: Oxford University Press. Eakin, J. (2010). Personal Communication. University of Toronto: Dalla Lana School of Public Health. Eccles, M. & The Improved Clinical Effectiveness through Behavioural Research Group (ICEBeRG). Designing theoretically-informed implementation interventions. Implement Sci. 2006 Feb 23;1:4. Eden, C., & Huxham, C. (2001). The negotiation of purpose in multi-organizational collaborative groups. Journal of Management Studies, 38(3), 351. Elliott, M. A. (2007). Stigmergic collaboration: A theoretical framework for mass collaboration. Unpublished PhD thesis, Centre for Ideas, Victorian College of the Arts, University of Melbourne Fennell, M. L., & Warnecke, R. B. (1988). The diffusion of medical innovations: An applied network analysis. New York, NY: Plenum Publishing. Foy, R., Eccles, M. P., Jamtvedt, G., Young, J., Grimshaw, J. M., & Baker, R. (2005). What do we know about how to do audit and feedback? pitfalls in applying evidence from a systematic review. BMC Health Services Research, 5, 50. Foy, R., Hawthorne, G., Gibb, I., Eccles, M. P., Steen, N., Hrisos, S., et al. (2007). A cluster randomised controlled trial of educational prompts in diabetes care: Study protocol. Implementation Science : IS, 2, 22. Frisby, W., Thibault, L., & Kikulis, L. M. (2004). The organizational dynamics of undermanaged partnerships in leisure service departments. leisure studies, 23(2), 109–126. Leisure Studies, 23(2), 109. Gabor A (1992) The Man Who Discovered Quality: How W. Edwards Deming Brought the Quality Revolution to America. Penguin. 109
Garland, A. (2005a). Improving the ICU: Part 1. Chest, 127(6), 2151-2164. Garland, A. (2005b). Improving the ICU: Part 2. Chest, 127(6), 2165-2179. Granovetter, M. (1985). Economic action and social structure: The problem of embeddedness. The American Journal of Sociology, 91(3), 481. Gray, B. (1989). Collaborating. finding a common ground for multiparty problems. San Francisco, CA: Jossey-Bass Publications. Gray, B., & Wood, D. J. (1991). Collaborative alliances: Moving from practice to theory. Journal of Applied Behavioral Science, 27(1), 3. Greenhalgh, T., Bate, P., & Kyriakidou, O. (2007). Diffusion of innovations in health service organizations: A systematic literature BMJ. Grilli, R., & Lomas, J. (1994). Evaluating the message: The relationship between compliance rate and the subject of a practice guideline. Medical Care, 32(3), 202-213. Grimshaw, J. M., Eccles, M., & Tetroe, J. (2004). Implementing clinical guidelines: Current evidence and future implications. The Journal of Continuing Education in the Health Professions, 24 Suppl 1, S31-7. Grimshaw, J. M., Eccles, M., Thomas, R., MacLennan, G., Ramsay, C., Fraser, C., et al. (2006). Toward evidence-based quality improvement. evidence (and its limitations) of the effectiveness of guideline dissemination and implementation strategies 1966-1998. Journal of General Internal Medicine, 21 Suppl 2, S14-20. Grimshaw, J. M., McAuley, L. M., Bero, L. A., Grilli, R., Oxman, A. D., Ramsay, C., et al. (2003). Systematic reviews of the effectiveness of quality improvement strategies and programmes. Quality & Safety in Health Care, 12(4), 298-303. Grimshaw, J. M., & Russell, I. T. (1993). Effect of clinical guidelines on medical practice: A systematic review of rigorous evaluations. Lancet, 342(8883), 1317-1322. Grimshaw, J. M., Shirran, L., Thomas, R., Mowatt, G., Fraser, C., Bero, L., et al. (2001). Changing provider behavior: An overview of systematic reviews of interventions. Medical Care, 39(8 Suppl 2), II2-45. Grimshaw, J. M., Thomas, R. E., MacLennan, G., Fraser, C., Ramsay, C. R., Vale, L., et al. (2004). Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technology Assessment (Winchester, England), 8(6), iii-iv, 1-72. Grimshaw, J. M., Zwarenstein, M., Tetroe, J. M., Godin, G., Graham, I. D., Lemyre, L., et al. (2007). Looking inside the black box: A theory-based process evaluation alongside a randomised controlled trial of printed educational materials (the ontario printed educational
110
message, OPEM) to improve referral and prescribing practices in primary care in ontario, canada. Implementation Science : IS, 2, 38. Groeger, J. S., Strosberg, M. A., Halpern, N. A., Raphaely, R. C., Kaye, W. E., Guntupalli, K. K., et al. (1992). Descriptive analysis of critical care units in the united states. Critical Care Medicine, 20(6), 846-863. Gubrium, J. F., & Holstein, J. A. (1997). The new language of qualitative method. New York, NY: Oxford University Press. Gulati, R. (1995). Social structure and alliance formation patterns: A longitudinal analysis. Administrative Science Quarterly, 40, 619. Gulati, R. (1998). Alliances and networks. Strategic Management Journal, 19(4), 293. Gulati, R., & Gagiulo, M. (1999). Where do interorganizational networks come from? American Journal of Sociology, 104(5), 1439. Gulati, R., Nohria, N., & Zaheer, A. (2000). Strategic networks. Strategic Management Journal, 21, 203. Guo, C., & Acar, M. (2005). Understanding collaboration among non-profit organizations: Combining resource dependency, institutional and network perspectives. Nonprofit and Voluntary Sector Quarterly, 34(3), 340. Halpern, N. A., Bettes, L., & Greenstein, R. (1994). Federal and nationwide intensive care units and healthcare costs: 1986-1992. Critical Care Medicine, 22(12), 2001-2007. Halpern, N. A., Pastores, S. M., & Greenstein, R. J. (2004). Critical care medicine in the united states 1985-2000: An analysis of bed numbers, use, and costs. Critical Care Medicine, 32(6), 1254-1259. Hamel, G. (1991). Competition for competence and inter-partner learning within international strategic alliances. Strategic Management Journal, 12, 83. Homer, C. J., Forbes, P., Horvitz, L., Peterson, L. E., Wypij, D., & Heinrich, P. (2005). Impact of a quality improvement program on care and outcomes for children with asthma. Archives of Pediatrics & Adolescent Medicine, 159(5), 464-469. Horbar, J. D., Carpenter, J. H., Buzas, J., Soll, R. F., Suresh, G., Bracken, M. B., et al. (2004). Collaborative quality improvement to promote evidence based surfactant for preterm infants: A cluster randomised trial. BMJ (Clinical Research Ed.), 329(7473), 1004. Horbar, J. D., Rogowski, J., Plsek, P. E., Delmore, P., Edwards, W. H., Hocker, J., et al. (2001). Collaborative quality improvement for neonatal intensive care. NIC/Q project investigators of the vermont oxford network. Pediatrics, 107(1), 14-22.
111
Howard, P. (2002). Network ethnography and the hypermedia organization: New media, new organization, new methods. New Media & Society, 44(4), 550. Hulscher, M. E., Laurant, M. G., & Grol, R. P. (2003). Process evaluation on quality improvement interventions. Quality & Safety in Health Care, 12(1), 40-46. Human, S. E., & Provan, K. G. (2000). Legitimacy building in the evolution of small-firm networks: A comparative study of success and demise. Admin Sci Quarterly, 45, 327. Huxham, C. (1993). Pursuing collaborative advantage. Journal of the Operational Research Society, 44(6), 599. Huxham, C., & MacDonald, D. (1992). Introducing collaborative advantage: Achieving interorganizational effectiveness through meta-strategy. Management Decision, 30(3) Huxham, C., & Vangen, S. (2000a). Ambiguity, complexity and dynamics in the membership of collaboration. Human Relations, 53(6), 771. Huxham, C., & Vangen, S. (2000b). Leadership in the shaping and implementation of collaborative agendas: How things happen in a (not quite) joined up world. Academy of Management Journal, 43(6), 1159. Huy, N. Q. (1999). Emotional capability, emotional intelligence and radical change. Academy of Management Review, 24(4), 325. Ilan, R., Fowler, R. A., Geerts, R., Pinto, R., Sibbald, W. J., & Martin, C. M. (2007). Knowledge translation in critical care: Factors associated with prescription of commonly recommended best practices for critically ill patients. Critical Care Medicine, 35(7), 1696-1702. Institute for Healthcare Improvement. (2003). The breakthrough series: IHI's collaborative model for achieving breakthrough improvement. Boston, MA: IHI Innovation White Paper. Institute of Medicine Board on Health Care Services. (2010). Future directions for the national healthcare quality and disparities reports Institute of Medicine Committee on Quality and Health Care in America. (2001). Crossing the quality chasm: A new health system for the 21st century. Washington, DC: National Academy Press. Johnson, P., & Duberley, J. (2000). Understanding management research: An introduction to epistemology. London, UK: Sage. Juran, Joseph M. (2004), Architect of Quality: The Autobiography of Dr. Joseph M. Juran (1 ed.), New York City: McGraw-Hill Kahn, J. M., & Fuchs, B. D. (2007). Identifying and implementing quality improvement measures in the intensive care unit. Current Opinion in Critical Care, 13(6), 709-713. 112
Kenney, C. (2008). The best practice: How the new quality movement is transforming medicine. New York, NY: PublicAffairs. Kilo, C. M. (1998). A framework for collaborative improvement: Lessons from the institute for healthcare improvement's breakthrough series. Quality Management in Health Care, 6(4), 113. Knight, L. (2002). Network learning: Exploring learning by interorganizational networks. Human Relations, 55(4), 427. Kohn, L. T., Corrigan, J. M., & Donaldson, M.S. (Institute of Medicine). (2000). To err is human: Building a safer health system. Washington, DC: National Academy Press. Kraatz, M. S. (1998). Learning by association? interorganizational networks and adaptation to environmental change. Academy of Management Journal, 41(6), 621. Landon, B. E., Wilson, I. B., McInnes, K., Landrum, M. B., Hirschhorn, L., Marsden, P. V., et al. (2004). Effects of a quality improvement collaborative on the outcome of care of patients with HIV infection: The EQHIV study. Annals of Internal Medicine, 140(11), 887-896. Larsson, R., Bengtsson, L., Henriksson, K., & Sparks, J. (1998). The interorganizational learning dilemma: Collective knowledge development in strategic alliances. Org Science, 9(3), 285. Lawrence, T. B. (1998). Examining resources in an occupational community: Reputation in canadian forensic accounting. Human Relations, 51, 1103. Leape, L. L., & Berwick, D. M. (2005). Five years after to err is human: What have we learned? JAMA : The Journal of the American Medical Association, 293(19), 2384-2390. Leatherman, S. (2002). Optimizing quality collaboratives. Quality & Safety in Health Care, 11(4), 307. Lorenzoni, G., & Lipparini, A. (1999). The leveraging of interfirm relationships as a distinctive organizational capability: A longitudinal study. Strategic Management Journal, 20, 317. Marsteller, J. A., Shortell, S. M., Lin, M., Mendel, P., Dell, E., Wang, S., et al. (2007). How do teams in quality improvement collaboratives interact? Joint Commission Journal on Quality and Patient Safety / Joint Commission Resources, 33(5), 267-276. May, C., & Finch, T. (2009). Implementing, embedding, and integrating practices: An outline of normalization process theory. Sociology, 43(3), 535. McGlynn, E. A., Asch, S. M., Adams, J., Keesey, J., Hicks, J., DeCristofaro, A., et al. (2003). The quality of health care delivered to adults in the united states. The New England Journal of Medicine, 348(26), 2635-2645.
113
McMillan, T. R., & Hyzy, R. C. (2007). Bringing quality improvement into the intensive care unit. Critical Care Medicine, 35(2 Suppl), S59-65. Merriam Webster Pocket Dictionary (2006). Merriam Webster; Springfield, MA Meyer, A. D., & Goes, J. B. (1988). Organizational assimilation of innovations: A multi-level contextual analysis. Academy of Management Review, 31, 897. Michie, S., Fixsen, D., Grimshaw, J. M., & Eccles, M. P. (2009). Specifying and reporting complex behaviour change interventions: The need for a scientific method. Implementation Science : IS, 4, 40. Mills, P. D., Weeks, W. B., & Surott-Kimberly, B. C. (2003). A multihospital safety improvement effort and the dissemination of new knowledge. Joint Commission Journal on Quality and Safety, 29(3), 124-133. Mittman, B. S. (2004). Creating the evidence base for quality improvement collaboratives. Annals of Internal Medicine, 140(11), 897-901. Morrison, T. (1996). Partnership and collaboration: Rhetoric and reality. Child Abuse & Neglect, 20(2), 127. Morrissey, J. P., Calloway, M., Bartko, W. T., Ridgley, S., Goldman, H., & Paulson, R. I. (1994). Local mental health authorities and service system change: Evidence from the robert wood johnson foundation program on chronic mental illness. Milbank Quarterly, 72(1), 49. Mosteller, F. (1948). A k-sample slippage test for an extreme population. The Annals of Mathematical Statistics, 19(1), 58. Murray, D. (1998). Design and analysis of group-randomized trials. New York, NY: Oxford University Press. Needham, D. M., Bronskill, S. E., Calinawan, J. R., Sibbald, W. J., Pronovost, P. J., & Laupacis, A. (2005). Projected incidence of mechanical ventilation in ontario to 2026: Preparing for the aging baby boomers. Critical Care Medicine, 33(3), 574-579. Nembhard, I. M. (2007). Organizational learning in health care: A multi-method study of quality improvement collaboratives. (PhD, Harvard University). Newton, P. J., Davidson, P. M., Halcomb, E. J., Denniss, A. R., & Westgarth, F. (2006). An introduction to the collaborative methodology and its potential use for the management of heart failure. The Journal of Cardiovascular Nursing, 21(3), 161-168. Nohria, N., & Eccles, R. (1992). Face to face: Making network organizations work. In N. Nohria, & R. G. Eccles (Eds.), Networks and organizations: Structure, form, and action (pp. 288308). Boston, MA: Harvard Business School Press.
114
Norris, C., Jacobs, P., Rapoport, J., & Hamilton, S. (1995). ICU and non-ICU cost per day. Canadian Journal of Anaesthesia = Journal Canadien d'Anesthesie, 42(3), 192-196. O'Looney, J. (1994). Modelling collaboration and social service integration: A single state's experience with developmental and non-developmental models. Administration in Social Work, 18(1), 61. Ontario Critical Care LHIN Leadership Table. (2007). Inventory of critical care services in ontario - an analysis of LHIN-level capacitiesOntario Ministry of Health and Long-Term Care. Ovretveit, J., Bate, P., Cleary, P., Cretin, S., Gustafson, D., McInnes, K., et al. (2002). Quality collaboratives: Lessons from research. Quality & Safety in Health Care, 11(4), 345-351. Oxford English Dictionary (1989). Simpson J. A., Weiner E. S. C. (Eds.), (2nd ed.). Oxford, UK: Oxford University Press. Panintz, T. (1998) A definition of collaborative vs. cooperative learning. Accessed May 2010, from http://www.londonmet.ac.uk/deliberations/collaborative-learning/panitz-paper.cfm Pentland, B. T. (1999). Building process theory with narrative: From description to explanation. Academy of Management Review, 24(4), 711. Pfeffer, J., & Salancik, G. R. (1978). The external control of organizations: A resource dependence perspective. New York, NY: Harper and Row. Pierce-Bulger, M., & Nighswander, T. (2001). Nutaqsiivik--an approach to reducing infant mortality using quality improvement principles. Quality Management in Health Care, 9(3), 40-46. Powell, W. W., Koput, K. W., & Smith-Doerr, L. (1996). Interorganizational collaboration and the locus of innovation: Networks of learning in biotechnology. Administrative Science Quarterly, 41(1), 116. Pronovost, P. J., Rinke, M. L., Emery, K., Dennison, C., Blackledge, C., & Berenholtz, S. M. (2004). Interventions to reduce mortality among patients treated in intensive care units. Journal of Critical Care, 19(3), 158-164. Provan, K. G., & Milward, H. B. (1995). A preliminary theory of inter-organizational network effectiveness: A comparative study of four community mental health systems. Administrative Science Quarterly, 40, 1. Provan, K. G., & Milward, H. B. (2001). Do networks really work? A framework for evaluating public-sector organizational networks. Public Administration Review, 61(4), 414. Provan, K. G., Nakama, L., Veazie, M. A., Teufel-Shone, N. I., & Huddleston, C. (2003). Building community capacity around chronic disease services through a collaborative 115
interorganizational network. Health Education & Behavior : The Official Publication of the Society for Public Health Education, 30(6), 646-662. Reerink, E. (1990). Improving the quality of hospital services in the netherlands. International Journal for Quality in Health Care, 2, 13. Reilly, T. (2001). Collaboration in action: An uncertain process. Administration in Social Work, 25(1), 53. Reitan, T. C. (1998). Theories of inter-organizational relations in the human services. Social Science Review, 72(3), 285. Rezmovic, E. L. (1982). Program implementation and evaluation results: A reexamination of type III error in a field experiment.”. Evaluation and Program Planning, 5, 111. Roberts, N. C., & Bradley, R. T. (1991). Stakeholder collaboration and innovation: A study of public policy initation at the state level. Journal of Applied Behavioral Science, 27(2), 209. Rogers, Everett M. (1983). Diffusion of Innovations. New York: Free Press. Rogowski, J. A., Horbar, J. D., Plsek, P. E., Baker, L. S., Deterding, J., Edwards, W. H., et al. (2001). Economic implications of neonatal intensive care unit collaborative quality improvement. Pediatrics, 107(1), 23-29. Rubenfeld, G. D., Caldwell, E., & Hudson, L. D. (2001). Publication of study results does not increase use of lung-protective strategy in patients with acute lung injury. American Journal of Respiratory and Critical Care Medicine, 163, A295. Rubenfeld, G. D., Caldwell, E., Peabody, E., Weaver, J., Martin, D. P., Neff, M., et al. (2005). Incidence and outcomes of acute lung injury. The New England Journal of Medicine, 353(16), 1685-1693. Rubenfeld, G. D., Cooper, C., Carter, G., Thompson, B. T., & Hudson, L. D. (2004). Barriers to providing lung-protective ventilation to patients with acute lung injury. Critical Care Medicine, 32(6), 1289-1293. Sandelowski, M. (1993). Rigor or rigor mortis: The problem of rigor in qualitative research revisited. ANS.Advances in Nursing Science, 16(2), 1-8. Schouten, L. M., Hulscher, M. E., van Everdingen, J. J., Huijsman, R., & Grol, R. P. (2008). Evidence for the impact of quality improvement collaboratives: Systematic review. BMJ (Clinical Research Ed.), 336(7659), 1491-1494. Schutz, A. (1967). The phenomenology of the social world. Chicago, IL: Northwestern University Press.
116
Scott, W. R. (1987). The adolescence of institutional theory. Administrative Science Quarterly, 32(4), 493. Selsky, J. W. (1991). Lessons in community development: An activist approach to stimulating inter-organizational collaboration. Journal of Applied Behavioral Science, 27(1), 91. Shojania KG, McDonald KM, Wachter RM, et al., eds. (2004), Closing the quality gap: a critical analysis of quality improvement strategies. Vol. 2; Technical Review 9 - Agency for Healthcare Research and Quality; 2004 Sep. AHRQ Publication No. 04-0051-2. Shortell, S. M., Bennett, C. L., & Byck, G. R. (1998). Assessing the impact of continuous quality improvement on clinical practice: What it will take to accelerate progress. Milbank Quarterly, 76(4), 593. Sinuff, T., Cook, D., Giacomini, M., Heyland, D., & Dodek, P. (2007). Facilitating clinician adherence to guidelines in the intensive care unit: A multicenter, qualitative study. Critical Care Medicine, 35(9), 2083-2089. A site dedicated to the life and works of dr. joseph M. juran. Accessed May, 2010, from www.jmjuran.com Society of Quality Control Engineers. (1967) "The QC Circle Phenomenon" in Industrial Quality Control. Buffalo, New York: Society of Quality Control Engineers. Steckler, A. e. a. (1992). Toward integrating qualitative and quantitative methods: An introduction. Health Education Quarterly, 19(1), 1. Suchman, M. C. (1995). Managing legitimacy: Strategic and institutional approaches. Academy of Management Review, 20, 571. The Canadian Patient Safety Institute. (2010). SaferHealthCare now! Retrieved May, 2010, from www.patientsafetyinstitute.ca/English/Initiatives/shn/Pages/default.aspx The Northern New England Cardiovascular Disease Study Group. (2009). The northern new england cardiovascular disease study group. Retrieved May, 2010, from www.nnecdsg.org The Royal College of Physicians and Surgeons of Canada. (1996). Specialty training requirements in critical care medicine The Vermont Oxford Network. (2010). The NIC/Q quality improvement collaboratives. Retrieved May, 2010, from www.vtoxford.org/quality/nicq/nicq.aspx The W. Edwards Deming Institute. The W. edwards deming institute. Retrieved May, 2010, from www.deming.org
117
Tsai, W. (2001). Knowledge transfer in intraorganizational networks: Effects of network position and absorptive capacity on business unit innovation and performance. Academy of Management Journal, 44(5), 996. United Kingdom National Health Service. (2010). NHS institute for innovation and improvement. Retrieved May, 2010, from www.institute.nhs.uk Wachter, R. M., & Pronovost, P. J. (2006). The 100,000 lives campaign: A scientific and policy review. Joint Commission Journal on Quality and Patient Safety / Joint Commission Resources, 32(11), 621-627. Walshe, K. (2007). Understanding what works--and why--in quality improvement: The need for theory-driven evaluation. International Journal for Quality in Health Care : Journal of the International Society for Quality in Health Care / ISQua, 19(2), 57-59. Walter, U. M., & Petr, C. G. (2000). A template for family-centred interagency collaboration. Families in Society: The Journal of Contemporary Social Services, 81(5), 494. Wasserman, S., & Galaskiewicz, J. (. (1994). Advances in social network analysis: Research from the social and behavioral sciences. Newbury Park, CA: Sage Publications. Waterman, H., Marshall, M., Noble, J., Davies, H., Walshe, K., Sheaff, R., et al. (2007). The role of action research in the investigation and diffusion of innovations in health care: The PRIDE project. Qualitative Health Research, 17(3), 373-381. Wilson, R. M., Runciman, W. B., Gibberd, R. W., Harrison, B. T., Newby, L., & Hamilton, J. D. (1995). The quality in australian health care study. The Medical Journal of Australia, 163(9), 458-471. Wilson, T., Berwick, D. M., & Cleary, P. D. (2003). What do collaborative improvement projects do? experience from seven countries. Joint Commission Journal on Quality and Safety, 29(2), 85-93. Winer, M., & Ray, K. (2000). Collaboration handbook: Creating, sustaining and enjoying the journey. St. Paul, MN: Amherst H. Wilder Foundation Publishing. Yin, R. (1989). Case study research: Design and methods. Beverly Hills, CA: Sage Publications. Zammuto, R. F. (1984). A comparison of multiple constituency models of organizational effectiveness. The Academy of Management Review, 9(4), 606. Zwarenstein, M., & Reeves, S. (2006). Knowledge translation and interprofessional collaboration: Where the rubber of evidence-based care hits the road of teamwork. The Journal of Continuing Education in the Health Professions, 26(1), 46-54.
118
Copyright Permissions Permission to use “Figure 1 – Napkin drawing of IHI Breakthrough Series concept” (p. 22) from Institute of Healthcare Improvement (Received via email August 6th 2010). From: Info Friday ‐ August 6, 2010 10:59 AM To: "Katie N. Dainty" Subject: RE: Request for Permission to use a Figure Dear Katie, Thanks very much for your patience. 1) Please use Figure title: Sketch of the IHI Breakthrough Series Model by Paul Batalden, MD (1994) 2) Please include credit line for Figure: Source: The Breakthrough Series: IHI’s Collaborative Model for Achieving Breakthrough Improvement. IHI Innovation Series white paper. Boston: Institute for Healthcare Improvement; 2003. (Available on www.IHI.org)
Please let us know if we can be of further assistance. Best, Matt Morse Customer Service & Systems Improvement Specialist Institute for Healthcare Improvement 20 University Rd., 7th Floor Cambridge, MA 02138 (617)‐301‐4914
[email protected]
119
APPENDIX A – Interview Guide I: Pre Interview Demographic Information 1. How long have you worked in Critical Care? 2. How long have you worked in this unit? 3. Were you involved in the ICUCBP project from the start? If not, when did you become involved? *
*
*
II: Interview 1. What is your roll in the ICU? 2. Can you describe your role in the project for me. How involved were you day to day? Prompt: o o o o o o
Central or leading role Complied with changes in practice Complied when possible or felt unable to comply Participated in educational aspects Was aware it was going on but only marginally involved Didn’t really know much about it
3. So overall, what was the experience of participating in the project like for you? Prompt: o o o o
Professionally/personally satisfying because … Burdensome to deal with additional expectations … Frustrating because of … Mixed because …
4. How were you first made aware that your unit would be participating in this project? Prompt: o o o o
Were you told face to face? Was information circulated in writing or by email? Had you been consulted in any way before this happened? How did you feel about your unit taking this project on?
5. Have you been asked to participate in other projects like this before or since? Prompt: o o o o
How do you feel about participating in (multiple) QI projects? Is it a positive experience for you? Does it feel like a burden? Do you see positive change happening in relation to these projects?
6. How was information related to the project generally shared/circulated? 120
Prompt: o o o o o o
Did you feel well-informed/in the loop? Was information provided in a way that was useful and meaningful to you? Was information provided in a timely manner? How often were you provided with information? Were there opportunities for you to feed back, discuss or seek information from your organizational leads/project directly? From your point of view what would be the best way for information about a project like this to be shared?
7. In terms of the information provided about how well your unit was doing at following best practices, what would be the best way of providing that? Prompt: o How often would you like to receive that information? o How should it be summarized and presented? o If not exposed to audit and feedback info/not aware: Would you have liked that information to be shared with you? Do you think it would have supported your efforts in any way? o If negative about audit and feedback: Tell me why you think audit and feedback wasn’t particularly useful for you/your unit.
8. As you went along were you aware of how your unit was doing compared with others? Prompt: o How did you find out about that? o Was this something you cared about? o Do you think this information had any impact on participation over time?
9. So overall, how much of a contribution do you think audit and feedback made to your participation in the project? In what ways was it most useful to you/your unit? 10. Your unit showed real improvement with DVT/VAP/CLI/SBT/PU/EEN. What do you think was behind that success? Institutional factors: o supportive culture o leadership o adequate resourcing Intervention: o Support elements o Education elements o Reminders o Audit and feedback
11. I understand that DVT/VAP/CLI/SBT/PU/EEN was more of a challenge for your unit. What do you think the main issues were there? Institutional factors: o Culture not receptive to change o Lack of leadership/engagement o Inadequate resourcing Intervention: o Support/education unsuited to this institution
121
o o o
Support/education unhelpful or inaccessible to someone in my position Support/education didn’t take day to day realities into account Audit/feedback burdensome or not shared meaningfully
12. In terms of the input coming from the project which elements were most valuable to you? What did you like about them? Prompt: o o o o
Support elements Education Reminders Audit and feedback
13. Which elements were least valuable to you? Why? Prompt: o o o o
Support elements Education Reminders Audit and feedback
14. Do you think that any of the elements supporting your success with ______ could be applied successfully to other best practices? 15. Do you think that some aspects of the support/education offered work better for some practices than other? In some contexts over others? 16. Is there anything the project could have done to support you better? 17. How important to you was it to be part of a network or a collaborative? 18. Is there anything else about your experience of participating in this project that we haven’t talked about yet that is important to you and you’d like us to be thinking about?
122
APPENDIX B – Open Coding Framework OPEN CODE LIST (KND & TS) • • •
Engagement Legitimacy Opportunity
•
Collaboration o Inter-unit o Intra-unit/team o No mention of it o Sharing
• • • • • • • •
Availability of Evidence Support of management Incorporation (of project) “standardization” Influence of others “empowering” Data collection Satisfaction
•
Workload o Human resources o Multiple projects o Role issues
•
Communications o Project coming o Information sharing during the project o Awareness o Audit and feedback
• • • • • • •
Multiple projects Size of hospital “best for patient” “benchmarking” “friendly competition” Data trust Proof of progress
•
Rewards o Internal within institution o External within collaboration 123
APPENDIX C - Summary of the ICU Collaborative Trial Results As described in Chapter 4, we conducted a cluster randomized trial to determine whether a collaborative best practice network used to deliver multifaceted KT interventions, including video-conference communication, various forms of education, reminders and audit and feedback, could improve the care provided to critically ill patients across geographically separated ICUs. This intervention led to only modest quantitative improvements in the rate of use of strategies to reduce ventilator associated pneumonia and the use of the sterile catheter insertion bundle to reduce catheter related bloodstream infections compared to baseline.
During the active intervention phase, we observed improvements in rates of compliance to semirecumbent positioning to prevent VAP [odds ratio 1.85 (95% CI 1.23,2.79) per month of active intervention] and in use of the sterile barrier precautions for central venous catheter insertion [odds ratio 3.06 (95% CI 2.09,4.49) per month of active intervention]. However, compared to control ICUs, the rate of improvement was only better in active ICUs for use of sterile barrier precautions during central venous catheter insertion (ratio of odds ratios 2.52 per month, 95% CI 1.55 to 4.10). Adherence to each of the other 4 evidence based practices was similar comparing active to control ICUs and showed no significant improvements over time. Overall, patients in ICUs receiving the active intervention were more likely to receive the targeted best practice than those in control ICUs (summary ratio of odds ratios 2.79, 95% CI 1.00, 7.74; p=0.05). However, our intervention had only modest impact on uptake and adoption of a few individual best practices, for example use of a bundle to prevent CRBSI, and no apparent impact on others, for example assessing risks of developing decubitus ulcers.
124
While we were able to see some significant results in two out of the six best practices, overall the quantitative outcomes were mediocre given the intensity of the effort put into the collaborative approach. These results combined with low attendance rates at group activities and heterogeneity of the ICUs approaches to the project provided evidence that the problems did not simply result from difficulty with KT and provider behaviour change. And so, a learning opportunity was born.
125
