Toward the Development of a Resilience Engineering ...

Proceedings of the Human Factors and Ergonomics Society Annual Meeting http://pro.sagepub.com/

Towards the Development of a Resilience Engineering Tool to Improve Patient Safety: The RETIPS Approach Sudeep Hegde, John Wreathall, A. Zach Hettinger, Rollin J. Fairbanks, Robert L. Wears and Ann M. Bisantz Proceedings of the Human Factors and Ergonomics Society Annual Meeting 2014 58: 803 DOI: 10.1177/1541931214581147 The online version of this article can be found at: http://pro.sagepub.com/content/58/1/803

Published by: http://www.sagepublications.com

On behalf of:

Human Factors and Ergonomics Society

Additional services and information for Proceedings of the Human Factors and Ergonomics Society Annual Meeting can be found at: Email Alerts: http://pro.sagepub.com/cgi/alerts Subscriptions: http://pro.sagepub.com/subscriptions Reprints: http://www.sagepub.com/journalsReprints.nav Permissions: http://www.sagepub.com/journalsPermissions.nav Citations: http://pro.sagepub.com/content/58/1/803.refs.html

>> Version of Record - Oct 17, 2014 What is This?

Downloaded from pro.sagepub.com at University at Buffalo Libraries on November 17, 2014

Proceedings of the Human Factors and Ergonomics Society 58th Annual Meeting - 2014

803

Towards the Development of a Resilience Engineering Tool to Improve Patient Safety: The RETIPS Approach Sudeep Hegde1, John Wreathall2, A. Zach Hettinger3,4,5, Rollin J. Fairbanks3,4,6 , Robert L. Wears7,8, Ann M. Bisantz1 1 Department of Industrial and Systems Engineering, University at Buffalo-SUNY, Buffalo, NY; 2John Wreathall & Co. Inc., Dublin, OH; 3National Center for Human Factors in Healthcare, MedStar Institute for Innovation, Washington DC; 4Department of Emergency Medicine, Georgetown University School of Medicine, Washington DC; 5Department of Emergency Medicine, MedStar Union Memorial Hospital, Baltimore, MD; 6Department of Emergency Medicine, MedStar Washington Hospital Center, Washington DC; 7Department of Emergency Medicine, University of Florida, Jacksonville, FL; 8Clinical Safety Research Unit, Imperial College London, Paddington, UK Over the last few years, patient safety research has seen a paradigm-shift marked by the advent of Resilience Engineering (RE). Findings from the team’s previous research on the efficacy of root-cause analysis in improving patient safety revealed the potential to analyze existing resilient system properties and leverage the same in system-design and improvement. A multi-phase research plan to develop a lessons-learned system for resilience engineering in healthcare is described. The focus of this paper is the first phase, which involved critical-incident interviews to elicit detailed information from frontline health care workers regarding real-life examples of resilience. 14 interviews were conducted with clinicians and nurses from a large, multi-hospital medical system. Multiple examples of resilience and factors pertinent to patient safety were extracted and aligned with system capabilities which are the cornerstones of resilience – learning, responding, anticipating and monitoring. Resilience was also seen to manifest at various levels of the work organization. These results demonstrate the feasibility of using the critical incident interviewing method to analyze resilience in a healthcare organization. The data from the interviews will further be used to develop a Resilience Engineering Tool to Improve Patient Safety (RETIPS) that can be implemented organization-wide for reporting and analysis of resilience-based cases.

Copyright 2014 Human Factors and Ergonomics Society. DOI 10.1177/1541931214581147

INTRODUCTION In the past few years, research on patient safety has seen a paradigm-shift with the emergence of Resilience Engineering (RE) (Hollnagel, 2011b; Hollnagel, Woods, & Leveson, 2006; Pariès, Hollnagel, Wreathall, & Woods, 2012). Traditional safety research has focused on system failures, ways to reduce error and how to deal with deviations from the planned direction of work. However, both failure and safety can be considered emergent features of a system (Hollnagel, 2011b). Performance varies as situations vary. These variations in performance can lead to failure – but paradoxically, can also, and often, lead to safe outcomes. Most of the focus of organizational learning and safety research is on failure, i.e. adverse events and near-misses. As a result, the potential to learn safety lessons during the majority of times when there are no negative outcomes is lost. A more recent approach has been to learn from the full range of outcomes including normal outcomes (when things go right), negative outcomes and everything in between. This approach has expanded the scope of research to understand what goes right in the work environment, i.e. the resilience of health systems, including the actions of front line providers, in preventing harm to the patient despite the inherent risks and complexity. Previous findings and Implications. Past research of this team evaluated the efficacy of root-cause analysis (RCA) solutions in improving patient safety (Hegde et al., 2013). The study, which analyzed semi-structured interviews of frontline providers at a large medical center, not only revealed the limited efficacy of RCA-based solutions, but also several instances wherein providers coped with design shortcomings, latent system errors and hazards by employing adaptations, workarounds and other frontline initiatives to ensure patient

safety. However, such instances of frontline providers’ role in achieving patient safety are usually not reported or formally acknowledged by an organization’s senior management or safety administrators. There is no formal bottom-up approach to bring positive lessons from everyday practice at the sharp end to bear on the safety policy-making process at the top-end, especially in the absence of incidents or sentinel events. Thus, the resilience of the system that makes safety a reality remains largely unacknowledged and under-studied. Findings from this initial study demonstrate the large, untapped potential of existing, resilient system properties which can be leveraged in system design and improvement. In general, resilience comprises a wide range of behaviors and system characteristics that may come into play during situations that are characterized by complexity, shortage of resources, time constraints, dynamism, ambiguity and threat to the safety of the patient. One way to promote a resilience-focused systems approach could be to provide a platform for sharing lessons contributing to patient safety. Lessons-learned Systems and the Power of Narratives. In several safety-critical industries including healthcare, lessons-learned systems have been in place that enable practitioners across the organization to report events that are relevant to safety. A leading example of a successful lessonslearned system is the Aviation Safety Reporting System (ASRS), established in 1976 (Billings, Lauber, Funkhouser, Lyman, & Huff, 1976). The success of the ASRS in enhancing safety standards in the aviation industry led to the recommendation by the Institute of Medicine (IOM) to adopt a similar system to improve patient safety (Kohn, Corrigan, & Donaldson, 1999). The ASRS is a combination of open-ended and structured questions with open space for narrative (NASA). Experts analyze the reported incidents and solutions

Downloaded from pro.sagepub.com at University at Buffalo Libraries on November 17, 2014

Proceedings of the Human Factors and Ergonomics Society 58th Annual Meeting - 2014

are recommended, which is pooled into a database of lessons learned. It is critical for the success of these systems to capture the narrative of the incident (both positive and negative) and then allow for encoding schemes so that not only may many patterns be discovered but also that the resulting narrative is descriptive, complete and precise (Holden & Karsh, 2007; Linde, 2001). Narratives provide rich contextual detail and the transfer of otherwise tacit knowledge, the information that is difficult to describe explicitly, but is a part of a practitioner’s knowledge structure (Linde, 2001; Polanyi, 1983). The perceived usefulness of such reporting systems is reinforced when reporters see evidence of their contribution to new safety interventions being developed. In a socio-technical system, individual and social knowledge can be demonstrated and learned without the need to formalize the same as propositions or rules (Linde, 2001). Therefore, in terms of the design of the reporting tool itself, one of the key factors that determines the format is the level of narrative detail it allows (Holden & Karsh, 2007). There are numerous examples of incident reporting systems in healthcare as well as other industries that have a semi-structured format that allows a narrative form, but also ensures that all information relevant in terms of safety is reported. Systems such as the Medical Event Reporting System include free-response fields of limited space (MERS). The Applied Strategies to Improve Patient Safety (ASIPS) (Westfall et al., 2004) form provides space for a minimal narrative about the event, with no prompts or cues to guide the response. However, they are mainly rooted in the traditional approach to safety, which is the reporting and analysis of nearmisses, close calls and adverse events. As a result, they may be afflicted with the limitations of the traditional approach to safety as discussed earlier. Nonetheless, they do provide evidence that implementing such event reporting systems is feasible within a health care setting. The findings from our previous study supporting the resilience approach, combined with the relevance of lessonslearned systems in healthcare reveal the potential to develop a similar system to capture the resilience characteristics of the organization. The idea behind such a system is to move beyond the search for latent errors and causes of failure, to uncover the capability of the system to overcome its shortcomings, risks, failures, and challenges to patient safety. This paper proposes a methodology to develop a Resilience Engineering Tool to Improve Patient Safety (RETIPS) to learn safety lessons from everyday practice in a large healthcare organization, and help build a comprehensive picture of work as it actually gets done. More specifically, such a system would help gather information about how practitioners adapt to handle complex and challenging situations, including:  Proactive measures taken at the operator-level to prevent hazards and mitigate risk of patient harm  Safety mechanisms that originate at the operator-level and evolve to become informal standards of practice  Role of frontline staff in creating resilience and factors that enhance this capability

    

804

Socio-technical relationships – both formal and informal that contribute to system resilience. Lesson-learning and –sharing mechanisms at the individual, unit and organizational levels Factors affecting performance variability Opportunities to improve resilience capability Other practices, observations, attributes etc. that might play a role in system resilience. RETIPS DEVELOPMENT APPROACH

Figure 1 shows the planned approach for developing such a voluntary reporting/information sharing tool, including the following phases:  Interviews of frontline practitioners in hospitals to collect an initial set of rigorous example cases  Thematic analysis of narratives and data from interviews to explicate elements relevant from a resilience and safety standpoint. The themes will then be used to form a draft version of the reporting tool  Iterative refinement of the tool by subject matter experts such as hospital safety professionals, clinicians, human factors researchers and resilience experts  Tool validation by implementing it in focus areas and evaluated by users and experts based on parameters of quality and usability  Final implementation of the tool in the hospitals and dissemination of lessons learned through an online database of responses and analysis thereupon.  A flow-map of the full methodology is shown in Fig.1. The focus of the remainder of this paper is the description of Phase 1 activity and a high-level analysis of resultant themes as support for the feasibility of this method. METHOD Interview Script Development. The interview script was primarily based on the Critical Incident Technique (CIT) (Flanagan, 1954), a robust procedure for gathering important facts concerning behavior in defined situations. Additionally, aspects of the Critical Decision Method (CDM) (Klein, Calderwood, & Macgregor, 1989) were adapted and included, such as questions about expertise, communication and decision making. The script was then adapted for resilience analysis based on a resilience-management framework proposed by Hollnagel called the Resilience Analysis Grid (RAG) (Hollnagel, 2011a). The RAG is intended as an initial framework to guide the development of more specific tools or grids’ to build a resilience profile of a system. The RAG consists of suggested probes or questions in each of four capabilities of resilience: (i) the ability to respond – knowing what to do, (ii) the ability to monitor – knowing what to look for, (iii) the ability to anticipate – knowing what to expect, and (iv) the ability to learn – knowing what has happened.

Downloaded from pro.sagepub.com at University at Buffalo Libraries on November 17, 2014

Proceedings of the Human Factors and Ergonomics Society 58th Annual Meeting - 2014

805

Figure 1: Overview of planned multi-phase research protocol to develop RETIPS The above adaptations resulted in questions to probe resilience within a structure prescribed by both CIT and CDM, i.e. to include four ‘sweeps’: The first sweep to identify a complex incident that has the potential to uncover cognitive and collaborative demands of the domain and the basis of resilience. The second sweep to develop a detailed incident timeline that shows the sequence of events. In particular, all communication events that took place were probed. The third sweep to examine key decision points more deeply by using a set of probe questions (e.g., "What were you noticing about the patient’s condition at that point?" "What was it about the situation that let you know what to do?" "What were your overriding concerns for the patient at that point?" “What went right for you in that situation?”). Finally, the fourth sweep to use "what-if" queries to elicit potential expert/novice differences (e.g., “Were there alternatives to the given course of action?” “Would this intervention have been easier if more resources were available?”). This version of the script was used to conduct pilot interviews of medical providers, some of whom were part of the research team. Based on responses to the pilot interviews, it was determined that resilience could be differentiated based on (i) actions based in one or more specific incidents (e.g. adapting clinical procedures to help a patient recover from a complication), and (ii) ‘everyday resilience’ actions that are incorporated as part of a routine procedure as a preventative measure or a workaround (e.g., reorganizing electric cables on the floor to eliminate a trip hazard in a patient room). In its final version, the script was further refined to distinguish between questions for each of these two classes of resilience. Participants. A total of 14 participants were interviewed, all employees of an academic health system, including seven attending physicians, five residents/fellows, and two nurses. There was broad representation across specialties, including emergency medicine, obstetrics, surgery, internal medicine, orthopedics, urology and critical care.

Interview Procedure. As a guideline to think of resilience examples, participants were asked to choose from one of four types of cases from their own experience that resulted in patient harm being averted: A – an instance in which they handled a challenging and complex patient situation; B – an instance in which they anticipated an unfavorable situation and took necessary steps; C – an instance in which they noticed a discrepancy during a process and took necessary steps; or D – a ‘good practice’ or preventative measure they might have incorporated as part of a routine procedure. Once they made their choice, the interviewer asked them to narrate the instance or practice in detail. The interviewer then repeated the case back to the participant to confirm their understanding of the case, and then began probing deeper as per the interview guide. The interviews were conducted over the phone at the participants’ convenience and were recorded for transcription and analysis. Analysis. A preliminary qualitative analysis was carried out by the first author based on initial readings of transcripts and notes written down during the interviews, in order to identify the types of resilience examples reported. In the first stage of the analysis, participants’ narratives and their responses to subsequent questions were reviewed and key examples of resilience or factors contributing to resilience were identified and extracted. In the second stage, these excerpts were further reviewed to determine their association(s) with the four capabilities of resilience, i.e. learning, monitoring, anticipating and responding. In the third stage of the analysis, a distinction was made between the various system-levels at which resilience originated or manifested, i.e. individual, group, unit and/or organizational levels. The second and third stages were completed iteratively.

Downloaded from pro.sagepub.com at University at Buffalo Libraries on November 17, 2014

Proceedings of the Human Factors and Ergonomics Society 58th Annual Meeting - 2014

RESULTS AND DISCUSSION From the preliminary analysis of the data, 28 instances were extracted that represented resilient actions or systems. It was seen that there were multiple sources and enablers of resilience for each case. Safe outcomes were often due to a combination of individual factors such as the provider’s skill, expertise and judgment as well as systems-level factors such as training, infrastructure and sound policies. In general, each case narrated by participants had elements of one or more resilience capabilities. As an example, an instance of good anticipation was followed by appropriate adaptation to do what was right in the given situation. Additionally, resilience was seen to be enabled by individual actions, group and unitlevel practices as well as organization-wide systems or protocols. Each of the capabilities, as defined in previous literature on RE (Hollnagel, 2011b), is discussed briefly with examples representing resilient systems below: Learning. From an RE perspective, learning has been defined as the ability to know what has happened or dealing with the factual. Coordination and communication systems are an important aspect of enabling resilience through learning and information sharing (Nyssen, 2011). In one hospital, a participant reported a relatively new protocol for a standardized handoff procedure. Prior to the new protocol, physician sign-outs were non-standardized and performed with significant variability. The newer protocol instituted a structured handoff and bedside rounding of every patient. Another participant from the same unit mentioned that the benefits of the new protocol are that a) the sign-overs tend to be more detail-oriented with a much lower chance of important details specific to each patient being missed; b) greater patient empowerment and satisfaction since the patients themselves have a chance to witness or even be involved in the communication; and c) it forces re-evaluation of each patient’s case by the two physicians and their teams. Monitoring is the ability to know what to look for or dealing with the critical. An emergency medicine physician explained the importance of monitoring security staffing levels in the emergency department (ED) given the high potential for erratic patient behavior. An interesting system being used in their unit is of mounting neon ticker-tape strips of different colors on a particular wall, where each color denotes security staffing levels in the unit. ED providers can tell from the color of the strip, whether the unit is under-, over- or at its capacity to respond to potentially violent situations. For instance, say, an orange strip would mean that the unit is close to its capacity to respond. A red strip would mean that all security personnel are engaged and in case of an anticipated violent situation, the providers would know to call for additional security staff from elsewhere in the hospital. The neon strips are managed by the charge nurse and located on the wall behind their desk where it is visible to everyone. Anticipating is the ability to know what to expect or dealing with the potential. One of the ways in which resilient systems demonstrate the ability to anticipate is to be able to

806

“recognize when to shift priorities across goal tradeoffs”(Woods, 2010). As an example of such a practice, reported by a physician in a surgical unit, each surgical team meets one week prior to surgery to cross-check the patient’s history of medical problems, medications, allergies etc. so that everyone is aware of the case details and clear on what needs to be done. This practice allows the group to anticipate any risks or issues that may arise either during or after the surgery for the given patient and take preventative measures accordingly. In addition to the group-level review, this participant mentioned that she makes it a point to do the crosscheck process for every patient herself a few days before surgery, as a double-check measure. In one instance, her individual practice of cross-checking led to her noticing that a patient’s white-blood cell count was very low. Since the surgery potentially involved infection, the surgery was cancelled and rescheduled after the patient’s cell count returned to normal. Responding is the ability to know what to do or dealing with the actual. A physician reported that at her hospital, it is mandatory for all security staff to undergo training in mediation skills in order to be able to handle patients showing hostility or a disposition towards violent behavior. While such formal training is not required to be taken by clinical providers, this physician underwent the training herself and said she now feels better equipped to respond to such patients. She recommended that all providers in the ED undergo the same training given the high frequency of such patients in the unit. A Systems-View of Resilience. As mentioned earlier in the analysis section, resilience was reflected at various levels. Resilience-systems at the macro-level are a result of senior management-driven interventions and policies. These were exemplified by formal protocols such as the formal sign-out protocol. Such formal systems however, are not free of caveats. For instance, even though formal sign-out process was considered by participants to be a much better way of doing sign-outs than was previously done, the participants also expressed that this new process was more time-consuming and needed to be streamlined to improve efficiency. Therefore, there is potential to improve the design and efficacy of macrolevel resilient systems using feedback of providers and other users in such a way that these systems align well with the way work actually is performed on the front lines. Resilience being enabled from the top-down, i.e. driven by the safety administration, has been called downward resilience (Woods, 2006). Resilience at the micro-level includes individual provider-driven practices such as cross-checking patient records before their surgery to anticipate potential risks. Often, providers evolve preventive measures or other resilient practices to mitigate a latent hazard. For instance, a highlyexperienced nurse makes sure to check with each patient all the medications they had been taking at home prior to the visit and compares those with current prescription to ensure there’s no discrepancy. However, the nurse also suggested that verification procedures like medication reconciliation should

Downloaded from pro.sagepub.com at University at Buffalo Libraries on November 17, 2014

Proceedings of the Human Factors and Ergonomics Society 58th Annual Meeting - 2014

be done at all levels in the chain of command, i.e. the physician and the pharmacist rather than just the nurse who happens to be the last safety barrier for the patient in the chain of care. Such micro-level practices illustrate the need for system-support to enhance resilience. Resilience evolving from the bottom-up, i.e. from the frontlines of care through higher levels in the organizational hierarchy, has been called upward resilience (Woods, 2006). Implications for RETIPS Development Since the current results are based only on a preliminary round of qualitative analysis, it is likely that in the next phase, a thematic analysis based on techniques like the Grounded Theory approach would yield a much more comprehensive understanding of system resilience and potential ways to leverage resilience better through systems-support. It may be argued that since the questions were focused on what went right, some latent errors in the system may have been undermined. However, as stated earlier, both success and failure result from performance variability. Therefore, appropriate analysis of responses to RETIPS would result in an understanding of the causes of success and failure in the system. Despite the current limitations, the results from the preliminary analysis of data demonstrate that the interview technique developed is feasible in terms of identifying factors that contribute to resilience as an emergent property of the system. The categories and features of resilience identified through the analysis of the interview data are congruent with the existing framework of Resilience Engineering. The data also reflects the challenges and constraints faced by frontline providers in ensuring patient safety due to lack of adequate systems-support. The next step in our research is to perform a full thematic analysis of the data from the interviews and identify key elements in the design of a tool that can be used by: (i) providers – to report instances of resilience; and (ii) safety administration – to systematically analyze emergent patterns and identify opportunities to improve resilience through system-support. CONCLUSIONS The proposed methodology for developing a systems approach to analyzing resilience in a healthcare organization is adapted from the CIT and CDM interview techniques, incident-reporting systems such as the ASRS, and Resilience Engineering principles. The results of the work presented here mark the conclusion of Phase 1 of the RETIPS-development plan. Preliminary analysis of data from the interviews revealed key categories and features of resilience across the system, including the role played by frontline providers in realizing patient safety, challenges and constraints to resilience and opportunities to improve systems-level support for resilience.

807

This stage of the research will be followed by a thematic analysis of the data, which will then be used to develop the Resilience Engineering Tool to Improve Patient Safety (RETIPS). REFERENCES Billings, C., Lauber, J., Funkhouser, H., Lyman, E., & Huff, E. (1976). NASA aviation safety reporting system. Flanagan, J. C. (1954). The critical incident technique. Psychological bulletin, 51(4), 327. Hegde, S., Hettinger, A. Z., Fairbanks, R. J., Wreathall, J., Lewis, V., Wears, R., & Bisantz, A. M. (2013). A Bottom-Up Approach to Understanding the Efficacy of Event-Analysis in Healthcare Paradigm Shift from Safety to Resilience Engineering. Paper presented at the Proceedings of the Human Factors and Ergonomics Society Annual Meeting. Holden, R. J., & Karsh, B. T. (2007). A review of medical error reporting system design considerations and a proposed cross-level systems research framework. Human Factors: The Journal of the Human Factors and Ergonomics Society, 49(2), 257-276. Hollnagel, E. (2011a). Epilogue: RAG: The resilience analysis grid. Resilience Engineering in Practice: A Guidebook, 275-296. Hollnagel, E. (2011b). Prologue: the scope of resilience engineering. Resilience Engineering in Practice: A Guidebook. Hollnagel, E., Woods, D. D., & Leveson, N. (2006). Resilience engineering: Concepts and precepts: Ashgate Pub Co. Klein, G. A., Calderwood, R., & Macgregor, D. (1989). Critical decision method for eliciting knowledge. Systems, Man and Cybernetics, IEEE Transactions on, 19(3), 462-472. Kohn, L., Corrigan, J., & Donaldson, M. (1999). To err is human: building a safer health system. Institute of Medicine: Washington, DC, 312. Linde, C. (2001). Narrative and social tacit knowledge. Journal of Knowledge Management, 5(2), 160-171. MERS. from http://www.mers-international.com/ NASA. Retrieved December 16th, 2012, from http://www.hq.nasa.gov/office/codeq/nsrs/index1.htm Nyssen, A. S. (2011). From myopic coordination to resilience in socio-technical systems. A case study in a hospital. Resilience Engineering in Practice: A Guidebook. Pariès, M. J., Hollnagel, E., Wreathall, M. J., & Woods, D. D. (2012). Resilience engineering in practice: A guidebook: Ashgate Publishing, Ltd. Polanyi, M. (1983). The tacit dimension. 1966. Gloucester, MA: Peter Smith. Westfall, J. M., Fernald, D. H., Staton, E. W., VanVorst, R., West, D., & Pace, W. D. (2004). Applied strategies for improving patient safety: a comprehensive process to improve care in rural and frontier communities. The Journal of Rural Health, 20(4), 355-362. Woods, D. D. (2006). Essential characteristics of resilience Resilience engineering: concepts and precepts, Aldershot: Ashgate (pp. 21-34). Woods, D. D. (2010). Resilience and the ability to anticipate. Resilience engineering in practice, 1-45.

Downloaded from pro.sagepub.com at University at Buffalo Libraries on November 17, 2014