Identification of leading indicators in healthcare

Identification of leading indicators in healthcare processes – developing a method

Ditte Caroline Raben Master of Science in Public Health

A thesis submitted for the degree of doctor of philosophy

Faculty of Health Sciences University of Southern Denmark

21st of July 2017

i

Declaration of Originality I hereby guarantee that this thesis does not include, without acknowledgement any material previously submitted for a degree or diploma at any university; and that to the best of my knowledge and belief it does not contain any material previously published or written by another person expect where due reference is made in the text.

Signed: 19.07.2017 __________________ on: _____________ Ditte Caroline Raben

ii

Supervisors Erik Hollnagel, Ph.D., Professor Centre for Quality, Region of Southern Denmark Middelfart, Denmark

Birgit Viskum, Consultant (Retired) Previous: Hospital of Southern Jutland Esbjerg, Denmark

Kim L. Mikkelsen, Ph.D. The Patient Compensation Association Copenhagen, Denmark

Assessment Committee Frans Boch Waldorff (Chairman), Ph.D., Professor Research Unit for General Practice, University of Southern Denmark Odense, Denmark Henning Boje Andersen, Ph.D., Professor (MSO) DTU Management Engineering Kgs. Lyngby, Denmark

Alastair Ross, Ph.D., Lecturer in Behavioural Science Dental School, University of Glasgow Glasgow, Scotland

Financial Support The University of Southern Denmark and the Region of Southern Denmark funded the study.

iii

Acknowledgements While the responsibility for the content of this thesis remains my own, I am indebted to the following people and institutions. First, this thesis would not have been possible without support from the Centre for Quality in the Region of Southern Denmark. A special thanks to Arne Poulstrup for welcoming me in the Centre, and to Christian von Plessen for letting me develop my personal and professional skills. I am very grateful to the Centre for funding the work and believing in my ability to take on the challenge and conduct the study. Second, I wish to thank the informants at Sygehus Lillebælt and Odense Universitets Hospital who allowed me to follow their work and ask them countless questions. Without their willingness to share their work with me and discuss their challenges and conditions, this work would not have been possible. My utmost gratitude goes to my main supervisor, Erik Hollnagel. You have truly opened my mind to a completely new way of thinking about science. One can never really know what questions and answers to expect from you. It is always a challenge and a learning experience to have a discussion with you. Thank you for always pushing me to do a little bit better, and always encouraging me to focus on the learning aspects of things. Thank you to my co-supervisor, Birgit Viskum. You have always taken a profound interest in my thesis and topic and tried to heighten my understanding of the concepts I have worked with in every way possible. Thank you to Kim L. Mikkelsen for stepping in midway through the project. You have heightened the quality of my work with your great comments and feedback. Further, thank you to Jeanette Hounsgaard for providing valuable feedback. I would also like to thank all my wonderful colleagues from the Centre for Quality. You have all participated in creating a wonderful, fun, warm and challenging environment for the writing of this thesis. Further, I have appreciated all the great discussions and learning experiences that I have been fortunate to participate in during this journey. This includes a special thank you to Jeffery Braithwaite, Mary Dixon-Woods, Mark-Alexander Sujan, Robert Wears and the rest of the members of Resilient Healthcare Net. A special thanks to Bettina Thude and Anne Mette Falstie-Jensen for being such calm and steady influences in our office. You are both truly inspirational regarding balancing life and work, and your

iv

example always motivates me to do my best. My biggest appreciation goes to my colleague and best office buddy anyone could ever imagine, Søren Bie Bogh. There is no doubt that an office without you will never be the same. You have filled my time here with so much laughter, fun, games, challenges, and serious and less serious discussions on every topic imaginable. I could not have completed this thesis without being able to share ups and downs along the way with you. Finally, this thesis would never have been possible without the support from my partner, soon-to-behubby and very best friend, Mads. I would never have been able to start or finish this without knowing that you would always have my back and take care of Agnes whenever I was overwhelmed or stressed on this journey. I cannot wait to start the next adventure with you by my side.

v

Thesis Papers Paper I Raben, D.C., Bogh, S.B. & Hollnagel, E. Suggesting an approach for developing leading indicators in healthcare - containing a review of the literature (Submitted) Paper II Raben, D.C., Viskum, B., Mikkelsen, K.L., Hounsgaard, J., Bogh, S.B. & Hollnagel, E. Application of a non-linear model to understand healthcare processes: using the functional resonance analysis method on a case study of the early detection of sepsis (Submitted - 3. Round of revision) Paper III Raben, D.C:, Viskum, B., Mikkelsen, K.L., Bogh, S.B. & Hollnagel, E. Learn from what goes right: a demonstration of a new systematic method for identification of leading indicators in healthcare (Submitted – 3. Round of revision) Paper IV Raben, D.C:, Viskum, B., Mikkelsen, K.L., Bogh, S.B. & Hollnagel, E. Proposing leading indicators for blood sampling - application of a method based on the principles of Resilient Health Care (Submitted)

vi

List of abbreviations: AVW: Acute Visitation Ward ER: Emergency Room FMEA: Failure Mode Effect Analysis FRAM: Functional Resonance Analysis Method GP: General Practitioner HMI: Human-Machine-Interface HSE: Health & Safety Executives HTA: Hierarchical Task Analysis LIIM: Leading Indicator Identification Method RCA: Root Cause Analysis RE: Resilient Engineering REWI: Resilience based Early Warning Indicators RHC: Resilient Health Care WAD: Work As Done WAI: Work As Imagined

vii

PhD Thesis—Leading indicators

Ditte Caroline Raben

Table of Content List of Tables ....................................................................................................................................... 4 List of Figures ...................................................................................................................................... 5 English summary.................................................................................................................................. 6 Dansk resumé ....................................................................................................................................... 8 Part I – Summary of studies ............................................................................................................... 10 Introduction ........................................................................................................................................ 10 Motivation ...................................................................................................................................... 10 Objectives and research questions .................................................................................................. 11 Scope .............................................................................................................................................. 12 Structure of the thesis ..................................................................................................................... 12 Background and theoretical approach ................................................................................................ 14 Safety and indicators in health care ............................................................................................ 14 Development of quality and safety in health care ...................................................................... 14 Resilient Health Care .................................................................................................................. 15 Safety-I and Safety-II ................................................................................................................. 17 Applying high-risk industry concepts in health care .................................................................. 20 Concept of indicators...................................................................................................................... 21 Leading v. lagging indicators ..................................................................................................... 23 Indicators in health care .............................................................................................................. 24 Characteristics of indicators and applied definition of leading indicators ................................. 26 Human performance based on Rasmussen ................................................................................. 28 Scientific standpoint ........................................................................................................................... 30 Presentation of studies ....................................................................................................................... 32 Study I: systematic literature review .............................................................................................. 32 Methods: study I ......................................................................................................................... 32

1



Results: study I ........................................................................................................................... 35 Concluding remarks: study I ....................................................................................................... 46 Study II: - applying the Functional Resonance Analysis Method to a complex process ............... 47 Methods: study II ........................................................................................................................ 47 Results: study II .......................................................................................................................... 53 Concluding remarks: study II ..................................................................................................... 62 Study III: developing a method for the identification of leading indicators .................................. 63 Methods: study III....................................................................................................................... 63 Results: study III ......................................................................................................................... 65 Concluding remarks: study III .................................................................................................... 68 Study IV: applying the systematic method to other FRAM models .............................................. 69 Methods: study IV ...................................................................................................................... 69 Results: study IV......................................................................................................................... 70 Concluding remarks: study IV .................................................................................................... 75 Discussion .......................................................................................................................................... 76 Summary of findings ...................................................................................................................... 76 Limitations of the study and consideration of a different method ................................................. 76 Reliability ................................................................................................................................... 77 Validity ....................................................................................................................................... 77 Transferability............................................................................................................................. 78 Comparison with existing literature on leading indicators ............................................................. 78 Contribution to the industry ........................................................................................................... 81 Conclusion ......................................................................................................................................... 84 Perspectives ........................................................................................................................................ 85 Suggestions for future research .......................................................................................................... 86 References .......................................................................................................................................... 89

2



Appendix 1 ......................................................................................................................................... 97 Part II – Thesis Papers ....................................................................................................................... 98 Paper I ................................................................................................................................................ 99 Paper II ............................................................................................................................................. 113 Appendix A .................................................................................................................................. 131 Appendix B................................................................................................................................... 133 Paper III............................................................................................................................................ 139 Paper IV ........................................................................................................................................... 161

3



List of Tables Table 1: Search terms for systematic literature review ...................................................................... 34 Table 2: Inclusion and exclusion criteria for systematic literature review ........................................ 34 Table 3: Key findings from literature review..................................................................................... 43 Table 4: Observations regarding early detection of sepsis ................................................................ 53 Table 5: Key findings of WAI of early detection of sepsis (Adopted from Paper II) ....................... 60 Table 6: Overview of common reasons for performance variability (Stanton et al., 2013, Hollnagel, 2012) .................................................................................................................................................. 64 Table 7: Potential measurable indicators for early detection of sepsis .............................................. 67 Table 8: Variability of functions in FRAM for blood samples .......................................................... 73

4



List of Figures Figure 1: The four abilities of RE (van der Vorm et al., 2011) ......................................................... 16 Figure 2: Safety-I according to Hollnagel (2008) .............................................................................. 17 Figure 3: Safety-II according to Hollnagel (2014b)........................................................................... 19 Figure 4: Equivalence of failures and successes ................................................................................ 31 Figure 5: Steps conducted to complete systematic review of the literature ....................................... 33 Figure 6: Flowchart summarising study selection ............................................................................. 36 Figure 7: FRAM function with aspects (Hollnagel, 2012) ................................................................ 49 Figure 8: Data collection strategy for Study II .................................................................................. 51 Figure 9: FRAM model (WAI) for 'early detection of sepsis' ........................................................... 54 Figure 10: FRAM model (WAD) of early detection of sepsis ........................................................... 56 Figure 11: FRAM model of blood sampling ...................................................................................... 71

5



English summary High-quality and safe health care services require the best conditions for the performance of everyday work processes. To manage these processes, it is necessary to have an informed and thorough understanding of them. This thesis explores the use and application of leading indicators for ensuring the management of successful health care processes. Leading indicators are found in many high-risk industries, where they serve as a basis for anticipating both unwanted and wanted events. Leading indicators are used to ensure that the necessary preconditions for successful performance are established and sustained. This thesis draws on knowledge from the use of leading indicators in high-risk industries and presents an approach for the identification of leading indicators in health care. The thesis comprises four connected studies. Study I systematically reviewed the literature on leading indicators, and especially the methods that have been used within high-risk industries. The review revealed that most approaches to identifying leading indicators began with mapping and modelling the process. The model was then analysed to identify appropriate leading indicators. The review showed that there was no consensus on how to identify leading indicators. Further, none of the studies presented the course of action in sufficient detail to allow replication, and the definition and understanding of leading indicators varied across different domains. Therefore, the recommendation of study I was to adopt the common traits from previous developed methods and focus on describing and explaining each step in detail, thereby making the method replicable. Study II developed a model using a case study of the early detection of sepsis in an Acute Visitation Ward in a Danish hospital. The aim of this study was to investigate whether this model could represent the process, with an emphasis on relations and connections between different tasks and functions. Additionally, the model was used to outline a method for identifying leading indicators. In study III, the results from study II were used to further develop and test a method for the identification of leading indicators using health care processes from another case study. The study concluded with a description of the Leading Indicator Identification Method (LIIM), which comprised six steps. Study IV tested the developed LIIM using the description of taking blood samples in a clinical biochemistry department in a Danish hospital. The study aimed to test the method in a different setting, and to potentially adjust and improve the six steps.

6



Combined, the four studies present and demonstrate a new approach to identifying leading indicators as a way of supporting high-quality and safe health care processes. The studies argue that the management of such processes can benefit from systematically examining the entire process and using that understanding as a basis for identifying the leading indicators that are needed to ensure successful outcomes.

7



Dansk resumé At skabe de bedste forudsætninger, for at gennemføre opgaver i sygehusvæsenet, er en vital del af at skabe et sygehusvæsen med fokus på kvalitet og sikkerhed. Men for at opnå denne høje sikkerhed og kvalitet i sundhedsprocesserne, er der behov for at skabe sig et informeret og grundigt overblik, over hvordan disse processer fungerer. Denne afhandling har forsøgt at udforske og anvende ledende indikatorer for at skabe de rette forudsætninger for at skabe, styre og sikre sådanne processer. Ledende indikatorer er ofte anvendt i høj-risiko brancher, hvor de fungerer som målinger, der bidrager til at forudse såvel utilsigtede og tilsigtede hændelser. Ledende indikatorer er nødvendige for effektivt at kunne styre en proces eller et forløb, og som basis for foregribende handlinger. Sådanne målinger bruges til kontinuerligt at sikre sig, at processer, har de nødvendige forudsætninger for at kunne fungerer optimalt og succesfuld. Denne afhandling anvender viden fra høj-risiko industrier og præsenterer en metode til at identificere ledende indikatorer indenfor sygehusvæsenet. Studie 1 inkluderede en systematisk litteratur gennemgang af litteraturen omkring ledende indikatorer, samt hvilke metoder har været anvendt til at identificere ledende indikatorer, særligt i høj-risiko brancher. Litteraturgennemgangen viste at de fleste påbegyndte identificeringen af ledende indikatorer ved at kortlægge den relevante proces. Efter dette indledende trin, blev den udviklede repræsentation af processen analyseret med henblik på at identificere indikatorer. Enten havde indikatorerne til formål, at forudsige utilsigtede hændelser eller at identificere faktorer forbundet med at opnå ønskede resultater. De følgende tre studier af afhandlingen fokuserede således på at forsøge at udvikle en metode til identificering af sådanne ledende indikatorer til sygehusvæsenet. Studie 2 indeholder en analyse af den tidlige opsporing af sepsis. Formålet med studiet var at undersøge, hvorvidt den anvendte metode, Functional Resonance Analysis Method (FRAM), kunne anvendes til at skabe en repræsentation af processen, og særligt afdække faktorer som relationer og forbindelser mellem de forskellige opgaver i processen. Ydermere, blev repræsentationen anvendt til at udvikle metoden til identificering af ledende indikatorer, der var omdrejningspunktet for studie 3. I studie 3, blev resultaterne fra studie 2 anvendt til at udvikle og teste, hvilke skridt var nødvendige i analysen af processen, og identificering af indikatorer. Dette arbejde blev afsluttet med, at metoden blev løsrevet fra casen omkring tidlig opsporing af sepsis, og navngivet LIIM (Leading Indicator Identification Method), der indeholder 6 skridt.

8



Studie 4 præsenterede, hvordan LIIM blev anvendt på en case omkring blodprøvetagninger på en klinisk biokemisk afdeling på et dansk sygehus. Formålet med dette studie var at teste om metoden også var anvendelig i en anden kontekst, samt eventuelt at udspecificere eller justere de seks trin. Kombineret præsenterer disse studier en ny og anderledes tilgang til forbedring af sygehusprocesser, med fokus på sikkerhed og kvalitet. Studierne viser, og konkluderer, at fremtidig styring af processer i sygehusvæsenet med fordel kan drage nytte af, at se på processer, ved at fokusere på ledende indikatorer, der er afgørende for ønskede resultater.

9



Part I – Summary of studies Introduction This chapter introduces the context and background of this thesis. It starts by presenting a short motivation for writing the thesis, followed by the objectives and aim of the thesis, and then the broader perspectives that this work has drawn upon. This presentation includes an introduction to working within quality and safety parameters in health care, followed by a description of the concept of indicators with a focus on the application of indicators in high-risk industries other than health care. The section on indicators concludes with the definition of indicators used in this thesis, as well as the epistemological standpoint. Motivation When I started researching the topic of indicators and safety in health care, my knowledge was limited. As I began studying the topic, I discovered that this field was characterised by strong opinions that played a significant role regarding which aspects of patient safety were primarily studied. The aim and topic of this thesis was based on curiosity regarding how safety is defined and understood in health care, especially in relation to the topic of indicators of safety. The topic of patient safety has received increasing attention, partly because of the growing number of adverse events and increasing political and public awareness. One way to improve safety and quality in high-risk industries is by applying indicators. In high-risk industries, indicators are commonly used to guide processes by highlighting aspects that are vital to achieving desired outcomes (Øien et al., 2011d). By considering these indicators when performing and planning processes, high-risk industries ensure that their processes always have the best preconditions for success (Hale, 2009). Based on this approach to controlling processes, this research proposed a similar method of understanding and developing indicators in health care. Hence, this research focused on identifying leading indicators in health care, and this objective was influenced by the way in which patient safety is usually understood (Vincent, 2008). This thesis does not focus on indicators as they are typically applied in health care—that is, as lagging measures that examine events that have already occurred and use what is learned to prevent future occurrences (Campbell et al., 2002). Instead, this thesis proposes the use of leading indicators, which are proactive. Leading indicators focus on what occurs prior to the unwanted event; we can either try to prevent it from happening, or we can investigate how the process can be remodelled to avoid the event in the future (Øien et al., 2011a). 10



Currently, there are no systematic guidelines for identifying leading indicators in health care (Reiman and Pietikäinen, 2014). To understand whether a system is safe and how it can stay safe, there is a need for methods that identify indicators of processes using a consistent and systematic approach. This thesis presents a method to meet this need. This thesis applies a new approach to patient safety that has been developed over the past decade and has gained increasing attention (Braithwaite et al., 2015). This approach argues that we should slow down efforts to constrain and regulate performance; instead, attention should be paid to how clinical care is supported for the number of intended outcomes to be as high as possible. This new approach has been given the name Safety-II, as opposed to the classical safety view, which is labelled Safety-I (Braithwaite et al., 2015). Safety-I is based on a ‘find and fix’ model. The emphasis is upon things that go wrong and hoping that this minimises the number of errors (Braithwaite et al., 2015, Hollnagel, 2014a). However, health care is much more complex than this, and fixing the cause of a problem or event does not necessarily mean that the event will not occur again—just that it may occur with a different root cause (Peerally et al., 2016). Instead, Safety-II examines how to enable things to go right more often by understanding and accepting that variations occur in any health care process (Hollnagel, 2014a). To improve the ability to manage complex processes and achieve a greater number of intended outcomes in health care, it is vital to understand how and under which conditions they function. Objectives and research questions The aim of this project was to develop a method for identifying leading indicators in health care based on the principles of Safety-II. Such a method will enable users to identify leading indicators in any given health care process. It includes: o a description of how to map a complex health care process o the finished model of a process, with clarification of the factors that contribute to the process progressing as intended o a set of guidelines for identifying leading indicators within this model that enable activities in the given process to proceed as intended or take necessary actions to improve conditions. Additionally, four research questions were answered during the studies conducted in this project. Each question was answered in a scientific article that has either been published or has been submitted to a scientific journal.

11





How are leading indicators defined, identified and selected in other industries with a history of leading indicators, and what can health care learn from the different approaches towards identifying leading indicators? (Study I) To answer this question, a systematic literature review was undertaken of leading indicators. The objective was to understand the topic and obtain knowledge of leading indicators.



Which methods are available for mapping a complex process? (Study II) The objective of this study was to explore and test the Functional Resonance Analysis Method (FRAM) to understand and map a selected case—that is, the early detection of sepsis.



How can leading indicators be identified within a given process? (Study II & III) The objective of this study was to develop a method of systematic steps to identify leading indicators to manage the investigated process.



Can these guidelines be applied to other processes related to patient safety? (Study IV) The objective of this study was to test whether the previously developed method could be applied to FRAM models representing other processes within the context of patient safety. If relevant and necessary, the method was modified and corrected.

Scope The scope of this work was to systemise and apply existing knowledge from other industries to develop a method that was applicable to health care. The thesis concentrates on method development using findings from a number of case studies. Attempts are made to investigate the generalisability of the method and possible limitations of the method. Validating, implementing and measuring the selected indicators are therefore beyond the scope of this work, but they may be considered in future studies. Structure of the thesis This thesis is divided into two main parts. Part I consists of four chapters, which constitute the overall framework for the thesis. Part II includes the four articles produced during the project period. In Part I, Chapter 1 presents the background and theory of the project. Chapter 2 presents the four studies. Each study is reported with methods and results, before continuing to the next study. Chapter 3 includes a discussion of methods, results and implications. Finally, Chapter 4 in Part I presents the conclusion of the thesis, perspectives and considerations of possible future studies. Part II of this thesis includes the articles produced during the research. An overview of the articles is presented below: 12


I.


Suggesting an approach for developing leading indicators in health care - containing a review of the literature

II.

Application of a non-linear model to understand health care processes: using the functional resonance analysis method on a case study of the early detection of sepsis

III.

Learn from what goes right: a demonstration of a new systematic method for identification of leading indicators in health care

IV.

Proposing leading indicators for blood sampling - application of a method based on the principles of Resilient Health Care

13



Background and theoretical approach Safety and indicators in health care To understand why this thesis presents a new and different approach to safety in health care, the first section introduces the current state of quality and safety in health care. Development of quality and safety in health care Medical error and patient harm was recognised as an undesirable side effect of the practice of medicine before the term ‘patient safety’ became a buzzword (Bagian, 2005). The phrase ‘First, do no harm’ can be traced back to Hippocrates (460–375 B.C.), who advised physicians to ‘abstain from harming and wronging any man’ (Wears et al., 2014, Vincent, 2010b). Since then, patient safety and health care quality has continually been championed. For example, during the nineteenth and twentieth centuries, Florence Nightingale, Ignaz Semmelweiss and Ernest Codman (Sharpe and Faden, 1998, Vincent, 2010b, Wears et al., 2014) were occupied with avoiding harm for patients, as well as understanding how harm occurred and how it could be minimised. This objective later became a central part in the creation of a high-quality health care system (Sharpe and Faden, 1998, Kohn et al., 2000). In the late 1980s, the field of patient safety started to evolve more rapidly, and data and evidence began gaining more attention. Early studies into patient safety included the Medical Insurance Feasibility Study and the Harvard Medical Practice Study, which were published in the late 1970s and early 1990s respectively (Mills, 1978, Leape et al., 1991). The latter study gained considerable attention because it used a number of strategies to highlight the severity of harm, including giving a specific number of deaths, implying that adverse events were not a reasonable price to pay for medical progress, and differentiating between negligence and error (Van Rite, 2011). Patient safety was placed on public and political agendas with the release of the United States Institute of Medicine’s report ‘To Err is Human’ (Kohn et al., 2000), which Vincent called a ‘stark, lucid and unarguable plea for action on patient safety’ (Kohn et al., 2000, Vincent, 2010a). In 2000, academic attention increased further when the British Medical Journal devoted an entire issue to the subject of medical error, thereby moving the field of research to mainstream academic and clinical enquiry (Leape and Berwick, 2000). The National Health Service (NHS) sparked a debate about how health care can learn from other high-risk industries on topics such as system thinking, culture, teamwork and reporting (Health, 2000).

14



Since the release of these reports, the field of patient safety has grown more rapidly and has become an acknowledged field of research. The publication of reports such as ‘To Err is Human’ and the UK equivalent ‘An Organisation with a Memory’ resulted in the establishment of the National Safety Agency in several countries, and scientific journals on patient safety emerged during this period. Today, patient safety is defined as ‘The avoidance, prevention and amelioration of adverse outcomes or injuries stemming from the process of health care’ (Vincent, 2008). Typical patient safety work addresses events related to errors or deviations that result in unwanted events or accidents (Cooper et al., 2000). Patient safety is related to ‘quality of care’, but it is typically viewed as a subset of quality, while quality also consists of activities that are not traditionally viewed as part of the patient safety agenda, such as accreditation (Cooper et al., 2000, Vincent, 2008). Resilient Health Care Recently, key thinkers in the field of patient safety have begun focusing on the shortcomings of current methods and approaches (Peerally et al., 2016, Card, 2016). Studies have shown that, despite the large amount of time and effort being put into reducing adverse events and iatrogenic effects, unwanted events still occur and patients still experience unacceptable levels of harm in health care (Rafter et al., 2015, Baines et al., 2013, Shojania and Thomas, 2013). There is still a need to consider how patients can be kept safe from harm during hospitalisation, as well as a desire to explore other approaches or perspectives to solve this challenge (Braithwaite et al., 2015). In 2011, Hollnagel, Braithwaite and Wears introduced Resilient Health Care (RHC), which presented a new distinction of safety, labelled Safety-I and Safety-II (Hollnagel, 2014a). RHC is defined as the ‘ability of the health care system to adjust its functioning prior to, during, or following changes and disturbances, so that it can sustain required performance under both expected and unexpected conditions’ (Hollnagel et al., 2013). RHC emerged from the field of Resilience Engineering (RE), which is described in an organisational context as the ability of an organisation to anticipate, prepare for, respond and adapt to incremental changes and sudden disruptions to survive and prosper (Woods, 2006). RE focuses on ways to enhance organisations’ ability to create processes that are robust yet flexible. An objective of RE is to foster the development of theories, methods and tools that proactively improve organisations’ ability to function effectively and safely (Herrera, 2012). Hollnagel (2011) proposes four essential aspects of a resilient system:

15


-


Knowing what to do by responding to both anticipated and unanticipated disruptions. This can be achieved either by implementing prepared sets of responses or by adjusting normal functioning that is due to disruptions. This addresses the system’s ability to handle events.

-

Knowing what to look for by monitoring things that are a threat or that may become a threat in the future. This monitoring should focus not only on aspects that occur within the system, but also on aspects in the environment that may affect the system. This addresses the system’s ability to handle critical events.

-

Knowing what to expect concerns the action of anticipating potential developments, threats and opportunities. This includes the ability to foresee potential changes, disruptions and pressures and considering the consequences they may have for the system. This addresses the system’s ability to handle potential events.

-

Knowing what has happened relates to learning from past experiences and using that knowledge to change the way the system performs or functions. It also focuses on learning the right lessons from the right experiences by focusing on the most important aspects. This final aspect addresses the system’s ability to handle factual events.

Figure 1: The four abilities of RE (van der Vorm et al., 2011)

RHC studies health care systems to understand how they work rather than focusing on how they fail (Hollnagel et al., 2013). This focus led to the development of two opposing, yet supplementary, views on safety (Hollnagel et al., 2013, Braithwaite et al., 2015). Safety-I and Safety-II attempt to challenge the classical and typically applied definition of safety in health care (Hollnagel, 2013).

16



Safety-I and Safety-II Safety is commonly defined as ‘reducing the risk of unnecessary harm to an acceptable minimum level’ (Vincent, 2010a). Another definition describes patient safety ‘as an attribute of health care systems that minimizes the incidence and impact of adverse events and maximizes recovery (Emanuel et al., 2008). Safety-I applies these definitions and acknowledges that it makes sense to focus on situations that go wrong because they are unexpected in nature and may lead to unintended and unwanted harm, loss of life or loss of property (Hollnagel, 2013). The ratio in health care for unintended events is estimated to be 1:10 (Carthey et al., 2001). However, the focus on unwanted events is understandable and is reinforced in many ways in the health care system. Authorities and regulations require adverse events to be measured using a variety of methods, including root cause analysis (RTA), case analysis, claims analysis and error reporting systems (Thomas and Petersen, 2003). Additionally, safety science offers many potential models for investigating failures and adverse events, including fault tree analysis, probabilistic risk assessment, failure mode and effects analysis (FMEA), and the Swiss cheese model (Hudson, 2003, Vincent, 2008). This focus unfortunately includes consequences in relation to competing for resources. Learning is limited because conclusions are drawn infrequently using only a small amount of the data and knowledge available (Hollnagel, 2013). Safety-I is defined based on this view; it aims to create a state in which the number of adverse outcomes is as low as possible. The philosophy of Safety-I is shown in Figure 2 and illustrates the binary view of work and activities.

Figure 2: Safety-I according to Hollnagel (2008)

17



Figure 2 shows that if everything works as it should, the outcome will be successful, and when something goes wrong, it will result in failure. A process that changes from normal to abnormal should be investigated to determine whether the change occurs abruptly or because of a drift into failure (Hollnagel, 2013, Dekker, 2012). The logic in Safety-I argues that adverse events can be avoided if the transition can be detected and blocked (Hollnagel, 2013). Hence, Safety-I focuses on finding failures and eliminating the cause or disabling the cause–effect link by inserting barriers. The traditional measure used to determine the effectiveness of this method is to count how many things go wrong after the intervention is performed. An unfortunate consequence of the focus on failures is the decreasing attention on elements and processes that function well and as intended. Most events do not go wrong, and much can be learned from understanding these positive events (Ball and Frerk, 2015). The learning and understanding from positive events is the opposite scenario of Safety-I; however, this perspective receives limited attention in health care. No authorities or regulations request this knowledge, and few methods or theories address the understanding of how human and organisational performance succeeds (Reason, 2008, Hollnagel, 2013). Safety-II argues that health care staff do not succeed in their everyday work because of rules, regulations, instructions and guidelines, but because they can adapt to their surroundings and the ever-changing conditions of the processes and systems they work within (Hollnagel et al., 2015, Ball and Frerk, 2015). Many factors in health care create uncertainty, intractability and complexity, and will contribute to things occasionally going wrong (Hollnagel et al., 2015). However, despite these factors, processes more often function well. Focusing on failure and error does not explain why people perform well most of the time—nor does it help to describe the characteristics of successful human performance (Hollnagel, 2014a). This perspective is the essence of Safety-II, which examines what goes right and attempts to understand the factors and mechanisms that contribute to successful or intended outcomes (Hollnagel, 2013). Safety-II encourages efforts to support improvisation and performance adjustments—for example, by helping staff to understand the resources or constraints of a situation and making it easier to anticipate the consequences of different actions (Hollnagel, 2013). According to Safety-II, performance variability is the essence of what makes processes go right in the complex system of health care. Therefore, efforts should be made to understand performance variability, how to dampen it when going in the wrong direction and how to amplify it when going in the right direction (Hollnagel, 2013). To do so, performance variability must be visible, understood, monitored and controlled—all central aspects of safety management in accordance with Safety-II (Hollnagel, 2013). As Safety-II proposes that 18



activities should occur the same way in the process, regardless of the outcome, it does not differentiate between mechanisms for things going wrong and things going right (Hollnagel, 2014a). This is illustrated in Figure 3.

Figure 3: Safety-II according to Hollnagel (2014b)

Figure 3 illustrates how performance variability is considered the cause for both acceptable and unacceptable outcomes. Successful performance variability can therefore be amplified in situations where variability has been contributing to successful outcomes. On the contrary, the variability should be dampened in situations where failures occur as a result of unwanted or unattractive performance variability. Safety management in this context is proactive so one can react before an event occurs (Hollnagel, 2014a). Reacting proactively requires an ability to anticipate what may occur in the future, as well as an understanding of how the system works and how changes and developments in an environment can affect different components of the system (Hollnagel, 2014a). There are different ways to proactively work with safety management—for example, by distinguishing between workas-imagined (WAI) and work-as-done (WAD). These terms also relate to the different perspectives of people working at the sharp and blunt ends. The sharp end refers to people who actualise the process and perform the task. These people are often referred to as working on the frontline of treating patients. The blunt end comprises people who are further away from the processes and actions. It can be described as the environment in which frontline care is planned by regulators, accreditors, administrators and designers. WAI is based on the perspective of people working at the blunt end and

19



the assumptions they make regarding the actions of workers at the sharp end (Medicine, 2010). One way to work proactively with this perspective is to align WAI with WAD. This will often result in the identification of previously unknown factors to be considered when planning work, such as how to behave more safely (Clay-Williams et al., 2015). Working proactively to achieve safe outcomes, and including the perspectives of workers at the blunt end, are core elements of this thesis. The studies within it examine the little-researched area of Safety-II and presents some of the challenges of working with subjects caught between the agendas of staff working at the blunt and sharp ends. Applying high-risk industry concepts in health care The application of concepts and methods developed in high-risk industries in health care is a common phenomenon. In particular, within the safety domain, most applied methods—for example, RCA, Plan-Do-Study-Act cycles, lean production and checklists—have been developed outside of health care (Hudson, 2003). Two main issues are identified when transferring high-risk industry concepts into health care, as presented below. Origin Health care is based on a set of complex and diverse activities. It embraces many highly specialised fields, including medicine, surgery, primary care, acute situations and administration of different elements (Woloshynowych et al., 2005). Although transferring concepts from other domains into a health care context is not a new idea, there is still scarce research on the challenges involved. This consideration has been stated in relation to RCA and its application in health care (Woloshynowych et al., 2005). Several publications note that the method has been applied insufficiently in health care, without thoroughly considering what makes it work in its context of origin and without adequately customising it to the specifics of health care (Peerally et al., 2016, Latino, 2015, Percarpio et al., 2008, Wu et al., 2008). Other researchers suggest that different high-risk industries have different ways of achieving a high level of safety and a high state of awareness of safety. Hudson (2003) argues that aviation and oil and gas have two different histories regarding safety. In aviation, performance has been achieved based on positive attitudes and encouragement towards safety initiatives (Hudson, 2003). In contrast, oil and gas have taken a more systematic approach to safety by implementing safety management systems and using senior managers as drivers towards higher safety states (Hudson, 2003). These examples address an important issue when applying methods to health care that previously achieved exemplary performance in other domains (Dixon-Woods et al., 2014). It is important, if not

20



crucial, to acknowledge how the method was applied in the original context (Vincent, 2008, Hudson, 2003). Searching for successful methods from other industries and directly transferring them to health care is not sufficient. The comparison of aviation and oil and gas highlighted the importance of considering why initiatives achieve positive results. To apply methods from either industry, understanding the different perspectives of safety could be a crucial factor in determining the success of the method elsewhere. Before transferring methods, the context and lessons learned in the original industries must be considered, and methods must be modified to fit the new context. Reactive v. proactive methods The second issue is the transfer of reactive and proactive methods. Reactive safety management typically starts as a response to an adverse event. Responses are often based on a desire to find the cause and develop appropriate solutions to avoid reoccurrence (Hollnagel, 2014a, Dekker, 2012). Health care is currently dominated by this reactive approach towards safety, which is managed through the application of methods such as incidence reporting and RCA (Barach and Small, 2000, Eagle et al., 1992). Despite a tendency to look towards high-risk industries and draw upon their extensive knowledge, the use of proactive methods is still modest within health care (Peerally et al., 2016, Dixon-Woods et al., 2014). Previous studies have discussed the benefits of proactive methods and noted a positive approach towards problems prior to accidents as a crucial factor for success (Chiozza and Ponzetti, 2009). Further, the proactive approach supports and strengthens one’s capability to self-organise in a safe manner and, if necessary, to adjust and redefine boundaries according to situational requirements (Reiman and Pietikäinen, 2014). An ability to adapt safety initiatives to the context is another advantage of the proactive approach, as the complex health care industry requires an approach that adapts to different settings and situations. Concept of indicators The first challenge that emerged in this study related to defining the meaning and purpose of indicators. Doing so in a health care context posed a further challenge. There are many variations of the term, which causes its meaning to occasionally be eliminated or forgotten in the applied context. This thesis attempts to define indicators and explain why they are needed in health care. An indicator is a pointer or instrument used to monitor the operation or condition of a system (de Vries, 2012). Indicators are needed to manage systems or processes in complex organisations (Øien et al., 2011a). To achieve the desired outcomes, there is a need to understand the process being investigated, as well as the interrelations affecting the process (Hollnagel, 2013). Depending on the

21



applied context, there are numerous ways to define indicators. The etymology of the word indicator is ‘one who points out’ as the term is related to the Latin indicare, which means ‘to point out’ (de Vries, 2012). Much of the research on indicators has been carried out in the interface between social and natural science (Øien et al., 2011b). Indicators are often used as markers in chemistry and biology within natural science. In social science, they are applied in several contexts, including economic and industrial. From an economic perspective, indicators are ‘statistical measures representing an economic variable’ (Manuele, 2009). Despite the many different applications, there is a consensus that indicators are a crucial part of a system, whether applied to understanding and tracking performance, measuring safety or foreseeing adverse events (Reiman and Pietikäinen, 2010). In this thesis, it was important to understand the concept of indicators and acknowledge the many different applications, as this could be crucial for understanding their success, as mentioned previously. Further, there was an opportunity to leverage the learnings from other industries, as highrisk industries have a history of using indicators to improve safety (Øien et al., 2011a). Although these industries often state the same purpose for developing indicators, their definitions of indicators differ. This inconsistency in the definition of indicators may be explained by the multidisciplinary field of safety (Øien et al., 2011a), in which each industry seems to apply its own modified definition of indicator. The literature further suggests that the concept of indicators must be carefully defined in regards to the given context whenever it is applied (Hopkins, 2009). Further, it is important to establish the purpose of indicators and describe their functions (Hopkins, 2009, Harms-Ringdahl, 2009). A broad definition of indicators by Øien (2011a) states that ‘an indicator is a measurable/operational variable that can be used to describe the condition of a broader phenomenon or aspect of reality’ (Øien et al., 2011a). This definition covers more than just safety, and it highlights the fact that an indicator needs to be both meaningful for the context it is applied in. There are several factors within a system, process or activity that may affect risk or safety levels, but that are not directly measurable or identifiable. Therefore, some definitions allow that an indicator may be a representation of an underlying concept. This is also apparent in Wreathall’s (2009) definition of indicators as ‘proxy measures for items identified as important in the underlying model of safety’. During the past 25 years, the field of safety science has focused on distinguishing different types of indicators from each other, often categorising them into leading and lagging indicators—a distinction that has become

22



increasingly discussed in this decade (Øien et al., 2011c). This distinction is explored further in the next section. Leading v. lagging indicators Leading indicators are used in financial and economic performance, where they are defined as ‘a measurable economic factor that changes before the economy starts to follow a particular trend or pattern’ (Manuele, 2013). In safety, the distinction has been applied during the past 25 years, but its use has been extensively discussed since 2009, when the distinction between leading and lagging indicators was questioned (Hopkins, 2009). However, most opinions are supportive of distinguishing between leading and lagging indicators, since they represent two different aspects of a process (Wreathall, 2009). Lagging indicators focus on the past performance of a system and the measurements of different elements that have already taken place (Dyreborg, 2009). Although the application of leading indicators is different from that of lagging indicators, a clear description and definition of leading indicators is still missing from the literature (Øien et al., 2011a). Some experts claim that appropriately developed leading indicators allow management to be proactive in managing precursor events such as accidents, incidents, near misses and undesirable safety states (Bergh et al., 2014, Edkins, 1998, Knijff et al., 2013a, Grabowski et al., 2007b, Broadribb et al., 2009, Institute, 2010). Other experts focus on leading indicators as indications of a high state of safety (Johnsen et al., 2012, Khan et al., 2010, Blair and O'Toole, 2010b, Herrera et al., 2010a, Grecco et al., 2012a, Øien et al., 2011a). The definition of leading indicator is not the only diverse in the literature; leading indicators are also referred to as ‘proactive’ and ‘process’ indicators (Øien et al., 2011a). However, in essence, many experts define leading indicators as conditions, events or measures that precede an undesirable event, and that have some value in predicting the arrival of the event (Øien et al., 2011b). Wreathall (2009) argued that the distinction between leading and lagging indicators is not necessarily the problem, but rather the interpretation of the causality of events. Others have argued the importance of understanding and highlighting the use of indicators, noting that this is often lacking in the literature (Harms-Ringdahl, 2009, Hopkins, 2009, Dyreborg, 2009). This thesis distinguishes between leading and lagging indicators because the distinction underpins the research method and is more relevant in the context of health care today (Mainz, 2003a). The term ‘leading indicator’ is used because this thesis attempts to develop a method that can help uncover some of the human, organisational and structural factors underlying and affecting health care work (Clay-Williams et al., 2015).

23



Indicators in health care Indicators are commonly used within health care and have a significant effect on organisations, quality assessment, political decisions and safety agendas (Mainz, 2003a). Based on a thorough review of health care literature, the application of indicators in this field is primarily based on lagging indicators (Mainz, 2003a). In general, the concept of indicators is not unknown in this field, but their use differs from that of other high-risk industries. Much of the work on developing, describing and presenting health care indicators in Denmark has been done in the context of the Danish National Indicator Project (Mainz, 2003b), which defines indicators ‘as measures that assess a particular health care process of outcome’ or as ‘quantitative measures for monitoring and evaluating quality’ (Mainz, 2003a). In health care, indicators are often used to provide a quantitative basis for clinicians, organisations and planners to achieve improvements in care. Their definition is based on standards of care, and it documents the quality of care (Mainz, 2003a, Mainz, 2003b). An example of indicators in patient safety is the Organisation for Economic Co-operation and Development’s (OECD) Quality Indicator Project, which aimed to develop indicators to be used internationally to assess the extent of patient safety issues (McLoughlin et al., 2006, Reiman and Pietikäinen, 2014). Selected indicators included post-operative complications, sentinel events, carerelated adverse events and a variety of disease-specific complications (McLoughlin et al., 2006). Similar attempts were made in the project ‘Safety Improvement for Patients in Europe’, which was funded by the European Union. In this context, the authors defined safety indicators as measures that assessed a particular health care process, structure or outcome to monitor, evaluate or improve quality of care, clinical support and organisational functions affecting safety of care (Kristensen et al., 2009, Kristensen et al., 2007). The work presented a method for selecting patient safety indicators, and it resulted in the development of 42 indicators, including factors such as culture, infections, surgical complications, medication errors, falls, obstetrics and specific diagnostic areas. Such indicators are often applied to diagnose the safety state, kick-start improvement work or compare countries, hospitals or units (Reiman and Pietikäinen, 2014). Despite the extensive application of indicators related to patient safety, the terms leading and lagging indicators are rarely applied in health care. It was only possible to find a reference to high-risk industries and the application of the term leading indicators in one study, which was presented in a book on patient safety culture and introduced patient safety indicators as tools for managing safety proactively (Reiman and Pietikäinen, 2014). The chapter by Reiman and Pietikäinen presented a

24



model of patient safety indicators using recent system safety theories and included examples of indicators that can be part of an organisation’s patient safety management (Reiman and Pietikäinen, 2014). The work introduced indicators to drive, monitor and measure outcomes of patient safety initiatives. The indicators were based on safety culture and were previously shown to be good indicators of organisational safety (Reiman and Pietikäinen, 2014). The authors examined safety from an organisational perspective and suggested applying their approach to create a safety management system in hospitals (Reiman and Pietikäinen, 2014). The indicators were created under organisational themes such as proactive safety development, hazard control, change and competence management, strategy management, work process and work condition management, safety leadership, supervisory support for safety and management of third parties (Reiman and Pietikäinen, 2014). The study distinguished itself from other leading indicator approaches because the indicators were not indicating specific unwanted events or precursors of harm. Rather, the indicators were an expression of how well the organisation was performing in terms of constructing a safety focus at the management level. Although the term ‘leading indicators’ is not usually used, the concept of looking to indicators as a precursor of future events or states has been applied previously in health care. An increasing number of studies have examined the effects of a safety culture and how it might affect the safety performance and state of a system in a positive way (Sammer et al., 2010, Flin, 2007). Sammer et al. (2010) focuses on measuring and understanding what safety culture includes and further investigates how factors such as leadership, teamwork, evidence base, communication, learning, a just culture and patientcentred care might affect safety. A further leading indicator approach in health care was presented by applying three steps to identify possible management changes to reduce the risk for harm in anaesthesia patients (Paté‐Cornell et al., 1997). First, the study assessed the contribution of different accident factors to the probability of anaesthesia accidents. Second, it investigated the effects of the state of the anaesthesia on the risk of death or brain damage. Third, the analysis resulted in the identification of possible management changes. A number of factors responsible for accidents were identified, including alertness and competence of the anaesthesiologist (Paté-Cornell, 2004, Paté‐Cornell et al., 1997). The study further identified management factors that affected the state of the anaesthesiologist. The study concluded that patient risk could be substantially reduced through closer supervision of residents, use of anaesthesia simulators and regular examinations for all anaesthesiologists.

25



Different types of early warning scores—sometimes termed rapid response systems—also draws on the concept of leading indicators, as they aim to create early recognition of patients whose clinical conditions are about to deteriorate (Subbe et al., 2001, Shearer et al., 2012, Schmidt and Wiil, 2015). The aim of the early warning score is to detect these patients before a serious adverse incident occurs (Subbe et al., 2001). Such scoring systems are believed to be relevant in the discussion of leading indicators in safety, where studies have shown one particular challenge related to implementation and effectiveness. When implementing such scores, systems rarely consider social and organisational factors (Shearer et al., 2012). One study showed that the lack of conforming to the rapid response system was the result of local sociocultural factors and intra-professional hierarchies. The study concluded that more attention needs to be paid to individual and bedside cultural issues to improve the effectiveness of these programs (Shearer et al., 2012). Further potential leading indicators of safety are the application of a safety management system with audits, the development of safety cases and the conduction of safety walk-arounds (Vincent et al., 2014). These approaches have been developed to gain a greater understanding of the factors that influence safety, either in a positive or negative direction. Further, we cannot exclude other examples of leading indicator concepts found in health care. However, as no consensus has been reached regarding the terminology, it is not possible to perform systematic searches in the literature, and some terms might be overlooked in a manual search. Nevertheless, leading researchers have stated that further development of the concept of leading indicators in health care is needed (Vincent et al., 2014). Characteristics of indicators and applied definition of leading indicators Several suggestions have been made regarding the characteristics of safety indicators. These characteristics can help guide the identification of indicators and assess quality and sustainability (Øien et al., 2011a, Herrera, 2012). As a result of the literature search, professional discussions and lessons learned during this study, the following important characteristics of leading indicators were noted: -

Meaningful: A vital characteristic widely agreed upon is meaningfulness (Øien et al., 2011c, Hollnagel et al., 2007). A good indicator is meaningful in relation to the process it measures or monitors, and it should guide future actions (Herrera, 2012).

26


-


Reliable: Indicators should be visible for different people, and examining the indicator should lead to the same or similar interpretations. Interpreting the indicator should provide knowledge about the process or system.

-

Sensitive: Indicators should be sensitive to changes. If changing circumstances affect the indicator, the changes should be visible.

-

Operational: The indicator should be a tool that can guide operational actions. It must support changes or modifications within the context.

Finally, many publications highlight that indicators should be measurable. If the indicator can be measured qualitatively and quantitatively, it must be possible to obtain measurements of the indicator (Vincent et al., 2014, Hopkins, 2009, Herrera, 2012, Harms-Ringdahl, 2009). However, measurability is not crucial in the studies performed for this thesis. First, it is important to be able to identify indicators that are meaningful for the process in which they are detected. They should be visible between different individuals helping to detect changes, and to guide in the direction of certain actions (Hopkins, 2009, Wreathall, 2011). If the indicators identified in this thesis meet these criteria, they may be made measurable at a later stage (Rasmussen, 1983). Searching for leading indicators in health care and patient safety requires awareness and clarification of definitions, characteristics, methods and their implications for the identification and selection of indicators (Øien et al., 2011c, Hale, 2009, Wreathall, 2011). In this thesis leading indicators are characterised as precursors of events, and if they are extracted they should contribute to a better understanding and management of processes in complex systems. As the literature shows, there are many ways to define leading indicators in high-risk industries; however, none are specific to health care. This thesis therefore suggests a new definition for leading indicators that is applicable to health care and is inspired by definitions used in high-risk industries. In the context of this thesis, two components must be included in the definition. First, this thesis focuses on the concept of Safety-II and the early detection and anticipation of intended and expected events (Harms-Ringdahl, 2009, Wreathall, 2011). Therefore, the definition of leading indicators must include the positive perspective, as the aim is to identify leading indicators for positive outcomes (Wreathall, 2011). Second, some definitions view indicators as quantitative measures that represent a technical or highly defined leading factor for unwanted events (Hinze et al., 2013, Paltrinieri et al., 2012, Grabowski et al., 2007b, Bennett and Foster, 2005). This thesis has a greater focus on human actions and behaviours, as well as organisational and structural factors, as 27



they play a major role in creating safe care in health care (Plsek and Greenhalgh, 2001, Shearer et al., 2012). Thus, human actions and organisational factors are important in defining the leading indicators used to guide the current and future behaviours and actions of health care staff. Thus, with the inclusion of these two important elements of leading indicators, the following definition was developed: Leading indicators are signs or symbols used as active monitoring to control current and future behaviours or actions to achieve desired safety outcomes. The terms ‘signs’ and ‘symbols’ were used by Rasmussen and are representations of how information or indications from our environment are perceived by human observers (Rasmussen, 1983). The concept is described in the next section. Human performance based on Rasmussen Jens Rasmussen was a pioneer in human behaviour. Among other things, he classified different types of processing information into three levels (Rasmussen, 1983, Embrey, 2005). The classification system is known as the skills, rule and knowledge-based approach (SRK), and it bases behaviour or actions on the familiarity of the task (Rasmussen, 1983). The development of the SRK approach was based on the assumption that humans are goal-oriented and seek solutions and relevant information to reach the desired outcome (Rasmussen, 1983). According to Rasmussen (1983), human activities in familiar environments are not goal-controlled, but oriented towards the goal and controlled by rules previously applied successfully (Rasmussen, 1983). This perspective is vital in understanding leading indicators, since the indicators control and guide the actions of individuals to achieve set goals. In health care, behaviour will typically be either rule- or knowledge-based. This means that rule-based performance draws upon empirical knowledge derived from previous occasions, explicit knowhow or conscious problem solving, whereas knowledge-based performance explicitly formulates a plan constructed from the understanding of the environment, components and predictions of the effect of the plan (Rasmussen, 1983). An important aspect of organising human performance into different categories is the role played by information from the surrounding environment (Rasmussen, 1983). For each type of performance, this information is different. Again, we focus on information received and perceived for rule- and knowledge-based behaviour. For rule-based behaviour, incoming information is defined as signs, because this information often activates previously performed actions in the individual. When

28



functional reasoning is required, human performance will be based on symbols (Rasmussen, 1983). Symbols consist of information linked to concepts and can be used to reason for different actions. The difference can be explained by using an example derived from health care. A patient’s heart rate, blood pressure or respiratory frequency are measured using numerical values. For the patient or relatives, the numbers derived from these measures can be perceived as signs. However, for the health care professionals treating the patient, these numbers will be perceived as symbols. They are given a meaning that will most likely cause the health care professionals to perform specific actions based on their previous experiences and knowledge of heart rate, blood pressure and respiratory frequency. Rasmussen’s framework has been used in this thesis for two reasons. First, it has previously been used as an effective tool for understanding human errors for adverse events in health care (Smits et al., 2010, Wang and Katz, 2007, Bracco et al., 2011). The framework can not only be used to explain failures, but also to provide an understanding of the leading signs or symbols used by health care professionals to make decisions and create desired outcomes. Second, the framework has been used to distinguish between different definitions of leading indicators during the Ph.D. project. Therefore, it has been included in the definition of leading indicators in the context of this thesis.

29



Scientific standpoint The aim of the studies summarised in this thesis was to develop a tool that can be used to understand the health care frontline. Therefore, it was important to consider how this work was affected by being developed by a health care professional without experience in working on the field. I found that it served as both an advantage and a disadvantage. Some perspectives might have been limited by not having hands-on knowledge of how things worked on an everyday basis. Therefore, it was important to occupy an observational role during this study. I tried to portray and map the investigated systems as they function, with an emphasis on the structural and organisational elements they included. Being an outsider allowed me to view situations in a more explorative way. In the words of a reviewer of this thesis, I analysed situations as ‘an interested and informed outsider, who was not burdened by the hidden assumptions of which insiders were unaware’ (Reviewer, 2017). Being an outsider enabled me to ask many questions, including those that might be characterised as being tacit knowledge for health professionals. I believe this helped me uncover many aspects that influenced the processes of care, some of which might not be obviously important for individuals with experience of working in the field. During this study, I also committed to a certain way of viewing and understanding the system and the safety of that system. Thus, the work in this thesis interprets safety based on two core principles (Hollnagel, 2012, Hollnagel, 2014a), which are summarised below. The first principle challenges the established understanding of causes for accidents. During the development of many methods and models for accident investigations, such as RCA, bow-tie model, FMEA, fault trees, and the hazard and operability study, it was believed that accidents could be traced back to underlying cause-and-effect chains (Hollnagel, 2012, Hollnagel et al., 2007). Therefore, investigating accidents consisted of searching for broken links in these cause–effect chains (Hollnagel, 2012). However, the studies in this thesis consider the equivalence of failures and successes because they are based on the same actions, and one cannot successfully determine whether actions are wrong before they have taken place and the result is known (Hollnagel, 2012, Hollnagel, 2014a, Hounsgaard, 2016). The principle is illustrated in Figure 4.

30



Figure 4: Equivalence of failures and successes

This understanding challenges the use of traditional methods, which attempt to investigate accidents or incidences through the understanding of linearity, as linear thinking is characterised by referring to a process following a chain of causal reasoning (Dekker, 2016). The second principle considers how humans within socio-technical systems are constantly challenged by limited resources such as time, materials, manpower and information (Hollnagel, 2012, Hollnagel, 2009). Additionally, large socio-technical systems are rarely described in detail (Lipsitz, 2012, Braithwaite et al., 2013, Plsek and Greenhalgh, 2001). Thus, people in such organisations are forced to match and adjust their performance and behaviour to surrounding conditions such as high pressure, different demands, work environment, expectations, constraints and working standards (Hollnagel, 2012). Therefore, adaptions are viewed as a way of coping with these many factors and are a necessary aspect of the system (Hollnagel, 2012, Hollnagel et al., 2015). Applying these principles to safety and the understanding of accidents gives rise to the involvement of non-classical methods to map the system.

31



Presentation of studies To develop something new in any field, there needs to be an understanding of what already exists. Therefore, this work began with a systematic literature review to uncover different ways to identify leading indicators. After the literature review, a common characteristic of applying system thinking became evident (Checkland, 1981). This led to the use of the FRAM to understand the system and identify leading indicators in health care processes. Study I: systematic literature review The first research question in this thesis asked: How are leading indicators defined, identified and selected in industries with a history of leading indicators, and what can health care learn from them? To answer this question, previous experiences with developing leading indicators were explored. Methods: study I A systematic literature review was conducted to explore different methods and understand the differences and similarities in approaches for developing leading indicators. Searches and assessments of databases showed that no systematic review previously attempted to sum up and describe the field of leading indicators. Few studies reported on different initiatives or applications, or considered theoretical foundations for safety indicators in general (Øien et al., 2011a, Øien et al., 2011c). However, these articles did not conduct a systematic search to include all methods, and they focused on several other aspects rather than the challenge of being able to identify leading indicators. The systematic review followed the steps presented in Figure 5, as inspired by (Brereton et al., 2007).

32


Phase 1 - Planing Review


Phase 3 Conduting Review

Phase 2 Selecting Papers

Specify research questions

Identify relevant research

Read included papers

Develop review protocol

1. review of all identified studies

Assess Study Quality

Validate review protocol

2. review of full text

Extract required data

Select included papers

Synthesis data

Phase 4 Documenting Review Write review report

Figure 5: Steps conducted to complete systematic review of the literature

Phase 1: Framing questions for the review (Khan et al., 2003). A series of questions were formulated to start the search for relevant work. The questions helped to find the literature that would answer the first research question. The following questions were asked: -

Which methods or models dominate the identification of indicators in high-risk industries?

-

Which differences and similarities characterise indicator development and use in these different sectors?

-

Which consequences do definitions and an understanding of indicators have on their use and development?

Phase 2: Identifying relevant work. At this point, the relevant search terms were identified along with the inclusion and exclusion criteria. A list of relevant databases was completed. In addition to databases containing scientific publications, the search included databases with conference papers and theses, as well as a search through relevant organisations. This was done to include literature that was not published scientifically, since a significant amount of knowledge on this topic is held in organisations that work with safety and leading indicators. The search terms, inclusion and exclusion criteria, and databases are listed in Tables 1 and 2. Searches were performed within Science Direct, Web of Science, Academic Search Premier, Scopus, EMbase and Engenta. Grey literature was found on dissertation abstracts, Worldcat and through hand search in relevant organisational websites.

33



Search word

Synonyms

Indicator

Measure

Precursor

-

Leading Indicators

Proactive Indicators

Safety Indicators

Early

Warning

Indicators Lagging Indicators

Reactive indicators

Incident Indicators

Outcome Indicators

Proactive

Prospective

Anticipation

Preparedness

Table 1: Search terms for systematic literature review

Inclusion criteria Studies

on

the

Exclusion criteria development

of

leading Languages other than Scandinavian or English.

indicators or early warning indicators within high-risk industries. Studies on the development of indicators, not In

high-risk

industries,

studies

not

necessarily defined as lead or lag indicators distinguishing between lead and lag indicators within health care.

will be excluded.

Published 1998–2013. In high-risk industries, this time was the beginning of the third age of safety. No definition of methods was necessary. Observational studies, method development, qualitative

studies,

cohort

studies

and

explorative studies should be included. Table 2: Inclusion and exclusion criteria for systematic literature review

Phase 3: Assessing study quality. At this point, some challenges arose in the process, since the included publications were different in nature and disposition, and applying established quality assessment criteria did not seem obvious, as this is typically based on studies including randomised controlled trials (Khan et al., 2003). The body of literature consisted of a variety of study designs and different solutions were considered to assess quality (Dixon-Woods et al., 2004). This review prioritised papers that appeared relevant in light of the outlined research questions, rather than focusing on study design or methodological standards (Dixon-Woods et al., 2006). The quality assessment was inspired by an article related to the overall subject of leading indicators (Øien et al., 2011b). The safety community within high-risk industries is multidisciplinary, and meanings are not shared across industries and authors. Therefore, the literature suggested carefully defining the concept

34



of indicators whenever the term is used, and establishing the purpose of the indicators and describing their functions (Hopkins, 2009, Øien et al., 2011c, Grote, 2009, Harms-Ringdahl, 2009). Based on these recommendations, it was decided to search for the following elements of the included article: (1) the definition of indicators, (2) the purpose of indicators and their primary functions, (3) details on which industries the approach was developed in, and (4) the use of a data collection method and theoretical framework or approach. This ensured that all relevant data were extracted from each paper, and it enabled comparisons of at least these elements between papers (Khan et al., 2003). Phase 4: Summarising evidence. The findings of the review were summarised in a table containing the abovementioned criteria. Papers were investigated with a focus on similarities and differences and the common applied steps of identifying leading indicators. Results: study I The results of the systematic review are presented below. In study I of this thesis, the results are discussed in relation to a health care context and what lessons can be applied in developing a method for leading indicators in health care. Descriptive and overall findings The review included 32 publications consisting of 27 articles, three reports and two theses. The publications were published between 1998 and 2014, with only one study published in the 1990s, 11 during the 2000s and 20 from 2010 until now. The study selection process is presented in Figure 6. The main findings of the literature review are summarised in Table 3.

35



Figure 6: Flowchart summarising study selection

Only a few articles presented the history of leading indicators, how they were previously applied in economics and other areas, and how these settings defined the term (Bennett and Foster, 2005); (Manuele, 2009). The main body of literature stated that a variety of accidents were the main concern that led to the development of leading indicators for safety in high-risk industries. The publications included references to the environmental accident in Seveso in 1976, the Chernobyl accident in 1986, the Piper Alpha oil platform accident in 1988, the Vapour Cloud Explosion at Buncefield in 2005, the Texas

36



City Refinery Explosion in 2005 and the Deepwater Horizon accident in 2010 (Mengolini and Debarberis, 2008a, Bergh et al., 2014, Paltrinieri et al., 2012, Øien et al., 2011b, Kongsvik et al., 2011). Before many of these accidents, safety performance was measured with ‘after-the-loss’ measures. However, afterwards, a growing consensus arose that these measures did not provide necessary insights to avoid future accidents (Grabowski et al., 2007a).

37



Reference

Industry

Definition of leading indicators

Purpose of indicators

Methods

Framework/approach

Grecco et al. 2014

Nuclear

Indicators that enable anticipation of performance evaluation are called leading indicators.

Leading indicators are used to assess safety culture.

Conversational action, systematic observations, documentary observations and questionnaire.

Fuzzy Set Theory approach to develop a method for safety culture measurement in organisation using leading safety performance indicators.

Edkins 1998

Aviation

Not defined—instead used the term proactive indicators which periodically monitor organisational latent failures that have appeared in catastrophic accidents.

Leading indicators should reflect the safety defenses or barriers put in place to protect from human and technical failures.

Focus-group discussions.

INDICATE programme—identifying needed defenses in the civil aviation transport environment based on six core safety activities.

Reiman & Pietikänen 2012

Not presented

The leading indicator identifies failings or ‘holes’ in vital aspects of the risk control system discovered during routine checks on the operation of a critical activity within the risk control system.

Article presents a number of errors enforcing factors targeted through the use of leading indicators.

Not presented.

Socio-technical system view on safety and safety performance indicators are organisational tools for the evaluation and improvement of the functioning of the socio-technical system used as part of the safety management process of the organisation.

Johnsen al. 2013

et

Oil & Gas

A proactive indicator indicates the performance of the key processes, culture and behaviour, or the working of protective barriers between hazards and harms, that are believed to control unwanted outcomes.

Proactive indicators are used for risk management and avoidance, as well as anticipation and mitigation of major accidents in the industry.

Interviews and discussions of questionnaires, investigation of accident reports.

High-reliability theory and theory of successful recoveries from RE to learn from both accidents and successes to avoid accidents and build resilience.

Bergh et al. 2014

Oil & Gas

Health and safety indicators should identify potential problems before an accident occurs, called leading indicators.

Leading indicators are used to manage psychosocial risk regarding managing operational risk resulting from human errors.

Not presented.

Psychosocial Risk Management Approach based on the principle of prevention in line with the control cycle, and it aims to reduce risk.

Knijff et al. 2013b

Chemical

Leading indicators for process safety offer a way of examining the contributing factors to accidents and

Leading indicators are mainly related to mechanical integrity, action

Interview of European Process Safety Centre on selection, development

Not presented.

38

PhD Thesis—Leading indicators Reference

Industry

Ditte Caroline Raben Definition of leading indicators


Methods

putting in place a system to monitor these factors in a tangible process.

item follow-up and training and competence.

and implementation leading indicators.

Framework/approach of

Khan et al. 2010

Chemical

Leading indicators are measures of process or inputs essential to deliver the desired safety outcomes.

The leading indicators are developed to control and monitor plant processes in a nuclear plant and categorised in three elements (1) operation integrity, (2) mechanical integrity, and (3) personnel integrity.

Not presented. Build upon UK HSE (Health & Safety Executives) guidelines on the development of process safety performance indicators.

A risk based approach for process safety performance indicators, which is built upon the UK HSE guidelines on the development if process safety performance indicators. Indicators are uniquely developed aggregated using a hierarchical aggregation scheme.

Blair O’Toole 2010a

Not presented

Leading indicators measure the actions, behaviours and processes (i.e., the things people do for safety) and not the failures caused by a lack of safety.

Not presented.

Case studies and analysis of qualitative data of activities.

Not presented.

Hinze et al. 2013

Construction

Leading indicators are measures that are not necessarily historical in nature, but rather can be used as predictors of future levels of safety performance.

Purpose of the leading indicators is to create a set of measures that describe the level of effectiveness of the safety process.

None presented.

Not presented.

Haight Thomas 2003

&

Chemical

Not presented.

Leading indicators should be used to optimise intervention strategies to decrease rates of injury and property damage and improve productivity.

Literature review.

Not presented.

Paltrinieri et al. 2012

Oil & Gas

Leading indicators are defined as a form of active monitoring on a few critical risk control systems to ensure their continued effectiveness.

Leading indicators monitor the system and help implement possible corrective actions.

In-depth analysis of accident scenarios. Not further described in article.

Resilience Base Early warning indicators (REWI) method and Dual Assurance method, which is a safetyperformance-based method aimed at establishing safety indicators to describe the safety level within an organisation, activity or work unit.

&

39



Reference

Industry



Methods

Framework/approach

Grabowski et al. 2010a

Marine transport

Not presented.

Not presented.

Value focused thinking sessions, safety culture surveys combined with investigation of safety performance.

Not presented.

Grabowski et al. 2007a

Tanker operations

Leading indicators, one type of accident precursors, are conditions, events or measures that precede an undesirable event and that have some value in predicting the arrival of the event, whether it is an accident, incident, near miss or undesirable safety state.

Leading indicators should improve safety in marine transportation.

Discussions with employees and safety climate, and safety performance surveys.

Six-step guide indicators.

Grabowski et al. 2007b

Virtual organisations

Leading indicators, one type of accident precursors, are conditions, events or measures that precede an undesirable event and that have some value in predicting the arrival of the event, whether it is an accident, incident, near miss or undesirable safety state.

Purpose of leading indicators is to measure safety and hence develop intervention strategies to avoid future accidents.

Expert elicitation sessions and discussion with senior staff.

Not presented.

Bennett & Foster 2005

Mining

Leading indicators are defined as a statistical time series that past experience has shown tends to reflect later changes and which thus can be used to forecast these changes because they precede the changes in a consistent manner and by relatively constant time interval.

Leading indicators should secure occupational safety and health management and enable control of the processes that lead to accidents, incidence etc.

Different presented.

Not presented.

Broadribb et al. 2009

Oil & Gas

Leading indicators provide early warning of major hazards.

Purpose of leading indicators is to monitor a number of crucial process safety performances.

Not presented.

40

approaches

for

developing

Three approaches are presented to develop Process Safety Performance Indicators being (1) proactive, (2) reactive, and (3) external learning.



Reference

Industry



Methods

Framework/approach

Sonnemans, et al. 2010

Chemical

Leading indicators are leadings metrics that are forward-looking.

Leading indicators should indicate safety risks and find the controlling latent conditions causing the trouble.

Case studies of three different sites extracting data from operational and tactical processes and analysed to find underlying problems.

Seven-stage protocol based on the concepts of precursors, control model, latent conditions and safety barriers.

Fearnley & Nair 2009

Not presented

Leading indicators are a form of active monitoring that determine that risk control systems are operating as intended.

Leading indicators should identify process safety and identify where there are weaknesses in the operating equipment to increase reliability.

Investigate incident reports step-by-step guide without further method description.

HSE step-by-step guide for chemical and major hazard industries to support improvement in process safety based on how safety and environmental performance measurement is related to major hazards associated with processes and site.

Mengolini & Debarberis 2008

Nuclear

Leading indicators are those on which the organisation can act to leverage achievement of the organisational goals monitored by the lagging indicators.

Indicators focus on operational performance and address organisational and safety culture aspects.

Not presented.

Systematic process-view approach to safety goals.

Moon & Hamilton 2012

Nuclear

Not presented.

Purpose of leading indicators is to monitor maturity and effectiveness of management activities.

Interviews observations employees.

Tomlinson et al. 2011

Shipping

Leading indicators are conditions, events and sequences that precede and lead up to accidents.

Purpose of leading indicators is to have value in predicting the arrival of future event whether accident, incident, near miss, or undesirable safety state.

Not presented.

41

with

and 300

Framework for Organizational Integrity, dividing the organisation into eight dimensions commonly associated with performance and safety failures and further dissects the dimensions into 16 factors to achieve a comprehensive model of the strengths and weaknesses of an organisation. ABS Safety Culture and Leading Indicators Model.



Reference

Industry



Methods

Framework/approach

Herrera et al. 2010b

Aviation/Oil & Gas

Leading indicators refer to current system status and their interpretation may be used to say something about future performance.

Leading indicators should provide an adequate understanding of the current state of the system and predict possible future events or consequences of changes.

Literature review, workshop, interviews and observations.

FRAM based on RE principles was used to identify leading indicators and Risk Influence Model (RIF) was used to identify lagging indicators.

Grecco et al. 2012b

Production

Indicators that provide information to be used to anticipate and improve organisational performance are called leading indicators.

Leading indicators should identify potential concern in the facilities performance based on central processes.

Conversations, systematic observations and document analysis.

Leading indicators were developed based on Ergonomic Work Analysis.

Toellner 2001

Not presented

Leading indicators are measurements linked to preventive actions.

Leading indicators selected should drive safety performance and help improve the overall safety performance.

Not presented.

Not presented.

Kongsvik et al. 2011

Oil & Gas

Safety climate is considered a leading indicator if it can be linked to later safety outcomes.

Leading indicators should monitor the safety climate, since this believes to give a glimpse into the culture or current safety state.

Survey.

Three-level theoretical framework of safety climate, including the member’s basic assumptions, their espoused values and social artifacts.

Øien et al. 2011a

Not presented

Leading indicators are a form of active monitoring used as inputs that are essential to achieve the desired safety outcome.

Leading indicators should help recognise signals to produce warnings and reduce risk of major accidents.

Not relevant.

Not presented.

Øien et al. 2011b

Not presented

Proactive indicators, leading indicators, assume measurements of underlying causes and contributing factors to accidental events.

Leading indicators should assume measurements of underlying causes and contributing factors to accidental events, such as training, supervision, etc., and thus provide early

Accident/Incidence analysis and identification of factors important to management.

Two approaches were pursued, being an incident/accident analysis-based approach and a resilience-based approach.

42

PhD Thesis—Leading indicators Reference

Industry

Ditte Caroline Raben Definition of leading indicators

Purpose of indicators warnings events.

of

Methods

Framework/approach

unwanted

Energy Institute 2010

Energy

Leading indicators are used to monitor the precursors to individual accidents.

Leading indicators should identify holes in processes or inputs essential to maintain critical aspects of the risk control system.

Not presented.

HSE Human Factors Framework – refers to environmental, organisational and job factors and human and individual characteristics, which influence behaviour at work which can affect health and safety.

Health & Safety Executives 2006

Chemical

Leading indicators provide information to be used to anticipate and develop organisational performance.

Leading indicators should be measures of processes or inputs essential to deliver the desired safety outcome.

Not presented.

Six steps method by HSE to setting performance indicators and implement a process safety management system.

American Bureau of Shipping 2012

Shipping

Leading indicators are a form of active monitoring focused on a few critical risk control systems to ensure their continued effectiveness.

Leading indicators should reveal areas of weakness in advance of adverse events, be associated with proactive activities that identify hazards and aid risk assessment and management.

Survey.

ABS Safety culture indicators model.

Manuele 2009

Not presented

Leading indicators are measures that engineers can use to predict how well a system or a project will perform before it is even finished.

Not presented.

Not relevant.

Not relevant.

Claude 2012

Construction

Leading indicators are measures of attitudes, behaviours, practices and conditions that influence construction safety performance.

Passive leading indicators are safety strategies implemented before construction phase and active leading indicators are safety related practices or observations measured during the construction phase.

Literature review and expert group selected leading indicators found in the literature review.

Not relevant.

Table 3: Key findings from literature review

43

and

leading



Definition and purpose of leading indicators The literature revealed many definitions of leading indicators and variations on the term, including proactive indicators, safety performance indicators and health and safety indicators. Only two of the publications in the study used the same definition of indicators, and they were published by the same author (Grabowski et al., 2007b). Some articles did not define or specify use of the term ‘leading indicator’, despite it being the main topic (Haight and Thomas, 2003, Moon and Hamilton, 2013). Definitions of leading indicators differed from each other in some respects. First, the specificity of the definition varied; some authors tended to define leading indicators only in relation to the study being described, resulting in definitions that were challenging to translate into other sectors, industries or processes. Reiman and Pietikänen (2012) stated that ‘The leading indicator identifies failings or “holes” in vital aspects of the risk control system discovered during routine checks in the operation of a critical activity within the risk control system’. Other definitions were shorter and less informative in relation to the specific process: ‘Leading indicators provide early warning of major hazards’ (Broadribb et al., 2009) and ‘Leading indicators are measurements linked to preventive actions’ (Toellner, 2001). Second, some definitions focused on accidents, hazards, failings, problems and undesirable outcomes. Few of the publications also considered an additional aspect of safety. These publications did not only screen for early warning symbols of unwanted events, but they also focused on the achievement of desired safety outcomes. Øien et al. (2011b) stated that ‘leading indicators are a form of active monitoring used as inputs that are essential to achieve the desired safety outcomes’. The variation in defining leading indicators is relatable to the concept of Safety-I and Safety-II, since Safety-II relates to safety as more than the absence of unwanted events, and it also focuses on achieved and desired safety outcomes (Hollnagel, 2014b). Third, the literature highlighted the importance of describing the purpose of the indicator and its function. This aspect was examined in each article. All but two of the included studies explicitly identified the purpose of the leading indicators (Blair and O'Toole, 2010a, Manuele, 2009). The publications often measured aspects believed to be associated with safety or risk, including safety culture and climate, psychosocial risk, mechanical integrity, operational integrity, personnel integrity and training supervision (Grecco et al., 2014, Kongsvik et al., 2011, Bergh et al., 2014, Knijff et al., 2013b, Khan et al., 2010, Øien et al., 2011d). Applied methods and approaches to identify leading indicators Details on which data collection methods were applied in the articles were inconsistent, and several publications did not mention applied research methods. Other publications briefly presented applied

44



methods but did not include details of the study population or size (Bergh et al., 2014, Hinze et al., 2013, Broadribb et al., 2009, Mengolini and Debarberis, 2008b). In general, very few of the studies enabled replication of findings. Most applied research methods were qualitative in nature, with a few studies using quantitative results through safety metrics, surveys and other previously registered data related to safety performance. Several articles used a mixed-method approach by combining interviews with observations or surveys. The qualitative approaches often applied were interviews and focus-group discussions conducted with both staff in operations and staff at the management level, as documented in nine studies (Edkins, 1998, Johnsen et al., 2013, Knijff et al., 2013b, Blair and O'Toole, 2010a, Grabowski et al., 2007b, Grabowski et al., 2007a, Moon and Hamilton, 2013, Herrera et al., 2010b, Baud, 2012). Further, five studies described in-depth analysis of accidents and incident reports (Johnsen et al., 2013, Paltrinieri et al., 2012, Fearnley and Nair, 2009, Øien et al., 2011d, Grecco et al., 2012b), and four studies used observations of operations to identify leading indicators (Grecco et al., 2014, Grecco et al., 2012a, Herrera et al., 2010a, Moon and Hamilton, 2013). Finally, two studies described the use of case studies investigating factors associated with safety (Blair and O'Toole, 2010a, Sonnemans et al., 2010). The lack of knowledge and details regarding research methods could affect the possibility of transferring methods into other contexts or industries. Further, the nature of the studies, which were often developmental or utilisation studies, posed challenges concerning the validity and reliability of the applied methods. Some publications presented a model or step-by-step guide for identifying leading indicators, whereas others based the study on an underlying theoretical foundation. Every publication used a different approach; thus, despite the similarities in some studies, it was difficult to compare the research findings. The concept of RE was part of the theoretical framework of four publications (Johnsen et al., 2013, Paltrinieri et al., 2012, Herrera and Hovden, 2008, Herrera et al., 2010a). The publications presented different uses of RE, mainly applying it as an underlying principle. A common feature of these publications was that they focused on identifying indicators with both an emphasis on accidents and a safety state, suggesting a more versatile approach. This approach complies with the principles of Safety-II. By examining aspects within an organisation that may cause accidents alongside aspects that secure a high safety state, this approach acknowledges that safety should be more than the absence of accidents (Hollnagel, 2014b).

45



Two additional concepts were replicated in several publications, namely the Health & Safety Executives guidelines (Fearnley and Nair, 2009, Health and Safety Executives, 2006) and the ‘American Bureau of Shipping’s Safety Culture and Leading Indicator Model’ (American Bureau of Shipping, 2012, Tomlinson et al., 2011). These approaches were developed for the chemical and shipping industries respectively, and they were only applied within those industries in the literature. Despite few similarities in the methods used, the large variety of frameworks and approaches underlined the fact that there was no standard for identifying leading indicators within or between industries. Concluding remarks: study I The literature review revealed a highly scattered field with different definitions of leading indicators and the application of different theoretical backgrounds and approaches. The use of leading indicators further lacked stringency, as some referred to leading indicators as measures that focused on measurability and others referred to them as conditions, events, current safety status or anticipation enablers. Similarities between different methods can be seen in the step-by-step process presented by several authors. This method of identifying leading indicators typically starts with a model of the system under investigation, followed by analysis of the activities in the system. Finally, the perspective of Safety-II and RE was also apparent in some approaches, which identified leading indicators based on positive factors or precursors for desired outcomes. These were the most valuable lessons learned from the literature review.

46



Study II: - applying the Functional Resonance Analysis Method to a complex process The literature review provided the researcher with an understanding and an argument to proceed with the next studies in the thesis. As established through the literature review, the identification of leading indicators depends on a thorough and detailed description and understanding of the process for which leading indicators are required. The second research question therefore asked: Which method can be applied for mapping a complex process? Different methods can be used to map complex processes. To stay true to the principles underlining this thesis, the method must be able to map processes in a non-linear way (Dekker, 2016, Hollnagel, 2012). Therefore, it was decided to apply the FRAM, since this method can assist in mapping complex processes with consideration of two important factors: the equivalence of failure and success, and the ability to illustrate aspects that are central for successful outcomes (Hollnagel, 2012). The FRAM is characterised as a method for modelling complex and dynamic socio-technical systems, and for providing an understanding of why things sometimes go wrong, but also, more importantly, why they usually succeed (Hollnagel, 2012). Methods: study II A case study was conducted to test the usefulness of the FRAM for this thesis. Qualitative case studies enable researchers to study complex phenomena within their natural context using a variety of data sources (Baxter and Jack, 2008). They are considered a valuable tool when developing new theories or interventions (Baxter and Jack, 2008). Thus, a case study seemed fitting to address the subject of developing a new framework for leading indicators in health care. In this thesis, the objective was to accomplish something more than understanding the particular case. This means that the actual case is of secondary interest because it played a supportive role in facilitating the construction of a method to identify leading indicators, which is the objective of study III (Baxter and Jack, 2008). In the terminology of qualitative case studies, an instrumental case study design is presented (Stake, 1995). The case was further characterised by being a multiple-case study, as the case was explored over several cases and across several units, helping to create a holistic and robust representation of reality (Baxter and Jack, 2008). During data collection for case studies, it is useful to apply a conceptual framework to ‘gather general constructs into intellectual bins’ and have a skeleton during data collection (Miles and Huberman, 1994). The skeleton was provided by the FRAM. The following sections present the data collection and the selected case.

47



Functional Resonance Analysis Method The FRAM evolved because of shortcomings of traditional safety models (Hollnagel et al., 2015, Hollnagel et al., 2007). The application of traditional models to map and analyse systems has affected the safety industry since the 1930s, helping to understand events and incidences as they develop in a step-by-step progression (Hollnagel, 2012, Hollnagel, 2014a). The FRAM instead adopts the thoughts of dynamic systems and parallel developments, assuming that things can be viewed as patterns of events (Hollnagel, 2012). Along with recent developments in safety understandings, presented through Safety-I and Safety-II, the FRAM has been shown to be useful in describing systems and their functions, and in understanding complexity, relations, connections and dynamics within sociotechnical systems (Herrera and Woltjer, 2010, Pickup et al., 2017, Clay-Williams et al., 2015, Laugaland et al., 2014, Herrera and Hovden, 2008). By focusing on these central and important features of processes, the FRAM can provide an understanding of why things usually succeed (Hollnagel, 2012, Braithwaite et al., 2015). The FRAM was introduced by Hollnagel (2006) and is applicable for both the accident investigation and risk assessment of a given process. The FRAM has three main purposes: (1) retrospective analysis, (2) prospective analysis and, (3) the basis for system design or redesign (Hollnagel, 2016b). When using the FRAM for risk assessment, the emphasis is on developing a description of what occurs on an everyday basis, and on understanding how performance variability can affect it either negatively or positively (Hollnagel et al., 2015). Further, the FRAM can illustrate differences between WAI and WAD, as mentioned in relation to Safety-I and Safety-II. The FRAM models of WAI are based on information from management, administrative staff and instructions or guidelines on the process. During the process of understanding the system based on experiences from studying and watching the system, a model of WAD can be processed to complement the model of WAI. This model will typically have a larger focus on organisational and structural elements, which are less often considered by workers at the blunt end of the system (Hollnagel, 2016a, Clay-Williams et al., 2015). The method is based on four steps, which are performed to map the system or process under investigation (Hollnagel, 2012). 1) Before starting the analysis of the model, the initial step is to define the purpose of the investigation and the process the FRAM is supposed to map. When this initial step is completed, the first step of the method can begin. During the identification of functions, it is

48



important to keep in mind the process under investigation. The focus should only be on the functions that are necessary for completing the process, and the functions that help achieve the desired outcome. 2) When the process is defined, all relevant functions need to be identified. Every function under investigation should include considerations on six underlying aspects. These aspects are central when investigating variability within functions, and they include Input, Output, Preconditions, Resources, Time and Control (Hollnagel, 2012). The input (I) aspect represents the part that starts the function, and output (O) is the result of the function and may vary based on the other four aspects in the FRAM. Many functions depend on preconditions (P) that must be carried out before successful output can be achieved. Resources (R) relate to things that are consumed or needed when carrying out a function (Hollnagel, 2012). Control (C) represents an important aspect in a function, and especially within a health care context, a large number of rules or regulations are connected to treatments. Time (T) represents the various ways in which time can influence a function (O) and cause variability. Figure 7 shows the FRAM hexagon and the six aspects.

Figure 7: FRAM function with aspects (Hollnagel, 2012)

3) Once all the functions within the process have been identified and described, each function will be investigated for possible variability. The investigation of performance variability can create a better understanding of the process and how the coupling of variability can lead to

49



unexpected outcomes. One way of describing variability is through different types of functions, which are categorised as technological, human or organisational (Hollnagel, 2012). 4) After understanding possible variability within functions, the next step in the FRAM is to understand how variability may combine. Initially, the aim is to understand how variability within one function in the process can spread and influence the rest of the process (Hollnagel, 2012). Selection of scenario and interviewees A case study was conducted to develop and demonstrate the method. The case study related to the early detection of sepsis. Sepsis is defined as the clinical syndrome that results from a dysregulated inflammatory response to an infection that is non-resolving and deleterious. Sepsis is associated with high mortality rates of 30–50% for patients with severe sepsis and 50–60% for patients with septic shock (Lever and Mackenzie, 2007). Further, sepsis is a complex process involving several professional groups. It requires rapid treatment and can sometimes mimic other conditions with different treatment requirements (Kent and Fields, 2012, Research, 2016). The first step in the analysis and the construction of the FRAM model was to decide how to define the process, and when it started and stopped (Hollnagel, 2016b). It was decided to focus on the early detection of sepsis, as staff during pre-meetings mentioned this as a challenge and an important factor in treating patients with sepsis and achieving a positive result. This resulted in observing and interviewing staff members on the process, from receiving the call from the doctor to initiating the treatment plan. The scenario was selected in cooperation with the included department and a number of resourceful experts who acknowledged that the case was a useful example and a relevant subject regarding patient safety. Additional characteristics that made this case relevant were its regular occurrence and its ability to develop into a serious condition within a short period. It was also an interesting case, as the detection of sepsis and sepsis itself are frequently highlighted during patient safety initiatives (Berezowicz, 2013, Burke, 2003). Observations and interviews were conducted with staff during shifts in the Acute Visitation Ward (AVW), which is part of the medical ward, and in the Emergency Room (ER) within a Danish hospital, as these were two common locations for receiving patients with potential or suspected sepsis.

50



Data collection methods To complete the second step of a FRAM analysis, the selected process needs to be investigated to identify all relevant functions performed. A variation of data collection methods were applied to analyse the cases of early detection of sepsis and to investigate the third and fourth steps of the method. The following section presents the data collection methods. The data collection strategy is illustrated in Figure 8. Document Review

•What are the standards, rules and regulations for the early detection of sepsis? How is the proces expected to run? Initial work includes analysing and using official documents on the selected process to gain understanding.

Interviews with blunt end

•After studying the guidelines, instructions and checklists, interviews with staff who formulated these documents were conducted..

WAI

•These first data collections resulted in a WAI model. The model helped get an initial understanding of the early detection of sepsis and reflected administratives point of view.

Observations

•After finishing this process followed observations to uncover all relevant aspects of the process as perceived from the frontline staff. This included observing staff meetings, morning conferences and patient treatments.

Interviews

•After observations two focus group interviews with staff followed. The interviews were conducted to confirm interpretations from observations and clarified unanswerded questions.

WAD Staff confirmation

•Based on the 2. round data collection the 'new' FRAM model of the process was developed. This map combined WAI and WAD. •After constructing the model, staff and experts were used to confirm the findings. This part also included an audit of past adverse events on the field, to search for patterns found in the data collection. Figure 8: Data collection strategy for Study II

Document analysis The documents investigated were used to obtain a thorough overview of how staff working at the blunt end planned the work at the sharp end. WAI is used especially within the FRAM and refers to the fact that there often a discrepancy between how we imagine work to be done and how it is actually done (Hollnagel, 2012, Clay-Williams et al., 2015). To create an interview and guide for observations of the early detection of sepsis, it was important to have an in-depth understanding of which standards, guidelines and instructions supported and guided the functions under consideration (Pickup et al., 2017). The document analysis was therefore applied in the preliminary phase of the data collection and as a tool for conducting the observations (Clay-Williams et al., 2015). In the creation of the WAI

51



model, four documents were analysed, and two focus-group interviews were conducted with the authors of the guidelines and the head of the wards. Qualitative semi-structured interviews Semi-structured interviews are characterised using an interview guide in which themes and a number of key questions are initially defined (Malterud, 2001, Brinkmann, 2014). The themes and questions for the interview guide were based on findings during the construction of the WAI model. The aim of the interviews was to obtain a thorough and in-depth understanding of the investigated process before conducting observations. Observations The aim of the conducted observations was to uncover aspects not reflected in guidelines and instructions, and to give attention to topics not previously investigated in the early detection of sepsis. This was especially important in steps two, three and four of the FRAM methodology. Observations, and specifically participant observations, are a useful tool for obtaining knowledge of the context or situation investigated (Dixon-Woods and Bosk, 2010, Leslie et al., 2014). It may not be completely unbiased, since the observer may interpret the observations, but it distinguishes itself from interviews, where situations and events are retold by staff (Kawulich, 2005). In conducting observations, the observer can take on different roles, such as total participation, participating as observer or observer (Silverman, 2013). The observations in this thesis were collected as a participating observer, which allowed the researcher to openly discuss the purpose and background of the observations (Kawulich, 2005). Combining the observations with semi-structured interviews and short informal interviews during observations further helped evaluate the findings with the involved parties and obtain response validity (Kawulich, 2005). The observations combined with the interviews with relevant staff were used to construct an in-depth FRAM model of the early detection of sepsis. During the observations, nurses, secretaries and doctors were followed through entire shifts, and patient pathways were observed from admission to the AVW or the ER until the diagnosis of sepsis was either confirmed or dismissed. The researcher conducted 89 hours of observations, including numerous short and informal interviews and clarification between observations. The interviews were used to enhance the understanding of the observations and elaborate on the aspects for each function, as well as variability and upstream–downstream couplings (Hollnagel, 2016b). By simultaneously collecting the data through interviews, it was possible to understand how the feelings, experiences and attitudes of the staff influenced the process. The same level of understanding might not be possible if the data

52



collection was solely based on observations of the process. Thus, the interviews were considered highly important for understanding and mapping the process (Leslie et al., 2014). The aim of the interviews was to gain knowledge of all functions influencing the process and to investigate the variability of each function along with possible consequences of variability (Hollnagel, 2012). Aspects were uncovered by asking questions related to time pressure, regular events and working conditions. Table 4 summarises the observations. Department/profession

Timeframe

AVW/nurse

12–20

AVW/nurse

7–15

AVW/nurse

15–23

AVW/nurse

12–20

ER/nurse and doctor

11–21

ER/nurse

11–21

ER/nurse and doctor

15–23

AVW/doctor

8–15

AVW/doctor

15–23

AVW/nurse

15–23

AVW/nurse

12–18

Table 4: Observations regarding early detection of sepsis

At the end of the observation period, the researcher conducted two focus-group interviews with four nurses to confirm the findings extracted from the observations (Dixon-Woods and Bosk, 2010). The interview guide can be found in Appendix 1. Results: study II To identify the leading indicators of a process, an understanding of this process needs to be established. As health care contains many individual agents, numerous systems and countless interactions, an understanding of the process requires a description of each function in the system. Therefore, the FRAM was used to fully understand the setting (Pencheon, 2008). It was also important to apply a framework that could cope with the complexity of health care, and the FRAM has been shown to be an effective method to achieve this in previous studies (ClayWilliams et al., 2015, Laugaland et al., 2014, Sujan and Felici, 2012, Herrera and Woltjer, 2010).

53



WAI Initially, a model of the early detection of sepsis, as described by instructions and managers, was created to illustrate the difference between how staff at the sharp and blunt ends understand a process (Clay-Williams et al., 2015). This model was constructed by reviewing the functions described in the four sepsis documents and transferring this information into a FRAM model. This is illustrated in Figure 9.

Figure 9: FRAM model (WAI) for 'early detection of sepsis'

The figure shows a simplified version of the process of the early detection of sepsis. It also shows the limited actions guiding knowledge available in guidelines and instructions. The model also symbolises the challenges faced by staff members in their work, as early detection includes many other organisational and structural factors than guidelines or managers in the ward consider. WAD Building the FRAM of WAD required data collected from the daily workplace and situations. The FRAM model was developed during the period of observations in AVW and the ER. The data collection helped identify 40 functions in the process. The FRAM model not only illustrates which functions are necessary for successfully conducting the early detection of sepsis, but also how the functions interact, relate and affect each other if performed with variability (Hollnagel, 2012). Figure 10 presents the FRAM model. The figure demonstrates that functions of early detection of sepsis are not connected in a linear way, but that they are coupled in various ways and include a complex 54



coordination between different staffing groups at different times. To increase visibility of the included functions, Table 5 shows all functions and presents a further description and interdependencies of the model.

55



Figure 10: FRAM model (WAD) of early detection of sepsis

56


Number 1

Function The General Practitioner (GP) or ER contacts the AVW.

2

The nurse, responsible for the telephone in AVW, will receive the call.

3

The nurse with the telephone must be available, to receive the call (Background function). The doctor refers the patient.

4

Brief description of function If the GP has a patient they suspect have an infection based on clinical signs, they will refer the patient to the medical ward, to be investigated and treated. The same is relevant for patients admitted to the ER. If they show signs of infections, doctors from the ER will also call the AVW. One of the nurses on shift will always be responsible for carrying the telephone were patients are called in. The nurse with the telephone may have one of several other functions in the ward, including admission, care of admitted patients etc. The nurse is responsible of several activities besides attending the telephone. Therefore, to be able to answer the telephone, she must be available.

Contribution to the model Activating the function where the ward receives the call from either GP or ER.

During the telephone conversation between the ER doctor or GP, the nurse will receive information regarding the patient’s condition. This may vary in detail and severity. The nurse with the telephone is obligated, besides the telephone to carry a sheet, which includes a number of aspects of the referral she needs to record. She will at this point also note if it will be necessary to order specific blood samples based on the knowledge from the doctor. This includes the nurses past experience with detecting sepsis and knowing what the typical symptoms are or what to ask for when talking with the doctor.

This activates the function, where the nurse will document the information she received from GP or ER doctor. This activates the function where the nurse will pass the sheet on to the secretary.

When the nurse is in charge of the telephone, she must go to the secretary’s office, and collect a stack of sheets, she will carry in her pocket and use when a referral is made.

Precondition for the nurse to be able to record the information necessary to receive the patient properly.

The Sheet will have to be developed to be used (Background function). The secretary receives the sheet from nurse, containing the information from the GP or ER doctor.

In advance, someone has developed and decided which categories go onto the sheet.

Control mechanism for writing down the necessary information.

The nurse will have to leave the activity she is performing or wait until she is finished, and walk to the secretary and hand over the sheet containing information on the referred patient.

The secretary must be available at the office to receive the

The secretary must be at her desk to directly accept the sheet containing the information.

This activates the function were the secretary takes the sheet, and transfers the information onto the electronical system. It also activates the function were the secretary will order blood samples for the patient, if this is noted on the sheet. This is a function affecting the preparedness for receiving the patient.

5

The nurse documents the information received from the doctor.

6

The nurse has the right experience to foresee potential sepsis. (Background function). The nurse with the telephone must have the sheet in her pocket to record onto it. (Background function).

7

8

9

10


57

Activating the function where the ER doctor or GP refers the patient to AVW. Precondition for the nurse to be able to receive the call from the GP or ER and handle the referral.

This function is a resource for the nurse in functions prior to the patient’s arrival.


11

12

13

14 15 16

17

18 19 20

21

22

23

sheet. (Background function). The secretary will transfer the information from the sheet into the electronical system. The secretary will order specific blood samples. (background function). The secretary will collect all necessary paperwork. The paperwork must be developed. The patient arrives at the ward. The secretary follows the patient into the admission room. The secretary updates the electronical system with information on the patient. The Nurse is aware of the patient’s arrival. The admission process begins. The electronical system is available and functioning. (Background function). Measuring vital signs.

Clinical Judgement is used as a tool when evaluating the patient (background function). Registration of the patient in the


The secretary is responsible for transferring the knowledge, which the nurse notes, on the sheet, into the electronical system.

If the information the nurse receives from the doctors indicates that specific conditions are likely she will note on the sheet that specific blood samples need to be ordered at the laboratory technician (LT). The nurse needs to fill out a number of paper during the admission of patients (sepsis checklist, PatientSafe admission, etc.) and the secretary will put these in a folder for the receiving nurse to collect. All the sheets necessary for admission must be developed to be used. The patient will contact the secretary at the reception desk. After the patient has announced their arrival at the secretary desk, the secretary will follow the patient to an admission room, which is available. After following the patient to the admission room, the secretary will update the electronical system, with a number of the room the patient is in.

This activates the function were the nurses on the ward, are informed that a patient has been referred and can be expected at the ward and the function where the secretary prepares paperwork for the admission of the patient. This is an output from when the secretary transfers the additional information into the electronical system. This is a precondition necessary to correctly admitting patients to the ward. This function is a precondition for preparing the folder for admission. This activates the function where the secretary will show the patient into an admission room. This activates the function were the secretary updates the electronical system with room number of the patient. This activates the function where the nurse is becoming aware of the patient’s arrival.

The nurse needs to keep updated with the electronical system to be aware of when the patient has arrived, and in which room the secretary has put the patient. The patient is admitted and the nurse notes the patient in the electronic patient journal, updates all relevant information and symptoms. Several of the functions in the model are dependent on the fact that the electronical system is functioning.

This activates the function where the nurse starts the admission of the patient. This activates the function where the vital signs are measured.

As soon as the nurse starts admitting the patient, the vital signs will be measured (Temperature, Blood pressure, respiratory frequency and pulse). These signs will be leading for how fast the patient will be seen by a doctor, when in the line of patients the patient will be admitted etc. Besides using vital signs and symptoms to evaluate the patient, staff will use their clinical judgement to examine the patient combined with past experience.

This function activates the function were other symptoms in the patient are evaluated and the patient is triaged. It also starts the function were the patient is registered in the Electronic Patient Journal.

To admit the patient, document vital signs and call a doctor later on the patient must be registered in the EPJ.

This is a precondition for the doctor to be able to document treatment for the patient.

58

This is a resource function for several functions.


24

Electronic Patient Journal (EPJ). The LT takes and analyses blood samples.

25

The patient is evaluated based on other symptoms.

26

Triaging patients based on vital signs and other symptoms. Triage tool is developed (Background function). The doctor is called for examination.

27

28

29

30 31

32

33

34

35 36

Using experience when talking to the nurse and evaluating the patient (background function). Looking into the patient’s record before assessment. Conducting the 30minute assessment.


After the secretary informs the LT about the patients, they will arrive at the admission room and take all necessary blood samples. They are then sent to the lab for analysis. After measuring the vital signs, the nurse will also explore other symptoms in the patient. This is a precondition for diagnosing and for how the patient is triaged. The patient is triaged using a form, were the triage is decided based on vital signs and other symptoms. The form must be accessible and developed.

This is a precondition for confirming or dismissing the sepsis diagnosis. This is a precondition for triaging the patient. This activates the function were the doctor is called for examination. It also decides how fast the doctor needs to arrive. This function controls the triage function.

The doctor will be called to examine the patient. If the triage is orange or red, the timeframe for examination is within 1 hour, if the triage is yellow or green it is 4 hours. The doctor will often depend on past experience and clinical judgement to evaluate how fast the patient should be assessed.

This activates the function were the doctor examines the patient.

The doctor will consider the record to be informed of former hospital stays, diagnosis, blood sample results etc.0. The doctor will initially examine the patient to review the patients general state and evaluate how fast treatment plans should be conducted.

This is both a function to activate the 30-minute assessment or the treatment plan. This function is the activating function for the examination of the patient and preparation of treatment plans. This activates the function were the diagnosis is either dismissed or confirmed.

Examining the patient for sepsis or other conditions and prepare treatment plan. The doctor must be available to examine the patient. (background function). The sepsis diagnosis will be dismissed or confirmed.

The doctor will examine the patient and look for diagnosing sepsis.

Treating with sepsis bundle. Registering signs in the sepsis sheet.

After the diagnosis is confirmed, the patient will be treated with all elements in the sepsis bundle. The checklist for sepsis must applied with all patients with reasonable suspicion of sepsis and completed with all patients with at least two of four criteria for sepsis.

This function serves as a resource and time element in the function of the evaluation of the patient.

To examine the doctor must be available and present in the ward.

This is a precondition for examining the patient.

The doctor will use blood sample results, vital signs measured and overall symptoms to confirm or dismiss diagnosis. The doctor will further warrant the treatment.

This activates the treatment with sepsis bundle if confirmed and dismissed other treatment will be started. This also starts the function were the diagnosis is registered. No output in this model.

59

No output in this model.


37

38

39

40

Developing the sepsis checklist (background function). Conducting the diagnosis within an hour (Background function). Continuing other treatment of patients (background function). Writing primary records and recording findings.


To be used the checklist must be developed and printed. According to instructions and the checklist, patient with suspicion of sepsis, must be diagnosed within 1 hour of admission.

This is a control function for the diagnosis of sepsis and a precondition for completing the checklist. This is a time control, for the function of diagnosing the sepsis.

If the sepsis diagnosis is dismissed the patient will continue in other examinations and treatments.


After assessing the patient and starting the treatment, findings ant treatments will be recorded in the patients file.

This function is part of the assessment of confirming or dismissing the sepsis diagnosis as a control element.

Table 5: Key findings of WAI of early detection of sepsis (Adopted from Paper II)

The WAI and WAD models present two different levels of granularity and focus areas. As seen in the WAI model, the management primarily focused their description of early detection on the medical treatments and actions of staff members in relation to the treatment, such as ‘examining patients for positive sepsis criteria’ or ‘conducting blood samples within 1 hour of admission’. The WAD model illustrated that other aspects, along with organisational and collegial aspects, affected whether early detection was conducted successfully. Examples of these functions were ‘transferring information on patients from doctor to referral sheet’, ‘evaluate patient’s other symptoms’ or ‘call the doctor to perform the examination’. These functions illustrate how important it is for staff members to have experience and clinical judgement, to examine the patient, and to detect sepsis patients early in the process. The process and complexity that the FRAM revealed is described in the following section, divided into four sets, each representing an overall task. A thorough description of the study and the model can be found in Paper II. Referring and obtaining information on the patient The early detection of sepsis was affected by activities performed early in the admission process. Observations and interviews revealed that the process of referring a patient included several activities for the nurse in charge of receiving the referral calls. Observations showed that nurses were required to take calls while admitting other patients or performing other tasks, resulting in interruptions in performing the task and potentially creating a lack of information transferred from the doctor to the nurse. Observations showed that nurses with experience had a different approach to acquiring information from the doctor than less or inexperienced nurses. Their experience made them alert to specific symptoms and signs from the patient, affecting their alertness towards possible sepsis early

60



in the process. Talking with the doctor required that the nurse knew which questions to ask to obtain the relevant information. Transferring information to electronic systems Some aspects of the early detection of sepsis related to the use of electronic systems. As soon as the secretary registered a patient in the system, the patient’s data appeared on an electronic board. In addition to registering new patients, the board served as an overview for expected patients, patients who had arrived in admission rooms and already-admitted patients in the ward. The board was an important tool for the early detection of sepsis and the human-machine-interface (HMI) design affected the ability to be alert to changing conditions and factors for patients (Stanton et al., 2013). The board was also used by doctors to keep track of admitted patients, expected patients and which patients needed to be examined first. It also contained information regarding staff functions and whereabouts, patients’ conditions, symptoms, treatments and how often they had to be attended. Admission process The third set of the model included admitting the patient to the ward and preparing for the doctor’s examination. This process was activated when the nurse became aware of the patient’s arrival. The patient’s vital signs were measured and other symptoms were monitored and evaluated before the patient was triaged. The triage decided how often the patient was attended, and it served as a tool when informing the doctor about the patient. The nurses used the triaging colour to underline the patient’s condition and as an argument for getting the doctor to examine the patient as early as possible. The measurement of vital signs was a function that produced many outputs. Completing the checklist required the measurement of vital signs, and the nurse was alert of a possible sepsis diagnosis. The triage was not conducted if the vital signs were not measured, possibly delaying the doctor’s arrival. Examining the patient and confirming or dismissing diagnosis The final set of the model included examining the patient and confirming or dismissing the diagnosis. Calling the doctor to plan the examination of the patient was the first function in this set. The time before the patient was examined depended on how the patient was triaged, but nurses occasionally called the doctor if they were worried or in doubt. Observations and talking to doctors showed that they used the nurses’ word to determine how fast the patient should be examined. Lack of experience in both nurses and doctors affected how fast the patient was examined, and this affected the rest of the process. Younger or less experienced doctors sometimes conducted the primary examination at the same time as the 30-minute assessment. Experienced doctors used the 30-minute assessment to 61



review the patient and order different test results, and they would often come back later to do the full assessment and primary record. The final function of diagnosing the patient included a number of preconditions. Preconditions differ from other aspects because they not only affect how the function is performed, but they also have the ability to stop the function from being performed (Hollnagel, 2012). There were five preconditions in this function, which illustrated that the performance of earlier functions had the ability to postpone or stop the diagnosis of sepsis. To conduct the diagnosis, the doctor examined the patient, examined the vital signs and blood sample results, and used information from the sepsis sheet and the information in the electronic patient journal. If any of these functions were not performed or were delayed, the diagnosis was delayed. Concluding remarks: study II The aim of study II was to identify and use a method to map and illustrate a complex process in health care. The choice of the FRAM was based on a desire to illustrate how work is performed at the sharp end of the system, and to include different factors that affected this work. The FRAM has previously been shown to be effective in the identification of not only activities directly related to the task, but also factors that can affect how tasks are performed, including elements such as time pressure, work relations, experience level of staff and the effect of the HMI designs. The study showed that the FRAM was a suitable tool for use in the first step of identifying the leading indicators of the health care process. It also became evident during this stage that the division of the model into smaller sets was a helpful and important element of analysing the system.

62



Study III: developing a method for the identification of leading indicators The results of study II confirmed that using the FRAM to map a complex process in health care resulted in a description of the selected case with much more detail and complexity than traditional methods usually provide (Kirwan and Ainsworth, 1992). This strengthened the consideration that guidelines might not be the most suitable tool for supporting and examining successful performance in complex health care processes (Clay-Williams and Colligan, 2015, Woolf et al., 1999). The study confirmed that the FRAM was suitable for creating a model of the processes under investigation, which was a necessary precondition for developing a method to identify leading indicators. In the aftermath of collecting data and developing the FRAM, the process of developing the method to identify leading indicators took place as an intuitive process in the pursuit of answering the third research question: How can leading indicators be identified within a given process? During the development of the steps to identify leading indicators, emphasis was placed on two of the features described in the methodology of the FRAM: variability and upstream–downstream functions (Hollnagel, 2012). These features are a central part of FRAM and meaningful in the identification of leading indicators, as they are able to identify functions or activities in a process that have the ability to affect the way the process plays out. Methods: study III Variability Variability can be defined as the range of possible outcomes of a given situation (BusinessDictionary.com, 2017). In relation to the FRAM, Hollnagel distinguished between output variability and performance variability (Hollnagel, 2016a). Performance variability refers to how functions are performed in a process, and this performance can create different outputs of the function, defined as output variability (Hollnagel, 2016a). The aim of investigating variabilities in the FRAM is to depict how some variabilities of the system are either necessary or important to achieve successful outcomes, whereas other variabilities may lead to unexpected and unintended outcomes (Stanton et al., 2013). Variability is important in relation to the identification of leading indicators, since functions that are more likely to be subject to variability are important to manage properly to create success in the process. Thus, these functions are likely or suitable to be considered leading indicators. Variability of functions can be investigated based on different parameters (Stanton et al., 2013). First, variability will typically be divided into the six aspects of the function, along with a

63



number of overall performance conditions, which can often result in variability of functions. Examples of performance conditions are listed in Table 6.

Theme

Reasons for performance variability

Personnel and material

Quality of personnel and necessary material

Training and experience

Quality and level of experience, clinical judgement and training

Quality of communication

Efficiency and adequacy of communications

HMI and operational

Quality of HMI design

support Access to procedures and

Ability to access procedures, regulations, guidelines, flowcharts and instructions

methods Conditions of work

Physical aspects of working conditions

Number of goals and

Number of tasks assigned to each team member and strategy for dividing tasks and

conflict resolution

resolving conflicts

Available time

Time available to perform each task

Circadian Rhythm

Adaption to working conditions and circumstances in different shifts (day-eveningnight)

Staff collaboration quality

Level and quality of formal and informal cohesion and collaboration between staff

Quality and support of

Quality of team roles and the organisational safety culture

organisation Table 6: Overview of common reasons for performance variability (Stanton et al., 2013, Hollnagel, 2012)

During the investigation of variability, each function of the process was analysed for variability in relation to I, C, T, P and R. Upstream–downstream couplings The FRAM describes functions as being either downstream or upstream. Functions that occur after other functions, to which they are linked, and may therefore be affected by them, are called downstream functions. An upstream function is defined as a function that might influence a later function, as it occurs before that function. Upstream and downstream functions are of interest in the analysis of the FRAM because of the coupling between them. In the FRAM, couplings are described as ‘the degree of which subsystems, functions, and components are connected or dependent upon each other’ (Hollnagel, 2016a, Perrow, 2011). Thus, when the variability of functions is analysed and described, it is particularly important to consider that variability affects downstream functions in the model, and that couplings to these functions are not always fixed and can change because of new or different variabilities (Hollnagel, 2016a). Therefore, the development of the method also included the 64



consideration of couplings. By combining variability and couplings, it was possible to point to specific functions in the model that were especially important for achieving intended outcomes. Results: study III Once the framework for modelling a complex process in health care was selected, the model was analysed to develop the LIIM. Developing the method was based on an iterative process. After this process, the actions undertaken in the development of the LIIM were extracted from the case and described in a systematic way, making them replicable for outsiders and in other cases. This section presents and describes the steps in LIIM. Paper III describes the steps using examples from the case of early detection of sepsis. The LIIM consists of six steps, where the first step represents the result from study II. (1) Identifying relevant functions It is important to identify and describe all relevant functions of the process. This was done in a systematic manner following the guidelines presented through the FRAM. Equally important was the explicit consideration of when to stop the description of the process. The FRAM is a proven tool for this, which also includes clear criteria for when the functional model is complete (Hollnagel, 2012). (2) Clustering of functions into sets Processes in health care and in other fields of activity typically comprise connected activities that are performed to complete a larger task. These activities are especially connected to each other and important to perform as they together serve a bigger purpose and are decisive for the completion of a crucial task in the process. In the model, these clusters of functions are referred to as sets. The sets are defined based on different tasks in the process. Processes in health care are typically divided into a number of smaller subtasks that are necessary to perform the overall task. Each subtask included a number of smaller tasks where variability differed. To assess the number of leading indicators of relevance for the process, the process was clustered into sets that were then analysed individually. (3) Identifying the variability of functions in each set The description of how each function was subject to variability helped identify important functions in the process. The functions that were most likely to vary under different conditions were further analysed in the next step of the method, where consideration was particularly given to the themes mentioned earlier. This was because factors such as personnel, experience, communication, HMI interface, access and quality of guidelines, working conditions, shift work, collaboration and support

65



from the organisation are associated with, and cause variability in, functions and processes in health care (Hollnagel, 2012). (4) Identifying upstream–downstream couplings of functions in each set This step dealt with the functions identified in the previous step as being subject to variability. Upstream–downstream couplings were analysed for functions with variability to assess how the variability affected downstream functions and whether these functions tended to dampen or amplify the variability. (5) Identifying the leading indicators for each set of the process This step combined the analysis of the functions subject to variability with functions where variability could either cause variability on downstream functions or where it was caused by variability on upstream functions. The functions that were subject to both upstream–downstream couplings and variability were identified as leading indicators for the process. (6) Confirming the relevance of identified leading indicators by reviewing previous adverse events and consulting peers or experts Finally, the method included confirmation of the relevance of the identified candidate leading indicators. This confirmation of relevance contained two elements. First, if available and possible, a review of previous adverse events was performed to determine whether the indicators were factors that were recognisable in adverse events. This review was conducted because adverse events are historical and detailed descriptions of the process being investigated. Although the focus of these events is the detection of errors or faults in the process, they can provide the basis for investigating whether the factors identified as leading indicators are relevant in the process. Second, the indicators were presented to staff working in the field through semi-structured interviews, where the process was presented along with a presentation of the proposed indicators. This process aligned the analytical work with the staff members’ own interpretation of important elements to ensure success in the process. At this point, the LIIM had helped to identify potential leading indicators for a given process. After identifying the leading indicators, measurable indicators were selected based on the identification of the functions in the process that were critical for success. A detailed description and selection of the four leading indicators of the process are available in Paper III. The four identified indicators were: o Obtaining the necessary and sufficient information on the patient from the referring doctor. 66



o Being alert and aware of electronic screens to register when a patient had been received at the ward and was ready for admission. o Using former experience and clinical judgement to evaluate the symptoms of the patient to supplement the vital signs. o Calling the doctor to conduct the examination and explaining the overall state of the patient to the doctor, including vital signs and other observed symptoms. As these characteristics are not all easily measured and collected through quantitative measures, this study proposes using a combination of quantitative and qualitative assessments to provide a broad understanding of the process and its successes (Herrera, 2012). Table 7 presents examples of how the identified leading indicators can be operationalised. Indicators Functions

Operationalisation of leading indicators

Receiving and obtaining the necessary and sufficient

Qualitative assessment of what the necessary information

information on the patient from the referring doctor.

includes. Quantitative measures of how often the necessary information is obtained.

Being alert and aware of the electronic system to

Quantitative assessment of time from arrival until first

register when a patient had been received at the ward

contact with staff.

and was ready for admission.

Duration of time from when a patient is registered in the electronic system until a nurse is assigned to the patient.

Using former experience and clinical judgement to

Qualitative assessment in the form of interviews with staff

evaluate the symptoms of the patient to supplement

regarding what signs were used to assess patients.

the vital signs.

Quantitative measures could be in form of Likert scale— necessity of using former experience in each case.

Calling the doctor to conduct the examination and

Qualitative measures applied to measure the nurse’s use

explaining the overall state of the patient to the doctor,

of wordings.

including vital signs and other observed symptoms.

Audits of information transfer between nurses and doctors.

Table 7: Potential measurable indicators for early detection of sepsis

Table 7 presents potential measurable elements to further assess each of the identified leading indicators. However, the operationalisation of indicators for this case primarily served as an example. Instead, it is recommended to perform additional data collections, such as observations and interviews with staff, to decide which measures best represent the identified leading indicators (Øien et al., 2010, Herrera et al., 2010a). Developing measurable indicators can serve two purposes. First, making the

67



indicators measurable can assist in evaluating whether they are decisive for and influence successful performance. Second, they can focus on proactive safety management strategies, thereby making them an integral part of the development of processes in the organisation (Herrera, 2012). Concluding remarks: study III This study included the development and presentation of the step that can be performed in the process of identifying leading indicators for activities in health care. The method was developed as a generic tool and should therefore be suitable for use in many different processes with different aims. However, the purpose of using the method should focus on two things: a desire to heighten an understanding of different variabilities in a process, as well as how they may affect the success of the process; and a need to identify limited activities in the process that can be subject to evaluation or improvement, since these will affect the continuing success of the activity. With the described case of the early detection of sepsis, it was possible to present the six steps contained in the model and note four areas that are important to remain alert to and improve for the success of early detection of patients with sepsis in the medical ward.

68



Study IV: applying the systematic method to other FRAM models Methods: study IV After the initial development of the steps required to identify the leading indicators for health care processes, the model was refined as it was tested in a different setting and case to answer the fourth and final research question: Can these developed guidelines be applied to other processes related to patient safety? The final study of this thesis therefore investigated whether the developed method—at this point, LIIM—is applicable to other health care cases. The aim was to identify possible shortcomings of the method and test whether it was applicable in other cases. In addition to testing it on a different case, we tried to use a case described by a different researcher. This step was taken to investigate whether the firsthand experience of conducting the observations and interviews with staff was a precondition for identifying leading indicators, or whether the use of the FRAM could provide a thorough understanding of the process for the identification of leading indicators. Case selection: blood sampling To test the method, it was applied to a FRAM model of blood sampling of patients in the Department of Clinical Biochemistry in a Danish hospital. The process was chosen based on a desire to investigate a common, yet complex, process taking place regularly. The model was developed by staff members in the Centre for Quality in the Region of Southern Denmark using data material gathered in 2014 (Hounsgaard, 2016). It covered the process from ordering blood samples in the primary ward until the sample was sent for analysis. Taking the correct sample from the correct patient within the desired timeframe was considered the desired and successful outcome of this process (Pickup et al., 2017). The material for developing the FRAM model was collected using qualitative data collection methods, including semi-structured interviews with all involved staffing groups (biomedical laboratory scientists, biomedical laboratory scientist working at the front desk, managing biomedical laboratory scientist, laboratory assistant). Interviews were supplemented with walk-arounds with staff in the working environment and through the collection of narrative stories (Hounsgaard, 2016). Five interviews were conducted with six staff members (Hounsgaard, 2016). The objective of using an unfamiliar case was to investigate the usefulness of the LIIM on a case with no preliminary understanding of the process that will typically evolve following data collection (Kawulich, 2005).

69



Results: study IV Study IV summarised the developed method and explained how it was applied to the described case of blood sampling. Further, it was important to assess the challenges each step could pose and whether these challenges could be overcome by describing the method differently or potentially more thoroughly. Overall results and candidate indicators The model of blood sampling was recreated, with the researcher initially collecting the data and developing the model. Figure 11 illustrates the model of blood sampling. The process contains 18 functions carried out during blood sampling from patients. During the analysis of the process, the FRAM model was divided into four sets of functions that, when combined, constituted the main tasks in the process of taking blood samples. The four sets were: (1) ordering blood samples in the primary ward, (2) receiving patients at the blood sample clinic and assigning queue numbers to them, (3) waiting for the patients’ number to be called and walking to the blood sample room, and (4) taking blood samples of patients and sending them for analysis. After identifying the four sets, the functions were analysed for variability and upstream–downstream functions, which were combined during step five of the LIIM. A more thorough description of the first five steps of the model can be found in Paper IV.

70



Figure 11: FRAM model of blood sampling

71



In the investigation of variability in functions, emphasis was placed on aspects such as quality and time, as shown in Table 6. Variability in functions was especially due to themes such as training and experience, quality and design of HMI, conditions of work, number of goals and conflict resolutions. Table 8 presents variability in functions.

72

PhD Thesis—Leading indicators Prior assessment and prescription of blood samples Conducting control visit in department Performed on time prior to clinic visit—on time Prescribing blood samples Prescription not sent—delay Prescription not received in clinic—delay IT system unavailable and manual prescription sent— delay Wrong patient or sample prescribed—delay Patient is walking into blood clinic Does not go straight to reception—delay Does not have social security card with him—delay

Ditte Caroline Raben Receiving patient and preparing paperwork Receiving patient in blood clinic Many patients at clinic—delay Experience level of front desk staff— delay Searching for prescription on blood samples Prescription not sent—delay Prescription not received/lost in system—delay Searching for paper if IT system is not functioning—delay Searching on patient’s name for missing sample—delay Prioritise queue number based on patient characteristics Evaluate patient to priorities queue— imprecise Wrong evaluation of patients—delay

Identification of special needs patients Ability to identify special needs— imprecise Experience of identification— imprecise

Waiting for turn and keeping track of special need patients Having the experience to identify special needs patients No experience—delay No experience—imprecise Keeping track of screen for patients Identification of wrong patients— delay or early Busyness on the ward—delay Busyness on the ward—imprecise

Taking blood samples and preparing for analysis Call the patient into the blood sample room Calling wrong patient—imprecise Patient is not hearing—delay Searching for patient—delay Print labels for blood samples Wrong labels printed—delay Printing not possible—delay

Patients wait until number called on screen Walking too early—early Not keeping track of number called—delayed Not keeping track of own number—imprecise Purposefully walking somewhere else than instructed—delay Guiding special need patients into blood sample room Not identified—delay Finding patient—delay Busyness prevents assisting— delay or imprecise Patient walks into blood sample room Walking into wrong room and going back—delay Walking into wrong room and having samples taken—precise

Check for urgency and potentially order urgent transport Urgency not detected—delay

Table 8: Variability of functions in FRAM for blood samples

73

Take blood samples from patient Personal factors—delay

Put labels on blood sample glass after sampletaking Wrong labels—delay Wrong labels—imprecise Prepare sample for further transport Patient arrives late—delayed Wrong labels on—imprecise Labels missing—imprecise Samples archived wrong—delay



After combining the features of variability and upstream–downstream couplings in the analysis, four functions emerged as being especially crucial for success and subject to variability. These functions were considered candidate indicators because they gave a good indication of whether the outcome of the process had the right preconditions for being successful. Using the steps of the LIIM reduced subjectivity and ensured that the indicators were identified through analysis results only. The analysis showed that the following functions were decisive: (1) prioritising the queue of patients based on patients’ characteristics and an assessment of the type of blood samples (2) being able to identify patients with special needs (3) keeping track of the progress of numbers on the screen and the flow of patients in the waiting area (4) ensuring that patients walk to the right blood sampling room when called. To assess the identified indicators and confirm their relevance to the process, the final step of the method included obtaining data from staff working in the ward. Similar to earlier studies, the effectiveness of the LIIM was improved during this process because the interviewed staff members recognised all identified functions as important contributors to the success of blood sampling. Emerging challenge During the analysis, a challenge and consideration related to the steps of the method emerged. This challenge did not compromise the success of LIIM, but it is important to acknowledge as the refinement and development of the method continues. During the final part of the study, in an interview conducted with staff a further function turned out to be decisive for success. This was the function of ‘prescribing the blood samples’. The function is performed prior to patients’ arrival at the blood clinic, and initial analysis of the model showed that the variability of this function was typically dampened at the receiving desk of the blood clinic. However, the interviewees mentioned different types of variability in this function that affected the success of the process. Variability appeared if the blood samples were not correctly prescribed from the ward. In some cases, the variability was dampened because staff members at the front desk were able to call the ward and reach the prescribing doctor to resolve the matter. However, in many cases, the doctor was not available and the secretary receiving the call was unable to advise which blood samples were required. Staff at the front desk of the blood clinic handled this variability in different ways. In some cases, they would be familiar with the patient and reorder previous blood samples, 74



they would talk to the patient and decide themselves what samples they believed had to be taken, or they were forced to send patients back to the ward or home if they were not able to decide on the samples. Thus, the prescription of blood samples in the ward is an important function for the successful process of taking blood samples, and hence should be considered a candidate for a leading indicator. It is uncertain whether this would have been detected if the analysis was based on data collection rather than an already evolved FRAM model. However, it seems plausible that this variability would have been considered because interviewees mentioned that the situation occurred 20–30 times each day. Therefore, it is believed that this method is best able to identify leading indicators in cases where data collection is conducted with this aim in mind. Concluding remarks: study IV The fourth study aimed at refining the LIIM and testing it on a process not observed personally. Applying the six steps of the LIIM resulted in the identification of four candidate leading indicators, however the sixth step of the method revealed that some aspects might not be covered completely with this approach. The conclusion of this study will therefore be, that the method was able to identify potential indicators but to ensure all crucial functions are considered, the data collection should be conducted with the LIIM in mind.

75



Discussion Summary of findings The four studies in this thesis have presented the background for and development of the LIIM. The results have confirmed that it was possible to use the FRAM as a foundation and framework for identifying leading indicators in health care processes. The LIIM was developed using a case of the early detection of sepsis, and it was refined using the case of blood samples. This section discusses the results and their implications. First, limitations of the results are analysed, and considerations regarding reliability, validity and transferability are presented. Second, the method is compared with existing methods. Third, the section discusses how leading indicators and the method can contribute to decision-making tools in health care processes as well as the industry of leading indicators. This discussion will refer to the distinction of Safety-I and Safety-II, along with the contribution of RHC to the management of health care processes. Limitations of the study and consideration of a different method The LIIM is not a quick fix for an organisation or department. The method can help identify leading indicators for a given process, and the indicators will always be context-specific for each process. Thus, the results at this point are limited to cases that have been investigated, namely the early detection of sepsis and blood sampling. Therefore, using the LIIM for future cases requires data collection and the involvement of frontline staff, as they have the best qualifications for presenting and describing the system, along with how performance adaptions in everyday work are part of successful outcomes. The established method—the LIIM—has partly been based on elements and ideas taken from the FRAM (Hollnagel, 2012), which is one of a number of different methods that have been developed to understand and describe socio-technical systems such as health care. The FRAM is a relatively new method and is therefore still in the developmental phase. Further, it has not yet been tested to determine whether different researchers will be able to produce similar FRAM models, or whether the application of the FRAM is dependent on the experience, knowledge and expertise of the researcher using the tool. Other methods, such as hierarchical task analysis (HTA), are commonly used for mapping and investigating complex systems (Kirwan and Ainsworth, 1992). HTA focuses on task analysis, with an emphasis on establishing which subtasks need to be carried out to meet a specific goal (Kirwan and Ainsworth, 1992). HTA differentiates itself from the FRAM because tasks are analysed hierarchically and are considered dependent on the primary, and the results can therefore

76



only be obtained if all activities are performed in the correct and predefined order. The method has shortcomings if the goal is illustrate how tasks can be performed if the order of activities is switched or if some activities are left out (Kirwan and Ainsworth, 1992). Further, HTA provides an effective means of how work should be organised to meet specific goals (Kirwan and Ainsworth, 1992). This distinguishes it from the FRAM that is useful in illustrating not how work should be organised, but how work is actually organised (Clay-Williams et al., 2015). Reliability The assessment of research reliability refers to the degree to which a method can be repeated with the same results, and it is typically considered in quantitative research (Golafshani, 2003, Kvale and Brinkmann, 2009). This study considers reliability in terms of whether other researchers would be able to identify the same leading indicators as described in this thesis. The FRAM model of the early detection of sepsis was derived from observations and interviews with staff members. All data collection and analysis was further conducted by the Ph.D. student, who has a background as a Master in Public Health and has no direct clinical training or education. This influenced how the process was observed, the questions asked during the interviews and observations, and the identified leading indicators. The Ph.D. student’s knowledge of theories, beliefs and the perceptual ‘lens’ influenced how the study evolved (Maxwell, 2012). Therefore, it is believed and acknowledged that the method (FRAM) chosen to identify the leading indicators greatly affected the conclusion of the study. To identify the influence of the Ph.D. student in the identification of the leading indicators, it was important to create transparency of the results. Thus, the results and method of this thesis were described thoroughly so that others will be able to understand and follow the process of identifying the leading indicators along with the development of LIIM. Validity Validity refers to the trustworthiness or credibility of the results of a study (Creswell and Miller, 2000). In quantitative research, validity is often defined by a ‘golden standard’ to which the results are compared (Maxwell, 2012). In qualitative research, validity is more often based on testing the results against the world and giving them a chance to prove the conclusions wrong (Maxwell, 2012). In this case, it primarily considers how to establish the extent to which the method has helped to identify the leading indicators that provide adequate coverage of the studied process (Yin, 2003). One way of investigating this is the triangulation of qualitative and quantitative methods (Golafshani, 2003). Therefore, it is recommended that the relevance of the identified leading indicators be tested

77



through the collection of data. This process could be commenced with the execution of focus groups or workshops, as suggested earlier, to determine how to best measure the identified leading indicators of the process. If such group sessions resulted in the identification of relevant measures—either current or new—data collection could begin. The data collection could be evaluated both qualitatively and quantitatively. The quantitative assessment could consider whether the fulfilment of indicators is associated with a high number of successful outcomes as well as the minimisation of unwanted events. For the qualitative assessment, another round of interviews could be conducted to investigate whether staff believe that the focus on leading indicators and the measures of these indicators have affected the process in a positive way. Transferability The transferability of the results of this thesis is limited, predominantly because the developed indicators are identified based on a clearly defined and specific process. Therefore, transferability is discussed in relation to whether the method has the potential to be applied in other cases and to produce useful indicators (Kvale and Brinkmann, 2009). During the thesis, the issue of transferability was addressed in study IV, which attempted to apply the steps of LIIM to a different case than the one for which the steps had been developed. The results of study IV suggest that the method provides a number of useful steps that point in the direction of possible leading indicators for success in two cases. More studies are required to further test the transferability of the method. During the first case (study II) and the development of the method (study III), both the data collection and the completion of the six steps were performed by the Ph.D. student. During the second case (study IV), the data collection was conducted by a different researcher, and the application of the LIIM was thereafter performed by the Ph.D. student. This could result in a question of whether the method is dependent upon the Ph.D. student. Therefore, it can be suggested that a third study could attempt to facilitate others in collecting data and then performing the six steps of LIIM. If this resulted in the identification of leading indicators, it would benefit and strengthen the method and minimise the consideration of whether the method is dependent on the Ph.D. student developing it. Comparison with existing literature on leading indicators This study provided knowledge of the diversity of methods and understandings of indicators both across and within domains. During the literature review, a number of different methods and approaches to identifying leading indicators in high-risk industries were investigated. The level of detail varied between publications, and the focus varied. Some reported on the concept of leading

78



indicators by presenting different approaches and types of leading indicators (Manuele, 2009, Bennett and Foster, 2005), whereas others presented their own approach to identifying leading indicators (Øien et al., 2010, Herrera et al., 2010a). Yet others presented the application of an already-defined way of identifying leading indicators and discussed the challenges or advantages of the approach (Khan et al., 2010, Health and Safety Executives, 2006). This section presents some of the earlier applied methods for identifying leading indicators and discusses some similarities and differences between these and the LIIM. In the literature, Grabowski et al. (2007a) proposed that identifying leading indicators should be done using a two-stage process. The first and fundamental stage is to identify the significant safety factors, and the second step is to identify the leading indicators from these safety factors (Grabowski et al., 2007a). The identification of safety factors can be based on a causal model, where the relationship between safety factors and performance measures is proposed, with consideration for the structure and functioning of the organisation or system (Grabowski et al., 2007a). This thesis has primarily focused on the identification of factors believed to be decisive for success, as the factors have not yet been compared with performance measures and validated for relevance. This was an active decision, as it is believed to be of vital importance to not just create a hypothetical model of how the system works, but to investigate the system in depth and determine what creates challenges and variability in the system (Hollnagel, 2012). This consideration relates back to the distinction of WAI and WAD, where study II investigated the difference between imagining and planning how the system works and actually observing and understanding how the system works (Clay-Williams et al., 2015). We therefore believed it was important to present a method that could primarily focus on the first step of the two-stage process presented by Grabowski et al. (2007a). By focusing on describing and explaining how systems work on a daily basis, we ensured that the later proposed indicators were identified from an informed basis. A different study identified leading indicators for helicopter operations in the Norwegian oil and gas sector. This study can be compared to the present study because it also uses the FRAM to identify leading indicators and a Risk Influencing Model to identify lagging indicators (Herrera et al., 2010a). The article presents a FRAM model of the process of a helicopter landing at night-time. The article further described and presented an interesting approach in the pursuit of making the identified leading indicators operationalised. Herrera et al. (2010a) explained how discussions, workshops and interviews with operational staff were used to discuss the indicators against preliminary criteria for

79



indicators (meaningful, reliable, available or affordable, operational and ownership). This presents an interesting approach, and it could be considered the next step in assessing the usefulness of the LIIM. The study further supports the findings from this study, since it also identified how the use of the FRAM as a theoretical framework can enhance one’s understanding of the system (Herrera et al., 2010a). We therefore believe that much can be learned from the high-risk industry methods assessed during the review, as long as differences between the practice environment and culture are considered. It is believed that the practice environment and the cultural aspects central in health care can be addressed through the FRAM. A concept or method for identifying leading indicators that has been used several times is the concept developed by HSE (Executives, 2006). HSE presents a step-by-step guide for the identification of safety indicators for major hazard industries. During six steps, users are instructed to: (I) construct organisational arrangements for implementation, (II) decide on the scope or aim of the indicators, (III) identify which risk systems are to be targeted and what the desired outcomes of them are, (IV) identify critical elements of the risk system, (V) establish how to collect data and report, and (VI) review the identified indicators (Executives, 2006). The term ‘risk control system’ covers a specific activity performed in an organisation and may therefore be compared to what is referred to as a health care process in this thesis (Executives, 2006). Despite also using a six-step approach, HSE distinguishes itself from the LIIM in several important aspects. First, when deciding on the scope of indicators, HSE presents three possible approaches: identifying incident scenarios, identifying causes of hazard scenarios and reviewing performance or non-conformances. These are all possibilities with a Safety-I focus because they aim to understand the underlying reasons for accidents or hazardous situations, along with investigating incident scenarios or reasons why staff do not perform as expected. However, the approach also considers a number of important elements, including a reflection of whether indicators are needed at an organisational, site or plant level (Executives, 2006). In health care, this can be compared to considering the use of leading indicators at an organisational, department or case level. Further, the approach shows an interesting understanding of how success and failure occur. According to HSE, establishing desired outcomes and afterwards finding out which elements of the risk control system must be undertaken correctly every time are sufficient activities to identify leading indicators. This is a well-thought-out approach that also helps to create a deeper understanding of the system. As presented earlier, activities can go well for different reasons but also fail for different reasons. Therefore, leading indicators can also be useful if they not only detect when

80



things are done in a specific way, but when they can also highlight if activities deviate or contain variability for a number of different reasons (Executives, 2006). Finally, a fourth approach was compared with the LIIM. This approach was also based on the principles of RE. The Resilience-based Early Warning Indicator (REWI) method presents a framework designed to help systems cope with unexpected situations (Øien et al., 2010). The paper divided the identification of indicators into three parts: (I) identification of attributes of resilience, (II) consideration of relevant issues ensuring that all attributes can be fulfilled, and (III) identifying how each attribute can be measured (Øien et al., 2010). The REWI method is interesting because it is a contributory method in which the users of the system identify the leading indicators. Through workshops, staff define their own issues based on the attributes of resilience, and they then define indicators to avoid these issues (Øien et al., 2010). The REWI method also draws upon the perspectives of Safety-II, such as this thesis, as the indicators are defined from success factors of the process. Thus, instead of identifying leading indicators that precede undesirable events, the indicators are defined based on how to achieve situations with the presence of positive factors (Øien et al., 2010). Like the LIIM, the REWI method is a generic method applicable for different industries and processes. The article highlights the importance of staying critical to the method and ensuring that the method is adapted to each specific system, plant or unit considered (Øien et al., 2010). The REWI method includes many relevant considerations and elements. The perspective of allowing involved staff to play a larger role in identifying leading indicators, as well as the operationalisation perspective, could have received more attention in this thesis. In hindsight, this part could have strengthened the relevance of the method in the wards, along with the applicability of the indicators. Contribution to the industry The LIIM contributes to the health care industry because it presents a first attempt to develop a generic method for the identification of leading indicators in health care. An important aspect after identifying leading indicators is to consider how they can contribute to improving the system. Typically, safety indicators in health care are often seen as part of large-scale political targets that have to be met, supporting the argument of making indicators measurable and systematically collecting data on them (Kristensen et al., 2007). However, the identification of leading indicators can contribute to the industry in other ways. They can be part of an organisation’s integral safety management. The traditional way of targeting patient safety and improvements in health care is currently based on knowledge drawn from the reporting of adverse events (Vincent, 2010c). Depending on the

81



tendencies revealed by the systems collecting these reports, we tend to consider the problems individually (Braithwaite et al., 2015). Each case is analysed using RCA, FMEA or other errordetection methods (Vincent et al., 2013, Vincent, 2008). Based on the results of these analyses, barriers will typically be put in place and consist of new guidelines, instructions, rules and regulations (Sujan et al., 2016). This thesis argues that the application of other perspectives can help to improve patient safety in a different way than previously seen. Examining processes with the objective of identifying leading indicators can provide the same knowledge as conducting an RCA. Nevertheless, we believe it can do more than that. The method does not offer a simple and single root cause for one adverse event; rather, it can provide a more holistic picture of the process, in which adverse events sometimes occur (Braithwaite et al., 2015). It can provide information on the performance of the organisation or system as a whole that is involved in the process. It can increase the focus of viewing processes through a safety lens and increase motivation among staff by examining all aspects of the process from an organisational, structural or human perspective. This might ultimately add to the organisational potential for safety. Further, if everyday operations are properly understood, it can enhance the ability to estimate how modifications of technology, as well as organisational factors, will affect the overall performance of the system (Praetorius et al., 2015). In developing this method, we have tried to contribute to a field that is constantly developing and changing. The studies and results of this thesis contribute to a new way of examining indicators in health care and a consideration of how knowledge on leading indicators from other industries can be made useful in health care, as well as how the FRAM and Safety-II can be used to develop a method that facilitates the identification of leading indicators in specific health care processes. When we present the results as a contribution to the industry, it is important to note that a balanced application of lagging and leading indicators is still advised (Herrera et al., 2010a). Emphasis has been placed on presenting the understanding of safety and how this can create new knowledge in the field of safety in health care. Choices have been made during this work by applying a specific and somewhat new way of viewing safety (Braithwaite et al., 2015). The understanding of how things go right and maximising the number of events with a successful outcome was described earlier and labelled as Safety-II (Braithwaite et al., 2015, Hollnagel, 2008). Staying true to this concept means that leading indicators are not understood as precursors of negative or unwanted events. They are supposed to be an expression of organisational attributes or aspects that enable safe interactions and patient treatment. Further, they can help managers create a system to meet desired future states (Reiman and Pietikäinen, 2014). 82



In many high-risk industries, safety management systems are considered a vital part of maintaining and developing safety in an organisation. However, the systematic approach to safety in health care largely focuses on the perspectives of Safety-I. Most interventions to improve safety in health care are therefore based on analysis and measurements of unwanted events (Vincent et al., 2013). The method presented in this thesis was developed from the perspective of offering a contribution or example of what methods could be applied in the introduction of a holistic safety management system in health care. The aim of applying the method and identifying leading indicators is to visualise aspects of health care processes that may not have previously been considered important for safety. If the organisation at a later stage decides to make the indicators measurable and collect data, it will create a strong signal to its staff that issues related to patient safety are important enough to be followed closely by management (Reiman and Pietikäinen, 2014). Thus, the LIIM can create a common language to discuss safety, and it offers a way to monitor and evaluate the safety management activities in the organisation (Øien et al., 2010). The benefits of the Safety-II approach are still being revealed and discussed among researchers (Sujan et al., 2016). However, as the concept receives increasing attention, the number of publications with positive results is also growing, stating that Safety-II has the potential to enhance safety in health care. Examples are presented below of other studies that have found that Safety-I and Safety-II can contribute to and supplement our understanding of safety in health care. A Scottish study on blood samples concluded that the FRAM and perspectives of Safety-II provided a realistic model of blood sampling and that the system could succeed through adaptability (Pickup et al., 2017). The study further stated that it showed where resilience could be enhanced in the system, and it pointed to areas that are relevant to improving the success of the process (Pickup et al., 2017). The advantages of Safety-II were further discussed in a conceptual paper on learning from incidence (Sujan et al., 2016). Here, the authors suggested that a Safety-II approach can help shift attention from extraordinary failures to ordinary performance adjustments that clinicians make every day (Sujan et al., 2016, Sujan et al., 2015). Such empirical studies further show that performance adjustments are a critical part of what keeps patients safe. Therefore, from a Safety-II perspective, it makes sense to focus on ordinary performance and understanding the leading indicators for successful performance rather than focusing solely on lagging indicators such as the number of adverse events. The application of Safety-II has also been suggested to gain a greater alignment between WAI and WAD. Studies have revealed a gap between how people working in administration (the blunt end) and people working on the frontline (the sharp end) view the processes described in guidelines (Clay83



Williams et al., 2015). The study showed that the perspectives of Safety-II helped to implement new guidelines in two different Intensive Care departments. It also revealed how staff modified their everyday performance to adhere to the guidelines of the process and created a clear link between the guidelines and the actual work performed in the department on a daily basis (Clay-Williams et al., 2015). These findings show that Safety-II perspectives can help identify indicators that might not be necessary according to staff at the blunt end, but that are useful for staff at the sharp end. The selected studies discussed above have investigated how Safety-II can be applied to health care and patient safety, and how it can help heighten awareness of other aspects of complex processes, which previous methods have paid limited attention to. The overall positive experiences and findings support the belief that Safety-II can be useful in relation to leading indicators and the monitoring of safety states and the improvement of how health care processes function.

Conclusion This thesis has presented a new view of how incorporate and apply indicators in the field of health care. It has discussed how knowledge and experience from high-risk industries can be useful, but not create a foundation for the development of new approaches in health care. Methods developed in high-risk industries however highlight the importance of remaining aware and sensitive to the importance of organisational, structural, cultural and personal factors. It has therefore presented the development of a systematic method for the identification of leading indicators in health care (LIIM). LIIM can help establish a number of candidate-leading indicators of a specific process or activity in health care. The study supports findings from previous studies on the usefulness and application of Safety-II and RHC to improve our understanding of processes in health care. By characterising health care as a complex system with numerous components, this thesis presents the identification of leading indicators for specific processes, with a focus on the importance of understanding how the system functions on a daily basis. Instead of focusing on adaptions and variations in performance as something that is unwelcome or undesirable, the perspective of RHC has helped detect variability and consequences and apply them as a way to identify leading indicators. However, further studies are recommended to elaborate and validate the method and findings of this thesis. Some perspectives and ideas for future research are presented in the next section.

84



Perspectives After finalising the literature review, I found the concept of indicators to be complex and varying across different domains. Since the literature review offered no direct guidelines for identification of leading indicators in health care a number of decisions were necessary regarding how the concept of leading indicators would make a difference in health care and a contribution to current practice and use. The choice of applying the concept of Safety-II was guided by the main supervisor, but during the past three years, the study showed that this perspective also resonated greatly among staff at the sharp end. They acknowledged and understood the perspective of Safety-II and welcomed the use of FRAM. Despite this positive feedback, a number of issues have not been investigated in this thesis, and should be considered in the future. The results of this thesis are the first step on the way to applying the concept of leading indicators in health care. This work will hopefully inspire others towards further development in the future, which could be focused on strengthening the concept of leading indicators in health care, and most importantly on aligning definitions and agreeing on common terms. This issue has been one of the major challenges for going forward in this field—both for leading indicators and safety in health care in general. The field of patient safety is a relatively new domain, where new approaches and ways of thinking are commonly introduced and old ways are discarded (Kohn et al., 2000, Reiman and Pietikäinen, 2014). Getting the full picture of terms, perspectives, important viewpoints and definitions has been difficult. The use and application of leading indicators is influenced by opinions and perspectives from different key stakeholders (Hale, 2009). Therefore, it requires work to understand the concept of leading indicators in a health care context and create consensus on a definition or method for the identification of leading indicators. Searches in a variety of databases during the past five years (2012–2017) revealed no scientific publications within health care using the terms ‘leading’ or ‘proactive’ indicators. Therefore, considering that this thesis presents one of the first attempts to define leading indicators in a health care context, it is still possible at this stage to further define the concept and method. A collected overview of different methods, approaches and history would be a good tool for new researchers in the field (Wears et al., 2014). The method contributes to improving the conditions of the system that enables patients to be treated safely. Applying the concepts of Safety-II in health care may help to improve conditions and decrease the rates of incidence and adverse events, which have remained steady since the topic of safety in

85



health care became a research area in the late 1990s (de Vries et al., 2008, Shojania and Thomas, 2013, Thomas and Petersen, 2003). More specifically, management and control of a complex system such as health care requires knowledge of what has happened, what happens and what may happen in the future (Hollnagel and Woods, 2007)—particularly in relation to making decisions that affect staff members on the frontline. However, the complexity of health care and the many agents involved in health care process, as well as the interrelations between different processes, can make it challenging for decision-makers to understand what is really going on (Ball and Frerk, 2015). Therefore, the LIIM might be used on the management level of a hospital to obtain an overview of complex processes, along with the ability to focus on smaller aspects of these processes, which will be important for successful performance. Thus, LIIM helps to visualise the complexity of the systems and points to specific elements of the system that can affect success. Therefore, the method may be useful in decision-making processes, such as reorganisation or as an aid in a situation that does not function as intended, but for which no obvious causes are found. In such cases, LIIM represents a way to systematically reveal issues or factors that are relevant in creating safe processes. The method can help determine which elements are important to focus on when organising or designing processes in health care. At the clinical level, the LIIM has been shown to be a tool that gives credit to frontline staff. As presented above, managers can have a simplified understanding of processes or activities at the frontline, and many of the important aspects for making things function on a daily basis are not considered when management designs or changes the conditions at the sharp end. At this level, the LIIM gives management an understanding of this complexity and gives recognition to staff members for their ability to work and adapt to the conditions of their daily working environment.

Suggestions for future research Several aspects of this work are relevant and interesting to explore further in the future. These aspects are divided into two areas. One direction would be the further refinement of the method. In order to strengthen the methods credibility and useability there is a need for testing the method, on more processes and by more individuals. This is especially crucial as the LIIM is a tool that is expected to work on an everyday basis in different hospitals and for different scenarios. Therefore it is vital that the six steps are explained thoroughly enough for application by different people. Easy understanding of the steps and

86



ability to apply them without the presence of the Ph.D. student, will therefore determine its success. One aspect of this, which was touched on earlier, is to ensure that the six step process involves more staff. Contributor-based indicators were explored by Herrera et al. (2010b) and Øien et al. (2010). This perspective is believed to strengthen the method and create useful reflections of everyday work along with needs and views of the staff. Including staff in the development and use of leading indicators might also increase the awareness and knowledge of leading indicators in the frontline staff that are primarily used to assess safety conditions and improvements based on past adverse events. By raising knowledge of how positive events and understanding of everyday work, the focus on benefits of Safety-II can be highlighted. This might create conditions for improve the system’s ability to accept performance variability. The second direction for further research is the validation of the identified indicators. This will require two tasks. First, the candidate indicators need to be made measurable. As mentioned during the results of study III, the identified indicators are still not considered measurable, but examples of how they could be made measurable were presented. Therefore, the first task in the validation or confirmation of the usefulness of the indicators will be the operationalisation. As also mentioned earlier, this process should include some kind of staff involvement. Staff involvement seems important as staff have the best understanding of the process they are working in and they have the best preconditions for exploring how concepts or situations in their work processes could be measured effectively. During this process, it is important to select a manageable set of indicators for implementation and use. After deciding on how to measure the identified selection of indicators they need to be routinely collected, measured and analysed. The data analysis might include analysis of indicator fulfilment along with the occurrence of adverse events. The analysis can be supplemented with the collection of qualitative data on the advantages and gains of the leading indicator system. Such an analysis might be able to determine a potential correlation between the leading indicators and the occurrence of unwanted events. If the fulfilment of indicators is associated with a decreasing number of unwanted events, it can indicate that there is an association between the indicators and the safety state of the process or organisation. When working with leading indicators it is also important to acknowledge that indicators might be subject to changes, since regular reviews of the usefulness of the applied indicators may show that the focus is no longer necessary. This may be because the increased focus on the selected indicators

87



has created a change in behaviour in the ward. However, this is not a bad thing as it might leave room for new and different indicators to be looked upon instead (Øien et al., 2010). With this project, I set out to investigate how leading indicators are understood and applied in both high-risk industries and health care. Based on the findings I suggest taking a greater consideration of new and different ways to view safety in health care. I have found that looking at things with a different and new perspective may help uncover new and undiscovered potentials for improvement in the health care system. However, despite trying to shift the focus of safety in health care, I hope this work still reflects that I greatly acknowledge the work that is done in order to deliver high quality and safe health care for patients.

88



References AMERICAN BUREAU OF SHIPPING 2012. Safety culture and leading indicators of safety. Houston, Texas, USA. BAGIAN, J. P. 2005. Patient safety: what is really at issue? Frontiers of health services management, 22, 3. BAINES, R. J., LANGELAAN, M., DE BRUIJNE, M. C., ASSCHEMAN, H., SPREEUWENBERG, P., VAN DE STEEG, L., SIEMERINK, K. M., VAN ROSSE, F., BROEKENS, M. & WAGNER, C. 2013. Changes in adverse event rates in hospitals over time: a longitudinal retrospective patient record review study. BMJ quality & safety, 22, 290-298. BALL, D. & FRERK, C. 2015. A new view of safety: Safety 2. British Jrnl Anaesthesia. BARACH, P. & SMALL, S. D. 2000. Reporting and preventing medical mishaps: lessons from non-medical near miss reporting systems. BMJ, 320, 759-63. BAUD, K. C. 2012. Passive leading indicators of construction safety performance. 1511943 M.S., University of Colorado at Boulder. BAXTER, P. & JACK, S. 2008. Qualitative case study methodology: Study design and implementation for novice researchers. The qualitative report, 13, 544-559. BENNETT, J. & FOSTER, P. 2005. Predicting progress: the use of leading indicators in occupational safety and health. Policy and Practice in Health and Safety, 3, 77-90. BEREZOWICZ, P. 2013. Sepsis: svær sepsis & septisk shock, retningslinier for diagnostik og behandling på intensiv afsnit; læge [Online]. Region Syddanmark: Region Syddanmark. Available: [Accessed 17-03-2017 http://ekstern.infonet.regionsyddanmark.dk/Files/dokument4281.htm 2017]. BERGH, L. I. V., HINNA, S., LEKA, S. & JAIN, A. 2014. Developing a performance indicator for psychosocial risk in the oil and gas industry. Safety Science, 62, 98-106. BLAIR, E. & O'TOOLE, M. 2010a. Leading Measures. Professional Safety, 55, 29-34. BLAIR, E. & O'TOOLE, M. 2010b. Leading Measures: Enhancing safety climate and driving safety performance. Professional Safety, 55, 6. BRACCO, F., BRUNO, A. & SOSSAI, D. Improving resilience through practitioners’ well-being: an experience in Italian health-care. Fourth Resilience Engineering Symposium, 2011. 43-49. BRAITHWAITE, J., CLAY-WILLIAMS, R., NUGUS, P. & PLUMB, J. 2013. Health care as a complex adaptive system. In: HOLLNAGEL, E., BRAITHWAITE, J. & WEARS, R. L. (eds.) Resilient Health Care - Volume 1. Surrey, England: Ashgate Publishing. BRAITHWAITE, J., WEARS, R. L. & HOLLNAGEL, E. 2015. Resilient health care: turning patient safety on its head. Int J Qual Health Care, 27, 418-20. BRERETON, P., KITCHENHAM, B. A., BUDGEN, D., TURNER, M. & KHALIL, M. 2007. Lessons from applying the systematic literature review process within the software engineering domain. Journal of systems and software, 80, 571-583. BRINKMANN, S. 2014. Interview. Encyclopedia of Critical Psychology. Springer. BROADRIBB, M. P., BOYLE, B. & TANZI, S. J. 2009. Cheddar or Swiss? How Strong Are Your Barriers? Process Safety Progress, 28, 367-372. BURKE, J. P. 2003. Infection control--a problem for patient safety. The New England journal of medicine, 348, 651. BUSINESSDICTIONARY.COM. 2017. variability [Online]. Available: http://www.businessdictionary.com/definition/variability.html [Accessed]. CAMPBELL, S. M., BRASPENNING, J., HUTCHINSON, A. & MARSHALL, M. 2002. Research methods used in developing and applying quality indicators in primary care. Quality and Safety in Health Care, 11, 358364. CARD, A. J. 2016. The problem with '5 whys'. BMJ Qual Saf. CARTHEY, J., DE LEVAL, M. R. & REASON, J. T. 2001. Institutional resilience in health care systems. Qual Health Care, 10, 29-32. CHECKLAND, P. 1981. Systems thinking, systems practice.

89



CHIOZZA, M. L. & PONZETTI, C. 2009. FMEA: a model for reducing medical errors. Clin Chim Acta, 404, 75-8. CLAY-WILLIAMS, R. & COLLIGAN, L. 2015. Back to basics: checklists in aviation and health care. BMJ Quality & Safety, 24, 428-431. CLAY-WILLIAMS, R., HOUNSGAARD, J. & HOLLNAGEL, E. 2015. Where the rubber meets the road: using FRAM to align work-as-imagined with work-as-done when implementing clinical guidelines. Implement Sci, 10, 125. COOPER, J. B., GABA, D. M., LIANG, B. A., WOODS, J. D. D. & BLUM, L. 2000. Agenda for Research and Development in Patient Safety. Chicago, National Patient Safety Foundation. CRESWELL, J. W. & MILLER, D. L. 2000. Determining validity in qualitative inquiry. Theory into practice, 39, 124-130. DE VRIES, B. 2012. Sustainability Science, Cambridge University Press. DE VRIES, E. N., RAMRATTAN, M. A., SMORENBURG, S. M., GOUMA, D. J. & BOERMEESTER, M. A. 2008. The incidence and nature of in-hospital adverse events: a systematic review. Quality and Safety in Health Care, 17, 216-223. DEKKER, S. 2012. Drift into failure: from hunting broken components to understanding complex systems, Ashgate Publishing, Ltd. DEKKER, S. 2016. Patient Safety: A Human Factors Approach, CRC Press. DIXON-WOODS, M. & BOSK, C. 2010. Learning through observation: the role of ethnography in improving critical care. Curr Opin Crit Care, 16, 639-42. DIXON-WOODS, M., CAVERS, D., AGARWAL, S., ANNANDALE, E., ARTHUR, A., HARVEY, J., HSU, R., KATBAMNA, S., OLSEN, R., SMITH, L., RILEY, R. & SUTTON, A. 2006. Conducting a critical interpretive synthesis of the literature on access to health care by vulnerable groups. BMC Medical Research Methodology, 6, 35. DIXON-WOODS, M., MARTIN, G., TARRANT, C., BION, J., GOESCHEL, C., PRONOVOST, P., BREWSTER, L., SHAW, L., SUTTON, L. & WILLARS, J. 2014. Safer clinical systems: evaluation findings. DIXON-WOODS, M., SHAW, R. L., AGARWAL, S. & SMITH, J. A. 2004. The problem of appraising qualitative research. Quality and Safety in Health Care, 13, 223-225. DYREBORG, J. 2009. The causal relation between lead and lag indicators. Safety Science, 47, 474-475. EAGLE, C. J., DAVIES, J. M. & REASON, J. 1992. Accident analysis of large-scale technological disasters applied to an anaesthetic complication. Can J Anaesth, 39, 118-22. EDKINS, G. D. 1998. The INDICATE safety program: evaluation of a method to proactively improve airline safety performance. Safety Science, 30, 275-295. EMANUEL, L., BERWICK, D., CONWAY, J., COMBES, J., HATLIE, M., LEAPE, L., REASON, J., SCHYVE, P., VINCENT, C. & WALTON, M. 2008. What exactly is patient safety? In: HENRIKSEN, K., BATTLES, J. B., KEYES, M. A. & GRADY, M. L. (eds.) Advances in Patient Safety: New Directions and Alternative Approaches (Vol. 1: Assessment). Rockville: Agency for Health care Research and Quality. EMBREY, D. 2005. Understanding human behaviour and error. Human Reliability Associates, 1, 1-10. EXECUTIVES, H. A. S. 2006. Developing process safety indicators - A step-by-step guide for chemical and major hazard industries. In: EXECUTIVES, H. A. S. (ed.). Suffolk, United Kingdom. FEARNLEY, J. & NAIR, S. R. Determining process safety performance indicators for major accident hazards using site process hazard information. 2009. 221-225. FLIN, R. 2007. Measuring safety culture in health care: A case for accurate diagnosis. Safety science, 45, 653667. GOLAFSHANI, N. 2003. Understanding reliability and validity in qualitative research. The qualitative report, 8, 597-606. GRABOWSKI, M., AYYALASOMAYAJULA, P., MERRICK, J., HARRALD, J. R. & ROBERTS, K. 2007a. Leading indicators of safety in virtual organizations. Safety Science, 45, 1013-1043. GRABOWSKI, M., AYYALASOMAYAJULA, P., MERRICK, J. & MCCAFFERTY, D. 2007b. Accident precursors and safety nets: leading indicators of tanker operations safety. Maritime Policy & Management, 34, 405425.

90



GRECCO, C. H. D. S., VIDAL, M. C. R., COSENZA, C. A. N., SANTOS, I. J. A. L. & CARVALHO, P. V. R. 2014. Safety culture assessment: A fuzzy model for improving safety performance in a radioactive installation. Progress in Nuclear Energy, 70, 71-83. GRECCO, C. H. S., VIDAL, M. C. R., COSENZA, C. A. N., SANTOS, I. J. A. L. & CARVALHO, P. V. R. 2012a. New Approach for Safety Management in Radiopharmaceutical Production Facilities. Latin American Journal of Pharmacy, 31, 1199-1202. GRECCO, C. H. S. D., VIDAL, M. C. R., COSENZA, C. A. N., SANTOS, I. J. A. L. & CARVALHO, P. V. R. 2012b. New approach for safety management in radiopharmaceutical production facilities. Latin American Journal of Pharmacy, 31, 1199-1202. GROTE, G. 2009. Response to Andrew Hopkins. Safety Science, 47, 478. HAIGHT, J. M. & THOMAS, R. E. 2003. Intervention effectiveness research:: A review of the literature on leading indicators. Chemical Health & Safety, 10, 21. HALE, A. 2009. Special Issue on Process Safety Indicators. Safety Science, 47, 459-459. HARMS-RINGDAHL, L. 2009. Dimensions in safety indicators. Safety Science, 47, 481-482. HEALTH AND SAFETY EXECUTIVES 2006. Developing process safety indicators - A step-by-step guide for chemical and major hazard industries. Health and Safety Executives. HEALTH, D. O. 2000. An Organisation with a Memory: Learning from Adverse Events in the NHS. In: OFFICE, T. S. (ed.). London. HERRERA, I. & HOVDEN, J. Leading indicators applied to maintenance in the framework of resilience engineering: a conceptual approach. Paper presented at The 3rd Resilience Engineering Symposium, 2008. 30. HERRERA, I. A. 2012. Proactive safety performance indicators. HERRERA, I. A., HOLLNAGEL, E. & HÅBREKKE, S. 2010a. Proposing safety performance indicators for helicopter offshore on the Norwegian Continental Shelf. PSAM. Seattle. HERRERA, I. A., HOLLNAGEL, E. & HÅBREKKE, S. 2010b. Proposing safety performance indicators for helicopter offshore on the Norwegian Continental Shelf. HERRERA, I. A. & WOLTJER, R. 2010. Comparing a multi-linear (STEP) and systemic (FRAM) method for accident analysis. Reliability Engineering & System Safety, 95, 1269-1275. HINZE, J., THURMAN, S. & WEHLE, A. 2013. Leading indicators of construction safety performance. Safety Science, 51, 23-28. HOLLNAGEL, E. 2008. Safety Management - Looking back or looking forward. In: HOLLNAGEL, E., NEMETH, C. P. & DEKKER, S. (eds.) Resilience Engineering Perspectives: Remaining sensitive to the possibility of failure. Ashgate. HOLLNAGEL, E. 2009. The ETTO principle: efficiency-thoroughness trade-off: why things that go right sometimes go wrong, Ashgate Publishing, Ltd. HOLLNAGEL, E. 2012. FRAM: the Functional Resonance Analysis Method - Modelling Complex Socio-technical Systems, Surrey, England, Ashgate Publishing. HOLLNAGEL, E. 2013. Making Health Care Resilient: From Safety-I to Safety-II. In: HOLLNAGEL, E., BRAITHWAITE, J. & WEARS, R. L. (eds.) Resilient Health Care - Volume 1. Surrey, England: Ashgate Publishing. HOLLNAGEL, E. 2014a. Safety-I and Safety-II - The Past and Future of Safety Management, Surrey, England, Ashgate Publishing. HOLLNAGEL, E. 2014b. Safety-I and Safety-II - The Past and Future of Safety Management, Farnham, Ashgate. HOLLNAGEL, E. 2016a. A FRAM glossary [Online]. Middelfart: Hollnagel, E. Available: http://functionalresonance.com/how-to-build-a-fram-model/index.html [Accessed 24.04.17 2017]. HOLLNAGEL, E. 2016b. This page describes how a FRAM model is built, and how it can be used for either event analysis of risk asssessment [Online]. Middelfart: Hollnagel, E. Available: http://functionalresonance.com/how-to-build-a-fram-model/index.html [Accessed 24.04.17 2017].

91



HOLLNAGEL, E., BRAITHWAITE, J. & WEARS, R. L. 2013. Preface: On the Need for Resilience in Health Care. In: HOLLNAGEL, E., BRAITHWAITE, J. & WEARS, R. L. (eds.) Resilient Health Care. Farnham: Ashgate Publishing. HOLLNAGEL, E., WEARS, R. L. & BRAITHWAITE, J. 2015. From Safety-I to Safety-II: A White Paper. England. HOLLNAGEL, E. & WOODS, D. D. 2007. Epilogue - Resilience Engineering precepts. In: HOLLNAGEL, E., WOODS, D. D. & LEVESON, N. (eds.) Resilience Engineering: Concepts and Precepts. Hampshire: Ashgate Publishing, Limited. HOLLNAGEL, E., WOODS, D. D. & LEVESON, N. 2007. Resilience engineering: concepts and precepts, Ashgate Publishing, Ltd. HOPKINS, A. 2009. Thinking about process safety indicators. Safety Science, 47, 460-465. HOUNSGAARD, J. 2016. Patient Safety in Everyday Work, Learning from things that go right. Master Master Thesis, University of Southern Denmark. HUDSON, P. 2003. Applying the lessons of high risk industries to health care. Quality and safety in health care, 12, i7-i12. INSTITUTE, E. 2010. Human factors performance indicators for the energy and related process industries. In: INSTITUTE, E. (ed.). London: Energy Institute. JOHNSEN, S. O., OKSTAD, E., AAS, A. L. & SKRAMSTAD, T. 2012. Proactive Indicators To Control Risks in Operations of Oil and Gas Fields. JOHNSEN, S. O., OKSTAD, E., AAS, A. L. & SKRAMSTAD, T. 2013. Proactive indicators to control risks in operations of oil and gas fields. SPE Economics and Management, 4, 90-105. KAWULICH, B. B. 2005. Participant Observation as a Data Collection Method. 2005, 6. KENT, N. & FIELDS, W. 2012. Early recognition of sepsis in the emergency department: an evidence-based project. J Emerg Nurs, 38, 139-43. KHAN, F., ABUNADA, H., JOHN, D. & BENMOSBAH, T. 2010. Development of Risk-Based Process Safety Indicators. Process Safety Progress, 29, 133-143. KHAN, K. S., KUNZ, R., KLEIJNEN, J. & ANTES, G. 2003. Five steps to conducting a systematic review. Journal of the Royal Society of Medicine, 96, 118-121. KIRWAN, B. & AINSWORTH, L. 1992. A guide to task analysis. KNIJFF, P., ALLFORD, L. & SCHMELZER, P. 2013a. Process Safety Leading Indicators-A Perspective From Europe. Process Safety Progress, 32, 332-336. KNIJFF, P., ALLFORD, L. & SCHMELZER, P. Process safety leading indicators - A perspective from Europe. 2013b. 54-66. KOHN, L. T., CORRIGAN, J. M. & DONALDSON, M. S. 2000. To err is human: building a safer health system, National Academies Press. KONGSVIK, T., JOHNSEN, S. A. K. & SKLET, S. 2011. Safety climate and hydrocarbon leaks: An empirical contribution to the leading-lagging indicator discussion. Journal of Loss Prevention in the Process Industries, 24, 405-411. KRISTENSEN, S., MAINZ, J. & BARTELS, P. 2009. Selection of indicators for continuous monitoring of patient safety: recommendations of the project 'safety improvement for patients in Europe'. Int J Qual Health Care, 21, 169-75. KRISTENSEN, S., MAINZ, J., BARTELS, P. & ALLÉ, O. P. 2007. Establishing a Set of Patient Safety Indicators. Safety Improvements for Patients in Europe Working package, 4. KVALE, S. & BRINKMANN, S. 2009. Interview, Hans Reitzel. LATINO, R. J. 2015. How is the effectiveness of root cause analysis measured in health care? J Healthc Risk Manag, 35, 21-30. LAUGALAND, K., AASE, K. & WARING, J. 2014. Hospital discharge of the elderly--an observational case study of functions, variability and performance-shaping factors. BMC Health Serv Res, 14, 365. LEAPE, L. L. & BERWICK, D. M. 2000. Safe health care: are we up to it? BMJ, 320, 725-6.

92



LEAPE, L. L., BRENNAN, T. A., LAIRD, N., LAWTHERS, A. G., LOCALIO, A. R., BARNES, B. A., HEBERT, L., NEWHOUSE, J. P., WEILER, P. C. & HIATT, H. 1991. The Nature of Adverse Events in Hospitalized Patients. New England Journal of Medicine, 324, 377-384. LESLIE, M., PARADIS, E., GROPPER, M. A., REEVES, S. & KITTO, S. 2014. Applying ethnography to the study of context in health care quality and safety. BMJ Qual Saf, 23, 99-105. LEVER, A. & MACKENZIE, I. 2007. Sepsis: definition, epidemiology, and diagnosis. BMJ : British Medical Journal, 335, 879-883. LIPSITZ, L. A. 2012. Understanding health care as a complex system: the foundation for unintended consequences. JAMA, 308, 243-244. MAINZ, J. 2003a. Defining and classifying clinical indicators for quality improvement. Int J Qual Health Care, 15, 523-30. MAINZ, J. 2003b. Developing evidence-based clinical indicators: a state of the art methods primer. Int J Qual Health Care, 15 Suppl 1, i5-11. MALTERUD, K. 2001. Qualitative research: standards, challenges, and guidelines. Lancet, 358, 483-8. MANUELE, F. A. 2009. Leading & Lagging Indicators. Professional Safety, 54, 28-33. MANUELE, F. A. 2013. On the Practice of Safety, Wiley. MAXWELL, J. A. 2012. Qualitative research design: An interactive approach, Sage publications. MCLOUGHLIN, V., MILLAR, J., MATTKE, S., FRANCA, M., JONSSON, P. M., SOMEKH, D. & BATES, D. 2006. Selecting indicators for patient safety at the health system level in OECD countries. Int J Qual Health Care, 18 Suppl 1, 14-20. MEDICINE, D. U. S. O. 2010. Vocabulary associated with errors [Online]. Available: http://patientsafetyed.duhs.duke.edu/module_e/vocabulary.html [Accessed 04-01 2017]. MENGOLINI, A. & DEBARBERIS, L. 2008a. Effectiveness evaluation methodology for safety processes to enhance organisational culture in hazardous installations. Journal of hazardous materials, 155, 243252. MENGOLINI, A. & DEBARBERIS, L. 2008b. Effectiveness evaluation methodology for safety processes to enhance organisational culture in hazardous installations. Journal of Hazardous Materials, 155, 243252. MILES, M. B. & HUBERMAN, A. M. 1994. Qualitative data analysis: An expanded sourcebook, sage. MILLS, D. H. 1978. Medical Insurance Feasibility Study: A Technical Summary. Western Journal of Medicine, 128, 360-365. MOON, G. & HAMILTON, W. 2013. Developing an Organisational Integrity framework for nuclear safety. Cognition, Technology & Work, 15, 39-45. PALTRINIERI, N., ØIEN, K. & COZZANI, V. 2012. Assessment and comparison of two early warning indicator methods in the perspective of prevention of atypical accident scenarios. Reliability Engineering & System Safety, 108, 21-31. PATÉ-CORNELL, E. 2004. On signals, response, and risk mitigation. Accident Precursor Analysis and Management: Reducing Technological Risk through Diligence, 45. PATÉ‐CORNELL, M. E., LAKATS, L. M., MURPHY, D. M. & GABA, D. M. 1997. Anesthesia patient risk: a quantitative approach to organizational factors and risk management options. Risk Analysis, 17, 511523. PEERALLY, M. F., CARR, S., WARING, J. & DIXON-WOODS, M. 2016. The problem with root cause analysis. BMJ Qual Saf. PENCHEON, D. 2008. The Good Indicators Guide: Understanding how to use and choose indicators. Coventry: NHS Institute for Innovation and Improvement. PERCARPIO, K. B., WATTS, B. V. & WEEKS, W. B. 2008. The effectiveness of root cause analysis: what does the literature tell us? Jt Comm J Qual Patient Saf, 34, 391-8. PERROW, C. 2011. Normal Accidents: Living with High Risk Technologies, Princeton University Press. PICKUP, L., ATKINSON, S., HOLLNAGEL, E., BOWIE, P., GRAY, S., RAWLINSON, S. & FORRESTER, K. 2017. Blood sampling - Two sides to the story. Applied Ergonomics, 59, Part A, 234-242.

93



PLSEK, P. E. & GREENHALGH, T. 2001. The challenge of complexity in health care. British Medical Journal, 323, 625. PRAETORIUS, G., HOLLNAGEL, E. & DAHLMAN, J. 2015. Modelling Vessel Traffic Service to understand resilience in everyday operations. Reliability engineering & system safety, 141, 10-21. RAFTER, N., HICKEY, A., CONDELL, S., CONROY, R., O'CONNOR, P., VAUGHAN, D. & WILLIAMS, D. 2015. Adverse events in health care: learning from mistakes. QJM, 108, 273-7. RASMUSSEN, J. 1983. Skills, rules, and knowledge; signals, signs, and symbols, and other distinctions in human performance models. IEEE transactions on systems, man, and cybernetics, 257-266. REASON, J. T. 2008. The human contribution: unsafe acts, accidents and heroic recoveries, Ashgate Publishing, Ltd. REIMAN, T. & PIETIKÄINEN, E. 2010. Indicators of safety culture–selection and utilization of leading safety performance indicators. Stockholm: Strål säkerhets myndigheten. REIMAN, T. & PIETIKÄINEN, E. 2014. Patient Safety Indicators as Tools for Proactive Management and Safety Culture Improvement. In: WATERSON, P. (ed.) Patient Safety Culture: Theory, Methods and Application Farnham: Ashgate. RESEARCH, M. F. F. M. E. A. 2016. Sepsis - Symptoms and causes [Online]. Rochester, Minnesota: Mayo Clinic [Accessed 23.02.2016 2016]. REVIEWER, A. March 2017 2017. RE: Reliability Engineering & System Safety. Type to RABEN, C. SAMMER, C. E., LYKENS, K., SINGH, K. P., MAINS, D. A. & LACKAN, N. A. 2010. What is patient safety culture? A review of the literature. Journal of Nursing Scholarship, 42, 156-165. SCHMIDT, T. & WIIL, U. K. 2015. Identifying patients at risk of deterioration in the Joint Emergency Department. Cognition, Technology & Work, 17, 529-545. SHARPE, V. A. & FADEN, A. I. 1998. Medical Harm: Historical, Conceptual and Ethical Dimensions of Iatrogenic Illness, Cambridge University Press. SHEARER, B., MARSHALL, S., BUIST, M. D., FINNIGAN, M., KITTO, S., HORE, T., STURGESS, T., WILSON, S. & RAMSAY, W. 2012. What stops hospital clinical staff from following protocols? An analysis of the incidence and factors behind the failure of bedside clinical staff to activate the rapid response system in a multi-campus Australian metropolitan health care service. BMJ Quality & Safety, 21, 569575. SHOJANIA, K. G. & THOMAS, E. J. 2013. Trends in adverse events over time: why are we not improving? : BMJ Publishing Group Ltd. SILVERMAN, D. 2013. Doing qualitative research: A practical handbook, SAGE Publications Limited. SMITS, M., ZEGERS, M., GROENEWEGEN, P., TIMMERMANS, D., ZWAAN, L., VAN DER WAL, G. & WAGNER, C. 2010. Exploring the causes of adverse events in hospitals and potential prevention strategies. Quality and Safety in Health Care, 19, e5-e5. SONNEMANS, P., KÖRVERS, P. & PASMAN, H. 2010. Accidents in “normal” operation–Can you see them coming? Journal of Loss Prevention in the Process Industries, 23, 351-366. STAKE, R. E. 1995. The art of case study research, Sage. STANTON, N., SALMON, P. M. & RAFFERTY, L. A. 2013. Human Error Identification and Accident Analysis Methods. In: STANTON, N., SALMON, P. M. & RAFFERTY, L. A. (eds.) Human Factors Methods: A Practical Guide for Engineering and Design Ashgate. SUBBE, C. P., KRUGER, M., RUTHERFORD, P. & GEMMEL, L. 2001. Validation of a modified Early Warning Score in medical admissions. QJM, 94, 521-6. SUJAN, M. A., CHESSUM, P., RUDD, M., FITTON, L., INADA-KIM, M., COOKE, M. W. & SPURGEON, P. 2015. Managing competing organizational priorities in clinical handover across organizational boundaries. Journal of health services research & policy, 20, 17-25.

94



SUJAN, M. A. & FELICI, M. 2012. Combining Failure Mode and Functional Resonance Analyses in Health care Settings. Computer Safety, Reliability, and Security. Springer Berlin Heidelberg. SUJAN, M. A., HUANG, H. & BRAITHWAITE, J. 2016. Learning from incidents in health care: Critique from a Safety-II perspective. Safety Science. THOMAS, E. J. & PETERSEN, L. A. 2003. Measuring errors and adverse events in health care. Journal of general internal medicine, 18, 61-67. TOELLNER, J. 2001. IMPROVING SAFETY & HEALTH PERFORMANCE: Identifying & Measuring Leading Indicators. (Cover story). Professional Safety, 46, 42. TOMLINSON, C. M., CRAIG, B. N. & MEEHAN, M. J. Enhancing safety performance with a leading indicators program. 2011. 43-51. VAN DER VORM, J., VAN DER BEEK, D., BOS, E., STEIJGER, N., GALLIS, R. & ZWETSLOOT, G. 2011. Images Of Resilience: The Resilience Analysis Grid Applicable At Several Organizational Levels?, Paris: TRANSVALOR-Presses des MINES. VAN RITE, E. 2011. The challenge of patient safety and the remaking of American medicine. VINCENT, C. 2008. Patient safety, Wiley-Blackwell. VINCENT, C. 2010a. The emergence of patient safety. In: BMJ (ed.) Patient safety. Oxford: Churchill Livingstone Edinburgh. VINCENT, C. 2010b. Medical harm: a brief history. In: LIMITED, B. P. G. (ed.) Patient safety. Oxford: BMJ Publishing Group Limited. VINCENT, C. 2010c. The Nature and Scale of Error and Harm. Patient Safety. John Wiley & Sons, Ltd. VINCENT, C., BURNETT, S. & CARTHEY, J. 2013. The measurement and monitoring of safety. London: The Health Foundation, 2013. VINCENT, C., BURNETT, S. & CARTHEY, J. 2014. Safety measurement and monitoring in health care: a framework to guide clinical teams and health care organisations in maintaining safety. BMJ Qual Saf, 23, 670-7. WANG, H. E. & KATZ, S. 2007. Cognitive Control andPrehospital Endotracheal Intubation. Prehospital Emergency Care, 11, 234-239. WEARS, R. L., SUTCLIFFE, K. M. & VAN RITE, E. 2014. Patient Safety: A Brief but Spirited History. In: ZIPPERER, L. (ed.) Patient Safety: Perspectives on Evidence, Information and Knowledge Transfer. Surrey, England: Ashgate Publishing. WOLOSHYNOWYCH, M., ROGERS, S., TAYLOR-ADAMS, S. & VINCENT, C. 2005. The investigation and analysis of critical incidents and adverse events in health care. WOODS, D. D. 2006. Resilience engineering: Redefining the culture of safety and risk management. Human Factors and Ergonomics Society Bulletin, 49, 1-3. WOOLF, S. H., GROL, R., HUTCHINSON, A., ECCLES, M. & GRIMSHAW, J. 1999. Potential benefits, limitations, and harms of clinical guidelines. British Medical Journal, 318, 527. WREATHALL, J. 2009. Leading? Lagging? Whatever! Safety Science, 47, 493-494. WREATHALL, J. 2011. Monitoring - A Critical Ability in Resilience Engineering. In: HOLLNAGEL, E., PARIÉS, J., WOODS, D. D. & WREATHALL, J. (eds.) Resilience Engineering in Practice: A guidebook. Surrey, England: Ashgate. WU, A. W., LIPSHUTZ, A. K. & PRONOVOST, P. J. 2008. Effectiveness and efficiency of root cause analysis in medicine. JAMA, 299, 685-7. YIN, R. K. 2003. Case Study Research: Design and Methods, SAGE Publications. ØIEN, K., MASSAIU, S., TINMANNSVIK, R. K. & STØRSETH, F. Development of early warning indicators based on Resilience Engineering. Submitted to PSAM10, International Probabilistic Safety Assessment and Management Conference, 2010. 7-11. ØIEN, K., UTNE, I. B. & HERRERA, I. A. 2011a. Building Safety indicators: Part 1 - Theoretical foundation. Safety Science, 49, 148-161. ØIEN, K., UTNE, I. B. & HERRERA, I. A. 2011b. Building Safety indicators: Part 1 – Theoretical foundation. Safety Science, 49, 148-161.

95



ØIEN, K., UTNE, I. B., TINMANNSVIK, R. K. & MASSAIU, S. 2011c. Building Safety indicators: Part 2 - Application, practices and results. Safety Science, 49, 162-171. ØIEN, K., UTNE, I. B., TINMANNSVIK, R. K. & MASSAIU, S. 2011d. Building Safety indicators: Part 2 – Application, practices and results. Safety Science, 49, 162-171.

96



Appendix 1 Interview guide for early detection of sepsis (In Danish)

Spørgeguide til ’tidlig opsporing af sepsis’

1. Prøv at tale et typisk forløb igennem, fra du bliver opmærksom på patienten indtil patienten er i sepsis behandling. 2. Hvor ofte oplever du at nogle af dine patienter udvikler sepsis? 3. Hvad får dig til at undersøge en patient for sepsis? 4. Hvor ofte sker det at patienter der kommer på afdelingen udvikler svær sepsis eller septisk chok? 5. Hvordan påvirkes et forløb hvis der er travlt på afdelingen, eller mange patienter? 6. Er der noget der har særlig betydning for at du formår at opdage sepsis? F.eks. erfaring, kliniske faktorer, skøn. 7. Hvilke redskaber hjælper med enten at opdage sepsis eller sikre ordentlig behandling? 8. Hvilke ting kan påvirke den tid det tager dig at behandler en patient med? 9. Hvilken betydning har triagering for hvor opmærksom du er på om en patient kan udvikle sepsis? 10. Hvordan påvirker TOKS scoren din opmærksomhed på en patient i relation til sepsis? 11. Hvordan be- eller afkræfter du en sepsis diagnose? 12. Hvornår screenes sepsis patienter for svær sepsis? 13. Hvordan forholder det sig med septisk chok? Hvor ofte sker det og hvordan ser et typisk forløb ud? 14. Er der noget andet der er vigtigt at have med i forbindelse med at opspore og behandle sepsis patienter?

97



Part II – Thesis Papers Paper I Raben, D.C., Bogh, S.B. & Hollnagel, E. Suggesting an approach for developing leading indicators in health care - containing a review of the literature (Submitted) Paper II Raben, D.C., Viskum, B., Mikkelsen, K.L., Hounsgaard, J., Bogh, S.B. & Hollnagel, E. Application of a non-linear model to understand health care processes: using the functional resonance analysis method on a case study of the early detection of sepsis (Submitted - 3. Round of revision) Paper III Raben, D.C:, Viskum, B., Mikkelsen, K.L., Bogh, S.B. & Hollnagel, E. Learn from what goes right: a demonstration of a new systematic method for identification of leading indicators in health care (Submitted – 3. Round of revision) Paper IV Raben, D.C:, Viskum, B., Mikkelsen, K.L., Bogh, S.B. & Hollnagel, E. Proposing leading indicators for blood sampling - application of a method based on the principles of Resilient Health Care (Submitted)

98



Paper I Title: Suggesting an approach for developing leading indicators in health care – containing a review of the literature Status: Submitted to Journal for Health Organization and Management (Submitted 8th of June 2017) Names and affiliations of contributing authors: Ditte Caroline Raben, MScPH, Institute of Regional Health Research, University of Southern Denmark and Centre for Quality, Region of Southern Denmark, Denmark. Address: P.V. Tuxensvej 5, DK-5500 Middelfart, Denmark. E-mail: [email protected] Søren Bie Bogh, MHSc, Ph.d, Centre for Quality, Region of Southern Denmark, Denmark. Address: P.V. Tuxensvej 5, DK-5500 Middelfart, Denmark. E-mail: [email protected] Erik Hollnagel, Ph.D., Professor, Institute of Regional Health Research, University of Southern Denmark and Chief Consultant, Centre for Quality, Region of Southern Denmark, Denmark. Address: P.V. Tuxensvej 5, DK-5500 Middelfart, Denmark. E-mail: [email protected]

Word Count: 3136

99



Abstract Purpose and Background: The issue of patient safety is an important subject within health care. Focus has mainly been on a reactive approach, working with unwanted events in order to avoid them in the future. This paper looks at the use and development of leading indicators (LI), through a proactive approach. The aim of this paper was to investigate how different approaches for the identification of leading indicators can be applied in the context of health care. The article introduces special characteristics of health care along with different approaches to identify leading indicator. Based on this it considers how these characteristics are best met and respected in the identification of leading indicators. Methods: We present knowledge derived through performing a systematic literature review on leading indicators. Different approaches for developing leading indicators in health care and other high-risk industries are presented and analysed with reference to special characteristics of health care. Findings: A variety of frameworks are presented including Resilience Based Early Warning Indicators, Dual Assurance Method, Functional Resonance Analysis Method. Main findings and common characteristics are summed-up and analysed. Value: Based on the understanding of complexity in health care, we propose the concept of Resilient Health Care to develop leading indicators in a health care context. Application: The results of this work are applied as a fundament for developing a new method for the identification of leading indicators. This method is developed with consideration for characteristics of health care and learnings from previous method developed in high-risk industries.

100



Introduction Indicators are a common tool in health care, especially in relation to measuring quality and safety initiatives (Mainz, 2003, Kristensen et al., 2009). They are often described as vital for tracking performance, safety or quality of care (Mainz, 2003, Kristensen et al., 2009, Campbell et al., 2002). However, the most widely used methods for identifying safety indicators are based on a reactive management, characterized by responding to something that either has gone wrong or identified as a risk, that could potentially cause harm (Hollnagel, 2013). This becomes evident as patient safety indicators are reviewed and include indicators like wound infections, mortality rates, complications of surgery, post-operative sepsis, adverse drug events and wrong blood type (Kristensen et al., 2009). These are categorized as being patient safety indicators, yet they represent events happening in the absence of safety. In this article, we present an argument of why representations of past events, may not be enough in order to manage a system as health care. We argue that the usefulness of indicators could be improved, if they could help indicate future events, through the application of a proactive management towards safety. The essence of proactive safety management is to create adjustments prior to potential unwanted happenings, by creating a greater understand of what factors exceed both unwanted and wanted events (Hollnagel et al., 2015). Other high-risk industries like oil and gas, aviation and nuclear have been frontrunners on promoting the proactive management especially towards safety. This has resulted in a distinction between indicators were they are either defined as lagging or leading indicators (Hale, 2009). Lagging indicators are measures, which describe events, which have taken place, like the ones often applied in patient safety. On the contrary leading indicators are used as precursors of incidence or undesirable events (Hale, 2009). Historically the desire to apply and develop leading indicators was influenced by unwanted disastrous events. In high-risk industries events as the Piper Alpha oil platform accident, the Texas City Refinery Explosion and the Deepwater Horizon blowout, to name a few, have resulted in the need to develop new ways to avoid major accidents (Bergh et al., 2014, Øien et al., 2011b, Broadribb et al., 2009, Vincent, 2008). Despite the fact that health care is faced with similar challenges regarding safety, the application of leading indicators is still novel here where a review of leading indicators showed few attempts have been made to identify such indicators (Braithwaite et al., 2015, Reiman and Pietikäinen, 2014, Birkmeyer et al., 2013, Paté‐Cornell et al., 1997). High-risk industries have provided us with various methods to identify leading indicators and analyzing these 101



methods can provide a valuable background for how to apply the concept of leading indicators in health care (Øien et al., 2011b). Therefore, the aim of this paper is to understand the characteristics of health care and investigate whether experiences from high-risk industry can give guidance on which theoretical framework is suitable for identifying leading indicators in health care (Hudson, 2003). The complexity of health care Reflecting on the usefulness of approaches developed in high-risk industries, requires a wellestablished understanding of important features and particularities of health care. For this purpose, describing complexity in health care gives an understanding of many of these features (Lipsitz, 2012). Health care consists of a collection of individual agents, which have the freedom to act in ways that are not always predictable and whose actions can change the context and conditions for others (Plsek and Greenhalgh, 2001, Rouse, 2008, Braithwaite et al., 2013). In contrast to mechanical systems, which are often well-defined with fixed and specific boundaries, health care includes changing memberships, and members often being part of several different systems (Plsek and Greenhalgh, 2001). Further, health care is dependent on humans and human behaviour, which is not always predictable or traceable. Therefore, actions in health care are often based on internalized rules, which can be expressed as instincts, constructs or mental models (Plsek and Greenhalgh, 2001, Rasmussen, 1983, Lipsitz, 2012, Braithwaite et al., 2013). This creates a challenge when trying to understand why things fail, as internalized rules rarely are visible, explicit or even logical for others (Plsek and Greenhalgh, 2001). Furthermore, health care must be able to anticipate, respond to, learn and adapt to changing conditions over time (Braithwaite et al., 2013, Plsek and Wilson, 2001). This adaptive nature of the system can lead to the emergence of new and novel behaviours. The outcome can be difficult to reproduce, track or understand, as it evolves as a result of interaction among agents (Plsek and Greenhalgh, 2001, Lipsitz, 2012). In the pursuit to understand or map the health care system, further challenges emerge. Health care is highly complex, both as a whole, but also as it is a combination of many different systems on several levels (Braithwaite et al., 2013). Often agents are linked to several levels of the system and must sometimes handle competing demands, different internalized rulesets and answer to several agendas (Plsek and Greenhalgh, 2001). A contributing factor to this challenge is the lack of linearity of health care. This means that it is difficult to anticipate future events, if the understanding of the system is

102



based on a linear understanding and events actually happen in a non-linear way (Plsek and Greenhalgh, 2001, Lipsitz, 2012). Application of leading indicators in high-risk industries Reviewing the literature of leading indicators in high-risk industries revealed a developed, but also a fragmented research field. An initial challenge emerged as the term ‘leading indicator’ is defined and used differently between industries, and much of the knowledge is not published scientifically, adding to the challenge of finding all relevant knowledge. Terms include precursors, proactive, leading, early warning or process indicators. Nonetheless, there are some common characteristics across the literature. Several industries have applied the concepts of leading indicators, with most findings in industries like oil & gas, chemical, aviation and nuclear (Øien et al., 2011a, Øien et al., 2011b). Traditionally these industries measured safety with ‘after-the-loss’ measures. Repeating major accidents, however, lead to a growing consensus that these measures do not provide necessary insights to avoid future accidents (Grabowski et al., 2007). This resulted in the development of leading indicators as they could contribute to measure and understand causes and contributing factors to accidental events (Øien et al., 2011b). The aim was to identify important factors, and have continuing measurements of these factors alert the system of possible occurrence of unwanted events, and thereby react proactively. After reviewing the literature, we have extracted some of the main tendencies in developing leading indicators, and presented them below. Grabowski et al., (2007) presented an approach for developing leading indicators in marine transportation. Using a quantitative approach the study found candidate indicators based on previously identified safety factors earlier associated with high levels of organizational safety performance. These potential safety factors were combined with safety performance to investigate correlations and validate leading indicators (Grabowski et al., 2007). A possible limitation of the study is that candidate indicators were based on previous studies and it was not reported how they are were found or selected (Grabowski et al., 2007). This inhibits the possibility to consider whether all potential factors were discovered with this method. Combining the study with a qualitative approach could have contributed with uncovering other safety factors. In comparison, Johnsen et al. (2013) presented a qualitative approach to identify indicators for oil and gas fields. Applying the term proactive indicators, but referencing to previous definitions of leading indicators and highlighting the focus of avoiding accidents by early warnings. The paper presented a five-step research design, using a double-loop safety learning approach to increase understanding of indicators. Data included 103



accident reports, description of successful recoveries, interviews, observations of processes and identification of indicators. Validation of indicators is suggested at a later stage, when the method is described and refined (Johnsen et al., 2012). Other studies applying a qualitative research design have also used a theoretical foundation for the development of leading indicators. These studies based the development of leading indicators on the concept of resilience engineering. Resilience engineering is a theoretical concept often applied in safety management for high-risk industry, and relates to a system’s ability to adapt to changing conditions (Hollnagel et al., 2007). Further, resilience engineering emphasizes, that in order to improve safety, focus should be divided between looking at accidents and looking at improving the conditions of the system (Hollnagel, 2008). Thus, Herrera et al. (2008) used the principles of resilience engineering to develop leading indicators for safety performance for offshore helicopter operations. They applied the Functional Resonance Analysis Method (FRAM), founded on resilience engineering, with the Risk Influencing Model, and lessons learned from previous studies, to identify leading indicators (Herrera et al., 2010, Herrera and Hovden, 2008). Additionally, Øien et al. (2010) conducted research related to oil and gas and presented the Resilience based Early Warning Indicator (REWI) method. Firstly, the paper identified eight attributes of a resilient organization. Secondly, a set of general issues for the contributing attributes were suggested, and thirdly, potential new issues were included through workshop sessions where indicators were selected. The article concluded that it is possible to develop ‘an indicator system’ based on resilience engineering theory, but as the article primarily presents the method rather than a case, it is difficult to assess the usability of the method (Øien et al., 2010). At a later stage the REWI approach was also applied by Paltrineri et al. (2012) and compared it to the ‘Dual Assurance’ method, developed by Health and Safety Executives (HSE) (Paltrinieri et al., 2012). HSE published a guide for developing process safety indicators (Executives, 2006), presenting a step-by-step guide with six steps towards identifying a set of lagging and leading indicators, guiding on aspects like, organizational level, scope of measurements, evaluating current risk controls, tolerance levels, collecting and reporting data and review of indicators (Executives, 2006). Paltrineri et al. (2012) found that both the REWI method and the Dual Assurance method by HSE were able to identify leading indicators, with potential to avoid a major accident. It further showed that using both approaches gave a better and more specific set of indicators than either one could give alone (Paltrinieri et al., 2012).

104



Khan et al. conducted a study in 2010 applying the six steps of HSE as they found that the nuclear industry had no unified approach concerning indicators, identifying a missing coherence, quantification, audibility and logical integration of both leading and lagging indicators. They concluded, that the HSE approach, could be used to model process safety performance, but authors highlight the importance of further testing and implementing by others (Khan et al., 2010). After reviewing different industries and various approaches, methods and guidelines towards identifying leading indicators some characteristics have been detected. The definition of leading indicators differs, not only in wording, but also in meaning. A part of the studies reviewed defined leading indicators as precursors of unwanted events, focusing on hazards, failings, accidents and undesirable outcomes (Johnsen et al., 2012, Grabowski et al., 2007). Others have a more holistic approach towards safety, and therefore leading indicators. These studies view leading indicators not only as precursors of unwanted events but also as predictors of future conditions (Paltrinieri et al., 2012, Herrera et al., 2010, Herrera and Hovden, 2008, Khan et al., 2010, Executives, 2006). Consensus was neither found regarding applied research methods towards identifying leading indicators. Secondly, some studies presented a complete quantitative approach (Grabowski et al., 2007), where others focused on a qualitative approach (Johnsen et al., 2012), and finally, others used mixed-methods to identify and validate indicators (Herrera et al., 2010). Despite differences in definitions and methodologies and frameworks, some common traits were identified. All studies approached the identification of leading indicators through (1) presenting an overall framework or theoretical background for the identification, (2) using different tools towards the understanding of the system investigated, (3) identifying potential leading indicators either through potential factors previously applied (Herrera et al., 2010, Grabowski et al., 2007, Øien et al., 2010) or through data collection (Johnsen et al., 2012, Khan et al., 2010, Executives, 2006), and finally (4) all studies included some sort of involvement from staff or experts, either during the identification process (Herrera et al., 2010, Johnsen et al., 2012)or in the validation and confirmation process (Øien et al., 2010, Grabowski et al., 2007, Khan et al., 2010). This comparison of the presented study underlined how experience prove the importance of applying a systematic and generic approach in the identification of indicators. Application of leading indicators in health care Extensive systematic searches across health care related databases and journals revealed no published literature using the term ‘leading indicator’. The idea of identifying relevant factors as precursors has 105



been attempted and proposed in health care, but no systematic way of doing it has been considered or developed (Birkmeyer et al., 2013, Paté‐Cornell et al., 1997). Further, the work found focused on developing context specific indicators rather than considering how these indicators are actually extracted from a given process (Øien et al., 2011b). In the literature, the concept of safety culture has been suggested as a leading indicator for safety performance and may affect the state of safety in a positive way (Flin, 2007, Sammer et al., 2010, Singer and Vogus, 2013, Birkmeyer et al., 2013). Results of studies are mixed. Some suggest that safety culture is a factor affecting performance in hospitals were other suggest that relying on safety culture alone cannot be applied as an isolated intervention, because it is unlikely to reduce the underlying causes of hospital errors (Sammer et al., 2010). The same study further underlines the importance of developing systematic interventions addressing interrelated processes of the system (Sammer et al., 2010). Others argue that safety culture is still not properly adopted and applied in health care, and calls for investigating the link between safety culture and staff behaviour and strengthening the accuracy of safety culture questionnaires (Flin, 2007). Other research suggested using Probabilistic Risk Analysis (PRA) as a tool. PRA has on previous occasions been applied to detect and analyze precursors to unwanted events in high-risk industries (Paté‐Cornell, 1986). In this case, PRA was used to link different accidents of anesthesia patients to a variety of root causes described as ‘the state of the anesthesiologist’ covering personal factors like competence, alertness, fatigue, suitability, distraction, abuse, aging, lack of training or supervision, along with organizational factors like staffing or busyness (Paté‐Cornell et al., 1997). The work identified a link between the two factors, and suggested these factors as leading indicators for accidents and that monitoring residence will improve patient safety (Paté‐Cornell et al., 1997). Finally, the search uncovered a proposition on patient safety indicators applying a proactive (Reiman and Pietikäinen, 2014). The work considered the distinction of leading and lagging indicators, and presented a theoretical framework for using safety performance indicators in safety-critical organizations (Reiman and Pietikäinen, 2014). Within this framework, the authors presented a number of necessary functions for achieving good safety culture. The authors presented a broader approach, connected to safety on a high management level (Reiman and Pietikäinen, 2014). Proposing a resilience engineering framework for leading indicators in health care

106



At this stage, common characteristics describing the complexity of health care have been presented. Further, we analyzed the work on leading indicators in patient safety and compared it to applications of leading indicators in high-risk industries. High-risk industries are often used as a reference in the development of safety management and we therefore propose applying the concept of Resilient Health Care, based on resilience engineering from high-risk industries, as it favors many of the traits characterizing health care (Hollnagel, 2008). Health care differs from high-risk industries on a number of aspects. This has also fostered a discussion of some of the tools or methods developed in high-risk industries and applied in health care (Hudson, 2003, Peerally et al., 2016, Card, 2016). Discussions have evolved around whether these methods can be directly translated into health care without careful consideration to the changing context. Firstly, many high-risk industries are centralized with a clear control structure, where health care is often a much more fragmented and decentralized in comparison (Vincent, 2008). This difference makes it difficult to regulate and standardize in the same degree in health care, where these standards or in this case indicators should be much more case specific, with consideration to the context and process they should fit into. Secondly, health care further needs to be much more adaptable towards specific and different cases. High-risk industries are often working with systems or products, which are highly technical, and the social aspect is primarily represented by staff working with technical components (Hudson, 2003). In health care, staff are faced with patients, which all react differently. This fosters health care to have a higher tolerance for uncertainty, as patients conditions may be masked, difficult to diagnose, treatment complicated by many factors as comorbidities, preferences and personal or social factors (Vincent, 2008). The concepts of Resilient Health Care accommodate many of the needs of the system as it acknowledges the importance of building a health care system with the ability to respond to unanticipated events, and fostering people to adapt to changing conditions and surroundings (Braithwaite et al., 2015, Braithwaite et al., 2013). Resilient Health Care is an approach fit to manage risk in a proactive way, by providing a framework that supports coping with complexity under pressure and still achieve success (Hollnagel et al., 2007). We present how methods development within the concept of Resilient Health Care can help identify leading indicators. Health care is a system of fuzzy boundaries, with agents often part of more than one system (Plsek and Greenhalgh, 2001). Resilient Health Care acknowledges and supports this characteristic, as initiatives based on resilience highlight the importance of defining the system boundary under 107



investigation (Braithwaite et al., 2013). Hence, methods applying the concept of Resilient Health Care will require mapping the investigated system before going forward (Øien et al., 2010, Herrera et al., 2010). This is possible by applying the previously presented FRAM, as the first step is to produce a representation of an activity or system in terms of the functions needed to carry out the activities and how they are coupled (Hollnagel, 2012). Such a visual model makes it possible to view activities, which are a premise of success, even if they are carried out on different levels of the system. FRAM can therefore function as a starting point for developing a systematic model of the system and potential help identify leading indicators of success (Herrera et al., 2010). Next, we identified how health care is highly influenced and dependent on human behaviour, and how this sometimes makes it challenging to understand and trace actions (Plsek and Greenhalgh, 2001). We also determined that health care is adaptive and must improve its ability to cope with variation, as this can often be a reason for performing well (Plsek and Wilson, 2001). In the mindset of Resilient Health Care, opposed to many other safety approaches, human behaviour is seen as a strength rather than a weakness of the system (Hollnagel, 2013). People are constantly learning during their work, and quick at discovering performance adjustments, which can be viewed as actual leading indicators of success. Therefore, to identify leading indicators it is necessary to understand performance adjustments, the reasons for them and how they contribute to successful outcomes (Hollnagel, 2013). Besides offering an understanding of successful outcomes, Resilient Health Care also suggests reasons why people in health care behave as they do (Plsek and Greenhalgh, 2001). Like FRAM is able to create an understanding of how the system looks, it is also able to create a model of the potential variations and adaptions of behaviour, which are a result of personal trade-offs (Hollnagel, 2012). Adaptation has previously been stressed as a crucial factor in any social system, arguing that organizational or human failures are caused by breakdowns in these necessary adaptions (Woods, 2000) and therefore sometimes these adaptions may be leading indicators of success. By identifying and understanding them, it might be possible to change or modify processes in the system, making them an intrinsic part of the system to embrace and permit these adaptions. This further argues for the case of investigating variations and adaptions in order to understand what behaviour might be relevant to dampen or amplify (Hollnagel et al., 2015). Conclusion Based on the high-risk industries we have found and considered a variety of approaches to identifying leading indicators. We have combined these experiences with the understanding of the complexity of

108



health care. On this background, we have illustrated how the concept of Resilient Health Care can function as a fundament for identifying leading indicators. We have suggested that applying the concept of Resilient Health Care for leading indicators is meaningful as many of identified characteristics of the health care system are considered in this concept. References: BERGH, L. I. V., HINNA, S., LEKA, S. & JAIN, A. 2014. Developing a performance indicator for psychosocial risk in the oil and gas industry. Safety Science, 62, 98-106. BIRKMEYER, N. J., FINKS, J. F., GREENBERG, C. K., MCVEIGH, A., ENGLISH, W. J., CARLIN, A., HAWASLI, A., SHARE, D. & BIRKMEYER, J. D. 2013. Safety culture and complications after bariatric surgery. Annals of surgery, 257, 260-265. BRAITHWAITE, J., CLAY-WILLIAMS, R., NUGUS, P. & PLUMB, J. 2013. Health care as a complex adaptive system. In: HOLLNAGEL, E., BRAITHWAITE, J. & WEARS, R. L. (eds.) Resilient Health Care - Volume 1. Surrey, England: Ashgate Publishing. BRAITHWAITE, J., WEARS, R. L. & HOLLNAGEL, E. 2015. Resilient health care: turning patient safety on its head. Int J Qual Health Care, 27, 418-20. BROADRIBB, M. P., BOYLE, B. & TANZI, S. J. 2009. Cheddar or Swiss? How Strong Are Your Barriers? Process Safety Progress, 28, 367-372. CAMPBELL, S. M., BRASPENNING, J., HUTCHINSON, A. & MARSHALL, M. 2002. Research methods used in developing and applying quality indicators in primary care. Quality and Safety in Health Care, 11, 358-364. CARD, A. J. 2016. The problem with '5 whys'. BMJ Qual Saf. EXECUTIVES, H. A. S. 2006. Developing process safety indicators - A step-by-step guide for chemical and major hazard industries. In: EXECUTIVES, H. A. S. (ed.). Suffolk, United Kingdom. FLIN, R. 2007. Measuring safety culture in health care: A case for accurate diagnosis. Safety science, 45, 653-667.

109



GRABOWSKI, M., AYYALASOMAYAJULA, P., MERRICK, J., HARRALD, J. R. & ROBERTS, K. 2007. Leading indicators of safety in virtual organizations. Safety Science, 45, 1013-1043. HALE, A. 2009. Special Issue on Process Safety Indicators. Safety Science, 47, 459-459. HERRERA, I. & HOVDEN, J. Leading indicators applied to maintenance in the framework of resilience engineering: a conceptual approach. Paper presented at The 3rd Resilience Engineering Symposium, 2008. 30. HERRERA, I. A., HOLLNAGEL, E. & HÅBREKKE, S. 2010. Proposing safety performance indicators for helicopter offshore on the Norwegian Continental Shelf. PSAM. Seattle. HOLLNAGEL, E. 2008. Safety Management - Looking back or looking forward. In: HOLLNAGEL, E., NEMETH, C. P. & DEKKER, S. (eds.) Resilience Engineering Perspectives: Remaining sensitive to the possibility of failure. Ashgate. HOLLNAGEL, E. 2012. FRAM: the Functional Resonance Analysis Method - Modelling Complex Socio-technical Systems, Surrey, England, Ashgate Publishing. HOLLNAGEL, E. 2013. Making Health Care Resilient: From Safety-I to Safety-II. In: HOLLNAGEL, E., BRAITHWAITE, J. & WEARS, R. L. (eds.) Resilient Health Care - Volume 1. Surrey, England: Ashgate Publishing. HOLLNAGEL, E., WEARS, R. L. & BRAITHWAITE, J. 2015. From Safety-I to Safety-II: A White Paper. England. HOLLNAGEL, E., WOODS, D. D. & LEVESON, N. 2007. Resilience engineering: concepts and precepts, Ashgate Publishing, Ltd. HUDSON, P. 2003. Applying the lessons of high risk industries to health care. Quality and safety in health care, 12, i7-i12. JOHNSEN, S. O., OKSTAD, E., AAS, A. L. & SKRAMSTAD, T. 2012. Proactive Indicators To Control Risks in Operations of Oil and Gas Fields. KHAN, F., ABUNADA, H., JOHN, D. & BENMOSBAH, T. 2010. Development of Risk-Based Process Safety Indicators. Process Safety Progress, 29, 133-143.

110



KRISTENSEN, S., MAINZ, J. & BARTELS, P. 2009. Selection of indicators for continuous monitoring of patient safety: recommendations of the project 'safety improvement for patients in Europe'. Int J Qual Health Care, 21, 169-75. LIPSITZ, L. A. 2012. Understanding health care as a complex system: the foundation for unintended consequences. JAMA, 308, 243-244. MAINZ, J. 2003. Defining and classifying clinical indicators for quality improvement. Int J Qual Health Care, 15, 523-30. PALTRINIERI, N., ØIEN, K. & COZZANI, V. 2012. Assessment and comparison of two early warning indicator methods in the perspective of prevention of atypical accident scenarios. Reliability Engineering & System Safety, 108, 21-31. PATÉ‐CORNELL, M. E. 1986. Warning systems in risk management. Risk analysis, 6, 223-234. PATÉ‐CORNELL, M. E., LAKATS, L. M., MURPHY, D. M. & GABA, D. M. 1997. Anesthesia patient risk: a quantitative approach to organizational factors and risk management options. Risk Analysis, 17, 511-523. PEERALLY, M. F., CARR, S., WARING, J. & DIXON-WOODS, M. 2016. The problem with root cause analysis. BMJ Qual Saf. PLSEK, P. E. & GREENHALGH, T. 2001. The challenge of complexity in health care. British Medical Journal, 323, 625. PLSEK, P. E. & WILSON, T. 2001. Complexity, leadership, and management in health care organisations. BMJ, 323, 746-749. RASMUSSEN, J. 1983. Skills, rules, and knowledge; signals, signs, and symbols, and other distinctions in human performance models. IEEE transactions on systems, man, and cybernetics, 257-266. REIMAN, T. & PIETIKÄINEN, E. 2014. Patient Safety Indicators as Tools for Proactive Management and Safety Culture Improvement. In: WATERSON, P. (ed.) Patient Safety Culture: Theory, Methods and Application Farnham: Ashgate.

111



ROUSE, W. B. 2008. Health care as a complex adaptive system: implications for design and management. Bridge-Washington-National Academy of Engineering-, 38, 17. SAMMER, C. E., LYKENS, K., SINGH, K. P., MAINS, D. A. & LACKAN, N. A. 2010. What is patient safety culture? A review of the literature. Journal of Nursing Scholarship, 42, 156-165. SINGER, S. J. & VOGUS, T. J. 2013. Reducing hospital errors: interventions that build safety culture. Annual review of public health, 34, 373-396. VINCENT, C. 2008. Patient safety, Wiley-Blackwell. WOODS, D. D. 2000. Behind human error: Human factors research to improve patient safety. Washington, DC: American Psychological Association. ØIEN, K., MASSAIU, S., TINMANNSVIK, R. K. & STØRSETH, F. Development of early warning indicators based on Resilience Engineering. Submitted to PSAM10, International Probabilistic Safety Assessment and Management Conference, 2010. 7-11. ØIEN, K., UTNE, I. B. & HERRERA, I. A. 2011a. Building Safety indicators: Part 1 - Theoretical foundation. Safety Science, 49, 148-161. ØIEN, K., UTNE, I. B., TINMANNSVIK, R. K. & MASSAIU, S. 2011b. Building Safety indicators: Part 2 - Application, practices and results. Safety Science, 49, 162-171.

112



Paper II Title: Application of a non-linear model to understand health care processes: using the functional resonance analysis method on a case study of the early detection of sepsis Status: Currently in Revision at Reliability Engineering & System Safety (3. Round of revision submitted May 12th 2017) Names and affiliations of contributing authors: Ditte Caroline Raben, MScPH, Institute of Regional Health Research, University of Southern Denmark and Centre for Quality, Region of Southern Denmark, Denmark. Address: P.V. Tuxensvej 5, DK-5500 Middelfart, Denmark. E-mail: [email protected] Birgit Viskum, Consultant, Public Health Medicine, Aborgmindevej 9, 5610 Assens, E-mail: [email protected] Kim L. Mikkelsen, Ph.d., Medical Coordinator, Danish Patient Insurance Associtation, Address: Kalvebod Brygge 45, 4.sal, 1560 København V, Danmark E-mail:[email protected] Jeanette Hounsgaard, MPQM, Centre for Quality, Region of Southern Denmark, Address: P.V. Tuxensvej 5, DK-5500 Middelfart, Denmark. E-mail: [email protected] Søren Bie Bogh, MHSc, Institute of Regional Health Research, University of Southern Denmark and Centre for Quality, Region of Southern Denmark, Denmark. Address: P.V. Tuxensvej 5, DK-5500 Middelfart, Denmark. E-mail: [email protected] Erik Hollnagel, Ph.D., Professor, Institute of Regional Health Research, University of Southern Denmark and Chief Consultant, Centre for Quality, Region of Southern Denmark, Denmark. Address: P.V. Tuxensvej 5, DK-5500 Middelfart, Denmark. E-mail: [email protected] Word count: 4608

113



Abstract The use of non-linear models to understand complex processes in health care is not a fully adopted concept. Current patient safety research focuses on events by studying adverse events, typically trying to understand the root causes of failures. This article describes an attempt in a Danish hospital to create an understanding of how complex processes produce positive outcomes despite variability and unforeseen factors, using the Functional Resonance Analysis Method (FRAM) to describe a frequent activity in health care: early detection of sepsis. The model presents 40 activities performed by nurses, doctors, secretaries, health workers and laboratory technicians; and illustrates possible and actual variability in the process. The results reveal that the application of FRAM helped to gain a heightened understanding of a complex health care process. The FRAM provided new insights to staff by focusing on aspects that previously had not been central when working with the patient safety during sepsis detection. This included aspects such as becoming aware of the importance of asking the right questions during the referral process from a general practitioner, using experience and clinical judgement during early assessment of patients and the importance of having a good collegial relationship between doctors and nurses. The method helped reveal how the process is often able to succeed despite variability, and how aspects like experience and clinical judgement play a vital role in adapting to everyday conditions. This knowledge can enhance the understanding of how complex processes develop and be useful in supporting their management and improving patient safety.

1. Background

114



Understanding and constantly improving the quality of complex processes in health care is an area of major importance (Vincent, 2008). However, to improve the way health care processes work and thereby improve the quality and safety of hospital care, it is crucial to develop a thorough understanding of what goes on in these processes and how variation occurs (Sujan et al., 2016). Certain aspects of health care can be characterised as a complex adaptive system (CAS) that exists in changeable and unpredictable settings (Braithwaite et al., 2013). Health care must balance many, often conflicting demands (Sujan et al., 2015). The fact that health care is characterised by high complexity implies that unwanted events may not be avoided by solely implementing more standards and rules or by investigating and describing every adverse event that occurs (Braithwaite et al., 2013). Viewing health care as a CAS can help reveal connections and relations, which models taking a linear approach might not be able to uncover. Ensuring patient safety tends to apply a reactive approach towards managing processes safely, based on the assumption that adverse events are caused by a linear and describable chain of events, where focus is on investigating adverse events and understanding root causes of these events (Vincent, 2010, Sujan, 2015). Ideally, the use of these methods and approaches should trigger improvements in the system that result in enhanced safety states (Sujan et al., 2016). However, studies suggest that 1 in 10 patients admitted to hospital will still experience an incident (Sujan et al., 2016). This knowledge suggests a gap in the way that adverse events or incidents are being processed for improvement or learning. Therefore, the foundation of the method applied in this study differs from previously used methods within patient safety. Instead of focusing on the root causes of unwanted events, or claims or error reporting with respect to past events (Vincent, 2010, Sujan et al., 2016), the method in this study focuses on understanding everyday performance (Hollnagel, 2012). Health care deals with a variety of factors including patients, different specialised settings, a variety of professions and changing working conditions (Herrera and Woltjer, 2010). This paper acknowledges that dealing with these factors requires continuous adjustment to ensure overall best performance and that health care provision functions under many different conditions and situations. However, few studies have been devoted to understanding performance variability and describing how this contributes to maintaining an effective and safe health care system (Sujan and Felici, 2012, Laugaland et al., 2014).

115



This study aims to understand how a health care system works on an everyday basis by using a systematic method, the Functional Resonance Analysis Method (FRAM) (Herrera and Woltjer, 2010). The study investigates the process of early detection of sepsis in a medical ward in a Danish hospital by applying the FRAM; this method has been shown to provide a good understanding of systems and how variability over time can develop into unanticipated events. The FRAM is based on system theory and was developed to understand systems, their complexity and performance adjustments (Hollnagel, 2012). The FRAM will result in a model of the process, which includes a visual representation of all activities connected to the early detection of sepsis. It is anticipated that using FRAM to investigate processes will provide new perspectives on important aspects of the early detection of sepsis applicable to the improvement of quality and safety. As the field of patient safety is dominated by linear models, this study investigates whether the application of a non-linear model like FRAM will add new and different insights to the early detection of sepsis, which is commonly the subject of patient safety improvement initiatives. In the following section, the research method is presented along with the investigated case, followed by a description of FRAM. Section 3 includes the resulting FRAM model that represents the investigated process via four themes. Section 4 includes a discussion of how the FRAM contributes to the understanding of complexity and performance variability. Finally, section 5 summarises the findings and implications for future research. 2. Methods A multiple-case study was conducted of the early detection of sepsis, a frequent and complex process in health care (Yin, 2003). Sepsis is a common condition in health care and may cause severe harm or death in patients (Lever and Mackenzie, 2007). The particular emphasis of the case study was upon understanding the early detection of sepsis and identifying how this process could vary under different conditions. To gather all the information on the selected process necessary to construct the FRAM model, an ethnographic approach was applied in the data collection, which included document reviews, interviews and participant observations through informal interviews (Dixon-Woods and Bosk, 2010). This approach allowed the researcher to observe the process several times and understand the different aspects that may influence or cause variability within the process (Dixon-Woods and Bosk, 2010).

116



2.1. Setting and case The process was investigated within a Danish hospital setting by collecting data in the acute visitation ward (AVW), which is part of the medical ward and emergency room (ER). These wards are part of a medical department with 66 beds and 5500 annual patient admissions. They were selected as they frequently receive patients suspected for sepsis, and had previously contributed to the research topic via implementation of a sepsis checklist. In cooperation with the hospital, observations primarily took place in the medical ward, which had a high frequency of patients with sepsis. The process was further investigated in the ER to gain an understanding of the process for patients admitted there and later transferred to the AVW. According to Lever and Mackenzie (Lever and Mackenzie, 2007) sepsis is defined as ‘a systematic illness caused by microbial invasion of normally sterile parts of the body’ and is a potential life-threating complication of an infection. Sepsis is defined as a three-stage syndrome starting with sepsis and progressing through severe sepsis and septic shock (Research, 2016). Early treatment of sepsis with antibiotics and treatment with intravenous fluids, improves chances of survival (Research, 2016). The signs of sepsis are often highly variable and can be confused with other causes (Kent and Fields, 2012). Therefore, it is important to either confirm or dismiss the sepsis diagnosis as soon as possible, as other conditions will require different treatments (Lever and Mackenzie, 2007, Research, 2016). The time critical aspect of the detection of sepsis makes it an interesting process to investigate. The early detection of sepsis takes place on a day-to-day basis. It involves many interdependent functions often requiring a multidisciplinary team of stakeholders: secretaries, nurses, health workers, doctors and laboratory technicians. Along with the frequency and severity, sepsis and its detection is a common focus area in efforts to improve patient safety. As the study case is highly researched it facilitates investigation of whether a non-linear model will contribute to new or different insights than have previous initiatives. 2.2. Functional Resonance Analysis Method The FRAM seeks to identify essential system functions and understand variability. The model aims to describe everyday practice and how processes succeed under varying conditions (Alm and Woltjer, 2011). Functions refer to the activities that are required to produce a certain outcome. A function describes what people do, either individually or collectively, to achieve a certain aim (Hollnagel,

117



2016). In some cases functions can also refer to what organisations as a whole or technological systems do, either by themselves or in collaboration with one or more human (Hollnagel, 2016). Building a FRAM model includes three steps. The initial step of the model is to determine what purpose the model should illustrate, and identify the functions and how they are coupled. The second step is to understand how each function may vary, with a focus on six aspects of each function. The third step focuses on identifying and describing why and how each function creates variability (Hollnagel, 2012). Figure 1 illustrates the representation of a function in FRAM.

Figure 1: Description and presentation of the six elements of the Functional Resonance Analysis Method (Hollnagel, 2012) 2.3. Data collection Data were collected by four methods: document reviews, focus groups, direct observations of clinical work and in-depth interviews. Four sepsis-related documents were reviewed to gain insight into the functions and activities described in the instructions relating to the selected process. These were the instructions for detecting and for treating patients with sepsis, a flowchart illustrating the process and the checklist that nurses are required to complete for all suspected sepsis patients. Prior to conducting the observations, two semi-structured group interviews were undertaken with key staff involved in constructing guidelines and planning work in the investigated wards. The interviews included first the head nurse and head doctor of the participating ward, and second the administrative 118



doctor of the ward and the senior doctor who developed the sepsis checklist. These interviews helped the researcher to understand the setting and how work was managed in the ward. Ninety hours of observations were performed. The observations took place in two different wards and followed both nurses and doctors. Using observations enabled the researcher to have natural conversations and informal interviews of various sorts with the informants during times of observations (Kawulich, 2005). The observations were conducted between December 2015 and April 2016, with the observer following the staff in all their activities during shifts, both day and evening. Observations were conducted through entire shifts, which enabled the researcher to follow staff during examination of suspected sepsis patients—both those who were later cleared or had a confirmed diagnosis. Observations were collected as hand-written notes, transcribed after shifts and eventually analysed using the concepts of the FRAM by highlighting observations that described functions, variability and the six aspects in the FRAM: time, control, resources, precondition, input and output (Hollnagel, 2012). Further, two semi-structured interviews were conducted with frontline staff after observations were performed as a validation exercise, to confirm findings from the observations. During these interviews, the researcher presented the findings to other staff members working in the ward, but who did not participate in the data collection. This process aimed to investigate whether others could recognise the results and whether they found the description useful. Further, the results were presented to experts working in the field of sepsis detection. This helped support the credibility of the FRAM as a tool for understanding and uncovering important activities and aspects in complex health care processes.

2.4. Ethical considerations The study was presented to the Regional Committees on Health Research Ethics for Southern Denmark and the Danish Data Protection Agency, but ethics approval was not required to conduct the study, according to these authorities, as no personally identifiable data was collected. The researcher signed a confidentiality statement regarding documenting personal data on patients and all observations were based on informed, voluntary consent from patients, their next of kin and health care personnel. Prior to treatment by health care personnel, patients and next of kin were asked whether a researcher was allowed to follow their treatment.

119



3. Results A total of 40 functions was identified in the sepsis identification process. There are no specific rules for the ‘level of granularity’, but functions are included or split into more specific functions if variability is detected (Herrera and Woltjer, 2010). Table 1 shows all functions; further description and interdependencies in the model can be found in Appendix A. The model represents the entire process from when the general practitioner (GP) or ER doctor contacts the department to refer a patient, and includes activities performed by nurses, secretaries, doctors and laboratory technicians. The model ends when a sepsis diagnosis is confirmed or dismissed. A visual representation of the model is provided in Appendix B, and offers a view of the many interrelationships in the model. The model confirms that sepsis detection is a complex multi-functional activity with several interdependencies. Table 1: Description of all functions in the early detection of sepsis Functions 1.

The general practitioner (GP) or emergency room (ER) contacts the acute visitation ward (AVW) (Background function)

2.

The nurse responsible for the telephone in the AVW receives the call

3.

The nurse with the telephone has to be available to receive the call (Background function)

4.

The doctor refers the patient

5.

The nurse documents the information received from the doctor

6.

The nurse has the appropriate experience to foresee potential sepsis (Background function)

7.

The nurse with the telephone must have the sheet in her pocket to record onto it (Background function)

8.

The sheet has to be developed to be used (Background function)

9.

The sheet is printed for the nurse to pick it up (Background function)

10. The secretary receives the sheet from the nurse containing the information from the GP or ER doctor 11. The secretary must be available at the office to receive the sheet (Background function) 12. The secretary transfers the information from the sheet into the electronic system 13. The electronic system is available and functioning (Background function) 14. The secretary orders specific blood samples 15. The secretary collects all necessary paperwork 16. The patient arrives at the ward 17. The secretary follows the patient into the admission room 18. The secretary updates electronic systems with information on the patient 19. The nurse and doctor are aware of the patient’s arrival

120



20. The admission process begins 21. The nurse measures vital signs 22. Clinical judgement is used as a tool when evaluating the patient (Background function) 23. The nurse registers the patient in the Electronic Patient Journal 24. The laboratory technician takes and analyses blood samples 25. The patient is evaluated based on other symptoms 26. The nurse registers signs in the sepsis checklist (Systemic inflammatory Response Syndrome - SIRS) 27. The sepsis checklist is developed (Background function) 28. The patient is triaged based on vital signs and other symptoms 29. The triage tool is developed (Background function) 30. The doctor is called for examination 31. The doctor reads the patient’s record before assessment 32. The doctor conducts a 30-minute assessment of the patient 33. The doctor uses experience when talking to the nurse and assessing the patient (Background function) 34. The patient is examined 35. The doctor records the primary record on the patient 36. The doctor must be available to examine the patient (Background function) 37. The sepsis diagnosis is dismissed or confirmed 38. The patient is treated with the sepsis bundle (If sepsis is confirmed) 39. The sepsis diagnosis is conducted within an hour of admission (Background function) 40. A patient with dismissed sepsis diagnosis is treated otherwise (Background function)

3.1. Dividing the model into sets The model of the process includes many relationships and details that are challenging to grasp in its entirety. To analyse the process it was divided into subsets of functions that each represent an important task in the early detection of sepsis. Dividing the model into sets enables easier identification of variability and eventually also identifies the functions most likely to cause variability and therefore likely to have the greatest effect on the success of the process. In the following, the four identified sets, are described in detail. 3.1.1. Referring and obtaining information on the patient The success of early detection of sepsis was influenced by activities performed early in the admission process. Receiving referrals from the doctor and transferring information into the system was the first group of functions causing variability in the overall process. Observations and interviews revealed 121



that the process of referring a patient included several activities performed by the nurse in charge of receiving the referral calls. Variability here occurred due to time pressure. Observations showed that nurses could be forced to take calls while admitting other patients or performing other tasks. This resulted in interruptions to the task being performed and could result in a lack of information transferred from the doctor to the nurse. The amount and detail of information regarding the patient’s symptoms varied due to time pressure and the experience of the nurse. Observations showed that nurses with experience had a different approach to acquiring information from the doctor than did less experienced nurses. Their experience made them alert to specific symptoms and signs from the patient, affecting their alertness with respect to possible sepsis early in the process. Speaking with the doctor required that the nurse knew which questions to ask to obtain the relevant information. To minimise variability caused by a nurse’s degree of experience when obtaining information from the doctor, nurses developed and used a form. The nurses in the ward were expected to carry the form in their pocket during work. A range of factors often affected the availability of the form, including that there were no more copies in the ward, nurses forgot to collect the form prior to their shift, or they had run out of forms. The nurses handled this variability by instead using either post-it notes or a clipboard available in admission rooms to record information. Since the developed form required consideration of different relevant factors and questions, this regularly affected the amount and quality of the information retrieved from the doctor. This also affected the secretary’s ability to understand and transfer the information to the electronic system. The availability of paper forms could be seen as a trivial aspect of patient care, but the observations showed that the forms were important cognitive artefacts on which both nurses and doctors heavily relied (Nemeth et al., 2005). When forms were not available, important information affecting the speed of treating the patient was sometimes reduced. 3.1.2. Transferring information to electronic systems Some aspects of the early detection of sepsis were connected with the use of electronic systems. As soon as the secretary registered a patient in the system, the patient’s data appeared on an electronic board. Besides registering new patients, the electronic board served as an overview of expected patients, those who had already arrived in admission rooms and those already admitted to the ward. The electronic board was also used by doctors to keep track of admitted patients and expected

122



patients, and which needed to be attended to first. It contained information regarding staff functions and their whereabouts, patient conditions, symptoms, treatments and how often they had to be attended to. If the electronic system was not available because of malfunction, the process was compromised. Staff members relied on the screen to organise work and also as a reminder to check patients requiring frequent attention. In the case of malfunction, staff used paper to replace the electronic system’s function, which heightened the risk of overlooking patients or forgetting to check vital signs for already admitted patients. It created more complexity and less cooperation between staffing groups, because the electronic system was used to organise work tasks, inform doctors of patients’ conditions and keep track of staff locations during shifts. Use of the electronic board was important for timely examination of new patients in the ward. It alerted staff about whether an expected patient had specific symptoms that needed to be treated quickly, and informed of the arrival of patients. Patient examination was delayed and variation occurred if the secretary had not transferred information into the system or if ward staff were not aware they needed to check the system for updates. 3.1.3. Admission process The next set included admitting a patient to the ward and preparing for examination by the doctor. This process was activated when the nurse became aware that the patient had arrived. The patient’s vital signs were measured and other symptoms monitored and evaluated before the patient was triaged. The triage outcome determined how often the patient was attended and served as a tool when informing the doctor about the patient. The nurses used triaging colours to underline the patient’s condition and to get the doctor to examine the patient as early as possible. The availability of different technologies further affected how these functions were performed. To triage patients, nurses relied on thermometers, sphygmomanometers, heart rate monitors and electronic access to a variety of diagrams necessary for conducting the triage. If any of these were not available, it affected the ability and speed of conducting the triage. The measurement of vital signs was a function that produced many outputs. This meant that variation in terms of time delay or output affected the process later in different ways. The triage was not conducted if vital signs were not measured, possibly delaying the doctor’s arrival. Additionally,

123



nurses used vital signs to confirm sepsis on the checklist. Completing the checklist required that vital signs were measured and the nurse was alert to a possible sepsis diagnosis. 3.1.4.Examining the patient and confirming or dismissing diagnosis The final section of the model included examining the patient and confirming or dismissing the diagnosis. Calling the doctor to plan the examination of the patient was the first function in this part in which variability occurred. The length of time before the patient was examined depended on how the patient was triaged, but nurses occasionally called the doctor if they were worried or in doubt. A call for the doctor and their time of arrival was subject to high variability and was affected by different factors. Some more experienced nurses relied on their clinical judgement, and symptoms were observed almost immediately after arrival. This ability made the nurse capable of reacting to symptoms, contacting the doctor and starting initial treatment fast. The speed and ability to make sound clinical judgement was affected by nurses’ experience: newly trained nurses found this process took longer and was sometimes difficult. This meant that the experience of the nurse in charge of admitting patients affected how quickly treatments were started, the doctor arrived to examine the patient and hence how fast sepsis was detected. Additional factors included the nurse’s relationship with the doctor, the nurse’s former experience with similar patients, the patient’s triage and the nurse’s use of specific words or highlighting of specific symptoms when talking to the doctor. Speaking to doctors during observations revealed that they weight the wording of nurses to determine how fast the patient should be examined. Level of experience by both nurses and doctors affected how quickly the patient was examined, which influenced the rest of the process. Variability in terms of how the patient was assessed by doctors also depended on doctors’ experience. Younger or less experienced doctors sometimes conducted the primary examination at the same time as the 30-minute assessment. Experienced doctors used the 30minute assessment to review the patient and order tests and would often come back at a later time to do the full assessment and primary record. This indicated that the experienced doctor had a better overview of the different patients in the ward, and could prioritise their time to the patients with urgent needs. The timing of the arrival of the doctor further depended on the time pressure in the ward in general. Doctors were often delayed if there were many patients in other parts of the ward. Some nurses mentioned that the doctor only came quickly if a patient’s triage colour was orange or red. This meant

124



that some nurses felt the need to raise the level of triage for a patient based on their clinical judgement and not on the patient’s vital signs. The final function of diagnosing the patient included a number of preconditions. Preconditions differ from other aspects, as they not only affect how the function is performed but also have the ability to stop the function from being performed (12). There were five preconditions in this function, which illustrated that the performance of earlier functions had the ability to postpone or prevent the diagnosis of sepsis. To complete the diagnosis, the doctor examined the patient, their vital signs and blood sample results, and had access to the sepsis sheet and the information in the electronic patient journal. If any of these functions were not performed, or were delayed, the diagnosis was delayed. 4. Discussion This study distinguishes itself from previous studies by applying a non-linear complex method to understand and investigate interrelations in the early detection of sepsis (Kent and Fields, 2012, Tromp et al., 2010). It does not consider which type of treatment should be preferred but applies an exploratory approach towards gaining understanding of organisational factors in the process. Emphasising how things go right has been labelled Safety-II (Braithwaite et al., 2015, Hollnagel, 2008), an approach that contributes a number of new aspects (5). There is a growing number of publications demonstrating that a Safety-II approach has the potential to enhance safety in health care. A Scottish study on blood samples in acute wards, emergency departments and outpatients wards concluded that the FRAM and perspectives of Safety-II provided a realistic model of blood sampling and how the system could succeed through adaptability (6). The study further demonstrated where resilience could be enhanced in the system and pointed towards areas that could be worked on to improve the success of the process (6). The advantages of a Safety-II perspective were further discussed in a conceptual paper on learning from incidence. Here the authors suggested that a SafetyII approach can help shift attention from extraordinary failures to the ordinary performance adjustments that clinicians make on a daily basis (5, 7). The FRAM has previously been shown to provide insights into how a system’s dynamic is affected by small variations in system functions (Sujan and Felici, 2012, Pickup et al., 2017). This can be useful information when developing guidelines and instructions or to manage the process better with the aim of achieving more positive outcomes in the treatment of sepsis patients.

125



Other methods could be used for the present purpose, each reflecting a specific perspective of the process and how it functions (Hollnagel, 2012). However, previous studies applying the FRAM have shown it to be a useful and powerful method for understanding health care processes; the current study supports these findings (Sujan and Felici, 2012, Clay-Williams et al., 2015). Studies have shown that the FRAM not only offers an understanding of a process under investigation, but has also raised awareness of contextual factors that are often not included in systematic descriptions of complex processes (Sujan and Felici, 2012, Herrera and Woltjer, 2010). In the current study, important contextual factors included the experience level of the staff, ability to multi-task, reliance on electronic systems and the hierarchical relationship and structure between nurses and doctors. These factors have previously been identified as important in several aspects of care including quality improvement, leadership and creativity (Kaplan et al., 2012, Kay Brazier, 2005, Benner, 1982). However, they are rarely considered as central in the implementation of new patient safety initiatives in Denmark, which often have a primary or ultimate focus on clinical aspects. The FRAM allows a focus on things that go right as well as things that go wrong. This enabled the researcher to investigate factors potentially associated with early detection of sepsis, to gain a better understanding of the complexity of the process. The method helps not only to illustrate what happens in the process, but to understand the ‘how and why’ aspects of the process. It provides a useful illustration of the dynamic interactions in the process and how these can vary. Four sets of functions became evident from the FRAM modelling process. A common trait for all four sets was that the variability was caused by individual differences and characteristics of staff members. These individual factors were their ability to conduct clinical judgements and their previous experience and knowledge in the field. These findings support assumptions that organisational and human aspects such as communication, knowledge sharing and relationships between staff members need to be considered fully in line with clinical aspects and when planning work (Pronovost et al., 2015, Sammer et al., 2010). These findings support those of previous studies applying FRAM to health care processes (Laugaland et al., 2014, Clay-Williams et al., 2015, Pickup et al., 2017). The method applied in this study could therefore make a relevant contribution in the development and design of processes in health care. Previous applications of the FRAM have also presented the method as a tool for creating a better connection between everyday work and how this is described in guidelines of everyday work. Clay-Williams et al. described how the FRAM helped create guidelines that were compatible with how staff were working. Through this approach, wards avoided

126



implementing guidelines that forced performance adjustments and created workarounds that compromised both safety and quality (8). Even though this study has established that using the perspective of the FRAM can offer new insights, it is also important to consider that sepsis cases in hospitals can develop differently. In some cases, severe sepsis or septic shock can develop while inpatients are treated for other conditions, primarily as a nonsomical infection (Research, 2016). This study focused on how sepsis is detected in patients immediately after admission and the results can therefore not be translated to patients who develop sepsis during their stay. Nonetheless, it is suggested that the same method could be used to investigate such cases and help identify important aspects, factors and relationships. 5. Conclusion The FRAM proved to be a useful method to understand a complex process in health care with the ability to highlight how a system might produce emergent outcomes due to unexpected combinations of performance variability in the process. This systematic method offers a novel, alternative way of investigating safety-critical processes in health care. The study showed that it is possible to apply the FRAM to patient safety processes and, through the conceptualisation, to gain new insights and perspectives on which factors play a central part in successful process outcomes. The method enabled the researcher to investigate the process in a systematic way and gain an understanding of how the process works and is adapted to everyday variability. The FRAM can create awareness among staff members of things they do to succeed, often without them noticing. It is further suggested that the FRAM can serve as a tool to develop a health care system in which management and guidance are based on an understanding of how things actually work on the frontline. The work with the FRAM was meaningful and according to staff in the organisation it also provided them with new insights as it took into account important factors like experience and clinical judgement, which are factors normally not considered important when developing tools like checklists or instructions. This study further suggests a heightened awareness of how we understand and view processes in health care, in advance of trying to improve or create changes. The results of this study question whether the application of linear models in root cause analysis provides us with the necessary insights and information to create a safer health care system, or if other models can raise awareness of new and equally important aspects, as presented in this study.

127



Abbrevations FRAM = Functional Resonance Analysis Method CAS = Complex adaptive system GP = General Practitioner ER = Emergency Room AVW = Acute Visitation Ward Acknowledgments We would like to acknowledge Alan Kimperkarl, Helle Skærbæk, Nick Pfaff Steen and Annette Fuhlendorff from the Medical Ward at Lillebaelt Hospital for allowing the study. A special thank you to Helen Bruun for coordinating the observations and setting up interviews with all particpants. Further thanks to all nurses, doctors and other staffmembers who shared their work with us for this study. ALM, H. & WOLTJER, R. 2011. Patient Safety investigation through the lens of FRAM In: WAARD, D. A., BERGLUND, A., PETERS, B. & WEIKERT, C. (eds.) Human Factors: A system of human, technology and organisation. Maastricht, Netherlands: Shaker Publishing. BENNER, P. 1982. From novice to expert. Am J Nurs, 82, 402-7. BRAITHWAITE, J., CLAY-WILLIAMS, R., NUGUS, P. & PLUMB, J. 2013. Health care as a complex adaptive system. In: HOLLNAGEL, E., BRAITHWAITE, J. & WEARS, R. L. (eds.) Resilient Health Care - Volume 1. Surrey, England: Ashgate Publishing. BRAITHWAITE, J., WEARS, R. L. & HOLLNAGEL, E. 2015. Resilient health care: turning patient safety on its head. Int J Qual Health Care, 27, 418-20. CLAY-WILLIAMS, R., HOUNSGAARD, J. & HOLLNAGEL, E. 2015. Where the rubber meets the road: using FRAM to align work-as-imagined with work-as-done when implementing clinical guidelines. Implement Sci, 10, 125. DIXON-WOODS, M. & BOSK, C. 2010. Learning through observation: the role of ethnography in improving critical care. Curr Opin Crit Care, 16, 639-42.

128



HERRERA, I. A. & WOLTJER, R. 2010. Comparing a multi-linear (STEP) and systemic (FRAM) method for accident analysis. Reliability Engineering & System Safety, 95, 1269-1275. HOLLNAGEL, E. 2008. Safety Management - Looking back or looking forward. In: HOLLNAGEL, E., NEMETH, C. P. & DEKKER, S. (eds.) Resilience Engineering Perspectives: Remaining sensitive to the possibility of failure. Ashgate. HOLLNAGEL, E. 2012. FRAM: the Functional Resonance Analysis Method - Modelling Complex Socio-technical Systems, Surrey, England, Ashgate Publishing. HOLLNAGEL, E. 2016. A FRAM glossary [Online]. Middelfart: Hollnagel, E. Available: http://functionalresonance.com/how-to-build-a-fram-model/index.html [Accessed 24.04.17 2017]. KAPLAN, H. C., PROVOST, L. P., FROEHLE, C. M. & MARGOLIS, P. A. 2012. The Model for Understanding Success in Quality (MUSIQ): building a theory of context in health care quality improvement. BMJ Qual Saf, 21, 13-20. KAWULICH, B. B. 2005. Participant Observation as a Data Collection Method. 2005, 6. KAY BRAZIER, D. 2005. Influence of contextual factors on health‐care leadership. Leadership & Organization Development Journal, 26, 128-140. KENT, N. & FIELDS, W. 2012. Early recognition of sepsis in the emergency department: an evidence-based project. J Emerg Nurs, 38, 139-43. LAUGALAND, K., AASE, K. & WARING, J. 2014. Hospital discharge of the elderly--an observational case study of functions, variability and performance-shaping factors. BMC Health Serv Res, 14, 365. LEVER, A. & MACKENZIE, I. 2007. Sepsis: definition, epidemiology, and diagnosis. BMJ, 335, 879-83. NEMETH, C., O'CONNOR, M., KLOCK, P. A. & COOK, R. 2005. Cognitive artifacts' implications for health care information technology: revealing how practitioners create and share their understanding of daily work.

129



PICKUP, L., ATKINSON, S., HOLLNAGEL, E., BOWIE, P., GRAY, S., RAWLINSON, S. & FORRESTER, K. 2017. Blood sampling - Two sides to the story. Applied Ergonomics, 59, Part A, 234-242. PRONOVOST, P. J., RAVITZ, A. D., STOLL, R. A. & KENNEDY, S. B. 2015. Transforming Patient Safety - A Sector-wide Systems Approach. WISH Patient Safety Forum 2015. RESEARCH, M. F. F. M. E. A. 2016. Sepsis - Symptoms and causes [Online]. Rochester, Minnesota: Mayo Clinic [Accessed 23.02.2016 2016]. SAMMER, C. E., LYKENS, K., SINGH, K. P., MAINS, D. A. & LACKAN, N. A. 2010. What is patient safety culture? A review of the literature. Journal of Nursing Scholarship, 42, 156-165. SUJAN, M. 2015. An organisation without a memory: A qualitative study of hospital staff perceptions on reporting and organisational learning for patient safety. Reliability Engineering & System Safety, 144, 45-52. SUJAN, M. A., CHESSUM, P., RUDD, M., FITTON, L., INADA-KIM, M., COOKE, M. W. & SPURGEON, P. 2015. Managing competing organizational priorities in clinical handover across organizational boundaries. Journal of health services research & policy, 20, 17-25. SUJAN, M. A. & FELICI, M. 2012. Combining Failure Mode and Functional Resonance Analyses in Health care Settings. Computer Safety, Reliability, and Security. Springer Berlin Heidelberg. SUJAN, M. A., HUANG, H. & BRAITHWAITE, J. 2016. Learning from incidents in health care: Critique from a Safety-II perspective. Safety Science. TROMP, M., HULSCHER, M., BLEEKER-ROVERS, C. P., PETERS, L., VAN DEN BERG, D. T., BORM, G. F., KULLBERG, B. J., VAN ACHTERBERG, T. & PICKKERS, P. 2010. The role of nurses in the recognition and treatment of patients with sepsis in the emergency department: a prospective before-and-after intervention study. Int J Nurs Stud, 47, 1464-73. VINCENT, C. 2008. Patient safety, Wiley-Blackwell. VINCENT, C. 2010. The Nature and Scale of Error and Harm. Patient Safety. John Wiley & Sons, Ltd. YIN, R. K. 2003. Case Study Research: Design and Methods, SAGE Publications.

130



Appendix A FRAM model of early detection of sepsis

131



132



Appendix B Description of functions in ’Early detection of sepsis’ No.

Function

1

The General Practitioner (GP) or Emergency room (ER) contacts the acute visitation ward (AVW)

2

The nurse, responsible for the telephone in AVW, will receive the call.

3

The nurse with the telephone has to be available, in order to receive the call (Background function)

4

The doctor refers the patient

5

The nurse documents the information received from the doctor.

6

The nurse has the right experience to foresee

Brief description of function If the GP has a patient they suspect have an infection based on clinical signs, they will refer the patient to the medical ward, in order to be investigated and treated. The same is relevant for patients admitted to the ER. If they show signs of infections, doctors from the ER will also call the AVW. One of the nurses on shift will always be responsible for carrying the telephone were patients are called in. The nurse with the telephone may have one of several other functions in the ward, including admission, care of admitted patients etc. The nurse is responsible of several activities besides attending the telephone. Therefore, in order to be able to answer the telephone, she has to be available.

Contribution to the model Activating the function where the ward receives the call from either GP or ER.

Variations

Activating the function where the ER doctor or GP refers the patient to AVW.

Variations occur if the nurse is occupied with something else, or does not carry the phone.

Precondition for the nurse to be able to receive the call from the GP or ER and handle the referral.

Variations occur if the nurse is not free and if the telephone is not at hand it will delay the process.

During the telephone conversation between the ER doctor or GP, the nurse will receive information regarding the patient’s condition. This may vary in detail and severity. The nurse with the telephone is obligated, besides the telephone to carry a sheet, which includes a number of aspects of the referral she needs to record. She will at this point also note if it will be necessary to order specific blood samples based on the knowledge from the doctor.

This activates the function, where the nurse will document the information she received from GP or ER doctor.

Depending on time available variations can occur when transferring the information and can stretch the time of the call.

This activates the function where the nurse will pas the sheet on to the secretary.

This includes the nurses past experience with detecting

This function is a resource for the nurse

Variations can occur if the nurse does not have paper to register the information, without paper the quality can be compromised of the referral information Further variations occur if the doctor is vague it will take longer to withdraw information This function affects variations in when the blood sample are request. Variations occur based on the experience and

133

The doctor has to be availability for calling and have a phone.



potential sepsis. (Background function)

sepsis and knowing what the typical symptoms are or what to ask for when talking with the doctor

in functions prior to the patients arrival

7

The nurse with the telephone has to have the sheet in her pocket in order to record onto it. (Background function)

Precondition for the nurse to be able to record the information necessary to receive the patient properly.

8

The Sheet will have to be developed to be used (Background function)

When the nurse is in charge of the telephone, she has to go to the secretary’s office, and collect a stack of sheets, she will carry in her pocket and use when a referral is made. In advance someone has developed and decided which categories go onto the sheet.

9

The secretary receives the sheet from nurse, containing the information from the GP or ER doctor.

The nurse will have to leave the activity she is performing or wait until she is finished, and walk to the secretary and hand over the sheet containing information on the referred patient.

10

The secretary must be available at the office in order to receive the sheet. (Background function) The secretary will transfer the information from the sheet into the computer system.

The secretary has to be at her desk in order to directly accept the sheet containing the information. The secretary is responsible for transferring the knowledge, which the nurse notes, on the sheet, into the computer system.

The secretary will order specific blood samples. (background function)

If the information the nurse receives from the doctors indicates that specific conditions are likely she will note on the sheet that specific blood samples need to be ordered at the laboratory technician (LT).

This activates the function were the secretary takes the sheet, and transfers the information onto the computer system. It also activates the function were the secretary will order blood samples for the patient, if this is noted on the sheet. This is a function affecting the preparedness for receiving the patient. This activates the function were the nurses on the ward, are informed that a patient has been referred and can be expected at the ward and the function where the secretary prepares paperwork for the admission of the patient. This is an output from when the secretary transfers the additional information into the computer system.

11

12

134

Control mechanism for writing down the necessary information.

clinical practice of the nurse and will improve the speed of the process, if the nurse is senior. This can cause variation in the time of when sepsis is detected. If the sheet is printed or not and can cause variations in the process.

The sheet helps control the information withdrawed. If the sheet was not developed the quality of information would be subject of variability. Variations occur when the secretary is delayed with receiving the sheet or has a hard time understanding notes from the nurse.

If the secretary is working somewhere else variations can occur. Unavailable secretary can cause variations if the transfer of observations is not completed, and the nurses in the ward have a harder time planning their work.

This function is subject of variability if the nurse receiving the call was not trained in being aware of sepsis symptoms.



13

The secretary will collect all necessary paperwork.

The nurse needs to fill out a number of paper during the admission of patients (Sepsis checklist, PatientSafe admission,etc.) and the secretary will put these in a folder for the receiving nurse to collect

This is a precondition necessary in order to correctly admitting patients to the ward.

14

The paperwork has to be developed

All the sheets necessary for admission must be developed in order to be used.

This function is a precondition for preparing the folder for admission.

15

The patient arrives at the ward.

The patient will contact the secretary at the reception desk.

16

The secretary follows the patient into the admission room

17

The secretary updates the computer system with information on the patient.

18

The Nurse is aware of the patient’s arrival.

After the patient has announced their arrival at the secretary desk, the secretary will follow the patient to an admission room, which is available. After following the patient to the admission room, the secretary will update the computer system, with a number of the room the patient is in. The nurse needs to keep updated with the computer system in order to get aware of when the patient has arrived, and in which room the secretary has put the patient.

This activates the function where the secretary will show the patient into an admission room. This activates the function were the secretary updates the computer system with room number of the patient. This activates the function where the nurse is becoming aware of the patients arrival.

19

The admission process begins.

The patient is admitted and the nurse notes the patient in the electronic patient journal, updates all relevant information and symptoms.

135

This activates the function where the nurse starts the admission of the patient.

This activates the function where the vital signs are measured.

Variability can occur if the secretary is occupied elsewhere, and unable to collect paperwork and this may cause variation in the process, for when the assessment of the patient can start and can affect the nurses awareness of possible sepsis. The design of the paperwork may cause either heightened or dampened variation in the process. Variations can be caused and delay the process if the patient is not registered immediately at arrival. This can cause delay in the process if the secretary is not available to follow the patient. Variations can occur if the secretary is busy after following the patient to the admission room, and cause delay later in the assessment. Variations can occur if the nurses are busy with other tasks, and not alert towards the computer system, the admission process can be prolonged. Further the amount of information on the computer screen can alert the nurse, of possible sepsis and cause variation in the initial treatment of the patient. Variation can occur if the necessary data is not available or not functioning. Variation can also occur based on nurses experience in specific symptoms.



20

Computer systems are available and functioning. (Background function)

Several of the functions in the model are dependent on the fact that computer or electronical systems are functioning.

This is a resource function for several functions.

21

Measuring vital signs.

22

Clinical Judgement is used as a tool when evaluating the patient (background function)

As soon as the nurse starts admitting the patient, the vital signs will be measured (Temperature, Blood pressure, respiratory frequency and pulse). These signs will be leading for how fast the patient will be seen by a doctor, when in the line of patients the patient will be admitted etc. Besides using vital signs and symptoms to evaluate the patient, staff will use their clinical judgement to look at the patient combined with past experience.

This function activates the function were other symptoms in the patient are evaluated and the patient is triaged. It also starts the function were the patient is registered in the Electronic Patient Journal.

23

Registration of the patient in the Electronic Patient Journal (EPJ)

24

The LT takes and analyses blood samples.

This is a precondition for the doctor to be able to document treatment for the patient. This is a precondition for confirming or dismissing the sepsis diagnosis.

25

The patient is evaluated based on other symptoms.

26

Triaging patients based on vital signs and other symptoms

In order to admit the patient, document vital signs and call a doctor later on the patient must be registered in the EPJ. After the secretary informs the LT about the patients, they will arrive at the admission room and take all necessary blood samples. They are then sent to the lab for analysis. After measuring the vital signs, the nurse will also look at other symptoms in the patient. This is a precondition for diagnosing and for how the patient is triaged. The patient are triaged using a form, were the triage is decided based on vital signs and other symptoms.

27

Triage tool is developed (Background function). The doctor is called for examination.

28

The form must be accessible and developed. The doctor will be called to examine the patient. If the

136

This is a precondition for triaging the patient.

This activates the function were the doctor is called for examination. It also decides how fast the doctor needs to arrive. This function controls the triage function. This activates the function were the

This can cause delays and lack of specific treatment or awareness of symptoms, since computer systems contain important information of the patient. Variation can occur if the necessary equipment is not available or not functioning.

Variations occur based upon each nurses experience and can affect the time before specific symptoms are linked towards a possible sepsis condition. Variation can occur if the data systems are not functioning properly and can cause lack of information or delay. Variation can occur based on expierence in LT and based on the difficulty of taking blood samples from each individual patient. This function may vary based on previously mentioned experience or clinical judgement and may be delayed or less thoroughly conducted. This function can vary based on the experience of nurses and the ability to rely on evaluation of patients rather than vital signs. No affecting variation. This can cause variation based on how



triage is orange or red, the timeframe for examination is within 1 hour, if the triage is yellow or green it is 4 hours.

doctor examines the patient.

29

Using experience when talking to the nurse and evaluating the patient (background function)

The doctor will often depend on past experience and clinical judgement to evaluate how fast the patient should be assessed.

This function serves as a resource and time element in the function of the evaluation of the patient

30

Looking into the patients record before assessment

The doctor will look into the record to be informed of former hospital stays, diagnosis, blood sample results etc.0

This is both a function to activate the 30 minute assessment or the treatment plan.

31

Conducting the 30 minute assessment

The doctor will initially examine the patient to review the patients general state and evaluate how fast treatment plans should be conducted

This function is the activating function for the examination of the patient and preparation of treatment plans.

32

Examining the patient for sepsis or other conditions and prepare treatment plan

The doctor will examine the patient and look for diagnosing sepsis.

This activates the function were the diagnosis is either dismissed or confirmed.

33

The doctor must be available to examine the patient. (background function) The sepsis diagnosis will be dismissed or confirmed

To examine the doctor must be available and present in the ward.

This is a precondition for examining the patient.

The doctor will use blood sample results, vital signs measured and overall symptoms to confirm or dismiss diagnosis. The doctor will further warrant the treatment.

This activates the treatment with sepsis bundle if confirmed and dismissed other treatment will be started. This also starts the function were the diagnosis is registered.

34

137

busy the doctor and the ward is, and the expierence of the nurse and ability to inform the doctor of the central aspects causing delay in the process. The experience can either cause variation in time of the examination, both before and under and cause variation in quality of the assessment. This function may cause variations in the how fast the doctor will assess the patient afterwards, and whether the doctor will conduct a 30 minute assessment or go straight into the full assessment with diagnosis assessment and preparation of a treatment plan. This function can vary based on how busy the doctor is but also based on the severeness of the patient’s condition. In some cases this function is not performed, if the doctor goes directly on to the full assessment of the patient. This function may vary based on pressure on ward and doctor and duration may vary based on information transferred from the nurse. No affecting variation other than pressure on the ward. Variations can occur based on the information accessible at the time, incl. blood sample report, information from nurses, and access to sepsis checklist.


35

Treating with sepsis bundle

36



Time variations based on previous functions.

Registering signs in the sepsis sheet

After the diagnosis is confirmed, the patient will be treated with all elements in the sepsis bundle. The checklist for sepsis must applied with all patients with reasonable suspicion of sepsis and completed with all patients with at least two of four criteria for sepsis.


37

Developing the sepsis checklist (background function)

In order to be used the checklist must be developed and printed.

38

Conducting the diagnosis within an hour (Background function)

According to instructions and the checklist, patient with suspicion of sepsis, must be diagnosed within 1 hour of admission.

This is a control function for the diagnosis of sepsis and a precondition for completing the checklist. This is a time control, for the function of diagnosing the sepsis.

Variations can occur if the sheet is not available, symptoms in the patient change quickly, if tools to measure signs are not available or lack of experience in the nurse. No affecting variations.

39

Continuing other treatment of patients (background function)

40

Writing primary records and recording findings.

If the sepsis diagnosis is dismissed the patient will continue in other examinations and treatments After assessing the patient and starting the treatment, findings ant treatments will be recorded in the patients file.

138

No output in this model This function is part of the assessment of confirming or dismissing the sepsis diagnosis as a control element.

All previous mentioned functions which may have variations connected to time, may have the ability to affect this function. Variations in this function may cause heightened awareness of sepsis state in patient. No affecting variations.

Lack of recording findings in electronic files can cause the delay of discovering urgent conditions.



Paper III Title: Learn from what goes right: a demonstration of a new systematic method for identification of leading indicators in health care Status: Status: Currently in Revision at Reliability Engineering & System Safety (3. Round of revision submitted May 22th 2017) Names and affiliations of contributing authors: Ditte Caroline Raben, MScPH, Institute of Regional Health Research, University of Southern Denmark and Centre for Quality, Region of Southern Denmark, Denmark. Address: P.V. Tuxenvej 5, DK-5500 Middelfart, Denmark. E-mail: [email protected] Søren Bie Bogh, MHSc, Institute of Regional Health Research, University of Southern Denmark and Centre for Quality, Region of Southern Denmark, Denmark. Address: P.V. Tuxenvej 5, DK-5500 Middelfart, Denmark. E-mail: [email protected] Birgit Viskum, Consultant, Public Health Medicine, Aborgmindevej 9, 5610 Assens, E-mail: [email protected] Kim L. Mikkelsen, Ph.d., Medical Coordinator, Danish Patient Insurance Associtation, Address: Kalvebod Brygge 45, 4.sal, 1560 København V, Danmark E-mail: [email protected] Erik Hollnagel, Ph.D., Professor, Institute of Regional Health Research, University of Southern Denmark and Chief Consultant, Centre for Quality, Region of Southern Denmark, Denmark. Address: P.V. Tuxenvej 5, DK-5500 Middelfart, Denmark E-mail: [email protected] Word count: 5216

139



Abstract The work in patient safety is often centred on adverse events and errors. Typical methods to improve patient safety are reactive and focus on understanding past failures. This article presents the development of a proactive method towards improving patient safety and understanding why processes function as intended on a daily basis. The paper presents the steps of how the method was developed and demonstrates it by using a former case study of early detection of sepsis. Emphasis is on understanding complex processes and identify aspects important for things going right and achieving intended outcomes. The study resulted in the development of six overall steps for identifying leading indicators in complex health care processes. These were (1) identification of relevant functions, (2) cluster of functions in sets, (3) identification of functions with variability, (4) identification of functions with upstream-downstream functions, (5) identification of leading indicators, and (6) confirmation of leading indicators through experts and adverse events. The study outlined the development a new method on the topic of leading indicators in the context of patient safety.

140



1. Introduction The aim of this study is to develop a systematic method for the identification of leading indicators for safety in health care based on a case study describing the early detection of sepsis in a Danish hospital ward. Based on these results and the literature on leading indicators, this study proposes a method to develop leading indicators in health care. The use of indicators in health care has been a priority for more than a decade in the Danish health care sector (Mainz, 2003). Indicators are commonly used to document and measure quality, set priorities, support patient safety initiatives and describe historical trends (Vincent et al., 2014). Another main aim of indicators is to support decision making with regard to when, where and how to take action (Broadribb et al., 2009). This paper argues that the current application of indicators is limited to taking actions. Studies of other industries suggest that indicators can be developed to prevent adverse events in health care. 1.1. Patient safety context The field of patient safety has developed rapidly during the past 15 years, expanding from the focus of a few researchers to be placed on national agendas (Wears et al., 2014). Still, patient safety approaches primarily rely on methods that assume adverse events can be explained and avoided using linear models and applying reactive thinking, labelled Safety-I (Braithwaite et al., 2015, Hollnagel, 2014b, Peerally et al., 2016, Sujan, 2016, Hollnagel, 2014a, Sujan et al., 2016). This approach has played an important role in avoiding harm for patients and improving the safety of health care services (Braithwaite et al., 2015, Peerally et al., 2016). However, adverse events still occur in up to 10% of acute admissions (Wears et al., 2014, Peerally et al., 2016, de Vries et al., 2008). This paper presents an alternative method for patient safety and a new way to identify indicators to ensure that processes function as intended. These indicators are termed ‘leading indicators’ and are used as precursors of events. They contribute to a better understanding and management of processes in complex systems such as health care (Øien et al., 2011b). In several high-risk industries, this approach has contributed to the viewpoint of safety (Øien et al., 2011a). 1.2. Leading indicators During the early 1980’s the use of measures of safety and major hazard risks emphasised direct or after-the-event indicators (Øien et al., 2011b, Vincent et al., 2014). This perspective helped to establish sets of incidences for accidents or near-misses in the sector, but it did not provide the

141



necessary information to avoid these unwanted events. To anticipate and create early warnings prior to accidents, one needs to understand the underlying relations and combinations of factors that enable the system to function on a daily basis (Broadribb et al., 2009, Braithwaite et al., 2015). This approach rapidly evolved in high-risk industries in the aftermath of major accidents such as the Piper Alpha oil platform accident, the Texas City Refinery explosion and the Deepwater Horizon blowout (Broadribb et al., 2009, Bergh et al., 2014, Vincent, 2008). Leading indicators can be used to actively monitor important components of a system to achieve desired safety outcomes and implement warnings prior to undesired outcomes (Øien et al., 2011a). Industries apply different approaches to identify the relevant leading indicators for their own systems or organisations. Although these approaches are developed for a specific context, they have some common traits. First, leading indicators are identified based on modelling or understanding the investigated safety system. Second, the model of the system is used to identify potential factors that are relevant for safety by either reviewing previous incidences or collecting data from staff (Grabowski et al., 2007a, Herrera et al., 2010, Øien et al., 2010, Johnsen et al., 2012, Executive, 2001). The health care industry has learned from and adopted approaches that have been developed in highrisk industries (Hudson, 2003). However, the concept of and distinction between leading and lagging indicators has not yet been established in health care (Kristensen et al., 2009). 1.3. Aim Inspired by the knowledge of leading indicators that have been developed and applied in fields such as aviation, nuclear power and offshore operations (Pronovost et al., 2015), this study aims to develop a generic method to identify leading indicators. Previous studies on the development of leading indicators have reported positive effects of using proactive methods to understand systems (Øien et al., 2011a). This paper attempts to move the focus from the prevention of adverse events to achieving a higher number of positive outcomes through a better understanding of the systems. This focus will provide a foundation for identifying factors that contribute to things going right, labelled Safety-II (Hollnagel, 2014b, Hollnagel, 2013, Sujan and Felici, 2012). 2. Background for the method Indicators are typically extracted by first creating an understanding and model of the process; thus, this paper applies the results of a case study that describes the early detection of sepsis in a Danish hospital ward. Raben et al. conduct an in-depth analysis of this caseThe case was used to test the

142



development of a method for detecting leading indicators in health care. The process is described using a method for modelling complex socio-technical systems called the Functional Resonance Analysis Method (FRAM) (2014). The FRAM is used to produce a model of what is required to achieve the intended outcomes of processes performed on a regular basis (17). The model is constructed around functions; it describes the activities of a process and helps to illustrate how the performance of these functions can vary and how this variability can affect other functions later in the process (Hollnagel, 2012). 2.1. Combining perspectives to identify leading indicators Leading indicators are elements or factors that can be used to indicate possible future states of a process; therefore, they can be used to manage desired processes (Raben and Hollnagel, 2014). The method was developed based on the variability of the functions in the process, as well as the concept of upstream and downstream functions. Variability is described and used to point to indicators that are crucial for processes to succeed. Identifying these indicators involves a focus on two features of functions: variability and upstream–downstream couplings. 2.2. Variability The FRAM comprises a number of functions that each describe an activity performed in the illustrated process. A key element of the FRAM is that it illustrates how different tasks are connected or coupled to each other and how earlier activities can affect later activities by delaying or affecting the quality of the activity. The visualisation of the FRAM illustrates that complex processes are difficult to describe in a linear way. Actions can vary or occur concurrently if the circumstances or surroundings change (Hollnagel, 2012). Therefore, variability is investigated to provide an understanding of the couplings of functions. Variability can occur for different reasons (Hollnagel, 2012). First, functions can be affected by internal variability caused by psychological or physiological factors. These can include stress, fatigue, well-being, decision-making ability, personal judgement and past experiences. Second, functions can vary due to the working environment in which they are carried out. This includes social factors such as group pressure, social norms, relations with and expectations of co-workers, and the overall organisational culture (Hollnagel, 2012). Finally, variability can evolve from upstream–downstream couplings (Hollnagel, 2012). Detecting such couplings is based on detecting potential variability and

143



considering how it may spread through the system and affect functions later in the process (Hollnagel, 2012). This view of processes and systems shows how variability emerges and how it is either amplified or dampened by actions later in the process. Analysis of variability and couplings can help highlight the emergence of unexpected outcomes; more importantly, it can describe how expected outcomes succeed despite variability in functions (Hollnagel, 2012). 3. Results 3.1. Systematic method for identifying leading indicators The systematic method outlined below was developed and applied using the case study of the early detection of sepsis in a Danish hospital ward. Step 1 - Identifying relevant functions. First, a very simple representation of early detection was developed. This included the main task of detecting sepsis. Functions such as referring the patient, examining the patient and calling the doctor were identified. Further variability of functions was considered to decide which functions should be described. If the function was likely to vary in relation to potential effects on the output, the researchers decided to describe it in detail. Using the FRAM to identify functions in a process provided an opportunity to create a very large and complex representation of reality. In this case, functions that were not related to early detection were not considered. Using this as a guideline, functions should be added by asking the question, ‘Can it in any way affect the intended outcome?’ until the answer to this question is ‘no’. Step 2 - Clustering of functions in sets. The second step was to cluster the functions into a number of sets. First, the model was used to examine the process and the chronological order of each function being performed. This provided an understanding of the process over time. Functions were then divided into sets, with each representing relevant tasks conducted with the aim of detecting sepsis. These tasks were the overall key tasks that staff listed when asked to explain the process of the early detection of sepsis. Figure 1 illustrates the sets. Clustering of functions was conducted based on the assumption that complex processes may contain many leading indicators. In this process, four sets of functions were included, but different numbers could occur in other cases depending on the size and complexity of the process. The collected FRAM model can be seen in Raben Et al. (2017).

144



The four sets of functions identified in this process were: referring and obtaining information on the patient, transferring information to electronic systems, receiving and triaging the patient, and examining the patient and confirming or dismissing the diagnosis (6). Each set included all activities that were conducted to fulfil the task. During the data collection process, informants mentioned the overall set as a single function (e.g., referral of the patient). The functions included in each set were identified as a result of observations and conversations with informants on smaller tasks performed to ‘refer the patient’. Using the first set as an example included having the sheet for referral in the pocket, receiving the call from the doctor, writing the information onto a sheet, asking additional questions and ticking blood samples to be ordered. Examining each set separately provided an opportunity to focus on the factors in each set that influenced whether the process ended as intended. Step 3 - Identification of the functions in each set with variability. To find the leading indicators, a number of underlying assumptions were considered. Variability was not necessarily a criterion for considering a function. To be considered, variability should be important for the process and should affect the process later. Variability can affect functions in different ways related to either time or quality. To determine how variability affected the process, functions were analysed based on whether they varied internally, externally or due to couplings. Upstream–downstream couplings will be described in further detail in the next step. In this case, internal variability was mainly considered in relation to human and organisational factors. An example of internal variability occurred in the second set in the function—‘to become aware of patients registered in electronic systems’ (see Figure 1). This variability was caused by the nurse’s experience, attention to electronic screens and high pressure on the ward. The nurse’s level of experience could affect whether he or she was able to keep track of electronic screens during highpressure times in the ward, thereby affecting when the examination started, the speed of detecting of a possible sepsis diagnosis and how fast a doctor would arrive and conduct a further assessment of the patient. The possibility and consequences of internal variability were considered for functions in all sets and in relation to how the variability affected the output of the functions.

145



Figure 1: Close-up of second set—‘transferring information into electronic systems’ Another example of internal variability was observed at the organisational level in the first set— ‘referring and obtaining information on the patient’. The effectiveness of communication caused variability at the organisational level, and it arose from the general practitioner’s (GP’s) ability and willingness combined with the nurse’s ability to extract information from the GP. This variability affected the speed of detecting a possible sepsis diagnosis; therefore, it affected whether sepsis was diagnosed in a timely manner. Further, external variability can be caused by factors in the working environment. In this case, external variability affected the function of ‘calling the doctor for examination’ (see Figure 2). The output of this function was affected by the nurse’s relationship with the doctor. If the nurse knew the doctor, the conversation could be straightforward and the nurse could inform the doctor of symptoms and vital signs and use his or her observations of the patient to ask the doctor to examine the patient shortly thereafter. Conversely, nurses who did not have a relationship with the doctor sometimes found it difficult to voice their assessment of the patient, thereby making it difficult for the doctor to assess how fast the patient should be examined.

146



Figure 2: Close-up of third set—‘Examining the patient and confirming or dismissing diagnosis’ Internal and external variability were considered for all functions in the model to provide a thorough understanding of how each function could cause different outputs and to determine which functions were likely to be subject to variability

147



Step 4 - Identification of functions in each set with upstream-downstream couplings. In addition to variations caused by internal and external factors, functions can vary because of upstream–downstream couplings. When attempting to understand a representation of reality, it is not only necessary to know how variability may occur for each function, but also how variability may be combined. Examining the model and considering upstream–downstream relations provided an understanding of the consequences of combinations of variability. Two characteristics were considered when examining how variability affected functions later in the process: timing and precision. This section presents examples for each aspect and discusses how variability caused changes in downstream functions later in the process. Examples are presented in Figure 3 and the function of ‘to triage patients based on vital signs and evaluation’.

Figure 3: Close-up of third set—‘Receiving and triaging the patient’

148


-


First, preconditions for triaging the patient were considered. The data showed that a delay in the evaluation could cause a delay in the diagnosis. Variability also occurred if the evaluation was not conducted precisely. Nurses improvised or relied on false assumptions, thereby causing the output of the function to be vague and creating a possibility of misunderstanding and misinterpreting the symptoms, eventually delaying the diagnosis.

-

Resources that are necessary to perform functions represent elements that are needed or consumed by a function or that must be present while a function is being carried out (Hollnagel, 2012)—for example, using a referral sheet to write down information from the doctor. In some cases, the sheet was not available for the nurse during referral, so the variability delaying the function and quality of the output was reduced. This led to imprecise and limited knowledge of the referred patient and a possible delay in blood sample results, thereby delaying the diagnosis. The findings further showed that precise referral of patients enabled nurses to be aware of sepsis at an early point, which saved time later in the process and reduced variability and delays caused by other functions.

-

Control is used to describe aspects within an activity that regulate how processes are conducted (17). Based on the theoretical background, control can cause variability if performed functions are imprecise, incomplete or incorrect. In this case, a sepsis checklist was used as a control aspect. If the sheet was not filled out correctly or in a timely manner, the staff were not aware of a possible sepsis condition and therefore would not react. This caused a delay in which the condition worsened. Further, given that the condition of patients with sepsis changes quickly, if the checklist was filled out too early, it would not consider symptoms that occurred later, thereby reducing its effect.

-

Time can relate to the time it takes to perform tasks or the time at which the tasks are performed. Time played a significant role in several functions in this case (see Figure 3). For example, to detect sepsis in a timely manner, the checklist had to be filled out within an hour of admission. This time constraint was set based on evidence that shows that the one-hour mark is critical in detecting sepsis. A delay in detection resonated throughout the system and affected the outcome.

-

Input represents the start of a function, and functions often ‘look’ for a signal to begin. Input can therefore affect the performance of a function if it is started by a wrong signal or if the signal is too weak, which may lead to a lack of response (Hollnagel, 2012). This aspect was important because admission started when the function of ‘to become aware of the patient’s

149



arrival on the electronical system screen’ was completed. If the input was not performed in a timely manner, it resulted in a delay in the admission process and thus the rest of the process. The process of considering the possible consequences of functions and aspects was conducted for the entire model of the selected process. Each function in the model was systematically considered and all aspects were investigated to detect possible variability and investigate how this variability affected the outcome of the model - in this case, the timely detection of sepsis. Step 5 - Identification of leading indicators for each set in the process This step summarises the two previous steps in the method and identifies the leading indicators for the process. All functions were systematically examined for variability and were considered based on the couplings this variability entailed. Both aspects were equally important, as some functions could be subject to variability, but it was not before involved staff associated the variability with future scenarios, that they are applied as leading indicators. This meant that staff were especially aware and attentive of variability if they had experienced previously that the variability lead to specific results (Rasmussen, 1983). For this reason, the FRAM was a suitable tool for identifying leading indicators, as the method assisted in identifying variability and the connections and results of this variability (Hollnagel, 2012, Rasmussen, 1983). Therefore, if a function with variability did not have consequences later in the model, it was not considered a potential leading indicator. Conversely, if a function with variability affected the timely detection of sepsis, it was considered a leading indicator. This work concluded with the selection of four functions that were considered precursors or leading indicators for positive and successful outcomes in this case: o Receiving and obtaining the necessary and sufficient information on the patient from the referring doctor. o Remaining alert and becoming aware of when a patient had been received at the ward and was ready for admission. o Using former experience and clinical judgement to evaluate the symptoms of the patient to supplement vital signs. o Calling the doctor to conduct the examination and explaining the overall state of the patient to the doctor, including both vital signs and other observed symptoms. Step 6 – Supporting the leading indicators through experts and adverse events

150



Next, it was important to support the findings of the first five steps of the model. To be meaningful and relevant for the organisation, it was crucial that they were relevant for the setting they were developed in and that the identified factors played a role in managing the process and securing positive outcomes. Two approaches were applied in this confirmation. First, relevant staff or experts in the field were consulted and presented with the work. This provided an opportunity to assess whether the indicators resonated with the staff. Second, a review was conducted of adverse events previously reported in the field to investigate whether it was possible to recognise the functions deemed to be leading indicators in any reported adverse events. The review showed that several reported adverse events occurred during the admission process, where the first assessment of the patient was made. These were often caused by not measuring vital signs and not considering other symptoms besides the vital signs. Next, the review showed that adverse events were caused by a lack of communication between the staff referring the patient to wards and the staff receiving the referral. This resulted in wards not being prepared for the severity of the patient’s condition, or they lacked information on the patient, identified as the first leading indicator for successful outcomes. In these cases, this led to, or could have led to, a delayed diagnosis. Finally, the examination was also mentioned in adverse events. Either the staff found it difficult to get the doctor to respond fast, or the assessment was not conducted with an emphasis on sepsis. This lack of response could be caused by relations between doctors and nurses, or nurses ability to voice suspicion or concern that was not necessarily based on clinical symptoms. This variability could be caused by the doctor’s workload or miscommunication between the doctor and nurse, thereby causing the examination to be delayed. In some cases, there was a delay in receiving, or a lack of, blood sample results, which prevented the confirmation of sepsis. Either the blood samples were delayed or they were not ordered early in the admission process. The review of adverse events helped support the findings from the first steps. We cannot conclude that the indicators were confirmed. However, it was important to investigate whether some of the factors identified as important for success were actually contained in previously registered adverse events. They gave an indication of whether the description of the process was recognisable in adverse events. 3.2. Summary of the method for identifying leading indicators (MILI)

151



Extracting the developed method from the illustration provided by a specific case study is an important step in the development of a useful method. The six steps of the systematic method are extracted and summarised below. 1) Identifying relevant functions It is important to identify and describe all relevant functions of the process in a systematic manner. Equally important is the explicit consideration of when to stop the description of the process. The FRAM is a proven tool for this, as it includes clear criteria for when the functional model is complete. 2) Clustering of functions into sets Processes in health care and other fields typically comprise a number of smaller activities referred to as sets. To assess the number of leading indicators that are relevant to the process, the process should be clustered into sets, which can then be analysed separately. 3) Identifying the variability of functions in each set The description of how each function might be subject to variability helps with the initial identification of important functions in the process. The functions that are most likely to vary under different conditions should be further analysed in the next step. 4) Identifying upstream-downstream couplings of functions in each set This step deals with all functions and emphasises the functions previously identified as being likely to vary. Upstream–downstream couplings are those in which the variability of the function early in the process affects functions later in the process. 5) Identifying the leading indicators for each set of the process The aim of this step is to propose and recognise observable characteristics of functions that may be variable and that can affect one or more of the functions that follow. Observable characteristics are candidates for leading indicators. This step serves to identify the leading indicators more precisely and to describe how they can be observed or measured. Measuring indicators as identified in this process, containing elements like communication, awareness, relationships or experience can be challenging to measure quantitatively, and therefore we recommend considering the inclusion of qualitative measurements.

152



6) Confirming the developed leading indicators through adverse events and experts Finally, the method includes confirmation of the developed leading indicators. This confirmation may contain two elements. First, if available and possible, the leading indicators can be compared to previous adverse events to detect whether the indicators are recognisable in adverse events. Second, the indicators should be presented to expert practitioners who can confirm the relevance of the proposed indicators. 4. Discussion This paper presents an alternative method to indicators (Mainz, 2003). Most research and progress in patient safety has tended to focus on reactive methods towards safety management (Peerally et al., 2016). Indicators developed in a patient safety context have primarily been outcome indicators (Mainz, 2003). However, these methods do not offer indicators and do not indicate which factors in a process are important to achieve success. Typically, patient safety is defined as avoiding, preventing and ameliorating adverse outcomes or injuries stemming from the process of health care (Vincent, 2010). This definition is widely used within the field of patient safety as a basis for developing patient safety initiatives. Thus, measurements should seek indications of a lack of safety, such as the unwanted events mentioned previously (Hollnagel, 2008). This work feeds into a discussion of how to improve work in health care. Understanding incidences using root cause analysis does not prevent the same mistakes from repeatedly occurring (Peerally et al., 2016, Sujan et al., 2016, Sujan, 2015). This study makes a contribution to the field of patient safety by trying to improve processes based on the identification of functions that are vital for success (Hollnagel, 2008). This study applies an alternative approach to the understanding of safety in health care. It argues that safety should not only be characterised by the lack of presence of accidents, but also by how often intended outcomes are achieved (Braithwaite et al., 2015, Hollnagel, 2014a, Hollnagel, 2008). Considering this alternative understanding of safety and applying it to the identification of indicators, the method helps to identify important aspects of a given process (Sujan et al., 2016). The developed indicators are signs for ‘what to focus on in the process in order to assure that things go right’ (Hollnagel, 2014b). This different perspective can be illustrated by comparing it to measures

153



previously applied in sepsis, such as the ‘number of patients transferred to intensive care with severe sepsis or septic shock’ and the ‘number of sepsis patients in intensive care who die’ (Berezowicz, 2013). These indicators represent actual measures used to track the safety of the hospital where this study was conducted. They are also a useful example of describing how this developed approach towards indicators is different from the reactive approach (Hollnagel, 2008). The indicators are not performance-shaping and do not indicate which factors in the process of sepsis treatment are important to achieve success (Berezowicz, 2013). The indicators measure unwanted events with a focus on death and worsened conditions. This fits with the perspective of Safety-I and reactive safety management, which focus on what has gone wrong (Hollnagel, 2014b). Identifying leading indicators of safety is typically a task for high-risk organisations (Grabowski et al., 2007a). Studies show that industries such as oil and gas, nuclear, aviation, production and transport apply the concept of leading indicators to a variety of critical safety processes (Mengolini and Debarberis, 2008, Johnsen et al., 2012, Edkins, 1998, Grecco et al., 2012, Grabowski et al., 2007b). When investigating previously applied methods for identifying leading indicators, no consensus was reached. Each study had its own approach or method towards identification that was specially fitted to the relevant setting (Raben and Hollnagel, 2014). The lack of a consistent method was both a challenge and an advantage. For example, it was not necessary to follow already-set guidelines and face the challenge of adapting it to a different context (Hudson, 2003). Conversely, there was a higher risk of developing a method that did not fit the context, and for which shortcomings might be discovered later in the process. The process of confirming or validating the method is timeconsuming when there is no available method to start with. Issues concerning whether the identified leading indicators were dependent on the researcher were considered. The results were reviewed by informants and employees in the ward and by experts in the studied field. These procedures showed that the results of this study align with those of relevant peers. However, it is not certain whether other individuals would have arrived at the same result. In relation to this statement, we considered the consequences of having limited knowledge of the investigated processes (Malterud, 2001). A lack of clinical training, and therefore a lack of clinical knowledge of the process, enabled the researcher to focus on the entire process and all aspects, as the researcher was not looking for specific issues that were known beforehand. Many actions performed during the daily work were based on tacit knowledge. By being an outsider, one may be more likely to ask questions to uncover such aspects of work. An insider might not consider the importance of

154



this and might not be able to extract it from the situation (Kawulich, 2005). As the objective of the study was to develop an applicable method, limited knowledge of the process enabled the researcher to focus on the method’s development rather than expose specific aspects of sepsis that are deemed important in the field. However, limited knowledge of the investigated process can also be a limitation, as there is a risk of not focusing on the aspects that are important in the case. The researcher may not have the ability to uncover all important factors because the relevant questions may not be asked. One of the main challenges relates to further development of this method and to make it more reliable. It has been developed based on the data collection of one case, as well as the observations, considerations and analysis of one researcher. This is a major consideration for the refinement of the method and will be addressed in a future study, which will focus on testing the methods in other settings and using different cases, as well as allowing other researchers to apply the method to investigate its usefulness and application. This approach will help test the reliability of the method and reveal aspects that need to be described more thoroughly. A final consideration relates to whether the proposed indicators are effectively leading. To determine this, the proposed indicators must be implemented and measured in the wards. Indicators for the early detection of sepsis cases could be used to investigate whether there is a correlation between not achieving the indicators and the occurrence of unwanted outcomes. However, it is important to consider the possibility of making the developed indicators measurable. The extracted indicators primarily describe implicit factors within a process, which can be difficult to measure on a daily basis, as they can be part of the intuitive actions of staff. This does not eliminate the possibility of measuring, but it might require the consideration of qualitative measures (Øien et al., 2011b, Reiman and Pietikäinen, 2014). As the identified indicators consider importance of aspects like communication and relation, heightened awareness and strengthening of ability to detect none clinical symptoms the collection of quantitative data can be limited. Instead, additional observations or interviews could be conducted to inform and support how these factors influence the success of the process and how they can be measured. Using the example of the first indicator ‘receiving and obtaining the necessary and sufficient information on the patient from the referring doctor’, could be assessed through focus group interviews with experienced staff-members in order to extract what questions they typically ask the referring doctor. This knowledge could be extracted and converted to the referring form that nurses already use during referral. Similar initiatives could also be pursued for the other indicators. In

155



addition the analysis of this collected information could be interesting to examine for variability. As we present indicators through the perspective of FRAM, we are interested in measuring and understanding variability because a heightened degree of variability may mean that the process is developing in an unintended direction. If further development results in a strengthening of the application of the method, it should still be considered that different researchers might obtain different results, as qualitative research is usually affected by the individual undertaking it (Malterud, 2001).

5. Conclusion This study used the FRAM and observational studies to understand and map a complex process that served as a framework to develop a systematic method to define leading indicators for the process using a proactive safety management view. This paper aimed to develop a number of systematic steps to identify the leading indicators of a given health care process using a systematic description of the process. This work suggests that these guidelines can help identify and manage important aspects of the process. To successfully identify the leading indicators, six steps were applied in a systematic manner: (1) identify relevant functions, (2) cluster functions into sets, (3) identify the variability of functions in each set, (4) identify upstream–downstream couplings of functions in each set, (5) identify the leading indicators for each set of the process and (6) confirm the developed leading indicators through adverse events and experts. This paper demonstrates how this method can be applied as a framework for studying and defining leading indicators. Further studies are necessary to validate the method developed in this study. Therefore, the systematic method is currently being applied to already-established descriptions of complex processes in health care to assess its usability in a different context. The results of this study will contribute to the discussion of whether safety in health care can be developed further than current methods and approaches allow. This study suggests that new perspectives of health care processes can be explored by focusing on positive outcomes. The proposed method offers a different way of investigating a process that may reveal aspects that have not been considered with methods such as the often-applied root cause analysis, which is driven by analysing near-misses, incidents and accidents. References 156



BEREZOWICZ, P. 2013. Sepsis: svær sepsis & septisk shock, retningslinier for diagnostik og behandling på intensiv afsnit; læge [Online]. Region Syddanmark: Region Syddanmark. Available: http://ekstern.infonet.regionsyddanmark.dk/Files/dokument4281.htm [Accessed 17-03-2017 2017]. BERGH, L. I. V., HINNA, S., LEKA, S. & JAIN, A. 2014. Developing a performance indicator for psychosocial risk in the oil and gas industry. Safety Science, 62, 98-106. BRAITHWAITE, J., WEARS, R. L. & HOLLNAGEL, E. 2015. Resilient health care: turning patient safety on its head. Int J Qual Health Care, 27, 418-20. BROADRIBB, M. P., BOYLE, B. & TANZI, S. J. 2009. Cheddar or Swiss? How Strong Are Your Barriers? Process Safety Progress, 28, 367-372. DE VRIES, E. N., RAMRATTAN, M. A., SMORENBURG, S. M., GOUMA, D. J. & BOERMEESTER, M. A. 2008. The incidence and nature of in-hospital adverse events: a systematic review. Quality and Safety in Health Care, 17, 216-223. EDKINS, G. D. 1998. The INDICATE safety program: evaluation of a method to proactively improve airline safety performance. Safety Science, 30, 275-295. EXECUTIVE, H. A. S. 2001. A guide to measuring health and safety performance. UK: Health and Safety Executive. GRABOWSKI, M., AYYALASOMAYAJULA, P., MERRICK, J., HARRALD, J. R. & ROBERTS, K. 2007a. Leading indicators of safety in virtual organizations. Safety Science, 45, 1013-1043. GRABOWSKI, M., AYYALASOMAYAJULA, P., MERRICK, J. & MCCAFFERTY, D. 2007b. Accident precursors and safety nets: leading indicators of tanker operations safety. Maritime Policy & Management, 34, 405-425. GRECCO, C. H. S., VIDAL, M. C. R., COSENZA, C. A. N., SANTOS, I. J. A. L. & CARVALHO, P. V. R. 2012. New Approach for Safety Management in Radiopharmaceutical Production Facilities. Latin American Journal of Pharmacy, 31, 1199-1202. HERRERA, I. A., HOLLNAGEL, E. & HÅBREKKE, S. 2010. Proposing safety performance indicators for helicopter offshore on the Norwegian Continental Shelf. PSAM. Seattle.

157



HOLLNAGEL, E. 2008. Safety Management - Looking back or looking forward. In: HOLLNAGEL, E., NEMETH, C. P. & DEKKER, S. (eds.) Resilience Engineering Perspectives: Remaining sensitive to the possibility of failure. Ashgate. HOLLNAGEL, E. 2012. FRAM: the Functional Resonance Analysis Method - Modelling Complex Socio-technical Systems, Surrey, England, Ashgate Publishing. HOLLNAGEL, E. 2013. Making Health Care Resilient: From Safety-I to Safety-II. In: HOLLNAGEL, E., BRAITHWAITE, J. & WEARS, R. L. (eds.) Resilient Health Care - Volume 1. Surrey, England: Ashgate Publishing. HOLLNAGEL, E. 2014a. Is safety a subject for science? Safety Science, 67, 21-24. HOLLNAGEL, E. 2014b. Safety-I and Safety-II - The Past and Future of Safety Management, Surrey, England, Ashgate Publishing. HUDSON, P. 2003. Applying the lessons of high risk industries to health care. Quality and safety in health care, 12, i7-i12. JOHNSEN, S. O., OKSTAD, E., AAS, A. L. & SKRAMSTAD, T. 2012. Proactive Indicators To Control Risks in Operations of Oil and Gas Fields. KAWULICH, B. B. 2005. Participant Observation as a Data Collection Method. 2005, 6. KRISTENSEN, S., MAINZ, J. & BARTELS, P. 2009. Selection of indicators for continuous monitoring of patient safety: recommendations of the project 'safety improvement for patients in Europe'. Int J Qual Health Care, 21, 169-75. MAINZ, J. 2003. Defining and classifying clinical indicators for quality improvement. Int J Qual Health Care, 15, 523-30. MALTERUD, K. 2001. Qualitative research: standards, challenges, and guidelines. Lancet, 358, 483-8. MENGOLINI, A. & DEBARBERIS, L. 2008. Effectiveness evaluation methodology for safety processes to enhance organisational culture in hazardous installations. Journal of hazardous materials, 155, 243-252.

158



PEERALLY, M. F., CARR, S., WARING, J. & DIXON-WOODS, M. 2016. The problem with root cause analysis. BMJ Qual Saf. PRONOVOST, P. J., RAVITZ, A. D., STOLL, R. A. & KENNEDY, S. B. 2015. Transforming Patient Safety - A Sector-wide Systems Approach. WISH Patient Safety Forum 2015. RABEN, D. C. & HOLLNAGEL, E. 2014. Unpublished Results - Exploring leading indicators in high-risk industries and what health care can learn? – a systematic review on the literature. RASMUSSEN, J. 1983. Skills, rules, and knowledge; signals, signs, and symbols, and other distinctions in human performance models. IEEE transactions on systems, man, and cybernetics, 257-266. REIMAN, T. & PIETIKÄINEN, E. 2014. Patient Safety Indicators as Tools for Proactive Management and Safety Culture Improvement. In: WATERSON, P. (ed.) Patient Safety Culture: Theory, Methods and Application Farnham: Ashgate. SUJAN, M. 2015. An organisation without a memory: A qualitative study of hospital staff perceptions on reporting and organisational learning for patient safety. Reliability Engineering & System Safety, 144, 45-52. SUJAN, M. A. 2016. What keeps patients safe? A Resilience Engineering perspective. Humanist on Human Centred Design for Intelligent Transport Systems. Loughborough: Lougborough University. SUJAN, M. A. & FELICI, M. 2012. Combining Failure Mode and Functional Resonance Analyses in Health care Settings. Computer Safety, Reliability, and Security. Springer Berlin Heidelberg. SUJAN, M. A., HUANG, H. & BRAITHWAITE, J. 2016. Learning from incidents in health care: Critique from a Safety-II perspective. Safety Science. VINCENT, C. 2008. Patient safety, Wiley-Blackwell. VINCENT, C. 2010. Integrating safety and quality. Patient Safety, 2nd edition, 31-46.

159



VINCENT, C., BURNETT, S. & CARTHEY, J. 2014. Safety measurement and monitoring in health care: a framework to guide clinical teams and health care organisations in maintaining safety. BMJ Qual Saf, 23, 670-7. WEARS, R. L., SUTCLIFFE, K. M. & VAN RITE, E. 2014. Patient Safety: A Brief but Spirited History. In: ZIPPERER, L. (ed.) Patient Safety: Perspectives on Evidence, Information and Knowledge Transfer. Surrey, England: Ashgate Publishing. ØIEN, K., MASSAIU, S., TINMANNSVIK, R. K. & STØRSETH, F. Development of early warning indicators based on Resilience Engineering. Submitted to PSAM10, International Probabilistic Safety Assessment and Management Conference, 2010. 7-11. ØIEN, K., UTNE, I. B. & HERRERA, I. A. 2011a. Building Safety indicators: Part 1 - Theoretical foundation. Safety Science, 49, 148-161. ØIEN, K., UTNE, I. B., TINMANNSVIK, R. K. & MASSAIU, S. 2011b. Building Safety indicators: Part 2 - Application, practices and results. Safety Science, 49, 162-171.

160



Paper IV Title: Proposing leading indicators for blood sampling - application of a method based on the principles of Resilient Health Care Status: Submitted to Cognition, Technology & Work – 24th of June 2017 Ditte Caroline Raben, MScPH, Institute of Regional Health Research, University of Southern Denmark and Centre for Quality, Region of Southern Denmark, Denmark. Address: P.V. Tuxensvej 5, DK-5500 Middelfart, Denmark. E-mail: [email protected] Søren Bie Bogh, MHSc & Ph.d., Institute of Regional Health Research, University of Southern Denmark and Centre for Quality, Region of Southern Denmark, Denmark. Address: P.V. Tuxensvej 5, DK-5500 Middelfart, Denmark. E-mail: [email protected] Birgit Viskum, Consultant, Public Health Medicine, Aborgmindevej 9, 5610 AssensE-mail: [email protected] Kim L. Mikkelsen, Ph.d., Medical Coordinator, Danish Patient Insurance Association, Address: Kalvebod Brygge 45, 4.sal, 1560 København V, Danmark E-mail: [email protected] Erik Hollnagel, Ph.D., Professor, Institute of Regional Health Research, University of Southern Denmark and Chief Consultant, Centre for Quality, Region of Southern Denmark, Denmark. Address: P.V. Tuxenvej 5, DK-5500 Middelfart, Denmark E-mail: [email protected] Word count: 4251

161



Abstract In recent years, health care has put a growing attention to the investigation of successful processes as a supplement to analyzing and investigating unwanted processes, like adverse events and near misses. This new perspective paves the way for developing methods and tools for investigating and understanding how processes function, and how variability can contribute to both success and failure. In the light of this, we have developed a method applicable for identifying leading indicators for successful outcomes of complex health care processes. The method, which is termed Leading Indicator Identification Method (LIIM) was inspired from similar methods applied in high-risk industries. To demonstrate the usefulness of the method we have conducted a case study with the aim of identifying leading indicators for blood sampling among patients in a Biomedical Department within a Danish hospital. The method builds on and uses steps from the Functional Resonance Analysis Method (FRAM). FRAM was developed to analyze how work is performed on a daily basis, in complex systems and can be used prospectively to monitor, manage and control such systems. The contribution of the work is to present the LIIM along with four leading indicators that are important to consider in the planning, management and monitoring of the blood sampling process.

162



Background Creating a safe health care system is a topic of great importance and growing attention. To investigate whether health care is getting safer, from the considerable efforts made to improve safety, measurements and evaluations need to be part of the agenda (Vincent et al., 2013). However, relying on measurements and hindsight is not always enough. Recently, a growing focus upon a more forward looking and proactive approach to safety than the traditional look at errors and incidences has suggested that investigating safety cannot solely be done by measuring harm. This proactive perspective focuses on how health care systems succeeds by rapidly responding and adapting performance in everyday work. But these aspects of creating a safe health care are often invisible to the system, and therefore challenging to measure and detect through adverse event measuring (Vincent et al., 2013). This new perspective of safety, labelled Safety-II stresses that the focus on harm, error or incidence can benefit from being supplemented with looking at aspects like anticipation, preparedness and resilience (Vincent et al., 2013). The relevance of Safety-II becomes evident as methods to improve safety in health care processes include root cause analysis, failure mode effect analysis and fault tree analysis all try to improve the system based on learning from errors (Peerally et al., 2016, Sujan et al., 2016). However, numerous studies have shown that accidents and adverse events still occur on a regular basis despite this continuing and increasing efforts to avoid them. Therefore, it seems increasingly relevant to investigate other ways to achieve safe health care processes (Rafter et al., 2015, de Vries et al., 2008). The limited effect of using traditional safety methods might be due to the fact that adverse events are often unique and emerge as a consequence of numerous combined factors (Hollnagel, 2012, Hollnagel et al., 2007). Using methods like root cause analysis or fault tree analysis will be able to uncover different factors contributing to a specific event investigated, but are not likely to prevent the same factors combined differently to create a different event (Braithwaite et al., 2013, Peerally et al., 2016). Working towards the goal of creating a safer health care system, the classical safety perspective states the question of why do things go wrong? (Pickup et al., 2017) This study, however, presents a different question that can be asked: how and why do things go right (Hollnagel, 2014)? The research on patient safety shows a gap in investigating the large number of times, that things in health care actually proceed as intended. Investigating preconditions and deciding factors for why things go right is sparsely represented in health care research (Singer and Vogus, 2013, Paté‐Cornell et al., 1997) despite several well-known institutions have stressed the importance of having a balanced approach towards measuring safety by combining both lagging and leading factors for successful health care 163



processes (Vincent et al., 2013, Executive, 2001). The inclusion of measuring leading factors for successful outcomes can complement the already large amount of data on past events, like incidence and near miss reporting. This dual approach to understanding processes and ensure safety has long been applied in high-risk industries (Øien et al., 2011a, Vincent et al., 2013, Reiman and Pietikäinen, 2014). This study addresses the leading indicators and presents a method, developed to identify leading indicators in health care processes. In this context, leading indicators are defined as signs or symbols used as active monitoring to control current and future behaviors or actions to achieve desired outcomes (Rasmussen, 1983, Øien et al., 2011a). Applying leading indicators to a complex system enables staff to gain a greater understanding of what goes on in the process and which parts have an impact on a desired outcome (Herrera et al., 2010). The method was developed with emphasis on heightening understanding of a complex system to change, manage or improve it. The method includes six step applied to a defined process to define leading indicators for the process (Raben et al., 2017). The method is still in an early stage and this article concentrates on developing LIIM further and to demonstrate how the method can be applied to a process in the field of patient safety. Testing it will help it become more robust and useful for others. To strengthen the method and uncover potential weaknesses or confusions, the aim of this study was to apply the method to a complex process and investigate applicability. Further, the article includes a reflection of the feasibility of the method and the challenges or difficulties revealed in the process. The LIIM - Leading Indicators Identification Method Developing methods to identify leading indicators for safety is a common process in many high-risk industries, including aviation, oil and gas and nuclear power (Edkins, 1998, Johnsen et al., 2012, Øien et al., 2011b). Uses in other high-risk industries show how different theoretical approaches help create a systematic guideline for identifying leading indicators often enabling a more holistic understanding of the system to create better safety awareness (Øien et al., 2011a). Previously, no attempts of developing such systematic guidelines have been reported in health care. The inspiration for the LIIM came from literature and history on leading indicators in high-risk industries. The literature revealed a fragmented field of methods for identifying leading indicators in high-risk industries that indicated that a context specific approach needed to be developed, fitted to a health care context (Øien et al., 2011b). Therefore, the LIIM was developed with industry knowledge, concepts drawn from system 164



theory, and application of the Functional Resonance Analysis Method (FRAM), previously shown to be useful in health care (Hollnagel, 2012, Checkland, 2000, Clay-Williams et al., 2015, Pickup et al., 2017, Laugaland et al., 2014). The early development and demonstration of LIIM had an experiential focus, and was carried out to test the applicability of LIIM in diverse systems (Raben and Hollnagel, 2014). The results are presented according to the six steps of LIIM, with a description of the steps followed by a section of how each step was applied to the case. The context: Blood sampling The LIIM method was applied to a case, describing blood sampling in a Department for Clinical Biochemistry in a Danish hospital in a FRAM model, developed from data material gathered in early 2014. The case analysis described a common, yet complex process taking place on a regular basis in the health care system. The FRAM model covered the process from ordering blood samples until the samples were sent of for analysis. The intended outcome and therefore success criteria of this process was to obtain the correct blood sample from the correct patient and having it sent for analysis within the intended timeframe, ensuring a safe flow of patients through the department (Pickup et al., 2017). The data for the model was collected by an independent researcher and not in relation to this study. The material for developing the FRAM model was collected through qualitative data collection methods, including semi-structured interviews with all involved staffing groups, walk-arounds with staff in the working environment and collection of narrative stories (Hounsgaard, 2016). A total of 5 interviews were performed with a six staff members (4 single interviews and 1 interview with two staff members) (Hounsgaard, 2016). The objective of using an unfamiliar case was to investigate the usefulness of the LIIM on a case with no preliminary understanding of the process that will typically following a data collection period (Kawulich, 2005). The process of applying the method The LIIM includes six steps (I) Identification of relevant functions, (II) clustering functions into sets, (III) identification of variability of functions, (IV) identification of upstream-downstream couplings of functions, (V) combining variability and upstream-downstream couplings to identify candidate leading indicators and (VI) confirming candidate leading indicators with experts. Step 1 – Identification of relevant functions

165



The process under investigation needed to be clearly defined to identify relevant functions. This step included developing a FRAM model of the case of analysis, typically including semi-structured interviews supplemented by observations or walk-throughs with staff members. All functions, which might influence the process, were included in the model at this point. How to identify relevant functions As the FRAM model of the process was not constructed by the present researcher, it was necessary to become familiarized with the model, by reconstructing the model, and all the connections and relations in the FRAM model were talked through in the research group. The model presented in Figure 1 illustrates the detailed and modified model.

166



Figure 1: Visual representation of process 'Taking blood samples'

167



Talking through the model and going through all the background material made it possible to understand the process illustrated, and the different relations and scenarios in the process. Going through and discussing all functions also helped in the overall understanding of the process and its complexity. In hindsight, it could add more understanding if the model was not only talked through in the research group, but also with frontline staff, conducting the process. Step 2 - Clustering functions into sets The aim of this step was to look at the process with the intention of clustering functions into different sets. This helped detect how sets of functions work together to achieve the desired outcome. Sets were identified by dividing the process into smaller subtasks important for achieving the overall task. How to cluster functions into sets In this process four sets of functions were found, each representing a subtask performed in relation to the final intended outcome, the successful blood sample taken from the patient. The four sets were (1) the prior assessment and prescription of blood samples, (2) receiving the patient in the blood sample clinic and preparing paperwork, (3) waiting for turn and keeping track of patients with special needs, and finally (4) taking correct blood samples and preparing them for analysis. The sets followed the patient pathway, in a chronological order ending with the final intended outcome, which in this case was ‘Taking the right blood samples from the right patient and preparing them for analysis within the right timeframe’. The division was thus based on sets of functions that each included a task that contributed to obtaining the overall intended result. Step 3 - Identifying variability of functions After categorizing functions into sets, each individual function was investigated. Special emphasis was on how variability in the functions could affect both the output of the immediate function as well as the outcome of the overall process. This could cover examples like; variability if the function was delayed, variability if information was not transferred accurate or variability if previous functions were not conducted properly. This step was done by systematically analyzing each function for variability. This process included analyzing whether variability in the function caused the process to be delayed or whether the quality of how functions were performed compromised the successfulness of the process as a whole. How to identify variability of functions

168



Uncovering variability in the process was conducted in a systematically way, where all four sets of functions were set up in tables. Under each set, the functions were listed and considered for variability. Functions like, prescribing blood samples, searching for prescription on blood samples, identifying patients with special needs, patients walking to room when called, calling the patient in and preparing the sample for transport were functions with several potential outcomes based on variability in the function, all having between 3-4 possible outcomes. All functions and variability can be found in Table 1. One of the main and important challenges of this step was to consider whether the detected functions with variability would potentially affect the intended outcome. The LIIM focused on identifying leading indicators, which guide decision-making and actions among staff members, therefore the challenge was not to be able to identify and name all potential variability of functions, but to be aware of the most occurring and typical variability.

169

PhD Thesis—Leading indicators Set 1: Prior assessment prescription of blood samples

Ditte Caroline Raben and

Set 2: Receiving patient and preparing paperwork

Set 3: Waiting for turn and keeping track of special need patients

Set 4: Taking blood samples and preparing for analysis

Conducting control visit in department Performed on time prior to clinic visit – on time

Receiving patient in blood clinic Many patients at clinic – delay Experience level of front desk staff delay Searching for prescription on blood samples Prescription not send – delay Prescription not received/lost in system – delay Searching for paper if IT system is not functioning – delay Searching on patient name for missing sample – delay

Having the experience to identify special need patients No experience – delay No experience -unprecise Keeping track of screen for patients Identification of wrong patients – delay or early Busyness on the ward – delay Busyness on the ward – unprecise

Call the patient into the blood sample room Calling wrong patient – unprecise Patient is not hearing – delay Searching for patient – delay Print labels for blood samples Wrong labels printed – delay Printing not possible – delay

Prioritize queue number based on patient characteristics Evaluate patient to priorities queue – unprecise Wrong evaluation of patients – delay

Patients wait until number called on screen Walking to early – early Not keeping track of number called – delayed Not keeping track of own number – unprecise Purposefully walking somewhere else than instructed - delay Guiding special need patients into blood sample room Not identified – delay Finding patient – delay Busyness prevents assisting – delay or unprecise Patient walks into blood sample room Walking into wrong room and going back – delay Walking into wrong room and having samples taken – precise

Check for urgency and potentially order urgent transport Urgency not detected – delay

Prescribing blood samples Prescription not send – delay Prescription not received in clinic – delay IT system unavailable and manual prescription send – delay Wrong patient or sample prescribed – delay Patient is walking into blood clinic Does not go straight to reception – delay Does not have social security card with him – delay

Identification of special need patients Ability to identify special needs – unprecise Experience of identification unprecise

Take blood samples from patient Personal factors – delay

Put labels on blood sample glass after sample taking Wrong labels – delay Wrong labels - unprecise Prepare sample for further transport Patient arrives late – delayed Wrong labels on – unprecise Labels missing – unprecise Samples archived wrong - delay

Table 1: All four sets of functions of ‘blood samples’ where variability is presented under each function

170



Step 4 - Identifying upstream-downstream couplings of functions After investigating the variability of functions in the process, the next step was to analyze whether the detected variability had the ability to affect downstream functions in the process. Each function was at this step investigated with emphasis on connections to other functions, and how different scenarios affected these connections. The main focus was to analyze whether the variability was able to resonate through the process and affect the quality of the desired outcome (Hollnagel, 2012). How to identify upstream-downstream couplings of functions This step started with an analysis of the variability, identified in step 3. Potential consequences of the variability to downstream functions was assessed. For example, for the function ‘prescribing blood samples’, the following potential delaying variabilities were identified: prescription not send, prescription not made, IT system unavailable, manual prescription send and wrong patient or sample prescribed. The different scenarios were then analyzed considering how they would affect downstream functions. During this analysis of functions and their couplings, it became evident that the variability typically was either detected and stopped or dampened when the patient arrived and received at the blood clinic. This was ensured as the front desk at the clinic served as a gatekeeper for ensuring right prescription and functioning as a helping service for getting patients organized. This process of considering consequences of variability was done for all functions, helping to identify the functions where variability actually affected downstream functions and eventually the outcome. This was exemplified in the function ‘Patients walking into the blood sample room’. The data material showed variability could happen if: patients walked into a wrong room and were send back to wait, when patient walked into a wrong room and blood samples were taken and labelled correctly, or if patients walked into a wrong room and samples were taken and labelled wrong. Both the second and the third variability affected the outcome, taking blood samples – either delaying it or causing an error, with wrong match of patients and sample glasses. The particular challenge at this step was to connect the variability of individual functions with the consequences the variability produced and how this progressed in the process. Typically, the model showed that variability was either was emphasized, by creating increasing variability later on or dampened by other functions during the process. This part was especially important in the verification process with staff members, as they had hands on experience with these consequences.

171



Step 5 – Combining variability and upstream-downstream couplings of functions to identify candidate leading indicators At this point, each function had been analyzed regarding variability and upstream-downstream couplings separately. To be considered as functions suitable for leading indicators, both previously mentioned aspects, variability in the function and couplings to other functions causing resonance in the system were important. These aspects were particularly important in the distinction of which functions were important factors or actual leading indicators for possible future events. Functions where variability occurred were important to analyze in regards to how they could resonate through the system. Functions with variability that was able to resonated through the system were detected as candidates for leading indicators. This was associated with staff behavior and attitudes as detected variability was often with potential future scenarios, both positive and negative. How to identify candidate leading indicators based on variability and upstream-downstream functions Combining the findings from steps 3 and 4 helped identify the functions that were often affected by variability and where this variability affected downstream functions. This process displayed that the functions of ‘prioritizing queue number based on patient characteristics’ and ‘Identification of patients with special needs’ were functions both identified as being leading functions for successful outcomes. This was evident as both functions could be subjects to variability and that this variability in both cases had the ability to create downstream variability. This meant that staff members through experience had identified that some patients had tendency to not follow the number system, or choose a preferred laboratory technician in spite of being assigned a different technician. This experience alerted them of being especially observant of these patients when they came in the clinic, as they made sure that the correct patient entered the correct room at the correct time. This process was an illustration of why this function is a relevant indictor for a successful process. However, the data also showed that different situations made it challenging to detect these patients due to time pressure, high work pressure or lack of experience in the staff. All of these variabilities could cause the patient to proceed in the process unassisted or unsupervised. Therefore, later functions like ‘patients wait until number called on screen’ or ‘patient walks into blood sample room’ transferred the variability further in the process.

172



This combination of variability and upstream-downstream couplings resulted in the selection of four functions in the process, identified as leading indicators. These were ‘to prioritize the queue’, ‘to identify patients with special need’, to keep track of screen’ and ‘to walk to blood sampling room when called’. By identifying these functions as leading indicators, they can be used as active tools to manage the process, for example in training of new staff or serve as a reminder of which organizational or structural aspects help achieve the intended outcome. Step 6 - Confirming the identified leading indicators through expert views Confirming usefulness and relevance of the proposed leading indicators is an important aspect of the method and two different possibilities have been pursuit in previous cases. Consulting experts in the field or staff working in the context and with the process on a daily basis represented a valuable source of knowledge for this case. How to confirm the identified leading indicators through expert views To investigate the relevance of the identified indicators a focus-group interview with three staff members working in the unit was conducted. The interview was set up semi-structured, allowing the staff to talk freely about the process, and what they found important, but still emphasizing that they covered and discussed the relevance of the leading indicators identified in this case. The interview was afterwards transcribed and categorized based on the four identified leading indicators. In the following, each of the leading indicators is presented from the perspective of the interviewees. Prioritizing the queue was a task partly administered by staff and by the computer systems. Interviewees mentioned that they had different numbering systems to put patients in. Normally, these number systems were used to categorize patients based on factors like accelerated patients, day patients, normal samples, children, project based samples and others. However, interviewees agreed that sometimes they used a different category, in order to accelerate a patient or getting samples taken without waiting in the queue. This was exemplified as interviewees mentioned: ‘..when we give them a number, we can give a them a accelerated number’ or ‘when there is an hour long queue and someone comes to deliver a urine sample, they should not sit and wait an hour, then we give them a glass and let them deliver the sample and then we prepare the sample when we are working at the front desk’. Identifying patients with special needs showed to be a central and important aspect of ensuring a good flow of patients through the ward. Interviewees mentioned both patients with walking disability,

173



foreign patients or patients that did not understand the numbering system and waiting queue. Staff coped with this by calling patients in advance, and having them sit on a chair in front of the treatment room. One interviewee mentioned ‘sometimes patients have walking disability, so it takes a long time to walk to the treatment room, so if we call them in before taking samples from the previous patient, then they have a couple of minutes to walk to the room.’ Sometimes challenges with these patients also resulted in taking a different patient in than scheduled, a scenario also represented in the model. As explained by an interviewee ‘Yes or sometimes we just take them in, some do not understand that they need to wait until there number is called, so they just come in and sit in front of the treatment room, and you don’t want to bother explaining it to them and then you take them in and sent them of. Why explain instead of just taking the samples?’ The third function ‘keeping track of screens’ also revealed to be a good indicator. It was a task typically performed by staff, when working at the front desk and receiving patients. This indicator was especially influenced by the previous indicator, as staff members typically monitored numbers of patients identified with special needs. Discussing this part in the interview also revealed, that the front desk personnel had many challenges and tasks. They need to make sure all categories were regularly taken in by checking numbers or sometimes answering to questions, of why patients had received a certain number. These aspects were illustrated as interviewees said, ‘when you stand at the front desk you function as a problem solver, I do not know what to do her, or why did you send this patient here..’ and ‘they (front desk staff) make sure all categories are being taken in, and they go in and say if a patient needs to be squeezed in’. Some challenges associated with the fourth function, identified as a leading indicator, ‘to walk to blood sampling room when called’ have been addressed previously, in the section of identifying patients with special needs. However, in some cases the system also challenged the flow of patients. An interviewee mentioned that some patients did not understand where to go when there number was called. So, instead of walking to and waiting in front of the treatment room they would act differently as they became confused if previous patients were not done. The following was explained, ‘no, many come and see that someone is having samples taken and they walk back and check the screen and come and look in the door, so you actively have to instruct them’ and ‘sometimes they just walk into the treatment room, even though we are dealing with another patient.’ The findings in the interview revealed that the functions identified as good candidates for leading indicators have an impact on how well the process is performed as well as how the flow in the ward functions to make sure that patients are being treated timely and safely. 174



Discussion LIIM adds to the quest of creating a safe and high quality health care system. It emphasizes that to enhance the safety of health care, a different and more holistic approach to safety can be relevant to explore that not only focuses on errors, but rather the functioning of a system, both in case of failures and success (Hollnagel, 2014). In this study, we present the second time the LIIM has been applied and tested on a health care process. Attempts so far have been positive and resulted in the identification of leading indicators, which previously have not been mentioned or highlighted as decisive factors in these processes. The presentation of the candidate functions for leading indicators even resulted in interesting and central perspectives of process management in health care, as the interviews mentioned ‘the ones that make the guidelines or instructions rarely have a feeling of how we work’ and ‘why do they never ask how it will function in reality before they implement something new..’. We believe that these statements underlines the fact that a different approach to viewing processes and the functioning of processes in health care can be rewarding and beneficial. Despite this positive aspect, a number of topics should be considered in the future applications of the LIIM and will be discussed in the following. The aim of identifying leading indicators is to increase the number of events with successful outcomes and understand how variability in some cases can lead to unintended outcomes (Herrera and Hovden, 2008, Øien et al., 2011a). However, we recommend that the identification of leading indicators should be done on clearly defined processes. This improves the indicators usability and relevance. Next, contextual understanding can have an impact on the identification of leading indicators. Visualizing the process through walk-arounds, interviews and narratives might strengthen the process and the overall understanding of the complexity of the process. Yet, the confirmation of the leading indicators with involved staff showed, that despite not having conducted the data collection the method is able to guide the identification of leading indicators based on material and data provided by a FRAM model. In this case experiences have shown, that the nature of the FRAM, of including variability of functions and the identification of upstream-downstream couplings, help extract what functions staff use to make decisions (Clay-Williams et al., 2015). Further, we considered that indicators developed to fit specific processes in health care are rarely measurable from existing data. This assumption is based on the traditions and current types of health care indicators. Error detection is still a central aspect in health care, and few measures are dedicated towards positive events. The LIIM is based on the understanding that leading indicators are

175



observable characteristics of a process, presumed to have a positive correlation with safety of the process (Herrera and Hovden, 2008). Therefore, leading indicators might not realistically be based on existing collected data nor are they necessarily aspects that have to be measured systemically. Rather they should be seen as part of the systems safety management and provide information on specific processes from a safety perspective, motivate people to work on safety and increase understanding of what contributes to success (Reiman and Pietikäinen, 2014). The indicators represent the organizational attributes that enable safe performance and interactions every day. Conclusion This study reports on the LIIM. Health care is a complex socio-technical system, which includes a large number of influencing factors. To develop a method to improve management and understanding of health care processes, it is relevant to consider which methods are able to carefully describe the complexity and interrelations of such a system. We believe we have presented a method that meets this criteria. The findings of this study support the assumption that the method is applicable on different processes related to health care processes and can help uncover the importance of functions or activities in health care processes that are normally not considered. The identification of candidate leading indicators can especially contribute to two aspects. Firstly, they can highlight factors and conditions that are influencers on successful performance. Secondly, they can pave the way for working proactively in the development of safety strategies for health care processes, as they offer pointing at some of the organizational, social and psychological factors that influence performance in health care process. References BRAITHWAITE, J., CLAY-WILLIAMS, R., NUGUS, P. & PLUMB, J. 2013. Health care as a complex adaptive system. In: HOLLNAGEL, E., BRAITHWAITE, J. & WEARS, R. L. (eds.) Resilient Health Care - Volume 1. Surrey, England: Ashgate Publishing. CHECKLAND, P. 2000. Soft systems methodology: a thirty year retrospective. Systems Research and Behavioral Science, 17, S11.

176



CLAY-WILLIAMS, R., HOUNSGAARD, J. & HOLLNAGEL, E. 2015. Where the rubber meets the road: using FRAM to align work-as-imagined with work-as-done when implementing clinical guidelines. Implement Sci, 10, 125. DE VRIES, E. N., RAMRATTAN, M. A., SMORENBURG, S. M., GOUMA, D. J. & BOERMEESTER, M. A. 2008. The incidence and nature of in-hospital adverse events: a systematic review. Quality and Safety in Health Care, 17, 216-223. EDKINS, G. D. 1998. The INDICATE safety program: evaluation of a method to proactively improve airline safety performance. Safety Science, 30, 275-295. EXECUTIVE, H. A. S. 2001. A guide to measuring health and safety performance. UK: Health and Safety Executive. HERRERA, I. & HOVDEN, J. Leading indicators applied to maintenance in the framework of resilience engineering: a conceptual approach. Paper presented at The 3rd Resilience Engineering Symposium, 2008. 30. HERRERA, I. A., HOLLNAGEL, E. & HÅBREKKE, S. 2010. Proposing safety performance indicators for helicopter offshore on the Norwegian Continental Shelf. PSAM. Seattle. HOLLNAGEL, E. 2012. FRAM: the Functional Resonance Analysis Method - Modelling Complex Socio-technical Systems, Surrey, England, Ashgate Publishing. HOLLNAGEL, E. 2014. Safety-I and Safety-II - The Past and Future of Safety Management, Surrey, England, Ashgate Publishing. HOLLNAGEL, E., WOODS, D. D. & LEVESON, N. 2007. Resilience engineering: concepts and precepts, Ashgate Publishing, Ltd. HOUNSGAARD, J. 2016. Patient Safety in Everyday Work, Learning from things that go right. Master Master Thesis, University of Southern Denmark. JOHNSEN, S. O., OKSTAD, E., AAS, A. L. & SKRAMSTAD, T. 2012. Proactive Indicators To Control Risks in Operations of Oil and Gas Fields. KAWULICH, B. B. 2005. Participant Observation as a Data Collection Method. 2005, 6.

177



LAUGALAND, K., AASE, K. & WARING, J. 2014. Hospital discharge of the elderly--an observational case study of functions, variability and performance-shaping factors. BMC Health Serv Res, 14, 365. PATÉ‐CORNELL, M. E., LAKATS, L. M., MURPHY, D. M. & GABA, D. M. 1997. Anesthesia patient risk: a quantitative approach to organizational factors and risk management options. Risk Analysis, 17, 511-523. PEERALLY, M. F., CARR, S., WARING, J. & DIXON-WOODS, M. 2016. The problem with root cause analysis. BMJ Qual Saf. PICKUP, L., ATKINSON, S., HOLLNAGEL, E., BOWIE, P., GRAY, S., RAWLINSON, S. & FORRESTER, K. 2017. Blood sampling - Two sides to the story. Applied Ergonomics, 59, Part A, 234-242. RABEN, C., BIE BOGH, S., VISKUM, B., MIKKELSEN, K. L. & HOLLNAGEL, E. 2017. Unublished Results - Learn from what goes right: a demonstration of a new systematic method for identification of leading indicators in health care. RABEN, D. C. & HOLLNAGEL, E. 2014. Unpublished Results - Exploring leading indicators in high-risk industries and what health care can learn? – a systematic review on the literature. RAFTER, N., HICKEY, A., CONDELL, S., CONROY, R., O'CONNOR, P., VAUGHAN, D. & WILLIAMS, D. 2015. Adverse events in health care: learning from mistakes. QJM, 108, 273-7. RASMUSSEN, J. 1983. Skills, rules, and knowledge; signals, signs, and symbols, and other distinctions in human performance models. IEEE transactions on systems, man, and cybernetics, 257-266. REIMAN, T. & PIETIKÄINEN, E. 2014. Patient Safety Indicators as Tools for Proactive Management and Safety Culture Improvement. In: WATERSON, P. (ed.) Patient Safety Culture: Theory, Methods and Application Farnham: Ashgate. SINGER, S. J. & VOGUS, T. J. 2013. Reducing hospital errors: interventions that build safety culture. Annual review of public health, 34, 373-396.

178



SUJAN, M. A., HUANG, H. & BRAITHWAITE, J. 2016. Learning from incidents in health care: Critique from a Safety-II perspective. Safety Science. VINCENT, C., BURNETT, S. & CARTHEY, J. 2013. The measurement and monitoring of safety. London: The Health Foundation, 2013. ØIEN, K., UTNE, I. B. & HERRERA, I. A. 2011a. Building Safety indicators: Part 1 - Theoretical foundation. Safety Science, 49, 148-161. ØIEN, K., UTNE, I. B., TINMANNSVIK, R. K. & MASSAIU, S. 2011b. Building Safety indicators: Part 2 - Application, practices and results. Safety Science, 49, 162-171.

179