Workshop Discussion Papers - UCL

Workshop Discussion Papers

With the support of the European Commission Sixth Framework Programme for Research and the Portuguese Ministry of Health

Contents

Understanding human action in preventing or causing adverse events René Amalberti

4

Can indicators of hospital clinical outcomes demonstrate patient safety? Paul D. Bartels

20

If a tree fell in the forest but no one heard, is it because they did not want to hear? Margarida Franca & Vahé A. Kazandjian

35

What roles for the patient in patient safety research? Jill Hall

47

What is effective training for patient safety researchers? Peter Norton

62

Estimating the level of harm from healthcare in data poor environments. Sisse Olsen

79

Effectiveness and sustainability of interventions to improve patient safety: what do we know and how can we know more? John Ovretveit

104

Root cause analysis: is it the answer? Dianne Parker

126

Can we control healthcare associated infections? Hervé Richet & Hugo Sax

135

The role of patient safety culture Cordula Wagner

146

Workshop discussion paper: Understanding human action in preventing or causing adverse events? Research Agenda Track: Parallel Workshops Session 2 Tuesday 25th Sept, 2007.

4

Patient Safety Research shaping the European agenda 24-26th September 2007: Porto, Portugal

WORKSHOP DISCUSSION PAPER

Understanding human action in preventing or causing adverse events René Amalberti

The author thanks Dr Rosemay Rushmer, PhD, and all the attendants of the session for their helpful commentaries and contribution. This paper includes a specific paragraph in each sub section (ITs, PSI, Resilience) summarizing the debates during the session.

René Amalberti Professor, IMASSA & HAS Vice CEO IMASSA, Resource specialist patient safety at HAS

5

Summary : The medical system is considerably error-prone for many reasons: (i) an incredible organizational complexity associated with contradictory stakes (wide hospital disparities, round-the-clock obligation of service, chronic understaffing, no possibility to turn down clients, multiples trades, budget constrains), (ii) a series of poor ergonomics design (as well for basic problems such as drug labelling as for macro-ergonomics and the design of Healthcare organizations), and (iii) a poor safety culture (craftsmen attitude, reluctance to report, punitive organizational culture). Short term progresses are associated with a better identification and comprehension of human error (HE), more standardization, better ergonomics, introduction of Information technology (IT), and growing auditing processes. Quality assurance and indicators are core concepts for achieving these goals. Mid term wins are expected with a change of culture, giving priority to any strategies favouring team building, reporting, disclosing errors, patient involvement, and conveying positive learning attitudes. The introduction of ITs and the selection of patient-safety indicators are proposed as two examples that deserve research attention. A third topic of research is the systemic approach of patient safety; indeed, an even greater challenge for patient safety leaders now addresses the choice of a win-win combination of local and global strategies on the long term. Human error cannot be totally eliminated. Moreover the considerable variety of patients’ cases asks for a fairly adaptive and resilient medical system. Betting exclusively on a greater constraint at the individual level to improve safety might have negative longer time consequences, either creating rooms for additional violations (unachievable goals with safety constraints imposed on the work), or restraining production (time consuming procedures turning down the demand and increasing staff shortage), or reducing resilience (possibly reducing additional opportunities to patients when all safety conditions are not realized). 1.

State of existing research

Human error in medicine is a recent field of research. It started to be taken into account in the early 90’s, with a few scientific papers and edited books [1-3], then took another halfdecade for this issue to be recognized as major by medical institutions [4]. Since the late 90s the number of special issues, journals, conferences, and reports dedicated to this topic has continuously grown, leading to a recent series of framework documents on effective interventions for improving patient safety [5-11]. This synthesis is necessarily schematic, emphasizing the innovations and the dynamic course of research in the past 5 years. The US, UK, Australia, and the Northern Countries were the pioneering nations. The other European countries are progressively recognizing the value of the concept, developing original home research, and joining large EC’s coordinated actions initiated either by DGSANCO or by the 7th framework program of DG research. The 06’s recommendations of the Council of Europe on patient safety show that the topic has become an EC political issue. Research in Adverse Event (AE) causation and prevention are conducted in 6 major directions, some situated at the micro clinical level, and others at the macro level. 1.1. Identifying the scale and the origins of AEs. The European and US tradition of experimental and ecological field studies, as well as chart reviews and national AE events studies have accumulated knowledge in the 90’s and the early 00’s on errors mechanisms in medicine (most in anaesthesiology). The knowledge

6

greatly contributed to the initiation of the patient safety movement. However these approaches were rapidly considered time consuming, only identifying specific problems, thus had limited applications in continuous medical risk monitoring and system changes. To bypass the constraint, the IHI suggested using simplified methods to proceed to chart review, using trigger tools [12]. Medical reporting systems (MRS) were finally logically preferred for a continuous identification and prevention of AEs although they were not exempt of problems. MRS range in three categories: (i) Cat1 relates to MRS relying on professionals’ reports, either voluntary or mandatory. These systems, by far the most widespread, are underperforming, and moreover are a formidable matter of misinterpretations [13-14]. They cannot serve as a tool for risk mapping. Nevertheless, they have a great value for the acquisition of a safety culture especially when they can be associated with RCA and M&Ms. (ii) Cat2 relates to MRS relying on patients’ reports and claims. Again the result is rather disappointing. Patients are usually acknowledged as good detectors of something going wrong, but the literature says that care givers are not paying enough attention to them [15]. (iii) Cat3 relates to systems catching information directly into patient medical and administrative charts and IT devices. The promises for assessing risk in a systematic manner are great [16-17], but the reality of the field remains limited due to the immaturity of the medical systems (electronic charts represent less than 20% of the reality in most Western countries). 1.2 Reducing obvious causes of errors, adopting ITs, improving ergonomics of drugs, materials, and work organizations. There is a series of areas where simple actions could rapidly reduce the rate of error and aid in the recovery. The introduction of Information Technology systems (ITs) in medicine goes much beyond electronic charts and has become a priority to reduce medication errors, whether they are due to improper staff’s hand-writings, or to routine errors (wrong drug, wrong dose, wrong patient). (See continuation in §2). Drug name confusion represents 15 percent of all reported errors. However, it is important to understand that people confuse drug products not just drug names. A focus on similarity in no-name attributes of drugs (e.g., strength, route, dosage form, and schedule) is required to improve medication safety. Studies and recommendations for better drug labelling and packaging have resulted in considerable improvements via the actions of several US and European focus groups (e.g., EC expert group on safe medication practices) [20-22]. Fatigue, stress, workload, and limitation of duty hours are a third area where potential simple changes might procure big wins. A great number of recent papers address the fatigue of residents, showing that eliminating interns’ extended work shift in an intensive care unit significantly increases sleep and decreases attentional failures during night work hours [2324] The maximum time on duty (especially for work shifts) became constrained either by general regulation in Europe (1993, European Working Time Directive), or by dedicated regulations in US (ACGME, 1987, 2002). Unfortunately, some recent papers show that despite regulatory improvements, the question of fatigue-induced AEs is not solved, more often because of staff shortage [25]. 1.3 Integrating Patients to detect and recover errors The role of patient as active persons detecting and recovering errors is becoming a strategic complement for reducing AEs [26, 9]. The problem for research is as well mobilizing patients’

7

organizations and designing procedures, as changing mentalities of doctors and nurses to increase their trust in these patients’ voices. This last prerequisite refers to a further point relative to the acquisition of safety culture. 1.4 Educating and adopting a safety culture The basic patient safety education of medical staff is clearly poor. The matter is just entering the curricula in medical schools. The US ARQH and the ACGME have pioneered this educational field. Obstacles are still numerous: missing adapted syllabus, lack of time, and lack of professors. Educating already licensed professionals is an even greater challenge. Only a few initiatives are made at national level with a standardized program (e.g., the program developed by the German agency for quality in medicine-AQuMEd/ÄZQ) Teaching basic knowledge in patient safety is not enough for transmitting an effective patient safety culture. All the literature points to the need for generalizing a collaborative and learning spirit. Some tools like CRM and measurement tools may help and have transitioned from aviation with a certain success [27]. RCA following an accident or incident report remains a preferred tool for setting a safety culture. Two slightly different contributions have enriched this field. First, the technique of RCA, exported from the chemical industry, uses the general concept of causal trees [28], which by means of successive WHY questions, identify successive causes until reaching the root cause(s). RCA may be cumulated for risk mapping, serving to identify sentinels events. Second, the ALARM technique [29-30] gives more freedom in the exploration of the complexity of contributing latent factors. The goal is not so much to identify a root cause, but to show the interaction between multiples factors, and understand how the latent conditions and the simultaneous cracking of all defences have provoked the AE. The method aims first at educating. In both cases, reporting of near misses offers numerous benefits over AEs: greater frequency, fewer barriers to data collection, limited liability. France has recently developed an original initiative with the new voluntary process for accrediting doctors (in risky professions) asking for doctors to individually and annually report and analyse a fix number of personnel near misses[31]. 1.5 Developing patient safety indicators. The disparity of initiatives proposing patient safety indicators [18-19, 6-8] first show the heterogeneity of approaches. A scientific debate continues on what might be the best priority level to address for risk monitoring. See continuation of this section in §2. 1.6 Adopting a system approach in patient safety interventions (See continuation of this section in §2) The problem of human reliability cannot be considered only at the individual level. Ergonomics has long shown that human are as well source of unreliability as well as of reliability, recovering errors and system failures, maintaining adaptation in adverse conditions generated by the conflicting demands of complex work situations. Macro organizational strategies in understanding and preventing AEs are becoming key factors for success, namely monitoring the whole dynamic system, establishing durable and viable compromises, improving system resilience, controlling the competitive medical market, and moving the legal system.

8

2.

Three examples of issues that deserve research attention:

2.1 Developing Information Technology systems (ITs) The introduction of Information Technology systems (ITs) is a priority to reduce a series of medical errors, whether these errors are due to incorrect physician order entry, poor handoffs (wrong drug, wrong dose, and wrong patient), or poor access to patient’s historical information (at admission, during hospitalization, and at reconciliation). Moreover, a common format of electronic charts will considerably ease a safer crossing of medical borders within European Countries. 2.1.1 Objectives Four generic types of systems are developed:Electronic patient records (EPR) are the building block medical information system that substitutes traditional paper medical records or “charts.” An EPR need not represent a single physical entity but can take a functional view assembled when needed from stored in various geographic locations, due to interoperability among EPR systems or use of intelligent agents to knit together data obtained from disparate sources into a single coherent record [17]. The term encompasses a spectrum of systems: imaged-based which paper is converted to electronic displays; that offer word-processing templates; point and-click that structure data capture. The examples of Kaiser Permanente and the Veterans Administration suggest that such systems may play a critical part in preventing medical errors [31]. The UK NHS and French Ministry of Healthcare, as well as northern countries have launched ambitious programs of EPR since 2004, but medical coverage is still limited and heterogeneous. Personal health record. A computer-based health record kept or controlled by a consumer (in contrast to a provider). Decision-support tools. If data is stored machine-intelligible form, it can be analyzed by clinicians and patients with alerts, reminders, other real-time decision aids. In the United States, 15 percent of hospitals report using computerized physician order entry (CPOE). Many studies in Europe and US test new systems of computerized physician order entry, computerized physician decision support (advertising on drug interactions, and even adverse reaction / allergies when coupled with patients’ data), and bar-coding. Electronic handoffs are also a priority to make safer the continuation of care. Improving and standardizing handoff is one of the National Patient Safety Goal of JCAHO, 2006, and an electronic version could clearly aids reaching this goal. The challenge in this area is to identify interventions which are most likely to result in the greatest benefits, and which are also feasible within these already protocol-saturated environments. 2.1.2 Remaining existing barriers and avenues for future research Technical barriers: The main difficulty of a full implementation of IT’s into hospitals remains the scalability. It is much easier to computerise small general practices than large complex hospitals, let alone provide integrated services across an organisation as large as the national authority. These other technical issues—which include patient record architecture, terminology, interoperability standards, security, and developments in computer technology—are in progress, although numerous problems will only become visible when the systems will be on use at a large scale. Usability and users’ trust: IT’s are clearly a priority for patient safety research, but will not solve all the problems (new wine means new errors), and moreover remain conditional on a substantial evolution of the culture of traceability and safety, including new protections for potential legal implications [32-33].

9

• •

The protection of medical confidentiality in electronic material (patient’s rights) is a potential obstacle for IT’s development in many European Countries, especially France, although medical staff’s protections are first front obstacle in US. Analyzing usability problems reveals clusters or categories of errors [34]. Some errors relate to coping strategies on the part of staff (i.e. attempts to work around poorly designed devices). There are special considerations when patients are the primary users of devices (e.g., batteries are a common problem). The configuration and programming (i.e., the basic set up of the device and its software) is a source of error because training in this area is inadequate. Some equipment collects essential data, but people are not aware of how the data are affected by routine changes. Patient monitoring systems, for example, might append data of a new patient in a particular bed to that of a former inhabitant if not reset properly. Other equipment changes require software to ensure compatibility, but not all staff understand this. Retraining is not always the solution to device problems, because basic software issues are common. One problem is multiple iterations of devices and software, so organizations have difficulty determining the compatibility of all the necessary components. Many errors result from communication breakdowns. Time for communication about how systems work should be built into workflows. Errors also result from uncertainty over new technology.

2.1.3 Approaches that could be used, and opportunities for collaboration (disciplines, countries) There is a serious lack of harmonization in the approach of Medical ITs in Europe. Programs of development are multiples at the nation or hospital levels, with a clear excess of emphasis on technical and business / commercial aspects, and not enough research on ergonomics and usability. Usability and Human factors studies should be conducted in parallel to the design of systems. Adopting a minimum technical platform for EPR at a European Level is a true challenge. The medical community must make a significant effort to design acceptable standards and not to continue designing ultra best techniques unaffordable by the majority of medical entities. There is now evidence in the literature that simple measures that are not so costly can show significant benefits. Low cost products should become a priority for research. This is an opportunity to involve new member countries while mobilising more advanced countries where up to date practices may be pursued at the expense of basic ones that have been shown to have a large impact on the safety of patients and that rely as heavily on a reasonable technology as on individual and group practices. This is a strong argument for the definition of specific safety priorities and for the organisation of external evaluation models around more targeted objectives. These specific efforts could take part in coordinated projects positioned under the authority of the European DG INFormation SOciety and Media (INFSO) and DG research. 2.1.4 Results of debates during the session • Great enthusiasm, great disappointment • A lot of money spent on projects, poor standardization of Hardware/software, only a few results (at present time) • Practitioners front line, hands on with immature systems, little education and little time to overcome problems, • Management enthusiastic for potential results, but often distant (with little vision of side effects associated with the introduction of non-optimal IT systems)

10

• • •

2.2

Issue of confidentiality and the use to which the information will be put forefront. This issue is especially strategic in Europe considering the importance of patients rights. Usability and Human factors studies should become first priority (not commercial issues) Harmonization required, adopting a minimum technical platform for EPR at a European Level is a true challenge and should be now considered as first priority..

Developing patient safety indicators

Mobility within borders and across EU is a benefit to citizens, able to obtain healthcare outside of their region / state, but represents at the same time a challenge in relation to the quality of the services provided. Unfortunately, there is still a lack of European consensus on the best way to identify and monitor most key patient safety issues. In addition, methodology and interventions for improving safety are diverse and partially not validated. There is a clear need for a concerted European approach. 2.2.1 Current efforts The measurement efforts of patient safety are difficult and slow to develop. Several proposals for Patient safety indicators (PSI) have been made in the last 5 years by research teams and expert committees without making the problem over. o The OECD Health Care Quality Indicator (HCQI) Project started in 2006. A proposal was issued in 2006, identifying and then rating 59 PSIs against three principal criteria: importance to patient safety, scientific soundness and potential feasibility. The project reviewed routine data collections as a starting point [35]. Twenty-one indicators were finally selected, ranging in 5 classes: Hospital acquired infections, Operative and post-operative complications, sentinel events, obstetrics, and other care-related adverse events. o Almost in the same time frame, the European DG Sanco SIMPATiE program (Safety IMprovement for PATients In Europe) [36] developed a common vocabulary on patient safety, then identified another set of PSIs. Eight experts from six nations evaluated the PSIs on a scale ranging from 1 to 9 for “Relevance”, “Validity and Reliability”, and “Feasibility”. Statistics for each dimension of the indicator formed the basis of recommendations in four categories from “recommended to be used throughout EU” to “not recommendable for implementation in EU”. Description of the PSIs can be found on www.simpatie.org. The PSIs are related to risk reduction and harm reduction and cover the dimensions; process and outcome. The PSIs are devided into subsets: “Institution Wide Measures”, “Specific Measures” and “Theme Related Measures” covering the themes: “infection control”, “surgical complications”, “medication errors”, “Obstetrics”, and “Fall”. In addition to the above mentioned efforts, most Western’ National and International Authorities (WHO) are proposing patient safety goals. In that case, specific PSIs are often associated with one or many of these goals on an ad hoc basis. Another type of clusters of safety indicators have also been successfully developed by the Institute for Healthcare Improvement in Boston (IHI) and may serve safety self assessment by means of triggers tools [12]. These triggers based on a risk-related aggregation of indicators (in most cases related to bundles of best practices) permit rapid identification of adverse events through simplified chart review. 2.2.2 Remaining barriers and avenues for future research

11

Heterogeneity: The multiplicity of proposals first show the heterogeneity of approaches. A scientific debate continues on what might be the best priority level to address for risk monitoring. Use of Process measure Vs outcomes measures Given the many problems surrounding the interpretation of routine data to assess PSIs, does this point towards the use of process measures or outcome measures? It is clear that concerns about PSIs apply to both. Beyond this, it is the interpretation of outcome measures which is most susceptible to the serious threats posed by issues of validity and reliability, the confounding effects of case mix and other factors, and the problem of chance variability. Furthermore, whereas assessments of ongoing processes can often be made from relatively small numbers, assessing such failures from outcomes alone requires much larger studies [33]. Standardization by use of IT-based PSI While some countries are developing multi-source monitoring systems these are not yet mature enough for international exchange. Use of micro PSIs Vs macro PSIs. Only indicators focusing on immediate deficiencies of patient care (bedside) are easy to compare, easy to measure, and non-problematic to standardise. However, an exclusive setting at that micro level might result in counterproductive actions, resulting in a litany of small, isolated, and equally mandatory standards [19]. Conversely, macro systemic indicators are uneasy to grasp and remain distant concepts for medical staff. There is nevertheless a growing tendency recommending a wholistic and systemic approach of all aspects of patient safety, with more system-oriented PSI. A first step is represented for example by the JCAHO’s Patient Tracer Methodology monitoring and detecting gaps in all aspects of patient trajectory and interactions with the medical system. A step further looks at the organisation and the culture of the medical staff and directors. The indicators focusing on safety culture are distinctly less consistent between countries, and over time. Another disadvantage is that they are much harder to collect objectively. Nevertheless a series of in-depth reviews of existing and validated methods has been recently published and can be repeated and possibly use as PSIs [37]. 2.2.3 Approaches that could be used and opportunities for collaboration (disciplines, countries) There is an urgent need for harmonization of PSIs. Rather continuing developing new indicators often with limited field testing, it should probably be preferable to encourage multiplying field testing for limited sets of PSIs and evaluating their potential to change safety figures. Attention should be paid to barriers and mistakes in implementation, data gathering and interpretation. The situation seems mature enough at the international level to conduct this field testing program in a consistent manner, allocating the testing of PSI to participative countries, with funding coming both from the two (three) sides of the Ocean. 2.2.4 Results of debates during the session • Don’t invent more PSIs, test them in the different countries • Need both process and outcome measures • Work on each PSI could occur at different levels of sophistication or advances in patient safety o One could imagine to select an agreed list of PSIs, with associated national’s cost of deployment and ambition for supervising patient safety, o Then let EU nations picking the relevant PSIs fitting their timely, local need, priorities from the list o Such a strategy should help consistent harmonization and efficacity, nevertheless respecting the tempo of national HC’s priorities and resources.

12

2.3 Developing a systemic approach in patient safety The system approach traces the causal factors back into the system as a whole. Remedial efforts are directed at situations, defenses, and organizations. [38]. One the most complex problem to solve for really gaining efficiency in patient safety at the whole level of the Healthcare system is to control and guide the multiplication of patient safety interventions made at micro and macro level in reaction to the flow of adverse events. Many of these interventions contradict one another, complicate the system, or are weakened by the demands of the field, thus are ineffective and often make the system more fragile and less safe. 2.3.1 Current efforts The system approach has long been only associated with the idea of latent factors. Two major sources of error-shaping factors are usually under focus: bad system design implying undue work constraints (general architecture, procurement, and personnel management), and bad organizational design (poor team work, governance, managerial policies, commitment, safety culture). There are three important avenues of efforts trying to go beyond these simple identifications of latent/systemic factors, modeling the reality gap, and pointing to concrete safety interventions at the system level. Developing safe organizations The first avenue is represented by High reliability organisations theory (HROs). HRO’s are those systems which have the potential for catastrophe to occur but which rarely experience large scale failures; HROs have the following properties: Firstly they are preoccupied with failure rather than success, encouraging staff to report actual and near misses and unexpected happenings. Secondly, HROs are reluctant to search for simple explanations. Thirdly, HRO’s are committed to resilience. These organizations develop capabilities to detect, contain, and bounce back from inevitable errors. Finally HROs show deference to expertise. Unfortunately, HRO research remains too vague, and has had little impact, both in terms of influencing the way safety performance is assessed by regulators, and in defining intervention strategies. [7]. Increasing the resilience of the system: The concept of organisational resilience has been recently introduced by the collective book of Hollnagel, Woods and Levison [41]. Resilience is the conjunction of three abilities: ƒ The ability to prevent something bad from happening ƒ The ability to prevent something bad from becoming worse ƒ The ability to recover something bad once it has happened Resilience could also be described as a system’s ability to resist a wide variety of demands from its whole domain of operation. The wider and better-controlled the open performance domain is, the higher the level of resilience is. This performance domain can open: (i) occasionally, in an exceptional situation; (ii) or more often as a result of a gradual opening. This last point brings us back to the pivotal question of trade-off between safety and performance. Often, the advantages of increasing production are immediately perceived and the domain opens out, while the associated risk-taking only implies drawbacks for safety at a later point in time. Deviations becomes the norm of resilient system, hence need to be address as such with dedicated intervention strategies [42-43]. Woods [44] has introduced the concept of “sacrificial decisions” to characterize this complex safety/performance conflict management. This trade-off is quite fundamental, and has always plagued discussions of safety: the safest aircraft never flies, the safest anesthesia is never given, etc. - so that all operations in risky domains must find and adjust their balance between acute FBC (FasterBetter-Cheaper) goals (or the tactics that will help to achieve these goals), and chronic goals

13

such as safety. As such, the resilience approach is a promising approach for evaluating the potential negative impact on medium term of prescriptive intervention strategies. Articulating the different approaches from micro to macro levels and from unsafe to ultra safe systems. A series of researches tend to fill the gap between micro and macro clinical approaches, and between unsafe and ultra safe systems. Researches try to capture the essence of contributing factors to safety at each level, trying to create consistency between the various interventions strategies [44,45]. 2.3.2 Remaining barriers and avenues for future researches Models are appealing but hard to measure and falsify. Systemic analysis is not familiar to the medical staff and medical research. Moreover, interventions strategies at this level are changing organizations, habits, and even profits, hence are generally much more demanding and disturbing for professionals than just changing a procedure. Individual responsibility may be hidden. It is recommended in the person approach to go beyond errors at the sharp end, and to consider systems as the potential cause of errors of frontline actors. At this point, some people jump forward to conclusions and say that if the system is the root-cause of accidents, the behavior of individual frontline operators is relatively unimportant: Each of them may then feel that what he or she does has little importance because he or she is controlled by the system. If something goes wrong, the blame is inevitably on the system or, possibly, on managers. Others take exactly the opposite view. Here comes the big word: responsibility. So who must be blamed in case of accidents? Those involved in the system, or the system which employs them? These two positions seem impossible to reconcile and may stay that way as long as the idea remains to find a culprit. These two apparently irreconcilable views are actually reconciled in the common term of professionalism. This term describes an entire set of qualities specific to an individual, including his or her determination, his or her ability to understand, his or her commitment to safety and, eventually, his or her ability to positively express his or her individual freedom. 2.3.3 Approaches that could be used and opportunities for collaboration (disciplines, countries) Acting on systemic factors is probably the (unique) possibility for real long term successes, but it needs deep and disturbing changes. It is definitely a long term effort asking for a global vision of the healthcare system. Research is required to continue developing ad-hoc interventions strategies rather than new models. 2.3.4 Results of debates during the session • •

•

Need for a radical change : get a big picture e.g. For engineers, a system approach means that the system may be broken down into workable pieces o …in engineering, many user-errors shows a product-error, but in healthcare, many user-errors means blaming the user. o Need a consistent and coordinated platform for engineering the global approach. Models are appealing but hard to measure and falsify. Systemic analysis is not familiar to the medical staff and medical research. o The value of the rule is seen when it relates to patient care, less when it relates to the system

14

Practitioners are not the best to look at system design, need to work in collaboration with social scientists. However, practitioners must be convinced of proposals. Joint research required. Should become first priority in patient safety agenda. o

•

3. Feasibility of doing this work across European Countries for potential mechanisms The DG Sanco and DG research are promoting networks to develop and share ideas, concepts, education, intervention strategies, and mapping exercises. o These Networks will produce European added value by mutual support, transfer of knowledge and good practice and exchange of ideas and materials for accelerating progress, based on the work already achieved by previous and ongoing projects (SIMPATIE, MARQUIS…). o

France has made such a proposal of European Network, so termed EUNetPaS (European Network for Patient Safety) at the 2007’ call for tender of DG Sanco. EUNEtPaS, if it is accepted, will connect all Member States and provide a platform to (i) Establish effective reporting and learning systems on adverse events in health care by clarifying the legal situation on health professionals’ liability issues and creating an environment where it is easy and safe to report and there is an opportunity to learn from mistakes without fear of censure, (ii) Educate and train health professionals and all staff working in health care settings, (iii) Develop organization models and patient involvement to ensure safer and cleaner health care systems, (iv) Develop a set of common core indicators at European level to measure patient safety performance and (v) Put emphasis on safe pharmacotherapy as an example to test Patient safety improvement policies as described below.

The main barriers to implementation and progresses are the economical and political differences of healthcare systems within Europe. These differences logically orient to develop at the bedside level the first step for harmonization of intervention strategies. This step is already en-route and significant results should reasonably be attainable within the next 5 years. It is clear that this step will not be enough for significant progresses in patient safety. Instead of delaying system approaches, it should be strategic to develop these holistic approaches in a coordinated way using common theoretical framework, but tuning results accordingly to national specificities. Harmonization will probably come later for this systemic level. All European countries might be associated in research and development whether they produce the concepts, or test the concepts. 4. Conclusion Human factors apply wherever humans work. Human factors acknowledge the universal nature of human fallibility. The traditional approach to human error might be called the “perfectibility” model that assumes that if workers care enough, work hard enough, and are sufficiently well trained, errors will be avoided. Our experience, and that of international experts, tells us that this attitude is counter-productive and does not work (The Australian Commission on Safety and Quality in Health Care - http://www.safetyandquality.gov.au/)

15

Standards and intervention strategies should take into account the HCO’s or the country’s economic and cultural work context which might encourage violations. This would result in a bureaucratic policy and a ‘virtual safety’ The last movement, the most recent and one which is still being constructed, concentrates on global management of safety, a systemic approach, and the culture of safety Global management of medical safety is currently being deployed in a number of countries, notably the United Kingdom, and is based on four areas of action : • Commitment by top management, in particular mentioning the importance of a senior member of staff responsible for patient safety at the highest level in the HCO • A proactive approach to risk, which does not confine itself to reacting to superficial causes identified in feedback as causing problems (the ‘react to what happened’ approach), but analyses and attacks the root causes of risks in the system • A sophisticated and multifaceted feedback system • An approach focusing on the responsibility and action of the actors, from top management to front-line staff, with particular emphasis on resilience strategies (robustness to destabilisation of the system, strategies of sacrifices between competing and contradictory goals, and crisis management).

16

REFERENCES 1. Bogner, M. (Ed.). (1994). Human error in medicine. Hillsdale, New Jersey: Lawrence Erlbaum Associates. 2. Brennan T, Leape, L., Laird, N., Localio, A., Lawthers, A., Newhouse, J., & al. Incidence of adverse events and negligence in hospitalized patients: results of the Harvard medical practice survey study I. New England Journal Med. 1991;324:370376. 3. Vincent, C. (Ed.). (2001, 2nd Ed). Clinical risk management. London: BMJ Publications, first Ed 1995. 4. Kohn, L., Corrigan, J., & Donaldson, M. (1999). To err is human - building a safer health system. committee on Quality in America. Washington DC: Institute of Medicine, National Academic Press. 5. Ovretveit J. Which interventions are effective for improving patient safety: a review of research evidence. Stockholm: Karolinska Insitute, Medical Managment centre, 2005. 6. Seven Steps to Patient Safety. National Patient Safety Agency, 2004, found at www.npsa.nhs.uk/sevensteps 7. Carthey J, Woodward S, Lewis R. Safety Management Systems, high reliability organisations and resilience engineering: Implications for strategies to improve patient safety (draft), NPSA, 2006 8. Standards setting and accreditation systems in health, Australian Council for Safety and Quality in Healthcare, 2003, www.aqc.org.au 9. World Alliance for Patient Safety, 2006, http://www.who.int/patientsafety/solutions/en/ 10. JCAHO, National patient safety goals, http://www.jointcommission.org/PatientSafety/NationalPatientSafetyGoals/ 11. CCHSAC, 2006, http://www.calgaryhealthregion.ca/qshi/accreditation/safety/PatientSafetyCCHSACom munique.pdf 12. Resar, R. Rosich, J., Classen, D., Methodology and rationale for the measurement of harm with trigger tools, Qual. Saf. Health Care, 2003, 12, 39-45 13. Cullen, D., Bates, D. Small,S. Cooper, J. Nemeskal, A Leape, L. The incident reporting system does not detect adverse drug events: a problem for quality improvement. J Qual Improv, 1995,21, 541-548. 14. Amalberti, R., Gremion, C., Auroy, Y., Michel P., Salmi R, Parneix P, Hoarau H., Pouchadon, M.L. Quenon, J.L., Hubert, B., Les systèmes de signalement médicaux, Etudes et résultats de la DRESSs, juillet, @-revue, 2007, http://www.sante.gouv.fr/drees/etude-resultat/ 15. Davies E. Cleary P. D. Hearing the Patient's Voice? Factors Affecting the Use of Patient Survey Data in Quality Improvement. Qual Saf Health Care 2005;14(6):428– 32 16. Murff, H., Patel,VL, Hripcsak, G., Bates, DW. Detecting adverse events for patient safety research: a review of current methodologies. J Biomed Inform., 2003, 36, 131143.

17

17. Goldschmidt, P., HIT and MIS implications of health information technology and medical information systems, Communications of the ACM, October 2005, 48 (10) 69-74 18. OECD Quality Indicator Project: The Patient Safety Expert Panel, 2006, http: //www. oecd. org/dataoecd 19. Shaw, C. Which way to organizational excellence?notthis way; ask a professional, JRSM, 2007,100, editorial 20. NPSA-HHRC, Information design for patient safety, a guide to the graphic design of medication packaging, 2006, NPSA-HHRC report 21. Pennanen, P. National handbook for pharmacotherapy in social and healthcare, Ministry of social affairs and health, Finland 22. Landrigan, C., Rotshchild, J., Chronin, J., Kaushal R., Burdick E., Katz J.et al. (2004). Effect of reducing interns’ work hours on serious medical errors in intensive care units. New England Journal of Medicine, 351, 1838–1848. 23. Lambert, Lin, J. Tan, H. Design of Safe Drug Names Drug Saf. 2005;28(6):495-512 24. Sim, D. Wrigley, S., Harris, S., Effects of the European Working time directive on anaesthetic training in the United Kingdom, Anaesthesia, 2004, 59 :781-84 25. Shetty, K., Bhattacharya, J., Changes in hospital mortality associated with residency work-hour regulations, Annals Int. Med, 2007, 147, 73-80 26. Weingart, S., Pagovitch, O., Sands, D., Li, J., Aronson, M., Davis, R., Bates, D., Phillips, R., What Can Hospitalized Patients Tell Us About Adverse Events? Learning from Patient-Reported Incidents, J Gen Int Med, 2005, 20 : 830 27. Grogan, E., Stiles, R., France, D., et al, The impact of Aviation-based teamwork training on the attitudes of Healthcare professionals, J.Am.Coll.Sur., 2004, 199,:84348 28. Battles, J., Shea, C. (2001). A system of analysing medical errors to improve GME curricula and programs. Academic Medicine, 76(2), 124-133 29. Vincent, C., Adams, S., & Stanhope, N. (1998). A framework for the analysis of risk and safety in medicine. BMJ 316, 1154-1157 30. Vincent C. A. Analysis of clinical incidents : a window on the system, not a search for root causes. Qual Saf Health Care 2004; 13:242-3. 31. Feachem RGA, Sekhri NK, White KL. Getting more for their dollar: acomparison of the NHS with California's Kaiser Permanente. BMJ 2002;324:135-43. 32. Benson, T. why General practitioners use computers and hospital doctors do not : Part1: incentives, BMJ 2002, 325: 1086-89 33. Powell, A. Davies, H., Thomson, R., Using routine comparative data to assess the quality of healthcare: understanding and avoiding common pitfalls, Qual. Saf. Health Care, 2003, 12, 122-28 34. Johnson C. Why did that happen? A brief explanation for the proliferation of barely usable software in healthcare systems. 2nd US/UK patient safety research methodology workshop. AHRQ, Rockville, MD, September 23-24, 2003. 35. McLoughlin, V. Millar, M. Mattke, S. Franca, M. Pia Maria Jonsson, David Somekh, and Bates, D. Selecting indicators for patient safety at the health system level in OECD countriesInternational Journal for Quality in Health Care 2006 18(Supplement 1):14-20 / OECD Quality Indicator Project: The Patient Safety Expert Panel, 2006, http: //www. oecd. org/dataoecd

18

36. European SIMPATIE project [Safety IMprovement for PATients In Europe]., 2006, WP4 37. Pronovost, P., Sexton, B., Assessing safety culture: guidelines and recommendations, Quality and Safety in Healthcare, 2005, 14:231-233. 38. Carthey, J de Leval M R and Reason, J T Institutional resilience in healthcare systems Qual Saf Health Care 2001;10: 29-32 39. Amalberti, R., Hourlier, S. Human error reduction strategies in health care. In P. Carayon (Ed.), Handbook of Human Factors and Ergonomics in Healthcare and Patient Safety. HillsdaleNew Jersey: LEA., 2006, Chapter 32: 562-77 40. Hollnagel, E., Woods, D.D., & Leveson, N. (Eds.). (2006). Resilience Engineering: concepts and precepts. Aldershot, UK: Ashgate. 41. Rasmussen J. Risk management in a dynamic society. Safety Science 1997;27:183214 42. Amalberti, R., Vincent, C., Auroy, Y., de Saint Maurice, G., (2006) Framework models of migrations and violations: a consumer guide, Quality and Safety in Healthcare, 2006;15(suppl_1):i66-i71 43. Woods, D.D. (2006a). Essential Characteristics of resilience. In E. Hollnagel, D.D. Woods & N. Leveson (Eds.), Resilience Engineering: Concepts and Precepts (pp. 6976). Aldershot, UK: Ashgate. 44. Amalberti, R. Auroy, Y. Berwick, D., Barach, P. Five System Barriers To Achieving Ultrasafe Health Care, Ann Intern Med. 2005;142, 9: 756-764. 45. Mohr JJ, Barach P, Cravero JP, Blike GT, Godfrey MM, Batalden PB, Nelson EC. Microsystems in health care: Designing patient safety into the microsystem. Jt Comm J Qual Saf. 2003 Aug;29(8):401-8.

19

Workshop discussion paper: Can indicators of hospital clinical outcomes demonstrate safety? Research Agenda Track: Parallel Workshops Session 1 Tuesday 25th Sept, 2007.

20

Patient Safety Research shaping the European agenda 24-26th September 2007: Porto, Portugal

WORKSHOP DISCUSSION PAPER

Can Indicators of Hospital Clinical Outcomes Demonstrate Patient Safety? Paul D. Bartels

Paul D. Bartels, MD Solvejg Kristensen, MHS Jan Mainz, Prof, MD, PH.D. Unit for Clinical Quality and Patient Safety Olof Palmes Allé 15, DK-8200 Aarhus N

21

Introduction and summary Qualitative analysis of adverse events and organisational practice in patient safety has proved to be a rich source of detailed information, which has increased knowledge of causation, preventability and safe practices. However, at the five years anniversary after the launching of the patient safety movement it was concluded (1;2), that there was an unmet need of quantitative assessment of patient safety – in terms of outcomes, in order to ascertain the global progress of patient safety. This demand for quantitative robust reproducible measurement of relevant outcomes as a condition for patient safety improvement has been extended to the clinical organisation (3);(4). Concurrently sets of indicators aiming at a broad assessment of patient safety at the clinical (5-7) or healthcare system (8) level are presently introduced in different healthcare systems. This creates a need for research-based validation (9). The more than 10 years of widespread experience with clinical indicators and epidemiological methods in classical quality improvement has created an extensive knowledge of the necessary scientific and technical principles for definition, design, implementation and interpretation of indicators (10-12)(13;14). The present paper aims at extending these principles to the domain of patient safety. At the same time specific aspects of methodology in relation to indicators in patient safety will be identified and discussed in terms of special needs for developmental work. The paper will give a general overview of the scientific foundations of clinical indicators, a summary of the existing work on patient safety indicators (PSI) and a discussion of four fields for possible development of indicators specifically aimed at patient safety: • global safety outcome measures, problems of construction and interpretation • development of surrogate measures of patient harm • modification and reinterpretation of “quality” datasets to cover the safety aspects of care • Use of safety culture staff surveys as data for new patient safety indicators. Clinical indicators, general principles of development and applications Clinical indicators related to patient safety have to comply with a number of general principles. Definitions Clinical indicators are defined in different ways (11): • As measures assessing a particular healthcare process, structure or outcome • As measuring tools, screens or flags used as guides to monitor, evaluate, and improve the quality of care, clinical support services and organisational functions affecting patient outcomes. Purposes of clinical indicators The purposes of implementing clinical indicators are to (11): • Document and improve the quality of care • Make comparisons and benchmarking over time between places (e.g. units, hospitals) • Support patients’ choice of providers.

22

The use of indicators enables professionals and organisations to monitor and evaluate what happens to patients as a consequence of how well organisational systems function. Indicators are based on standards of care. These can be evidence-based and derive from academic literature. When scientific evidence is lacking, indicators can also be determined by an expert panel of health professionals in a consensus process based on their experience. Thus, indicators and standards must be described according to the strengths of scientific evidence of their ability to predict outcomes (10;12). Key characteristics of the ideal indicator The ideal clinical indicator has the following key characteristics (11): • Is based on agreed specified definitions and is described exhaustively and exclusively • Is valid and reliable • Is highly or optimally specific and sensitive, i.e. detecting few false positives and false negatives • Relates to clearly identifiable events for the user (e.g. if meant for clinical providers it is relevant to clinical practice) • Is evidence-based • Is feasible in terms of data collection. Each indicator must be defined in detail with explicit data specifications in order to be specific and sensitive. Indicators may vary in their validity and reliability. Validity is the degree to which the indicator measures what it is intended to measure, i.e. the result of a measurement corresponds to the true state of the phenomenon being measured. A valid indicator discriminates between care otherwise known to be of good or bad quality and concurs with other measures intended to measure the same dimension of quality. Reliability is the extent to which repeated measurements of a stable phenomenon by different data collectors, judges or instruments at different times and places get similar results. Reliability is important when using an indicator to make comparisons among or within groups over time. A valid indicator must be reproducible and consistent. Reliability is highly dependent of the size of the population being characterized by the indicator and of the incidence of the disease/event. Indicators should be based on the best available evidence which can be described as the integration of the best research evidence with clinical expertise and patient values. The strengths of evidence of an indicator will determine its scientific soundness or the likelihood that improvement in the indicator will produce consistent and credible improvements in the quality of care. Indicators related to structure, process and outcome Indicator can be related to structure, process or outcome of healthcare. Structure denotes the attributes of the settings in which care occurs. This includes the attributes of material resources such as facilities, equipment and financing, of human resources such as the number and qualifications of staff, and of organisational structure such as medical staff, organisation, methods of pure review, and methods of reimbursement. Process denotes what is actually done in giving and receiving care, i.e. the practitioners’ activities in making a diagnosis, recommending or implementing treatment or other interaction with the patient.

23

Outcome measures attempt to describe the effects of care on the health status of patients and populations. Improvements in the patient’s knowledge and salutary changes in the patient’s behaviour may be included under a broad definition of outcome and some may represent the degree of the patient’s satisfaction with care. For a process indicator to be valid its use must previously have been demonstrated to produce a better outcome. Similarly, using structural indicators for quality assessment is only possible if structural components have been shown to increase the likelihood of either a good outcome or a process that has previously been shown to yield better outcomes. Therefore it is necessary to establish such relationships between any particular component of structure or process that is used to assess quality/safety. These linkages may be based on scientific literature. If little evidence exists professional experience concerning these linkages can be distilled using consensus message. Only clinical indicators which are evidence-based have had the linkage between structure or process and patient health outcomes confirmed. The ability to assess the quality of medical, technical care is bound to the strengths and weaknesses of clinical science. Generic and disease-specific indicators Disease-specific indicators measure particular aspects of care related to specific patient populations characterized by diagnostic criteria or specific interventions while generic indicators measure aspects of care relevant to most patients. The main body of experience – and success in terms of proven effects on outcome – lies with disease-specific clinical indicators. These indicators are closely linked to development of evidence-based clinical guidelines and are usually interpreted within the framework of evidence-based medicine. Generic indicators are more difficult to interpret – especially when making comparisons among hospitals or providers – since the evidence base tend to be weaker and as there may be profound differences in patient mix. Risk adjustment Usually multiple factors contribute to a patient’s survival and health outcome. Therefore, outcome measures must be adjusted for factors outside the health system influence if fair comparisons are to be made. In quality/safety assessment, components relating to the medical care system should be isolated. This is accomplished by controlling for significant confounding factors that contribute to the outcome. Factors that are frequently included in risk adjustment models include patient demographic, psycho-social characteristics (such as age, sex and functional status), lifestyle factors (smoking and alcohol consumption), and severity of the illness that is the focus of measurement, health status and co-morbid conditions. Risk adjustment is essential prior to comparing patient outcomes across hospitals or providers. Risk adjustments are most important for the interpretation of outcome indicators. There are alternative methods to ensure that other differences among patient groups are not influencing comparisons of process or outcome indicators – e.g. the population of patients for whom the indicator is measured can be carefully restricted. Alternatively, stratified analyses can be performed to examine specific types of patients within a small and overall sample. Clinical indicators applied within patient safety overview and methodological issues This special type of clinical indicators are recognized by their purposes, which is monitoring preventable adverse events – directly or indirectly (5;15). Patient safety indicators are thus indicators (16), which have to be valid within a specific framework of interpretation referring

24

to preventable events or medical errors within a given clinical setting (organisation, treatment setting or patient population): •

In terms of structure and process indicators: Healthcare organisational features, practices or interventions, with a supposed positive or negative effect on injuries caused by care (e.g. safety culture, preoperative antibiotics, washing hands practices, screening of schizophrenic patients for suicidal risk). It is important to note that the present evidence base for many patient safety interventions is comparatively weak (17;18)

•

In terms of outcome: harm/injuries which is or may be conceived to be caused by preventable events in healthcare: e.g. death, temporary or permanent disability, infections and falls (5;19).

The development and interpretation of indicators as true measures of patient safety is been demanding: Firstly because the concept of preventable harm implies a process of judgement rather than objective criteria. Secondly because of the probabilistic nature of the relationship between exposure (structure-process) and harm (outcome) which creates special problems in defining the denominator of rate-based indicators (20). Thirdly because of the complex interactions between system and patient factors which influences outcomes such as mortality rates. Moreover the tendency to poor and variable clinical documentation of treatment complications requires special attention to data quality problems. The few categories of directly measurable preventable adverse events (wrong sided surgery, wrong blood type, in-hospital suicide) occurs so infrequently, that corresponding indicators has the characteristics of sentinel indicators (21) with very limited value in monitoring changes over time or for benchmarking purposes at the clinical/hospital level. However, these indicators have been found to be useful at the system (national/regional) level e.g. as part of the recommended patient safety indicators from OECD (8). In practice this means that evaluation of patient safety status by indicators requires access to multiple sources and kinds of data and has the characteristics of screening-tools for safety problems, rather than making a definitive diagnosis. Present state of the art Presently practical assessment of patient safety occurs in four settings: 1. Generic organisation oriented indicators as measurements of global outcomes like risk-adjusted/standardized mortality rates (22-24). Measurement of a broad spectrum of preventable treatment complications (AHRQ patient safety indicators), rates of adverse events experienced by patients (Danish National Patient Satisfaction Study) (25). 2. Clinical indicators used in generic clinical conditions concerning complications in surgery, control of hospital infections (26-29). The experience with these clinical indicators is of long duration. The original framework of reference is evidence based quality improvement but increasingly these data are used in the context of patient safety.

25

3. Single indicators specifically related to patient safety problems which forms part of diagnose-specific clinical indicator sets (e.g. rate and timing of preoperative and antibiotics in specified surgical conditions, assessment of suicidal risk in schizophrenic patients) (6). 4. Structure/context indicators considered as specific to patient safety: Culture/climate investigations. It is characteristic for these initiatives that there is a preponderance of outcome indicators reflecting the sporadic evidence base for patient safety interventions. Clinical indicators patient safety The patient safety movement has been initiated and perpetuated by descriptive epidemiological studies which consistently has showed considerable mortality associated with preventable adverse events. The methodology which essentially consists in chart review by structured implicit audit. It is therefore natural that overall mortality in hospitals or departments could be used as the universal outcome indicator for patient safety (30;31). However the conclusions of these studies have been challenged on the basis of possibility of selection bias and lack of appropriate correction for confounding factors (32;33). This may reflect a conflict between the patient safety approach to harmful events and the clinical epidemiological approach usually used in construction and interpreting clinical indicators. The discrepancy is illustrated on case level by figure 1 and figure 2. Figure 1: Overview of core patient safety terms and their relations

26

Figure 2 Overview of relations between process and outcome RELATIONS BETWEEN PROCESS OR OUTCOME + PATIENT Age, sex, lifestyle Psychosocial factors Co-morbidity + DISEASE Stage/seriousness Complications + TREATMENT Evidence-based pratice Competence of practitioners Technical equipment Safeguards + ORGANIZATION/LEADERSHIP Culture Leadership Communication Resource allocation Teamwork

OUTCOME Despite good empirical evidence that interventions or organisation changes which are believed to influence patient safety, we are still lacking studies which weigh those factors against other determinants of mortality. The required research effort will be demanding in terms of scale methodology development and intensity. A model for such a project may be found in the recent published results of a U.S. patient safety surgery study (27;34-36). This study has addressed the data standardization problem, the problems of risk adjustments/standardization of mortality rates in surgery and seems to be near to identifying safety promoting factors in organisations (parallel with specifying patient characteristics of vulnerability to adverse events). A possibility of similar research concerning the development of organisation based mortality rates as broad measures of safety might be emergent to the recent efforts of several European nations; UK, Denmark, Netherlands, and Sweden in applying hospital standardized mortality ratios (HSMR) to quality and safety assessment of hospitals (37-39). An alternative and controversial approach to the general quantitatively based evaluation and monitoring global patient safety organisations might be based on patient surveys. The fraction of patient experiencing errors has been part of The Danish National Patient Satisfaction Survey in hospitals since 2004. This indicator proves to vary significantly between hospitals (range 15-20 %) but the content validity is ambiguous. There are only few studies (40-42) concerned in in-depth analysis of patients experiences of adverse events. Except from this there is considerable discrepancy between “objective assessment of adverse events done by retrospective auditing of medical nursing records and the recorded – this can partly be explained by differences in professional and patient perceptions of errors, but may also be due to incomplete documentation in medical records. In light of the rising importance of patient involvement in improvement of patient safety, especially in the form of reporting adverse events this type of indicator might represent a valuable complementary addition to the global outcome measures.

27

However, interpretation will only be meaningful in the light of further research concerned with association between patient perception of safe care, medical injury service quality coordination and communication. Development of surrogate outcome measures of safety The group of clinical indicators which assess rate of harmful outcomes such as the AHRQPSI set, rates of hospital infections or surgical complications – apart from this has a number of inherent technical problems which have proved difficult or resourced demanding to solve. The clinical documentation and coding or several of these indicators are of varying completeness and accuracy. It has moreover been repeatedly demonstrated that change in management or clinical attention to surveillance of adverse events has strong effect on observed and coded rates. The inaccuracy is partly accounted for by the very variable manifestations of harmful outcomes (e.g. hospital/postsurgical infections, decubitus, ulcers, adverse medication events). The consequence is that if the indicator has to be exhaustive and inclusive the definition will automatically be broad, leaving room for variation of interpretation. At the same time several of the outcome indicators, in for example the AHRQ-set, implies a professional judgement of causative factors (e.g. failure to rescue complications of anaesthesia) which again leaves room for local and subjective interpretation of the definition. These issues are of special importance, if these types of clinical indicators are used for diagnostic purposes to assess safety towards an external standard or in relation to benchmarking between organisations or countries and in external accountability in the form of publication or pay for performance. The problem seems to be especially pronounced with data derived from administrative databases (43), but also data from direct medical abstraction or clinical registrations have to be treated cautiously (44). Long experience with documentation and registration of indicators concerning hospital infection rates or surgical complication rates (45) has shown that the problem can be alleviated by employing observation and data registration by specially trained staff. This is however a very resource demanding strategy which rarely allows comprehensive on-line documentation of safety/quality with these indicators. It must be concluded that the construction of valid reliable and visible clinical indicators of harmful outcomes requires development of methodology in the form of “surrogate measures” (45;46) (the idea of a surrogate measure is analogous to the intermediate outcomes employed in quality monitoring of diabetes and hypertension where patterns of objective data HbA1c blood pressure predict the risk of unfavourable outcomes for these groups of patients). There are few recent examples of the approach in patient safety related indicators, firstly by prudent selection of index population as marker for more general patient safety problem, the obvious example being the rate of hip fractures caused by in-hospital falls or postoperative infections in tracer conditions as hip-replacement surgery. This approach requires a relatively high rate of harmful tracer outcomes and is therefore limited to a few areas of patient safety monitoring. A more promising approach has recently been described for surveillance of hospital- acquired infections based on electronic hospital registries (47). On the basis of recognition of patterns of data which represents antibiotic treatments, microbiology findings, clinical

28

biochemistry findings, radiological findings and objective patient administrative data (length of stay) it is possible recognize cases of various types of hospital acquired infection with a high probability. The rate derived from these cases represents an on-biased probability based estimate of the true infection rates. It seems obvious that similar constructs can be made in the field of medication safety (electronic trigger tools) (48;49) and probably that the approach can be extended to other generic fields of safety. The increasing spread of electronic health records it seems to be an extremely important field for research and development of probability based clinical indicators of harmful outcomes. Can culture surveys form the basis of clinical indicators for patient safety? “In short the importance of culture is overwhelmingly supported by the evidence. If the harmful aspects of the culture of care providers are not addressed most problems will remain regardless of the extent to which we complete research, improve the accuracy of measurement of errors, increase health care resources and improve the knowledge and skills of clinical staff” (50). This authoritative statement it is not surprising that surveys which aim at giving a quantitative assessment of staff safety culture/organisational safety climate have become increasingly popular during the last five years. Recent reviews demonstrate that there now exist an abundance of instruments for this type or surveys with different focus an aspect of organisational culture (teamwork, open communication about errors, perceived attitudes of top-management towards safety) (51;52). There also appears to be ongoing active research activities on the validation of these questionnaires in terms of validity and reliability (53). Since this type of assessments accurately used as practical tools for quality and safety managing is an obvious undeveloped research issue concerning appropriateness of application of the various different tools in various health care settings. It is however more important that the evidence for causal relationships between the results of quantitatively survey of staff patient safety culture and patient safety outcomes (relevant change in care processes or incidence of adverse event/values of indicator based assessment of harmful outcomes is minimal. One study demonstrate that top-management interventions to improve patient safety culture result in a moderate change in plausible direction of survey values from the staff group exposed to the managerial intervention. A second study shows that unit safety climate measured by staff survey correlates with the incidence of nosocomial infections and medication errors (54;55). If patient safety culture measurements have to be justified as a tool for monitoring rather than a way to creation of attention to patient safety matters in organisations much more rigorous research is needed to create the evidence for a cause-effect relationship between the results of culture surveys and actual rates of harmful events (56). In view of the recent rise in interest of culture surveys in European settings (57) this research should be highly prioritized. Conclusions • There is a general need for development of quantitative estimates of outcomes in the form of harm and processes in the form of safe practices for sustaining patient safety improvement, evaluating effectiveness of interventions, priority setting and accountability.

29

• •

• • •

Patient safety indicators are to be considered as quality indicators which can be interpreted within a framework of patient safety (adverse events, preventable/nonpreventable). Patient safety indicators presently described are mainly outcome measures – derived from existing administrative datasets. There is a need for establishing relationships with known effective interventions and these outcomes. Secondly there is a need for calibration by sophisticated risk adjustment in order to be able to use these indicators for benchmarking. In terms of evaluation of cross border differences in patient safety in Europe two outcome measures seem especially interesting for further development and validation: Hospital standardized mortality rates (HSMR) and the AHRQ patient safety indicators. There is a need for development of valid and feasible structure and process indicators for patient safety. In this context safety culture surveys should be validated against outcomes measures Patient safety indicators are still to be considered as preliminary in need of validation. They are therefore still problematical in terms of use for accountability in the present stage.

30

Reference List (1) Brennan TA, Gawande A, Thomas E, Studdert D. Accidental deaths, saved lives, and improved quality. N Engl J Med 2005 Sep 29;353(13):1405-9. (2) Leape LL, Berwick DM. Five years after To Err Is Human: what have we learned? JAMA 2005 May 18;293(19):2384-90. (3) Pronovost PJ, Miller MR, Wachter RM. Tracking progress in patient safety: an elusive target 2. JAMA 2006 Aug 9;296(6):696-9. (4) Tomson CR, Berwick DM. What can the UK learn from the USA about improving the quality and safety of healthcare? 1. Clin Med 2006 Nov;6(6):551-8. (5) McDonald KM, Romano PS, Geppert JJ. Measures of Patient Safety Based on Hospital Administrative Data - The Patient Safety Indicators. University of California San Francisco-Stanford Evidence-Based Practice Center under contract no. 290-970013. Agency for Healthcare Research and Quality, Rockville, Maryland. Publication Number 02-0038; Technical Review no. 5; 2002 Aug. (6) Kristensen S, Mainz J, Bartels P. Establishing a set of Patient Safety Indicators. Aarhus; 2007. (7) Agency for Healthcare Research and Quality. Guide to Patient Safety Indicators. http://www.qualityindicators.ahrq.gov/downloads/psi/psi_guide_v31.pdf. 2006. (8) Armesto SG, Kelley E, Wei L. Health Care Quality Indicators Project. Patient Safety Data Systems in The OECD: A Report of a Joint Irish Department of Health - OECD Conference. http://ec.europa.eu/health/ph_information/indicators/docs/HCQI_systems_report_en.p df. Organisation for Economic Co-operation and Development; 2007 Mar 22. Report No.: JT03224252. (9) Perneger TV. A research agenda for patient safety 19. Int J Qual Health Care 2006 Feb;18(1):1-3. (10) Mainz J. Defining and classifying clinical indicators for quality improvement. Int J Qual Health Care 2003 Dec;15(6):523-30. (11) Mainz J. Developing evidence-based clinical indicators: a state of the art methods primer. Int J Qual Health Care 2003 Dec;15 Suppl 1:i5-11. (12) Mainz J. Quality Indicators: Essential for Quality Improvement. Int J Qual Health Care 2004;16:1-2. (13) Willis CD, Evans SM, Stoelwinder JU, Cameron PA. Measuring quality. Aust Health Rev 2007 May;31(2):276-81. (14) Wollersheim H, Hermens R, Hulscher M, Braspenning J, Ouwens M, Schouten J, et al. Clinical indicators: development and applications 1. Neth J Med 2007 Jan;65(1):15-22. (15) Pronovost PJ, Berenholtz SM, Goeschel CA, Needham DM, Sexton JB, Thompson DA, et al. Creating high reliability in health care organizations 6. Health Serv Res 2006 Aug;41(4 Pt 2):1599-617.

31

(16) Kazandjian VA, Wicker K, Ogunbo S, Silverman N. Understanding safer practices in health care: a prologue for the role of indicators. J Eval Clin Pract 2005 Apr;11(2):161-70. (17) Pizzi LT, Goldfarb NI, Nash DB. Making Health Care Safer: A Critical Analysis of Patient Safety Practices. Chapter 40. Promoting a Culture of Safety. 20-7-2001. IHI. Ref Type: Generic (18) Ovretveit J. Which interventions are effective for improving patient safety? - A review of research evidence. 1-8-2005. Karolinska Instituttet - Medical Management Centre. Ref Type: Generic (19) Kazandjian VA, Matthes N, Thomas T. Errors: can indicators measure the magnitude? J Eval Clin Pract 2001 May;7(2):253-60. (20) Pronovost P, Holzmueller CG, Needham DM, Sexton JB, Miller M, Berenholtz S, et al. How will we know patients are safer? An organization-wide approach to measuring and improving safety. Crit Care Med 2006 May 17. (21) Rivard PE, Rosen AK, Carroll JS. Enhancing Patient Safety through Organizational Learning: Are Patient Safety Indicators a Step in the Right Direction? Health Serv Res 2006 Aug;41(4 Pt 2):1633-53. (22) Jarman B, Gault S, Alves B. Explaining differences in English hospital death rates using routinely collected data. British Medical Journal 1999 May 6;318:1515-20. (23) Aylin P, Bottle A, Jarman B, Elliott P. Paediatric cardiac surgical mortality in England after Bristol: descriptive analysis of hospital episode statistics 1991-2002 19. BMJ 2004 Oct 9;329(7470):825. (24) Wachter RM, Pronovost PJ. The 100,000 Lives Campaign: A scientific and policy review 1. Jt Comm J Qual Patient Saf 2006 Nov;32(11):621-7. (25) Patients' experiences in Danish hospitals, http://www.patientoplevelser.dk/log/medie/Rapporter/Survey_2006_english.pdf. Copenhagen: Unit of Patient Evaluation; 2007. Report No.: ISBN: 978-87-91520-174. (26) Kelly JV, Hellinger FJ. Physician and hospital factors associated with mortality of surgical patients 2. Med Care 1986 Sep;24(9):785-800. (27) Khuri SF. Safety, quality, and the National Surgical Quality Improvement Program 13. Am Surg 2006 Nov;72(11):994-8. (28) Peterson AM, Walker PH. Hospital-acquired infections as patient safety indicators. Annu Rev Nurs Res 2006;24:75-99. (29) Russell EM, Bruce J, Krukowski ZH. Systematic review of the quality of surgical mortality monitoring. Br J Surg 2003 May;90(5):527-32. (30) Thomas EJ, Studdert DM, Burstin HR, Orav EJ, Zeena T, Williams EJ, et al. Incidence and types of adverse events and negligent care in Utah and Colorado 8. Med Care 2000 Mar;38(3):261-71. (31) Brennan TA, Leape LL, Laird NM, et al. Incidence of adverse events and negligence in hospitalized patients: Results of the Harvard Medical Practice Study I.

32

N.Engl.J.Med. 324. 7-2-1991. Ref Type: Generic (32) McDonald CJ, Weiner M, Hui SL. Deaths due to medical errors are exaggerated in Institute of Medicine report. JAMA 2000 Jul 5;284(1):93-5. (33) Meurer LN, Yang H, Guse CE, Russo C, Brasel KJ, Layde PM. Excess mortality caused by medical injury. Ann Fam Med 2006 Sep;4(5):410-6. (34) Main DS, Henderson WG, Pratte K, Cavender TA, Schifftner TL, Kinney A, et al. Relationship of processes and structures of care in general surgery to postoperative outcomes: a descriptive analysis 2. J Am Coll Surg 2007 Jun;204(6):1157-65. (35) Main DS, Cavender TA, Nowels CT, Henderson WG, Fink AS, Khuri SF. Relationship of processes and structures of care in general surgery to postoperative outcomes: a qualitative analysis 1. J Am Coll Surg 2007 Jun;204(6):1147-56. (36) Schifftner TL, Grunwald GK, Henderson WG, Main D, Khuri SF. Relationship of processes and structures of care in general surgery to postoperative outcomes: a hierarchical analysis 1. J Am Coll Surg 2007 Jun;204(6):1166-77. (37) Jarman B, Bottle A, Aylin P, Browne M. Monitoring changes in hospital standardised mortality ratios. BMJ 2005 Feb 12;330(7487):329. (38) Wright J, Dugdale B, Hammond I, Jarman B, Neary M, Newton D, et al. Learning from death: a hospital mortality reduction programme. J R Soc Med 2006 Jun;99(6):303-8. (39) Kristensen S. Hospital Standardized Mortality Rates (work accepted for publication). Ugeskr Laeger 2007. (40) Gut R, Osterbye T, Freil M, Schulze SB. To sider - samme sag Patienters oplevelser af fejl under indlæggelse og patientjournalers oplysninger om utilsigtede hændelser. Unit of Patient Evaluation, Copenhagen County. http://www.patientoplevelser.dk/log/medie/Rapporter/poster10.pdf. 2006. (41) Weingart SN, Pagovich O, Sands DZ, Li JM, Aronson MD, Davis RB, et al. What can hospitalized patients tell us about adverse events? Learning from patient-reported incidents. J Gen Intern Med 2005 Sep;20(9):830-6. (42) Weingart SN, Price J, Duncombe D, Connor M, Sommer K, Conley KA, et al. Patientreported safety and quality of care in outpatient oncology 1. Jt Comm J Qual Patient Saf 2007 Feb;33(2):83-94. (43) Zhan C, Miller MR. Administrative data based patient safety research: a critical review. Qual Saf Health Care 2003 Dec;12 Suppl 2:ii58-ii63. (44) Wilson AP, Gibbons C, Reeves BC, Hodgson B, Liu M, Plummer D, et al. Surgical wound infection as a performance indicator: agreement of common definitions of wound infection in 4773 patients. BMJ 2004 Sep 25;329(7468):720. (45) Nguyen GT, Proctor SE, Sinkowitz-Cochran RL, Garrett DO, Jarvis WR. Status of infection surveillance and control programs in the United States, 1992-1996. Association for Professionals in Infection Control and Epidemiology, Inc 89. Am J Infect Control 2000 Dec;28(6):392-400. (46) Gaynes RP, Platt R. Monitoring patient safety in health care: building the case for surrogate measures. Jt Comm J Qual Patient Saf 2006 Feb;32(2):95-101.

33

(47) Leth RA, Moller JK. Surveillance of hospital-acquired infections based on electronic hospital registries. J Hosp Infect 2006 Jan;62(1):71-9. (48) Szekendi MK, Sullivan C, Bobb A, Feinglass J, Rooney D, Barnard C, et al. Active surveillance using electronic triggers to detect adverse events in hospitalized patients. Qual Saf Health Care 2006 Jun;15(3):184-90. (49) Kilbridge PM, Classen DC. Automated surveillance for adverse events in hospitalized patients: back to the future. Qual Saf Health Care 2006 Jun;15(3):148-9. (50) Braithwaite J, Ledema R. Patient Safety Research: a review of the tecnical literatur. http://www.med.unsw.edu.au/medweb.nsf/resources/Projects4/$file/cec_patient_safet y_30111.pdf. The Centre for Clinical Governance Research, University of New South Wales, Sydney.; 2005. Report No.: ISBN 0 7334 2176 8. (51) Singla AK, Kitch BT, Weissman J S, Campbell EG. Assessing Patient Safety Culture: A Review and Synthesis of the Measurement Tools. Journal of Patient Safety 2007;2, Number 3, September 2006:105-15. (52) Pronovost P, Sexton B. Assessing safety culture: guidelines and recommendations. Qual Saf Health Care 2005 Aug;14(4):231-3. (53) Sexton JB, Helmreich RL, Neilands TB, Rowan K, Vella K, Boyden J, et al. The Safety Attitudes Questionnaire: psychometric properties, benchmarking data, and emerging research. BMC Health Serv Res 2006 Apr 3;6(1):44. (54) Hofmann DA, Mark B. An investigation of the relationship between safety climate and medication errors as well as other nurse and patient outcomes. Personnel Psychology 2006;59(4):847-69. (55) Thomas EJ, Sexton JB, Neilands TB, Frankel A, Helmreich RL. The effect of executive walk rounds on nurse safety climate attitudes: a randomized trial of clinical units. BMC Health Serv Res 2005 Apr 11;5(1):28. (56) Madsen MD, Andersen HB, Itoh K. Assessing Safety Culture in Healt Care. In: Carayon P, editor. A Haandbook of Human Factors and Ergonomics in Healthcare and Patient Safety. 2006. p. 693-713. (57) Kristensen S. Patient Safety Culture: Assessment instruments by Kristensen, S og Bartels P. In: J.J.E.van Everdingen, S.M.Smorenburg, W.Schellekens, S.Cucic, editors. Patient Safety Toolbox. 2007. p. 3-159.

34

Workshop discussion paper: What's best in accreditation and remediation? Research Agenda Track: Parallel Workshops Session 1 Tuesday 25th Sept, 2007.

35

Patient Safety Research shaping the European agenda 24-26th September 2007: Porto, Portugal

WORKSHOP DISCUSSION PAPER

If a tree fell in the forest but no one heard, is it because they did not want to hear? Margarida França i), Vahé A. Kazandjian ii)

Margarida França i), Vice President of the National Institute of Emergency Medicine, Lisbon, Portugal.. [email protected] Vahé A. Kazandjian ii) President of the Center for Performance Sciences, Maryland, USA; Senior Vice President for the Maryland Hospital Association; Co-Chair of the Board for the Maryland Patient safety Center. [email protected]

36

Summary The objective of this paper is to raise awareness about the value of quality improvement (QI) methodologies and performance measurement techniques to the safety of care. Since the times where quality improvement techniques were of exclusive use of industrial sector to nowadays, there has been a great challenge on the application of performance measurement and improvement to the delivery of health care. Health care organizations that implement QI effectively expect to contribute for the global safety of the delivery of care. Yet, there is a lack of research to systematically demonstrate how organizational experience in QI strategies can contribute to the improvement of safety of care as well. The increase in mobility of patients within the European countries is requesting safe care across borders and the adoption of common techniques and evidence-based guidelines. A few projects have already introduced the concept of European harmonization within external evaluation models with recommendations. Accreditation and indicators play a senior role on the external auditing methodologies and they constitute the core subject of this paper on patient safety research. This paper presents for discussion the idea that the next step should belong to the research area within a cooperative agenda between European countries in order to set up evidence based guidelines and tools, that may contribute to the harmonization of health services delivery of care and to the excellence of European medical services.

______________________________________________________

Introduction Health is a multi factorial reality where the health care plays a significant role (1). Pressure on the health services all over the world has created the need to look to quality of services as a measurable attribute of care that needs an economic valorization and social value. Medical care initially viewed as a professional responsibility has been raised as a national policy issue and more recently as a European level policy with the patient safety agenda. Quality improvement methodologies and techniques were originally developed in the manufacturing sector and its application to service delivery has been a challenge that has motivated experts on the last two decades. Regarding the health sector the big challenge has been on the application of performance measurement and improvement to the delivery of care. The factors for the increasing interest on health service and systems performance are identified and consensual around main topics: rising costs of health care delivery, technological advance, aging populations, health market failures, poor quality and variations in practice, medical errors and injuries, lack of accountability, inequalities and sheer uncertainty (2).

37

Quality services, in healthcare or other industries, have safety as an integral dimension of their responsibility. It is therefore not possible to think of quality separately from safety, but as an inclusive concept and strategy to assure that not only the right things are done but that they are done without harming the recipient or the provider. Paradoxically, the healthcare systems around the world are now facing a dilemma when dealing with the “old” concept of quality and the “new” imperative of safety: are strategies to enhance safety different from those already in place for quality/performance improvement? We, across the Portuguese healthcare system, may soon face the same questions as others in Europe and other continents. It is therefore important that a clear framework be designed and adopted for our healthcare system where not only the strategies for improving performance are considered, but also the way we will demonstrate that improvement has taken place and is sustained. The purpose of this paper is to address questions that are applicable to all European healthcare systems. Also the continuous quality improvement philosophy has been the scope of the interventions to improve quality of care and reduce undesirable events despite the lack of scientific and documented evidence about the effectiveness of these efforts. Much more research has to be developed to get conclusions on these questions and on the attempt to link with performance improvement of health care organizations. Traditional strategies and tools toward quality assurance (QA) and quality improvement (QI) as regulation, accreditation, indicators, clinical audit and peer review, etc, are accepted as positive ways to improve safety despite the lack of evidence as well. The search for evidence of the main interventions should be addressed on international basis and as reference for policymakers and governmental strategies, both national and international. At the same time we have observed a general growing of external inspection by the Governments interested in showing citizens that quality of care is a top priority of the political agenda. Quality and accountability concepts started to show up on most of European plans of health care. Again most of the external assessment initiatives are lacking on validity and evaluation about results of care (3). In fact accreditation, ISO certification, EFQM projects side by side with statutory inspection is disseminated without evidence about the clinical benefit and the differences between them. It is a fact that a pool of several initiatives lacking integration and comparison has appeared in most of the European countries. So two research questions ought to be addressed: 1. In what way QA and QI techniques are able to contribute to the safety of patients and direct care providers? 2. What specific elements of each technique or methodology are associated with safe care performance and which ones are not related to the improvement of patient/provider safety?

38

What is safety in the delivery of health services? And who is affected? Safety is on the international agenda and on most of the national agendas as well. The launch of the World Alliance for Patient Safety has helped to raise patient safety to the top priorities of the political agendas. The same idea came out from the Luxembourg Declaration with a clear emphasis on the establishment of structures and processes to demonstrate tangible improvement on the safety of care. At the same time European citizens appear to be more concerned about the risk of facing a medical error when dealing with health care services (4). Therefore there is an urgent need to assess the impact of safety strategies. The immediate question remains strategic, or at least a tactical one: should safety be part of QI programs, including accreditation, or should it be developed independently and in parallel ? There are four “audiences” affected by the level of safety in health services: 1. the recipients (patients and indirectly their families); 2. the providers (medical team); 3. the community; and, 4. the financing organizations (government, private). A safe system is therefore one that minimizes the physical and psychological harm to recipients of services; does not put the providers at risk for harm; protects the community from preventable consequences of harmful or potentially harmful care delivery; and, does not unnecessarily burden the financing of the system by providing redundant, inappropriate or inconsiderate services. This definition aims at stressing that safety in care is more than patient safety. The comprehensiveness of this definition should be reflected in modern healthcare system’s vision to propose, implement, and evaluate strategies where quality and safety of care are used as a continuum, and their evaluation within the context of social expectations. Should safety be measured differently from quality? A look at the “safety” measures shows that the traditional and existing measures of quality/performance improvement are proposed to also be measures of safety. For example: 1. Complications of surgery (due to misuse of prophylaxis or the surgical act itself); 2. Falls (due to inappropriate medication/dose, risky environment, patient-tonurse ratio); 3. Readmissions (due to complications or inappropriate patient/family education); 4. Returns to operating room (due to improper surgical care); 5. Mortality; 6. Patient restraints and their unnecessary harmfulness; and, 7. Etc…. These measures are at the core of many initiatives to improve safety in healthcare. Two international projects have already proposed such measures and are in the process of

39

testing them. The WHO’s Europe Office has embarked upon an indicator project to assist European countries in introducing the concepts and tools for quantifying key aspects of clinical and organizational performance. The PATH Project was initiated in 2003 and is now in it second phase of testing (5). The OECD Health Care Quality Indicator Project was launched in 2001 with the long term objective of developing a set of indicators that reflect a robust picture of care quality that can be reliable reported across countries with the help of a set of comparable date (6). A set of indicators to be included in the OECD Health Data is the expected result from the second phase of this project. According the most initial findings of the OECD HCQP nowadays there are a limited number of indicators available for cross-national comparison of care delivered. Patient Safety has been identified has one among five top areas of research and consequently of improving of data systems (7). At the present phase it involves twenty three countries. Yet, within the framework of the more global definition of safety, we propose that additional measures should eventually be part of the safety/performance improvement strategies and tools. For example: • Injuries to physicians/nurses (needle sticks, assaults, infectious diseases). • Additional cost to the healthcare system due to redundant and unnecessary services. • Level of patient and family compliance post discharge due to inappropriate education about the regimen, and potential side-effects. Quality improvement initiatives within the Portuguese healthcare system The Portuguese Ministry of Health, back in 1999 (8) following the steps of other European countries as England with the National Institute of Clinical Excellence (NICE) and France with the creation of the Agence Nationale d’Accreditation et d’Evaluation en Santé (ANAES) have initiated a strategy on quality for the NHS health care units and adopted a dual strategy of healthcare performance improvement as a partner to Portuguese hospitals and long-term care organizations: 1. 2.

A.

helping hospitals achieve accreditation; and, providing hospitals, primary care centers and long-term care organizations tools to improve performance.

Accreditation

Accreditation of an organization is similar to the licensure of an individual to provide services. It is an accountability framework whereby through the accreditation process organizations discover the areas in need for improvement; and, when accredited they demonstrate to society that they are qualified to provide high levels of services. Accreditation has got its genesis on a set of minimum standards created in 1917 by the American College of Surgeons. From a professional and corporative initiative the accreditation has turned in to the most common and longest established program for external assessment of healthcare organizations (www.isqua.org.au).

40

In the last decade accreditation programs have grown in number of initiatives and countries adherence, especially European. By 2004 year 26 programs were active or in development in 18 countries (9). Most of these programs were launched in western European countries. The initial focus has been on the improvement of hospitals to extend on a second phase to secondary and tertiary care services (9). Accreditation has shown a capacity to adapt easily to different objectives and national needs and characteristics as well. However some of the programs seem not to achieve the sustainability to succeed after the initial enthusiasm and do not show significant growth. Most of European countries have accreditation programs follow the worldwide trend of using accreditation for the dual purpose of helping hospitals to improve their care and provide a platform for external accountability about their qualifications to provide high quality care. The philosophy that is under the political decision is the attempt to create organizational based approaches to improve care through systematic peer review. Accreditation has often been contrasted to licensure, while licensing requires compliance with minimum standards in order to protect public safety; accreditation requires compliance with “good practice” standards (10). The direct relationship that accreditation has to safety of care results from the activities on risk reduction in the care processes. Most accreditation programs have around 60% of standards related to risk management and patient safety with a clear tendency of reinforcement and autonomy. Consequently, the ongoing assessment and evaluation of compliance with accreditation standards produce valuable data which can be used as a resource for safety of care research and performance improvement. Yet, the sustainable and direct link between accreditation and performance improvement (quality and safety as performance dimensions) has not been established. While it is believed that accreditation will help establish a culture of quality and safety (11, 12), the hypothesis that accredited hospitals do offer a better performance remains more a belief (and perhaps a wish) than a demonstrated, replicated, and sustained reality. What are the confounders in establishing a link between performance improvement activities and accreditation? First, the sequence of the observations should be clarified: is it that hospitals that already have embarked upon performance improvement activities do get more easily accredited? If so, is accreditation an external validation of a strategy already adopted internally by those hospitals? Or, is the process of getting ready for accreditation a facilitator for embarking upon performance improvement activities? If so, it may be argued that accreditation is among the requisites for improving quality and safety of care, not by the act of accreditation per se, but by its requirements and systematic evaluation of the hospitals activities and outcomes.

B.

Performance improvement tools

In 2003, Portugal joined through a Ministry of Health service, the Institute for Quality in Health (IQS) twelve other countries as a participant in the International Quality Indicator Project (IQIP) (13;14). The IQIP is largely documented on European and Asian webpages and peer reviewed literature (http://www.ospfe.it/index.phtml?id=974; http://www.epos-bg.de/cms/index.php?id=95; http://www.tjcha.org.tw/S_english.asp?catid=4).

41

While Portuguese hospitals joined the IQIP (called PQIP for Portugal QIP) to improve their overall performance through indicators and learn about better practices from more than 1,000 other hospitals in the QIP, it has been shown that the “quality” indicators of the PQIP are identical to the “safety” indicators now proposed in other European or international initiatives. Indeed, the PQIP have access to the following indicators that can be used for both quality and safety improvement: • Antibiotic Prophylaxis, • Unscheduled Readmissions, • Unscheduled Admissions, • Unscheduled Returns to ICU, • Unscheduled Returns to OR, • Physical Restraint, • Documented Falls, • Unscheduled Returns to ED, • Patients Leaving ED Before Treatment is Complete. The Ministry of Health, through central services, will continue to coordinate the PQIP in Portugal, and we believe that the increasing interest of Portuguese healthcare organizations will be well served with the tools and teachings of the PQIP which links safety and quality of care performances through indicators and participants’ training in improvement practices. The worldwide use of the IQIP as a quality improvement and safety enhancement system has been documented (15) In November 2002 the International Society for Quality in Health Care (ISQua) organized the 5th International Indicators Summit in Paris. Since that time European countries initiatives have arise on quality indicators. At the same time most of countries that have implemented accreditation and quality indicators are facing the same challenge of integrating quality indicators within the certification and accreditation processes. As mentioned at the outset, the linkages between performance improvement and accreditation (or vice versa) remain mostly not-established. And, with the purposeful focus on safety, hospitals and health care systems seem to be faced yet with a new challenge beyond that of evaluating the role of accreditation – should there be two distinct philosophies and tools for measuring and improving safety of care as contrasted to quality of care? So, quality or safety? We propose that there is no such distinction. Safety is part of quality and the two will be improved simultaneously. Further, the tools for improving safety are identical to those used in performance improvement – what will differ is the set of incentives provided to those organizations showing commitment to and results from improving their performance and accountability. In particular, the use of these tools should be provided training and guidance to realize that existing “quality improvement” tools do measure overall performance, leaving the interpretation about safety to the analyzers of the data. We propose that it should be known if a tree falls in the forest, even if no one was listening at that very moment!

42

Discussion Many emphasize that the effectiveness of QI initiatives when combined with accreditation models has not been duly demonstrated despite the common belief that quality of care should be improved. Specially, regarding overall organizational performance, the evidence is still less than systematically reported. However the pressure on health organizations to produce high quality care with decreasing budgets seems to have affected their adoption of QI initiatives, accreditation, or both simultaneously. Some health managers have still difficulties in embarking upon QI projects under financial constraints as recently has happened in Portugal where new initiatives have decreased significantly with the end of the seven year of frame of European funds to QI in health organizations. And this phenomenon can be encountered in other European countries and worldwide (16). The challenge seems to be one of accountability, where clinical, organizational and financial dimensions need to be reflected in services relevant and appropriate to patients and communities. Also there’s a need to look for evidence about safety solutions and their availability to be applied worldwide. In the non existence of clear evidence and or technical orientation most European countries have seen the appearance of multiple initiatives to assess and improve the delivery of care. There is consensus that the main benefit of accreditation process comes from the effort of preparing for the survey and from the increasing awareness of staff about quality issues (17). In what way the awareness about patient expectations about outcomes of care will shape the national strategies of building QI initiatives along with accreditation is perhaps the next phase of quality improvement in healthcare, in Europe and worldwide.

43

References 1. Lalonde M. A new Perspective on the Health of Canadians. Ottawa, Office of the Canadian Minister of National Health and Welfare, 1974. 2. Kelley, E and Hurst, J. OECD Health Working Papers, N.º 23. Health Care Quality Indicators Project Conceptual Framework Paper, OECD Publications Service, Paris, March 2006b: 36, Available at http://www.oecd.org/dataoecd/1/36/36262363.pdf 3. Shaw, C. External assessment of health care. BMJ. 2001 April 7; 322(7290): 851–854. 4. European Commission. DG Health and Consumer Protection – Luxembourg Declaration – Patient Safety. Making It Happen, Luxembourg, April 2005: 2. 5. Veillard J, Champagne F, Klazinga NS, Kazandjian VA, Arah OA and Guisset .A Performance Assessment Framework for Hospitals; The WHO Regional Office for Europe PATH Project. International Journal for Quality in Health Care, September 2005: 1-10. 6. Kelley, E and Hurst, J. OECD Health Working Papers, N.º 22. Health Care Quality Indicators Project. Initial Indicators Project, OECD Publications Service, Paris, March 2006a, 151, Available at http://www.oecd.org/dataoecd/1/34/36262514.pdf 7. Mattke Soeren, A. M. Epstein and S. Leatherman. The OECD Health Care Quality Indicators Project: history and background, International Journal for Quality in Health Care, Vol. 18, Sup. I, September 2006: 1- 4. 8. França Margarida, A. Boavista and Luis M.C. Ribeiro. A experiência portuguesa de acreditação de hospitais, Qualidade em Saúde, N.º 0, June 2000: 28-32. 9. Shaw, Charles D.. Accreditation in European Health Care, Joint Commission Journal on Quality and Patient Safety, Vol. 32, N.º 5, May 2006: 266-275. 10. Roa, Donna Vincent and Anne Rooney. Improving Health Services Delivery with Accreditation, Licensure and Certification, QA Brief, The Quality Assurance Project’s Information Outlet, Bethesda, Vol. 8, N.º 2, Fall 1999: 4-11. 11. Alexander J.A., Weiner and Griffith. Quality improvement and hospital financial performance. Journal of Organizational Behavior. Volume 27, Issue 7 , Pages 1003 – 1029. 2007 12. Kelder J.A, Health service organisations and professionals: an information systems model for transforming the nexus between accreditation and practice. Medinfo. 2007; 12 (Pt 2):1189-93. 13. Kazandjian VA, Matthes N, Wicker KG. Are performance indicators generic? The international experience of the Quality Indicator Project(R). Journal of Evaluation in Clinical Practice, May 2003, 9(2): 265-276.

44

14. Thomson R, Taber S, Lally J, Kazandjian V. UK Quality Indicator Project(R) (UK QIP) and the UK independent health care sector: a new development. International Journal for Quality in Health Care. April 2004, 16 Suppl 1: I51-I56. 15. Kazandjian V.A, Wicker K, Ogunbo S, Silverman N. Understanding Safer Practices in Health Care: A Prologue for the Role of Indicators. Journal of Evaluation in Clinical Practice, April 2005, 11(2):161-170. 16. Fabry J., Morales I., Metzger M.H., Russell I., Gastmeier P., Quality of information: a European challenge. J Hosp Infect. 2007 Jun;65 Suppl 2:155-8. 17. Scrivens Ellie. Impact of accreditation systems upon patient care, J. Clin. Effect. Vo.. 2, N.º 4. December 1997: 102-105. AMERICAN College of Surgeons. «The Minimum Standard of the American College of Surgeon’s Hospital Standardization Program», Bulletin of the American College of Surgeons, Vol. 8, N.º 4, 1924. Council of Europe. Committee of Ministers. Recommendation Rec (2006)7 of the Committee of Ministers to member states on management of patient safety and prevention of adverse events in health care, 2006. Mainz, Jan, Defining and classifying clinical indicators for quality improvement., International Journal for Quality in Health Care, Vol. 15, N.º 6, 2003: 523-530. Millar John, Soeren Mattke and the members of the OECD Patient Safety Panel. OECD Health Working Papers, N.º 18. Selecting Indicators for Patient Safety at the Health Systems Level in OECD Countries, OECD Publications Service, Paris, October 2004, 44, Available at http://www.oecd.org/dataoecd/53/26/33878001.pdf Mcloughlin, Vivienne et al. Selecting indicators for patient safety at the health system level in OECD countries, International Journal for Quality in Health Care, Vol. 18, Sup. I, September 2006: 14-20. NATIONAL Clinical Assessment Authority, Understanding performance difficulties in doctors, NCAA report, London, November 2004: 45. OECD Directorate for Employment, Labour and Social Affairs. Health Committee. Health Care Quality Indicators Project. Patient Safety Data Systems In The OECD: A Report of A Joint Irish Department Of Health – OECD Conference, Paris, March 2007: 35. René, Amalberti et al. The SIMPATIE project. Safety Improvement In Europe Work Package 5 – Improving patient safety through External auditing, Haute Autorité de Santé, December 2006: 29. Available at http://www.simpatie.org/Main/pf11175587453/wp1175588035/wp1175589333

45

Vincent Charles et al. How to investigate and analyse clinical incidents: Clinical Risk Unit and Association of Litigation and Risk Management Control, BMJ, Vol. 320, March 2000: 777-781.

46

Workshop discussion paper: What roles for the patient in patient safety research? Research Agenda Track: Parallel Workshops Session 1 Tuesday 25th Sept, 2007.

47

Patient Safety Research shaping the European agenda 24-26th September 2007: Porto, Portugal

WORKSHOP DISCUSSION PAPER

What are the roles for the patient in patient safety research? Jill Hall

Jill Hall Research Fellow Department of Health Sciences, University of York

48

Summary Adverse events present a substantial issue in the health care environment. There is now increasing interest in involving patients in strategies which aim to reduce their occurrence. Patient involvement can be broadly divided into: individual patients (or their family/friends/representative) taking on roles to help ensure their own safety in health care contexts; patients acting as representatives to help ensure the safety of others; and patients taking part in research. At present there are a wide variety of strategies employed in order to facilitate patient involvement at both an individual and representative level and also to engage patients in research activities. There is little evidence for the effectiveness of many interventions (in particular those at an individual level) and patients’ views and beliefs on the topic remain largely unexplored. Since there is no overall clear picture of strategies to involve patients, nor of their effectiveness, we propose that a broad mapping process is undertaken in order to establish what efforts are currently being employed to involve patients. In addition, a survey of patient and professional attitudes to such involvement should be undertaken. However, areas that appear to be particularly worthy of further research are an investigation into patients’ attitudes and beliefs; patient held records or patient access to records; and patient contributions to incident reporting systems. Existing research Background Adverse events present a substantial issue in the health care environment. The World Alliance for Patient Safety defines an adverse event as ‘an injury related to medical management, in contrast to complications of disease’. International estimates suggest that the incidence of adverse events that occur each year is between 3.8% and 16.6% (average 8.9%) and that nearly 4% of those adverse events are potentially preventable (1). Strategies to reduce adverse events have so far mainly focused on the change of systems of care and professional behaviour. However, more recently there has been a growing interest in involving patients in safety initiatives. This increasing interest in patient involvement can be broadly divided into three main areas; • Individual patients (or their family/friend/representative) taking on roles to secure their own safety in health care contexts. In particular the development and use of interventions to promote and support their (or their families’/friends’/representatives’) involvement. • Patients as representatives to help ensure the safety of others. For example, as people with experience of being a patient in hospital they may be asked to attend the hospital safety committee or act as a patient representative on patient safety ‘walk-arounds’. • Taking part in research. Patients may be involved in areas such as the setting of research agendas/priorities (both nationally and internationally), piloting interventions and participating in research design and conduct.

49

This paper aims to summarise the current literature pertaining to patient involvement in patient safety and also to highlight the gaps in the evidence base along with suggestions for future research. Roles as individuals The concept of individual patients (or their family, friends or representatives) taking on roles in order to enhance their own safety while using healthcare services, along with efforts to facilitate that contribution, is most advanced in the USA but is increasingly receiving attention elsewhere.1 Indeed, patients are potentially ideally placed to notice patient safety issues in their care that health care workers perhaps would not. There are a number of roles that patients could potentially play; for example, helping to reach an accurate diagnosis; deciding on an appropriate treatment or management strategy; choosing a suitably experienced and safe provider; ensuring that treatment is appropriately administered, monitored and adhered to; and identifying side effects or adverse events quickly and taking appropriate action (2). In addition, there are a wide variety of interventions that have been designed to promote patients’ roles in securing safety such as information guides, education programmes, web portals to access health records (3, 4) and taking part in surgical site marking (5, 6, 7). The content of any one role or intervention, in general, tends to consist of several individual behaviours that patients are being encouraged to adopt. In order to help make sense of the current literature it is useful to identify and separate these behaviours since the acceptability, practicality and effect may differ for each behaviour. Patient behaviours that are encouraged by the current literature include: • Receiving safety-related information. For example, leaflets, brochures, posters and verbally from health care professionals (HCPs). • Following instructions given to them regarding their healthcare or condition. • Seeking out knowledge, for instance about their condition. • Sharing information with HCPs about, for example, one’s medication or medical history. • Asking for clarification if they have not understood something. • Checking various aspects of the healthcare they receive. • Taking part in their care, such as contributing to their treatment plan. • Raising concerns about their safety. For instance, this may be to a HCP about an aspect of their care or, by contributing to a reporting system. Receiving safety-related information One of the most widespread behaviours that patients are encouraged to adopt on which they can act to ensure their safety is receiving safety-related information or advice. This can be, for example, verbally via HCPs, patient package inserts that are distributed with medications, videos, posters or information brochures. There are numerous books, brochures, posters, videos and other materials (particularly in the USA) offering advice to patients about the kinds of action they should take to ‘help to prevent errors’ and ‘get safer health care’ (5, 8, 9). Within a single advice leaflet the tips that are advocated generally require patients to adopt many of the other behaviours that are highlighted below.

1

From this point, where individual patients’ taking on roles to help ensure their own safety are referred to this is taken to include their family, friends or representative.

50

Following instructions These may be instructions or education given verbally by HCPs or from written brochures and leaflets covering many aspects of healthcare eg following instructions about how to store and administer medication correctly or following pre-operative instructions prior to some sort of procedure. Seeking out knowledge For example, patients using the internet to access information about their condition, so that they are better informed to help ensure they keep safe. Increasingly, patients can also seek out information on quality measures of hospitals and clinics, such as rates of infection, to assist them in making decisions about their healthcare. Sharing information Sharing information with HCPs, for instance about one’s medical history or medication, is another action that patients are increasingly encouraged to undertake in order to help ensure their safety. A popular approach is the ‘brown bag’ initiative where patients are asked to bring all their medications with them on admission to hospital (10) or more innovatively, patients having electronic access to their records and patients carrying their own (or their child’s) health records (11). Asking for clarification For instance, during or after consultation with a health care professional if some aspect of the information given was unclear or not understood. Checking Much of the current literature also advocates that patients’ ‘check’ various aspects of the healthcare they receive. For example, checking the site marked for surgery in order to prevent error (6) or checking that HCP has got the ‘right patient/right medication/procedure’ by offering ID bracelet to HCP to verify identity prior to administration of medication or procedure (12). Taking part By ‘taking part’ in, for example, their own treatment plan. For example in mental health service users may help plan their care for acute psychiatric emergencies when they are ‘well’, in order to help avoid serious escalations (13). Raising concerns Many interventions advise or encourage patients to ‘raise concerns’ that they may have about their safety. This could be concern, for example, about medication administration (eg where tablets look different from those usually prescribed) or concern about HCP behaviour (eg the ‘Please Ask’ campaign in the UK advises, amongst other things, that patients ask the doctor/nurse whether they have washed their hands, http://www.npsa.nhs.uk/pleaseask). Alternatively, patients in Europe can now increasingly raise their concerns by contributing to reporting systems when an incident occurs. For instance, these systems exist in the UK, Netherlands, Denmark and Ireland. By reporting an adverse event or ‘near miss’ to these systems patients are not only acting to ensure their own safety but, in addition, their action may also help to ensure the safety of others if the system is used as a learning tool.

51

Comment on roles for individuals There are several issues with patients taking on roles to help ensure their own safety that should to be considered: • There appears to be a wide range of different interventions employed many of which are US based and not necessarily transferable to a European setting. • There is little evidence of the effectiveness of many of the interventions intended to facilitate patient participation (14). It is also imperative that where interventions are implemented and they consist of written advice, patients’ literacy is considered and that their understanding of the information given is assessed afterwards. • There is a dearth of evidence from non-hospital care settings (eg primary care) and long term care settings (eg residential and nursing homes). • With the exception of a handful of studies, patients’ views and preferences regarding these roles and behaviours remain largely unexplored. • Potential, unwanted consequences of patients being encouraged to adopt these roles have rarely ever been considered eg. the impact on patient perceptions of safety (maintaining the balance between raising awareness and raising anxiety), patient-professional interactions, the attribution of responsibility for health care status, the impetus for system improvement and social inequalities in health care experiences and outcomes (15). Roles as representatives Patients may be involved in patient safety as representatives to present patient perspectives. Their involvement may be facilitated by HCPs, policy makers and patient safety organisations. This involvement may take a number of forms: • as people with experience of a particular condition – piloting/evaluating an intervention; for example, patients with rheumatoid arthritis piloting a ‘methotrexate diary’ which provides patients and health care professionals with information such as the “dos and don'ts” of treatment, symptoms to look for, monitoring results and a record of appointments (16). • as people with general experience of being a patient in hospital – attending meetings; acting as patient representative on patient safety walk-arounds; at significant event audit meetings (17). • as people with experience of a particular condition acting as intermediary or bridge between patients, their families & clinical team, for example peer counsellors working within mental health services in New York (18). • providing a patient viewpoint when designing/evaluating healthcare facilities, systems or procedures, for example o users of mental healthcare services on responses to psychiatric emergencies (19). o commenting on and suggesting changes to the layout of medication inserts (20). At an international level patient representatives, who may be working at a national level in their own country, may be called on to take part in forums which aim to promote international collaboration, such as The WHO Patients for Patient Safety Workshops. These are facilitated workshops which aim to recruit and develop patients and consumer partners who are or wish to become champions in advancing patient safety (21).

52

In contrast to patients being called on by HCPs, academics or policy makers to contribute as representatives, some patients and families, who have been directly affected by medical mistakes, have established support groups and online communities which set out to reduce the incidence of medical error, act as a voice for patients and consumers and provide support for those affected by medical error. Examples of these include: o Kilen – a consumer group set up in Sweden which collects patient reports of medical adverse events (www.kilen.org/indexe.htm) o Sufferers of Iatrogenic Neglect (SIN) are a UK based patient support and pressure group for people affected by medical error (www.sinmedicalmistakes.org) o Josie King Foundation in the USA. It was set up after the death of Josie King due to medical error and aims to ‘prevent others from dying or being harmed by medical errors. By uniting healthcare providers and consumers, and funding innovative safety programs, we hope to create a culture of patient safety, together’. (www.josieking.org). These groups and organisations tend to be web based and have grown up from patient concerns and desires to be involved. Increasingly, these organisations are being consulted by policy makers to provide the perspectives of people who have been harmed in some way by a medical error. There are a number of issues with patients as representatives that should be considered. Organisation of patient involvement can be problematic and involves time and resource commitment. There are problems with integrating ‘lay’ people into areas which are traditionally populated with HCPs such as the established HCP culture, language and the patient role which is traditionally a passive one, deferring to the greater knowledge and expertise of the health care professional. Involving patients as representatives requires agreement and co-operation from other stakeholders. This can be problematic especially when time and resources are at stake and this may lead to ‘token’ patient involvement. As always, hard to reach groups are under-represented. This is a significant problem as people in these groups are more likely to have low health literacy and, consequently, to experience a patient safety event (22). Taking part in research Increasingly, patients are being called on to provide input for patient safety research. This may include agenda setting, research design and management. In the United States, The National Patient Safety Foundation’s (NPSF) Patient and Family Advisory Council, which provides guidance and patient perspective for all NPSF activities, has produced an agenda for action which includes suggested areas for research (9). INVOLVE, an advisory group on public involvement in research and development in the UK, lists a number of patient safety research projects which have involved patients in all stages of the research process from prioritising topic areas and designing research instruments to analysis and dissemination (www.invo.org.uk). Although some patient representatives are involved in setting the research agenda, they are often called later in the research process to be involved in priorities identified by others. True involvement may then be more problematic.

53

Issues that deserve further research attention Taking into consideration the dearth of research in many aspects of patient involvement in patient safety, the topic as a whole would benefit from further investigation in a broad, scoping manner, in order to map the strategies currently being used (see Watt et al (27) for research currently ongoing) and in addition, some exploration of both patient and professional attitudes. However, specific areas worthy of further research include: • • •

Investigation into the attitudes of patients towards suggested roles in order to gain a deeper insight of their views and beliefs. This is, perhaps, crucial to the understanding and development of any potential interventions. Development and evaluation of interventions which aim to give patients greater access to their records. Specifically, patient held records and patient access to electronic personal records. Evaluation of incident reporting systems which accept contributions from patients and the public.

Patients’ attitudes and beliefs Patient safety initiatives and interventions which aim to enable and support patients’ roles in securing their own safety tend to be designed and implemented by HCPs with little input from patients, despite the fact that their success depends on the willingness and ability of patients to adopt them. The views of patients and their likely uptake of interventions remain largely unexplored. The few studies that have explored patients’ attitudes suggest that they may have issues with current initiatives that could potentially make them more or less likely to adopt them. Advice that requires patients to directly challenge or question health care professionals, or appear ‘rude’ is deemed to be more problematic (23, 24, 25), in part due to the fear of recriminations (24). Conversely, patients may feel more comfortable with roles which they perceive as realistically achievable, such as keeping their doctor informed about their medications and allergies. However, the limited evidence currently available also suggests that even behaviours that patients are comfortable with do not necessarily translate into action (25). Whilst this information on patient views and attitudes towards potential patients’ roles provides some degree of insight, much more research is needed (26). If patients are to be involved in ensuring their own safety in a meaningful manner then interventions which aim to facilitate this should be based on an in-depth understanding of patients’ views on safety issues, their willingness to take on roles and those factors that might make them more or less likely to do so. Some of the key issues that should be considered include: • A better understanding of patients’ awareness of, and attitude towards, safety issues. What do they understand by the terms ‘patient safety’ and ‘medical error’? Are they concerned about safety when they use health care services? Their views are likely to differ from those of HCPs and may affect their ability to

54

•

•

• •

be involved. The development of future interventions will need to reflect those opinions. Do patients see a role for themselves in helping to ensure their own safety while using healthcare services? Desire and ability to participate are likely to differ between individual patients depending on a multiplicity of factors and circumstances. For instance, what influence do factors such as past experience of health care services, knowledge of their condition, age, education, culture and social circumstances exert on a patient’s ability and desire to be involved. In addition, does an individual patient’s ability and desire to take on a role alter over the trajectory of a single episode of health care? If so, when and why does this occur? More information regarding patient roles. What kind of roles are patients willing to adopt and which present difficulties for them? In addition, where patients are given information, what modes of delivery work best eg. written leaflet, videos, posters? What are the barriers to patients taking on certain roles? The reasons may be many and wide ranging but it is likely that staff attitudes will be hugely influential. If a patient speaks up, how do they think it will be received by the HCP? What type of support would patients require in order to undertake these roles? For instance, support from HCPs, so that patients know that if they raise an issue their concern will be acted upon. Do patients require practical support to act on advice given eg who to contact and when, knowledge of the ‘system’?

A number of these issues will be addressed, in a UK population, by research currently ongoing (27). However, this work will not address the significance of international contextual and demographic variables on patients’ attitudes and beliefs. Patient records Medical records are an area in healthcare where patients are already able to contribute to ensuring their own safety. For sometime, it has been customary for patients receiving obstetrics and paediatric care to hold their own records (28, 29) although their use has not spread to other disciplines for which they could be potentially useful (30). Increasingly, the internet is being utilised, both in the US and Europe, as a means of patients gaining access to their medical records in order to enhance their own safety; web based systems have been introduced where patients can connect to their doctor’s office, access and check the information in their record, add data and review online health information (3, 4, 31, 32). Other systems require less active involvement from the patient such as the UK based HealthSpace (www.healthspace.nhs.uk). HealthSpace currently enables patients to store their own health notes, keep a record of, for example, their weight and blood pressure and provides links to sources of online health information. Patient access to, and patient held, records have the potential to impact on patient safety in a number of ways. Patient held records may help the patient share information about their medical history, whereby they can help ensure that their records can be viewed each time they see a health care professional. This is particularly important in situations where patients see a wide variety of different HCPs. Patients’ electronic access to their records could also potentially encourage the sharing of information, particularly if patients are able to add to the data that is held, but they also act as a tool whereby the patient can check the information that is held eg checking test results.

55

In order to develop and evaluate these interventions, further research is required. In particular, the following issues should be considered: •

• •

• •

•

For patient held records, what potential is there for expanding their use to other conditions? A Cochrane review found little evidence of their use in patients with psychotic illness, despite their potential to engage a frequently marginalised group of patients, who tend to come into contact with a large number of different HCPs (30). Access to personal electronic records: how practical is this for patients? Is accessibility to the internet an issue? How does this differ by country and region eg rural versus urban? The nature of record keeping by HCPs. How intelligible and legible are medical records to patients? If patient roles are to be developed in this area, what changes in record keeping may be necessary to ensure that they are intelligible and legible for patients? How applicable and suitable are these interventions in relation to the differing health care schemes of European countries eg social and private insurance schemes, services free at point of contact? Patients’ views of these roles should be considered. Would patients be willing to adopt these kinds of roles? In the case of access to electronic personal records what level of involvement would patients prefer? Does the use of these types of interventions introduce a risk of unwanted consequences such as the feeling of a ‘burden of responsibility’? However, in addition it is also imperative that alongside those views the attitude of staff towards a system of patient access to records is investigated. How does the acceptance of this type of intervention differ according to the culture of differing institutions and countries? Patient roles do not occur in isolation from staff; interventions that are implemented without taking into consideration the views and beliefs of staff may well serve only to raise patients’ expectations, potentially resulting in disappointment, and, in doing so, the risk of doing more harm than good. Does the implementation of these roles lead to improvements in patient safety? High quality studies, using appropriate research methods will be required in order to evaluate the effectiveness of these interventions.

Reporting systems There are many countries that now have well established incident reporting systems; some run on a national level and others are more local or organisation based. Increasingly, a number of these systems accept report of adverse events from patients and/or the public (eg Denmark, Netherlands, Ireland, UK, and Sweden). Patients who report an incident to such a system are not only acting to ensure their own safety but are also acting for the benefit of other patients; lessons learned can potentially ensure the same doesn’t happen to future patients. Whilst all these systems broadly aspire to similar aims of improving patient safety they all have different characteristics and structures. Some systems accept all patient safety concerns what ever they may be (eg NPSA in the UK) whilst others focus on particular types of incidents eg adverse drug reactions (Netherlands). Other systems have the facility to provide feedback to a patient on the issue they have raised (eg Sweden),

56

whilst others are anonymous and do not have this facility. Furthermore, the predominant method of reporting to many of these systems is electronically via the internet, although some still offer a paper based option (eg Yellow Card system, UK). A recent review indicates that the quality of patient reporting is similar to that of health care professionals (33). The review also noted that patients also identify adverse drug effects that have not been reported by healthcare professionals and additionally, many of the reports are classed as serious. However, Blenkinsopp also reported that patient reports take longer to process as patients are often unfamiliar with medical language although the quality of the information is often ‘richer’ than that submitted by health care professionals. Some systems have substantial numbers of patients’ reporting, whereas other systems struggle to engage patients (34). Due to the wide variation in structure and functioning of reporting systems, there is clear opportunity for further research and collaboration in order to establish the most appropriate mechanism for patients to be able to contribute to these systems. Issues for research include: • Mapping of the current systems that are in place. What are the characteristics of the systems that have higher levels of patient reporting? Collaboration across Europe would enable a clearer picture of best practice to emerge. • Patients’ views on reporting systems remain a largely unexplored area. At present, the systems and their structures appear to be geared to HCPs. What are patients’ views towards reporting? What factors, of both the safety issue and the reporting system, would make them more or less likely to report the incident? What sorts of things would they report? How would they like to be able to make reports eg web based, paper, telephone? Do patients prefer to receive individual feedback on their concern or are they happy for systems to remain anonymous? Feasibility of this work across European countries Due to a lack of evidence that currently exists within countries regarding the patient role, sharing of best practice across Europe is premature at this time. The patient role in patient safety would, therefore, benefit from further in-depth research at a national level. Once the research base has expanded at the national level, there may well be scope for collaboration across European countries. By sharing information on what already exists, different countries would have the opportunity to learn and collaborate; to highlight areas where similarities exist and to pool efforts to develop those strategies and interventions. Through a deeper understanding of patient and professional attitudes across the region there is also the potential to take into account the differences that exist across European countries and the impact this may potentially have on strategies to involve patients in helping to ensure their own safety whilst using health care services. Conclusion There are a number of potential roles for the patient to be involved in patient safety. These roles may be on an individual or representative level or involve taking part in research. Most areas require more research to be undertaken: however, areas of particular need or interest include: a deeper understanding of patients’ attitudes and

57

beliefs towards patient safety; investigation of promising interventions such as patient held, or patient access, to medical records and patient contributions to reporting systems.

58

References 1.

Neale et al. Unpublished material. Adapted from Sisse Olsen, E Jane chapman, Ara Darzi, Charles Vincent, Graham Neale. A new structure for audit. Presentation made by Graham Neale at Making Health Care Safer 2004, Royal College of Physicians, London, 21/22 October 2004. Used with permission. Quoted in Emslie, International Perspectives on Patient Safety, National Audit Office 2005.

2.

Vincent C, Coulter A. (2002). Patient safety: what about the patient? Quality and Safety in Health Care 11:76-80.

3.

CiminoJJ, Li JH et al. An evaluation of patient access to their electronic medical records via the World Wide Web. Journal of the American Medical Informatics Association 2000;151-5.

4.

Raths D. Gateway to success. Healthcare Informatics 2006;23(4):30-6.

5.

AHRQ. 20 tips to help prevent medical errors .Rockville, MD: Agency for Healthcare Research and Quality 2002. (Available at: http://www.ahrq.gov/consumer/20tips.htm)

6.

Arbor A. Ensuring Correct Surgery. , VA national Center for Patient Safety 2006.

7.

NPSA. Alert: Correct Site Surgery. NPSA 2005 (www.npsa.nhs.uk/site/media/documents/885_CSS%20PSA06%20FINAL.pdf).

8.

JCAHO. Speak up: help prevent errors in your care [web document]. Oakbrook Terrace, IL: Joint Commission on Accreditation of Health Organizations 2002. (Available from: http://www.jcaho.org/accredited+organizations/speak+up/speak+up+index.htm

9.

NPSF. National Agenda for Action: Patients and Families in Patient Safety. NPSF 2003. http://npsf.org/pdf/paf/AgendaFamilies.pdf).

10.

Cantrell M. A double dose of caution. Medication reconciliation programs are helping caregivers keep track of patients' meds throughout the continuum of care. Nursing Spectrum 2005;18(22):10-1.

11.

Brayden RM, Kempe A et al. Innovative approaches to child health information. Pediatric Annals 1997; 26(3):194-9.

12.

Cohen MR. Medication Errors. The Name Game: Inviting patients to play. Nursing 1999, 20.

13.

Langan J, Lindow V. Living with risk: mental health service user involvement in risk assessment and management. Joseph Rowntree Foundation 2004; vii:25-31.

59

14.

Pizzi LT, Goldfarb NI, Nash DB. Other practices related to patient participation. In: Shojania KG Duncan BW, McDonald KM, Wachter RM, eds. Making health care safer: a critical analysis of patient safety practices. Rockville, MD: Agency for Healthcare Research and Quality 2001. (Available from: http://www.ahrq.gov/clinic/ptsafety/)

15.

Entwistle, V. A., M. M. Mello, et al. (2005). "Advising patients about patient safety: current initiatives risk shifting responsibility." Joint Commission Journal on Quality & Patient Safety 31(9): 483-94.

16.

Anonymous. Patient to have new role in improving methotrexate safety. Pharmaceutical Journal News 2003.

17.

Ponte et al. Making patient-centred care come alive: achieving full integration of the patient's perspective. Journal of Nursing Administration 2003;33:2:83-90

18.

Wale JB, Moon RR. Engaging patients and family members in patient safety - the experience of the New York City Health and Hospitals Corporation, Psychiatric Quarterly 2005;76:1:85-95.

19.

Allen M, Carpenter D, Sheets JL, Miccio S, Ross R. What do consumers say they want and need during a psychiatric emergency? Journal of Psychiatric Practice 2003;9:1:39-58.

20.

Fuchs J, Hippiusb M, Schaefer M. Design of package leaflets for drugs. Results of a questionnaire among patients. Pharmazeutische Industrie 2003;65:4:302-6

21.

WHO 2006. http://www.who.int/patientsafety/events/media/dublin_call_03_09_2007.pdf)

22.

Joint Commission 2007 ‘ What did the doctor say? Improving health literacy to protect patient safety’ http://www.jointcommission.org/NR/rdonlyres/D5248B2EE7E6-4121-8874-99C7B4888301/0/improving_health_literacy.pdf

23.

Hibbard, J. H., E. Peters, et al. (2005). "Can patients be part of the solution? Views on their role in preventing medical errors." Medical Care Research & Review 62(5): 601-16.

24.

Swift, E., C. P. Koepke, et al. (2001). "Preventing Medical Errors: Communicating a Role for Medicare Beneficiaries." Health Care Financing Review 23(1): 77-85.

25.

Waterman AD, Burroughs TE, Garbutt J, Fraser VJ, Dunagan WC, Waterman BM, Levinson W, Gallagher TH. Hospitalized patients' comfort with and participation in medical error prevention. Journal of General Internal Medicine 2004;19:159.

26.

Coulter, A. (2006). "Patient safety: what role can patients play?" Health Expectations 9(3): 205-6.

27.

Watt I et al. A review of strategies to promote patient involvement, a study to explore patient’s views and attitudes and a pilot study to evaluate the acceptability

60

of selected patient involvement strategies. http://www.pcpoh.bham.ac.uk/publichealth/psrp/Publication_PS034.htm 28.

Elbourne D, Richardson M, Chalmers I, Waterhouse I, Holt E. The Newbury Maternity Care Study: a randomised controlled trial to assess a policy of women holding their own obstetric records. British Journal of Obstetrics and Gynaecology 1987;94:612-9.

29.

Brayden, R. M., A. Kempe, et al. (1997). "Innovative approaches to child health information." Pediatric Annals 26(3): 194-9.

30.

Henderson C, Laugharne R. Patient held clinical information for people with psychotic illnesses. Cochrane Database of Systematic Reviews 1999, Issue 3. Art. No.: CD001711. DOI: 10.1002/14651858.CD001711.

31.

Pagliari C, Detmer D, Singleton P. Potential of electronic personal health records. BMJ 2007;335:330-3.

32.

Wald JS. Requirements Development for a Patient Computing System. American Medical Informatics Association; Annual symposium of the American Medical Informatics Association/visions of the future and lessons from the past/a medical informatics odyssey, Washington, DC, Hanley and Belfus, 2001.

33.

Blenkinsopp A, Wilkie P, Wang M, Routledge PA. Patient reporting of suspected adverse drugy reactions: a review of published literature and international experience. British Journal of clinical Pharmacology 2006;1-9.

34.

Mansell P. A strategic look at patient reporting to NPSA. NPSA 2006, London.

35.

Devos A, Mermans E et al. Safety first! Are patients and their relatives aware of the risks posed by the toxic medication they receive? Bone Marrow Transplantation 2006; 37: S283-S283.

61

Workshop discussion paper: What is effective training for patient safety researchers? Research Agenda Track: Parallel Workshops Session 2 Tuesday 25th Sept, 2007.

62

Patient Safety Research shaping the European agenda 24-26th September 2007: Porto, Portugal

WORKSHOP DISCUSSION PAPER

What is effective training for patient safety researchers? Peter G. Norton

Peter G. Norton Professor of Family Medicine University of Calgary Calgary, Alberta, Canada UCMC North Hill, 1707, 1632 - 14 Street NW Calgary, AB, Canada T2N 1M7 email: [email protected]

63

1.

Background

We begin by arguing that there is a need for enhanced capacity in patient safety research and so enhanced training opportunities are also needed. Then based upon a particular model of patient safety research propose capabilities patient safety researchers will need in the future. We conclude with a brief list of present and future training opportunities. 1.1 Is enough patient safety research going on? When one examines the published literature one might conclude that patient safety research is alive and well. Indeed, as shown in Figure 1, there has been a steady increase in the number of citations for patient safety listed in PubMed since the landmark publication of To Error Is Human1 in 1999.

Figure 1: Citations for patient safety in PubMed on July 1, 2007. However the number of entries in PubMed increases each year and so one could ask “Is patient safety keeping up?” Figure 2 shows that it is in fact increasing faster that the size of the PubMed database. Patient safety comprised 0.3% of the listings in 1998 and in 2006 this had more than doubled to 0.7%.

64

If one adds a trend line to the graph in Figure 2 and projects to 2035 one sees that if everything remains as it is patient safety papers will be over 2% of all the publications in the PubMed by 2035 (Figure 3).

Figure 3: Trend in patient safety publications in PubMed. Should we then leave well enough alone since things are going in the right direction? We contend that the answer is no. There are at least two substantive reasons. First, patient safety is a substantial public health problem with medical errors ranking as the eighth leading cause of death in the USA killing more Americans than motor vehicle accidents, breast cancer, or AIDS. These areas have far greater publication rates now and even, if thing remain stable, in the future than does patient safety. As shown in Table 1 breast cancer citations alone accounted for 1.58% of the PubMed listings in 2006 while cardiovascular disease comprised

65

9.66%. In relation to the burden on the population it seems that patient safety research is lagging behind and needs a boost - more intellectual effort should be devoted to patient safety in the future.

Citation family medicine stroke breast cancer trauma cardiovascular disease

% of all Number in PubMed in 2006 2006 2328 0.37% 8764 1.41% 9856 1.58% 25223 4.05% 60188 9.66%

Table 1: Selected PubMed citation rates in 2006 The second reason involves the need for basic research in patient safety. True progress in an area is often based on basic research in that area. In patient safety we have borrowed the basic research from other areas. Cook addresses this issue in his editorial “Lessons from the War on Cancer: The Need for Basic Research on Safety” which appeared in 20052. In this editorial he writes “For basic research we have tended to rely on other domains, hoping that the lessons learned there can be imported directly and painlessly into health care. But healthcare is quantitatively and qualitatively different than commercial aviation or manned spaceflight.” Cooke notes that the war on cancer was declared in the US in 1971 but that “sustained progress on cancer has come through basic research on the mechanisms of cell metabolism, transcription and processing of its DNA, and the control of the cell cycle, and not via improvements in the techniques and applications already established in 1971”. His argument for the need for more basic patient safety research is persuasive. Based on these observations the need to train future patient safety researchers is pressing. Such capacity building must be informed as far as possible by our perception of the future needs and directions of the patient safety movement. Training must be efficient and opportunities must exist to allow those involved to have satisfying and productive careers. To better understand the ways in which this training might be developed it will be useful to have a conceptual model of the patient safety research domains. 1.2 The domains of patient safety research Patient safety research can be conceptualized as having three domains which are interrelated and inform each other. These research domains are: basic, applied and spread, or knowledge translation research. These domains are pictured in Figure 1. The main purpose of basic patient safety research is to expand knowledge of and insight into patient safety. Some areas that would benefit from basic research are obvious and certainly others will arise in time. At the present time we have a need for basic research in human factors and the role of cognition. As Cook2 states we must understand “how people

66

cope with the actual demands and uncertainties of the technical work of healthcare”. Measurement is another fundamental problem that must be addressed. Efficient and effective ways to establish reliable and valid estimates or indices of harmful and non-harmful events are lacking. Possibly this gap can be addressed by the electronic medical record but if so basic research concerning the care-provider IT interface will be necessary. Another area where basic research is lacking concerns the dynamic and complex nature of the organizations and systems where health care is delivered. This complexity poses deep and fundamental questions about how to accomplish change. Applied research is that research which designed to address the practical issues of and directly facilitate improvement in patient safety. Much of our current effort falls into this category. Several themes come to mind when one considers future research of this kind. One will be a fuller and more complete understanding of quality improvement demonstrations such as the 100,000 Lives Campaign3 in the US and Safer Healthcare Now!4 in Canada. Such projects, based upon proven interventions or established theoretical models, require evaluation of both the processes and outcomes including cost-benefit considerations. Another theme that could be helpful to those managing and working in the system would be practical demonstrations of incident reporting. Initiatives are currently underway in several jurisdictions and will hopefully assess the effect of these systems and again address cost-benefit issues. A third area involves mitigation of harm. Safety is not only about the prevention of adverse events but is also about minimizing the effects of such events when they are not prevented. We need to demonstrate “just in time” mitigation systems that work and are replicable. In this area it will be critical that the researchers work in partnership with those who fund and manage the system. Such alliances will ensure that the demonstration are practical and have the possibility of system wide adoption. In this part of the work there appear to be significant issues with the academy. Work of this type is less valued in much of the academy and so does not contribute to promotion and tenure to the same extent as does basic research output. Finally we need to systematically investigate how to spread new techniques and models from their original ‘test beds’ throughout the hospital, region and country. This knowledge translation work will be difficult. Currently theories used in the field come largely from the organizational behaviour and social sciences fields and have not been thoroughly tested or validated in healthcare. We will have to understand how spread our innovations to both the sharp and blunt ends to reach our goal of a safer system. With respect to this challenge elucidating the role of organizational context may be fruitful. We will need to clarify which contextual factors enable spread as well as those that work against it.

67

Human factors Organization & system theory Role of cognition

Basic research Discovery of new knowledge

Quality improvement Incident reporting Just in time mitigation

Applied research Making it work

Spread Application across the system Knowledge translation Role of context Sharp vs. blunt end issues

Figure 1: The domains of patient safety research 1.3 The work of patient safety research In each of these patient safety research domains there is and will continue to be a need for creativity and innovation. Who should do this work? Expertise and knowledge from other disciplines must be brought to bear on the issues involved in patient safety. The involved disciplines vary somewhat from domain to domain. For example in the basic research area some of the relevant disciplines include cognitive

68

psychology, organizational management, industrial engineering and traditional safety science. In the applied area those with expertise in quality improvement methods, human behaviour, economics and evaluation will be useful colleagues. In the area of spread scientists working in the area of knowledge translation will be key. However patient safety research will also require meaningful involvement of clinicians and those who manage the system. Only they can bring the knowledge of the day-to-day challenges, the human variability and vulnerability and the taxing demand of the provision of healthcare to the research. We will need to offer training and productive work environments for these clinician and healthcare management scientists. The training must ensure that they can and will communicate and work effectively and efficiently across the relevant disciplinary boundaries. These investigators will have to interact with a cadre of scientists from the other disciplines who wish and are able to extend their models and knowledge to the complex dynamic area of healthcare systems and are eager to work with these clinician and managers as equal research partners. These two groups of individuals will have to strive to understand each other’s language, culture and research paradigms. Thus the work must be truly multidisciplinary and programs of training will have to ensue that the relevant skills are fostered. 1.4 Strategic directions Following from the material presented above there are several strategies that a jurisdiction might employ that have the potential to accelerate progress in development of an appropriate cadre of patient safety scientists. The following list is neither exhaustive nor is it validated but hopefully can serve as a basis for discussion and reflection. 1. Active dissemination about the importance, value, challenges and successes of patient safety research will be important to secure fair and sustainable funding both for capacity building and the actual research. 2. Specific strategies need to be developed to attract capable students to safety research careers. 3. At all stages of patient safety research training there must be emphasis on the multidisciplinary approach. Practical aspects of training must involve multidisciplinary teams. 4. There should be active involvement with those who manage and fund the system in the research enterprise. This will facilitate both the applied and spread aspects and will generate important questions for the basic researchers. 5. Training must not be restricted to the pre and post graduate levels. It is important that the academy ensure the existence of continued professional development for patient safety scientists at all career stages. 6. A potentially valuable strategy will be the (continued) funding of programmatic patient safety research teams which include specific training components and specific accountabilities for this capacity building work. 7. Development of real and virtual patient safety research networks can be undertaken to encourage multicentre studies and cross fertilization of training.

69

2. Present capacity building initiatives Training opportunities in patient safety and patient safety research are expanding at a rapid rate. A simple Google search “patient safety” and “research training” resulted in 65,000 citations. A substantial proportion of these appear to refer to actual training opportunities. In this paper we do not attempt an exhaustive review or even cataloguing of these opportunities. Instead we have chosen to look at a few examples that may be of interest. 2.1 Patient safety in clinical training programs This important area for consideration but it is not directly related to the subject of this paper. However incorporation of patient safety material in clinical training programs will help to meet strategic direction 2 - to attract capable students to safety research careers. Such training will also ensure clinicians can work actively in the applied and spread domains. It is therefore incumbent upon those with accountability for the development of capacity in patient safety research to monitor the progress in this area and to aid and assist in development of appropriate curriculum. 2.2 Specific patient safety research fellowship programs Recently some jurisdictions have announced fellowship training opportunities that are specifically in patient safety research. Here are two examples: 2.2.1 Emergency Medicine Patient Safety Foundation Emergency Medicine Patient Safety Research Fellowship Award5 This award, announced this year, is open to a residency trained emergency physician in the US. It is a one year award with a stipend of $75,000 (US) for salary support and requires that the trainee “devote 80% effort to focused research training in patient safety”. The research training plan may include, but is not limited to, formal didactic course work and less formal training under the [required]research mentor. 2.2.2

The Dr. David Rippey Patient Safety Fellowship Award6

The Canadian Patient Safety Institute and the Canadian Institutes for Health Research, two federal funding agencies in Canada, formed a partnership to fund this award in 2007. It is targeted at post-doctoral and post-graduate health professionals interested in pursuing careers in health services research, particularly in patient safety and will provide funding to a maximum of $55,000 (Can) per annum for two years. Limited clinical practice is allowed. The fellowship will support a research project but also requires a learning plan goes beyond the research project to the involvement of decision makers and policy leaders. The application must contain specific information regarding how the candidate's and/or supervisor's relationship with decision makers enables active involvement in the training environment/learning program.

70

2.3 Graduate studies in patient safety Another emerging trend is the development of specific graduate degree programs in patient safety. These programs have the potential to develop many new patient safety scientists. Examples follow. 2.3.1

The Imperial College London7

The Imperial College London has recently announced their MSc in Quality and Safety in Healthcare. This two-year part time course is designed for experience healthcare professionals and has didactic and research components. The tuition fee will be £3,185 for home and EU students and £20,750 for students from outside the EU. 2.3.2

Northwestern University8

The Institute for Healthcare Studies in the Feinberg School of Medicine at Northwestern University in Chicago has a part-time MS program in Healthcare Quality and Patient Safety targeted at working professionals, medical students and fellows. Students are expected to obtain the degree in 18 months. The goal of the program in is “to educate healthcare clinicians and administrative professionals to become effective healthcare quality and patient safety practitioners, researchers, and thinkers”. Students have the option to complete a mentored thesis project. 2.3.3 University of Wisconsin-Madison9 A different model is being offered at the University of Wisconsin-Madison which is offering a Graduate Certificate in Patient Safety. Eligible students must be accepted into a graduate or professional degree program at the university. In addition they must either have degree in a health care related field (including health care management), work experience in health care delivery or have taken one or more specific courses in health care systems that the University offers in its engineering, nursing and law faculties. Required course work includes a Patient Safety Research Seminar and a Patient Safety Project. Another innovative approach might be a ‘laddered’ graduate program. In this model students could complete say three courses and then receive a diploma. If they wished to continue then a further two courses might be sufficient for a non-thesis degree. The program could also offer a research based route to the master`s degree with a thesis requirement. If such a program was part time and tailored to accommodate to the work of practicing health care professionals it might have the potential to develop substantial practice and research skills in mid-career practitioners. 2.4

Programs of research

Several jurisdictions have established or are establishing funding for programs of research in patient safety. These appear to be very promising vehicles for capacity building and often contribute to some or all of the strategic directions put forth above. We will examine four of these.

71

The Agency for Healthcare Research and Quality10

2.4.1

The 1999 US report To Err is Human1 recommended the establishment of centers of excellence in patient safety research and in response The Agency for Healthcare Research and Quality (AHRQ) announced grant funding for such centers in October 2000. The grants were to support the development of multidisciplinary research teams which would enhance patient safety through fundamental and applied research. This funding was intended for sites with demonstrated track records in patient safety research. Specific activities mentioned in the announcement included educational programs, professional career development and knowledge translation of findings. In 2001 three centers funded. The sites and their project tiles principal investigators and total five year funding are listed in Table 211 below. The funding was to allow the centers to complete four to seven major research projects which had been outlined in their applications. Location Brigham and Women's Hospital, Boston

Title Improving Medication Safety Across Clinical Settings

PI Bates, David W.

Five year funding $5,545,748

University of Pennsylvania, Philadelphia

Improving Patient Safety by Reducing Medication Errors

Strom, Brian L.

$6,795,283

University of Texas Health Science Center, Houston

Translating Safety Practices from Aviation to Healthcare

Thomas, Eric J.

$7,206,859

Table 2: Centers of Excellence for Patient Safety funded by AHRQ in 2001 An evaluation of this program does not appear to be available at this time. There has not been another cycle of funding or an announcement of one at this time. 2.4.2

Developing centers of excellence in patient safety research12

At the same time AHRQ also announced funding for “Developing centers of excellence in patient safety research” (DCERPS). Funding was available for projects that lasted up to three years. The request for application (RFA) stated this funding envelope was “targeted toward "new" researchers and research teams—individuals or teams who have not conducted a lot of patient safety research, have not conducted specific portions of the patient safety research agenda and would like to expand in that area, or have not previously worked together as a full team.” The RFA went on to state that after completion “it is expected that all awardees will be eligible to compete for funding similar to the Centers of Excellence” and that “the overall goal of the RFA is to build capacity related to patient safety research”. Plans related to three specific activities that were to be included in the applications: • •

Building a multidisciplinary team Forming ties to the delivery system

72

•

Producing education materials about the importance of patient safety and the current evidence about improvement strategies

Eighteen grants were awarded in 2001. Seven examples are listed in Table 311.

Location Hektoen Institute, Chicago University of Washington, Seattle RAND, Santa Monica, California

Title CCH/RUSH Diagnostic Error Evaluation and Research Center Center for Evaluation and Research in Pediatric Safety

PI Schiff, Gordon

Three year funding $592,156

Taylor, James A.

$599,000

Center for Patient Safety at the End of Life

Lynn, Joanne

$150,000

University of Chicago

Creating High Reliability Healthcare Organizations

Small, Stephen D.

$480,010

University of Wisconsin, Madison

DCERPS on Systems Engineering-Wisconsin Patient Safety

Carayon, Pascale

$600,000

New York University

Patient Safety in Home Care

Kovner, Christine

$599,685

Hickner, John M.

$595,014

The American The American Academy of Academy of Family Physicians DCERPSFamily Physicians, PC Leawood, Kansas

Table 3: Examples of the developing centers of excellence in patient safety research funded by AHRQ in 2001 In 2005 Rand13 carried out an evaluation of all of the AHRQ initiatives and issued several reports. In one of these, Assessment of the National Patient Safety Initiative, they recommend that “continued funding support should be provided for DCERPS and other projects that are beginning to build patient safety research infrastructure in order to enable them to become self-sustaining.” 2.4.3

Centre for Research Excellence (Australia)14

In January 2004 the National Health and Medical Research Council in partnership with the Australian Council for Safety and Quality in Health Care called “expressions of interest” for the Centre for Research Excellence (CRE) in Patient Safety. Planned infrastructure funding for the CRE in Patient Safety was to be $400,000 (AUS) yearly for five years. The aims that were enunciated for this center were to: a) undertake high quality research in the field of patient safety

73

b) build capacity by developing the next generation of research leaders c) facilitate better practice through dissemination of knowledge d) engage stakeholders through national and international collaborations The CRE was established at Monash University in late 2005 and involves a range of universities and organisations from across Australia15. The CRE has garnered over $4.2 million in competitive research grants and has 3 post doctoral and 7 PhD students. 2.4.4 Research Centres for Patient Safety & Service Quality (England)16 The National Institute for Health Research has recently established two Research Centres for Patient Safety & Service Quality. These centres will bring together researchers from a wide range of backgrounds, including management and the social sciences with managers and clinicians investigate ways to improve the care of patients. The details are in Table 4. NHS Organisation St Mary's and Hammersmith NHS Trust

King's College Hospital NHS Trust

Academic Partner Imperial College London

Specialism Safety, quality, resilience and reliability of technology; effective use of information technology; the role of NHS managers and staff in enhancing patient safety.

King's College London

New and emerging health technologies; the organisation and management of health services and staff to reduce risk.

Table 4: Newly funded Research Centres for Patient Safety & Service Quality in England Funding for the two Centres will be £9.5m over five years and is intended to support: • research staff • communications, linkage and partnership costs involved in the translation, implementation and evaluation of innovative patient safety and service quality research in NHS practice • NHS service support costs of patient safety and service quality research • research training, leading to a higher degree by research (eg MPhil, MD, PhD), for staff.

3.

Summary

In this paper we first presented data that supports the need to build research capacity in patient safety. Then we presented a framework for patient safety research and from it defined several strategic directions for patient safety research development. The first two of theseactive dissemination about the importance, value, challenges and successes of patient safety research; and, the development of specific strategies need to be developed to attract capable students to safety research careers – are less program specific and more system wide

74

suggestions. The remaining five if present in a jurisdictional program may enhance capacity for patient safety research. These directions were: MD (multidisciplinary) IDM (Involve decision makers CPD (Continuing professional development) PR&T (Programmatic research and training) RN (Research Networks)

At all stages of patient safety research training there must be emphasis on the multidisciplinary approach. Practical aspects of training must involve multidisciplinary teams. There should be active involvement with those who manage and fund the system in the research enterprise. This will facilitate both the applied and spread aspects and will generate important questions for the basic researchers. Training must not be restricted to the pre and post graduate levels. It is important that the academy ensure the existence of continued professional development for patient safety scientists at all career stages. A potentially valuable strategy will be the (continued) funding of programmatic patient safety research teams which include specific training components and specific accountabilities for this capacity building work. Development of real and virtual patient safety research networks can be undertaken to encourage multicentre studies and cross fertilization on training.

Table 5: Strategic directions The paper then presented detail of some of the opportunities that are in place at this time. These were clearly complementary and all have the potential to contribute substantially to capacity building in patient safety research. For example fellowship programs are a tried and true method for developing clinician scientists. In addition to training new scientists they also integrate those trainees into established research teams. Mentoring relationships formed during fellowships often last for years after the fellowship has finished. Similarly graduate training has a long history of success and will build capacity. The examples presented all have a focus on bringing together students from diverse disciplinary backgrounds. Hopefully this will lead the graduates into interdisciplinary research in patient safety. The examples of programmatic funding for patient safety research, while expensive, appear to have the potential to meet all the strategic directions we have put forth. These programs can allow trainees to be mentored in an interdisciplinary research environment and to see productive interactions between the policy, management and research sectors. The success of all of these initiatives will of course depend on their support from the funders and the hosting organizations. Their development and sustainability will be further enhanced if this work is valued by the academy and is relevant to the policy and managerial sectors. The former need will require engagement of senior members of the academy while the latter need will be more likely to be met is those involved in policy and management are active partners in program development. Any agency considering developing such programs will need to consider these factors. The agency will need to ensure that appropriate career tack options are available for those who are trained in their programs.

75

The examples presented are all recently developed and so have little or no evaluation available. Each will have the possibility to contribute to the strategic directions shown in Table 5. Clearly some by design or by scope will have more opportunity to do this. In Table 6 we show the mandated or stated areas of contribution.

Examples Emergency Medicine Patient Safety Research Fellowship David Rippey Patient Safety Fellowship Award Imperial College London MSc in Quality and Safety Northwestern University MS program in Healthcare Quality and Patient Safety

MD ?

Strategic Priorities IDM CPD PR&T ? ? ?

RN ?

? Yes Yes

Yes ? Yes

? ? ?

? ? ?

? ? ?

University of Wisconsin-Madison Graduate Certificate in Patient Safety.

Yes

Yes

?

?

?

AHRQ centers of excellence in patient safety research AHRQ developing centers of excellence in patient safety research

Yes Yes

Yes Yes

Yes Yes

Yes ?

? ?

Centre for Research Excellence (Australia) Research Centres for Patient Safety & Service Quality (England)

Yes Yes

Yes Yes

? ?

Yes Yes

Yes Yes

Table 6: Possible contribution to strategic directions of the examples In any particular jurisdiction the type of initiatives which should be undertaken will depend on a myriad of factors including: • • •

The existing expertise in patient safety research. Are there enough experts to undertake an advanced “Centers of Excellence” program? How much investment will be made? We have indicated where possible the costs for each of the examples. Particular local needs. Is there a need to begin in say emergency care or palliative care now?

The challenge is clear. We need more capacity in patient safety research to enable us to develop a safer system for care delivery. Countries will address the issue of increased capacity for patient safety research in different ways. Hopefully we will learn from each other and so accelerate the agenda.

76

References 1. Kohn LT, Corrigan JM, Donaldson MS, eds. To Error Is Human. Building a Safer Health System. Institute of Medicine. Washington, DC: National Academy Press; 1999. 2. Cook RI. Lessons from the War on Cancer: The Need for Basic Research on Safety. Journal of Patient Safety 2005; 1(1):7-8. 3. Berwick DM, Calkins DR, C. McCannon CJ, Hackbarth AD. The 100 000 Lives Campaign: Setting a Goal and a Deadline for Improving Health Care Quality JAMA. 2006;295:324-327. 4. Canadian Patient Safety Institute. (2005) Safer Healthcare Now! http://www.saferhealthcarenow.ca/ (8 July 2007) 5. Emergency Medicine Patient Safety Foundation. http://www.empsf.org/ (8 July 2007) 6. Canadian Institutes of Health research (2007) Fellowship Priority Announcement: Patient Safety http://www.cihr-irsc.gc.ca/e/32756.html (8 July 2007) 7. Medicine, Imperial College London (2007) MSc in Quality and Safety in Healthcare http://www1.imperial.ac.uk/medicine/about/divisions/sora/teaching/postgraded/msc_quality_and _safety/ (8 July 2007) 8. Northwestern University (2006) Healthcare Quality and Patient Safety http://www.tgs.northwestern.edu/academics/schooloverview/feinberg/HQS/ (8 July 2007) 9. University of Wisconsin-Madison () Graduate Certificate in Patient Safety http://www.engr.wisc.edu/ie/current/patientsafety/patientsafetycurr.pdf (8 July 2007) 10. National Institutes of Health () RFA-HS-01-002: Centers of Excellence for Patient Safety Research and Practice http://grants.nih.gov/grants/guide/rfa-files/RFA-HS-01-002.html (8 July 2007) 11. Agency for Healthcare Research and Quality () Developing Innovative Approaches to Improving Patient Safety http://www.ahcpr.gov/qual/newgrants/dcerps.htm (8 July 2007) 12. Agency for Healthcare Research and Quality (2000) Developmental Centers for Evaluation & Research in Patient Safety http://grants.nih.gov/grants/guide/rfa-files/RFA-HS-01-007.html 13. Farley DO, Morton C, et al. Assessment of the AHRQ Patient Safety Initiative, RAND Corporation, 2005 http://www.rand.org/pubs/technical_reports/2005/RAND_TR203.sum.pdf (8 July 2007) 14. petam (2006) About the NHMRC Centre of Research Excellence in Patient Safety http://www.crepatientsafety.org.au/about/ (8 July 2007) 15. crepatientsafety (2006) NHMRC Centre of Research Excellence in Patient Safety http://www.crepatientsafety.org.au/ (8 July 2007)

77

16. Marinos Panayi (2006) NIHR - National Institute for Health http://www.nihr.ac.uk/infrastructure_research_centres_for_nhs_patient_safety_and_service_qua lity.aspx

78

Workshop discussion paper: Estimating the level of harm from healthcare in data poor environments? Research Agenda Track: Parallel Workshops Session 2 Tuesday 25th Sept, 2007.

79

Patient Safety Research shaping the European agenda 24-26th September 2007: Porto, Portugal

WORKSHOP DISCUSSION PAPER

Estimating the level of harm from healthcare in data poor environments Sisse Olsen

Abstract The epidemiology of harm has been extensively studied in the developed world but reliable information on harm from data poor regions is still relatively scarce. This paper consists of 3 main sections. Firstly it provides an overview of the literature pertaining to methods of estimating harm from healthcare in data poor environments with a particular focus on developing and transitional countries and in addition gives a brief overview of the strengths and weaknesses of each. Secondly, as this is an emerging area of research, the main areas which need further research efforts are identified and discussed in detail. These include retrospective record review for identification of adverse events, research methodologies based on interview with clinical staff to determine the level of harm from healthcare and methodologies using direct observation and focus groups to examine injection safety. The final part of this paper presents suggestions for mechanisms to further develop this are of research.

Dr Sisse Olsen Clinical Research Fellow, Clinical Safety Research Unit Imperial College London UK

80

Introduction The level of harm from healthcare has been extensively studied in developed countries since the early 1990’s(1-8). This wave of research was initiated by the publication of the Harvard Medical Practice Study (HMPS) in 1991(1;2). Although large scale epidemiological studies have been carried out based largely on the methodology of retrospective record review developed by the HMPS, several alternative methods exist. Information gained through incident reporting, routine hospital data, claims and complaints analysis and central national/regional audits or enquiries have all played a part in understanding the patterns and burden of harm from healthcare in resource rich countries. For resource poor regions however much of this data will not be routinely available and the level of detail and quality of information recorded in the medical case notes varies greatly and may not be sufficient to support traditional retrospective record review(9). This brings about the challenge to develop research strategies for investigating the level, pattern and causes of harm from healthcare in resource and/or data poor environments. This paper consists of 3 main sections. First it will provide an overview of the literature pertaining to methods of estimating harm from healthcare in data poor environments with a particular focus on developing and transitional countries. The aim will not be to rank methods but to provide a picture of where each may be more suitable and how these may be used in combination. Secondly, as this is an emerging area of research, the main areas which need further research efforts will be identified and discussed in detail and thirdly some mechanisms for undertaking such research will be suggested.

Overview of the literature to date There are a number of methods of studying errors and adverse events, each of which has evolved over time and been adapted to different contexts. Each of the methods has particular strengths and advantages, but also weaknesses and limitations. The choice of method should be based on the question one seeks to answer, taking into account the particular characteristics of each method as well as local data availability and expertise. Understanding different methods of studying harm Failing to understand that different methods have different purposes has led to considerable confusion and much fruitless debate over the years. For instance, the major retrospective record reviews have sometimes been criticised for not providing data on human factors and other issues not identified in medical records. In fact such studies are not intended to provide such information. Their primary purpose is to assess the nature and scale of harm, although recent review techniques also suggest that valuable information on cause and prevention can be extracted. In all cases the methodology of a study will depend on the questions being addressed, the resources available and the context of the study(10). Thomas and Petersen have classified methods of studying errors and adverse events into eight broad groups and reviewed the respective advantages and disadvantages of each method(11). Table 1 below summarises the main types of studies of errors and adverse events, and their respective advantages and limitations. The language and 81

content of the table has been adjusted from Thomas and Petersen’s original source version and is taken from Vincent’s 2006 book Patient Safety. Methods vary in several respects including relying on different data such as medical records, observations, claims data, voluntary reports etc. Some methods focus on single cases or small numbers of cases with particular characteristics, such as claims, while others attempt to randomly sample a defined population. Some are oriented towards detecting incidence of errors and adverse events, while others address their causes and contributory factors. Thomas and Petersen suggest that the methods can be placed along a continuum with active clinical surveillance of specific types of adverse event (e.g. surgical complications) being the ideal method for assessing incidence, and methods such as claims analysis and morbidity and mortality meetings being more oriented towards causes. There is no perfect way of estimating the incidence of adverse events; each gives a partial picture. Record review is comprehensive and systematic, but by definition is restricted to the data recorded in the medical record. Reporting systems are strongly dependent on the willingness of staff to report and are a very imperfect reflection of the underlying rate of errors or adverse events though they have other uses(10).

82

Table 1

Methods of studying errors and adverse, (From Vincent 2006, adapted from Thomas and Petersen, 2003)

Study Method

Advantages

Disadvantages

Morbidity and mortality conferences and autopsy

Can suggest contributory factors Familiar to health care providers

Hindsight bias Reporting bias Focused on diagnostic errors Infrequently use

Case analysis/ Root cause analysis

Can suggest contributory Structured systems approach Includes recent data from interviews

Hindsight bias Tends to focus on severe events Insufficiently standardized in practice

Claims analysis

Provides multiple perspectives (patients, providers, lawyers)

Hindsight bias Reporting bias Non-standardised source of data

Error reporting systems

Provide multiple perspectives over time Can be a part of routine operations

Reporting bias Hindsight bias

Administrative data analysis

Uses readily available data Inexpensive

May rely upon incomplete and inaccurate data The data are divorced from clinical context

Record review/chart review

Uses readily available data Commonly used

Judgements about adverse events not reliable Medical records are incomplete Hindsight bias

Review of electronic medical record

Inexpensive after initial investment Monitors in real time Integrates multiple data sources

Susceptible to programming and/or data entry errors Expensive to implement

Observation of patient care

Potentially accurate and precise Provides data otherwise unavailable Detects more active errors than other methods

Time consuming and expensive Difficult to train reliable observers Potential concerns about confidentiality Possible to be overwhelmed with information

Active clinical surveillance

Potentially accurate and precise for adverse events

Time consuming and expensive

83

Search methodology The WHO World Alliance for Patient Safety commissioned an overview of available methods of estimating harm carried out by Dr Michel and published in 2003: Strengths and weaknesses of available methods for assessing the nature and scale of harm caused by the healthcare system; literature review(9). This document specifically addressed the question of assessment methods used in developing countries and although evidence was found of use of a wide range of research methodologies the body of literature in this area was limited. Building on the findings of this review, the literature search for this paper as was built as an update of Dr Michel’s original search strategy to identify papers published after 2003. The search terms remained the same, however due to the large number of references retrieved some additional search terms were restricted to the title only. The full search strategy can be found in appendix 1. Search strategy 1 retrieved 929 references but only 10 of these related to data and studies from developing/transitional countries or data poor environments. Search strategy 2 found a total of 1429 titles. Review of titles led to full review of 179 abstracts and of these 51 full papers were retrieved. 32 contained suitable studies. The strengths and weakness of each research methodology for detecting and analysing harm from health care has been extensively described by Dr Michel (9). Only a brief summary of these will be presented here. Types of studies Record review based studies The majority of studies identified were ad hoc studies based on epidemiological designs using record review or systematic prospective data collection based on the medical records or prescription charts in combination with further demographic information collected from the patients. These covered a range of topics but the majority had a limited focus such as prescription/medication errors, maternal and neonatal complications following delivery or outcome and compilations following surgical procedures (12;13;13-28). Evidence was found of three studies looking for adverse events using a retrospective record review methodology based on the HMPS. All of these were in single institutions and from large tertiary referral hospitals(29-31). The number of record review based methodologies and the range of information collected by this method suggest an improved effectiveness in capturing the extend of harm and also, at least in larger health facilities availability, sufficient and reliable information contained in the case notes. There were no studies identified which specifically addressed the reliability and accuracy of the information contained in the medical records in this setting. Only two examples were identified of record or medication chart review were used on a large scale covering several institutions (12;14). The suitability of retrospective record

84

review to provide large scale epidemiological studies depends largely on the organisation of and information contained in the medical record. Hence the suitability of this methodology will vary between regions and countries. It is clear that the methodology is not of use in smaller poorly resourced health facilities were both the organisation of and information contained in the medical notes is limited. Interview or questionnaire based studies There were only a small number of studies based on staff interviews or questionnaires. Two concerned prescribing and medication errors(32;33). A further study used a combination of data collection from the case notes and interviews with patients, relatives and healthcare professionals to gather data to determine the level and causes of adverse events associated with maternity care(34). This combination of data sources is potentially very useful in environments where there is a scarcity of information in the medical notes and it has already been shown also to be of value in developed countries’ health care facilities (8). Studies of direct observation Studies of direct observation of care delivery described assessment of safe injection practices in health facilities in Swaziland, adverse drug reactions in hospitalized children in Brazil and the use of direct clinician observation and vignettes for evaluation of the quality of health services (35-37). The utilisation of direct observation was different in each of these studies but the methodology has the potential to detect both the level and causes of harm in data poor environments as it is not dependent on previously collected information. Instead it relies on adequately trained skilled observers and can be deployed in a range of settings. In addition the cost, by international standards, of the time of skilled observers is likely to be relatively low in resource poor regions. Studies of incident reporting data A new development in the literature over the past 4 years is the emergence of publications based on data collected by incident reporting systems in particularly transitional countries such as India (38), Pakistan (39) and Brazil (40). Although the number of reports in this area is still low, it is likely to increase over the next few years as more data collected is analysed and the implications understood. The studies identified in this search related to vaccination and drug related incidents. Effectiveness of incident reporting systems in capturing the extent of harm is limited and it is well recognised that underreporting is widespread and the reliability of the data is moderate(41;42). The utilisation of the data collected by this method is key and should be an important consideration before embarking on the cost of initiating such systems in resource poor regions as it has proved a challenge to the effectiveness of reporting systems in the developed world (43). Studies of external audits and confidential enquiries There was no evidence of publication of data from large scale external audits or confidential enquiries identified in the literature from 2004-2007. This type of data is reliant on a large central organisation for data collation and analysis as well as widespread local engagement to ensure reporting of data. This is costly and although

85

systems such as NCEPOD in the UK and JCAHO in the US have had an impact on the provision of safer healthcare, it is questionable if the cost associated with such systems is justified in resource poor environments. Two papers concerning external audits of individual units were identified. Both reported a positive impact on the quality of care delivered locally(44;45). Studies of claims and complaints Studies investigating information gained from claims and complaints was found originating from Taiwan(46) and Turkey(47;48), both relatively well developed transitional economies. Claims for malpractice in healthcare are of less relevance in poorer environments. Although data from claims and complaints, where available, can yield information on the causes of harm this method is not suitable or estimating the scale of harm. The information is often several years out of date due to the time lag involved in bringing cases to court and the cost of analysing large numbers of claims is high. This methodology is not suited for the majority of data poor areas and its use in developed countries has largely been superseded by other methods. Studies of information technology, electronic medical records and routine administrative data No studies of the use of information technology, electronic medical records or routine administrative data were found as these require a high level of data availability and investment. Studies of autopsy data Autopsies are being carried out increasing rarely in many developed countries (49-52) but this trend does not seem to be replicated in the developing world. The facilities for autopsy however are mainly concentrated in larger cities as many rural and mission hospitals often do not have the facilities for autopsy or a pathologist available. Of the studies of autopsy findings identified in this search, two explored the concordance between clinical and post mortem diagnosis(53;54) and one study was specifically designed to explore the true cause of deaths and the occurrence of adverse events following cardiac surgery(55). Discordance between pre and post-morbid diagnosis is not necessarily an indication of misdiagnosis or medical error but may be a reflection of atypical symptoms or limited diagnostic testing facilities. The availability of data is variable and the reliability depend on the underlying autopsy rate, facilities available for histopathological testing, the experience of the pathologist and availability of final reports. Methodologies based on autopsy findings are not suitable for large scale studies. Studies of Mortality and Morbidity reports Three studies were identified based on data obtained from morbidity and mortality meetings in the clinical areas of obstetrics(56;57) and paediatric care(58). The use of morbidity and mortality (M&M) meetings in determining the scale and causes of harm is dependent on the selection of cases, varying definitions of adverse events and the style of analysis of cases (root cause or ad hoc). The lack of a structured method of analysis in most M&M meetings brings the validity and reliability of the data into question. The

86

practice of M&M meetings also vary between specialties, facilities and countries making it unsuitable for large scale studies of harm from healthcare. Strengths and Weaknesses of different study methods A summary of the strengths and weaknesses of individual methods in the context of data poor environments can be found in table 2 (updated and adapted from Dr Michel’s 2003 report) Table 2 Summary and rating of methods of study of harm from healthcare

Criteria

Effectiveness in capturing the extent of harm Availability of reliable data Suitability for large- scale studies Suitability for small, repeated studies Costs*

Ad hoc studies based on epidemiological designs and systematic data collection Review Studies Direct of based on observation medical interviews records with healthcare providers 3 3-4 3-4

Incident reporting systems

External audit and confidential inquiries

2-3

3-4

2

3-4

2-3

2-3

2-3

Methods based on reporting

Analysis of routinely collected and existing data Autopsy Mortality and reports morbidity conferences

2

2-3

2-3

3-4

3-4

4 for harm assessment, 2-3 for cause analysis 1

1-2

2-3

1

2-3

3

3-4

2-3

3

3-4

2-3

3-4

2

3-4

2-3

2-3

2-3

3-4

1

3-4

3-4

3-4

1-2

2-3

2-3

1-2

1

1-2

3-4 3-4 3-4 Effectiveness 3-4 in influencing policy** 3-4 3-4 3 Effectiveness 2-3 in influencing hospital and local safety procedures and outcomes 3 4 2-3 Synergy with 3-4 other domains of quality of care (Rating scale from 1 to 4, the most favourable level being 4)

87

The available methods for assessing the nature and scale of harm caused by health care systems have widely differing purposes, strengths and limitations, and should be considered as complementing each other by providing different levels of qualitative and quantitative information. Selection of a particular method for study must take into consideration the environment in which it is being carried out in terms of resources and level of data available as well as the purpose of the study. Their suitability, validity and reliability may vary widely, depending on their objectives and on the context. An understanding of the reliability and validity of the data obtained is also crucial for using the data to influence policy and creating change. This review has limitations. The available literature varies in quality and quantity and there are still relatively few studies of methodologies for studying harm from healthcare originating from developing countries. The majority of the studies originate from tertiary referral centres where larger amount of routine data is available. There is still a gab in the body of research evaluating methods for studying harm in very data poor environments. Avenues for developing this research will be discussed in the remaining parts of this paper.

Scope for further research More data are needed on the reliability and validity of methods, especially methods used in very data poor settings. When commenting on research methodologies for assessing harm the following must be born in mind: •

Rapid assessment methods are needed as data collection is not an end in itself but a necessary prelude to effective action. Whilst wide ranging changes should be based on available data, efforts to collect data should not delay action on immediate and obvious local problems.

•

The relevance of the method depends on the goal of the study and the methodology should suit the aim of the data collection in terms of examining active and latent errors (11)

•

The selection of the methodology depends on the availability of data. Whilst some methodologies traditionally used in developed countries requiring good organisation and baseline collection of data such as record review and studies based on incidence reporting may be suitable for use in tertiary centres in the developing world, more work needs to be done exploring methods independent of these such as direct observation and studies based on structured interviews of health care professionals.

With these points in mind the discussion of this paper will concentrate on ongoing developments and suggestions for further research in 3 main areas: The use retrospective record review in developing world, direct observation of delivery of care and interview based methodologies. Retrospective record review 88

Retrospective record review has been the most important methodology in terms of studies seeking to estimate the level of harm from healthcare in the developed world since the publication of the HMPS in 1991(1;2). Studies based on this methodology have been replicated in a large number of countries since then although not all have been fully reported in the international literature. This methodology has also been used in well developed transitional countries such as Brazil. The suitability of retrospective record review for large scale epidemiological studies depends largely on the organisation of and the information contained in the medical record. Hence the suitability of this methodology will vary between regions and countries. There are no national or large scale studies from poorer countries assessing the scale of harm from healthcare based on retrospective record review despite the proliferation of the methodology in the developed world. With this in mind the World Alliance for Patient Safety in 2004 supported efforts to set up studies in developing and transitional countries based on retrospective record review to explore the scale of the problem of harm from healthcare and to evaluate the usefulness of the retrospective record review in this environment. This work has been led Dr Ross Wilson (Australia), Dr Philippe Michel (France), Charles Vincent (UK) and Dr Sisse Olsen (UK) . Studies are currently being completed in several countries in the WHO EMRO and AFRO regions. In addition further studies are being undertaken in Latin America supervised by members of the team behind the national Spanish study of adverse events. Although data from these studies are not yet available some interesting findings are already clear. In the countries, with which the author has worked from the EMRO/AFRO region, the experience has revealed some surprising findings. In the initial planning stages of the project there were concerns with regards to the organisation of medical records, the availability of the medical records within individual hospitals, the language within the medical records, the willingness of hospitals to participate in the study and the overall feasibility of retrospective record review studies in this type of environment. In all countries however it was possible to identify hospitals with sufficient organisation of the medical records to generate random lists of admissions for review. There was much less resistance to participation in the project than anticipated from local hospital management. Although there were deficiencies in many medical records this was overcome by oversampling in order to get sufficient numbers of record reviewed in each institution. In most countries medical records in the major city hospitals were adequate for retrospective review of adverse events and the organisation of the records allowed access to these by researchers. Some technical difficulties were experienced during the data collection with electronic transfer of the results for central analysis but these have now largely been overcome. The overall reliability and validity of the data remains to be seen but the experiences from these studies have been testament to the fact that retrospective record review does have a place outside well developed counties at least within the main flagship healthcare facilities. More results of from this study will hopefully be available for discussion during the conference. However it is clear that the methodology is not suitable in smaller poorly resourced health facilities where both the organisation of and information contained in

89

the medical notes is limited. To investigate the level and causes of adverse events in such environments there is a need for other methods independent of pre-recorded information such as staff interviews and direct observation of care delivery. The research questions which remain a priority in assessing the further use and value of retrospective record review in data-poor environments are: •

To assess the feasibility of the data collection

•

To estimate the workload involved

•

To evaluate the acceptability to participating professionals and institutions

•

To identify any unexpected obstacles (cultural or organizational)

•

To assess how this methodology may stimulate action and influence policy developments for improving patient safety

•

To further refine the data collection tools and assess their suitability in this environment.

•

To further address the reliability and validity of retrospective record review in this context

Interview based methodologies Methodologies based on direct data collection from local healthcare workers have tremendous potential in the developing world and their use remains to be fully explored. Studies in the developed world has given an insight into the potential strengths of such methodologies but due to the large amount of available existing data such methodologies have not been explored more widely in this setting. The two most important examples of such methodologies is the 1997 study from Boston(US) by Andrews et al(59) and the methodology developed for the French national survey described by Michel at al in 2004 (8). In the US study data was collected by trained external observers who collected information from local healthcare professionals by attending ward-rounds, operating theatre sessions, morbidity and mortality meetings and through general discussions with staff. The AE rate discovered in this prestige surgical unit was much higher at 17.7% than those previously suggested by the US record review studies (3.2-2.9%) The method was originally developed in France by the French national working group in charge of defining the methodology of a national study of the incidence of adverse events. The mainstay of the methodology moved away from traditional medical record review to data collection primarily based on interview with hospital staff. This has the advantage, particularly in data poor environments, of losing the reliance on pre-recorded information about care. It has also been suggested that such methods may increase validity of AE assessment, better investigate the main contributory factors and also identify types of AE rarely 90

recorded such as pain (8). The resulting method was strongly based on the record review method: external reviewers collect information using a two-stage methodology for data collection. Stage 1 is a screening stage conducted by a research nurse charged with selection patients with positive screening criteria for further in-depth review by a medical/physician reviewer. The second stage review is carried out by a senior physician who assesses the presence of adverse events, their preventability and the main contributory factors. The major difference is the primary source of information. For stage 1 this is the ward head nurse, and in stage 2 the information is collected by interview with the doctor in charge of the patient. In addition in the French study all additional existing documentation could be consulted, particularly further questioning of the ward nurses and any information recorded in the medical records. When using this methodology the patient sample can be selected by the patients present at the time of review, either in a cross-sectional way (assessment of the patients who are hospitalised the day of the nurse review) or in a longitudinal way (for example, all the patients hospitalised during a given period (7 days in the French study)). In the crosssectional approach, all adverse events present (or under treatment) on the day of assessment are taken into account, and a prevalence rate is calculated. The effectiveness, reliability, and acceptability of estimating rates of adverse events and rates of preventable adverse events were tested during a pilot study reported in full by Michel et al in 2004(8). Reliability of the results based on staff interview was deemed to be good and likely to be better than that of record review. The prospective and retrospective methods identified similar numbers of medical and surgical AE but the prospective method identified more preventable cases, in addition this interview based methods showed higher face validity. Since the publication, development of the methodology particularly for use in data-poor environments has continued. This has included attempts to improve the staff interview methods and combine it with systematic use of the medical records, even if they contained limited information. Although pilot testing of this method has commenced within the WHO EMRO-AFRO patient safety project much more research into the use, validity and reliability of the methodology is urgently needed particularly as it is thought to be the only methodology which can efficiently provide epidemiological estimates of all AE in data-poor environments. As pilot testing of this methodology has only just commenced more information on the progress, challenges and success of its use will be available for discussion during the September conference. The main research priorities/questions requiring assessment in this area are itemised below: •

To test the feasibility of the data collection

•

To estimate the workload involved

91

•

To test the acceptability of the methodology to participating professionals and institutions

•

To get information about the most relevant process - The degree of information we can get from different staff groups i.e. nurses/doctors - The accuracy of the information we can get from the nurses / doctors / records

•

To identify any unexpected obstacles (cultural or organizational)

•

To assess how this methodology may stimulate action and influence policy developments in improving patient safety

•

To further refine the data collection tools and assess their suitability and reliability in this environment.

It is expected that the EMRO/AFRO pilot testing will provide an initial insight into the issues but much more research in this area is needed given the potential of this methodology.

Observation based studies Injection safety is an area of care delivery which is particularly suitable for assessment by direct observation. The standards of safe care are well understood and can be assessed using staff trained in observation and using a relatively simple checklist. To further support the findings of direct observation, additional information can be gained from focus groups or interview/questionnaires with individual healthcare workers of their practice. The WHO has provided guidelines for the conduct of focus groups addressing injection safety. These were initially designed for a major country survey, but the guidelines conducting focus groups are applicable to a project of any size(60). An example of a study combining direct observation and data collection directly from healthcare staff is the study by Daly et al of the safety of vaccines in Swaziland(35). Injection safety is a major health issue in many developed countries and it is estimated that 550 million injections are administered annually in the Expanded Programme on Immunisation in developing countries world wide(35). The potential for associated complications and the transmission of infectious diseases remains a particular concern. Further research into direct observation of injection practices has been suggested and limited pilot testing of a simple methodology is due to start in the EMRO/AFRO group of countries. This methodology is outlined below. Several sites will be selected within each country to represent the types of healthcare facilities offered. At each site two focus groups will be conducted: (i) focus group with patients receiving injections; (ii) focus group with providers of injections. A minimum of 25 injections should be observed per site. The observation of injection procedure will be structured and recorded according to existing WHO guidelines and all forms and instructions are freely available from WHO(60). For each region or country assessed a 92

short report should be prepared summarising the findings from focus groups, reporting data on observations and providing recommendations for improving injection safety practice. The report should also offer reflections on the implications of the findings and provide a stimulus for safety initiatives. The main research questions which need clarification in developing this methodology are: • To assess the value of direct observation and focus groups as means of assessing patient safety both in the context of injection safety and in general •

To test the feasibility of data collection using this method

•

To assess the workload involved

•

To test the acceptability from participating professionals and institutions

•

To assess how this methodology may stimulate action and influence policy developments in improving patient safety

Feasibility and suggestions for mechanisms for further research Understanding of the level and causes of harm from healthcare in data and/or resource poor environments is urgently needed. Improvement of patient safety in resource poor regions is at least as important as in the developed world to ensure minimal wastage of limited resources. Despite this there are only limited data available at present and there is an important need to further develop and understand research methodologies suitable for this environment in order to prioritise research efforts and ensure reliable data. Some methodologies which deserve further attention have been suggested above but this is not intended to be an exhaustive list. In order to maximise the efforts and resources for patient safety research in resource poor countries, it is important to further develop some of the research networks and links which already exists such as programs within the World Alliance for Patient Safety. A model which should be encouraged is the development of country networks for research where countries with previous research experience can support others wishing to replicate or further develop methodologies within their context. Such sharing of learning is important for both resource rich and poorer countries which can have equally gain from each others experiences. A valuable part of the conference discussion has been the opportunities given to establish such contacts and collaboratives. Outcomes of the conference session The conference session on 25th September 2007 on “Estimating the level of harm from healthcare in data poor environment” was well attended by delegated from more than 15 nations both from the developed and developing world and gave rise to fruitful debate of the tools needed to enhance research in this area. It was acknowledged from all counties represented bith richer and poorer nations that there were challenges to using traditional data collection tools based on hospital data sets and medical records. There were calls for development of validated research tools to explore different types of data 93

such as observation of care delivery, interview with healthcare providers and patients and information contained in narratives of adverse events. It was suggested that a standardised portfolio of such tools could be available to patient safety researcher to avoid expensive and unnecessary duplication of tools development. However it is important to ensure that standardised tools are validated locally within different cultures and settings so that the strengths of weaknesses of individual tools can be assessed in different contexts. To ensure progression of this agenda the session raised the important suggesting of collaboration between the north and south to encourage and promote patient safety research where resources are scarce. Furthermore patient safety research estimating the level of harm form healthcare the developing word will benefit from a shift of focus from instead of thinking of these as data poor, to focusing on development of research methodologies which utilities the sorts of data which are universally available where healthcare is provided such as observation o care delivery and interview of healthcare staff and patients so that these may be used in both developing and developed countries.

94

Reference List (1) Brennan TA, Leape LL, Laird NM, Hebert L, Localio AR, Lawthers AG, et al. Incidence of adverse events and negligence in hospitalized patients. Results of the Harvard Medical Practice Study I. N Engl J Med 1991 Feb 7;324(6):370-6. (2) Leape LL, Brennan TA, Laird N, Lawthers AG, Localio AR, Barnes BA, et al. The nature of adverse events in hospitalized patients. Results of the Harvard Medical Practice Study II. N Engl J Med 1991 Feb 7;324(6):377-84. (3) Wilson RM, Runciman WB, Gibberd RW, Harrison BT, Newby L, Hamilton JD. The Quality in Australian Health Care Study. Med J Aust 1995 Nov 6;163(9):45871. (4) Vincent C, Neale G, Woloshynowych M. Adverse events in British hospitals: preliminary retrospective record review. BMJ 2001 Mar 3;322(7285):517-9. (5) Davis P, Lay-Yee R, Briant R, Ali W, Scott A, Schug S. Adverse events in New Zealand public hospitals I: occurrence and impact. N Z Med J 2002 Dec 13;115(1167):U271. (6) Davis P, Lay-Yee R, Briant R, Ali W, Scott A, Schug S. Adverse events in New Zealand public hospitals II: preventability and clinical context. N Z Med J 2003 Oct 10;116(1183):U624. (7) Baker GR, Norton PG, Flintoft V, Blais R, Brown A, Cox J, et al. The Canadian Adverse Events Study: the incidence of adverse events among hospital patients in Canada. CMAJ 2004 May 25;170(11):1678-86. (8) Michel P, Quenon JL, de Sarasqueta AM, Scemama O. Comparison of three methods for estimating rates of adverse events and rates of preventable adverse events in acute care hospitals. BMJ 2004 Jan 24;328(7433):199. (9) Philippe Michel. Strengths and weaknesses of available methods for assessing the nature and scale of harm caused by the health system: literature review. 2003 Dec 1. (10) Vincent C. Patient Safety. Edinburgh: Churchill Livingstone; 2006. (11) Thomas EJ, Petersen LA. Measuring errors and adverse events in health care. J Gen Intern Med 2003 Jan;18(1):61-7. (12) Al Khaja KA, Al Ansari TM, Damanhori AH, Sequeira RP. Evaluation of drug utilization and prescribing errors in infants: a primary care prescription-based study. Health Policy 2007 May;81(2-3):350-7. (13) Anwari JS, Butt AA, Al-Dar MA. Obstetric admissions to the intensive care unit. Saudi Med J 2004 Oct;25(10):1394-9. (14) Danchaivijitr S, Assanasen S, Apisarnthanarak A, Judaeng T, Pumsuwan V. Effect of an education program on the prevention of ventilator-associated pneumonia: A multicenter study. J Med Assoc Thai 2005 Dec;88 Suppl 10:S36S41. (15) Daniel E, Ffytche TJ, Sundar Rao PS, Kempen JH, ener-West M, Courtright P. Incidence of ocular morbidity among multibacillary leprosy patients during a 2 year course of multidrug therapy. Br J Ophthalmol 2006 May;90(5):568-73.

95

(16) Ekere AU, Yellowe BE, Umune S. Mortality patterns in the accident and emergency department of an urban hospital in Nigeria. Niger J Clin Pract 2005 Jun;8(9):14-8. (17) Ersumo T, Meskel Y, Kotisso B. Perforated peptic ulcer in Tikur Anbessa Hospital: a review of 74 cases. Ethiop Med J 2005 Jan;43(1):9-13. (18) Hailu D, Worku B. Birth trauma among live born term neonates at a referral hospital in Addis Ababa, Ethiopia. Ethiop Med J 2006 Jul;44(3):231-6. (19) Hesse IF, Mensah A, Asante DK, Lartey M, Neequaye A. Adult tetanus in Accra, why the high mortality? An audit of clinical management of tetanus. West Afr J Med 2005 Apr;24(2):157-61. (20) Jaimes F, De La RG, Gomez E, Munera P, Ramirez J, Castrillon S. Incidence and risk factors for ventilator-associated pneumonia in a developing country: where is the difference? Respir Med 2007 Apr;101(4):762-7. (21) Mousavi S, Nobahar M, Vafaei AA, Malek M, Babaei M, Malek F, et al. Complications of upper gastrointestinal endoscopy in unsedated patients. Indian J Gastroenterol 2004 Sep;23(5):193. (22) Mteta KA, Mbwambo J, Mvungi M. Iatrogenic ureteric and bladder injuries in obstetric and gynaecologic surgeries. East Afr Med J 2006 Feb;83(2):79-85. (23) Nwofor AM, Ekwunife CN. Tube thoracostomy in the management of pleural fluid collections. Niger J Clin Pract 2006 Jun;9(1):77-80. (24) Oliveira AC, Carvalho DV. Postdischarge surveillance: the impact on surgical site infection incidence in a Brazilian university hospital. Am J Infect Control 2004 Oct;32(6):358-61. (25) Padua CA, Cesar CC, Bonolo PF, Acurcio FA, Guimaraes MD. High incidence of adverse reactions to initial antiretroviral therapy in Brazil. Braz J Med Biol Res 2006 Apr;39(4):495-505. (26) Passarelli MC, Jacob-Filho W, Figueras A. Adverse drug reactions in an elderly hospitalised population: inappropriate prescription is a leading cause. Drugs Aging 2005;22(9):767-77. (27) Trotman H, Barton M. The impact of the establishment of a neonatal intensive care unit on the outcome of very low birthweight infants at the University Hospital of the West Indies. West Indian Med J 2005 Oct;54(5):297-301. (28) Yousif E, Ahmed AM, Abdalla ME, Abdelgadir MA. Deficiencies in medical prescriptions in a Sudanese hospital. East Mediterr Health J 2006 Nov;12(6):9158. (29) Herrera-Kiengelher L, Chi-Lem G, Baez-Saldana R, Torre-Bouscoulet L, Regalado-Pineda J, Lopez-Cervantes M, et al. Frequency and correlates of adverse events in a respiratory diseases hospital in Mexico city. Chest 2005 Dec;128(6):3900-5. (30) Matsaseng T, Moodley J. Adverse events in gynaecology at King Edward VIII Hospital, Durban, South Africa. J Obstet Gynaecol 2005 Oct;25(7):676-80. (31) ud-Gallotti R, Dutilh Novaes HM, Lorenzi MC, Eluf-Neto J, Namie OM, Tadeu V, I. Adverse events and death in stroke patients admitted to the emergency

96

department of a tertiary university hospital. Eur J Emerg Med 2005 Apr;12(2):6371. (32) Gordon PC, Llewellyn RL, James MF. Drug administration errors by South African anaesthetists--a survey. S Afr Med J 2006 Jul;96(7):630-2. (33) Miasso AI, Grou CR, De Cassiani SH, de Silva AE, Fakih FT. [Medication errors: types, causes and measures taken in four Brazilian hospitals]. Rev Esc Enferm USP 2006 Dec;40(4):524-32. (34) Okong P, Byamugisha J, Mirembe F, Byaruhanga R, Bergstrom S. Audit of severe maternal morbidity in Uganda--implications for quality of obstetric care. Acta Obstet Gynecol Scand 2006;85(7):797-804. (35) Daly AD, Nxumalo MP, Biellik RJ. An assessment of safe injection practices in health facilities in Swaziland. S Afr Med J 2004 Mar;94(3):194-7. (36) dos Santos DB, Coelho HL. Adverse drug reactions in hospitalized children in Fortaleza, Brazil. Pharmacoepidemiol Drug Saf 2006 Sep;15(9):635-40. (37) Leonard KL, Masatu MC. The use of direct clinician observation and vignettes for health services quality evaluation in developing countries. Soc Sci Med 2005 Nov;61(9):1944-51. (38) Jose J, Rao PG. Pattern of adverse drug reactions notified by spontaneous reporting in an Indian tertiary care teaching hospital. Pharmacol Res 2006 Sep;54(3):226-33. (39) Khan FA, Hoda MQ. Drug related critical incidents. Anaesthesia 2005 Jan;60(1):48-52. (40) Fernandes GC, Camacho LA, Sa CM. Surveillance system of vaccine adverse events and local data analysis--the experience in a middle-sized city in Brazil, 1999-2001. Vaccine 2005 Mar 18;23(17-18):2349-53. (41) Olsen S, Neale G, Schwab K, Psaila B, Patel T, Chapman EJ, et al. Hospital staff should use more than one method to detect adverse events and potential adverse events: incident reporting, pharmacist surveillance and local real-time record review may all have a place. Qual Saf Health Care 2007 Feb;16(1):40-4. (42) Sari AB, Sheldon TA, Cracknell A, Turnbull A. Sensitivity of routine system for reporting patient safety incidents in an NHS hospital: retrospective patient case note review. BMJ 2007 Jan 13;334(7584):79. (43) Sari AB, Sheldon TA, Cracknell A, Turnbull A. Sensitivity of routine system for reporting patient safety incidents in an NHS hospital: retrospective patient case note review. BMJ 2007 Jan 13;334(7584):79. (44) Olowonyo T, Oshin S, Obasanjo-Bello I. Registering in a health facility for delivery protects against maternal mortality in a developing country setting. J Obstet Gynaecol 2005 Oct;25(7):638-41. (45) Shakespeare TP, Back MF, Lu JJ, Lee KM, Mukherjee RK. External audit of clinical practice and medical decision making in a new Asian oncology center: results and implications for both developing and developed nations. Int J Radiat Oncol Biol Phys 2006 Mar 1;64(3):941-7.

97

(46) Tsai WC, Kung PT, Chiang YJ. Relationship between malpractice claims and medical care quality. Int J Health Care Qual Assur Inc Leadersh Health Serv 2004;17(7):394-400. (47) Tumer AR, Dener C. Evaluation of surgical malpractice in Turkey. Leg Med (Tokyo) 2006 Jan;8(1):11-5. (48) Ozdemir MH, Cekin N, Can IO, Hilal A. Malpractice and system of expertise in anaesthetic procedures in Turkey. Forensic Sci Int 2005 Oct 29;153(2-3):161-7. (49) Petri CN. Decrease in the frequency of autopsies in Denmark after the introduction of a new autopsy act. Qual Assur Health Care 1993 Dec;5(4):315-8. (50) Eriksson L, Sundstrom C. Decreasing autopsy rate in Sweden reflects changing attitudes among clinicians. Qual Assur Health Care 1993 Dec;5(4):319-23. (51) Hill RB. The current status of autopsies in medical care in the USA. Qual Assur Health Care 1993 Dec;5(4):309-13. (52) Shojania KG, Burton EC, McDonald KM, Goldman L. Changes in rates of autopsy-detected diagnostic errors over time: a systematic review. JAMA 2003 Jun 4;289(21):2849-56. (53) Daramola AO, Elesha SO, Banjo AA. Medical audit of maternal deaths in the Lagos University Teaching Hospital, Nigeria. East Afr Med J 2005 Jun;82(6):2859. (54) Kusum DJ, Jaya RD, Gayathri PA. Medical autopsy: whose gain is it? An audit. Indian J Pathol Microbiol 2006 Apr;49(2):188-92. (55) Zhang FQ. [Analysis of autopsy and clinical findings on medical malpractice cases after heart operation]. Fa Yi Xue Za Zhi 2007 Feb 15;23(1):46-8. (56) Dumont A, Gaye A, de BL, Chaillet N, Landry A, Delage J, et al. Facility-based maternal death reviews: effects on maternal mortality in a district hospital in Senegal. Bull World Health Organ 2006 Mar;84(3):218-24. (57) Filippi V, Brugha R, Browne E, Gohou V, Bacci A, De B, V, et al. Obstetric audit in resource-poor settings: lessons from a multi-country project auditing 'near miss' obstetrical emergencies. Health Policy Plan 2004 Jan;19(1):57-66. (58) Krug A, Pattinson RC, Power DJ. Saving children--an audit system to assess under-5 health care. S Afr Med J 2004 Mar;94(3):198-202. (59) Andrews LB, Stocking C, Krizek T, Gottlieb L, Krizek C, Vargish T, et al. An alternative strategy for studying adverse events in medical care. Lancet 1997 Feb 1;349(9048):309-13. (60) injection safety WHO. 2007. Ref Type: Internet Communication

98

Appendix 1 - Results of literature search The main Medline search strategy (search strategy No. 1) was as follows: (Medical errors [title] OR Medication errors[title] OR Diagnostic errors[title] OR Iatrogenic disease[title] OR Malpractice[title] OR Medica* error* [title] OR Sentinel event*[title] OR Adverse event*[title] OR Human error* [title] OR Sentinel event*[title]) AND (Adverse drug reaction reporting systems OR Sentinel surveillance OR (Retrospective studies AND Record*) OR (Prospective studies AND Observation* [title]) OR Data collection OR Record review [title] OR Risk management OR Safety management OR Medical audit OR Audit OR Mandatory reporting OR Autopsy OR Reporting system [title] OR Morbidity mortality conference OR Mortality morbidity committee *[title]) The results from this search were only considered further if the studies originated from developing/transitional countries or data poor environments In addition, a broader search was performed for the developing countries (search strategy No. 2): (Adverse drug reaction OR reporting system OR risk management OR Safety management OR Medical audit) AND (Developing countries OR Africa OR India OR Brazil) Below is a list of all included studies and papers organised into the methodology employed. Review of medical records - Selective audits K. A. Al Khaja, T. M. Al Ansari, A. H. Damanhori, and R. P. Sequeira. Evaluation of drug utilization and prescribing errors in infants: a primary care prescription-based study. Health Policy 81 (2-3):350-357, 2007. medication/prescription r/v only J. S. Anwari, A. A. Butt, and M. A. Al-Dar. Obstetric admissions to the intensive care unit. Saudi.Med J 25 (10):1394-1399, 2004. S. Danchaivijitr, S. Assanasen, A. Apisarnthanarak, T. Judaeng, and V. Pumsuwan. Effect of an education program on the prevention of ventilator-associated pneumonia: A multicenter study. J Med Assoc.Thai. 88 Suppl 10:S36-S41, 2005. E. Daniel, T. J. Ffytche, P. S. Sundar Rao, J. H. Kempen, M. ener-West, and P. Courtright. Incidence of ocular morbidity among multibacillary leprosy patients during a 2 year course of multidrug therapy. Br.J Ophthalmol. 90 (5):568-573, 2006. A. U. Ekere, B. E. Yellowe, and S. Umune. Mortality patterns in the accident and emergency department of an urban hospital in Nigeria. Niger.J Clin.Pract. 8 (9):14-18, 2005.

99

T. Ersumo, Y. Meskel, and B. Kotisso. Perforated peptic ulcer in Tikur Anbessa Hospital: a review of 74 cases. Ethiop.Med J 43 (1):9-13, 2005. M. N. Gandhi, T. Welz, and C. Ronsmans. Severe acute maternal morbidity in rural South Africa. Int J Gynaecol.Obstet. 87 (2):180-187, 2004. D. Hailu and B. Worku. Birth trauma among live born term neonates at a referral hospital in Addis Ababa, Ethiopia. Ethiop.Med J 44 (3):231-236, 2006. I. F. Hesse, A. Mensah, D. K. Asante, M. Lartey, and A. Neequaye. Adult tetanus in Accra, why the high mortality? An audit of clinical management of tetanus. West Afr.J Med 24 (2):157-161, 2005. F. Jaimes, Rosa G. De La, E. Gomez, P. Munera, J. Ramirez, and S. Castrillon. Incidence and risk factors for ventilator-associated pneumonia in a developing country: where is the difference? Respir.Med 101 (4):762-767, 2007. S. Mousavi, M. Nobahar, A. A. Vafaei, M. Malek, M. Babaei, F. Malek, and S. Yaghmaie. Complications of upper gastrointestinal endoscopy in unsedated patients. Indian J Gastroenterol. 23 (5):193, 2004. K. A. Mteta, J. Mbwambo, and M. Mvungi. Iatrogenic ureteric and bladder injuries in obstetric and gynaecologic surgeries. East Afr.Med J 83 (2):79-85, 2006. A. M. Nwofor and C. N. Ekwunife. Tube thoracostomy in the management of pleural fluid collections. Niger.J Clin.Pract. 9 (1):77-80, 2006. A. C. Oliveira and D. V. Carvalho. Postdischarge surveillance: the impact on surgical site infection incidence in a Brazilian university hospital. Am.J Infect.Control 32 (6):358361, 2004. C. A. Padua, C. C. Cesar, P. F. Bonolo, F. A. Acurcio, and M. D. Guimaraes. High incidence of adverse reactions to initial antiretroviral therapy in Brazil. Braz.J Med Biol.Res. 39 (4):495-505, 2006. M. C. Passarelli, W. Jacob-Filho, and A. Figueras. Adverse drug reactions in an elderly hospitalised population: inappropriate prescription is a leading cause. Drugs Aging 22 (9):767-777, 2005. H. Trotman and M. Barton. The impact of the establishment of a neonatal intensive care unit on the outcome of very low birthweight infants at the University Hospital of the West Indies. West Indian Med J 54 (5):297-301, 2005. 0 E. Yousif, A. M. Ahmed, M. E. Abdalla, and M. A. Abdelgadir. Deficiencies in medical prescriptions in a Sudanese hospital. East Mediterr.Health J 12 (6):915-918, 2006. prescription review only - AE based reviews L. Herrera-Kiengelher, G. Chi-Lem, R. Baez-Saldana, L. Torre-Bouscoulet, J. RegaladoPineda, M. Lopez-Cervantes, and R. Perez-Padilla. Frequency and correlates of

100

adverse events in a respiratory diseases hospital in Mexico city. Chest 128 (6):39003905, 2005. using HMPS methodology T. Matsaseng and J. Moodley. Adverse events in gynaecology at King Edward VIII Hospital, Durban, South Africa. J Obstet.Gynaecol. 25 (7):676-680, 2005. HMPS like methods on specialist hospital R. ud-Gallotti, H. M. Dutilh Novaes, M. C. Lorenzi, J. Eluf-Neto, Okamura M. Namie, and Velasco Tadeu, I. Adverse events and death in stroke patients admitted to the emergency department of a tertiary university hospital. Eur.J Emerg.Med 12 (2):63-71, 2005. in CVA patients only

Interview based studies A. I. Miasso, C. R. Grou, S. H. De Cassiani, A. E. de Silva, and F. T. Fakih. [Medication errors: types, causes and measures taken in four Brazilian hospitals]. Rev.Esc Enferm.USP. 40 (4):524-532, 2006. P. Okong, J. Byamugisha, F. Mirembe, R. Byaruhanga, and S. Bergstrom. Audit of severe maternal morbidity in Uganda--implications for quality of obstetric care. Acta Obstet.Gynecol.Scand. 85 (7):797-804, 2006. combined with rr - Questionnaires P. C. Gordon, R. L. Llewellyn, and M. F. James. Drug administration errors by South African anaesthetists--a survey. S.Afr.Med J 96 (7):630-632, 2006. Direct observation A. D. Daly, M. P. Nxumalo, and R. J. Biellik. An assessment of safe injection practices in health facilities in Swaziland. S.Afr.Med J 94 (3):194-197, 2004. D. B. dos Santos and H. L. Coelho. Adverse drug reactions in hospitalized children in Fortaleza, Brazil. Pharmacoepidemiol.Drug Saf 15 (9):635-640, 2006. K. L. Leonard and M. C. Masatu. The use of direct clinician observation and vignettes for health services quality evaluation in developing countries. Soc.Sci.Med 61 (9):19441951, 2005. Incident reporting systems G. C. Fernandes, L. A. Camacho, and Carvalho M. Sa. Surveillance system of vaccine adverse events and local data analysis--the experience in a middle-sized city in Brazil, 1999-2001. Vaccine 23 (17-18):2349-2353, 2005. reporting of vac ae only

101

J. Jose and P. G. Rao. Pattern of adverse drug reactions notified by spontaneous reporting in an Indian tertiary care teaching hospital. Pharmacol.Res. 54 (3):226-233, 2006. F. A. Khan and M. Q. Hoda. Drug related critical incidents. Anaesthesia 60 (1):48-52, 2005. External audit and confidential enquiries T. Olowonyo, S. Oshin, and I. Obasanjo-Bello. Registering in a health facility for delivery protects against maternal mortality in a developing country setting. J Obstet.Gynaecol. 25 (7):638-641, 2005. T. P. Shakespeare, M. F. Back, J. J. Lu, K. M. Lee, and R. K. Mukherjee. External audit of clinical practice and medical decision making in a new Asian oncology center: results and implications for both developing and developed nations. Int J Radiat.Oncol.Biol.Phys. 64 (3):941-947, 2006. Studies of claims and complaints A. R. Tumer and C. Dener. Evaluation of surgical malpractice in Turkey. Leg.Med (Tokyo) 8 (1):11-15, 2006. W. C. Tsai, P. T. Kung, and Y. J. Chiang. Relationship between malpractice claims and medical care quality. Int J Health Care Qual Assur.Inc.Leadersh.Health Serv. 17 (7):394-400, 2004. M. H. Ozdemir, N. Cekin, I. O. Can, and A. Hilal. Malpractice and system of expertise in anaesthetic procedures in Turkey. Forensic Sci.Int 153 (2-3):161-167, 2005. Information technology and Electronic Medical Records None identified Administrative data None identified Autopsy reports F. Q. Zhang. [Analysis of autopsy and clinical findings on medical malpractice cases after heart operation]. Fa.Yi.Xue.Za Zhi. 23 (1):46-48, 2007. A. O. Daramola, S. O. Elesha, and A. A. Banjo. Medical audit of maternal deaths in the Lagos University Teaching Hospital, Nigeria. East Afr.Med J 82 (6):285-289, 2005. D. J. Kusum, R. D. Jaya, and P. A. Gayathri. Medical autopsy: whose gain is it? An audit. Indian J Pathol.Microbiol. 49 (2):188-192, 2006. Mortality and morbidity reports

102

A. Dumont, A. Gaye, Bernis L. de, N. Chaillet, A. Landry, J. Delage, and M. H. BouvierColle. Facility-based maternal death reviews: effects on maternal mortality in a district hospital in Senegal. Bull.World Health Organ 84 (3):218-224, 2006. V. Filippi, R. Brugha, E. Browne, V. Gohou, A. Bacci, Brouwere De, V, A. Sahel, S. Goufodji, E. Alihonou, and C. Ronsmans. Obstetric audit in resource-poor settings: lessons from a multi-country project auditing 'near miss' obstetrical emergencies. Health Policy Plan. 19 (1):57-66, 2004. A. Krug, R. C. Pattinson, and D. J. Power. Saving children--an audit system to assess under-5 health care. S.Afr.Med J 94 (3):198-202, 2004.

103

Workshop discussion paper: Effectiveness and sustainability interventions to improve patient safety: What do we know and how can we know more? Research Agenda Track: Parallel Workshops Session 2 Tuesday 25th Sept, 2007.

104

Patient Safety Research shaping the European agenda 24-26th September 2007: Porto, Portugal

WORKSHOP DISCUSSION PAPER

Effectiveness and sustainability of interventions to improve patient safety: what do we know and how can we know more? John Ovretveit

Abstract The purpose of this paper is to provide a reference to help plan future research in Europe into patient safety interventions. It summarise the state of knowledge about the effectiveness of safety interventions and notes European research and initiatives. It then identifies research methods, issues especially about evaluating large scale complex social interventions and providing decision-makers with actionable knowledge for their settings. It finishes by noting gaps in the research and opportunities for collaboration, especially between the western and newly independent states of the 53-country membership of WHO Europe.

Dr John Øvretveit, ([email protected]) Director of Research, The Karolinska Institute Medical Management Centre, Stockholm, and Professor of Health Policy and Management, Faculty of Medicine, Bergen University, Norway. Address: Medical Management Centre, Karolinska Institute, Floor 5, Berzelius väg 3, STOCKHOLM, Sweden SE-171 77 Tel (46) 31 69 39 28. Fax (46) 31 69 1777.

105

1. Introduction We know more about the size and seriousness of the problem than we know about the effectiveness and implementation of solutions. It is research into solutions – safety interventions – on which this paper concentrates. Its purpose is not to present evidence of effectiveness of patient safety interventions. Rather, to give a neutral overview of some starting points for research on patient safety intervention research in Europe, note the issues and gaps in research and outline some possible next steps to advance this specific field. The review was not based on a systematic study for this paper, but drew on the author’s research reviews and research, his selective and partial knowledge of research carried out as a result of his international collaborations and networks, and includes comments from the workshop session at the WHO safety research conference in Porto, Portugal in 2007. It aims to contribute to new safety research and provide material for discussion rather than a definitive overview. The first part considers issues to bear in mind when reviewing the field for planning future research. The next part give a short overview of research into safety interventions, signalling where the reader can find more details from some full reviews of the field which have been undertaken. This is followed by a discussion of methods and approaches for developing safety research to make a stronger practical and scientific contribution. The appendix gives a list of how certain terms are used in this paper and notes the limitations of the concepts which underlie these terms (eg “safety intervention”, “implementation”, “target”, “strategy”, “outcomes”). 2. Considerations in reviewing the field to plan research action The purpose of the paper is to: -

Give an overview of the safety intervention research which has been done and of the methodological issues, in order to strengthen safety research in the future. Researchers need to be aware of who is doing which research in this rapidly developing field so as to build on and contribute to the body of knowledge, and if they are to connect with researchers in the same area and avoid duplication. Safety research can particularly benefit from theories and empirical research from different disciplines. At present, little research is multidisciplinary.

-

Identify limitations in research methods used to study safety. There is a need to continue to use and develop experimental methods to study effectiveness of clinical level interventions. However, the less stardardisable and controllable interventions, for example a large scale and complex evolving programmes, then the more observational and social science methods are needed to understand implementation issues for these types of interventions and the many types of consequences. This is also necessary even for some simple clinical level interventions when controls or even comparisons are not possible to exclude other explanations for “outcomes”.

-

Identify gaps in the research There are a number of potential interventions developed by other industries which have not been studied in health care. There are also interventions being used, which may spread without proper evaluation and become standard practice too-early before interventions which may be more effective are considered. Much of the research also lacks a theoretical basis or a theoretical contribution, which restricts the cumulation and growth of knowledge, but also makes it more difficult for managers and others to apply the research in their settings because there are rarely theoretical explanations of why an intervention has its effects. 106

Planning safety research also needs to bear in mind: -

-

-

-

-

-

The need to build a European and individual country capacity to undertake research in terms of the individuals, centres, and collaboration networks, Building on research and research capabilities and collaborations already established, and increasing their effectiveness with different forms of support and networking for synergy, The needs of all of the 53 WHO European members states for relevant research and research strengthening, not only the high resource western states. The latter have much to learn from implementation in low resources settings, as has been shown in quality improvement research, Research is inevitably driven in part by the political profile of the issues: safety is only beginning to appear on the agendas of a few countries, and leading activity is often more on a local or regional basis, Increasing consumerism, education, worker and patient movements, human rights and patients rights, and the growing power and participation of patients in safety issues in some countries, Significant changes are occurring in a number of countries because national government and other bodies have exercised leadership to highlight safety problems and carry forward actions. This has usually involved the formation of a national association for patient safety independent from government. In some countries there is a notable lack of leadership of safety improvement, with some organisations taking some initiatives but no authoritative body taking a strong role in coordinating and promoting action. Misunderstandings due to differences in interpretation of terms and definitions. Patient safety, risk, and safety culture, for example, are all defined and understood differently. Research and discussions would be more productive if the terms used were defined by those using them, but also if the often simplistic concepts underlying them were questioned and developed (eg about causality and actors participation in implementation)

3. What is known about the effectiveness and sustainability of safety interventions? Actions to improve patient safety are increasingly being used at different levels of the health system and many more are being considered. Some have been tested and found effective in other industries, and some have been studied in health care. However, more are being carried out without sufficient evidence of effectiveness. In part this is because of the challenges in studying the effectiveness and costs of these actions in health care, especially those which cannot be standardised and where research methods to control confounders are difficult to use. An overview shows most of the research has been undertaken in the USA and Canada, and, more recently, an increasing amount in the UK and Australia. The Netherlands stands out in Europe for its quality research, but safety research in mainland Europe is scarce and mostly comprises the recent EU funded initiatives which, in the initial phases, have been surveys of EU members safety activities and regulations. The following gives more details of specific reviews of research into safety interventions. In recent years there have been five comprehensive and rigorous reviews of the effectiveness of safety interventions: the first covering all types of healthcare (AHRQ 2001), the second covering hospitals (Wong & Beglaryan 2004). The third and fourth are WHO evidence reviews oriented to Europe, covering quality as well as safety improvement interventions in hospitals (Øvretveit 2003) with a complementary review of quality and safety methods (Øvretveit 2005). The fifth more recent and covering safety interventions and costs (Øvretveit 2007). Good evidence exists that certain clinical level interventions will improve safety for patients in most settings (eg prophylaxis to prevent venous thromboembolism in patients at risk). These are described in the AHRQ 2001 review and some in the NQF 2003 document which is a practical guide (eg “Verbal orders should be recorded whenever possible and immediately read back to 107

the prescriber”). Recent experience shows that a combination of specific interventions (“bundles”) can be effective for addressing some clinical safety problems. Gaps in research in this category of intervention include some clinical level interventions which have not been well studied in different settings, especially low resource settings, and very little research on implementation and sustainability issues, which psychological, social and organisation research perspectives would illuminate. The evidence is less strong about some other complex clinical level interventions (e.g. rapid response or medical emergency teams, and crew resource management). In part this is because the research where it has been done shows varying results, and often these variations are due to the large differences between the actual interventions carried out but categorised under the same name. There is also less strong evidence about the effectiveness of different methods for safety data collection, analysis and follow up action (eg different root cause methods and mechanisms for linking these to action). Evidence, where it exists for the latter, is mostly from outside of health care, but many of these methods have strong face validity and did not need strong evidence of effectiveness to be widely adopted in other industries. In this respect health care has both gained and lost from the dominant ideology of the hierarchy of evidence which has taken over the evidence based medicine movement. Turning to more complex social interventions with broader targets lead by higher levels of the health system, there is evidence that some organization- and system-level interventions can be effective, if implemented properly. Some of these are listed in the NQF 2003 recommendations. Many interventions at this level have not been systematically evaluated in health care or in nonUSA settings (e.g. crew resource management). Economics of safety There is limited scientific research into the costs of implementation of different interventions or of economic benefits, even if only to the provider. There is experiential- and practitionerevidence of savings. There is virtually no research into the additional income which providers might generate from the increased bed and service capacity which might be released by effective safety interventions, where providers are in systems which reward extra throughput. There is little research into the wider non-provider costs of adverse events, and an under-use of claims databases to assess volume and costs of safety problems. There is no research into the interventions for healing the harm caused to patients and providers by involvement in a serious adverse event, such as the use of modified versions of post-traumatic stress disorder treatments. Although there is evidence that financial incentives influence physician- and organisational behaviour, there is little evidence of the results of financial incentives for safety interventions. There is evidence that lack of finance constrains investing in safety interventions, especially when not required to by regulatory or inspection agencies. Face validity suggests financial intervention would have some effect, but negative consequences are possible and difficult to predict. The costs of an incentive scheme should also be considered, including the costs of verifying compliance (eg from provider reports or from other data). There is evidence that safety problems have a high financial cost, but little evidence of the cost of the interventions and of savings. Some research limitations Most research evidence of the effectiveness of safety interventions has been carried out in other industries, US health care or in health care systems different from those in Europe. Whilst some interventions are likely to be effective in any setting, full implementation may be more or less difficult in a particular country’s health care system. Finance and skills for research into these interventions is limited in many countries, especially for social science approaches rather than 108

traditional medical science experimental methods, and will take time to become available. Other action is needed to promote testing and experience exchange, possibly using experience in quality improvement and research-practice collaborations as a model for developing research networks and expanding the concept of research to include practical projects by health care providers with less rigorous data. Research is growing but still scarce and limited mostly to the UK into the growing power and participation of patients in safety issues, including education and their prevention role. There is little research into the advantages and benefits of publicity of different types to mobilise patients to demand action or “protect themselves”, and about movements which are starting to take this approach – there is increasing cross-boarder collaboration between these patient groups. Research is lacking about interventions outside of hospitals, especially interventions to address safety problem arising in transfers and communications between practitioners and organisations. One notable exception is studies of medication in homes for older people – one example of empirical research which has had a significant policy and practice impact and started a new sub-filed of interventions and research into these (eg community pharmacy programmes). There is little research about safety programmes in organisations or national strategies by government and other bodies to improve safety, although some recent EU surveys have shown the level of activity across EU-Europe. More knowledge about these strategies using social science research methods could provide ideas and some evidence for designing strategies in a particular country. A WHO Europe programme will publish a review of the research to provide guidance for decision makers on how to develop a strategy in early 2008. Recent knowledge about the environment factors which are important for implementation One development in the field has been a recognition that interventions are often dependent upon and interact with the conditions surrounding them.. There is some evidence that safety improvement requires a number of supporting conditions and critical success factors for some interventions to be effective and sustained (e.g. new technologies). It is likely that different conditions are necessary for different interventions to be successfully implemented and effective in different settings, although some conditions may be common to some groupings of interventions. Some “barriers” research and theories into necessary conditions does not sufficiently recognise these differences in the type of intervention. Research does suggest a number of conditions which both organizations and national bodies need to create to stimulate and sustain safety actions. Some theoretical discussions suggest that, when applied together, a number of complementary changes to conditions and interventions at different levels of the health care system are likely to be effective: interventions at one level of the system can create conditions which support or impede implementing safety interventions at the level below. There is reason to believe that organisations and nations wishing to improve safety are more likely to be successful if there are coordinating mechanisms at different levels to ensure their actions are mutually-reinforcing or at least not conflicting, and if they add and link actions to have a larger overall effect. 4. Consequences of research limitations - scientific and practical The deficiencies in the quality and quantity of scientific knowledge about the effectiveness of safety interventions have been described in a number of reports and articles. Knowledge about how interventions are implemented in different settings and what helps and hinders implementation is even scarcer. One consequence is that theory predicting, explaining or understanding implementation and effects of these changes to health care is limited, as shown not only by the lack of theoretical discussions in papers, but also by mechanistic linear cause-effect assumptions which do not take into account individual and group interpretations, power or system interactions. This 109

restricts the development of safety science in health care, in part because researchers are not able to build on theories which have already had some testing when planning their research, and later in their research further to refine theories as a result of their findings. Another consequence is that actors seeking solutions to safety problems have little scientific knowledge to inform their choice of intervention and implementation strategy. Actors such as managers and policy makers need to prioritise which problems to address. Effective prioritisation of which of the many problems to address is not possible without information about the effectiveness, cost and ease of implementation of different possible interventions. In a few cases where research is available, there are rarely definitive conclusions or recommendations, even for other settings similar to those in which the research was conducted. The indeterminacy of the knowledge increases with the complexity of the intervention where controls become more difficult, and the more we move from the clinical level to the national level and the more we consider complex social interventions rather than standard physical or technical interventions. Actors have to make difficult assessments of whether similar results would occur in their settings and often lack the descriptions of implementation or context which would help their assessment. The deficiencies of knowledge thus have serious consequences. The challenges are daunting, but there are ways forward and great opportunities to develop scientific methods and multidisciplinary working, but only if a broad range of approaches are considered, not only controlled experimental methods. 5. Challenges and solutions in studying effectiveness Research methods can be considered at three levels • Philosophical approach: eg, positivist, phenomenological/naturalistic, and action science • Design: eg experimental and quaisi-experimental, survey, case study, action research • Data collection and analysis: eg quantitative, qualitative. Traditionally there has been alignment between the approaches used in each level: typically experimental studies involve gathering quantitative data about outcomes and are based on positivist conceptions of knowledge. If the aim of the study is to discover if an intervention or action does increase safety, then all studies need to address these questions: 1. What exactly was the intervention or actions carried out in practice (description)? 2. What was the environment surrounding the intervention or actions (environment description)? 3. Were the data about results reliable? (data reliability)? 4. Were the data about results valid? (data validity – they represent a valid dimension of outcome) 5. What else apart from the intervention or actions could explain all or some of the results? 6. How certain are we that the results were primarily caused by the intervention or actions and not other primary cause? (attribution)? Often the aim is also to assess sustainability: whether and how the intervention actions are maintained over time and whether and how the results are sustained. “Strong evidence” of effectiveness and sustainability requires that the study gives satisfactory answers to these questions. The attribution question is more difficult to answer the longer the time the result data are collected after the initial introduction of the intervention, and if the time is to short for the intervention to affect the data. This can make sustainability assessment difficult. In addition, the more we move from safety interventions which are physical artefacts (eg forcing functions preventing nitrous oxide gas tubes being plugged into oxygen-gas delivery sockets) 110

towards complex social interventions at local and then national levels (eg national accreditation programme), then the more difficult it is to use designs which require control of the intervention, the contexts and the subjects. Different philosophical approaches have different criteria for what constitutes a satisfactory answer to these questions. The purpose of design and data gathering methods is to guide the study to produce satisfactory answers. Complex social interventions Patient safety research is particularly needed into implementation processes and environments. Even simple interventions often require active participation of health workers (the primary “targets”) and others for full implementation and depend on many surrounding environmental factors. The importance of participation and of the surrounding environmental helpers and hinderers increases the more we move from simple clinical level interventions towards large scale multiple component programmes, which are often evolving over time, especially if action evaluation provides feedback. Research would be improved by a greater recognition that safety interventions are, in their implementation: - a series of actions carried out by human beings (with varying intensity and quality protocols often cannot be enforced) - to influence other human beings (who have choice, and are exposed to competing influences) - in a changing social, economic and political situation. The desired effects depend on the intervention actions being carried out – by “implementation leaders” and the “targets”, as well as the social conditions which support the actions: the degree to which the actions can be fully and consistently carried out depends on the conditions surrounding the actors. For example higher level support, resources, time, knowledge, and skills. The effects of these actions also depends on conditions which help and hinder the actions to have the desired results (eg media or cultural influences which contradict or complement the actions and which interact with the actions). The sustainability of safety interventions and their results depends even more on the surrounding conditions – short-term evaluations are less sensitive to the role of surrounding conditions and how they change. To understand which conditions are necessary for implementation and desired effects, it does not always help to “control these out” as “confounders”. It may be better to understand how they have their influence using naturalistic social science research designs. These ideas can be summarised by describing many safety interventions as complex social interventions, carried out under changing conditions which both help and hinder the intervention and its effects. These surrounding conditions become more influential the longer the programme continues or after the recipients of the programme are no longer exposed to it. One conclusion is to extend research before and after the traditional focus which is on “secondary implementation” (eg implementing an intervention which is already planned) to study the “before” phases of “primary implementation” (preparation, assessing readiness, translating the intervention to situation and planning implementation structure and processes). In addition, to study later in time the phases of “tertiary implementation” (evolving the intervention after testing, sustaining the intervention and spreading it beyond the patient, organisations or areas used in the test).

111

Primary intervention What implementers do to assess, translate and prepare the situation

Secondary Implementation What implementers do to carrying out planned actions to change and data on shortterm results

Tertiary intervention What implementers do to extend in time and areas covered (sustainability and spread)

6. Challenges and solutions for users of research The interventions studied in research are specific intervention involving actions over a period of time in one place. Researchers define the boundaries of the intervention, often narrowly, and place everything else as “context”. Evaluation research gathers evidence to assess the effects of this intervention on certain measures such as reduction in patient mortality or morbidity. An assessment then has to be made as to whether the effects are due to the intervention or to other changes (internal validity), and also whether the intervention can be repeated and the same effects produced elsewhere (external validity). Research sometimes does not describe the actual intervention in detail, but refers to a category of interventions – for example, “automation”, or a subcategory, “computer order entry “– but there are many different examples or ways to implement the intervention. Poor descriptions make it difficult to asses the external and internal validity of the study. The key issues are whether the actual intervention is sufficiently well described, reproducible elsewhere and likely to produce the same results elsewhere. Users of the research need to assess the likely effectiveness and ease of full implementation in their setting. This is because the strength of evidence of effectiveness is different for different interventions. In addition, some interventions depend more on the context within which they are applied than others. Little research has been carried out into cost-effectiveness – users may decide that the likely cost of implementation may be too high for the possible safety gain achieved. Stronger evidence is required to justify implementation where the intervention may be high cost, difficult to implement, or may expose patients and personnel to harm. Removing concentrated potassium chloride from patient care areas did not need a number of randomised controlled trials to prove effectiveness. Some safety interventions do carry a cost, take some effort to implement, but do not carry risks: an example is a rapid response team for critically-ill patients in hospital. There is an ethical case for implementing these interventions sooner rather than later and without strong evidence. Evidence may not exist because the research has not been done, or because clear evidence of effectiveness is difficult to establish, yet the case for the intervention may be strong. There is a balance between the amount of evidence required to justify implementation and the patient suffering which could be prevented. The consideration of how many patients will die before a certain standard of proof is reached needs to be balanced with a consideration of the costs and possible dangers of implementing an intervention for which evidence of effectiveness is scarce (the “proportionality of burden of proof”). 6.1. Matching research approach to purpose of the study Who the study is for, and the information they need should decide the research approach. Many research funding agencies in the past have funded research which is primarily used by other researchers or academics and contributes to the body of empirical and theoretical scientific knowledge on the subject. More recently governmental and other research funders have emphasised that the research should also be for, or exclusively for practitioners or the public in order to help them to act in a more informed way. One view is that this is an unnecessary distinction as “good science is also useful”, but it is a challenge for researchers to design research which satisfactory contributes both to science and practical decision-making, not least because the timescales for both are different. 112

There is an argument that safety research is best designed for one user and with an understanding of the information they need and the decisions the research is to inform. That a study designed for many different users and purposes results in gathering too many different data but superficially and does not serve any one purpose or user well. A user-focused approach to safety research often involved participation with users in defining the end information required and the purpose of the study. Both practical and scientific research is required, and methods which allow both. 7. Conclusions. There is some strong evidence that certain interventions will increase safety in any setting, but little evidence of the costs and savings. Two areas where research is particularly needed are into: - Interventions outside of hospitals and to improve interprofessional working and cross-unit communications and coordination. - Interventions which increase the role of patients in safety at all levels and in interventions led by the growing patient safety consumer and rights movements. There is little research into implementation and sustainability or which provides decision makers with information to assess, in their settings, the likely results or plan implementation. Most research has been carried out in health systems different to those in Europe, and in high resourced well organised and managed health systems. It is likely that intervention success depends on certain organisation and environmental conditions, but there is little research indicating which these are for different interventions, and many of these preconditions may not be present in many European states – this is a key research area. Much research assumes a limited conception of evidence and causality which is not appropriate for understanding more complex social interventions such as hospital safety programmes. There is a case for a more cross-disciplinary and research-practice view of evidence which includes experiential evidence and assessments by stakeholders about different effects of the intervention as well as the expected effects and sustainability. Few safety interventions are simple controllable interventions to stable situations. For full implementation, even physical artefacts and forcing functions require surrounding conditions which support implementation (not the least of which is finance), adaption to the specific situation, a process of implementation and often the active participation of educated and autonomous professionals. Social sciences have methods and theories to understand and explain intervention success and failures, but which are under-used at present. Open system theory is particularly useful for highlighting the permeable boundaries between intervention and context and showing the arbitrary boundaries which current methods force evaluators to draw between the two. Open systems theory helps create more satisfactory understanding safety problems and shows the limitations of linear causality, but to date has been less useful for evaluation or planning better interventions and implementation development in this field could significantly advance the science of patient safety by using methods used in research from health promotion, community public health, and safety science. Building on the above, the following provides an research strategy which promotes a multidisciplinary approach, and a broad approach to evidence for increasing scientific knowledge about what we know about the effectiveness and sustainability of safety interventions: 1. Design research for one user and the actions the research will inform, drawing on previous theory in the design, and refining this theory from the results, 2. Before data gathering, define which effects the research expects to find and the theory about the action paths which lead to effects and the conditions which help and hinder the implementation of the actions, 113

3. Collect data about the actions actually taken to implement the safety intervention so as to provide better descriptions of what was assessed, 4. Collect and provide data about the environment within which the intervention was carried out, using theory about which factors would help and hinder implementation to guide data gathering 5. Collect and present data about results which would allow users to make more informed decisions or actions, or which would test or change theories or previous findings. 6. List all possible other explanations for expected results before the study, and control for or collecting data to assess these explanation’s degree of influence when presenting results. Attention to these issues would make it possible to developing methods better to answer the attribution question, especially for complex social interventions carried out over time. Developments in evaluation science have much to offer, especially in evaluating public policy programmes, including theory based evaluation, realist evaluation, action evaluation and the theory of change approach.

114

Appendix 1: Terms, concepts, evidence and knowledge types The focus of this paper is research into effectiveness of “safety interventions” and how to develop this research, rather than, for example, research into “the size of the problem” and other types of safety research. One factor constraining productive discussion and scientific advance is different uses of the same term, which sometimes indicates deeper differences in conceptual understandings and assumptions. The safety field is multidisciplinary and needs to be developed more in this respect, so attention to defining terms in papers and discussion is even more necessary for progress. This paper does not propose definitions or advocate for agreement in terms, but it does define some of the meanings of terms used in the paper. A patient “safety intervention” is any action taken to prevent or minimise harm to a patient, ranging from a non-slip floor mat to prevent a patient from falling to a 5 year national strategy. “Intervention” is used generically to describe actions taken at the clinical, organisational and national level by different actors. “Strategy” is more often used to describe a collection of activities or interventions carried out by an organisation or a national body. “Outcomes” describes a range of effects on health providers and patients and others in the short or long term which can be attributed to the intervention-actions. Figure 1: Intervention coverage (target) and intervention complexity Intervention scope (target) National or Eg Directive to all Eg a national Europe collaborative health care facilities to breakthrough programme remove concentrated to carry out a number of potassium chloride from actions to reduce patient care areas and ventilator associated actions to ensure this is pneumonia or improve carried out hand hygiene Region Interventions to apply the directive continuously throughout the regional Facility (eg Interventions to apply hospital) the directive continuously throughout the hospital Clinical Interventions to apply team/unit the directive continuously in the unit eg a combination of Eg directive to remove Individual education and physical concentrated potassium and other actions to chloride from patient increase hand-hygiene care areas Single simple Single “bundle” Intervention complexity

Eg a national patient safety programme with many projects

Eg a regional patient safety programme with many projects Eg a hospital patient safety programme with many projects Eg a unit patient safety programme with many projects

Multiple complex

“Intervention” is a term commonly used in medical research mostly to describe a treatment for a patient. “Implementation” are the actions actually taken to put the intervention into practice, most of which are carried out by the “targets” of the intervention. For example, patients implement a treatment when they go to a pharmacy and buy and take the full course of the prescribed medication. Language and assumptions have not recognised the level of participation or self-direction by the “targets of the intervention” and has assumed the full implementation which is ensured in controlled medical research. In addition some safety research and much practice appears to 115

be based on liner models of causality which are often not appropriate to the problem (eg “root cause” analysis) and are carried into intervention assessment. Even epidemiology models (latent causes) do not take into account more advanced accident theory which include understanding of system interaction, and where “causation” is the concurrence or coincidence of factors in a situation. The “mirror image” of this type of theory about how adverse vents are caused (cause of the problem) is theories about how interventions cause effects (cause of the solution). Research using controlled experimental trials involves testing a theory that certain actions would cause a reduction in the phenomena represented by the outcome measures. Often these theories are implicit and there is no theory about how the intervention has any effects detected – the focus is on what works? not how. However most safety interventions involve many actions of many types – a conceptualisation far removed from the idea of a treatment intervention. In some safety interventions a model of the target and the intervention as both complex social systems in interaction evolving over time may be more relevant than a surgeons knife cutting tissue where consciousness and social systems are not considered. Implementation research, social and psychological research and health promotion contributes new thinking which is particularly relevant to understanding interventions, causality and implementation adaption and innovation in different settings, and what helps and hinders. The paper therefore argues that research into safety interventions needs to broaden the concept of ‘evidence” only referring to evidence of effectiveness to include other types of knowledge such as systematic experiential evidence documentation (SEED). Safety Intervention Specific action

Method or strategy used by an organisation National or regional intervention

Conceptual Strong evidence Weak evidence Experiential of effectiveness of effectiveness reports and discussion (C) (E1) (E2) observations (O) E.g. Appropriate E.g. Medical E.g. Using a E.g. Many use of prophylaxis emergency label sticker to human factors to prevent venous team (or rapid remind nurses engineering thromboembolism response team) to check safety methods used in patients at risk. item in other industries E.g. USA E.g. Root None veterans health cause analysis Creating a administration 10 year strategy safety culture None None E.g. NHS adverse event reporting system

116

Patient safety intervention research: Bibliography AHA (2004) Partnering With Patients to Reduce Medical Errors (Guidebook for Professionals), Rockville, Ill: American Hospital Association, 2004 AHRQ (2000) Medical Errors: The Scope of the Problem. Fact Sheet. Rockville, MD: Agency for Healthcare Research and Quality; 2000. Publication no. AHRQ 00- P037. Accessed at www.ahrq.gov/qual/errback.htm on 7 December 2004. AHRQ (2001) Making Health Care Safer. A Critical Analysis of Patient Safety Practices: Summary. AHRQ Publication No. 01-E057. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/clinic/ptsafety/summary.htm AHRQ's Patient Safety Initiative: Building Foundations, Reducing Risk. Interim Report to the Senate Committee on Appropriations. AHRQ Publication No. 04-RG005, December 2003. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/qual/pscongrpt/ Amalberti R Auroy, Y Berwick, D Barach, P 2005 Five System Barriers to Achieving Ultrasafe Health Care Ann Intern Med. 2005;142:756-764. Andrews L, Stocking C, Krizek T, Gottlieb L, Krizek C, Vargish T Siegler M (1997), “An alternative strategy for studying adverse events in medical care”, The Lancet, 349: 309-313. Arah, A Klazinga N How safe is the safety paradigm? Qual Saf Health Care 2004;13:226–232 Audit Commission (2001) A spoonful of sugar, Audit commission publications, PO Box Wetherby LS23 7JA (author, Mapstone, N) Australian report 1999: Implementing safety and quality enhancement in health care. Bagian J.P., et al. (2002) VA’s root cause analysis system in action. Jt Comm J Qual Improv 28:531–545, Oct. 2002. Bagian J.P., et al.: VA’s root cause analysis system in action. Jt Comm J Qual Improv 2002 28:531–545 Bagian JP, Lee C, Gosbee J, DeRosier J, Stalhandske E, Eldridge N, et al. (2001) Developing and deploying a patient safety program in a large health care delivery system: you can't fix what you don't know about. Joint Commission Journal on Quality Improvement 2001;27(10):522-532. Bagian, J Lee, C Gosbee, J DeRosier, J Stalhandske, E Eldridge, NWilliams, R Burkhardt, M Developing and Deploying a Patient Safety Program in a Large Health Care Delivery System: You Can't Fix What You Don't Know About Joint Commission Journal on Quality and Safety, The Joint Commission Journal on Quality Improvement, October 2001, 27, (10): Baker, G. R., Norton, P. (2001). “Making Patients Safer! Reducing Error in Canadian Healthcare.” Healthcare Papers 2(1): 10-31. Baker, G. R., Norton, P. (2002) “Patient Safety and Healthcare Error in the Canadian Healthcare System: A Systematic Review and Analysis of Leading Practices in Canada with Key Initiatives Elsewhere.” Report to Health Canada. . Banning et al (1993) Reduction of errors in laboratory test reports using CQI, Clin Lab Man rev, 7 424-36. Bates et al (1998) DW, Leape LL, Cullen DJ, Laird N, Petersen LA, Teich JM, et al. Effect of computerized physician order entry and a team intervention on prevention of serious medication errors. JAMA. 1998;280:1311-6. Bates, D Spell, N Cullen, D et al (1997) “The cost of adverse drug events in hospitalised patients” Journal of the American Medical Association, Vol. 277, No 4, pp 307-311. Bates, D. W., Boyle, D. L., et al. (1995). “Relationship between medication errors and adverse drug events.” Journal of General Internal Medicine. 10(4): 199-205. 117

Bent, P Bolsin, S Creati, B Patrick A, Colson M Professional monitoring and critical incident reporting using personal digital assistants MJA 2002, 177: 492-97. Berte, L & Nevalainen, D (1997) Quality pays – in every business” Transfusion science, 18, 4: 589-596. Bigelow B & Arndt M The more things change the more they stay the same, Health Care Management Review2000 25 (1) 65-72 Birkmeyer JD, Birkmeyer CM, Wennberg DE, Young M (2001) Leapfrog patient safety standards: The potential benefits of universal adoption. 2001. Washington, DC, The Leapfrog Group. Bogner M, editor.(1994) Human error in medicine. Hillsdale, NJ: Lawrence Erlbaum Associates. 1994. Borbas C, Morris N, McLaughlin B, Asinger R, Gobel F The role of clinical opinion leaders in guideline implementation and quality improvement Chest. 2000 Aug;118(2 Suppl):24S-32S. Bradley EH, Holmboe ES, Mattera JA, Roumanis SA, Radford MJ, Krumholz HM. The roles of senior management in quality improvement efforts: what are the key components? J Healthc Manag. 2003 Jan-Feb;48(1):15-28. Brennan T, Leape L, Laird N, Hebart L, Loralio A, Lawthers A Newhouse, Weiler, & Hiatt (1991) Incidence of adverse events and negligence in hospitalised patients”, New England Journal of Medicine, 324: 370-6, 1991. Brennan, T. A., Localio, A. R., et al. (1990). “Identification of adverse events occurring during hospitalization. A cross-sectional study of litigation, quality assurance, and medical records at two teaching hospitals.” Annals of Internal Medicine. 112(3): 221-6. Brown et al (2002) (in New Zealand each adverse event cost an average of $NZ 10,264 per patient, with a total cost of $NZ 870 million or 30% of total public hospital expenditure, of which $NZ 590 million was associated with preventable adverse event. Brown, P., McArthur, C., Newby, L., Lay-Yee, R., Davis, P., & Briant, R. 2002, "Cost of medical injury in New Zealand: a retrospective cohort study", J.Health Serv.Res.Policy, vol. 7 Suppl 1, p. S29-S34. Burton, R (1995) Reductions in unwanted complications and lengths of inpatient stay by use of “dynamic guidelines” Physician Executive Journal of Management, Vol 21, No 11, pp 19-21. Cabana D, Rand C, Powe N, Wu A, Wilson M, Abboud P, & Rubin H Why Don't Physicians Follow Clinical Practice Guidelines?: A Framework for Improvement JAMA, Oct 1999; 282: 1458 - 1465. Caplan RA, Posner KL, Cheney FW (1991) Effect of outcome on physician judgments of appropriateness of care. JAMA. 1991;265:1957-1960. Carlisle D (2005)”Points mean prizes for GPs, but what’s in it for patients?” Health Services Journal, 26May 2005:16-17. Chaix-Couturier C, et al (2000). Effects of financial incentives on medical practice: results from a systematic review of the literature and methodological issues. Int J Qual Health Care 2000;12:133. Classen, D. C., Pestotnik, S. L., et al. (1997). “Adverse drug events in hospitalized patients. Excess length of stay, extra costs, and attributable mortality.” Journal of the American Medical Association. 277(4): 301-6. Connell, Kubisch, A Schorr L, and Connell, J (1998) New approaches to evaluating community initiatives, Vol.2 Theory measurement and analysis, Washington, DC: Aspen Institute Couch, N. P., Tilney, N. L., Rayner, A. A., & Moore, F. D. 1981, "The high cost of lowfrequency events: the anatomy and economics of surgical mishaps", N.Engl.J.Med., vol. 304, no. 11, pp. 634-637. 118

Danish health care act 2003, Act on patient safety in the Danish health care system; Danish parliament act 429, of 10/06/2003). Davis D, Evans M, Jadad A, et al. The case of knowledge translation: shortening the journey from evidence to effect. BMJ 2003; 327: 33-35. Davis, D. A., Thomson, M. A., et al. (1995). “Changing physician performance. A systematic review of the effect of continuing medical education strategies.” Journal of the American Medical Association. 274(9): 700-5. DeBrantes et al 2004 Financial Incentives for Adoption of Health Information Technology by Healthcare Deliverers, E health initiative foundation www.bbriefings.com/pdf/1251/deBrantes.pdf Devers, K Pham, H Liu, G 2004 What Is Driving Hospitals’ Patient-Safety Efforts? Health Affairs 2004 23 (2): 103-115. Dhillon, B (2003) Methods for performing human reliability and error analysis in health care” International Journal for Health Care Quality Assurance 2003 Vol 16 No 6&7: 306-317. Donchin Y, Gopher D, Olin M, et al (1995) A look into the nature and causes of human errors in the intensive care unit. Crit Care Med 23 (2):294-300, Dopson, S., Fitzgerald, L., Ferlie, E., Gabbay, J. and Locock, L. No magic targets. Changing clinical practice to become more evidence based. Health Care Management Review 2002; 37: 35–47. Dudley RA, Frolich A, Robinowitz DL, et al (2004) Strategies To Support Quality-based Purchasing: A Review of the Evidence. Summary, Technical Review Number 10. AHRQ Publication Number 04-P024, July 2004. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/clinic/epcsums/qpurchsum.htm (accessed July 22, 2005) ehealth (2003) Financial Incentives: Innovative Payment for Health Information Technology, from www.ccbh.ehealthinitiative.org. (accessed July 22, 2005) Ehman, A. J. (2003). “Saskatchewan first with mandatory reporting of medical errors.” Canadian Medical Association Journal 168(4): 471. Ferlie EB, Shortell SM. Improving the quality of health care in the United Kingdom and the United States: A framework for change. Milbank Quarterly 2001;79(2):281-315. Flum, D. R. & Koepsell, T. 2002, "The clinical and economic correlates of misdiagnosed appendicitis: nationwide analysis", Arch.Surg., vol. 137, no. 7, pp. 799-804. Fullbright-Anderson (theories of change approach to evaluation based on the logic of theorybased evaluation Funnel, S (1997) Programme logic: an adaptable tool for designing and evaluating programmes, Evaluation news and comment, July 5-17. Gebhart F. (1999) VA facility slashes drug errors via bar-coding. Drug Topics;1:44. Getting evidence into practice. NHS Centre for Reviews Dissemination Effective Health Care Bulletin 5 (1) 1999. Gosbee J (2004) Introduction to the Human Factors Engineering Series Joint Commission on Quality and Safety Journal April 2004 Volume 30 Number 4 215-219. Gosbee J and Anderson T (2003) Human factors engineering design demonstrations can enlighten your RCA team, Qual. Saf. Health Care 2003; 12: 119-121. Gosbee John W.Conclusion: You Need Human Factors Engineering Expertise to See Design Hazards That Are Hiding in "Plain Sight!" Joint Commission Journal on Quality and Patient Safety, December 2004, vol. 30, no. 12, pp. 696-700(5) Gosbee, J Introduction to the Human Factors Engineering Series, Joint commission journal on paitent safety, 2004 30; 4: 215-218) 119

Gosden T, Forland F, Kristiansen IS, et al.(2001) Impact of payment method on behaviour of primary care physicians: a systematic review. J Health Serv Res Policy 2001;6:44–55. Gosfield, A. G. & Reinertsen, J. L. (2003). Doing well by doing good: Improving the business case for quality. Available at www.uft-a.com. Gosfield, A. G. & Reinertsen, J. L. (2004, January 22). Paying physicians for high quality care. New England Journal Medicine. http://222.uft-a.com/PDF/NEJM%20Epstein%20letter.pdf. Gosfield, A. G. (2004). The doctor-patient relationship as the business case for quality. Journal of Health Law 37, 2, 197-223. Available at www.gosfield.com/PDF/DrPatientRelationship.pdf. Gosfield, A. G., "Contracting for Provider Quality: Then, Now and P4P," in Gosfield, (Ed.), Health Law Handbook, 2004 edition, West Group, pp. 103-183, available at http://www.gosfield.com/PDF/ch3PDF.pdf (accessed July 22, 2005) Gray, A 2003 Adverse events and the National Health Service: an economic perspective- a report to the National Patient Safety Agency, Health Economics Research Centre Department of Public Health, University of Oxford, November 2003. Greenhalgh T, Robert G, Bate P, Kyriakidou O, Macfarlane F, Peacock R. A systematic review of the literature on diffusion, dissemination and sustainability of innovations in health service delivery and organisation. London, NHSSDO Programme, 2004. www.sdo.lshtm.ac.uk Grimshaw J McAuley M Bero L et al Systematic reviews of the effectiveness of quality improvement strategies and programmes, Quality and Safety in Health care, 2003; 12:298-303. Grimshaw JM, Thomas RE, MacLennan G, et al. Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technology Assessment, 2004, Vol 8, No6. Grol & Wensing 2004). Grol R Wensing M (2004) What drives change? Barriers to and incentives for achieving evidence based practice. Medical Journal of Australia, 2004; 180: s57-s60. Haycox, M & Bagust, A (1999) Clinical guidelines: the hidden costs BMJ 1999; 318:391-393. HCPAC(2000) Management and control of hospital acquired infection in Acute NHS Trusts in England, UK House of Commons Public Accounts Committee, www.publications.parliament.uk/pa/cm/cmpubacc.htm Hebert, P. C., Levin, A. V., et al. (2001). “Bioethics for clinicians: 23. Disclosure of medical error.” Canadian Medical Association Journal. 164(4): 509-13. HF (2004) Briefing: Making healthcare safer for patients, London: Health Foundation , www.health.org.uk Hibbard JH, Stockard J, Tusler M. Does publicizing hospital performance stimulate quality improvement efforts? Health Aff (Millwood) Mar-Apr 2003;22(2):84-94. Hickson GB, Altemeier WA, Perrin JM. Physician reimbursement by salary or fee-for-service: effect on physician practice behavior in a randomized prospective study. Pediatrics Sep 1987;80(3):344-50. Hillman AL, Ripley K, Goldfarb N, Weiner J, Nuamah I, Lusk E. The use of physician financial incentives and feedback to improve pediatric preventive care in Medicaid managed care. Pediatrics Oct 1999;104(4 Pt 1):931-5. Hunt, D RB Haynes, SE Hanna, K Smith. (1998) Effects of computer-based clinical decision support systems on physician performance and patient outcomes. JAMA 280: 1339-1346. Iles,V Sutherland, K. (2001) Managing Change in the NHS, Organisational Change. London:National Co-ordinating Centre for NHS Service Delivery and Organisation R & D, 2001. Institute of Medicine. (1990). Medicare: A strategy for quality assurance volume II. K. N. Lohr (Ed.). Washington, DC: National Academy Press, 116-139. Ioannidis, J. P. & Lau, J. 2001, "Evidence on interventions to reduce medical errors: an overview and recommendations for future research", J.Gen.Intern Med, vol. 16, no. 5, pp. 325-334. 120

IOM (2000) Institute of Medicine, Committee on Quality of Health Care in America. (2001). Crossing the quality chasm: A new health system for the 21st century. Washington, DC: National Academy Press. IOM (2000) To err is human: Building a safer health system. L. T. Kohn, J. M. Corrigan, & M. S. Donaldson, (Eds.). Washington, DC: National Academy Press, Institute of Medicine, Committee on Quality of Health Care in America. IOM 2003 Priority Areas for National Action: Transforming Health Care Quality, Institute of Medicine, National Academy Press, Washington. Jacobs, B (1999) Partnership in Pittsburgh: the evaluation of complex local initiatives, in Osbourne, S (ed) (1999) Managing Public-Private Partnerships for Public Services: An International Perspective, London: Routledge. JCAHO (2002). “Sentinel event policy and procedures.” Joint Commission on the Accreditation of Healthcare Organizations. Johnson, J. A. & Bootman, J. L. 1995, "Drug-related morbidity and mortality. A cost-of illness model", Arch.Intern Med, vol. 155, no. 18, pp. 1949-1956. Johnson, W. G., Brennan, T. A., Newhouse, J. P., Leape, L. L., Lawthers, A. G., Hiatt, H. H., & Weiler, P. C. 1992, "The economic consequences of medical injuries. Implications for a no-fault insurance plan", Journal of the American Medical Association, vol. 267, no. 18, pp. 2487-2492. Judge, K & Bauld, K Stong (2001) Theory, flexible methods: evaluating complex community based initiatives, Critical Public Health 2001 Vol 11 pp 19-38. Judge, K 2000 Testing the limits of evaluation: Health Action Zones in England, Journal of Health Services Research and Policy, 5:1. Juran Institute (2003) Cost Management Update, Sigma Issue No 140, The Cost Management Group of the IMA from www.juran.com, also reported in BMJ News, Medical errors kill almost 100000 Americans a year. BMJ 1999 319: 1519. Kalish, R. L., Daley, J., Duncan, C. C., Davis, R. B., Coffman, G. A., & Iezzoni, L. I. 1995, Costs of potential complications of care for major surgery patients", Am.J.Med Qual., vol. 10, no. 1, pp. 48-54. Kaplan HS, Battles JB, Van der Schaaf TW, Shea CE, Mercer SQ (1998) Identification and classification of the causes of events in transfusion medicine. Transfusion. 1998;38:1071-78. Karsh, T Beyond usability: designing effective technology implementation systems to promote patient safety, Qual Saf Health Care 2004;13:388–394. Kaushal, R., Shojania, K. G., & Bates, D. W. 2003, "Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review", Arch.Intern Med, vol. 163, no. 12, pp. 1409-1416. Kirkland, K. B., Briggs, J. P., Trivette, S. L., Wilkinson, W. E., & Sexton, D. J. 1999, "The impact of surgical-site infections in the 1990s: attributable mortality, excess length of hospitalization, and extra costs", Infect.Control Hosp.Epidemiol., vol. 20, no. 11, pp. 725- 730. Kuperman, G. J., Teich, J. M., et al. (2001). “Patient safety and computerized medication ordering at Brigham and Women's Hospital.” Joint Commission Journal on Quality Improvement. 27(10): 509-21. LA Times (2003) “Cedars-Sinai Medical Center suspends use of computerized physician order entry system. Los Angeles Times. 22 January 2003:B.1. Leape LL, Brennan TA, Laird N, Lawthers, Localio, Barnes, Hebert, Newhouse, Weiler, & Hiatt (1991) The nature of adverse events in hospitalized patients: Results of the Harvard Medical Practice Study II. N Engl J Med 1991;324:377-84. Leape LL. Reporting of adverse events. N Engl J Med 2002;347:1633-8. Leape, L Kapcenell et al (2000) Reducing adverse drug events: lesions from a breakthrough series collaborative, Joint commission journal on quality improvement, 26, 6 ;321-331. 121

Leape, L. L., Bates, D. W., et al. (1995). “Systems analysis of adverse drug events. ADE Prevention Study Group.[comment].” Journal of the American Medical Association. 274(1): 3543. Leape, L. L., Cullen, D. J., Clapp, M. D., Burdick, E., Demonaco, H. J., Erickson, J. I., & Bates, D. W. 1999, "Pharmacist participation on physician rounds and adverse drug events in the intensive care unit", Journal of the American Medical Association, vol. 282, no. 3, pp. 267-270. Lewis Q Fletcher M Implementing a national strategy for patient safety: Lessons from the National Health Service in England Qual. Saf. Health Care 2005;14;135-139 Marshall M Harrison S (2005) It’s about more than money: financial incentives and internal motivation Qual. Saf. Health Care 2005;14;4-5 Marx D. (2001) “Patient safety and the “just culture”: A premier for health care executives.” Columbia University Mayo Alumni (1995) “Can we be doing better”, Fall issue, p 7. Merry, A., McCall, S. A. (2001). “Errors, medicine and the law.” Cambridge, Cambridge University Press. Michie, S, Johnston, M Abraham, C Lawton, R Parker, D Walker, A Making psychological theory useful for implementing evidence based practice: a consensus approach Qual. Saf. Health Care 2005;14;26-33 Milstein, A Galvin, R Delbanco,S Salber, P Buck, C Improving the Safety of Health Care: The Leapfrog Initiative, Effective Clinical Practice, 2000. Mission, J (2001) A review of clinical risk management J. Qual. Clin. Practice 2001, 21, 131–134 Mulrow C & Oxman A (1997) How to conduct a Cochrane systematic review, Cochrane Collaboration: San Antonio, 1997. Mutter, M (2003) One Hospital's Journey Toward Reducing Medication Errors Source: Joint Commission Journal on Quality and Patient Safety, June 2003, vol. 29, no. 6, pp. 279-288(10) National Coalition on Healthcare (NCH) and Institute for Healthcare Improvement (2000) Reducing medication errors and improving patient safety: success stories from the front lines of medicine, Washington DC, National Coalition on Healthcare.(http://www.nchc.org/). National Coalition on Healthcare (NCH) and Institute for Healthcare Improvement (2002) Curing the System: Stories of Change in Chronic Illness Care, Washington DC, National Coalition on Healthcare.( http://www.nchc.org/). NHCPI (2001) The Growing Case for Using Physician Incentives to Improve Health Care Quality, Washington: National Health Care Purchasing Institute, 2001.(from www.nchpi.net NHS (2001) Building a safer NHS for patients, London: department of health, 2001. NHS Centre for Reviews Dissemination (1999) Getting evidence into practice. NHS Centre for Reviews Dissemination Effective Health Care Bulletin 5 (1) 1999. NHSE, 1993 Risk management in the NHS, London:HMSO,1993 NQF 2003 Safe practices for better healthcare, Washington: National Quality Forum, 2003 (accessed May 30, 2005, www.qualityforum.org) NSCPS 2002, National Steering Committee on Patient Safety Building a Safer System: A National Integrated Strategy for Improving Patient Safety in Canadian Health Care Orser, B. A., Chen, R. J., et al. (2001). “Medication errors in anaesthetic practice: a survey of 687 practitioners.” Canadian Journal of Anaesthesia. 48(2): 139-46. Øvretveit (2007) Economics and effectiveness of interventions for improving safety of health care - A review of research. Available from Karolinska Institutet, Medical Management Centre, Stockholm, and download from: Click “iDisk Public Folder (Graphite)” at top of web site: http://homepage.mac.com/johnovr/FileSharing1.html 122

Øvretveit J (2003) “What is the best strategy for improving quality and safety of hospitals – a review and synthesis of the evidence“ WHO, Copenhagen. Øvretveit J (2005) “What are the advantages and disadvantages of different quality tools– a review and synthesis of the evidence“ WHO, Copenhagen. (www.euro.who.int/hen) Øvretveit J, Andreen Sachs M. Brister i patientsäkerhet och övrig kvalitet - vilka är kostnaderna? Läkartidningen 2005;102:140-2. (What is the cost of the patient safety and quality problem in healthcare? A review of the research). Øvretveit, J (1998) Evaluating Health Interventions, Open University Press, Milton Keynes. Øvretveit, J (1999) Comparative & Cross-Cultural Health Research , Radcliffe Medical Press, Oxford. Øvretveit, J (2002) Action Evaluation of Health Programmes and Change, Radcliffe Medical Press, Oxford Perrow C (1999) Normal accidents : Living with High-Risk Technologies. With a New Afterword and a Postscript on the Y2K Problem. Princeton, NJ: Princeton University Press. 1999. Pittet D, Hugonnet S, Harbarth S, Mourouga P, Sauvan V, Touveneau S, et al. (2000) Effectiveness of a hospital-wide programme to improve compliance with hand hygiene. Lancet 2000;356:1307-12. Plowman, R., Graves, N., Griffin, M. A., Roberts, J. A., Swan, A. V., Cookson, B., & Taylor, L. 2001, "The rate and cost of hospital-acquired infections occurring in patients admitted to selected specialties of a district general hospital in England and the national burden imposed", J.Hosp.Infect., vol. 47, no. 3, pp. 198-209. Pronovost P, Holzmueller, C (2004) Partnering for Quality, Journal of Critical Care, 2004 19(3):121-29. Pronovost P, Waters H, Dorman T (2001)The Economic Impact of the Leapfrog Group intensive care unit physician staffing standard, in Birkmeyer J, Birkmeyer C, Skinner J, (eds): Economic Implications of the Leapfrog Safety Standards. Washington, DC, The Leapfrog Group, 2001 Pronovost P, Weast B, Bishop K, et al: Senior executive adopt-a-work unit: A model for safety improvement. Jt Commission Joint Commission Journal on Quality and Safety 30: 59-68, 2004. Pronovost PJ, Jenckes MW, Dorman T, et al: Organizational characteristics of intensive care units related to outcomes of abdominal aortic surgery. JAMA 1999, 281:1310-1317, QuIC (2000) Doing What Counts for Patient Safety: Federal Actions to Reduce Medical Errors and Their Impact, Report of the Quality Interagency Coordination Task Force To the President (QuIC), Agency for Healthcare Research and Quality., (www.Quic.org) Raschke, R et al.(1998) A computer alert system to prevent injury from adverse drug events. Development and evaluation in a community teaching hospital. JAMA 280: 1317-1320. Rasmussen J.(1990) Human error and the problem of causality in analysis of accidents. Philos Trans R Soc Lond B Biol Sci. 1990;327:449-460. Resar, R Pronovost, PHaraden, C Simmonds, T Rainey, T Nolan, T Using a Bundle Approach to Improve Ventilator Care Processes and Reduce Ventilator- Associated Pneumonia Joint Commission Journal on Quality and Safety 2005 31 (5):243-251. Rex JH, Turnbull JE, Allen SJ, Vande Voorde K, Luther K. (2000) Systematic root cause analysis of adverse drug events in a tertiary referral hospital. Jt Comm J Qual Improv. 2000;26:563Rigby, K & Litt, J (2000) Errors in care management: what do they cost? Quality in Health Care, 9 216-21. Robert Trowbridge, R & Wachter, R (2001) Legislation, Accreditation, and Market-Driven and Other Approaches to Improving Patient Safety Ch55 in AHRQ 2001, Making Health Care Safer. A Critical Analysis of Patient Safety Practices: Summary. AHRQ Publication No. 01-E057. 123

Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/clinic/ptsafety/summary.htm Rosenthal, M. B., Fernandopulle, R., Song, H. R., & Landon, B. (2004). Paying for quality: Providers' incentives for quality improvement. Health Affairs 23 (2): 127-141. Runciman W (2002) Lessons from the Australian Patient Safety Foundation Quality and Safety Health Care 2002;11:246-251. Runciman WB, Sellen A, Webb RK, Williamson JA, Currie M, Morgan C, et al. (1993) The Australian Incident Monitoring Study. Errors, incidents and accidents in anaesthetic practice. Anaesth Intensive Care. 1993;21:506-519. Runciman, B., Merry, A., et al. (2001). “Improving patients' safety by gathering information. Anonymous reporting has an important role.” British Medical Journal. 323(7308): 298. Runciman, W. B., Merry, A. F., et al. (2003). “Error, blame, and the law in health care—an antipodean perspective.[comment].” Annals of Internal Medicine. 138(12): 974-9. Schneider, P. J., Gift, M. G., Lee, Y. P., Rothermich, E. A., & Sill, B. E. 1995, "Cost of medication-related problems at a university hospital", Am.J.Health Syst.Pharm., vol. 52, no. 21, pp. 2415-2418. Schyve, P An Interview with Lucian Leape, Joint Commission Journal on Quality and Safety, 2004;30 (12): 653-58. Scott T, Mannion R, Davies HTO, Marshall MN. Organisational culture and performance in the NHS: a review of the theory. instruments and evidence. York: Centre for Health Economics; 2001. Scott, T, Mannion, R, Davies, H & Marshall, M (2003) The Quantitative Measurement of Organizational Culture in Health Care: A Review of the Available Instruments.Health Services Research 38 (3), 923-945. Smetzer, J (2005) Reducing at risk behaviours, Joint Commission Journal on Quality and Safety, 2005, 31(5): 294-299. Solberg L, Brekke M, Fasio J, et al. Lessons from experienced guideline implementers: attend to many factors and use multiple strategies. Jt Comm J Qual Improv 2000;26:171–88. Sorensen, J Safety culture: a survey of the state of the art, Reliability Engineering and System safety 2002; 76:189-204. Spath PL, Nash DB 2004 A guide to involving patients in error prevention and the role of hospital leadership. Chicago, Illinois: American Hospital Association; 2004. Spooner A, Chapple A, Roland M. What makes British general practitioners take part in a quality improvement scheme? J Health Serv Res Policy 2001;6:145–50. Stalhandske E, Bagian, J Gosbee, J (2002) Department of Veterans Affairs Patient Safety Program Am J Infect Control 2002;30:296-302. Stanhope N, Crowley-Murphy M, Vincent C, et al. An evaluation of adverse incident reporting. J Eval Clin Pract 1999;5:5–12. Thomas EJ, Peterson LA (2003) Measuring errors and adverse events in health care. J Gen Intern Med 2003;18:61–7. Thomas, E Studdert, D Newhouse, J Zbar, B Howard, K Williams, E and Brennan, T (1999) Costs of medical injuries in Utah and Colorado, Inquiry, 36(3) 255-64. Thompson, R McElroy, H Kazandjian, V (1997) “Maryland hospital quality indicator project in the UK”, Quality in Health Care, 6,1, 49-55. Van der Schaaf T. (1992) Near miss reporting in the Chemical Process Industry [Doctoral thesis]. Eindhoven, The Netherlands: Eindhoven University of Technology. 1992. Vincent C, Ennis M, Audley RJ (1993) Medical accidents. Oxford ; New York: Oxford University Press. 1993. 124

Vincent, C. (2001). “Clinical risk management: Enhancing patient safety.” London, British Medical Journal Publishing Group. Vincent, C. A. and A. Coulter (2002). “Patient safety: What about the patient?” Quality & Safety in Health Care. 11(1): 76-80. Vincent, C., Neale, G., & Woloshynowych, M. 2001, "Adverse events in British hospitals: preliminary retrospective record review", British Medical Journal, vol. 322, no. 7285, pp. 517519. Vincent, C., Taylor-Adams, S., et al. (1998). “Framework for analysing risk and safety in clinical medicine.” British Medical Journal. 316(7138): 1154-7. von Laue NC, Schwappach DL, Koeck CM. (2003) The epidemiology of medical errors: a review of the literature. Wien Klin Wochenschr. 2003:115(10):318-25. Wachter, R Shojania, K The Faces of Errors: A Case-Based Approach to Educating Providers, Policymakers, and the Public About Patient Safety, Joint Commission Journal on Quality and Safety in Health Care, 2004 30 (12): 665-673. Walshe, K Shortell, S What happens when things go wrong: how healthcare organisations deal with major failures in care Health Affairs May 2004, 23, 3: 103-11. Walton RT, H. E. D. S. F. N. 2002, Computerised advice on drug dosage to improve prescribing practice (Cochrane Review)., Update Software, Oxford, The Cochrane Library, 1, 2002: CD002894. Wang (1998) (also Wang et al. 2003).Leape et al (1999) Ioannidis and Lau (2001). Wang, S. J., Middleton, B., Prosser, L. A., Bardon, C. G., Spurr, C. D., Carchidi, P. J., Kittler, A. F., Goldszer, R. C., Fairchild, D. G., Sussman, A. J., Kuperman, G. J., & Bates, D. W. 2003, "A cost-benefit analysis of electronic medical records in primary care", Am.J.Med, vol. 114, no. 5, pp. 397-403. Wanzel, K. R., Jamieson, C. G., et al. (2000). “Complications on a general surgery service: incidence and reporting.[comment].” Canadian Journal of Surgery. 43(2): 113-7. Weeks WB, Bagian JP: Developing a culture of safety in the Veterans Health Administration. Eff Clin Pract 3:270-276, 2000. Weiss, C 1997a How can theory based evaluation make greater headway” Evaluation review, 21: 501-8. Weiss, C 1997b Theory-based evaluation: past present and future, New directions for evaluation, 76 41-55. Wilson T & Sheika, A (2002) Enhancing patient safety in primary care, BMJ 324; 584-7. Wilson, R Runciman, W Gibberd, B Harrison, B Newbury, L & Hamilton, J (1995) “The quality in Australian healthcare study”, Medical Journal of Australia, 163 (9): 458-71. Wilson, W. (2002). “To Catch an Error: Without Good Protective Legislation Reporting Medical Mistakes is Risky. So Safety Experts Are Focusing on Close Calls And Other Techniques To Cut Medical Errors.” Medical Post. 38(29): 45. Wong J & Beglaryan H (2004) Strategies for Hospitals to Improve Patient Safety: A Review of the Research, Ontario; The Change Foundation, 2004 Zhan, C & Miller, M (2003) Excess length of stay, charges, and mortality attributable to medical injuries during hospitalization, JAMA, 290 (14), 1868-74.

125

Workshop discussion paper: Root cause analysis: Is it the answer? Research Agenda Track: Parallel Workshops Session 1 Tuesday 25th Sept, 2007.

126

Patient Safety Research shaping the European agenda 24-26th September 2007: Porto, Portugal

WORKSHOP DISCUSSION PAPER

ROOT CAUSE ANALYSIS: IS IT THE ANSWER? Dianne Parker

Professor Dianne Parker, School of Psychological Science University of Manchester

127

1. What is it? Since the mid-1990s there has been growing interest, in many parts of the world, in medical error and patient safety. Several very high profile scandals, prompted the realisation within healthcare that, as in the domain of quality assurance in manufacturing, when there is a failure we need to know what happened, why it happened and how to fix it. As expertise from a range of academic perspectives was brought to bear on the issue, one of the key messages to emerge, from social and organisational psychology, was that in order to understand adverse events in complex organisations, there is a clear need to focus not just on individual errors made by those delivering patient care, but also on any organisational/latent failures that made a contribution. In the famous phrase, it was crucial to have an ‘Organisation With a Memory’. One of the basic requirements of such an organisation is a systematic approach to the investigation of failure. Specifically, in terms of investigating the causes of adverse events, and near misses, the search for a systematic, organisation-wide approach led to the adoption of several formal techniques for analysing adverse events. Root Cause Analysis (RCA) is the umbrella name for a family of techniques that involve a formal investigation of an adverse event or near miss, and that rely on an impartial, analytical approach. Many analytical tools based on RCA have been developed and marketed. Some, such as timelines, accident fault trees and failure event trees focus on the reconstruction of the adverse event, with the intention of clarifying precisely what happened. They are often used together with analyses that attempt to determine why these events occurred. A slightly different approach involves the construction of flowcharts that specify the conjunction of several factors. Popular techniques of this type include MORT (Management Oversight Risk Trees), PRISMA, TRIPOD, MTO, SCART (Systematic Cause Analysis Technique) and STAMP (Systems Theory Accident Modelling and Process). Many of these are software based, commercially available tools. I have to thank my colleague John Ovreteit for letting me know that of these techniques MTO, which stands for Man-Technology-Organisation, is the most widely used in Sweden and Norway. Whatever the details, the basic principles of root cause analysis remain the same. Statistical analysis is undertaken of a body of reported incidents and/or near misses, in order to identify trends and patterns in the data. This analysis allows for the targeting of interventions, and the sensible application of limited resources. The root cause analysis approach brings to light both the active (‘sharp end’) failures and the latent (organisational) failures that conspire to produce an adverse event. RCA is recommended in UK healthcare by the National Patient Safety Agency (NPSA), by the US Joint Commission in Healthcare Safety, and by the States and Territories in Australia, for example New South Wales Health. RCA is widely used. Indeed a comprehensive review of the quality and safety tools used in healthcare, carried out for the Health Education Network of the WHO by John Ovretveit in 2005 (1), found RCA was one of the most widely reported of all safety specific quality tools. Different countries have instituted a range of different ways to persuade healthcare organisations to carry out RCAs when necessary. For example

128

in Denmark a 2004 Act on Patient Safety obligates healthcare professionals in the secondary care sector to report adverse events to a national database (2). All serious adverse events prompt a RCA. In the US, the Joint Commission on Accreditation of Healthcare Organisations (JCAHO) requires all healthcare organisations to perform RCAs and develop action plans following any sentinel events. In Great Britain the NPSA have offered training in RCA to up to eight people in each NHS trust FMEA, the Failure Modes and Effects Analysis tool, is a variation on this theme. Recommended by the US Institute for Healthcare Improvement, FMEA is a type of proactive RCA. To use FMEA you do not have to wait until an adverse event of patient safety incident occurs. The analysis can be carried out in relation to any highrisk process 2. How to carry out a basic Root Cause Analysis The London Protocol (3) was one of the early attempts to move the focus away from the individual and onto organisational factors. It provides a formal protocol that allows a structured approach to thinking about contributory factors at 7 levels: • Institutional • Organisational • Work environment • Team • Individual staff member • Task • Patient An alternative set of factors for consideration are as follows: • Patient factors • Communication factors • Knowledge skills and competence • Work environment and scheduling • Equipment • Policies, procedures and guidelines • Safety defences Although these two sets factors appear very different each one prompts a thorough and systematic review, and consideration of the same incident via either set of factors will probably lead to the identification of similar issues. Whatever organising structure you decide to use, the first step is to collect all available information about the incident under investigation. Cast the net wide drawing on documents, witness statement, interviews, etc. Ask what happened, where and when did it happen, who or what was involved, how did it happen. The second step is to construct a timeline or flowchart to visually depict the incident under investigation. This should help to identify steps in the process where failures occurred. There are likely to several. For each step consider each of the list of possible contributing factors you are using to structure the analysis, and decide whether or not it had a part to play. Use the data you have about the incident to identify contributing factors and causes. These are the issues that will need to be fixed.

129

The next step is key. Having identified all the contributing factors, investigators should draw up recommendation to prevent or minimise the chances of a similar incident happening again. For each recommended action, a responsible person should be identified, and a timeframe for implementation agreed. This is the step at which the process most often fails. Unless people with the power to make decisions and allocate resources are involved, it is unlikely that the actions identified will be closed out, or that their impact will be evaluated. The US perspective At around the same time, Jim Bagian (4) , at the US Department of Veterans’ Affairs National Centre for Patient Safety in Ann Arbor, Michigan, was introducing Root Cause Analysis. Previously, a focused review system had been used to study adverse events. An early evaluation of the new system, published in 2002 (in the Joint Commission’s Journal of Quality Improvement) found that RCA had shifted the focus of investigations away from individual error towards actionable system vulnerabilities. One of the basic differences is that RCA does not settle for the first answer. In fact it is sometimes known as the Five WHYs approach. Here is why… 1. Why did the child get the wrong dose of a drug? The answer might be: because the doctor drew up the wrong amount into the syringe In RCA the next question would be 2. Why did the doctor draw up the wrong amount into the syringe? To which the answer might be: He had not given this drug before and was unable to check the dosage. The next question then would be: 3. Why was he unable to check the dosage? To which the answer might be: His supervisor was not on the ward and he could not find the drug on the hospital IT system. The next question then would be: 4. Why was his supervisor not on the ward, and why could he not find the drug on the hospital IT system? To which the answer might be: The supervisor was covering two wards, and the IT system only gets updated every 12 months. I hope you can see that this relentless probing has lead us away from blaming the doctor who administered the incorrect dose, and lead us into a consideration of staffing arrangements, and the reliability of the IT system. 3. Is RCA the answer? RCA is becoming widely adopted, and has undoubtedly moved the patient safety agenda forward. However recently, research papers have begun to emerge that illustrate that RCA is, unsurprisingly, not the magic bullet some may have initially hoped. A recent Australian paper (5), looked at the recommendations arising from RCAs conducted in an 18 month period in the Sydney area. 59 RCAs had led to 328 recommendations. These were reviewed by 12 doctors and 17 nurses. Results showed that the nurses were more enthusiastic than the doctors about the

130

recommendations, rating them as more relevant, and achievable. Perhaps equally interesting is the fact that of the 29 healthcare professionals taking part in the study, while 24 had themselves taken part in an RCA, less than half (11) had had any training in how to conduct and RCA. Lack of training of patient safety managers in how to conduct an effective investigation was also identified by Evans (6) as contributing to clinicians’ lack of trust in the system and thereby reducing the likelihood of reporting. Again in Australia a study in New South Wales (7) investigated the perceived benefits of a training programme, the NSW Safety Improvement Programme (SIP). They surveyed 252 hcps who had been on the 2-day training course, and found that although the course was felt to be beneficial, in improving the skill set and commitment to safety of those trained, a number of organisational barriers had been encountered by professionals subsequently conducting RCAs. The seven principal barriers identified were (beginning with the most problematic): Lack of time, lack of resources, unco-operative colleagues, lack of data, difficulty with teams, interprofessional differences, and lack of support from management. Respondents indicated that on the whole, recommendations from RCAs were partly , but not entirely, taken up (51%). These findings are echoed by those of Louise Wallace et al (8) who evaluated the 3-day training course in RCA provided by the NPSA in England. Lack of feedback about the learning and outcomes associated with RCAs, and the difficulties experiences by healthcare organisations in implementing change were identified as significant barriers. Rick Iedema Professor of Communication at Sydney, has done some extremely interesting work from a sociological perspective, looking at how clinicians share information about adverse events. Recent papers (9, 10) have analysed transcripts of RCA meetings, and showed how challenging RCA is to clinicians, as it involves close scrutiny of others’ practice and of their errors. This means that they have to develop an understanding of the technical complexities of colleagues work, that they not only resist the traditional pressures of collegiality, but manage to develop trust, respect and confidence in each other. Professor Iedema’s close analysis reveals that RCA teams sometimes have difficulties deriving organisational level generalities from specific clinical situations. He further suggests that the expectation that an RCA will lead to new or improved procedures produces real difficulties in complex and uncertain clinical contexts. There is very little published research on RCA in the European context (a point worthy of discussion in itself) One notable exception is a very recent Danish review of medication errors in community pharmacies (11). The researchers here investigated 401 cases of medication error from 40 community pharmacies in Denmark. Root cause analysis was used, and identified four root causes. These were handwritten prescriptions, error traps such as similarity in the packaging or names of drugs, work interruptions leading to lack of concentration in prescribers, and lack of effective controls such as the use of barcodes, or double checking, that might detect errors. While the authors acknowledge that improvements to systems and technology would lead to improvements in safety, the culture of community pharmacies also deserves attention. In my opinion, it is very unlikely that any root cause analysis, however thorough, would identify culture as a root cause. 4. What more is needed? Weick (12) on sensemaking – aims to build understanding to inform both elimination of risk and hazards. Can work reactively or proactively. Sensemaking provides a

131

framework for bringing together the results of techniques such as RCA, FMEA and PRA. Battles et al (13) suggest that sensemaking is most effective when these fairly mechanistic approaches are supplemented by input from the ‘experts’ – those involved at the shop floor, who can ‘make sense’ of the risks and hazards identified,, and assume ownership of, any agreed actions. 5. The way forward 1. RCA should be embedded in an integrated safety management system (SMS). This has been suggested in a recent paper published by Bill Runciman and colleagues (14). They point out that a lot of good work on patient safety is being done in many healthcare organisations but the approach is usually fragmented. They point to the World Health Organisation’s World Alliance for Patient Safety as providing a possible platform for developing an integrated approach that covers safety, quality and risk management in healthcare. Two of the Alliance’s first set of initiatives involved developing a patient safety taxonomy, and learning from reported adverse events. Runciman suggests that taking the output of these two initiatives together could provide the basis for the integrated approach he advocates. 2. Performing root cause analyses can be a significant burden, and a full analysis may not always be necessary for less serious events. Research should be carried out to develop an analytical method that would provide the same level of insight, but represent a lighter touch. At the same time, the development of a common method for recording the results of root cause analyses would allow comparisons to be made more easily. 3. Waring and colleagues, based in Manchester and Nottingham in the UK have also taken a sociological approach and suggest that RCA is one example of the regulation of medicine to achieve patient safety (15). They suggest that the development of expertise in such techniques allows managers to scrutinise medical performance in a way previously not possible. Doctors may actively resist this change and look for ways to subvert the process, in order to maintain their self-regulatory independence, and limit the power of managers. Given that the reform of healthcare is on the agenda in many countries, the tension between managers and medics is likely to increase. Research is needed to investigate ways in which this tension can be managed to produce a fruitful co-operation. If the results of RCAs are to be accepted as credible by clinicians, those carrying out the investigations must have both clinical expertise and political negotiating skills. 4. The suggestions for research made above require considerable investment and time, and it is unlikely that the way in which RCA is used will change in the near future. Therefore, it is imperative to ensure that there is proper training, and organisational support for implementations of solutions generated. Without proper resourcing and top-level backing, those who carry out RCAs will soon become disillusioned and the system will fall into disrepute. It is difficult enough to win the trust of healthcare professionals and persuade them that the aim of RCA is not to find someone to blame and shame. If RCA is instituted and then the results are not acted upon, any goodwill generated will soon dissipate, and RCA will come to be seen as a pointless paper exercise. Wallace and colleagues (6) specifically suggest an international project to identify best practice in RCA given the resource constraints typically experienced in healthcare organisations.

132

References 1.

World Health Organisation World Alliance for Patient Safety (2005) WHO Draft Guidelines for adverse event reporting and learning systems. Geneva: WHO

2.

World Health Organisation Europe Health Evidence Network (2005) What are the advantages and limitations of different quality and safety tools for health care? Copenhagen: WHO

3.

Vincent, C, Taylor-Adams, S, Chapman, EJ, Hewett, D Prior, S, Strange, P and Tizzard, A (2000). How to investigate and analyse clinical incidents: Clinical Risk Unit and Association of Litigation and Risk Management Protocol. British Medical Journal, 320, 777-781

4.

Bagian, JP, Gosbee, J, Lee, CZ… (2002) The Veterans Affairs root cause analysis system in action. Journal of Quality Improvement, 28, 531-45

5.

Middleton, S, Chapman, B, Griffiths, R, Chester, R. Reviewing recommendations of root cause analyses. Australian health review: a publication of the Australian Hospital Association. Vol. 31, no. 2, p. 288-95.

6.

Evans S, Smith B, Esterman A, Runciman W, Maddern G, Stead K, Selim P, O’Shaughnessy J, Muecke S, Jones S. Evaluation of an intervention aimed at improving voluntary incident reporting in hospitals. Quality and Safety in Health Care 2007;16:169-175

7.

Brathwaite, J, Westbrook MT, Mallock, NA, Travaglia, JF and Iedema, RA (2006) Experiences of health professionals who conducted a root cause analyses after undergoing a safety improvement programme. Quality and Safety in Healthcare,15, 393-399.

8.

Wallace, L, Spurgeon, P and Earl. L (2006) Evaluation of the NPSA’s 3-day RCA programme. Report to Department of Health Patient Safety Research Programme. Coventry University, UK.

9.

Iedema, RAM, Jorm, C, Long, D, Braithwaite, J, Travaglia, J and Westbrook, M (2006) Turning the medical gaze on itself: Root cause analysis and the investigation of error. Social Science and Medicine, 62, 1605-1615.

10.

Iedema, RAM, Jorm, C, Braithwaite, J, Travaglia, J and Lum, M (2006) A root cause analysis of clinical error: Confronting the disjunction between formal rules and situated clinical activity. Social Science and Medicine, 63, 1202-1212.

11.

Knudsen, P, Herborg, H, Mortensen, AR, Knudsen, M and Hellebek A (2007) Preventing medication errors in community pharmacy: Root cause analysis of transcription errors. Quality and Safety in Healthcare, 16, 285-290

12.

Weick, K (1995) Sense-making in organisations. London: Sage.

13.

Battles, JB, Dixon, NM, Borotkanics, RJ, Rabin-Fastmen, B and Kaplan, HS (2006) Sensemaking of patient safety risks and hazards. Health Services Research, 41, 1555-75

14.

Runciman W, Williamson J, Deakin A, Benveniste K, Bannon K, Hibbert P. An integrated framework for safety, quality and risk management: an information

133

and incident management system based on a universal patient safety classification. Quality and Safety in Health Care 2006;15(Supplement 1 ):i82i90 15.

Waring, J, McDonald, R. and Harrison, S. (2007) Adaptive regulation or governmentality: patient safety and the changing regulation of medicine. Sociology of Health & Illness, 29, 163-179

134

Workshop discussion paper: Can we control hospital infections? Research Agenda Track: Parallel Workshops Session 2 Tuesday 25th Sept, 2007.

135

Patient Safety Research shaping the European agenda 24-26th September 2007: Porto, Portugal

WORKSHOP DISCUSSION PAPER

Can we prevent Healthcare-Associated Infections? Hervé Richet, MD1, Hugo Sax MD2

Hervé Richet, MD 1) Global Patient Safety Challenge, World Health Organization (WHO) World Alliance for Patient Safety, WHO Headquarters, Geneva, Switzerland and Hospital Epidemiology Unit, La Timone Hospital, Marseille, France. Hugo Sax MD 2) Infection Control Program, University of Geneva Hospitals, Geneva, Switzerland Corresponding author: Prof. Hervé Richet Unité des Rickettsies, Faculté de Médecine, Université de la Méditerranée 27 Boulevard Jean Moulin, 13385 Marseille cedex 05, France E-mail: [email protected] Tel: 33 (0) 4 91 32 44 80 Fax: 33 (0) 4 91 38 77 72.

136

INTRODUCTION The prevention of infections has been a preoccupation for mankind all along his history due to the disastrous outcome of infections until the discovery of antimicrobials. The human disasters associated with epidemics or pandemics had challenged our ancestors to find ways of preventing the spread of infections much before micro-organisms were identified and their modes of transmission analyzed and they indeed succeeded. The most effective preventive measures, still in use nowadays, such as isolation precautions, hand hygiene, sterilization, and antisepsis were developed long before or just shortly after micro-organisms were identified. All these discoveries have had an important impact on the prevention of healthcare-associated infections and, for instance, Joseph Lister with the introduction of antisepsis during surgical procedures was able to significantly reduce the mortality rate of surgical interventions from 46% to 15%. In turn, this reduction in mortality rate enabled surgeons to perform longer and more complex surgical procedures. Of interest, one of the most difficult challenges faced by hospital epidemiologist today is often less to invent new preventions or control methods than to find ways to make sure that old methods that have shown their efficacy, such hand hygiene are applied. However, many factors interfere with the prevention of infections. They include, just to cite a few, the emergence of more virulent micro-organisms, the progress of antimicrobial resistance, the development of more aggressive therapeutic procedures and a population of hospitalized patients with more frequent impaired immunity due to age, illness and treatments. In this article, we will review the zones of uncertainties surrounding HCAI and what research programs should be funded by the EU in order to optimize their prevention knowing that the list of potential research topics presented in this article is far from being exhaustive and should serve only as a basis for discussion. This article is divided into three parts, the first presents the framework for research programs, the second is a table showing in more details some questions that should be addressed as well as some examples, and the third shows all category IA recommendations present in the Centers for Disease and Prevention guidelines. Because they are strongly recommended for implementation and strongly supported by well designed experimental, clinical, or epidemiologic studies they give evidence of the research done in the field of prevention of HCAI. A. WHAT DO WE KNOW ABOUT THE FREQUENCY OF HCAI 1. Frequency of HAI and burden of diseases The European Centre for Disease Prevention and Control (ECDC) in its First European Communicable Disease Report states that according to the recent national HCAI prevalence surveys in Europe and based on the results of hospital-wide surveillance programmes of nosocomial bacteremia in different EU member states, the total number of patients acquiring HCAI in the EU every year can be estimated at 3,000,000. Approximately 50,000 deaths are estimated to occur every year as a consequence of the infection. The report also tells that several EU Member States still do not have a national surveillance network since such programs requires a financial investment at the national and hospital level for setting up or reinforcing infection control programmes including surveillance. In addition, in a table showing a summary of general trends (1995-2005), EU incidence (2005),

137

main age groups affected (2005), and major threats detected (2005) for diseases reported on EU level, the results regarding HCAI are as follow: - General 10-year trends: no reliable data - EU incidence par 100,00: not applicable - Main age groups affected: no data - Major threats monitored/detected: 0 (1). This report clearly shows a conflicting situation regarding the frequency of HCAI in the EU member states since it appears that: 1. Even if no reliable data exist, HCAI may be the most frequent communicable diseases and top more “visible” infectious diseases such as HIV (77,553 new cases in 2005) (2), or tuberculosis (426,717 in 2005 in the WHO European Region) (3) without generating the same public health response, 2. HCAI are severe diseases probably associated with high mortality 3. Reliable data do not exist at the EU level, 4. Early warning systems do not formally exist since no major threats were either monitored or detected. Representatives from national surveillance networks have worked together in the Hospitals in Europe Link for Infection Control through Surveillance (HELICS) to analyze inter-country differences and work toward comparable surveillance methods. Data are being collected in the fields of surveillance of intensive care unit (ICU) infections in 724 ICUs from 7 EU member states and of surgical site infections in 642 hospitals from 12 EU member states. However, these surveillance programs involve a subset of all HCAI and a major research item would be to organize the collection of reliable data on the whole spectrum of HCAI in all the varieties of healthcare settings enabling to perform research on frequency and causes of all types of infection and their impact on public health that could be used by public health authorities to define prevention strategies, to enable benchmarking and to be used as evaluation tools. Most of the work on surveillance has been done in ICU patients and in surgical site infections where potential confounding factors may be controlled by using score of severity of illness or other modes of stratification of patients according to the risk of infection. Such scoring systems do not exist for patients admitted in services other than ICU and there is a need to develop severity of illness scoring systems for non-ICU patients. Another major research item would be to assess the burden of HCAI by evaluating the cost, morbidity, functional health, quality of life, and mortality associated with these infections. Cost related to surveillance, control and prevention of HCAI should also be evaluated to demonstrate cost effectiveness of infection control activities and encourage EU member states or hospitals to invest in infection control activities and hospital epidemiology. This is especially important at a time when the modes of HCF financing are changing with the implementation in EU member states, such as France, of reimbursement linked to the activity of the facility or to diagnosis related groups (DRG). Such research programs should be multidisciplinary involving epidemiologists and economists. An early warning and response system (EWRS) has been set up under a decision of the European Parliament in 1998 in order to establish permanent communication between European Union (EU) Member States’ public health authorities, which are responsible for determining the measures required to control communicable disease-related events. Between 1998 and December 2005, a total of 583 messages were circulated through the EWRS, notifying 396 events. However, none of these messages involved HCAI (4).

138

Therefore, an early warning system dedicated to HCAI could be implemented. Over the last thirty years, deaths from disasters have been declining, in part thanks to the role of early warning systems and associated preparedness and response systems. To be effective, early warning systems must be people-centred and must integrate four elements - (i) knowledge of the risks faced; (ii) technical monitoring and warning service; (iii) dissemination of meaningful warnings to those at risk; and (iv) public awareness and preparedness to act (5). 2. Factors influencing the occurrence of HCAI a. Patients characteristics The last national prevalence survey performed in France in May/June 2006 provides important information about the characteristics of the population of hospitalized patients. Among the 358,353 patients hospitalized in the 2,337 healthcare facilities participating in the survey, the median age was 69 years, 55.7% were older than 65 years and 18.4% older than 85 years. In addition, 9.5% of the patients were immunosupressed and 29.2% of them had a MacCabe severity score of 1 or 2, which predict a fatal issue within the following 1 and 5 years, respectively. All these factors have an impact on the occurrence of HCAI since the prevalence of these infections was 1.8 times higher in patients older than 65 years (6.1 vs. 3.5), 4.13 times higher in patients with a MacCabe score of 2 (13.15% vs 3.2%) for a score of 0), and 2.47 times higher in patients with impaired immunity (6). The fact that public healthcare facilities care for severely ill patients is also shown by mortality data. In 2005 in France, 49.3% of the persons who died (219,975/445,810), died in public hospitals while only 8.3% died in private hospitals. These data are important because they suggest that all HAI may be not preventable due in part to the severity of the diseases of hospitalized patients as well as to the presence in HCF of large number of end of life patients who are subjected to invasive procedures and/or treatments. The EDC’s annual report on infectious diseases also mentions that approximately 20-30% of HCAI are considered to be preventable by intensive infection prevention and control programmes including surveillance. Harbarth et al have performed a systematic review of articles published during the last decade, in order to obtain a crude estimate of the proportion of potentially preventable nosocomial infections. Their evaluation of 30 reports suggests that great potential exists to decrease nosocomial infection rates, from 10% to 70%, depending on the setting, study design, baseline infection rates and type of infection. The most important reduction effect was identified for catheter-related bacteraemia, whereas a smaller potential for prevention seems to exist for other types of infections. Based on these estimates, the authors consider that at least 20% of all nosocomial infections as probably preventable. (7). A major research project should be to assess what infections in what type of patients are preventable, to quantify the non-preventable infections and to evaluate what makes those infections non-preventable. Are they due to severely impaired immunity or to the type of care provided? An additional question would be to assess if acute care facilities are the best equipped to treat end of life patients. b. Evolution of micro-organisms Unexpected changes have taken place in HCF worldwide regarding the epidemiology of micro-organisms responsible for HCAI. For instance, in the late 1990s and early 2000s, experts in the field of antimicrobial resistance were convinced that methicillin-resistant Staphylococcus aureus (MRSA) were headed toward pan-drug resistance as suggested by the emergence of S. aureus with decreased susceptibility to glycopeptides (GISA) (8).

139

A survey performed in 90 healthcare facilities worldwide in 1998 for the International Network for the Study and Prevention of Emerging Antimicrobial Resistance (INSPEAR) showed that indeed pan-drug resistance was possible because the median rates of resistance of 82% for kanamycin, 81% for tobramycin, 58% for gentamicin, 76% for erythromycin and clindamycin, 60% for tetracycline, and 74% for ofloxacin (9). However, what happened instead was the emergence of MRSA strains much less resistant and more virulent. As an example in the Assistance Publique – Hôpitaux de Marseille facilities (France), a tertiary care referral center, MRSA strains susceptible to all antimicrobials (with the exception of methicillin) represented 5.5% (481/878) of the strains isolated in 2001 and 12.7% (130/1025) in 2006. In addition, a preliminary study seeking to assess the mortality associated with infections caused by the more susceptible strains showed that the attributable mortality was 32% in ICU patients (mortality in MRSA patients: 45% - baseline mortality rate: 13%). (Richet H. personal communication). This is not surprising considering the fact that Durand et al. have detected new MRSA clones containing the toxic shock syndrome toxin 1 gene. Those strains were isolated from hospital- and community-acquired infections, mostly in children, and caused a variety of clinical syndromes, including toxic shock syndrome (10). The same trend has been observed all over the world due to the emergence of MRSA in the community followed by its introduction and spread within healthcare facilities. Studies also indicate that the massive geographic spread of MRSA results from the dissemination of relatively few epidemic clones (11). The emergence of these new clones has also an impact on the prevention of HCAI by making it more difficult. The case of Denmark can be used as an example. This country had kept the incidence of MRSA infections very low since the 1970s because of the implementation of strict infection control measures. However, the number of MRSA strains belonging to the USA300 clone has increased several-fold in Denmark since 2003. The infection control measures used in Denmark, which had been very effective until the emergence of the new clones of MRSA, seems to be less effective for preventing the spread of the new clones. The authors of this article conclude by writing that with the US experience in mind, this is of great concern, especially since this is observed in a country with a long reputation for controlling MRSA (12). Unfortunately, most of the research done in the field of MRSA infection has been microbiological, including molecular epidemiology, multi locus sequence typing, analysis of the mec gene, antimicrobial resistance, research of toxins. As a consequence much knowledge has been acquired on the strains and on microbial epidemiology but very few is known on the modes of transmission of the strains, the clinical epidemiology and outcome of the infections caused by the new clones, and the design and evaluation of infection control measures. To investigate such matters should be a priority in regards of its potential public health impact in the EU. To do such studies at the EU level is of great importance because all the EU member states are involved due to the circulation of the MRSA clones from a country to another.

B. WHAT CAN BE DONE TO IMPROVE THE PREVENTION OF HCAI In this chapter we will focus on a prevention measure, hand hygiene, as an example and on the prevention of two infections; urinary tract infections and catheter-related infections. We will not cover surgical wound infections and pneumonia because those infections are the one for which European networks exist and most research has been performed or is on-going.

140

1. Hand hygiene Hand hygiene has been shown to the most important measure for preventing the spread of pathogens in healthcare settings by Ignaz Semmelweis in Vienna and Oliver Wendell Holmes in Boston in the mid-1800s when they established that hospital-acquired diseases, now known to be caused by infectious agents, were transmitted via the hands of HCWs. Despite the fact that hand hygiene is among the few prevention measures ranked IA (Strongly recommended for implementation and strongly supported by well designed experimental, clinical, or epidemiologic studies) by the Centers for Disease Control and Prevention guidelines (cf annex), adherence of HCWs to recommended hand hygiene procedures has been unacceptably poor, with mean baseline rates ranging from 5% to 81%, with an overall average of about 40%(13). Why is this so? What can be done to induce improvement? Motivation to improve is always especially weak if the perceived benefices of certain behaviour are deferred in time and/or are not readily manifest. This applies for most preventive topics. For hand hygiene, however, an additional barrier to compliant behaviour is the fact that good compliance does benefit to the actor him/herself but to a stranger, the patient. Current knowledge in determinants for good hand hygiene behaviour comes from a handful of studies applying behaviour psychology. Thus, more research is needed to understand the crucial factors that are in the way of high compliance with what would seem logic behaviour. This issue has been addressed by the WHO World Alliance for Patient Safety and the following recommendations have been made in order to promote research programs aimed (1) at improving hand hygiene compliance: • Assess the key determinants of hand hygiene behaviour and promotion among the different populations of healthcare workers (HCWs) • Develop methods to obtain top management support • Implement and evaluate the impact of the different components of multimodal programmes to promote hand hygiene • Assess impact on hand hygiene compliance and untoward consequences of patient involvement in hand hygiene promotion (2) Improving the knowledge on microorganisms’ transmission and monitoring and effectiveness of hand hygiene: • Develop experimental models for the study of cross-contamination • Monitor hand hygiene adherence by using new devices or adequate surrogate markers , allowing frequent individual feedback on performance • Determine the percentage increase in hand hygiene adherence required to achieve a predictable risk reduction in infection rates • Generate more definite evidence for the impact on infection rates of improved adherence to recommended hand hygiene practices • Provide cost-effectiveness evaluation of successful and unsuccessful promotion campaign (13). 2. Prevention of urinary tract infections There is much complacency about healthcare-associated urinary tract infections (HCAUTI) both in the medical and infection control communities. More than 25 years ago the authors of the CDC Guideline for Prevention of Catheter-Related Urinary Tract Infections wrote that the urinary tract was the most common site of nosocomial infection, accounting for more than 40% of the total number reported by acute-care hospitals and affecting an estimated 600,000 patients per year. Most of these infections--66% to 86%--followed instrumentation of the urinary tract, mainly urinary catheterization. Although not all catheter-associated urinary tract

141

infections can be prevented, it is believed that a large number could be avoided by the proper management of the indwelling catheter (14). Twenty five years later the same report could be made because the situation has not changed. HCAUTI remain at the same time the most common and preventable healthcare-associated infections, although the effective infections control measures are well known. Another sign of this neglect is shown by the following facts. The Guideline for Prevention of HCAUTI is the only CDC guideline which has not been updated and a Medline search using as key words “prevention” “nosocomial” “urinary tract infections” retrieved only three hundred and thirty three publications testifying of the very limited amount of research performed in this field. Across all services UTI are the most common HCAI accounting for 30.3% of all HCAI in the 2006 National French Prevalence Survey. In the same survey, 6.2% of the patients (22,259) had an indwelling urinary catheter (IUC) on the day of the survey and 3.2% of the patients had such device during the 7 days preceding the survey. The prevalence rate of nosocomial infections was 19.7% among patients who had IUC on the day of the survey and 13.5% among patients who had IUC during the previous 7 days. The risk of having HCAI was respectively 4.4 and 3 times higher in patients with IUC on the day of the survey and during the previous 7 days. In the US, it is estimated that urinary tract catheterization is performed 7 to 8,000,000 times a year in acute care hospitals. Approximately 5% to 8% of catheterized uninfected patients will acquire HCAUTI for each day of catheterization leading to a cumulative rate of 40% after 10 days. In addition, a study performed in 1,458 patients demonstrated that mortality was higher in catheterized patients who acquired bacteriuria than in those who did not. By extrapolation as many as 56,000 deaths a year in the US could be related to catheteracquired UTI (14). All these data, plus the fact that HCAUTI is a major reservoir of multidrugresistant organisms should make the prevention of these infections a priority. However, this is not an easy task because many risk factors are involved including host factors such age, gender, underlying diseases, the use of invasive devices, improper aseptic techniques, the types of devices used, faulty aseptic management of the indwelling catheter and behavioural factors. Multimodal prevention strategy seems to be a promising field of study as shown by a study published in August 2007 that is shown as an example of such approach. The authors of this study developed a project with qualitative and quantitative goals. The qualitative goals were to promote a culture of healing, to promote teamwork, and to achieve optimal clinical and financial outcomes for patients. The quantitative goals were to reduce the adverse events per ICU days by 50%. The strategy for change involved four changes: physician led multidisciplinary rounds, daily bed flow meetings, bundles, and culture change. The UTI bundle included regular assessment of continued need for catheter, sterile techniques of insertion, perineal care daily, drainage bag lower than patient’s bladder at all times, secure all catheters, and use of silver coated catheters in selected cases. As a result of this strategy, adverse events and HCAI declined following the introduction of changes in the system of care. The authors cannot surmise which of the four elements contributed to the change. However, the bundles approach facilitated the active implementation of evidence base medicine. The bundles provided for completeness, consistency, and application of evidence based medicine interventions by acting as a reminder system (16). Doyle et al have summarized the research projects to conduct in order to improve prevention of HCAUTI. They include: • Evaluation of institutional barriers to change in practice patterns • Evaluation of culture change required to decrease catheter use

142

• • •

Evaluation of costs Measurement of outcomes How to achieve sustainability (17).

3. Prevention of healthcare-associated bloodstream infections (HCBI) Wenzel et al. estimate that the number of HCBI occurring each year in the US would range between 87,500 and 350,000 and would represent the eight leading cause of death in the US (18). The NNIS study has shown that primary bloodstream infections were the most common infections in paediatric intensive care units in the US and those coagulase negative staphylococci were the most common pathogens reported representing 38% of all isolates (19). Studies have repeatedly shown that various factors may influence the risk of developing catheter-related infections including host factors such age, severity of illness, skin disinfection, difficulties during insertion, parenteral nutrition, blood product transfusion duration of use, poor patient hygiene Despite a relatively large number of randomized controlled trials performed, the routine use of antimicrobial impregnated catheters as prevention against catheter-related infections (CRI) remains controversial and more studies needed (20). As with other HCAI like HCAUTI, changing the behaviour of healthcare-workers who insert and care for intravascular catheters needs to be evaluated as well as multimodal preventive strategies directed at risk factors for CRI (21). CONCLUSIONS AND RECOMMANDATIONS OF THE PARTICIPANTS There are gaps in knowledge, and in the implementation of what is already known resulting in risks both to individuals and to systems. International multi-disciplinary research focusing on these two areas is needed. Short-term goals should include work on indicators and benchmarks, pathogens, and process measures. Medium-term goals should include the implementation of an early warning system within Europe to identify both problem pathogens and organisms that are considered less pathogenic Long-term goals should include sustainability and education, with behaviour change as a key issue

143

REFERENCES 1. The First European Communicable Disease Report. http://www.ecdc.eu.int/pdf/Epi_report_2007.pdf 2.

HIV / AIDS Surveillance in Europe End-year report 2005 - 2006, No. 73. http://www.invs.sante.fr/publications/2006/eurohiv_rapport73/report_eurohiv_73.pdf

3.

Surveillance of tuberculosis in Europe, EuroTB. http://www.ecdc.eu.int/documents/pdf/EuroTB_report.pdf

4.

Guglielmetti P, Coulombier D, Thinus G, Van Loock V, Schreck V. The early warning and response system (EWRS) for communicable diseases in the EU: an overview from 1999 to 2005. Euurosurveillance 2006; 11:215-220

5.

Global Survey of Early Warning System. Final Version. A report prepared at the request of the Secretary-General of the United Nations. http://www.unisdr.org/ppew/info-resources/ewc3/Global-Survey-of-Early-WarningSystems.pdf

6.

Enquête nationale de prevalence des infections nosocomiales, juin 2006. Résultats préliminaires, 12 janvier 2007. http://www.invs.sante.fr/publications/2007/enp2006_resultats_preliminaires/

7.

Harbarth S, Sax H, Gastmeier P. The preventable proportion of nosocomial infections: an overview of published reports. J Hosp Infect 2003;54:258-266

8.

Hubert SK, Mohammed JM, Fridkin SK, Gaynes RP, McGowan JE Jr, Tenover FC. Glycopeptides-intermediate Staphylococcus aureus: evaluation of a novel screening method and results of a survey of selected U.S. hospitals. J Clin Microbiol. 1999;37:3590-3

9.

H. Richet, M. Benbachir, D.F. Brown, H. Giamarellou, I. Gould, M. Gubina, P. Heczko, S. Kalenic, M. Pana, D. Pittet, S. Ben Redjeb, J. Schindler, C. Starling, M. Struelens, W. Witte, W. Jarvis. How do health-care facilities worldwide diagnose, conduct surveillance for, and control methicillin-resistant Staphylococcus aureus? Infection Control and Hospital Epidemiology 2003;24:334-341

10.

Durand G, Bes M, Meugnier H, Enright MC, Forey F, Liassine N, Wenger 1, Kikuchi K, Lina F, Vandenessch F, Etienne J. Detection of new methicillin-resistant S. aureus clones containing the toxic shock syndrome toxin 1 gene responsible for hospital- and community-acquired infections in France. J Clin Microbiol 2007;45:2881-2888

11.

Stefani S, Varaldo PE. Epidemiology of methicillin-resistant S. aureus in Europe. Clin Microbiol Infect 2003;12:1179-1186

12.

Larsen AR, Stegger M, Goering RV, Sørum M, Skov R. Emergence and dissemination of the methicillin resistant Staphylococcus aureus USA300 clone in Denmark (2000-2005). Euro Surveill. 2007;12:28-9. Available from:http://www.eurosurveillance.org/em/v12n02/1202-222.asp

13.

WHO World Allaince for Patient Safety. WHO Guidelines on Hand Hygiene in Health Care (Advanced Draft). World Health Organization 2006. http://www.who.int/patientsafety/events/05/HH_en.pdf

14.

Wong ES, Hooton TM. Guideline for Prevention of Catheter-Related Urinary Tract Infections. http://www.cdc.gov/ncidod/dhqp/gl_catheter_assoc.html#

15.

Dellinger EP. Nosocomial infection: Discussion. Clinical Review. ACS Surgery Online 2002

16.

Jain M, Miller L, Belt D, King D, Berwick DM. Decline in ICU adverse events, nosocomial infections and cost through a quality improvement initiative focusing on temwork and culture change. Qual Saf Health Care 2006;15:235-239.

144

17.

Doyle B, Mawji Z, Horgan M, Stillman P, Rinehart A, Bailey J, Mullin E. Decreasing nosocomial urinary tract infection in a large community hospital. Lippincott’s Case Management 2001;6:127-136

18.

Wenzel RP, Edmond MB. The impact of hospital-acquired bloodstream infections. Emerg Infect Dis 2001;7:174-177

19.

Richards MJ, Edwards JR, Culver DH, Gaynes RP. Nosocomial infections in Paediatrics Intensive Care Units in the United States, Pediatrics 1999;104:e39.

20.

Gastmeier P, Zuschneid I, Geffers C. Antimicrobially impregnated catheters: An overview of randomized controlled trials. J Vasc Acess 2003;4:102-110.)

21.

Juan-Torres A, Harbarth S. Prevention of primary bacteremia. Int J Antimicrobial Agents 2007

145

Workshop discussion paper: What role for safety cultures? Research Agenda Track: Parallel Workshops Session 1 Tuesday 25th Sept, 2007.

146

Patient Safety Research shaping the European agenda 24-26th September 2007: Porto, Portugal

WORKSHOP DISCUSSION PAPER

The role of patient safety culture Cordula Wagner

Cordula Wagner NIVEL and EMGO/VUmc EMGO Institute Department of Public and Occupational Health VU University medical center Van der Boechorststraat 7 1081 BT Amsterdam

147

1

Introduction

In the last decade our health care services have become more complex. Technologically, there are a lot more possibilities. Diseases that could not be treated in the past can be treated nowadays. Thanks to improvements in health care services people live longer and live healthier. To guarantee good quality of care, health care organizations in European countries are more or less obliged to implement continuous quality improvement and quality management systems for monitoring and evaluation. Despite of that, there is growing evidence of various countries that things go wrong and that our health care system is not as safe for patients as it could be. Therefore, patient safety is a critical component of health care quality. Patient safety can be defined in different ways, however, by patient safety, we mean that there is (practically) no chance that the patient suffers physical and/or psychological injury due to the failure of health providers to deliver care according to professional standards and/or to shortcomings of the health care system 1 2 3. As health care organizations continually strive to improve, there is a growing recognition of the importance of establishing a culture of patient safety. Achieving a culture of patient safety requires an understanding of the values, beliefs, and norms about what is important in an organization and what attitudes and behaviours related to patient safety are supported, rewarded and expected. The aim of this paper is to give background information, based on existing literature and evidence, about the relation between safety culture and actual patient safety in health care, e.g. more specific in hospitals. This information will be used for discussion during the Portoworkshop about safety culture. The aim of the Porto-workshop is to develop a research agenda for the next five years. The central questions of the workshop are: •

•

Do we need more research to investigate the meaning of ‘safety culture’? (e.g. in western and non-western countries) Or, do we know what safety culture means in various countries and for various professional groups? Do we need more research on safety culture assessment instruments? Or, can we agree on using one or two instruments that makes comparisons possible?

•

Do we need more research into safety cultures that have the most positive impact on patient safety? Is there one best safety culture for every organisation?

•

Do we need more research to get more profound insight into the ways we can change the existing culture to improve patient safety?

•

What kind of research is needed to get more evidence for the relation between patient safety and culture?

148

2

The relation between patient safety and culture

2.1 Patient safety: adverse events around the world For years now, there is evidence that the safety of patients in hospitals can be improved. The first article reporting on the incidence of adverse events in hospitals has been published in 19914. Other countries, e.g. Australia, New Zeeland, Canada, the UK, France, Denmark and the Netherlands have replicated the Harvard Medical Practice study and came up with similar results5 6 7 8 9 10 11. The results show that between 2,9% and 16,6% of patients admitted to a hospital suffer some kind of unintentional harm. Nearly half of the adverse events may have been avoidable. Adverse events can result from failure to follow accepted standards by health care providers or shortcomings in the health care system. Besides the negative consequences for patients there are also negative consequences for health care providers or organizations themself. Patient safety campaigns As soon as the character, seriousness and extent of adverse events are common knowledge, a feeling of urgency evolves resulting in campaigns for improving patient safety being started. This is the case in the United States of America12, Australia, Canada, the United Kingdom, Denmark and the Netherlands. It usually concerns programs of several years, because a change in culture is also needed. Therefore, in most countries the culture in hospitals has to change from a blame and shame culture (looking for who is to blame) to a blame-free culture focused on improving processes and systems. In general, patient safety improvement comprises activities for the identification, analysis and management of patient-related risks and incidents, in order to make patient care safer and minimise harm to patients1. Other high-risk industries have integrated these activities into a safety management system. Based on their experience, safety improvement can only be successful if an open safety culture supports the improvement activities and if leaders are actively involved and responsible for the results. 2.2 Causes of adverse events At first sight, most adverse events are caused by human action or, just the reverse, lack of action. Further analysis shows that these adverse events can be the result of a care process or system that has not been set up rightly13 14, which is, in turn, influenced by the organisation of the health care system, current law and regulations and the demands of parties, such as care insurers and patients. The prevailing opinion in literature is that people will always make mistakes which can cause adverse events15. In an organisation culture that enables people to report adverse events without being blamed for it, it is possible to learn from mistakes16. Adverse events could be prevented more effectively if the process or system would be set up differently, in such a way that chances for making mistakes are reduced (for example electronic prescriptions instead of handwritten ones), or these mistakes are tackled before the patient is harmed by it17.

149

Complexity and high-reliability The number of adverse events caused by failure of material or technical equipment has decreased during the past few decades, in contrast with the number of adverse events in which human factors play a role18. As a result, people are now inclined to focus on persons involved in the adverse events (sharp end professionals). Frequently, health care consists of a care process with several links that are interdependent in order to get the best result. For example, a surgeon who is operating on a patient is dependent on the anaesthetists and the operation nurse. The operating team is dependent on a good and clean operating theatre with reliable equipment and materials. Additionally, the recovery room, the hospital department and the support services in the hospital are important for good care before and after the operation (blunt end). However, the hospital is not self-supporting and is, in turn, dependent on personnel in primary care and third line medicine, both before hospitalisation and after discharge. This interdependency causes the adverse event to develop not always at the point of surgery, but often days, weeks or even months before the operation. Failures within care processes, manifest as ‘breakdowns’ within inter-departmental relationships, lead to situations of constraint, rapid change and uncertainty in the work of for example the operating staff. To cope with these situations staff have to break with established routines or work under increased time and emotional pressures. The focus will probably shift from working safely to working quickly. Creating safe processes depend on working together and looking beyond the scope of departmental management19. Chances to prevent adverse events just in time depend also on the safety structure within the organisation and the extent to which sufficient barriers are placed in the care processes. Small process deviations at the beginning of the chain can lead to large process deviations and adverse events at the end of the chain, unless deviations are spotted in time. That is why extra attention for the culture and context in which adverse events occur is often pleaded20 21. To improve patient safety it is generally assumed that an open and pro-active safety culture is needed in hospitals.

3

Safety culture: background, concept and definition

3.1 Growing interest in safety cultures The notion of organizational culture has long before been mentioned in the management literature as a crucial variable for organizational performance. Nevertheless, cultural change needs to go along with structural reorganisation. Culture reflects the mental program of people within one group. This mental program distinguishes one group from another. It includes the shared beliefs, attitudes, values and norms of behaviour22. One expression of a group is the way team members talk about their team. Discourse analysis can help to explore the discursive patterns within teams, and more specific the use of the pronouns we, they and I. Members of teams may use the pronoun ‘we’ to claim superiority in relation with the surrounding community (the others)23.

150

Cultures are visible in common symbols, hero’s, rituals and values. Symbols are words, gestures, images or objects with a meaning that is only understood by members of that culture. Hero’s are persons that are highly esteemed by the group, and rituals are common activities that are in fact not necessary to achieve the objective of the organisation. Ethnographic research in the operating theatre has shown that surgeons and anaesthetists have ritualistic behaviour to normalize risks. These behaviours lead to the possibility that some threats will escape appropriate attention and may lead to patient harm. It’s a safety paradox: routine decreases risks for patients, but, on the other hand acting routinely inhibits learning and increases the chance that incidents occur again24. The core of a culture consists of values, standards, views and matters that are taken for granted 25. Values are feelings with a plus pole or minus pole, like good versus bad, beautiful versus ugly, normal versus abnormal. These values and standards determine the behaviour of the staff, visible in the way people communicate, what people do or choose not to do, the way people are managed26. There are competing interpretations of what culture is27. In general there are two broad schools. One that regards culture as something intangible, something that an organization is. The other assumes that the culture of an organization can be derived into various aspects or variables. This makes it possible to create, change, and manage culture in the pursuit of wider organizational objectives. The same distinction can be made with regard to safety culture. Safety culture consists of the norms, values and views of the members of a specific group (e.g. units, nurses or medical specialists) with regard to the key elements of patient safety (e.g. error disclosure, reporting adverse events, risk analysis). The concept of safety culture is derived from industries and organizations such as aerospace (NASA), nuclear power, aviation, and the military that are known for their ability to reliably deal with risky processes. One important driver of safety is the very explicit commitment to safety by leaders28. Other components of a good safety culture include a focus on system improvement instead of blaming individuals for mistakes, reporting and learning from errors, and infrequent unsafe acts, attitudes about teamwork (e.g. the difficulty to speak up, not resolving disagreements, input in decision making), communication between health care providers, group cohesion. A defective safety culture was highlighted as one of the organizational causes of big incidents in hospitals in for example the Netherlands. Hospitals are feeling pressure to act to improve the safety related attitudes that are part of safety culture. The purpose of, for example executives’ walk rounds, teamwork training and culture assessments is to foster a culture that encourages open communication and identifies ways to improve systems29. In healthcare quite often the definition of Nieva & Sorra, (2003)30 has been used: “The safety culture of an organization is the product of individual and group values, attitudes, perceptions, competencies, and patterns of behaviour that determine the commitment to, and

151

the style and proficiency of, an organization’s health and safety management. Organizations with a positive safety culture are characterized by communications founded on mutual trust, by shared perceptions of the importance of safety and by confidence in the efficacy of preventive measures” Another very simple definition for culture is: Culture reflects the way we do things around here. If someone, who is not working in a hospital, would observe the daily routines of nurses and especially doctors, he/she would see possibly the following: •

Anaesthetists in the operating room get telephone calls;

•

Interruptions during patient visits;

•

Transfer of patient information from one shift to another by summing up the facts about 10, 20 or 30 patients;

•

Describing the status of patients within a couple of seconds, because the senior doctor is in a hurry.

All these kinds of distraction, interruption and information overload increases the risk on medical errors. To improve patient safety it is necessary to get more insight into the cultures of various disciplines or departments, and to better understand the risks in relation to human limitations. Open incident reporting, discussion and reflection may stimulate that insight. The safety culture of teams generally develops from a stage of denial to a stage of total integration in daily practice 31 32. A culture of denial is characterised by professionals whose prevailing attitude is ‘why waste our time on safety? We do our best, we don’t have that kind of incidents’. A reactive culture is when safety is taken seriously, but only after things have already gone wrong. After a period of time attention is decreasing again. In a bureaucratic and calculative culture systems are in place to manage all possible safety risks. The professionals’ attitude is that as long as one keeps to procedures nothing can ever happen. In a proactive culture, thought is given to what might go wrong in the future and professionals take preventive steps in practice before something might go wrong. Finally, in a generative culture, risk management is an integral part of the thinking of professionals and managers. The self-evaluation of the MaPSaF33 and the Dutch IZEP34 are both based on the five developmental stages.

4

Safety culture assessment instruments

Before implementing patient safety initiatives in health care organizations, one important step is to understand the safety culture within teams. Different methods may be employed to achieve this goal. In-depth individual interviews and focus group interviews provide detailed insights about individual and collective perceptions. But, these methods are time consuming. Self-administered surveys are more efficient and can also help to understand perceptions of safety culture35. Uniform concepts are needed in order to facilitate comparing research

152

results from one country with another country. One of the six action areas of the WHO’s World Alliance for Patient Safety is the development of a ‘patient safety taxonomy’. Scott et al. (2003)36 have published a review of available instruments for the quantitative measurement of organizational culture in health care. The authors found nine instruments (published up to June 2001) with a track record in health care that satisfied the inclusion criteria, e.g. quantitative in nature, good face validity, addressing different layers of culture, data on statistical validity and reliability. The results showed that the instruments varied considerably in terms of their grounding in theory, format, length, scope, and scientific properties. Therefore, the authors concluded that instruments with different characteristics were available to researchers and that all of them had some kind of limitations. The choice of an instrument should be determined by the purpose of investigation, the way how organizational culture is conceptualized, the intended use of the results, and the availability of resources. Based on the cultural dimensions of the instruments, none of them is focusing on patient safety and a safety culture. Kristensen & Bartels (2007)37 have published an overview of five patient safety culture assessment instruments that have been used in the UK or the USA. The five instruments and the subject they cover are presented in the table below. The instruments are adapted versions from earlier developed instruments in other industries. As Scott, Kristensen & Bartels conclude that the instruments vary in focus, aim, method, application and feasibility. The choice depends on the aim of the specific project. Examples of purposes are: - Gaining information about staff’s perceptions of safety culture; - Facilitating self-reflection and discussions on safety culture; - Helping to understand features of an organisation and/or a team and the safety culture; - Making comparisons over time; - Making judgements and setting priorities for developing the patient safety culture; - Supporting and directing cultural interventions to improve safety; - Surveillance of the impact of initiatives to change the safety culture; - Provide the basis for designing a reporting system.

153

Table 1 Overview of subjects covered by five patient safety culture assessment instruments Subjects covered Instruments CAIR HSPSC MaPSa SAQ SCS F Attention and priority given to patient safety + + + + + Communication + + + + Error management + + + + + Flow of information and processing + + + Identification of causes of patient safety + + incidents Job satisfaction + Leadership + + + + + Learning from patient safety incidents + + + + Patients are involved in patient safety + + Perceptions and recognition of stress + Perceptions of causes of patient safety + + + incidents Personnel management + + + Reporting of adverse event + + + + + Training and education + + + Work environment + + Working as a team + + + CAIR: The Checklist for assessing institutional resilience, University of Manchester, UK; HSPSC: The Hospital Survey on Patient Safety Culture, Agency for Healthcare Research and Quality (AHRQ), USA38; MaPSaF: Manchester Patient Safety Assessment Framework, University of Manchester, UK; SAQ: The Safety Attitudes Questionnaire, University of Texas, USA39; SCS: The Safety Climate Survey, Institute of Healthcare Improvement, USA.

Flin and colleagues have used the ORMAQ (Operating Room Management Attitude Questionnaire) to measure the attitudes of anaesthetists (2003)40 and surgical team members (2006)41 towards human and organisational factors that can have an impact on effective team performance and consequently on patient safety. The questionnaire covers the topics leadership, communication, teamwork, stress and fatigue, work values, human error and organisational climate. Vogus & Sutcliffe (2007) have developed the safety organizing scale (SOS), a 9-item unidimensional measure of self-reported behaviours enabling a safety culture. The SOS discriminate between related concepts like organizational commitment and trust42. 5

Activities to change the patient safety culture

Odwazny et al. (2005)43 describe the approach and experience of the Department of Medicine of the Rush University Medical Center with fostering a culture of patient safety. They have developed an online adverse event reporting system, established a model for

154

change using a) safety rounds with residents; b) e-mail safety alerts; and, in some cases c) decision alerts using electronic order entry. The support of senior management of a culture of learning and prevention, and an organizational structure that promotes collaboration has provided an environment in which patient safety initiatives can flourish. There is growing literature on teamwork44 45 and the impact of team training on attitudes of health care providers and team communication46 47 48. These trainings are mainly based on aviation Crew Resource Management (CRM) and emphasize six key areas, e.g. managing fatigue, creating and managing teams, recognizing adverse situations (red flags), crosschecking and communication, decision making, and performance feedback. Participants that have followed the training agreed that CRM training will reduce errors and improve patient safety. In a separate study researchers found that formal team briefing has shown positive results on team morale, shared understanding and efficiency49. Pronovost et al. (2006)50 described a comprehensive unit-based safety program (CUSP) to improve culture and guide organizations in learning from mistakes. The CUSP is based on a framework for a “Patient Safety Score Card” with four measures. These measures provide an answer to four questions: 1) How often do we harm patients?, 2) How often do we provide the interventions that patients should receive?, 3) How do we know we learned from defects?, 4) How well have we created a culture of safety?. The program was used in over 100 intensive care units focusing on eliminating catheter-related blood stream infections. Safety culture was seen as a vital component for system redesign.

Future research topics The goals we share for patient safety should guide the focus of future research. The aim of patient safety is to reduce the risk of harm to patients from the structure and process of care, and human slips, lapses and mistakes. Therefore, the aim of safety research must be to improve patient outcomes by minimizing harm to patients. High reliability organizations have proven that the context in which care is delivered, in other words the organizational culture, has important influences on patient safety. In this paper the role for safety culture is explored. It is shown that various articles give a description of safety culture definitions. To measure safety culture several assessment instruments were developed. Some activities are promising in changing patient safety culture. Despite of that the relation between safety culture and performance is mostly unknown. For the next five years we need to get more insight into: - the risks specific patient groups have during the care process in hospitals or other health care organizations and the influence of safety culture on these risks. The focus should lie on departments rather than on hospitals, e.g. in the Netherlands the differences in rates of adverse events between departments were larger than between hospitals; 155

-

-

the safety practices that really reduces patient harm and how safety culture can stimulate the implementation of these practices; the behaviour that fit with a positive safety culture (proactive and integrative) – we know for example that nurses should question physicians when necessary, but we don’t know how to learn nurses to do so; the barriers of various professional groups to change the existing culture; the best way to use the results of safety culture surveys to change the culture within a hospital department.

Questions for the workshop discussion •

•

Do we need more research to investigate the meaning of ‘safety culture’? (e.g. in western and non-western countries) Or, do we know what safety culture means in various countries and for various professional groups? Do we need more research on safety culture assessment instruments? Or, can we agree on using one or two instruments that makes comparisons possible?

•

Do we need more research into safety cultures that have the most positive impact on patient safety? Is there one best safety culture for every organisation? Are there differences between countries?

•

Do we need more research to get more profound insight into the ways we can change the existing culture to improve patient safety?

•

What kind of research is needed to get more evidence for the relation between patient safety and culture?

Summary of workshop discussion The workshop has started with a short presentation by the author of this paper. The discussion has been facilitated by Prof. Aneez Esmail from the school of Community Based Practice, University of Manchester. The central questions for the workshop discussion were focussing on gaps in the knowledge base: Which areas need further attention in order to develop our knowledge? What are the research gaps, and where are the difficulties converting research into practice? The group participants was convinced that research to investigate the real meaning of patient safety culture is important, e.g. recognition of culture at unit, organizational and professional level. This should be done by a combination of qualitative and quantitative methods. Therefore, a greater theoretical understanding of the meaning of culture and its relationship with performance is needed. There was no consensus under the participants about research into the most effective patient safety culture and the concept of safety leadership.

156

With regard to research on assessment instruments there was consensus that there are sufficient instruments for hospitals. This is not the case for other health care sectors like mental health, primary care or care for the disabled. But, at the same time the participants realised that the existing instruments, which are based on survey instruments, have limitations. In general, more work is needed on translating concepts to develop a common language. There is more research needed to get a better understanding into the way we can change patient safety culture, e.g. develop methods to get an understanding of unit, organizational and professionals cultures (ethnographic and conceptual), and into the relation between patient safety culture and performance. Finally, some of these gaps can be addressed in the short term, such as: the development of a common language and taxonomy (translating into different languages), the assessment of culture at unit, organizational and professional level, and the development of methods for monitoring change (e.g. observational, ethnographic, immediate outcome measures). The above mentioned research gaps can be addressed more efficiently by developing a network of researchers focusing on patient safety culture.

If you don’t like something, change it; if you can’t change it, change the way you think about it. Mary Engelbreit

157

References 1 WHO. World Alliance for Patient Safety: Forward Program. WHO, Geneva, 2004. 2 Wagner C, Wal G van der. Voor een goed begrip. (For a good understanding) MC 2005;60(47):1888-1891. 3 Wagner C, Cuperus-Bosma J., Wal G van der. Patient Safety in Dutch hospitals. Italian Journal of Public Health 2005;2(3-4)59-63. 4 Brennan TA, Leape LL, Laird NM, Hebert L, Localio AR, Lawthers AG et al. Incidence of adverse events and negligence in hospitalized patients. Results of the Harvard Medical Practice Study I. N.Engl.J.Med. 1991;324:370-6. 5 Wilson RM, Runciman WB, Gibberd RW, Harrison BT, Newby L, Hamilton JD. The Quality in Australian Health Care Study. Med.J.Aust. 1995;163:458-71. 6 Thomas EJ, Studdert DM, Burstin HR, Orav EJ, Zeena T, Williams EJ et al. Incidence and types of adverse events and negligent care in Utah and Colorado. Med.Care 2000;38:26171. 7 Vincent C, Neale G, Woloshynowych M. Adverse events in British hospitals: preliminary retrospective record review. BMJ 2001;322:517-9. 8 Schioler T, Lipczak H, Pedersen BL, Mogensen TS, Bech KB, Stockmarr A et al. [Incidence of adverse events in hospitals. A retrospective study of medical records]. Ugeskr.Laeger 2001;163:5370-8. 9 Davis P, Lay-Yee R, Briant R, Ali W, Scott A, Schug S. Adverse events in New Zealand public hospitals I: occurrence and impact. N.Z.Med.J. 2002;115:U271. 10 Baker GR, Norton PG, Flintoft V, Blais R, Brown A, Cox J et al. The Canadian Adverse Events Study: the incidence of adverse events among hospital patients in Canada. CMAJ. 2004;170:1678-86. 11 Bruijne MC de, Zegers M, Hoonhout L, Wagner C. Onbedoelde schade in Nederlandse ziekenhuizen. Utrecht/Amsterdam, NIVEL/VUmc, 2007. 12 www.ihi.org 13 Berwick DM. A primer on leading the improvement of systems. British medical journal 1996;(312):619-622. 14 IOM. To Err is human. Washington , Institute of medicine, 1999. 15 Battles JB, Lilford RJ. Organizing patient safety research to identify risks and hazards. Quality & Safety 2003;12(2):ii2-ii7. 16 IOM. Crossing the quality chasm. Washington , Institute of medicine, 2001. 17 IOM. Priority areas for national action: Transforming healthcare quality. Washington, Institute of Medicine, 2003 18 Reason J. Human Error. Cambridge University Press, New York, 1990. 19 Waring J, McDonald R, Harrison S. Safety and complexity: inter-departmental relationships as a threat to patient safety in the operating department. J Health Organ Manag 2006;20(2-3):227-42. 20 Davies HT, Nutley SM, Mannion R. Organisational culture and quality of care. Quality in Health Care 2000;9(2):111-119. 21 Carrol JS, Quijada. Redirecting traditional professional values to support safety: changing organisational culture in health care. Quality and Safety in Health Care 2004;suppl 2:ii16-ii21.

158

22 Hofstede G. Cultures and organizations. Software of the mind. London, McGraw-Hill, 1991. 23 Kvarnstrom S, Cedersund E. Discursive patterns in multiprofessional healthcare teams. J Adv Nurs 2006;53(2):244-52. 24 Warin J, Harrison S, McDonald R. A culture of safety or coping? Ritualistic behaviours in the operating theatre. J Health Serv Res Policy 2007;12 Suppl 1:S1-3-9. 25 Schein, E. Organizational Culture and Leaderschip. San Francisco: Jossey-Bass, 1985. 26 Keuning D, Eppink DJ. Management en organisatie. Stenfert Kroese, Leiden, 1993. 27 Davies HTO, Nutley SM, Mannion R. Organisational culture and quality of health care. Quality in Health Care 2000;9:111-119. 28 Schein EH. Organizational culture and leadership, second edition. San Francisco, Jossey Bass, 1992. 29 Thomas EJ, Sexton JB, Neilands TB, Frankel A, Helmreich RL. The effect of executive walk rounds on nurse safety climate attitudes: a randomised trial of clinical units. BMC Health Services Research 2005;5(28). 30 Nieva VF, Sorra J. Safety culture assessment: a tool for improving patient safety in healthcare organizations. Qual. Saf Health Care 2003;12(2):ii17-ii23. 31 Westrum R. A typology of organisational cultures. Qual. Saf. Health Care 2004;13:22-27. 32 Parker D, Lawrie M, Hudson P. A framework for understanding the development of organisationa safety dulture. Safety Science 2006;44:551-562. 33 Ashcroft DM, Morecroft C, Parker D, Noyce PR. Safety culture assessment in community pharmacy: development, face validity, and feasibility of the Manchester Patient Safety Assessment Framework. Qual Saf Health Care 2005;14(6):417-21. 34 Struben V, Wagner C. Instrument voor Zelfevaluatie van de Patientveiligheidscultuur (Self-assessment instrument of patient safety culture). Utrecht, NIVEL, 2006. 35 Kho ME, Carbone JM, Lucas J, Cook DJ. Safety climate survey: reliability of results from a multicenter ICU survey. Qual. Saf. Health Care 2005;14:273-278. 36 Scott T, Mannion R, Davies H, Marshall M. The quantitative measurement of organizational culture in Health Care: a review of the available instruments. HSR 2003;38(3):923-945. 37 Kristensen S, Bartels P. Patient safety culture: assessment instruments. In: Patient Safety Toolbox, Bohn Stafleu van Loghum, Houten, 2007. 38 Rockville w, Sorra J, Nieva V, Famolaro T, Dyer N. Hospital Survey on patient safety culture: 2007 comparative database report. AHRQ Publication no. 07-0025. 39 Sexton JB, Helmreich RL, Neilands TB, Rowan K, Vella K, Boyden J, Roberts PR, Thomas EJ. The safety attitudes questionnaire: psychometric properties, benchmarking data, and emerging research. BMC Health Services Research 2006;6:44. 40 Flin R, Fletcher G, McGeorge P, Sutherland A, Patey R. Anaesthetits’ attitudes to teamwork and safety. Anaesthesia 2003;58(3):233-42. 41 Flin R, Yule S, McKenzie L, Paterson-Brown S, Maran N. Attitudes to teamwork and safety in the operating theatre. Surgeon 2006;4(3):145-51. 42 Vogus TJ, Sutcliffe KM. The safety organizing scale: development and validation of a behavioral measure of safety culture in hospital nursing units. Med Care 2007;45(1):46-54.

159

43 Odwazny R, Hasler S, Abrams R, McNutt R. Organizational and cultural changes for providing safe patient care. Qual Manag Health Care 2005;14(3):132-43. 44 Undre S, Sevdalis N, Healey AN, Darzi SA, Vincent CA. Teamwork in the operating theatre: cohesion or confusion? J Eval Clin Pract 2006;12(2):182-9. 45 Thomas EJ, Sexton JB, Helmreich RL. Discrepant attitudes about teamwork among critical care nurses and physicians. Crit Care Med 2003;31(3):992-3. 46 Grogan El, Stiles RA, France DJ, Speroff T, Morris JA, Nixon B,, Gaffney FA, Seddon R, Pinson CW. The impact of aviation-based teamwork training on the attitudes of health-care professionals. J Am Coll Surg. 2004;199(6):843-8. 47 Award SS, FaganSP, Bellows C, Albo D, Green-Rahad B, De la Garza M, Berger DH. Bridging the communication gap in the operating room with medical team training. Am J Surg. 2005;190(5);770-4. 48 Dunn EJ, Mills PD, Neily J, Crittenden MD, Carmack AL, Bagian JP. Medical team training: applying crew resource management in the Veterans Health Administration. Jt Comm J Qual Patient Saf 2007;33(6):317-25. 49 Allard J, Bleakley A, Hobbs A, Vinnell T. “Who’s on the team today?” The status of briefing amongst operating theatre practitioners in one UK hospital. J Interprof Care 2007;21(2):189-206. 50 Pronovost PJ, Berenholtz SM, Goeschel CA, … Hunt E. Creating High Reliability in Health Care Organizations. HSR 2006;41(4);part II:1599-1617.

160