Profiles in Patient Safety: Confirmation Bias in Emergency Medicine Jesse M. Pines, MD, MBA
Abstract Confirmation bias is a pitfall in emergency care and may lead to inaccurate diagnoses and inappropriate treatments and care plans. Because of the increasing severity and volume of emergency care, emergency physicians often must rely on heuristics, such as rule-out protocols, as a guide to diagnosing and treating patients. The use of heuristics or protocols can be potentially misleading if the initial diagnostic impression is incorrect. To minimize cognitive dissonance, clinicians may accentuate confirmatory data and ignore nonconfirmatory data. Clinicians should recognize confirmation bias as a potential pitfall in medical decision making in the emergency department. Reliance on the scientific method, Bayesian reasoning, metacognition, and cognitive forcing strategies may serve to improve diagnostic accuracy and improve patient care. ACADEMIC EMERGENCY MEDICINE 2006; 13:90–94 ª 2006 by the Society for Academic Emergency Medicine Keywords: diagnostic accuracy, confirmation bias, emergency medicine, medical error
M
r. W is a 51-year-old diabetic male who presents to the emergency department (ED) with a seven-day history of lumbar lower back pain that occurred immediately after lifting a heavy box at work. He is triaged at 2:00 AM and is seen by Dr. J at 2:45 AM. He reports radiation of pain down the front of his leg and denies trauma, and bowel or bladder abnormalities. He has been using high-dose Motrin (600 mg every 6 hours) to relieve the pain. He reports a pain severity of 10/10. He has no other medical problems, smokes marijuana occasionally, and has a distant history of IV drug abuse. Triage vitals are as follows: blood pressure, 150/91; heart rate, 105 beats per minute; temperature, 100.5ºF; and respiratory rate, 16 respirations per minute. He took 600 mg of Motrin 1 hour before ED arrival. He reports that he has been unable to work all week and needs a written excuse for his boss. The nurse approaches the emergency physician (EP) and states, ‘‘Mr. W is here again. He is here all the time re-
From the Department of Emergency Medicine, University of Pennsylvania (JMP), Philadelphia, PA. Received August 6, 2004; revision received March 21, 2005; accepted July 26, 2005. Series editors: Pat Croskerry, MD, PhD, Dartmouth General Hospital Site, Dalhousie University, Halifax, Nova Scotia, Canada; and Marc J. Shapiro, MD, Rhode Island Hospital, Brown University School of Medicine, Providence, RI. Address for correspondence and reprints: Jesse M. Pines, MD, MBA, Department of Emergency Medicine, University of Pennsylvania, 3400 Spruce Street, Ground Ravin, Philadelphia, PA 19104. Fax: 215-662-3953; e-mail:
[email protected].
90
ISSN 1069-6563 PII ISSN 1069-6563583
questing pain medicine and work excuses for lower back pain. He was even here yesterday and was seen by your colleague, Dr. S, [was] diagnosed as having a muscle strain or a herniated disk, [was] given two Percocet orally, and [was] told to follow up with his primary physician. Let’s get him out of here.’’ Because it is a busy night, no rooms are available and Mr. W is examined in the hall. He states that he was told to return if he had a fever. Mr. W states that he thought he had a fever at home but did not have a thermometer. On exam, he is very uncomfortable lying recumbent on a stretcher next to his wife, who looks very concerned. Head, neck, heart, lung, and abdominal examination are normal. Back examination reveals diffuse lumbar bony and paraspinous tenderness. He is unable to tolerate a straight-leg raise because of pain. The neurological examination is grossly nonfocal, and he has no major deficits in sensation or motor ability. No rectal or perineal examinations are performed because he is in the hall. He states that the Percocet that he received last night helped ‘‘a little’’ with the pain but did not relieve it completely.
Scenario 1 While the ED nurse is writing the chart, she again approaches Dr. J: ‘‘Come on.Dr. S saw him last night and thought he was fine. I dipped his urine again tonight and it was normal. Let’s discharge him. There are 10 people in the waiting room.’’ Dr. J agrees that this is likely drug-seeking behavior and discharges the patient, giving him two Percocet to go, and instructs him again to see his primary physician. Two ED technicians help Mr. W to his car so that his wife can drive him home.
ª 2006 by the Society for Academic Emergency Medicine doi: 10.1197/j.aem.2005.07.028
ACAD EMERG MED January 2006, Vol. 13, No. 1 www.aemj.org
Scenario 2 Dr. J insists that Mr. W’s pain be controlled in the ED and that he be examined in a private room. His urine is dipped and is negative. Mr. W changes into a gown, and he is found to have severe pain with standing and then becomes diaphoretic. Rectal tone and perineal sensation and skin examination are normal. Four milligrams of IM morphine sulfate are ordered. On reexamination, the EP notices that Mr. W is diaphoretic and that his temperature now has risen to 102.2ºF. He states that his pain now rates 9/10. On further questioning, he admits using IV heroin 3 weeks before onset of the pain. Complete blood count, chemistries, erythrocyte sedimentation rate (ESR), a chest radiograph, and lumbar plain films are ordered. His white blood cell count (WBC) is 11.8 3 103/mm3, his ESR is 47 mm/h, and the rest of his labs are unremarkable. Chest radiograph shows no acute disease. Lumbar films show vertebral endplate and disk destruction. Emergency magnetic resonance imaging with gadolinium enhancement is ordered and reveals findings consistent with epidural abscess. The neurosurgery consultant is immediately notified of the results and decides to take Mr. W to the operating room for emergent drainage. DISCUSSION It is easy to imagine how the first scenario might happen in a busy ED. EPs repeatedly are challenged to rapidly diagnose and treat multiple patients, some of whom present with potentially life-threatening illness. EPs increasingly are being forced to work in crowded conditions and to focus on efficiency of patient throughput while attempting to maintain the highest possible quality of care. Because of the depth, scope, and volume of cognitive thinking required to manage patient information, medical errors of cognition are a significant issue in emergency medicine (EM) practice.1–5 EM particularly is susceptible to cognitive errors, because clinicians are required to integrate their knowledge base with new situations to create a diagnostic and management plan.6 EPs face a very high cognitive load and frequently manage many patients simultaneously who have life-threatening and potentially life-threatening conditions. Many studies have confirmed that the major cause of malpractice claims in EDs is a failure to diagnose.7–9 A 1993 study found that about 2% of patients with acute myocardial infarction mistakenly are sent home.10 Because of the rapidity with which EPs must work and the importance of an accurate diagnosis, it is important that EPs be cognizant of the possibility that diagnoses may be compromised by confirmation bias. Put simply, this means that one may have an initial or a preconceived idea about something and interpret subsequent information or data so as to confirm that idea (or in the case of EPs, to confirm the diagnoses). As a specialty, EPs have developed skills that open them to potential errors in cognition such as confirmation bias. In the case presentation, an initial biased approach may be for Dr. J to confirm Dr. S’ diagnosis of musculoskeletal back pain without further in-depth examination and investigation. Certain elements in the history confirm his judgment. The natural inclination of a busy EP is to sort patients quickly by categorizing them by diagnostic or
91
treatment strategy.11 In this case, the EP may accentuate the historical elements confirming the diagnosis of musculoskeletal back pain (preceded by injury, previous diagnosis, and history of many ED visits) and not investigate further pertinent historical elements (e.g., when pressed, Mr. W admitted to recent intravenous drug abuse). Confirmation bias is related closely to anchoring bias, which can come into play when there is an incorrect initial impression and the focus of the evaluation is centered on that initial impression. For example, in this case, our patient who presents with classic musculoskeletal back pain (and multiple visits) actually has an epidural abscess, or a patient treated frequently for migraine headaches actually has an acute subarachnoid hemorrhage. Because of the volume and acuity of care in an ED, quick sorting and categorization can serve to reduce the already high cognitive load required to manage multiple patients.12 One study of trauma patients found that there were reasoning errors in 100% of trauma resuscitations.13 The use of heuristics is a necessary evil in caring for ED patients. The use of heuristics is inevitable to allow clinicians to maintain efficiency and not chase the metaphorical zebras (a colloquial term designating those possible diagnoses that are least likely and most difficult to confirm on the basis of given clinical data, as in the saying, ‘‘when you hear hoofbeats, think horses, not zebras’’). Thus, attaching safeguards to the heuristics, rather than avoiding the heuristics, has been a solution for error avoidance. However, sometimes the initial clinical suspicion is not borne out by results of diagnostic tests, repeated examinations, and observation. When the initial clinical suspicion is high for a particular illness, the EP may place more emphasis on confirmatory data than on nonconfirmatory data. For example, in this case, because the nurse and Dr. S both see the likely diagnosis as musculoskeletal pain, Dr. J may preferentially search for information that confirms that diagnosis and not approach Mr. W as if he were a new case of severe lower back pain. Thus, the influence of confirmation bias can lead to errors in medical decision making. This can be even more powerful when, in a clinician’s judgment, a constellation of signs and symptoms appears pathognomonic of a particular illness, or when another physician has already made a diagnosis. The tendency to overemphasize confirmatory data (confirmation bias) often can compromise the ability of EPs to accurately diagnose and treat patients. This can lead to EPs overlooking vital information and not asking all the right questions needed to diagnose and treat patients accurately. Confirmation bias occurs when people selectively focus upon evidence that supports their beliefs or what they want or believe to be true, while ignoring evidence that serves to disconfirm those ideas. Confirmation bias is a very human way of thinking. Francis Bacon described confirmation bias as follows in 1620: The human understanding when it has once adopted an opinion (either as being the received opinion or as being agreeable to itself) draws all things else to support and agree with it. And though there be a greater number and weight of instances to be found on the other side, yet these it either neglects and despises, or else by some distinction sets aside
92
Pines CONFIRMATION BIAS
and rejects; in order that this great and pernicious predetermination the authority of its former conclusions may remain inviolate.14 Confirmation bias is well documented in the behavioral and economic literatures.15–17 It empirically is even stronger when information is presented sequentially, as it is in clinical emergency care, compared with when all the information is available up front.18 Variability in the temporal processing and receipt of information may influence decision making because the longer a person holds onto a decision or approach, the more difficult it becomes for him or her to break from that thinking. When multiple providers are caring for a patient, confirmation bias can have a variable effect on guiding accurate diagnosis and treatment. Other providers caring for the same patient may verbally confirm a diagnosis or reinforce the initial categorization of a patient by the initial diagnostic impression. In the case of an EM resident presenting a patient in the assessment-oriented way, in which the assessment precedes the presentation, confirmatory data may be highlighted to reinforce the assessment, whereas nonconfirmatory data (that may or may not have been asked for) may be omitted.19 However, the presence of multiple providers may help prevent confirmation bias because one provider may get a critical bit of information that differs from the first providers’, and he or she accordingly changes the plan of care. When a medical student spends an hour taking an exhaustive history and physical examination, that student’s lack of direction (and lack of knowledge of heuristics) ultimately can lead to pertinent information being found that may not have been found in a briefer encounter. Further examples of confirmation bias affecting a conclusion on a less individual basis include a drug vendor’s interpretation of a study designed to validate the use of its product (and publication of the same). In addition, when clinical policies or pathways designed by clinicians occur in a hospital with a particular research interest in a mode of therapy, or sponsorship by a particular vendor, confirmatory data may be accentuated in a nonscientific manner. An additional level of complexity in the EM decisionmaking process occurs when a clinical impression is strong enough to guide diagnosis without the support of confirmatory data. An example of this is a patient with typical features of chest pain resembling an acute coronary syndrome. Adjunctive data may not support this, such as a negative cardiac marker or electrocardiogram (that may be normal in the early stages of acute myocardial infarction), but often definitive diagnostic workup may not be immediately available to emergency healthcare providers (cardiac catheterization), and a judgment must be made on clinical grounds. Confirmatory tests in this situation must be taken with an in-depth understanding of both the value (sensitivity and specificity) and limitations of available historical information, ED testing, and appropriate use of all the available data in clinical decision making. Here confirmation bias can be helpful because even in the face of negative data (ECG, cardiac markers), a high level of clinical suspicion is not changed by the objective data. A related concept is cognitive dissonance, which holds that it is psychologically uncomfortable to hold contra-
dictory cognitions.20 It can be confusing when an unexpected result (usually negative) comes back on a patient in whom the illness in question was highly suspected. This can lead to disposition issues. When the initial impression is highly suspicious for serious illness and the initial search for a cause is not fruitful, EPs sometimes may accentuate any positive data to support a justification for hospital admission. This again is a beneficial effect of confirmation bias when it leads to appropriate patient care. Physicians and scientists are prone to confirmation bias, as are practitioners in many other academic disciplines. The more that researchers believe that they are right, the greater weight they place on confirmatory information. One study in which journal reviewers were asked to evaluate manuscripts that described identical experimental procedures reporting variable results (positive, negative, or mixed) found that reviewers were strongly biased against manuscripts that reported results contrary to their theoretical perspective.21,22 One solution to managing data and decision making in high workload situations is the presence of automation. Automation in clinical medicine is analogous to clinical guidelines, such as protocols that may be present in a chest pain center (rule-out protocols).23 These may even be built into clinical information systems or ED protocols. A recent study in the aviation literature showed that automation was helpful in guiding initial plans but found that one third of pilots failed to revise flight plans as a result of change in conditions.24 One could argue that the presence of automation may even hinder the ability to reconsider alternative diagnoses when there exists a location bias (chest pain center). This may lead to a patient who has chest pain secondary to cholelithiasis being misdiagnosed after a cardiac evaluation. Automation in clinical information systems may be very useful in reducing medication errors,25 but built-in forcing strategies such as ‘‘trauma labs,’’ ‘‘toxicology labs,’’ or ‘‘rule out cholecystitis with labs and ultrasound,’’ may result in missed diagnosis if the initial impression (anchoring bias) is incorrect. Additionally, the use of so-called screening labs should be used principally in the way they were designed: as screening tests (i.e., not as diagnostic tests). For example, when a patient who is a poor historian presents with nonspecific symptoms, the use of screening labs most often is not helpful to the patient, and further historical evaluation must be done to identify a source for the complaint (i.e., calling family and other providers). Aside from consuming health care dollars by performing tests that are very unlikely to help the patient, when abnormal test results return that were not appropriately ordered in the first place, it might lead physicians to jump to inappropriate diagnostic conclusions. Schermer26 stated, ‘‘Smart people believe weird things because they are skilled at defending beliefs they arrived at for nonsmart [sic] reasons.’’ Because of the increasing complexity of cases and the cognitive load required to take care of them, EPs must rely on heuristics to care for many patients. One solution to confirmation bias is the application of the scientific method, in which the intent is to disprove a belief as opposed to searching for only confirmatory evidence. Although this makes sense
ACAD EMERG MED January 2006, Vol. 13, No. 1 www.aemj.org
in theory, it takes an experienced clinician to be unaffected by the confirmation bias of the initial diagnostic impression. The scientific method also can have its own pitfalls. In the case of chest pain of potential cardiac origin, one cannot start with the impression that it is present and then look for evidence to disprove it, because obtainable data beyond the clinical impression are not strong enough to overrule the initial impression. Evidence that disproves can be just as suspect as evidence that confirms; it depends on the likelihood ratios (or sensitivity and specificity) of the evidence used. A particularly robust approach that is particularly useful in EM involves Bayesian reasoning, in which known data on tests are combined with initial clinical impressions (pretest probabilities) to derive accurate diagnostic probabilities of disease (posttest probability).27 For example, when the clinician has a high clinical suspicion (pretest), nonconfirmatory data are more likely to be false (false negative), and confirmatory data are more likely to be true, than if the pretest clinical suspicion was low. It becomes appropriate to emphasize nonconfirmatory data to a lesser degree. Changing the diagnostic impression as a result of a negative test is more likely to cause a diagnostic error. Examples of this include using a normal WBC to exclude the diagnosis of appendicitis in a classic clinical presentation, or excluding pulmonary embolism in a patient with a moderate clinical probability but a normal D-dimer. Key skills in Bayesian reasoning are deciding whether and how the test will contribute to diagnostic certainty before ordering it and interpreting the result in light of the pretest probability. If the test is ordered specifically to confirm a positive diagnosis, then the clinician should disregard a negative result. For example, in the hypotensive trauma patient, free abdominal fluid on the FAST exam should lead directly to laparotomy (if there is no other reason for hypotension); however, if no free fluid is seen, the test is not sensitive enough to rule out intraabdominal injury, and the patient should be further evaluated (laparotomy if the patient remains unstable; abdominal CT if they are stable). If it is done to refute a diagnosis (i.e., as rule-out), a positive diagnosis similarly only calls for further tests. An example of this is the use of the ESR for temporal arteritis in an otherwise lowrisk patient. If the ESR is elevated, a biopsy still is needed before a definite diagnosis can be made; if the ESR is low, clinicians can be fairly certain that temporal arteritis is not present. In these ways, the use of heuristics and confirmation bias actually may prevent misdiagnosis in that they may prevent the physician from leaving the most probable diagnosis to chase a zebra. Confirmation bias is an issue for clinicians taking the initial history when the first impression steers the history in such a way that the physician poses questions that confirm the impression and may not ask the ones that might suggest a different diagnosis. The physician is not necessarily ignoring relevant data; however, the chain of thought that he or she follows simply is not allowing him or her to steer in the direction of seeking truth. Sometimes, because of distractions, such as thoughts of other patients who are being treated concurrently, the EP may gloss over certain things in the history while seeking the list of typical presenting features of the suspected disease. This last situation was eloquently
93
expressed by Simon and Garfunkel, who sang, ‘‘A man hears what he wants to hear and disregards the rest.’’ The practice of EM requires the processing of multiple complex data elements in a real-time, high-stakes environment. Patient re-evaluation ideally should occur at every step of the process as new data become available, and EPs should update posttest probabilities on the basis of new information. A solution suggested by Croskerry and Sinclair6 is the use of metacognition by experienced providers and education of medical students and residents on the use of this cognitive strategy. Metacognition involves stepping back and thinking about the cognitive process that goes into making a decision. Specifically, medical educators should focus on teaching the cognitive process to students and residents and should realize the limitations of medical data by using cognitive aids (such as computers and personal data assistants). They need to consciously step back and see the broader range of possibilities, reexamine decision making as new data become available, avoid overconfidence, and effectively select strategies to deal with problems in decision making. Another potential solution is the use of cognitive forcing strategies. These can be categorized into universal, generic, and specific strategies. A universal cognitive forcing strategy is defined as a generalized understanding of the error theory and the appreciation and application of metacognition. A generic cognitive forcing strategy involves understanding the general heuristics in medical decision making and under what circumstances they fail. Understanding and recognizing confirmation bias is a subset of this process, and clinicians must be aware of such bias to adjust initial impressions on the basis of new objective data. Search-satisficing, or calling off a search once a positive result is found, is an example of a generic strategy that could be applied to stopping a search for coingestants in a toxic poisoning once a primary ingestant is found. A specific cognitive forcing strategy relates to known pitfalls in specific diagnostic workups.6 There are many pitfalls in clinical EM; for example, failure to consider a closed-head injury in an inebriated patient. In this sense, the usage of a cognitive forcing strategy is the deliberate usage of a particular strategy in a specific situation that optimizes medical decision making and minimizes error.6 Still, urgent clinical decisions must be made in EDs without complete information. These strategies need to be balanced against the dangers of indecision in cases (and busy departments) in which delay can have adverse consequences to the patient or to those waiting to be treated. The nature of EM is, whether providers like it or not, tied to situations in which the ED is crowded and the demand for services is pushed to capacity. Simply acknowledging this brings providers no further toward a safer system for patients or providers. Even the information that is available is imperfect: historical information and physical examination results both have high interrater variability, and diagnostic tests ordered in the ED each have an intrinsic error rate (sensitivity and specificity) that must be considered. Recognition and understanding that confirmation bias may exist may help clinicians to rethink the objective data when using a specific data point to guide medical decision making.
94
CONCLUSIONS Stepping back and reconsidering the objective facts may guide clinicians to reconsider the initial impression and pursue a completely different diagnostic strategy. Using and teaching metacognition and an understanding of error theory and cognitive forcing strategies may be helpful in minimizing confirmation bias. When the initial clinical impression is not corroborated by objective data, EPs must be open to revisiting the possibility of an inaccurate diagnosis and may have to start again at diagnostic time zero or, alternatively, defer to an appropriate inpatient or outpatient workup. References 1. Brennan TA. The Institute of Medicine report on medical error—could it do harm? N Engl J Med. 2000; 342:1123–5. 2. Famularo G, Salvini P, Terranova A, Gerace C. Clinical errors in emergency medicine: experience at the emergency department of an Italian teaching hospital. Acad Emerg Med. 2000; 7:1278–81. 3. Glick TH, Workman TP, Gaifberg SV. Suspected conversion disorder: foreseeable risks and avoidable errors. Acad Emerg Med. 2000; 7:1272–7. 4. Kuhn GJ. Diagnostic errors. Acad Emerg Med. 2002; 9:740–50. 5. Croskerry P, Sinclair D. Emergency medicine: a practice prone to error? CJEM. 2001; 3:271–6. 6. Croskerry P. Cognitive forcing strategies in clinical decision-making. Ann Emerg Med. 2003; 41:110–20. 7. U.S. General Accounting Office, the Ohio Hospital Association and the St. Paul (MN) Insurance Company. 1998 Data. Available at: http://hookman.com/mp9807. htm. Accessed Aug 6, 2004. 8. McQuade JS. The medical malpractice crisis—reflections on the alleged causes and proposed cures: discussion paper. J R Soc Med. 1991; 84:408–11. 9. Kronz J, Westra W. The role of second opinion pathology in the management of lesions of the head and neck. Curr Opin Otolaryngol Head Neck Surg. 2005; 13:81–4. 10. McCarthy BD, Beshansky JR, D’Agostino RB, Selker HP. Missed diagnoses of acute myocardial infarction in the emergency department; results from a multicenter study. Ann Emerg Med. 1993; 22:579–82. 11. Kovacs G, Croskerry P. Clinical decision making: an emergency medicine perspective. Acad Emerg Med. 1999; 6:947–52. 12. Croskerry P. The cognitive imperative: thinking about how we think. Acad Emerg Med. 2000; 7:1223–31.
Pines CONFIRMATION BIAS
13. Clarke JR, Spejewski B, Gertner AS, et al. An objective analysis of process errors in trauma resuscitations. Acad Emerg Med. 2000; 7:1303–10. 14. Bacon F. Novum Organum. New Organon, 1620. 15. Gilovich T. How We Know What Isn’t So: The Fallibility of Human Reason in Everyday Life. New York, NY: Free Press, 1993. 16. Mynatt CR, Doherty ME, Tweney RD. Consequences of confirmation and disconfirmation in a simulated research environment. Q J Exp Psychol. 1978; 30: 395–406. 17. Dave C, Wolfe KW. On Confirmation Bias and Deviations from Bayesian Updating. Available at: http://www.peel.pitt.edu/esa2003/papers/wolfe_ confirmationbias.pdf. Accessed Apr 21, 2004. 18. Jonas E, Schulz-Hardt S, Frey D, Thelen N. Confirmation bias in sequential information search after preliminary decisions: an expansion of dissonance theoretical research on selective exposure to information. J Pers Soc Psychol. 2001; 80:557–71. 19. Maddow CL, Shah MN, Olsen J. Efficient communication: assessment-oriented case presentation. Acad Emerg Med. 2003; 10:842–7. 20. Festinger L. A Theory of Cognitive Dissonance. Palo Alto, CA: Stanford University Press, 1957. 21. Mahoney M. Publication prejudices: an experimental study of confirmatory bias in the peer review system. Cog Ther Res. 1977; 1:161–75. 22. Mahoney MJ, DeMonbreun BG. Confirmatory bias in scientists and non-scientists. Cog Ther Res. 1977; 1:176–80. 23. Fesmire FM, Hughes AD, Fody EP, et al. The Erlanger chest pain evaluation protocol: a one-year experience with serial 12-lead ECG monitoring, two-hour delta serum marker measurements, and selective nuclear stress testing to identify and exclude acute coronary syndromes. Ann Emerg Med. 2002; 40:584–94. 24. Muthard EK, Wickens CD. Factors that Mediate Flight Plan Monitoring and Errors in Plan Revision: Planning under Automated and High Workload Conditions. Presented at the 12th International Symposium on Aviation Psychology. 2003. Available at: http://www.aviation.uiuc.edu/UnitsHFD/conference/ Dayton03/mutwic.pdf. Accessed April 21, 2004. 25. Bates DW. Using information technology to reduce rates of medication errors in hospitals. Br Med J. 2000; 7237:788–91. 26. Schermer M. Smart people believe weird things. Sci Am. 2002. 27. El-Gamal MA, Grether DM. Are people bayesian? Uncovering behavioral strategies. J Am Stat Assoc. 1995; 90:1137–45.
