Reducing False-Negative and False-Positive Diagnoses in Anatomic Pathology Consultation Material Andrew A. Renshaw, MD; Edwin W. Gould, MD
Context.—Previous studies have shown that there are disagreements in interlaboratory consultation, including false-negative and false-positive diagnoses. To date, methods to reduce false-negative and false-positive diagnoses have been poorly documented. Objective.—To identify features associated with falsenegative and false-positive diagnoses in anatomic pathology. Design.—We reviewed the results of interlaboratory consultation in our institution during a 9-year period. For false-negative and false-positive diagnoses, methods that might have prevented the error were identified. Results.—Disagreements were identified in 810 of 8082 consults (10%). Fifty-four false-negative cases (0.7% of all consults) and 27 false-positive cases (0.3%) were identified. False-negative cases were more common in breast (20 of 1131; 1.8%), genitourinary (16 of 970; 1.7%), hematologic (3 of 242; 1.3%), and cytology (3 of 404; 0.8%) than in all other sites combined (P , .001); no
significant difference in sites were identified for falsepositive cases. Overall, there was no difference in the percentage of cases that were reviewed by more than one pathologist in either false-negative cases (109 of 810; 13.5%) or false-positive cases (135 of 810; 16.7%), compared with all other consults (858 of 7272; 11.8%) (P ¼ .74 and .59, respectively). However, on review, 12 of all 27 false-positive cases (44%) might have been prevented by the use of immunohistochemistry alone, and 36 of all 54 false-negative cases (67%) might have been prevented by the use of a second review; special stains, including immunohistochemistry; additional levels; changes in processing; and hedges. Conclusion.—Approximately one-half of false-negative and false-positive cases (48 of 81; 59%) might be preventable by the use of a combination of pathologic methods. (Arch Pathol Lab Med. 2013;137:1770–1773; doi: 10.5858/arpa.2013-0012-OA)
I
ses because they may lead to inadequate, delayed, or inappropriate treatment for patients. Although some pathologists recommend second reviews for all initial diagnoses of malignancy to reduce the incidence of false-positive diagnoses, the data to support that policy is unclear. To further investigate this area for data to form evidence-based recommendations for reducing false-negative and falsepositive diagnoses, we reviewed our records of interinstitutional consults for features that might help reduce the incidence of these cases.
nterinstitutional review of pathology material, either directed by the pathologist or by the patients or clinicians, has been previously studied1–18 and is often recommended or required by pathology organizations and medical institutions when patients change facilities for treatment.7,16–17 These studies have shown that the rate of disagreement depends not only on the type of tissue but also on the type of sampling.7,16 In addition, when the result is determined by a third opinion or clinical follow-up, in some cases, the original diagnosis, rather than the consult diagnosis, is correct.14,16 Although some studies suggest review of material by a second pathologist before sign-out leads to reduced rates of disagreement with in-house material,15,19 that improvement is not consistent among different types of interinstitutional consultations.20 To date, few recommendations have been put forth for reducing the disagreement rates in interinstitutional consultation. Some of the most concerning and feared types of disagreements are false-negative and false-positive diagnoAccepted for publication March 7, 2013. From the Department of Pathology, Baptist Hospital of Miami, Miami, Florida. The authors have no relevant financial interest in the products or companies described in this article. Reprints: Andrew A. Renshaw, MD, Department of Pathology, Baptist Hospital of Miami, 8900 N Kendall Dr, Miami, FL 33176 (e-mail:
[email protected]). 1770 Arch Pathol Lab Med—Vol 137, December 2013
METHODS We reviewed results of both pathologist-directed and patient- or clinician-requested interinstitutional consultation (a large, community hospital) received at, or sent from, our institution between January 2004 and December 2012. This included both incoming and outgoing consultations for both pathologist- and patient– or clinician-requested cases. These were all final case results; intraoperative or immediate frozen-section diagnoses were not included in this series. For all cases, the pathologist of record compared both reports and determined whether a disagreement was present or not. For all cases with disagreements, one author (A.A.R.) reviewed the case and classified it as previously described.21 False-negative cases were those in which the lesion itself or its atypicality appeared to be entirely missed by the original pathologist. It did not include cases in which there was a disagreement about the degree of atypia present. For example, a breast case with a diagnosis of atypical ductal hyperplasia and a Reducing FN and FP Diagnoses—Renshaw & Gould
consult diagnosis of ductal carcinoma in situ would not be considered a false-negative case because the original pathologist clearly saw the lesion and recognized it was atypical. In contrast, a breast biopsy with a diagnosis of fibroadenoma, which, on consultation, was diagnosed as a phyllodes tumor would be a false-negative diagnosis because there is no indication that the original pathologist thought the lesion was atypical. Although the original pathologist may have considered such a diagnosis, there was no documentation of that possibility. In contrast, if the original pathologist had made a diagnosis of fibroadenoma but included a note saying that the tumor was hypercellular but lacked stromal overgrowth, then that would not count as a false-negative diagnosis because the report makes it clear there was a differential to consider. Similarly, false-positive cases were those in which a definite diagnosis was made and the second pathologist thought that it could definitely be stated the lesion was not there. For example, a diagnosis of carcinoma changed to suspicious for carcinoma would not qualify as a false-positive case because the lesion was clearly seen by both pathologists, and the difference between them rested on how sure each was of the diagnosis. However, if the diagnosis was changed from carcinoma to a benign mass, without any qualification, that would qualify as a false-positive diagnosis. For the purposes of this report, there was no attempt at correlating the biopsy diagnoses with additional follow-up or studies. Statistical analysis was done using a 2-tailed v2 test. A P value of .05 was considered significant.
RESULTS During the 9 years of the study, approximately 270 000 cases were reviewed at our large, community hospital. During that period, there were 8082 consults (approximately 3% of all anatomic pathology cases). Consults included bone and soft tissue (6%; n ¼ 485), breast (14%; n ¼ 1131), cardiovascular (0.2%; n ¼ 16), cytology (5%; n ¼ 404), endocrine (4%; n ¼ 323), gastrointestinal (12%; n ¼ 970), genitourinary (12%; n ¼ 970), gynecologic (7%; n ¼ 566), head and neck (4%; n ¼ 323), neurologic (6%; n ¼ 485), hematologic (3%; n ¼ 242), pulmonary (5%; n ¼ 404), skin (22%; n ¼ 1755), and other (0.1%; n ¼ 8). Diagnoses included carcinoma (39%; n ¼ 3152), lymphoma (4%; n ¼ 323), melanoma (5%; n ¼ 404), other malignancies (7%; n ¼ 566), precursors/dysplasia (7%; n ¼ 566), atypical/suspicious (8%; n ¼ 647), benign tumor (10%; n ¼ 808), inflammatory (11%; n ¼ 889), negative (4%; n ¼ 323), and other (56%; n ¼ 404). Disagreements were identified in 810 of 8082 consults (10%); 78% (632 of 810) were of minor clinical significance, and 22% (178 of 810) were of major clinical significance. This included 54 false-negative cases (0.7% of all 8082 consults) and 27 false-positive cases (0.3%). In addition, there were 49 cases (0.6%) in which the original diagnosis was positive, but the review diagnosis was atypical or suspicious. These cases were not studied further. False-negative cases were more common in breast (20 of 1131; 1.8%), genitourinary (16 of 970; 1.7%), hematologic (3 of 242; 1.3%), and cytology (3 of 404; 0.8%) than they were in all other sites (P , .001); no significant difference in sites were identified in false-positive cases. Overall, there was no difference in the percentage of cases that were reviewed by more than one pathologist in either false-negative (109 of 810; 13.5%) or false-positive cases (135 of 810; 16.7%) compared with all other consults (858 of 7272; 11.8%) (P ¼ .74 and .59, respectively). Of the 54 false-negative cases, 36 (67%) might have been prevented by the use of a second review; special stains, Arch Pathol Lab Med—Vol 137, December 2013
including immunohistochemistry; additional levels; changes in processing; and/or hedges (Table 1). Although all 13 false-negative cases (100%) for prostate biopsies were reviewed by a single pathologist, that was not significantly different than the rate of cases reviewed by a single pathologist overall (7128 of 8081; 88.2%) (P ¼ .21). In the case of the missed neuroblastoma, there was only 1 focus of tumor that was present on 1 of 10 smears (10%) made from the case. On review, 12 of all 27 false-positive cases (44%) might have been prevented by the use of immunohistochemistry, based on the clinical history alone (Table 2). Overall, the rate of immunohistochemistry use in these false-positive cases (3 of 27; 11%) was significantly less than the rate of immunohistochemistry use for all remaining consults (274 of 783; 35%) (P ¼ .002). COMMENT Although there is literature on disagreements in interinstitutional consultations, that literature does not focus on false-negative and false-positive diagnoses and does not focus on ways to reduce those disagreements. As a result, the data presented here are both similar and different than previous studies on disagreements in interinstitutional consultation. The overall disagreement rate of 10% (810 of 8082) of the cases is similar to that reported in other studies,13,16,21 although disagreement rates as high as 40% have also been reported (reviewed in Renshaw22). However, our results show that false-negative and false-positive diagnoses occur in only 1% of consult cases, and falsenegative cases appear to be more common than falsepositive diagnoses, although that depends on the criteria used to define those diagnoses. We specifically excluded cases in which all observers agreed there was an atypical focus but disagreed only on the whether the features were diagnostic or not. If those 49 of the consult cases (6%) were included, there would have been slightly more false-positive cases (76 of 130; 58%) than false-negative cases (54 of 130; 42%). Previous studies have suggested that the rate of disagreement depends on the site and type of biopsy.7,16 In the current study, this was significant for false-negative cases but not for false-positive cases. The types of cases that were significant for false-negative cases (breast, genitourinary, hematologic, and cytology) were similar (cytology) but not identical (serosal surfaces, gynecologic tract, gastrointestinal tract) to the results reported by others.7,16 As a result, targeted second reviews, to reduce disagreements and potential errors, may be more effective for false-negative cases than they are for false-positive cases. We believe that a focused second review, when thoughtfully done, can be effective at reducing errors, including false-negative and false-positive errors, and that targeting those second reviews can make them more efficient in time and effort. A structured program of second reviews is an appropriate technique for many pathology programs. Nevertheless, there are many unknowns about second reviews. First, there are no current data, to our knowledge, on how pathologists might respond to a targeted review. Specifically, if pathologists know specific types of cases are more likely to be examined by a second pathologist than are other types of cases, the pathologists are likely to alter their practice to reduce disagreement before that second review occurs and, concurrently, to spend less time on cases that Reducing FN and FP Diagnoses—Renshaw & Gould 1771
Abbreviations: BMs, bone marrow tests; DCIS, ductal carcinoma in situ; LCIS, lobular carcinoma in situ.
Melanoma in situ 1 Skin reexcision
No residual melanoma
2 Skin biopsy
Benign
Metastatic papillary thyroid carcinoma Basal cell carcinoma 1 Lymph node
Benign
1 Bone marrow
No neuroblastoma
Found on deeper levels throughout block (5 times) No immunohistochemistry
Two-thirds seen by only one pathologist Only on 1 of 10 smears (10%) Low-grade phyllodes tumor Neuroblastoma 3 Breast core
Fibroadenoma
No stains done Corynebacterium infection 1 Breast biopsy
Nonspecific inflammation
Reviewed by one pathologist Calcifications 1 Breast core for calcifications
No calcifications
Microinvasion High-grade DCIS, no invasion 3 Breast core/biopsy
LCIS No LCIS 10 Breast biopsy
All cases reviewed by only one pathologist
Perform E-cadherin on any suspicious breast biopsy, second review Second review of all high-grade DCIS cases; hedge with comment ‘‘the lesion is large and architecturally complex, and invasion can not be ruled out’’ All breast cores without calcifications reviewed by 2 pathologists Gram stain on all cases with cystic, neutrophilic, granulomatous inflammation.26 Correlation with microbiologic studies Hedge all cellular fibroglandular lesions on core biopsy; second review Put all tissue in clot, rather than smears, for BMs for metastatic disease Use keratin on selected lymph nodes with known carcinoma Provide cut levels on all benign biopsies done for the clinical diagnosis of basal cell carcinoma Use Melan-A on all negative reexcision results
Second review of all benign cases
Method to Potentially Prevent Error Comment
All cases reviewed by only one pathologist E-cadherin not performed Carcinoma
Consult Diagnosis Original Diagnosis
Benign 13
Cases, No. Biopsy Site/Type
Prostate biopsy
Potential Ways to Decrease False-Negative Cases Table 1.
1772 Arch Pathol Lab Med—Vol 137, December 2013
are less likely to be reviewed. Whether that might result in increased error rates in cases not selected for targeted review is not known. In addition, previous studies have shown that the effect of a case being reviewed by a second pathologist before consultation on the disagreement rate is not always consistent.19–20 Our results also suggest that the effect of a second name is inconsistent regarding false-negative and false-positive cases. Although, overall, we could find no consistent trend that showed that a second pathologists’ review affected false-negative or false-positive diagnoses, in specific instances (false-negative prostate biopsies, missing breast calcifications) the data suggested that a second review might be of value; those results, however, were not statistically significant. Further study of that effect may be of benefit. Nevertheless, some pathology groups insist that all new diagnoses of a malignancy require a second pathologist to review them on the presumption that such a review will reduce the rate of false-positive diagnoses. Unfortunately, data to support that practice are scant. Although the practice is unlikely to reduce the accuracy of diagnoses, our data clearly show that false-positive cases still occur, even when more than one pathologist’s name is on the report. Our opinion is that only a subset of new diagnoses of cancer warrant review by a second pathologist. For example, second review of cases that are pathologically and clinically obvious (colonic adenocarcinoma with a mass, a liver mass with a markedly elevated serum a-fetoprotein) would appear to be unlikely to add value. Indeed, mandatory second review of all such cases is likely to simply increase the workload, provide no benefit to patients, and result in decreased vigilance23–25 and a reduced value to the second review in cases in which it is more useful. As a result, we believe that each pathology group should define for itself what is clinically and or pathologically obvious, based on the strengths, weaknesses, and idiosyncrasies of their own pathologists and clinicians. Requiring that all new diagnoses of malignancy require a 2-pathologist review implies that a single pathologist in that laboratory is not able to make a diagnosis of malignancy by him or herself, an interpretation that may not be correct and may have unintended consequences. In contrast, our results suggest that many false-positive cases might be avoided by a more-liberal use of immunohistochemistry, rather than by having a second pathologist review the case. Our results demonstrate that, in falsepositive cases, immunohistochemistry is used less often than it is in consultations without false-positive results, which implies that immunohistochemistry may be underused. A more-liberal use of immunohistochemistry might be able to reduce both false-positive and false-negative diagnoses and improve the accuracy of diagnoses overall. In addition to using second review and immunohistochemistry, other options were available that might have resulted in fewer false-negative cases. In most bone marrow cases, the direct-smear tests are used for their excellent morphology, rather than as a screening test. In cases in which the bone marrow test is being performed for known metastatic disease, the utility of those smears for diagnosis is likely less, and they may present a screening challenge. One way to avoid that situation is to make only a few direct smears (or even none) and to place all that material in the clot section, where screening may be more easily performed. In addition, there were disagreements about microinvasion Reducing FN and FP Diagnoses—Renshaw & Gould
Table 2.
Potential Ways to Decrease False Positive Cases
Biopsy Site/ Type
Cases, No.
Original Diagnosis
Cervical biopsy Prostate biopsy Breast biopsy
3 4 3
Pleural fluid Stomach biopsy
1 1
SIL Carcinoma Invasive carcinoma arising in DCIS Adenocarcinoma Adenocarcinoma
Consult Diagnosis
Comment
Method to Potentially Prevent Error
Benign Benign DCIS
No IHC done No IHC done No IHC done
p16, MIB-1 in selected cases p63, HMWK, AMACR in selected cases p63 in selected cases
Benign Benign granulation tissue
No IHC done No IHC done
BER-EP4, MOC31 in selected cases Keratin, CD68 in selected cases
Abbreviations: DCIS, ductal carcinoma in situ; HMWK; high–molecular-weight keratin; IHC, immunohistochemistry; SIL, squamous intraepithelial lesion.
on a needle core biopsy of the breast in several cases. Because the treatment is likely the same regardless of the presence or absence of microinvasion, which might be more easily resolved at excision, a note stating the difficulty may be enough to avoid that type of disagreement. A similar approach may be taken with cellular fibroglandular lesions. In contrast, some lesions, such as skin biopsies for basal cell carcinoma, simply required more sectioning to identify the lesion. With a clinical suspicion of basal cell carcinoma, multiple sections may need to be performed routinely to identify all such lesions. Finally, recognition of unusual and specific diagnostic patterns, such as cystic neutrophilic granulomatous inflammation in the breast,26 is necessary to obtain the appropriate microbiologic stains. In conclusion, false-negative and false-positive cases occur in approximately 1% of interlaboratory consultation. Approximately one-half of false-negative and positive cases might be prevented by the use of a combination of additional pathologic methods. Further study to define the best use of those multiple methods appears warranted. References 1. Jacques SM, Qureshi F, Munkarah A, Lawrence WD. Interinstitutional surgical pathology review in gynecologic oncology, II: Endometrial cancer in hysterectomy specimens. Int J Gynecol Pathol. 1998;17(1):42–45. 2. Safrin RE, Bark CJ. Surgical pathology signout: routine review of every case by a second pathologist. Am J Surg Pathol. 1993;17(11):1190–1192. 3. Whitehead ME, Fitzwater JE, Lindley SK, Kern SB, Ulirsch RC, Winecoff WF. Quality assurance of histopathology diagnoses: a prospective audit of three thousand cases. Am J Clin Pathol. 1984;81(4):487–491. 4. Lind AC, Bewtra C, Healy JC, Sims KL. Prospective peer review of surgical pathology. Am J Clin Pathol. 1995;104(5):560–566. 5. Abt AB, Abt LG, Olt GJ. The effect of interinstitution anatomic pathology consultation on patient care. Arch Pathol Lab Med. 1995;119(6):514–517. 6. Epstein JI, Walsh PC, Sanfilippo F. Clinical and cost impact of secondopinion pathology. Review of prostate biopsies prior to radical prostatectomy. Am J Surg Pathol. 1996;20(7):851–857. 7. Kronz JD, Westra WH, Epstein JI. Mandatory second opinion surgical pathology at a large referral hospital. Cancer. 1999;86(11):2426–2435. 8. Jacques SM, Qureshi F, Munkarah A, Lawrence WD. Value of second opinion pathology review of endometrial cancer diagnosed on uterine curettings and biopsies [abstract]. Mod Pathol. 1997;10(1):103A.
Arch Pathol Lab Med—Vol 137, December 2013
9. Bruner JM, Inouye L, Fuller GN, Langford LA. Diagnostic discrepancies and their clinical impact in a neuropathology referral practice. Cancer. 1997;79(4): 796–803. 10. Scott CB, Nelson JS, Farnan NC, et al. Central pathology review in clinical trials for patients with malignant glioma: a report of Radiation Therapy Oncology Group 83-02. Cancer. 1995;76(2):307–313. 11. Aldape K, Simmons ML, Davis RL, Miike R, Wiencke J, Barger G, et al. Discrepancies in diagnosis of neuroepithelial neoplasms: the San Fransisco Bay Area Adult Glioma Study. Cancer. 2000;88(10):2342–2349. 12. Hahm GK, Niemann TH, Lucas JG, Frankel WL. The value of second opinion in gastrointestinal and liver pathology. Arch Pathol Lab Med. 2001; 125(6):736–739. 13. Grzybicki DM, Raab SS, Janosky JE, et al. Anatomic pathology and patient safety: it’s not an error: it’s a diagnostic misadventure! Am J Clin Pathol. 2008; 129(1):167–168; author reply, 169. 14. Renshaw AA, Gould EW. Comparison of disagreement and error rates for three types of interdepartmental consultations. Am J Clin Pathol. 2005;124(6): 878–882. 15. Tsung JS. Institutional pathology consultation. Am J Surg Pathol. 2004; 28(3):399–402. 16. Manion EM, Cohen MB, Weydert JA. Mandatory second opinion in surgical pathology referral material: clinical consequences of major disagreements. Am J Surg Pathol. 2008;32(12):732–737; erratum, Am J Surg Pathol. 2008; 31(12):1919–1922. 17. Gupta D, Layfield LJ. Prevalence of inter-institutional anatomic pathology slide review: a survey of current practice. Am J Surg Pathol. 2000;24(2):280–284. 18. Jacques SM, Qureshi F, Munkarah A, Lawrence WD. Interinstitutional surgical pathology review in gynecologic oncology, I: cancer in endometrial curettings and biopsies. Int J Gynecol Pathol. 1998;17(1):36–41. 19. Renshaw AA. Measuring and reporting errors in surgical pathology: lessons from gynecologic cytology. Am J Clin Pathol. 2001;115(3):338–341. 20. Renshaw AA, Gould EW. Measuring the value of review of pathology material by a second pathologist. Am J Clin Pathol. 2006;125(5):737–739. 21. Renshaw AA, Gould EW. Comparison of disagreement and amendment rates by tissue type and diagnosis: identifying cases for directed blinded review. Am J Clin Pathol. 2006;126(5):736–739. 22. Renshaw AA. Comparing methods to measure error in gynecologic cytology and surgical pathology. Arch Pathol Lab Med. 2006;130(5):626–629. 23. Fowkes FGR. Diagnostic vigilance. Lancet. 1986;327(8479):493–494. doi: 10.1016/S0140-6736(86)92942-9. 24. Evans KK, Tambouret RH, Evered A, Wilbur DC, Wolfe JM. Prevalence of abnormalities influences cytologists’ error rates in screening for cervical cancer. Arch Pathol Lab Med. 2011;135(12):1557–1560. 25. Ducatman BS, Bentz JS, Fatheree LA, et al. Gynecologic cytology proficiency testing failures: what have we learned?: observations from the College of American Pathologists gynecologic cytology proficiency testing program. Arch Pathol Lab Med 2011;135(11):1442–1446. 26. Renshaw AA, Derhagopian RP, Gould EW. Cystic neutrophilic granulomatous mastitis: an underappreciated pattern strongly associated with grampositive bacilli. Am J Clin Pathol. 2011;136(3):424–427.
Reducing FN and FP Diagnoses—Renshaw & Gould 1773
