Journal of Internal Medicine 1997; 242: 395–400
Improving empirical antibiotic treatment: prospective, nonintervention testing of a decision support system L. LEIBOVICI a b , V. GITELMAN c , Y. YEHEZKELLI a b , O. POZNANSKI a , G. MILO a , M. PAUL a & P. EIN-DOR c From the aDepartment of Medicine, Rabin Medical Centre, Beilinson Campus, Petah-Tiqva, bSackler Faculty of Medicine, and cFaculty of Management, Tel-Aviv University, Ramat-Aviv, Tel-Aviv, Israel
Abstract. Leibovici L, Gitelman V, Yehezkelli Y, Poznanski O, Milo G, Paul M & Ein-dor P (Rabin Medical Centre, Petah-Tiqva, Sackler Faculty of Medicine, and Faculty of Management, Tel-Aviv University, Ramat-Aviv, Tel-Aviv, Israel). Improving empirical antibiotic treatment: prospective, nonintervention testing of a decision support system. J Intern Med 1997; 242: 395–400. Objectives. Develop a problem-orientated and databased decision support system (DSS) to improve empirical antibiotic treatment, and compare the performance of the system to that of the physician. Design. The DSS was tested in a prospective, noninterventional, comparative cohort study. Setting. University hospital in Israel. Subjects. Consecutive patients (n 5 496) in four departments of internal medicine suspected of harboring a moderate to severe bacterial infection. Interventions. None. Main outcome measures. The percentage of appropriate empirical antibiotic treatments. Results. Out of 496 patients included in the study, 219 had positive cultures or serological tests. The
Introduction Severe bacterial infections are associated with an inhospital fatality rate of about 30%, and indicate a guarded long-term prognosis [1–4]. Appropriate
Grant Support: Chief Scientist, Ministry of Health, Jerusalem, Israel. Preliminary results were presented at the 17th Annual Meeting of the Society for Medical Decision Making, October 1995, Tempe, Arizona. © 1997 Blackwell Science Ltd
physicians prescribed inappropriate empirical antibiotic treatment in 91 of 219 patients (42%); whilst the recommendations of the system were inappropriate in 50 patients (23%) (P , 0.05). Superfluous treatment was prescribed in 15% of patients by the physician, and in 11% by the system. Out of the 91 patients given inappropriate treatment by the physician, the DSS advised treatment to which the pathogens were susceptible in 61 patients. The advantage of the DSS over the physician was most evident in multiresistant gramnegative isolates, enterococci and Staphylococcus aureus. Out of the 277 patients with negative cultures, the DSS advised narrower-spectrum antibiotic treatment than prescribed by the physicians in 27% of patients, and broader-spectrum in 13%. Conclusion. A problem-orientated, data-based DSS outperformed physicians in the choice of appropriate empirical antibiotic treatment, and recommended less broad-spectrum antibiotics. Keywords: antibiotic treatment, decision-support system.
empirical antibiotic treatment (i.e. to which the infecting pathogen will be shown to be susceptible in vitro) reduces mortality by half [1,4]. Unfortunately, physicians prescribe inappropriate antibiotics in 20%–50% of patients with severe infections [5–7]. In about a quarter of patients, superfluous antibiotic treatment is given [6–8]. Data available within hours of suspecting a moderate to severe infection can probably be used to improve the physician’s ability to predict the severity of disease, the most likely pathogen and its suscepti395
396
L. LEIBOVICI et al.
Table 1 The susceptibility of six pathogens to antibiotic drugs in Beilinson Campus during 1994–96 Gram-negative:
Drug ————— Piperacillin Ceftazidime Gentamicin Amikacin Imipenem Ofloxacin
Gram-positive:
E. coli Pseudomonas Acinetobacter Staph. aureus Coag.-neg. Staph. Enterococcus ———————————————————————————————————————————————————— C N C N C N Drug C N C N C N ————— — 47 32 71 50 50 22 Methicillin 71 34 50 33 71* 62* 94 69 83 69 40 16 Vancomycin 100 100 100 100 92 88 83 64 79 50 40 28 90 76 90 74 80 47 100 98 83 67 100 94 82 55 71 46 40 13
Percentage of patients; data are based on results of blood cultures. C, community acquired; N, nosocomial. * Susceptibility to ampicillin.
a b
bility to antibiotics [8–13]. An attractive way for improving use of such data is incorporating them in decision-support-systems (DSSs). However, large diagnostic DSSs, portraying a whole domain, did not prove successful [14,15]. For the present study, we compiled a DSS to improve empirical antibiotic treatment of suspected moderate to severe bacterial infections. The DSS is based on the following premises: (i) it is targeted at cases in which the physician often errs in prescribing appropriate empirical antibiotic treatment, (ii) it is based on prediction models derived from large databases — most of the models were validated in sites other than the original one, and (iii) the DSS addresses local data on prevalence of pathogens and susceptibility to antibiotics. These databases can be changed when transporting the DSS to other sites. Seven pathogens (Staphylococcus aureus, coagulase-negative staphylococci, Pseudomonas aeruginosa, Acinetobacter sp., enterococci, anaerobes and Candida sp.) accounted for 60% of bloodstream infections given inappropriate empirical antibiotic treatment, but only for 20% of those given appropriate treatment [5]. The system is designed to support the physician’s decision, mainly in patients at high risk for infections caused by these pathogens. We compared the prediction of the system as to the most likely pathogens and the appropriateness of the antibiotics prescribed by the DSS to those of the attending physicians in a group of patients suspected of harboring moderate to severe bacterial or fungal infections in a department of medicine. The aim of the present study was to test whether the system is more accurate than the attending physicians, and we did not interfere with the clinical decisions of the physician.
Patients and methods Setting Beilinson Campus is a 900-bed university hospital serving an elderly, urban population of about 300 000. During the study period, 86% of medical patients were admitted through the emergency ward and distributed at random between six departments of internal medicine. There were no guidelines for antibiotic treatment in the hospital, but for neutropenic patients. The use of several antibiotic drugs is restricted. In general, the resistance of microorganisms to antibiotic drugs in our hospital is high (Table 1). The decision support system Questions as to demographic details, data on underlying disorders, functional capacity, use of antibiotics and immunosuppressive drugs, presentation of the infectious episode, the most likely source of infection and laboratory test results are supplied by the user to the DSS. Using previously derived clinical prediction rules, the DSS presents the following predictions: 1 In a patient without an obvious source of infection, the probability of a bacterial infection [16,17] 2 Probability of bacteraemia [10,18] 3 In patients with community-acquired pneumonia, the probability of a severe infection or a fatal outcome [11]. This is the single rule that was neither derived nor validated in our hospital before the trial. Rules 2 and 3 and the presence of frank septic shock were used as an indication for the severity of infection. 4 Probability of an infection caused by each of the seven pathogens at high risk for inappropriate treatment [12,13,19 and unpublished material]. © 1997 Blackwell Science Ltd Journal of Internal Medicine 242: 395–400
DSS FOR ANTIBIOTIC TREATMENT
397
5 If the patient is not at high risk for any of the seven pathogens, the DSS presents a simple list of pathogens according to the site of infection and whether it is nosocomial or community-acquired, based on local databases. For two infections (community acquired pneumonia and meningitis) the data were taken from the literature, and not from the local databases. 6 For patients suspected of harboring a gram-negative pathogen, the probability of the pathogen being a multiresistant one [8,20,21] 7 For patients suspected of harboring an infection caused by Staphylococcus aureus, the probability of methicillin resistance [unpublished data]. At this stage, the DSS chooses a ‘coverage rate’, i.e. the percentage of pathogens that the empirical treatment must cover. During a preliminary test of the system on retrospective cases, this coverage rate was arbitrarily chosen as 80% for severe infections, and 70% for moderate ones. The system then searches arrays of antibiotic drugs to select the least member of the array that still affords the coverage rate. The arrays, arranged in order of ascending coverage, but also of ascending cost and ecological impact, were as follows: 1 b–lactams and other drugs for gram-negative infections: ampicillin , cephalothin , cefuroxime , piperacillin , cefotaxime , quinolons , aztreonam , ceftazidime , imipenem. 2 Aminoglycosides: gentamicin , tobramicin , amikacin. 3 For gram-positive infections: penicillin , ampicillin , cloxacillin , cephalothin , piperacillin , clindamycin , vancomycin. Several treatments were obligatory: metronidazole or clindamycin for patients at medium to high risk for anaerobic infections; erythromycin for mycoplasma pneumonia; and amphothericin B for severe fungal infections. For patients at moderate to high risk for a pseudomonal infection, the array contained only antipseudomonal drugs. The decision whether to prescribe a combination of drugs for gram-negative infection or a single one was not made by the system, but left to the physician.
boring a moderate to severe bacterial infection (either community-acquired or nosocomial) were included in the study. Suspicion of a bacterial disease was raised if the patient was febrile (. 38 8C) or hypothermic (, 36 8C) without an obvious reason other than an infectious one; had a local finding compatible with infection as the main presenting complaint or sign; or had frank septic shock. A patient was included in the study only once. Blood cultures were drawn and an X-ray of the chest was performed for each patient. Cultures from other sites, direct stains and serological tests were ordered if indicated. The patients were reached by one of us (YY, OP, GM, MP, or LL) within 24 h of admission (in communityacquired cases) or of suspecting an infection (in hospital-acquired ones). The empirical antibiotic treatment (or lack thereof) was obtained from the chart and registered. The attending physician was asked as to the likelihood of a bacterial aetiology and bacteraemia (low, moderate or high), and the most likely pathogens (three choices). Data available at that point were then entered into the DSS. The patients’ data and the prediction of the DSS were automatically recorded. The attending physician was not aware of the recommendations of the system, and no changes in the treatment were made because of the study. Ten to 14 days later data as to the results of the cultures, serological testing, and final diagnosis of the episode were recorded. The recommendations of the system and the actual empirical antibiotics prescribed were compared to the results of cultures and serological tests. Antibiotic treatment was defined as inappropriate if the pathogen was not susceptible in vitro to the prescribed antibiotic/s, or if no empirical antibiotic treatment was prescribed for a proven bacterial infection; superfluous, if the antibiotic was too broad-spectrum (judging by the in vitro susceptibility), or not needed; and appropriate in the rest of the cases. As both the recommendations of the DSS and the physicians’ choices were compared to the results of the cultures, we used the McNemar test to look for statistical significance.
Testing the DSS
Results
From September 1994 through June 1995, and February through April 1996, all patients in four departments of internal medicine suspected of har-
Out of 496 patients included in the study, 250 (50.4%) were men. The median age was 75 years, range 16–100. Twenty-two per cent of episodes were
© 1997 Blackwell Science Ltd Journal of Internal Medicine 242: 395–400
398
L. LEIBOVICI et al.
Table 2 Final diagnosis in all 496 patients suspected of harbouring a moderate to severe bacterial infection, and in patients with a known pathogen Diagnosis
All patients (n 5 496)
Lower respiratory tract infection Urinary tract infection Soft tissue and bone infection Infection of unknown source Neutropenic fever Viral infection Acute sinusitis or pharyngitis Biliary tract infection Intra-abdominal infection Other infections Noninfectious
179 (36)
21 (10)
126 (25) 47 (9) 45 (9) 15 (3) 10 (3) 8 (2)
118 (58) 13 (6) 25 (11) 9 (4) 4 (2) 5 (2)
5 (1) 3 (1) 13 (3) 45 (9)
Patients with a known pathogen pathogen (n 5 219)
2 (1) 1 (1) 11 (5) –
Percentages are given in parantheses.
a
Table 3 Pathogens that grew in cultures from blood, urine, and other sites Pathogen
Blood culture
Urine culture
Other sites
Escherichia coli Proteus mirabilis Pseudomonas sp. Klebsiella sp. Acinetobacter sp. Salmonella sp. Coagulase negative staphylococci Staph. aureus Enterococcus sp Strep. pneumoniae Other streptococci Others Total
24 (28.9) 7 (8.4) 3 (3.6) 4 (4.8) 2 (2.4) 1 (1.2) 4 (4.8)
57 (48.3) 11 (9.3) 15 (12.7) 22 (18.6) 2 (1.7) 0 0
14 (25.0) 2 (3.5) 11 (19.6) 3 (5.4) 1 (1.9) 0 0
3 (3.6) 8 (9.6) 7 (8.9) 3 (3.6) 17 (20.5) 83 (100)
0 1 (0.8) 0 0 10 (8.5) 118 (100)
5 (8.9) 3 (5.4) 3 (5.4) 2 (3.5) 12 (21.4) 56 (100)
a
Percentages are given in parentheses.
hospital-acquired. The final diagnosis of the episodes are given in Table 2. Blood cultures were positive (and not contaminated) in 83 patients (16.7%). Of the 126 patients with urinary tract infections, 118 had positive urine cultures. Twenty-seven of them were bacteraemic. Cultures from other sites were positive in 56 patients, nine of them with positive blood cultures. Pathogens grown in positive cultures are detailed in Table 3. Two patients with pneumonia had positive serological tests for Mycoplasma pneumoniae. The true pathogen was included amongst the three choices of the physician in 55% of the 219 patients
with a known pathogen, versus 78% true prediction by the DSS (P , 0.05). The most common antibiotic drugs prescribed by the physician were cefuroxime, erythromycin and gentamicin for communityacquired infections; and cefotaxime, cefuroxime, gentamicin and ceftazidime for hospital-acquired infections (data not shown). Table 4 shows the appropriateness of the empirical antibiotic treatment prescribed by the physicians versus the one recommended by the DSS. The physicians prescribed inappropriate treatment in 42% of patients. Superfluous treatment was given in 15% of patients. The recommendations of the DSS were inappropriate in 23% of patients, and superfluous in 11% (P , 0.05). Use of the system would have reduced the rate of inappropriate treatments (in patients with a known pathogen) by 19% (95% confidence interval 14%–24%) (41 of 219 patients) without increasing the rate of superfluous treatments. Of the 91 patients in which the physician prescribed inappropriate antibiotic treatment, the DSS prescribed antibiotics to which the pathogen was shown to be susceptible (appropriate and superfluous) in 61 (67%). The major pathogens in this group were Pseudomonas aeruginosa in 15 patients, Escherichia coli in 13 patients, Klebsiella pneumoniae in 10 patients, enterococci in five patients, Staphylococcus aureus and Acinetobacter sp. in four patients each. Of 50 patients for which inappropriate treatment was prescribed by the DSS, 20 were given appropriate or superfluous treatment by the physician (40%). Eighteen patients out of the 50 had a wrong initial diagnosis. Overall, the final diagnosis of the presenting episode was other than the initial one in 69 of the 496 patients (14%). The pathogen was unknown (or the aetiology noninfectious) in 277 patients. In 76 of these (27%) the DSS recommended antibiotic treatment that was less broad-spectrum than the one actually given. In most of these cases the DSS prescribed penicillin for lower respiratory tract infection, or no antibiotic treatment at all. In 36 patients out of this group (13%), the DSS advised on a broader-spectrum antibiotic than was given, mainly in cases of hospital-acquired infections.
Discussion The use of the DSS would have reduced the rate of inappropriate antibiotic treatment by half. The DSS recommended antibiotic treatment to which the © 1997 Blackwell Science Ltd Journal of Internal Medicine 242: 395–400
DSS FOR ANTIBIOTIC TREATMENT
399
Table 4 Antibiotic treatment recommended by the DSS compared to that by the physician in 219 patients with positive cultures or serological testing
(DSS) ———— Inapprop Approp Superfl Total
Inappropriate
Appropriate
Superfluous
Total
30 43 18 91
9 81 5 95
11 21 1 33
50 145 24 219
(Physician)
Inapprop, inappropriate treatment; Approp, appropriate treatment; Superfl, superfluous treatment.
a
pathogen was shown to be susceptible in 77% of patients with a documented bacterial aetiology of infection versus 58% prescribed by the physician. The number of superfluous antibiotic treatments was not increased. In the whole group of patients (including those with negative cultures) the DSS recommended less antibiotics, and less broad-spectrum ones. As expected from the formulation of the system, the major advantages of the DSS were in covering multiresistant gram-negative isolates, enterococci and Staphylococcus aureus. The suggestions of the DSS are based on the source of infection entered by the physician. In 14% of cases, the final diagnosis was different from the original one. A careful examination of the mistakes and the addition of a module to improve recognition of the source of infection might improve the performance of the DSS. The mistakes of the physicians and the DSS did not overlap. A significant percentage of patients prescribed inappropriate treatment by the physicians could have benefitted from the recommendations of the DSS. It might be argued that our hospital has a high rate of resistant strains, and of inappropriate empirical antibiotic courses, and thus a DSS performed better than the physicians but would not necessarily help in other settings. However, the proportion of inappropriate antibiotic treatments was similar to ours in hospitals with medium [6,7,9] and very low [22] percentages of resistant bacteria; and a DSS improved accuracy in both these settings [9,22]. Overall, it seems that systems that are problem-orientated, and systems that serve to alert and remind, have a better success than large diagnostic ones [14,23]. Our system does not interfere with the whole diagnostic process. In patients likely to be treated correctly, the system presents and uses a simple table of pathogens’ prevalence. It alerts the physi© 1997 Blackwell Science Ltd Journal of Internal Medicine 242: 395–400
cian when the patient is at risk for an infection caused by seven pathogens, that are often treated inappropriately. Another point worth emphasizing is that the system incorporates both knowledge and data. The DSS was not operated by the physicians themselves. We do not know if the physicians will be ready to adopt the recommendations of the DSS and to change their practice in accordance. The performance of the DSS may deteriorate when transferred to another site, although the system was built with this option in mind. Most of the prediction rules on which the system is based were validated in a site other than the original one, and the system addresses local databases. The experience of the group at LDS Hospital (Salt Lake City, Utah, USA) [9,24] is encouraging. They have shown that the use of computer decision support to improve antibiotic practice was well accepted by physicians, improved empirical antibiotic selection, reduced costs and impeded the emergence of resistant pathogens. In view of their experience, our results show that a data-based and problem-orientated DSS can significantly reduce the number of patients given inappropriate empirical antibiotic treatment for moderate to severe infections, without increasing the percentage of superfluous treatments. We intend to test the system in a controlled trial at several sites, to show that it can improve patients’ outcome.
References 1 Leibovici L, Samra Z, Konigsberger H, Drucker M, Ashkenazi S, Pitlik SD. Long-term survival following bacteremia or fungemia. JAMA 1995; 274: 807–12. 2 Perl TM, Dvorak M, Hwang T, Wenzel RP. Long-term survival and function after suspected gram-negative sepsis. JAMA 1995; 274: 338–45.
400
L. LEIBOVICI et al.
3 Geerdes HF, Ziegler D, Lode H, Hund M, Loehr A, Fangmann W et al. Septicemia in 980 patients at a university hospital in Berlin: prospective studies during 4 selected years between 1979 and 1989 Clin Infect Dis 1992; 15: 991–1002. 4 Bryan CS, Reynolds KL, Brenner ER. Analysis of 1,186 episodes of gram-negative bacteremia in non-university hospitals: the effects of antimicrobial therapy. Rev Infect Dis 1983; 5: 629–38. 5 Leibovici L, Konigsberger H, Pitlik SD, Samra Z, Drucker M. Patients at risk for inappropriate antibiotic treatment of bacteremia. J Intern Med 1992; 231: 371–4. 6 Dunagan WC, Woodward RS, Medoff G, Gray JL, Casabar E, Lawrenz C et al. Antibiotic misuse in two clinical situations: positive blood cultures and administration of aminoglycosides. Rev Infect Dis 1991; 13: 405–12. 7 Maki DG, Schuna AA. A study of antibiotic misuse in a university hospital. Am J Med Sci 1978; 275: 271–82. 8 Leibovici L, Konigsberger H, Pitlik SD, Samra Z, Drucker M. A predictive index optimizing empiric treatment of gram-negative bacteremia. J Infect Dis 1991; 163: 193–6. 9 Evans RS, Classen DC, Pestotnik SL, Lundsgaarde HP, Burke JP. Improving empiric antibiotic selection using computer decision support. Arch Intern Med 1994; 154: 878–84. 10 Leibovici L, Greenstain S, Cohen O, Mor F, Wysenbeek AJ. Bacteremia in febrile adults: a clinical index for diagnosis. Arch Intern Med 1991; 151: 1801–6. 11 Fine MJ, Smith DN, Singer DE. Hospitalization decision in patients with community-acquired pneumonia — A prospective cohort study. Am J Med 1990; 89: 713–21. 12 Leibovici L, Gransden WR, Eykyn SJ, Konisberger H, Drucker M, Pitlik SD, Phillips I. Clinical index to predict bacteraemia caused by staphylococci. J Intern Med 1993; 234: 83–9. 13 Gransden WR, Leibovici L, Eykyn SJ, Pitlik SD, Samra Z, Konigsberger H et al. Risk factors and a clinical index for diagnosis of Pseudomonas aeruginosa bacteraemia. J Clin Microbiol Infect 1996; 1: 119–23. 14 Johnston ME, Langton KB, Haynes RB, Mathieu A. Effects of computer-based clinical decision support systems on clinical performance and patient outcome: A critical appraisal of research. Ann Intern Med 1994; 120: 135–42. 15 Berner ES, Webster GD, Shugerman AA, Jackson JR, Algina J,
16
17
18
19
20
21
22
23
24
Baker AL et al. Performance of four computer-based diagnostic systems. N Engl J Med 1994; 330: 1792–6. Mellors JW, Horwitz RI, Harvey MR, Horwitz SM. A simple index to identify occult bacterial infections in adults with acute unexplained fever. Arch Intern Med 1987; 147: 666–71. Leibovici L, Cohen O, Wysenbeek AJ. Occult bacterial infection in adults with unexplained fever: validation of a diagnostic index. Arch Intern Med 1990; 150: 1270–72. Yehezkelli Y, Subah S, Elhanan G, Raz R, Porter A, Regev A, Leibovici L. Early prediction of bacteremia: Testing of two prediction rules in a university and community hospital. J Gen Intern Med 1996; 11: 98–103. Leibovici L, Gransden WR, Weinberger M. Predicting candidemia. Presented at the 9th International Symposium on Infections in the Immunocompromised Host, Assissi, Italy, 1996 (abstract). Gransden WR, Phillips I. Predictive index for optimizing treatment of gram-negative bacteria. J Infect Dis 1991; 164: 211–1 (letter). Leibovici L, Greenstain S, Cohen O, Wysenbeek AJ. Toward improved empiric treatment of moderate to severe urinary tract infections. Arch Intern Med 1992; 152: 2481–6. Andreassen S, Leibovici L, Schøenheyder HC, Olesen KG, Kristensen B. Decision support system in bacteremia: evaluation of a prototype for empirical treatment of Danish patients with urosepsis. Presented at the Bi-National Israel-Denmark Symposium on Optimal Antibiotic Treatment of Severe Bacterial Infections, Herzlia, Israel, 1996, (abstract). Heathfield HA, Wyatt J. Philosophies for the design and development of clinical decision-support systems. Meth Inform Med 1993; 32: 1–8. Pestotnik SL, Classen DC, Evans RS, Burke JP. Implementing antibiotic guidelines through computer-assisted decision support: Clinical and financial outcomes. Ann Intern Med 1996; 124: 884–90.
Received 10 March 1997; accepted 10 June 1997. Correspondence: Dr L. Leibovici, Department of Medicine E, Beilinson Campus, Rabin Medical Centre, Petah-Tiqva 49100, Israel (fax: 1972 39376505; e-mail:
[email protected].).
© 1997 Blackwell Science Ltd Journal of Internal Medicine 242: 395–400
