Failure of Three Decision Rules to Predict the Outcome of In-hospital Cardiopulmonary Resuscitation MARK H. EBELL, MD, MS, JAMES A. KRUSE, MD, MINDY SMITH, MD, MS, JEANNE NOVAK, RN, JOELLE DRADER-WILCOX The objective of this study was to evaluate three decision-support tools (the Pre-Arrest Morbidity or PAM score, the Prognosis After Resuscitation or PAR score, and the Acute Physiology and Chronic Health Evaluation or APACHE Ill score) for their abilities to predict the outcomes of in-hospital cardiopulmonary resuscitation (CPR). The medical records of all 656 adult inpatients undergoing CPR during a two-to-three-year period in three large hospitals were retrospectively reviewed, and demographic and clinical variables were abstracted. Of 656 patients undergoing resuscitation, 246 (37.6%) survived the resuscitation attempt long enough to be stabilized (immediate survival), but only 35 (5.3%) survived to discharge. Only 11 patients had PAM scores higher than 8, none of whom survived to discharge; 131 patients had PAR scores above 8, of whom six survived to discharge. The PAR score and the APACHE Ill score had the greatest areas under the receiver operating characteristic curves (when predicting the outcome of survival to discharge), although no individual area for either outcome was greater than 0.6. None of the decision-support tools studied was able to effectively discriminate between survivors and non-survivors for the outcomes of immediate survival and survival to discharge following in-hospital CPR. This is consistent with previous work utilizing the APACHE II score, which did not identify a threshold above which patients did not benefit from CPR. The findings for the PAR score and the PAM score stand in contrast to previous studies that found them to be potentially useful decision rules. Further work is needed to develop a decision-support tool that better discriminates between survivors and non-survivors of in-hospital CPR. Key words: resuscitation; prognosis; cardiopulmonary resuscitation; decision support; decision making. (Med Decis Making 1997;17:171-177)
Accurate prediction of the outcomes of in-hospital cardiopulmonary resuscitation (CPR) would be helpful to patients and their physicians who must decide whether to forego this intervention. In fact, a recent study showed that information about the prognosis of CPR significantly altered the decisions of a group of patients about do-not-resuscitate (DNR) orders.’ Previous studies have addressed the question of outcome prediction using decision rules derived from multivariate models, 2-5 existing measures of severity of illness, 6 and artificial intelligence techniques.’ To
be accepted by clinicians, however, it is important that any proposed decision support tool be validated in a population or populations distinct from that in which it was developed.’ Several decision-support tools have been suggested in recent years for the prediction of survival. The first proposed was the Pre-Arrest Morbidity (PAM) score. 3 It consists of 15 variables, each given a value of 0, 1, or 3; a score of more than 8 predicts failure to survive to hospital discharge. Variables were chosen based on the results of a study9 and the authors’ clinical! judgment. The Prognosis After Resuscitation (PAR) score was based on the results of a meta-analysis, and consists of 8 variables, each assigned a score of -2 to 10 points.’ Again, a score higher than 8 predicts failure to survive to discharge following in-hospital CPR. Both of these rules were evaluated in a study that took place in an Irish population distinct from that used to develop the original decision rules. Using the original threshold of 8, the PAR score identified 25 of 274 of patients as nonsurvivors to discharge, none of whom survived (sensitivity 9.1%, specificity 100%); the PAM score identified 5 of 274 as unlikely to benefit, none of whom
Received August 24, 1995, from the Department of Family Medicine (MHE) and the Division of Critical Care Medicine, Department of Internal Medicine (JAK), Wayne State University; and the Department of Family Practice, Michigan State University (MS). Ms. Novak is in practice in Urbana, Illinois, and Ms. Drader-Wilcox is on the ‘staff of the School of Medicine, Wayne State University. Revision accepted for publication September 4, 1996. Supported in part by a grant from the American Academy of Family Physicians/Foundation. Address correspondence and reprint requests to Dr. Ebell: Department of Family Practice, A 100 Clinical Center, Michigan State University, East Lansing, MI 48824. e-mail: (
[email protected]).
171
172
l
Ebeii, Kruse, Smith, Novak, Drader-Wilcox
MEDICAL DECISION MAKING
survived (sensitivity 1.8%, specificity 100%). The PAR and PAM scores performed better using a lower threshold of 5, with sensitivities of 21.5% and 6.2%, respectively; both rules maintained a specificity of 100% at this threshold (no patient who eventually survived to discharge was incorrectly identified as a
Table 1
l
non-survivor). Areas under the receiver operating characteristic (ROC) curves were 0.74 and 0.69, respectively.10 While not specifically designed to predict the outcome of CPR, previous work with the Acute Physiology and Chronic Health Evaluation (APACHE) II
Clinical and Laboratory Characteristics of immediate Survivors and Non-survivors, and Survivors and Non-survivors to Discharge immediate Survivors and Non-survivors Survivors
Demographic information Age Male gender Black race Nursing home resident Admitting diagnosis Myocardiai infarction Pneumonia Diagnoses and comorbidities Sepsis w/ lievated temperature Sepsis w/out elevated temperature Congestive heart failure Coronary artery disease Malignancy Metastatic malignancy Laboratory and clinical values Hematocrit Serum albumin (g/L) Serum creatinine Serum glucose Urine output in first 24 hours White blood ceil count Predictive scores PAR score PAM score APACHE iii
Nonsurvivors
Odds Ratio* (95% Cl)
Survivors and Non-survivors to Discharge Survivors
P
Nonsurvivors
Odds Ratio* (95% Cl)
P
62.3 years 49.6% 56.5% 8.1%
62.5 years 61.8% 62.0% 12.0%
7.3%
8.8%
13.3%
13.2%
1 .01(0.63-l .60)
0.987
5.1%
8.5%
0.58(0.29-1.13)
0.108t
14.3%
17.4%
0.79(0.51-1.23) 0.302t 2 0 . 6 %
26.6%
27.2%
0.97(0.68-1.39) 0.868f
27.0%
23.8%
1.19(0.83-1.70) 0.353.f
12.9% 6.5%
13.5% 4.7%
0 . 9 5 ( 0 . 6 0 - l .52) 0.833t 1.41(0.7-2.79) 0.321t
34.1% 30.5 g/L
36.0% 31.1 g/L
NA NA
34.9% 0.0066 0.354$ 3 1 . 3 g / L
35.3% 30.8 g/L
NA NA
0.810$ 0.6755
2.38 (pmoi/L) 176.3 (mmoi/L) 1,624 mL
NA
0.282ll
2.39 (pmoi/L) 180.0 (mmoi/L) 1,652 mL
NA
0.099ll
NA
0.7531
NA
1.61 (FmoWL) 163.1 0.421ll (mmoi/L) 0.345ll 1 , 9 8 4 mL
NA
0.0557
14.3 (109/L)
12.3 (109/L)
NA
0.2715
11.5 (109/L)
13.2 (1 OVL)
NA
0.1558
4.4 3.0 61.2
4.8 3.1 60.7
NA NA NA
0.348 0.585 0.831
3.7 2.9 52.8
4.7 3.1 61.4
NA NA NA
0.258 0.568 0.058
“2.29 (kmoi/L) 183.7 (mmoi/L) 1,748 mL
NA
0.8495 6 5 . 8 years 0.61(0.44-0.84) 0.002t 4 2 . 9 % 0.86(0.62-1.19) 0.368.f 3 7 . 1 % 0.64(0.37-1.11) O.lllt 8.6%
0.81(0.45-l .46) 0.480.t
NA
8.6%
62.2 years 58.0% 62.0% 10.6%
NA
0.178
0 . 5 4 ( 0 . 2 8 - l .07) 0.079t 0.36(0.18-0.72) 0.003.f 0.79(0.23-2.64) 0.700t
8.2%
1.05(0.31-3.54)
0.940.f
12.88%
1.69(0.72-3.97)
0.227.r
7.09%
1.27(0.37-4.31)
0.704t
16.0%
1.36(0.58-3.21)
0.476f
28.6%
26.9%
1.09(0.51-2.31)
0.828t
34.3%
24.5%
1.61(0.79-3.30)
0.192t
5.7% 0.0%
13.7% 5.6%
0 . 3 8 ( 0 . 0 9 - l .54) 0.176t 0.23(0.01-3.87) 0.248$
20.0%
8.82%
*Odds ratios and 95% confidence intervals (Cls) shown for dichotomous independent variables only. (Note: patients may have more than one comorbidity, and not all admitting diagnoses were categorized. tPearson chi-square; $Fisher’s exact test; two-tailed; PStudent’s t test: llMann-Whitney U test.
VOL 17/NO 2, APR-JUN 1997
score showed that higher scores are associated with failure to survive to discharge following CPR. However, there was no threshold above which failure to survive could be confidently predicted.’ The APACHE III score has been designed to more accurately measure severity of illness in critically ill patients,l1l and may therefore be a better predictor of CPR survival. The purpose of this study was to evaluate the APACHE III score, the PAR score, and the PAM score for their abilities to predict the outcomes of in-hospital CPR in a racially mixed population of patients in the United States. Because physicians are encouraged to discuss the desire for CPR with their patients while they are still competent, and because the onset of cardiopulmonary arrest cannot be predicted, it was felt that a decision rule would be most useful if it used information gathered within the first 24 hours of admission. Two outcomes were evaluated: immediate survival (return of spontaneous circulation with transfer to the intensive care unit) and survival to discharge. It was hypothesized that the PAR score would perform better than the PAM score, and that there would be no APACHE III score above which failure to survive could be reliably predicted.
Methods A detailed description of the methods appears elsewhere as part of an analysis of the association between race and survival following CPR.12 Briefly, patients were identified from the log of cardiopulmonary arrests from each of three participating hospitals (an urban trauma center, an urban teaching hospital, and a large university hospital), and this list was supplemented by a search of all patients with discharge diagnoses of cardiac arrest or CPR. Patients resuscitated in the operating room, whose only resuscitation occurred in the emergency department, or for whom the use of chest compressions, artificial ventilation or rescue breathing, and cardiac medications was not documented were excluded. Each medical record was reviewed, and pertinent demographic, clinical, and laboratory variables available to a treating physician within the first 24 hours of hospital admission were abstracted. Sepsis with fever was defined as clinical evidence suggestive of infection (dysuria, cough, mental status change, etc.), plus tachypnea (>20 breaths/min), tachycardia (> 90 beats/mini, and either hyperthermia (temp > 38.4C) or hypothermia (I5 >20 >20
30.3 19.7 29.6
30/l 52 (19.7%) 36/l 92 (I 9.6%)
37/l 65 (22.4%)
4166 (6.1%) 2147 (4.3%) 3l70 (4.3%)
Bialecki and Woodward, 1995” Bialecki and Woodward, 1995”
APACHE II PDR APACHE II PDR
Admission Prior to arrest
>50 >50
10.0 14.5
391215 (16.1%) 39/201 (19.4%)
l/24 (4.2%) l/34 (2.9%)
Ebeil et al., 1996 (current) Ebeil et al., 1996 (current)
APACHE Iii APACHE iii
Admission Admission
>90 >I10
12.3 4.9
331575 (5.7%) 351624 (5.6%)
2/61 (2.5%) 0132 (0%)
Ebeil et al., 1996 (current) O’Keeffe and Ebeli, 1994” George, et al., 196g3 O’Keeffe and Ebeil. 1994”
PAM PAM PAM PAM
Admission Admission Uncertain Prior to arrest
>6 >6 >7 >5
1.7 1.6 17.1 6.2
351645 25/269 34/l 16 251257
O/l 1 (0%)
Ebeli et al., 1996 (current) O’Keeffe and Ebell, 1994”’ Ebeil and Preston, l9936 O’Keeffe and Ebell, 1994”
PAR PAR PAR PAR
Admission Prior to arrest Admission Prior to arrest
>8 >8 >8 >5
20.3 8.8 17.0 21.5
29/515 (5.6%)
Ebell, 1992”
Neural network
Admission
