Renal Failure, 27:547–556, 2005 Copyright D 2005 Taylor & Francis Inc. ISSN: 0886-022X print / 1525-6049 online DOI: 10.1080/08860220500198771
CLINICAL STUDY
Mortality Risk Factors and Validation of Severity Scoring Systems in Critically Ill Patients with Acute Renal Failure Emerson Quintino Lima Division of Nephrology, School of Medicine—University of Sao Paulo, Sa˜o Paulo, Brazil and Department of Nephrology, Sao Jose do Rio Preto Medical School, Sa˜o Paulo, Brazil
Dirce Maria T. Zanetta Department of Epidemiology, Sao Jose do Rio Preto Medical School, Sa˜o Paulo, Brazil
Isac Castro and Luis Yu Division of Nephrology, School of Medicine—University of Sao Paulo, Sa˜o Paulo, Brazil
Background. Risk stratification and prediction of outcome in acute renal failure patients in the intensive care unit are important determinants for improvement of patient care and design of clinical trials. Methods. In order to identify mortality risks factors and validate general and specific predictive models for acute renal failure (ARF) patients in the intensive care unit (ICU), 324 patients were prospectively evaluated. Multivariate analysis by logistic regression was utilized for identification of mortality risk factors. Discrimination and calibration were used to evaluate the performance of the following models at referral to nephrologist and at initiation of renal replacement therapy: APACHE II, SAPS II, LODS, and ATN-ISI. Organ failure was assessed by SOFA and OSF. Results. The hospital mortality rate was 85%. The identified mortality risk factors were: age 65 yr, BUN 70 mg/dL, ARF of septic origin, and previous hypertension. Serum creatinine 3.5 mg/dL, systolic blood pressure 100 mm Hg, and normal consciousness were associated with mortality risk reduction. Performance of all prognostic models was disappointing with unsatisfactory calibration and underestimation of mortality on the day of referral to the nephrologist and at initiation of renal replacement therapy. Conclusions. Cross-validation of prognostic models for ARF resulted in poor performance of
Address correspondence to Dr. Emerson Quintino Lima, Faculdade de Medicina de Sa˜o Jose´ do Rio Preto, Av. Brigadeiro Faria Lima 5416, Sa˜o Jose´ do Rio Preto, Sa˜o Paulo 15090-000, Brazil; Fax: 55-17-2105712; E-mail:
[email protected]/
[email protected]
all studied scores. Therefore, a specific model is still warranted for the design of clinical trials, comparison of studies, and for prediction of outcome in ARF patients, especially in the ICU. Keywords
INTRODUCTION In the last decades, significant findings on the understanding of the pathophysiological mechanisms of experimental acute renal failure (ARF) have been reported. However, few of them resulted in clinical measures to prevent, treat, or accelerate renal function recovery in patients with ARF. Despite the apparent improvement in survival rates resulting from new dialysis strategies, the mortality rate of ARF remains high and may reach 80% in intensive care unit (ICU) patients.[1 – 4] There are multiple difficulties involved in the comparison of clinical studies in ARF patients: use of different definitions of ARF, study design, different evaluated outcomes, heterogeneity of patients, and associated therapies. Moreover, the assessment of severity of ARF patients by prognostic mortality models or severity scores systems has not been standardized yet. Prognostic scores may be used for disease severity stratification, comparison of patients, therapeutic response follow-up, or evaluation of medical assistance among different institutions. Although the general prognostic scores utilized in the ICU have not been originally developed for specific
547 Order reprints of this article at www.copyright.rightslink.com
acute renal failure, intensive care unit, mortality, prognostic scores
548
E. Q. Lima et al.
populations, they are still widely used in patients with ARF.[4 – 6] Mortality prediction for ARF patients is likely to be more accurate if specific prognostic scores were used for this population.[7,8] Controversial results have been reported on the ability of prognostic scores to predict mortality in ARF and which model for these patients is more appropriate.[2,9] Moreover, good performance of a prognostic model in one institution does not indicate that it will be as effective in other institutions or in different groups of patients.[10] On this background, the aims of this study were to identify mortality risk factors and determine the performance of general and specific prognostic models in severely ill patients with acute renal failure.
MATERIALS AND METHODS Study Cohort From June 2000 to June 2001, adult patients ( 18 years old) diagnosed with ARF in the ICU, and referred to the nephrologists, were prospectively evaluated. Acute renal failure was defined as serum creatinine increase to at least 2.0 mg/dL or an increase of 1.0 mg/dL in patients with baseline serum creatinine lower than 1.5 mg/dL. For patients with baseline creatinine ranging from 1.5 mg/dL to 3.0 mg/dL, ARF was defined as an increase of 50% or more.
hemolytic-uremic syndrome, and other obstetric complications), and medical (patients not included in previous categories). Initiation of renal replacement therapy and modality were indicated by the nephrologist. Renal function recovery was defined as a return to baseline serum creatinine values and partial recovery as a decrease of serum creatinine above baseline levels.
Prognostic Scores Acute Physiology Age and Chronic Health Evaluation II (APACHE II), Simplified Acute Physiology Score II (SAPS II), Logistic Organ Dysfunction System (LODS), Sequential Organ Failure Assessment (SOFA), Acute Tubular Necrosis Individual Severity Index (ATNISI), and Organ System Failure (OSF) were calculated by the same investigator on the day of referral to the nephrologist and at initiation of renal replacement therapy, according to the recommendations of the original studies.[7,12 – 16] The definition of renal failure of the original OSF score was modified (described earlier) and hepatic failure was defined as prothrombin activity < 50% and/or bilirubin> 2.0 mg/dL. The highest number of organ failures was six (cardiovascular, respiratory, renal, hematologic, hepatic and neurological). If a variable was not measured for a patient, it was assumed to be within the range of normal.
Exclusion Criteria Statistical Analysis Patients with baseline serum creatinine higher than 3.0 mg/dL, ARF etiology other than acute tubular necrosis (ATN), and those who received nephrologic care prior to ICU admission were excluded. The cause of ARF was considered to be ATN when a prerenal cause, urinary tract obstruction, acute interstitial nephritis, glomerulopathies, vascular, or systemic diseases were ruled out as the etiology of ARF.
ARF Classification Each patient’s ARF was classified as nephrotoxic or ischemic. Nephrotoxic ARF was classified when associated with nephrotoxic drugs and ischemic ARF when caused by hypotension, hemorrhage, low cardiac output, or sepsis. Sepsis was defined according to the American College of Chest Physicians/Society of Critical Care Medicine Consensus Conference.[11] The ARF was also classified as surgical (ARF within 72 hours of surgery), obstetric (HELLP syndrome, eclampsia, post-delivery
Performance of prognostic models that predict the risk of hospital mortality (APACHE II, SAPS II, LODS, and ATN-ISI) was tested by means of discrimination and calibration. Discrimination is the ability of the score to predict the outcome of patients. Discrimination was assessed by the ROC (receiver operating characteristic) curve area, which shows the rate of sensibility (correct identification of nonsurviving or true positive patients) and 1-specificity (incorrect identification of surviving or false positive patients). The values of the area under the ROC curve indicate the discrimination power of the score. Prognostic models with no discrimination power have an area under the ROC curve lower or equal to 0.5; scores with an area greater than 0.7 have satisfactory discrimination power, and those with an area greater than 0.9 have excellent discrimination power.[17] The area under the ROC curve was expressed in absolute values and dispersion was expressed by the 95% confidence interval (95% CI). Calibration evaluates the ability of a prognostic model to accurately predict mortality in all risk groups.
Mortality Risk Factors and Validation of Severity Scoring Systems
549
Table 1 Patients demographics
N (%) Age Sex Male Female Type of ARF Medical Surgical Obstetrical Etiology of ATN Ischemia Nephrotoxins
All
Survivors
Nonsurvivors
P
324 60.1 ± 16.7
49 (15.1%) 52.9 ± 17.6
275 (84.9%) 61.4 ± 16.2
0.003
213 (65.7%) 111 (34.3%)
33 (65.3%) 16 (32.7%)
180 (65.5%) 95 (34.5%)
0.79
221 (68.2%) 102 (31.5%) 1 (0.3%)
26 (53.1%) 22 (44.9%) 1 (2%)
195 (70.9%) 80 (29.1%) 0 (0%)
0.005
276 (85.2%) 48 (14.8%)
37 (75.5%) 12 (24.5%)
239 (86.9%) 36 (13.1%)
0.04
Table 2 Laboratory tests at referral to nephrologist and on the day of first dialysis
BUN (mg/dL) Referral Dialysis Creatinine (mg/dL) Referral Dialysis Sodium (mEq/L) Referral Dialysis Potassium (mEq/L) Referral Dialysis pH Referral Dialysis Bicarbonate (mEqL) Referral Dialysis Albumin (g/L) Referral Dialysis Protrombin time (%) Referral Dialysis Hematocrit (%) Referral Dialysis Leukocyte (mm3) Referral Dialysis Platelets ( 103/mm3) Referral Dialysis
All
Survivors
Nonsurvivors
P
68.7 ± 29.4 88.8 ± 35.9
57.5 ± 26.2 79.9 ± 35.5
70.8 ± 29.4 90.2 ± 35.5
0.002 0.23
3.8 ± 1.3 5.2 ± 2.8
4.0 ± 1.4 5.9 ± 3.1
3.7 ± 1.3 5.1 ± 2.8
0.15 0.26
139 ± 8 140 ± 8
140 ± 8 140 ± 6
139 ± 8 140 ± 8
0.80 0.85
4.8 ± 1.2 4.9 ± 1.2
4.6 ± 1.2 4.9 ± 1.2
4.8 ± 1.2 4.9 ± 1.2
0.19 0.68
7.29 ± 0.11 7.27 ± 0.10
7.35 ± 0.09 7.31 ± 0.09
7.29 ± 0.11 7.27 ± 0.10
0.001 0.06
17.8 ± 5.8 17.5 ± 5.4
19.6 ± 5.3 18.3 ± 3.3
17.4 ± 5.8 17.4 ± 5.6
0.017 0.47
23.7 ± 6.7 24.1 ± 6.5
25.8 ± 6.3 25.6 ± 6.9
23.4 ± 6.8 23.9 ± 6.5
0.026 0.33
47.8 ± 22.2 46.6 ± 21.0
51.3 ± 23.9 50.1 ± 24.0
47.2 ± 21.9 46.2 ± 20.7
0.27 0.44
29.9 ± 5.7 29 ± 5
29.2 ± 5.7 28 ± 4
30.0 ± 5.7 29 ± 5
0.22 0.97
12945 ± 7507 13927 ± 8073
11683 ± 5316 13994 ± 7516
13176 ± 7829 13920 ± 8147
0.24 0.93
133 ± 105 134 ± 103
148 ± 127 152 ± 141
131 ± 100 132 ± 99
0.41 0.54
E. Q. Lima et al.
550
Table 3 Severity scores at referral to nephrologist and on the day of first dialysis
APACHE II Referral Dialysis SAPS II Referral Dialysis ATN-ISI Referral Dialysis LODS Referral Dialysis SOFA Referral Dialysis OSF Referral Dialysis
All
Survivors
Nonsurvivors
P
31 ± 8.3 33 ± 7.4
24 ± 8.7 26.9 ± 7.5
32 ± 7.7 33.7 ± 7.1
< 0.001 < 0.001
49.4 ± 14.3 52.4 ± 13.4
37.2 ± 13.3 44.4 ± 12.6
51.5 ± 13.3 53.3 ± 13.2
< 0.001 0.002
0.57 ± 0.20 0.63 ± 0.18
0.42 ± 0.21 0.52 ± 0.20
0.59 ± 0.18 0.64 ± 0.17
< 0.001 0.004
7.5 ± 2.2 7.9 ± 2.2
6.3 ± 1.9 7.3 ± 1.9
7.6 ± 2.1 7.9 ± 2.2
< 0.001 0.25
12.8 ± 4.3 13.9 ± 4.1
10.5 ± 4.1 12.4 ± 4.5
12.9 ± 4.2 14.2 ± 3.9
< 0.001 0.07
3.0 ± 0.9 3.0 ± 0.9
2.7 ± 0.9 2.7 ± 0.9
3.0 ± 0.9 3.0 ± 0.9
0.06 0.06
Patients were divided according to the mortality intervals predicted by the scores in risk decile and calibration was defined as the degree of agreement between modelpredicted mortality and actual mortality for each risk strata. Adequacy of curve adjustments was assessed by Hosmer-Lemeshow chi-square test. A nonsignificant result suggests that the model has good calibration. Continuous data were analyzed by Student’s t test and by Mann-Whitney’s U test and expressed as mean ± standard deviation or by median and twenty-fifth to seventy-fifth percentile when appropriate. Categorical variables were expressed as percents and were analyzed by Pearson’s chi-squared test or Fisher’s exact test when appropriate. Multivariable logistic regression was carried out to obtain mortality risk factors using variables with a trend towards significance in the univariate analysis (P 0.15) on the day of referral to the nephrologist. Results were expressed by odds ratio (OR) and 95% CI. Differences were considered statistically significant when P< 0.05. Statistical analysis was carried out by SPSS version 10.0 (SPSS Inc., Chicago, IL, USA) and BMDP programs (BMDP Statistical Software Inc., Los Angeles, USA).
RESULTS During the study period, 324 patients were included for analysis and the hospital mortality rate was 85%. The ARF incidence in the ICU requiring nephrologic care was 4.6% (7080 patients admitted to the ICU in this period).
The ARF etiology was multifactorial, the mean age was 60.1 ± 16.7 years and the nonsurviving patients were older (Table 1). Among the comorbidities, systemic hypertension was the most prevalent (42.3%), followed by immunosuppression (20.7%), diabetes mellitus (19.7%), congestive heart failure (16.4%), neoplasias (15.2%), coronary disease (12.3%), obstructive pulmonary disease (12%), liver cirrhosis (10.2%), stroke (6.5%), and AIDS (3.4%). Only hypertension was more prevalent in the nonsurviving patients compared with surviving patients (45.1% vs. 26.5%, P < 0.05). Median time between the onset of ARF and consultation with nephrologist was 2 days (1 –5 days). On the day of referral to the nephrologist, most patients were on mechanical ventilation (77.5%) with higher prevalence among the nonsurvivors (81.5% vs. 55.1% in
Table 4 Mortality risk factors for ARF in the ICU at referral to nephrologist Risk factor BUN 70 mg/dL Sepsis ATN Age 65 years History of hypertension Creatinine 3.5 mg/dL Normal consciousness SBP 100 mmHg
Odds ratio
95% CI
P
6.02 3.09 2.81 2.53 0.42 0.27 0.25
2.13 – 17.0 1.27 – 7.51 1.25 – 6.34 1.08 – 5.95 0.19 – 0.94 0.13 – 0.57 0.11 – 0.53
< 0.01 < 0.05 < 0.05 < 0.05 < 0.05 < 0.001 < 0.001
Mortality Risk Factors and Validation of Severity Scoring Systems
551
1 - Specificity
1 - Specificity
Mortality (%)
1 - Specificity
Figure 1. ROC curves (A) and Calibration analysis (B), constructed by plotting observed death rate (- -& - -) and predicted death rate (— —) by APACHE II, SAPS II, LODS, and ATN-ISI stratified by deciles of risk at referral to nephrologist (*HosmerLemeshow chi-square test).
.
E. Q. Lima et al.
552
Figure 2. ROC curves (A) and Calibration analysis (B), constructed by plotting observed death rate (- -& - -) and predicted death rate (— —) by APACHE II, SAPS II, LODS, and ATN-ISI stratified by deciles of risk on the day of first dialysis (*HosmerLemeshow chi-square test).
.
Mortality Risk Factors and Validation of Severity Scoring Systems
surviving patients, P
