Clinical Chemistry 59:10 1497–1505 (2013)
Lipids, Lipoproteins, and Cardiovascular Risk Factors
Incremental Prognostic Value of Biomarkers beyond the GRACE (Global Registry of Acute Coronary Events) Score and High-Sensitivity Cardiac Troponin T in Non-ST-Elevation Acute Coronary Syndrome Christian Widera,1† Michael J. Pencina,2† Maria Bobadilla,3† Ines Reimann,1 Anja Guba-Quint,1 Ivonne Marquardt,1 Kerstin Bethmann,1 Mortimer Korf-Klingebiel,1 Tibor Kempf,1 Ralf Lichtinghagen,4 Hugo A. Katus,5 Evangelos Giannitsis,5 and Kai C. Wollert1*
BACKGROUND: Guidelines recommend the use of validated risk scores and a high-sensitivity cardiac troponin assay for risk assessment in non-ST-elevation acute coronary syndrome (NSTE-ACS). The incremental prognostic value of biomarkers in this context is unknown. METHODS: We calculated the Global Registry of Acute Coronary Events (GRACE) score and measured the circulating concentrations of high-sensitivity cardiac troponin T (hs-cTnT) and 8 selected cardiac biomarkers on admission in 1146 patients with NSTE-ACS. We used an hs-cTnT threshold at the 99th percentile of a reference population to define increased cardiac marker in the score. The magnitude of the increase in model performance when individual biomarkers were added to GRACE was assessed by the change (⌬) in the area under the receiver-operating characteristic curve (AUC), integrated discrimination improvement (IDI), and categoryfree net reclassification improvement [NRI(⬎0)]. RESULTS: Seventy-eight patients reached the combined end point of 6-month all-cause mortality or nonfatal myocardial infarction. The GRACE score alone had an AUC of 0.749. All biomarkers were associated with the risk of the combined end point and offered statistically significant improvement in model performance when added to GRACE (likelihood ratio test P ⱕ 0.015). Growth differentiation factor 15 [⌬AUC 0.039, IDI 0.049, NRI(⬎0) 0.554] and N-terminal pro–B-type natriuretic peptide [⌬AUC 0.024, IDI 0.027, NRI(⬎0) 0.438] emerged as the 2 most promising biomarkers.
1
Division of Molecular and Translational Cardiology, Department of Cardiology and Angiology, Hannover Medical School, Hannover, Germany; 2 Department of Biostatistics, Boston University and Harvard Clinical Research Institute, Boston, MA; 3 F. Hoffmann-La Roche, Pharma Research & Early Development, Basel, Switzerland; 4 Department of Clinical Chemistry, Hannover Medical School, Hannover, Germany; 5 Department of Medicine III, University of Heidelberg, Heidelberg, Germany. † Christian Widera, Michael J. Pencina, and Maria Bobadilla contributed equally to the work, and all should be considered as first authors. * Address correspondence to this author at: Molekulare und Translationale Kardiologie, Hans-Borst-Zentrum fu¨r Herz- und Stammzellforschung, Medizinische Hochschule Hannover, Carl-Neuberg-Str. 1, 30625 Hannover, Germany. Fax ⫹49-511-532-5307; e-mail
[email protected].
Improvements in model performance upon addition of a second biomarker were small in magnitude. CONCLUSIONS: Biomarkers can add prognostic information to the GRACE score even in the current era of high-sensitivity cardiac troponin assays. The incremental information offered by individual biomarkers varies considerably, however.
© 2013 American Association for Clinical Chemistry
Guidelines for the management of non-ST-elevation acute coronary syndrome (NSTE-ACS)6 emphasize the importance of risk stratification to match the intensity of therapy with an individual patient’s risk (1, 2 ). The guidelines recommend a standardized approach that uses scoring systems such as the Global Registry of Acute Coronary Events (GRACE) score to calculate risk and guide management decisions (1, 2 ). Because scores reflect only some disease dimensions related to outcome in NSTE-ACS, biomarkers addressing separate aspects of NSTE-ACS pathophysiology may provide additional information (3 ). With an increasing number of studies relating individual biomarkers to prognosis in NSTE-ACS, there is a growing uncertainty about the differential performance of these biomarkers (4 –17 ). Moreover, only a few studies have examined the incremental prognostic value of biomarkers when used in combination with risk scores such as GRACE, which already incorporates
Received March 15, 2013; accepted June 18, 2013. Previously published online at DOI: 10.1373/clinchem.2013.206185 Nonstandard abbreviations: NSTE-ACS, non-ST-elevation acute coronary syndrome; GRACE, Global Registry of Acute Coronary Events; NSTEMI, non-ST-elevation myocardial infarction; cTnT, cardiac troponin T; hs-cTnT, high-sensitivity cTnT; CRP, C-reactive protein; FGF23, fibroblast growth factor 23; GDF-15, growth differentiation factor 15; NT-proBNP, N-terminal pro–B-type natriuretic peptide; sST2, soluble suppression of tumorigenicity 2; IQR, interquartile range; AUC, area under the ROC curve; IDI, integrated discrimination improvement; NRI, net reclassification improvement; OR, odds ratio.
6
1497
information from biomarkers of myocardial damage and renal function (16 –21 ). Differences in study design preclude comparisons of biomarker performances based on these studies. Recently, cardiac troponin assays with increased analytic performance have been introduced into clinical practice (22 ). With the use of these assays, patients presenting with troponin elevations just above the 99th percentile of a reference population, which could not be detected reliably with previous less-sensitive assays, were shown to be at increased risk of death and ischemic events (23, 24 ). It is not known if biomarkers can enhance risk stratification on top of GRACE and a high-sensitivity cardiac troponin measurement. Materials and Methods STUDY POPULATION AND FOLLOW-UP
The study was approved by the institutional committees on human research at both institutions. The study included 1146 consecutive patients with NSTE-ACS who were selected retrospectively from a cohort of 2445 patients with nontraumatic chest pain suggestive of myocardial ischemia who were admitted to our cardiology departments in Hannover or Heidelberg. Serum samples were obtained on admission from all 2445 patients. Among these patients, 1146 had NSTE-ACS (NSTEMI or unstable angina pectoris), 710 STEMI, 279 other cardiac disease, and 310 noncardiac chest pain. All patients provided written informed consent within 24 h after admission. Refusal to provide consent was the only exclusion criterion. All treatment decisions were left to the discretion of the attending cardiologists. According to the Universal Definition of Myocardial Infarction, NSTEMI was diagnosed in patients showing a rise and/or fall of cardiac troponin T (cTnT) above the diagnostic threshold for MI during serial testing for at least 12 h. Patients were diagnosed with unstable angina if they had cTnT concentrations consistently below the diagnostic threshold for MI or persistent cTnT elevations (no rise and/or fall) during serial testing for at least 12 h. To make the diagnosis of unstable angina, we also required that these patients had signs of myocardial ischemia on electrocardiogram, or a history of coronary artery disease, or at least 1 stenosis of ⱖ50% in a major coronary artery on angiography. Patients were recruited between August 2007 and August 2011. Initially, cTnT was measured with the fourth generation Roche Diagnostics assay with 30 ng/L as diagnostic threshold for MI, a concentration that this assay can measure with ⬍10% total imprecision (25 ) (495 patients). During the course of the study, the fourth generation assay was replaced by the Roche Diagnostics high sensitivity cTnT (hs-cTnT) assay in both institutions, and 14 ng/L was used as the 1498 Clinical Chemistry 59:10 (2013)
diagnostic threshold for MI, the 99th percentile of a reference population which this assay can measure with ⬍10% total imprecision (26 ) (651 patients). Therefore, in some patients with a diagnosis of unstable angina based on serial measurements with the fourth generation assay, NSTEMI would have been diagnosed with the hs-cTnT assay (27 ). Between 2009 and 2011, 6 lots of the hs-cTnT assay displayed a downward shift at low concentrations of the measurement interval (28 ). Thus, in some patients with a diagnosis of unstable angina determined with the use of these lots, NSTEMI would have been diagnosed with a well-standardized hs-cTnT assay (28 ). Results from these measurements were used for diagnostic purposes only and were not used in the calculation of the GRACE score or to assess the prognostic value of hs-cTnT. Please note that the GRACE score considers all NSTE-ACS patients and does not distinguish between unstable angina and NSTEMI. Deaths and nonfatal MIs were recorded from admission to 6 months. Follow-up was accomplished in all patients by telephone contact or questionnaire at 6 months. When a patient, spouse, or primary care physician reported another hospital admission for a cardiovascular reason, hospital discharge letters (or final reports on deceased patients) were obtained and searched for a diagnosis of a fatal cardiac event or nonfatal MI (STEMI or NSTEMI). Cardiac death was defined as death during hospitalization with MI, death from progressive heart failure, death from documented cardiac arrhythmias, or sudden or unwitnessed death not related to accidents, suicide, terminal cancer, or other ominous diagnoses. We provide this information on the mode of death and type of MI to characterize our patient population. The GRACE score however, predicts all-cause mortality and all nonfatal MIs. All end points were adjudicated by 2 cardiologists. In rare cases of disagreement, a third cardiologist was consulted to reach consensus. BIOMARKER ASSAYS
Serum samples were obtained by venipuncture on admission and stored at ⫺70 °C. Samples were thawed once to measure the following biomarkers. cTnT was measured with the electrochemiluminescence hs-cTnT assay from Roche Diagnostics. Recently, the company reported a downward shift at low concentrations of the measurement interval in 6 lots of the assay, including lot 163704, which we had used for our measurements. This shift was not due to a change in reagent formulation but to the standardization against the master lot (28 ). Applying an adjusted standardization procedure (28 ), Roche Diagnostics recalculated the hs-cTnT values using the calibration data and original signal counts that we had obtained in our patients (see Fig. 1 in the Data Supplement that accompanies the online version of this report at http://www.clinchem.org/content/
Incremental Value of Biomarkers in NSTE-ACS
vol59/issue10). The corrected hs-cTnT values were used to calculate the GRACE score. Because the GRACE score considers troponin only as a binary variable (increased cardiac marker), we also explored the incremental prognostic value of hs-cTnT when added to GRACE as a continuous variable, again using the corrected hs-cTnT values. Copeptin was measured with an immunofluorescence assay (Thermo Fisher Scientific). C-reactive protein (CRP) was measured with an immunoturbidimetric assay (hs-CRP, Roche Diagnostics). Cystatin C was measured with an immunoturbidimetric assay (Roche Diagnostics). Fibroblast growth factor 23 (FGF23) was measured with an ELISA (Kainos Laboratories). Galectin-3 was measured with an ELISA (BG Medicine). Growth differentiation factor 15 (GDF-15) was measured with a precommercial electrochemiluminescence assay (Roche Diagnostics) used here for the first time in an ACS population. GDF-15 values obtained with this assay correlate closely with the values measured with a previously described immunoradiometric assay (29 ) (r ⫽ 0.980, slope 1.049, intercept ⫺136 ng/L, 45 samples with GDF-15 concentrations ranging from 567 to 13 334 ng/L). N-terminal pro–B-type natriuretic peptide (NT-proBNP) was measured with an electrochemiluminescence assay (Roche Diagnostics). Soluble suppression of tumorigenicity 2 (sST2) was measured with an ELISA (Medical & Biological Laboratories). See online Supplemental Table 1 for more information on the biomarkers and assay systems.
The GRACE score is derived from 8 variables that are available on admission (age, heart rate, systolic blood pressure, serum creatinine concentration, Killip class, cardiac arrest, presence of ST-segment deviation, and increased cardiac marker) (30 ). Values for these variables were entered into the GRACE risk calculator (www.outcomes-umassmed.org/grace/) to obtain estimates of the cumulative risks of all-cause mortality and all-cause mortality or nonfatal MI in the period from admission to 6 months. The combined end point of all-cause mortality or nonfatal MI was the prespecified primary end point of the study. All-cause mortality was analyzed as a secondary end point. An hs-cTnT concentration ⬎14 ng/L was used to define increased cardiac marker in the score. Serum creatinine concentrations were measured at the local study sites.
pmol/L); 22 patients had an FGF23 concentration below the LOD (3 ng/L); for these samples we imputed the midpoint between zero and the LOD (31 ). Eight patients had a cystatin C concentration below the instrument minimal reading of 0.4 mg/L; for these samples we imputed a concentration of 0.2 mg/L. Four patients had an hs-cTnT concentration below the limit of the blank (3 ng/L); for these samples we imputed a concentration of 1.5 ng/L. The probabilities derived from the GRACE score were transformed into their linear predictor form. Continuous variables were compared with the Mann–Whitney test and categorical variables with the 2 test. The Pearson correlation coefficient was calculated to assess the correlations between individual biomarkers and GRACE score probabilities. We examined the associations of individual biomarkers with outcome events using logistic regression models. Natural log-transformed biomarker distributions were standardized to a mean of zero and standard deviation (SD) of 1 to facilitate comparison of effects sizes between biomarkers. We determined the area under the ROC curve (AUC) to assess the discriminatory performance of the GRACE score and individual biomarkers. We assessed the incremental prognostic value of biomarkers when added to the GRACE score by the likelihood ratio test (32 ). We used 3 complementary measures of discrimination improvement to assess the magnitude of the increase in model performance when individual biomarkers were added to GRACE: change in AUC (⌬AUC), integrated discrimination improvement (IDI), and continuous and categorical net reclassification improvement (NRI) (33 ). To get a sense of clinical usefulness, we calculated the NRI(⬎0.02), which considers 2% as the minimum threshold for a meaningful change in predicted risk. Moreover, 2 categorical NRIs were applied with prespecified risk thresholds of 6% and 14%, chosen in accord with a previous study (17 ), or 5% and 12%, chosen in accord with the observed event rate in the present study. Categorical NRIs define upward and downward reclassification only if predicted risks move from one category to another. Since the number of biomarkers added to GRACE remained small (maximum of 2), the degree of overoptimism was likely to be small (34 ). Still, we reran the ⌬AUC and IDI analyses using bootstrap internal validation and confirmed our results. Analyses were performed using SPSS statistics version 19 and SAS 9.3.
STATISTICAL ANALYSIS
Results
CALCULATION OF THE GRACE SCORE
Categorical variables are reported as numbers and percentages, continuous variables as medians with interquartile range (IQR). To limit the influence of extreme observations, all biomarker values were natural logtransformed. A total of 233 patients had a copeptin concentration below the limit of detection (LOD) (4.8
PATIENT POPULATION, GRACE SCORE, AND BIOMARKER CONCENTRATIONS
Baseline characteristics, GRACE score variables, and biomarker concentrations are reported in Table 1. Six months after admission, 78 patients (6.8%) had reached Clinical Chemistry 59:10 (2013) 1499
Table 1. Patient population.a All (n ⴝ 1146)
Male sex
No event (n ⴝ 1068)
758 (71)
Pb
52 (67)
0.42
11 (3–54)
11 (3–56)
13 (4–49)
0.80
Age, years
69 (60–76)
69 (59–76)
74 (68–80)
⬍0.001
Heart rate, min⫺1
71 (62–83)
70 (61–82)
80 (67–99)
⬍0.001
Systolic blood pressure, mmHg
146 (130–159)
146 (130–159)
140 (120–159)
0.05
Creatinine, mg/dL
0.94 (0.79–1.16)
0.93 (0.79–1.13)
1.20 (0.90–1.65)
⬍0.001
Killip class I
1048 (91)
986 (92)
62 (79)
Killip class II
76 (7)
64 (6)
12 (15)
Killip class III
16 (1)
14 (1)
2 (3)
Killip class IV
6 (1)
4 (0)
2 (3)
Delay time, h
810 (71)
Event (n ⴝ 78)
GRACE variables on admission
Cardiac arrest
⬍0.001
0 (0)
0 (0)
0 (0)
ST-segment deviation
142 (12)
122 (11)
20 (26)
⬍0.001
Elevated cardiac marker
822 (72)
748 (70)
74 (95)
⬍0.001
133 (103–159)
130 (101–156)
GRACE score points
165 (145–188)
⬍0.001
Biomarker concentrations on admission Copeptin, pmol/L
11.3 (6.0–27.6)
10.8 (5.8–23.7)
40.1 (13.3–110.3)
⬍0.001
Cystatin C, mg/L
0.93 (0.80–1.17)
0.92 (0.79–1.12)
1.30 (0.98–1.82)
⬍0.001
FGF23, ng/L
40.6 (29.0–57.1)
40.3 (28.9–55.5)
56.3 (29.9–103.5)
⬍0.001
Galectin-3, g/L
21.8 (17.2–28.0)
21.4 (17.0–27.0)
29.0 (22.0–44.3)
⬍0.001
GDF-15, ng/L
1770 (1262–2981)
1720 (1241–2727)
4180 (2482–7990)
⬍0.001
hs-CRP, mg/L
2.93 (1.26–8.40)
2.73 (1.18–7.49)
10.97 (3.33–34.54)
⬍0.001
hs-cTnT, ng/L
30.6 (12.7–161.3)
27.2 (12.2–144.7)
92.7 (38.1–401.4)
⬍0.001
NT-proBNP, ng/L
392 (131–1363)
363 (125–1176)
1958 (613–10476)
⬍0.001
sST2, g/L
0.48 (0.38–0.61)
0.47 (0.38–0.60)
0.60 (0.43–1.37)
⬍0.001
Coronary angiography
989 (86)
920 (86)
69 (88)
Percutaneous coronary intervention
657 (57)
612 (57)
45 (58)
0.94
34 (3)
31 (3)
3 (4)
0.64
Unstable angina
554 (48)
538 (50)
16 (21)
⬍0.001
NSTEMI
592 (52)
530 (50)
62 (79)
Patient management and diagnosis at discharge
Coronary artery bypass graft
a b
0.57
Data are shown as n (%) or median (IQR). Delay time refers to time from symptom onset to admission. Events vs no events.
the combined end point of death (41 cardiac, 12 noncardiac deaths) or nonfatal MI (3 STEMI, 22 NSTEMI). Online Supplemental Fig. 2 shows the timing of events. Patients who reached the combined end point presented with higher GRACE score points and higher biomarker concentrations compared with patients who did not (Table 1). The GRACE score was moderately correlated with NT-proBNP (r ⫽ 0.59) and GDF-15 (0.58); more modestly with cystatin C (0.48), hs-cTnT (0.45), copeptin (0.43), galectin-3 (0.40), hs-CRP (0.33), and sST2 (0.30); and weakly with FGF23 (0.11) (all P ⬍ 0.001). 1500 Clinical Chemistry 59:10 (2013)
DIFFERENTIAL PROGNOSTIC PERFORMANCE OF INDIVIDUAL BIOMARKERS
Increasing concentrations of all biomarkers were associated with the risk of death or nonfatal MI at 6 months (Fig. 1). Based on the odds ratio (OR) per 1 SD increase in the natural log-transformed biomarker concentration, NT-proBNP (OR 2.4) and GDF-15 (OR 2.4) showed the strongest associations to outcome (Fig. 1). The GRACE score and all biomarkers provided significant discriminatory information, as evidenced by their AUC with corresponding 95% CIs that were nonover-
a
0 OR (per 1 SD in ln biomarker) 1 2 3
Fig. 1. Associations of individual biomarkers with outcome.
ORs for 6-month all-cause mortality or nonfatal MI per 1 SD increase in the natural log-transformed biomarker (ln biomarker) levels.
lapping with the nondiscriminatory value of 0.5 (Table 2); the GRACE score alone had an AUC of 0.749. GDF-15 (AUC 0.771) and NT-proBNP (AUC 0.745) emerged as the biomarkers with the greatest discriminatory strength (Table 2).
INCREMENTAL PROGNOSTIC INFORMATION PROVIDED BY
BIOMARKERS WHEN ADDED TO GRACE
All biomarkers offered statistically significant improvements in model performance when added to GRACE (likelihood ratio test in Table 3). The magnitude of model improvement varied considerably, how-
Table 2. Discrimination of outcome events by individual biomarkers or the GRACE score.a
AUC (95% CI) P
GDF-15 0.771 (0.711–0.830) ⬍0.001
GRACE 0.749 (0.696–0.801) ⬍0.001
NT-proBNP
0.745 (0.682–0.808)
⬍0.001
Cystatin C
0.742 (0.681–0.802)
⬍0.001
Copeptin
0.718 (0.655–0.782)
⬍0.001
Galectin-3
0.714 (0.654–0.774)
⬍0.001
hs-CRP
0.704 (0.641–0.767)
⬍0.001
hs-cTnT
0.687 (0.634–0.741)
⬍0.001
sST2
0.657 (0.583–0.732)
⬍0.001
FGF23
0.623 (0.544–0.702)
0.002
Discrimination of 6-month all-cause mortality or nonfatal MI by individual biomarkers or the GRACE score as assessed by the AUC.
0.015
⫹ FGF23
0.777 (0.724 to 0.829)
0.752 (0.693 to 0.810)
0.763 (0.710 to 0.816)
0.770 (0.716 to 0.823)
0.771 (0.714 to 0.827)
-
0.003 (⫺0.027 to 0.033)
0.014 (⫺0.005 to 0.034)
0.021 (⫺0.002 to 0.044)
0.022 (⫺0.005 to 0.049)
0.028 (0.001 to 0.056)
0.030 (0.008 to 0.052)
0.033 (0.004 to 0.062)
0.024 (⫺0.010 to 0.058)
0.039 (0.009 to 0.070)
9
8
7
6
4
3
2
5
1
-
0.013 (0.003 to 0.024)
0.009 (0.000 to 0.018)
0.023 (0.004 to 0.043)
0.018 (0.008 to 0.029)
0.021 (0.008 to 0.034)
0.020 (0.005 to 0.036)
0.016 (0.003 to 0.028)
0.027 (0.012 to 0.041)
0.049 (0.024 to 0.074)
8
9
3
6
4
5
7
2
1
-
0.273 (0.047 to 0.498)
0.400 (0.171 to 0.628)
0.237 (0.007 to 0.466)
0.361 (0.135 to 0.586)
0.283 (0.054 to 0.513)
0.377 (0.148 to 0.606)
0.385 (0.157 to 0.612)
0.438 (0.212 to 0.664)
0.554 (0.329 to 0.779)
1
8
3
9
6
7
5
4
2
25
20
19
18
15
13
13
9
3
Rank sum
Biomarkers were added individually to GRACE. Improvements in model performance concerning the end point of all-cause mortality or nonfatal MI were assessed by the change (⌬) in AUC, the IDI, and the NRI(⬎0). For each metric, biomarkers are ranked according to their performance relative to the other markers. Rank sum was calculated as rank(⌬AUC) ⫹ rank(IDI) ⫹ rank[NRI(⬎0)]. Likelihood ratio test.
0.006
⬍0.001
⫹ sST2
⫹ hs-cTnT
⬍0.001
⬍0.001
⫹ Galectin-3
⫹ Copeptin
0.779 (0.726 to 0.831)
0.781 (0.729 to 0.833)
⬍0.001
⬍0.001
⫹ hs-CRP
⫹ Cystatin C
0.772 (0.716 to 0.829)
⬍0.001
⫹ NT-proBNP
0.749 (0.696 to 0.801)
0.788 (0.732 to 0.844)
⬍0.001
Rank NRI(>0)
OR (95% CI)
b
1.7 (1.4–1.9)
1.6 (1.2–2.0)
a
sST2
FGF23
⫹ GDF-15
1.8 (1.4–2.2)
GRACE
1.9 (1.6–2.3)
hs -c TnT
NRI(>0) (95% CI)
Cystatin C
Rank IDI
2.1 (1.7–2.6)
1.9 (1.6–2.4)
IDI (95% CI)
Galectin-3
hs CRP hs-CRP
Rank ⌬AUC
2.1 (1.7–2.6)
⌬AUC (95% CI)
2.4 (2.0–3.0)
Copeptin
AUC (95% CI)
2.4 (1.9–3.1)
Pb
GDF-15
Biomarker
NT-proBNP
Table 3. Improvements in model performance upon addition of individual biomarkers to the GRACE score.a
Incremental Value of Biomarkers in NSTE-ACS
Clinical Chemistry 59:10 (2013) 1501
Table 4. Improvements in model performance upon addition of GDF-15 or NT-proBNP to the GRACE score plus hs-cTnT.a
Biomarker
Pa
GRACE ⫹ hs-cTnT ⫹ GDF-15 ⫹ NT-proBNP
⌬AUC (95% CI)
AUC (95% CI)
0.763 (0.710 to 0.816)
-
IDI (95% CI)
NRI(>0) (95% CI)
-
-
⬍0.001
0.791 (0.735 to 0.846)
0.028 (0.002 to 0.054)
0.049 (0.024 to 0.074)
0.531 (0.306 to 0.755)
0.001
0.773 (0.717 to 0.829)
0.010 (⫺0.014 to 0.034)
0.020 (0.009 to 0.030)
0.484 (0.263 to 0.705)
a
GDF-15 or NT-proBNP was added to a model including GRACE plus hs-cTnT as a continuous variable. Improvements in model performance concerning the end point of all-cause mortality or nonfatal MI were assessed by the change (⌬) in AUC, the IDI, and the NRI(⬎0). b Likelihood ratio test.
ever: the ⌬AUC ranged from 0.003 to 0.039, the IDI from 0.009 to 0.049, and the category-free NRI(⬎0) from 0.237 to 0.554 (Table 3). To illustrate biomarker performance across these statistical metrics, we ranked biomarkers according to their performance relative to the other markers; for each metric, the best marker was attributed rank 1, the worst marker rank 9 (Table 3). The ranks in the ⌬AUC, IDI, and NRI(⬎0) categories were added up to calculate a rank sum. On a scale of 3 to 27 (with higher numbers indicating worse performance), GDF-15 achieved a rank sum of 3 because it ranked in first place in all 3 categories. The second best marker was NT-proBNP, with a rank sum of 9 (Table 3). In a sensitivity analysis, we examined the incremental prognostic value of our 9 biomarkers only in patients who were included after the hs-cTnT assay had been introduced in our institutions (n ⫽ 651). GDF-15 and NT-proBNP emerged again as the 2 most promising biomarkers adding prognostic information to GRACE [GDF-15, likelihood ratio test P ⫽ 0.003; ⌬AUC 0.028; IDI 0.030; NRI(⬎0) 0.484; NT-proBNP, P ⫽ 0.012; ⌬AUC 0.024; IDI 0.016; NRI(⬎0) 0.467]. When we assessed the incremental prognostic information provided by our biomarkers upon addition to a model containing the individual GRACE variables (not combined into a score), the magnitude of the incremental prognostic information remained similar to that shown in Table 3; GDF-15 emerged again as the most promising biomarker. RECLASSIFICATION ACROSS PREDEFINED RISK THRESHOLDS
The category-free NRI(⬎0) reflects any reclassification in the correct direction regardless of its magnitude. Potential for clinical benefit achieved when GDF-15 or NT-proBNP were added to GRACE was therefore assessed by calculating the NRI(⬎0.02), which considers 2% as minimum threshold for a change in predicted risk. Both markers enabled substantial net reclassification across this threshold: GDF-15, NRI(⬎0.02) 0.373 (95% CI 0.201– 0.546); NT-proBNP, 0.260 (0.071– 0.449). 1502 Clinical Chemistry 59:10 (2013)
Along the same line, we calculated a category-based NRI to assess the ability of GDF-15 and NT-proBNP to reclassify patients across predefined risk thresholds classifying patients as low (⬍6%), intermediate (6%–14%), or high risk (⬎14%). Both markers enabled net reclassification across these thresholds: GDF-15, NRI(6/14) 0.187 (95% CI 0.051– 0.323); NT-proBNP, 0.154 (0.042– 0.267) (see online Supplemental Table 2). Because net reclassification depends on the chosen risk thresholds (33 ), in sensitivity analysis we defined 5% and 12% as alternative thresholds that were more evenly distributed around the event rate of 6.8%. GDF-15, but not NT-proBNP, enabled net reclassification across these thresholds: GDF-15, NRI(5/12) 0.183 (95% CI 0.052– 0.315); NT-proBNP, 0.108 (-0.021– 0.237). INCREMENTAL VALUE BEYOND GRACE AND hs-cTnT
Guidelines recommend that cardiac troponin be measured with a high-sensitivity assay on admission in all NSTE-ACS patients (2 ). Since GRACE considers the results from this troponin test only as a binary variable, we explored whether GDF-15 or NT-proBNP add prognostic information to a model including GRACE plus hs-cTnT as a continuous variable (because this information is clinically available before any other biomarker is measured). Both markers added statistically significant prognostic information to GRACE plus hscTnT (Table 4). The magnitude of the improvement in model performance appeared to be greater for GDF-15 than NT-proBNP (Table 4). INCREMENTAL VALUE OF ADDING A SECOND BIOMARKER TO GRACE PLUS GDF-15 OR TO GRACE PLUS NT-proBNP
Based on the ⌬AUC, IDI, and NRI(⬎0), GDF-15 and NT-proBNP emerged as the 2 most promising biomarkers adding discriminatory information to GRACE (Table 3). We therefore explored if any of the other 8 biomarkers provided prognostic information on top of GRACE plus GDF-15 or GRACE plus NT-proBNP. In-
Incremental Value of Biomarkers in NSTE-ACS
creases in model performance were relatively small (see online Supplemental Table 3). SECONDARY ANALYSES
In a posthoc secondary analysis, we assessed the incremental prognostic information provided by biomarkers on top of GRACE concerning the end point of allcause mortality. All biomarkers, except for FGF23, offered statistically significant improvements in model performance (see online Supplemental Table 4). NTproBNP and GDF-15 emerged as the most promising markers. Discussion This is the first study to compare the incremental prognostic value of biomarkers beyond GRACE and cTnT measured with a high-sensitivity assay in NSTE-ACS. We found that the ability of biomarkers to add prognostic information to GRACE and hs-cTnT varies considerably. All biomarkers investigated in the present study, except for FGF23, have been linked to adverse outcomes in NSTE-ACS (5–21, 23 ). FGF23 is emerging as a prognostic biomarker in chronic kidney disease (35 ) and stable coronary artery disease (36 ), which prompted us to explore it for the first time in NSTEACS. Consistent with these studies, we found all biomarkers to be associated with death or MI and to discriminate patients with or without events (Fig. 1 and Table 2). The GRACE score alone had an AUC of 0.749 in our patients, thus confirming the score as a valuable tool for risk assessment (30 ). Three complementary statistical metrics were employed to examine the incremental value of biomarkers beyond GRACE. Reviewing biomarker performances across these metrics, GDF-15 and to a somewhat lesser extent NT-proBNP emerged as the most promising biomarkers (Table 3). Please note that the rank orders in Table 3 are meant to illustrate biomarker performances relative to the other markers across the 3 statistical metrics. Although these ranks help to identify promising biomarkers and less promising markers as a basis for future studies, it is not our intention to ascribe statistical properties to these ranks (i.e., it is not possible to conclude on the basis of its rank that one marker is significantly better than another marker). Emphasizing the potential of GDF-15 and NTproBNP to add information to what is clinically available, we found that GDF-15 and to a somewhat lesser extent NT-proBNP also added discriminatory information to GRACE when hs-cTnT was considered as an additional continuous variable (Table 4). Addition of a second biomarker on top of GRACE plus GDF-15 or
GRACE plus NT-proBNP added little discriminatory information, perhaps because the score already includes information from biomarkers of renal function and myocardial damage. Notably, some biomarkers performed differently across the 3 statistical metrics. Biomarker performance across these metrics depends on the strength of the baseline model (GRACE score) and the effect size of the candidate marker and its correlation with GRACE. The ⌬AUC heavily depends on the strength of the baseline model, which is true to a lesser degree for the IDI. On the other hand, the NRI(⬎0) depends mainly on the effect size of the candidate marker and its correlation with GRACE (33 ). Moreover, the NRI(⬎0) tends to favor markers that offer smaller improvements for many people, whereas the IDI picks up large improvements for few people (33 ). The increases in the AUC achieved when individual biomarkers were added to the GRACE score ranged from 0.003– 0.039, which is relatively modest. However, even variables with strong effect sizes lead to only small numerical increases in the AUC when added to a strong baseline model such as GRACE (33 ). Indeed, employing the NRI(⬎0) as a metric not as easily influenced by the strength of the baseline model, we conclude that GDF-15 offers an increment consistent with a medium-to-strong effect size and NT-proBNP an increment consistent with a medium effect size (33 ). The strength of GDF-15’s impact was confirmed by its impressive IDI of 0.049 (with an event rate of 0.068, GDF-15 separated events from nonevents by a further 0.049). We are not aware of a previous study comparing the incremental prognostic value of multiple biomarkers beyond a validated risk score such as GRACE and a high-sensitivity cardiac troponin. Some previous studies have examined the incremental prognostic value of individual candidate biomarkers beyond GRACE, but results were heterogeneous (16 –21 ). Differences in study design (some studies including STEMI), timing of blood sampling (up to 72 h after admission), end points (some studies including heart failure and revascularization end points), and length of follow-up (between 30 days and 1 year) preclude comparisons of biomarker performances based on these studies (16 – 21 ). We focused on NSTE-ACS because risk stratification guides management decisions in this setting (1, 2 ). We created optimal conditions for the GRACE score to predict outcome by utilizing the follow-up interval (admission to 6 months) and the end point (all-cause mortality or nonfatal MI) for which the score was developed. Moreover, we incorporated results from hscTnT testing in the score to reflect the use of highsensitivity troponin assays in current clinical practice. Thus, our study is the first to provide a comparative Clinical Chemistry 59:10 (2013) 1503
analysis of the incremental value of biomarkers in the current clinical environment. Unlike many previous studies, our study was not focused on statistical significance but assessed the magnitude of model improvement achieved by individual biomarkers. In fact, all our biomarkers added statistically significant prognostic information, but the incremental prognostic value of individual markers differed widely. Thus, we identified strong biomarker candidates for future studies on biomarker strategies in NSTE-ACS. The change in the cTnT assay used for diagnostic purposes from the fourth generation to the highsensitivity assay during the study is a limitation because it may have resulted in patients with small NSTEMIs not being included in our cohort and in small NSTEMIs not being diagnosed during follow-up. Importantly, we found our results to be robust when we analyzed only the patients that were included after the hs-cTnT assay had been introduced. Use of hs-cTnT lots with a downward shift at low concentrations for diagnostic purposes on admission and during follow-up will have resulted in very few (if any) missed diagnoses of NSTEMI (28 ). In conclusion, our study shows that biomarkers can add prognostic information to GRACE even in the current era of high-sensitivity cardiac troponin assays. Although bootstrap internal validation confirmed our results, the differential performance of the biomarkers investigated here needs to be reexamined in other, preferably larger study populations with more events. More studies are needed to assess whether biomarkerenhanced risk stratification can guide management decisions in NSTE-ACS, as it has previously been suggested for the GRACE score alone (37 ). As demonstrated here, GDF-15 and NT-proBNP have the potential to reclassify patients in the appropriate direc-
tions across risk thresholds, which may trigger changes in treatment decisions in the future. More widespread use of risk scores and prognostic biomarkers in clinical practice will help to match the intensity of therapy with an individual patient’s risk and will enable judicious allocation of healthcare resources.
Author Contributions: All authors confirmed they have contributed to the intellectual content of this paper and have met the following 3 requirements: (a) significant contributions to the conception and design, acquisition of data, or analysis and interpretation of data; (b) drafting or revising the article for intellectual content; and (c) final approval of the published article. Authors’ Disclosures or Potential Conflicts of Interest: Upon manuscript submission, all authors completed the author disclosure form. Disclosures and/or potential conflicts of interest: Employment or Leadership: M. Bobadilla, Hoffmann La Roche. Consultant or Advisory Role: K.C. Wollert, Roche Diagnostics. Stock Ownership: None declared. Honoraria: H.A. Katus, Roche Diagnostics; E. Giannitsis, Roche Diagnostics; K.C. Wollert, Roche Diagnostics. Research Funding: Hannover Medical School; K.C. Wollert, German Ministry of Education and Research (BMBF, BioChancePlus), F. Hoffmann-La Roche, and Roche Diagnostics. Stock Ownership: None declared. Expert Testimony: None declared. Patents: T. Kempf, European patent 2047275B1 (GDF-15) and US patent application 12/363932 (pending; GDF-15); H.A. Katus, patent number not available (cTnT); K.C. Wollert, European patent 2047275B1 (GDF-15) and US patent application 12/363932 (pending; GDF-15). Role of Sponsor: The funding organizations played no role in the design of study, choice of enrolled patients, review and interpretation of data, or preparation or approval of manuscript. Acknowledgments: We thank Dr. Jochen Jarausch for discussion and comments on the manuscript.
References 1. 2012 Writing Committee Members, Jneid H, Anderson JL, Wright RS, Adams CD, Bridges CR, et al. 2012 ACCF/AHA focused update of the guideline for the management of patients with unstable angina/Non-ST-elevation myocardial infarction (updating the 2007 guideline and replacing the 2011 focused update): a report of the American College of Cardiology Foundation/ American Heart Association Task Force on practice guidelines. Circulation 2012;126:875–910. 2. Hamm CW, Bassand JP, Agewall S, Bax J, Boersma E, Bueno H, et al. ESC Guidelines for the management of acute coronary syndromes in patients presenting without persistent ST-segment elevation: the Task Force for the management of acute coronary syndromes (ACS) in patients presenting without persistent ST-segment elevation of the European Society of Cardiology (ESC). Eur Heart J 2011;32:2999 –3054. 3. Scirica BM. Acute coronary syndrome: emerging tools for diagnosis and risk assessment. J Am Coll
1504 Clinical Chemistry 59:10 (2013)
Cardiol 2010;55:1403–15. 4. Morrow DA. Cardiovascular risk prediction in patients with stable and unstable coronary heart disease. Circulation 2010;121:2681–91. 5. Lindahl B, Venge P, Wallentin L. Relation between troponin T and the risk of subsequent cardiac events in unstable coronary artery disease. The FRISC study group. Circulation 1996; 93:1651–7. 6. Morrow DA, Rifai N, Antman EM, Weiner DL, McCabe CH, Cannon CP, Braunwald E. C-reactive protein is a potent predictor of mortality independently of and in combination with troponin T in acute coronary syndromes: a TIMI 11A substudy. Thrombolysis in Myocardial Infarction. J Am Coll Cardiol 1998;31:1460 –5. 7. James SK, Armstrong P, Barnathan E, Califf R, Lindahl B, Siegbahn A, et al. Troponin and C-reactive protein have different relations to subsequent mortality and myocardial infarction after acute coronary syndrome: a GUSTO-IV substudy.
J Am Coll Cardiol 2003;41:916 –24. 8. James SK, Lindahl B, Siegbahn A, Stridsberg M, Venge P, Armstrong P, et al. N-terminal pro-brain natriuretic peptide and other risk markers for the separate prediction of mortality and subsequent myocardial infarction in patients with unstable coronary artery disease: a Global Utilization of Strategies To Open occluded arteries (GUSTO)-IV substudy. Circulation 2003;108:275– 81. 9. Jernberg T, Lindahl B, James S, Larsson A, Hansson LO, Wallentin L. Cystatin C: a novel predictor of outcome in suspected or confirmed non-STelevation acute coronary syndrome. Circulation 2004;110:2342– 8. 10. Ristiniemi N, Lund J, Tertti R, Christensson A, Ilva T, Porela P, et al. Cystatin C as a predictor of all-cause mortality and myocardial infarction in patients with non-ST-elevation acute coronary syndrome. Clin Biochem 2012;45:535– 40. 11. Eggers KM, Armstrong PW, Califf RM, Simoons ML, Venge P, Wallentin L, James SK. ST2 and
Incremental Value of Biomarkers in NSTE-ACS
12.
13.
14.
15.
16.
17.
18.
19.
mortality in non-ST-segment elevation acute coronary syndrome. Am Heart J 2010;159:788 –94. Grandin EW, Jarolim P, Murphy SA, Ritterova L, Cannon CP, Braunwald E, Morrow DA. Galectin-3 and the development of heart failure after acute coronary syndrome: pilot experience from PROVE IT-TIMI 22. Clin Chem 2012;58:267–73. Scirica BM, Sabatine MS, Jarolim P, Murphy SA, de Lemos JL, Braunwald E, Morrow DA. Assessment of multiple cardiac biomarkers in non-STsegment elevation acute coronary syndromes: observations from the MERLIN-TIMI 36 trial. Eur Heart J 2011;32:697–705. Dhillon OS, Narayan HK, Quinn PA, Squire IB, Davies JE, Ng LL. Interleukin 33 and ST2 in nonST-elevation myocardial infarction: comparison with Global Registry of Acute Coronary Events Risk Scoring and NT-proBNP. Am Heart J 2011; 161:1163–70. Kohli P, Bonaca MP, Kakkar R, Kudinova AY, Scirica BM, Sabatine MS, et al. Role of ST2 in non-ST-elevation acute coronary syndrome in the MERLIN-TIMI 36 trial. Clin Chem 2012;58:257– 66. Narayan H, Dhillon OS, Quinn PA, Struck J, Squire IB, Davies JE, Ng LL. C-terminal provasopressin (copeptin) as a prognostic marker after acute non-ST elevation myocardial infarction: Leicester Acute Myocardial Infarction Peptide II (LAMP II) study. Clin Sci 2011;121:79 – 89. Widera C, Pencina MJ, Meisner A, Kempf T, Bethmann K, Marquardt I, et al. Adjustment of the GRACE score by growth differentiation factor 15 enables a more accurate appreciation of risk in non-ST-elevation acute coronary syndrome. Eur Heart J 2012;33:1095–104. Schiele F, Meneveau N, Seronde MF, Chopard R, Descotes-Genon V, Dutheil J, et al. C-reactive protein improves risk prediction in patients with acute coronary syndromes. Eur Heart J 2010;31: 290 –7. Meune C, Drexler B, Haaf P, Reichlin T, Reiter M, Meissner J, et al. The GRACE score’s performance in predicting in-hospital and 1-year outcome in the era of high-sensitivity cardiac troponin assays
20.
21.
22.
23.
24.
25.
26.
27.
28.
and B-type natriuretic peptide. Heart 2011;97: 1479 – 83. Ang DS, Wei L, Kao MP, Lang CC, Struthers AD. A comparison between B-type natriuretic peptide, global registry of acute coronary events (GRACE) score and their combination in ACS risk stratification. Heart 2009;95:1836 – 42. Beygui F, Silvain J, Pena A, Bellemain-Appaix A, Collet JP, Drexler H, et al. Usefulness of biomarker strategy to improve GRACE score’s prediction performance in patients with non-STsegment elevation acute coronary syndrome and low event rates. Am J Cardiol 2010;106:650 – 8. Apple FS, Collinson PO, IFCC Task Force on Clinical Applications of Cardiac Biomarkers. Analytical characteristics of high-sensitivity cardiac troponin assays. Clin Chem 2012;58:54 – 61. Lindahl B, Venge P, James S. The new highsensitivity cardiac troponin T assay improves risk assessment in acute coronary syndromes. Am Heart J 2010;160:224 –9. Bonaca M, Scirica B, Sabatine M, Dalby A, Spinar J, Murphy SA, et al. Prospective evaluation of the prognostic implications of improved assay performance with a sensitive assay for cardiac troponin I. J Am Coll Cardiol 2010;55:2118 –24. Hermsen D, Apple F, Garcia-Beltran L, Jaffe A, Karon B, Lewandrowski E, et al. Results from a multicenter evaluation of the 4th generation Elecsys Troponin T assay. Clin Lab 2007;53:1–9. Saenger AK, Beyrau R, Braun S, Cooray R, Dolci A, Freidank H, et al. Multicenter analytical evaluation of a high-sensitivity troponin T assay. Clin Chim Acta 2011;412:748 –54. Giannitsis E, Kurz K, Hallermayer K, Jarausch J, Jaffe AS, Katus HA. Analytical validation of a high-sensitivity cardiac troponin T assay. Clin Chem 2010;56:254 – 61. Apple FS, Jaffe AS. Clinical implications of a recent adjustment to the high-sensitivity cardiac troponin T assay: user beware [Letter]. Clin Chem 2012;58:1599 – 600. Hallermayer K, Jarausch J, Menassanch-Volker S, Zaugg C, Ziegler A [Reply]. Clin Chem 2013;59:572– 4. Kavsak PA, Hill SA, McQueen MJ, Devereaux PJ. [Reply]. Clin Chem
2013;59:574 – 6. 29. Kempf T, Horn-Wichmann R, Brabant G, Peter T, Allhoff T, Klein G, et al. Circulating concentrations of growth-differentiation factor 15 in apparently healthy elderly individuals and patients with chronic heart failure as assessed by a new immunoradiometric sandwich assay. Clin Chem 2007; 53:284 –91. 30. Fox KA, Dabbous OH, Goldberg RJ, Pieper KS, Eagle KA, Van de Werf F, et al. Prediction of risk of death and myocardial infarction in the six months after presentation with acute coronary syndrome: prospective multinational observational study (GRACE). BMJ 2006;333: 1091. 31. Hornung RW, Reed LD. Estimation of average concentration in the presence of nondetectable values. Appl Occup Environ Hyg 1990;5:46 –51. 32. Vickers AJ, Cronin AM, Begg CB. One statistical test is sufficient for assessing new predictive markers. BMC Med Res Methodol 2011;11:13. 33. Pencina MJ, D’Agostino RB, Pencina KM, Janssens AC, Greenland P. Interpreting incremental value of markers added to risk prediction models. Am J Epidemiol 2012;176:473– 81. 34. Steyerberg EW, Harrell FE Jr, Borsboom GJ, Eijkemans MJ, Vergouwe Y, Habbema JD. Internal validation of predictive models: efficiency of some procedures for logistic regression analysis. J Clin Epidemiol 2001;54:774 – 81. 35. Isakova T, Xie H, Yang W, Xie D, Anderson AH, Scialla J, et al. Fibroblast growth factor 23 and risks of mortality and end-stage renal disease in patients with chronic kidney disease. JAMA 2011; 305:2432–9. 36. Udell JA, O’Donnell T, Morrow D, Jarolim P, Omland T, Sloan S, et al. Association of fibroblast growth factor (FGF)-23 levels with risk of cardiovascular events in patients with stable coronary artery disease [Abstract]. J Am Coll Cardiol 2012; 59:E1480. 37. Mehta SR, Granger CB, Boden WE, Steg PG, Bassand JP, Faxon DP, et al. Early versus delayed invasive intervention in acute coronary syndromes. N Engl J Med 2009;360:2165–75.
Clinical Chemistry 59:10 (2013) 1505
