Continuous Probabilistic Prediction of Angiographically Significant Coronary Artery Disease Using Electron Beam Tomography Matthew J. Budoff, MD; George A. Diamond, MD; Paolo Raggi, MD; Yadon Arad, MD; Alan D. Guerci, MD; Tracy Q. Callister, MD; Daniel Berman, MD Background—We sought to incorporate electron beam tomography– derived calcium scores in a model for prediction of angiographically significant coronary artery disease (CAD). Such a model could greatly facilitate clinical triage in symptomatic patients with no known CAD. Methods and Results—We examined 1851 patients with suspected CAD who underwent coronary angiography for clinical indications. An electron beam tomographic scan was performed in all patients. Total per-patient calcium scores and separate scores for the major coronary arteries were added to logistic regression models to calculate a posterior probability of the severity and extent of angiographic disease. These models were designed to be continuous, adjusted for age and sex, corrected for verification bias, and independently validated in terms of their incremental diagnostic accuracy. The overall sensitivity was 95%, and specificity was 66% for coronary calcium to predict obstructive disease on angiography. With calcium scores ⬎20, ⬎80, and ⬎100, the sensitivity to predict stenosis decreased to 90%, 79%, and 76%, whereas the specificity increased to 58%, 72%, and 75%, respectively. The logistic regression model exhibited excellent discrimination (receiver operating characteristic curve area, 0.842⫾0.023) and calibration (2 goodness of fit, 8.95; P⫽0.442). Conclusions—Electron beam tomographic calcium scanning provides incremental and independent power in predicting the severity and extent of angiographically significant CAD in symptomatic patients, in conjunction with pretest probability of disease. This algorithm is most useful when applied to an intermediate-risk population. (Circulation. 2002;105:17911796.) Key Words coronary disease 䡲 tomography 䡲 angiography 䡲 calcium
C
oronary artery calcification (CAC) scores derived from electron beam tomography (EBT) are predictive of subsequent cardiac events in both symptomatic1,2 and asymptomatic populations.3–5 As opposed to other noninvasive modalities to diagnose coronary artery disease (CAD) focusing on physiological consequences of obstruction, EBT coronary calcium represents an anatomic measure of plaque burden.6 Studies comparing pathological and EBT findings have demonstrated that the degree of luminal narrowing is weakly correlated with the amount of calcification on a site-by-site basis,7 whereas total calcium score is more closely associated with the presence and severity of maximum angiographic stenosis.8 –10 These studies have a substantial degree of verification bias as a consequence of the preferential referral of positive test responders to angiography and of negative test responders away from angiography.8 We sought to overcome these difficulties and thereby construct and validate a clinically relevant model to predict
the likelihood of angiographically significant CAD based on the magnitude and distribution of calcification within each region of the coronary circulation. The rationale for this study was to create a new model based on a much larger data set than previous models, with certain design criteria. The a priori criteria for the design of this model were that it be (1) continuous, (2) adjusted for age and sex, (3) corrected for the effects of verification bias, (4) independently validated, and (5) capable of being integrated with additional diagnostic information.
Methods Participating Centers The EBT and angiographic studies were conducted at Harbor-UCLA Medical Center, Torrance, Calif; EBT Research Foundation, Nashville, Tenn; and Saint Francis Hospital, Roslyn, New York, and approved by the respective Institutional Review Boards. Investigators recorded patient information using a case-report form that
Received November 9, 2001; revision received February 1, 2002; accepted February 4, 2002. From the Division of Cardiology, Harbor-UCLA Research and Education Institute, Torrance, Calif (M.J.B.); the Division of Cardiology, Cedars-Sinai Medical Center, Los Angeles, Calif (G.A.D., D.B.); the Division of Cardiology, Tulane University School of Medicine, New Orleans, La (P.R.); Saint Francis Heart Center, Roslyn, NY (Y.A., A.D.G.); and EBT Research Foundation, Nashville, Tenn (T.Q.C.). Correspondence to Matthew Budoff, MD, Harbor-UCLA REI, 1124 W Carson St, RB-2, Torrance, CA 90502-2064. E-mail
[email protected] © 2002 American Heart Association, Inc. Circulation is available at http://www.circulationaha.org
DOI: 10.1161/01.CIR.0000014483.43921.8C
1791
1792
Circulation
April 16, 2002
included biographical data including age, sex, dates, and results of EBT and angiographic studies. The data were forwarded to the coordinating and data elaboration center at Cedars-Sinai Medical Center in Los Angeles, Calif.
Patient Population The patient population consisted of 1851 patients (1169 men, 682 women; mean age, 58⫾11 years; range, 21 to 86) undergoing coronary angiography for clinical indications. Nonconsecutive patients were enrolled from January 1996 to September 1999. After signing informed consent, EBT scanning was performed to assess presence and estimate extent of CAC. EBT was performed within 2 weeks of angiography in 92% of patients. Exclusion criteria included EBT scans performed ⬎3 months from the angiogram and patients who had previous coronary interventional procedures or known CAD. The vast majority (⬎90%) of patients underwent angiography before EBT (81%) or were already scheduled to undergo angiography before EBT (10%). No patient was included in the previous multicenter publication.8 To adjust for verification bias, we evaluated 4103 asymptomatic persons referred for CAC assessment to measure cardiovascular risk. Persons were asymptomatic, with no history of cardiovascular disease.
Angiographic Protocol All patients underwent coronary angiography and were referred on the basis of the primary physician’s concern for presence of myocardial ischemia. Positive noninvasive stress testing, abnormal echocardiograms, and clinical history were all used for referring patients for cardiac catheterization. The angiograms were analyzed by an experienced reader at each institution, blinded to the results of the EBT coronary scan. Each epicardial coronary vessel was assessed, and the visual estimation of the percent luminal stenosis of each lesion was reported as previously described.8 Angiographic abnormalities were considered significant if ⱖ50% luminal diameter stenosis was found in any vessel.
Electron Beam Tomography The EBT studies were performed with an Imatron C-150 XL ultrafast CT scanner in the high-resolution volume mode, with a 100-ms exposure time. ECG triggering was used, corresponding to 80% of the R-R interval for standardized calcium scoring.11 Thirty to 40 consecutive, noncontrast images were obtained at 3-mm intervals to include the entire coronary tree. Total radiation exposure with the use of this technique was 0.6 Rad per patient, significantly less than multislice spiral CT (estimated between 2.8 to 10 Rad12,13) or conventional angiography(4 to 8 Rad14). The scans were scored by means of the algorithm developed by Agatston et al11 by a cardiologist blinded to the clinical, ECG, and angiographic information.
Multivariate Logistic Prediction Model The baseline data set of 1851 patients was divided into two samples by a random number generator. The training sample of 932 patients was used for generation of 4 different logistic prediction models. The pretest model (1) was based on age, age squared, and sex. The test model (2) was based on the square root of each vessel-specific EBT score (left main, left anterior descending, left circumflex, and right coronary artery). The combined model (3) was based on sex, the 4 vessel-specific calcium scores, and 2 age-dependent interaction terms: age times square root of the total calcium score (when total calcium score ⬎0) or age alone (when total calcium score⫽0). The adjusted model (4) used the same variables as the combined model but was corrected for verification bias15,16 by use of the HarborUCLA data on the total population of 4982 patients referred for testing (879 referred for angiographic verification and 4103 who were not). The resultant prediction model was used to estimate the posterior or post-test probability of angiographically significant CAD in each of these 932 patients from which the model was derived
(training sample) as well as in the independent 919 patients (validation sample).
Statistical Analysis All values are reported as mean⫾SD. Categorical data were analyzed by means of 2 analysis and Fisher’s exact test. A 2 goodness-of-fit statistic was used to assess the calibration of each model relative to the line of identity (the agreement between observed and expected prevalence of disease in each decile of predicted probability; a 2 of zero indicating perfect calibration). A receiver operating characteristic (ROC) curve was created from each model to determine its discrimination (the ability to distinguish between those with and those without disease; a value of 0.5 indicating chance discrimination and a value of 1 indicating perfect discrimination).15 Statistical comparisons among the four models were made by computing the ratio of the 2 statistics, which were then tested with of an F test for the appropriate degrees of freedom (1, the number of nonzero deciles) in the numerator and denominator. All tests of significance were 2-tailed, and significance was defined at the ⱕ0.05 level.
Results Electron Beam Tomography Of the 1851 symptomatic patients, 1466 (79%) had a total calcium score ⬎0. Total calcium scores ranged from 1 to 6649. The mean and median scores were 380 and 116, and 25th and 75th percentiles were 3 and 456, respectively. Overall sensitivity was 96%, and specificity was 40% for any CAC to predict obstructive angiographic disease. With calcium scores ⬎20, ⬎80, and ⬎100, the sensitivity to predict stenosis decreased from 90% to 79% and 76%, respectively; the specificity increased from 58% to 72% and 75%, respectively. The mean square root of the score in men was significantly higher than that in women (23.8⫾67.5 versus 13.3⫾51.3, P⬍0.0001), and each sex demonstrated significant increases in scores with age. We found no significant differences of sensitivity for obstructive disease (men, 97%; women, 96%), although women had significantly higher specificity (men, 36%; women, 47%; P⬍0.001). Participants had, on average, 2.6 cardiovascular risk factors.
Angiography Of 1851 patients, 983 (53%) had obstructive disease (luminal stenosis ⱖ50% in 1 or more vessels). In this cohort, mean total calcium score was 608 (range, 0 to 6646). Calcium scores were significantly lower for individuals without obstructive disease (n⫽868; 123; range, 0 to 3761; P⬍0.001).
Multivariate Prediction of the Probability of CAD The variables incorporated into the model are prospectively defined and summarized in Table 1. The calibration and discrimination for each of the four prediction models are summarized in Table 2. Although the pretest model based only on age and sex was well calibrated, especially in the middle range of probability (Figure 1A), discrimination was relatively low (ROC area⫽0.672⫾0.019). In contrast, the test model based on the EBT calcium scores (Figure 1B) was significantly less well calibrated (F[7,5]⫽15.4, P⫽0.03) but higher in discrimination (ROC area⫽0.842⫾0.020). The combined model, incorporating all the variables, was well calibrated (Figure 1C) and provided a significant increase in
Budoff et al TABLE 1. Baseline Characteristics in the Training and Validation Samples Training Sample (n⫽932)
Validation Sample (n⫽919)
Mean
SD
Mean
SD
Age, years
57.716
10.942
57.653
11.159
Male sex
0.646
0.479
0.617
0.486
LM calcium score*
1.915
4.022
1.859
4.072
LAD calcium score
8.970
9.182
9.440
9.943 7.270
Square root of vessel score
LCX calcium score
4.497
7.121
4.952
RCA calcium score
5.720
8.097
6.294
9.122
Total calcium score
13.475
13.129
14.251
14.323
10.230
21.220
10.732
20.961
(Total calcium score⫽0)⫻age† (Total calcium score)⫻age‡
827.956
851.585
888.403
948.435
1-Vessel disease§
0.219
0.414
0.205
0.404
2-Vessel disease
0.147
0.354
0.161
0.368
3-Vessel disease
0.165
0.372
0.165
0.372
ⱖ2-Vessel disease
0.312
0.464
0.326
0.469
ⱖ1-Vessel disease
0.531
0.499
0.531
0.499
LM disease
0.027
0.162
0.027
0.163
LAD disease
0.379
0.485
0.375
0.485
LCX disease
0.282
0.450
0.286
0.452
RCA disease
0.318
0.466
0.331
0.471
LAD indicates left anterior descending; LCX, left circumflex; LM, left main; and RCA, right coronary artery. *Calcium scores represent the square root of the raw measurements. †(Total calcium score⫽0)⫻age is the interaction term in the absence of calcification. ‡(Total calcium score)⫻age is the interaction term in the presence of calcification. §Values for individual vessels represent population proportions.
discrimination over the pretest model (Figure 2) but also exhibited a significant amount of verification bias (the ratio of the angiographic referral rate in those with a calcium score ⬎0 [628/3157⫽0.199] divided by the angiographic referral rate in those with a calcium score⫽0 [251/1825⫽0.138] was 1.44). Adjustment for this verification bias did not affect calibration (Figure 1D) or discrimination (Table 2) but significantly raised specificity (66%).
Discussion Coronary risk factors do not allow for discrimination of severity of angiographic CAD17 or adverse outcomes in individuals but are useful prognostic indicators in large TABLE 2. Calibration and Discrimination in the Validation Sample for Each Model Model Pretest (1)
Calibration (Goodness of Fit)
Discrimination (ROC Area⫾SEM)
1.8 (P⫽0.94)
0.672⫾0.019
15.4 (P⫽0.03)
0.842⫾0.020
Combined (3)
9.0 (P⫽0.44)
0.842⫾0.023
Adjusted (4)
12.4 (P⫽0.19)
0.830⫾0.024
Test (2)
Angiographic Disease by EBT
1793
patient populations. Detection of CAC by EBT has been demonstrated to be highly sensitive for the presence of significant CAD.8 A positive EBT study (presence of CAC) is nearly 100% specific for atheromatous coronary plaque.6,7 Recent guidelines support the use of EBT in symptomatic persons, stating that EBT is “sufficiently accurate for predicting the presence of angiographic stenosis.”18 Previous reports tried to “merge’ multiple data sets with different entry criteria and different definitions of obstruction, demonstrating a very high sensitivity but lower specificity.18,19 Because CAC is a continuous variable with increasing calcium scores leading to increasing specificity, there is much greater information content than use as a dichotomous variable. The EBT-derived predictors of angiographic and multivessel disease can be useful as marker of underlying coronary atherosclerotic plaque burden, thus having a continuous association with atherosclerotic disease severity.10,20 Calcium scores from EBT correlate with severity of CAD and are useful in predicting angiographic 3-vessel and/or left main disease.20 This study documents the clinical applicability of EBT-derived calcium scores for the noninvasive diagnosis of CAD in two broad areas: for diagnosis, by the correlation of posterior probability with angiographic prevalence; and for evaluation, by the relation of posterior probability to the anatomic severity of disease. As in other studies, the addition of EBT scores added independent and incremental information to predict obstructive CAD over age and sex (Figure 2), dramatically altering the post-test probability of disease across a wide range of patients (Table 3). The identification of symptomatic patients with severe angiographic disease is becoming increasingly important.21 Our results demonstrate that in symptomatic patients, EBT in conjunction with a clinically derived pretest probability can noninvasively predict angiographic disease, with post-test probabilities for vessel-specific disease and multivessel disease. Not accounting for the increasing incidence of CAD with increasing age leads to a loss of information. By adding to the model an additional interaction term defined as (Total Score⫽0) · Age, the absence of calcification is weighted relatively more heavily in the model (proportional to age), whereas the overall calibration of the model remains essentially unchanged. For example, a 55-year-old man with a pretest probability of 53% (mean of our study population) and no calcification (calcium score⫽0) has a post-test probability of 9.9%. This is consistent with the raw negative predictive accuracy of 90% in this population: 315 of 350 patients with no CAC have normal coronary arteries. Similarly, the same man with a score of 380 (mean for the population studied) has a post-test probability of 87%. A variety of post-test probabilities are displayed in Table 3. Those patients who exhibited the greatest change from pretest to post-test probability were those patients with pretest probability ranging from 20% to 70%. This is consistent with many other cardiovascular tests, including exercise treadmill and thallium testing. Posterior probability also served as a predictor of the extent of angiographic disease, because the prevalence of multivessel disease increased as a monotonic curvilinear function of probability to a high of 92% (Figure 1C).
1794
Circulation
April 16, 2002
Figure 1. Calibration of multivariate logistic regression models for prediction of presence of angiographically significant CAD. Observed frequency of angiographically significant disease in each decile of predicted probability is represented. Number of observations comprising the proportion is noted below each bar. A, Model based only on age and sex; B, model based only on calcium scores; C, combined model based on age, sex, and calcium scores; and D, combined model after adjustment for verification bias.
A graphical comparison of the pretest model with the combined model is illustrated in Figure 2. The comparison is statistically significant (P⬍0.0001), indicating that EBT adds an increment of diagnostic information to age and sex. The resultant prediction model estimated the posterior probability of significant CAD in each of these 932 patients (training sample) and the 919 patients (validation sample). This approach ensures the capability of integrating additional vari-
Figure 2. ROC curves for pretest model based on age and sex and for the combined model based on age, sex, and calcium scores. Vertical and horizontal bars represent 95% CIs for each plotted true-positive rate and false-positive rate derived from computed logistic probabilities. Addition of EBT increased the area under the ROC curve from 0.67 to 0.84 (P⬍0.001).
ables or adjusting for differences in baseline prevalence of disease.22 The ROC curves for test and adjusted models were essentially identical to that for the combined model. The estimates of sensitivity and specificity are biased, however, because information regarding the ability of EBT to predict obstructive CAD was obtained from symptomatic patients who had CAD status verified with coronary angiography. As a result of this verification bias, sensitivity will be overestimated (positive EBT results are overrepresented) and specificity will be underestimated (negative EBT results are underrepresented). Bielak et al23 adjusted for verification bias, raising the specificity of EBT (for obstructive CAD) over the unadjusted specificity in the angiography group (72% versus 39%), with little change in the sensitivity (97% versus 99%). The verification bias in our study was 1.44, significantly greater than a value of 1 (no verification bias, P⬍0.0001) but lower than the value by Bielak et al.23 The patient enrollment in our study was mostly after the angiogram, so there is a higher likelihood that verification bias was reduced. Correction of the verification bias (Figure 1D) significantly raised the expected specificity for obstructive CAD (40% versus 66%, P⬍0.001), without significant change in sensitivity (96% versus 95%). A large angiographic and EBT study (n⫽1764) recently reported similar findings, with a higher sensitivity and lower specificity,24 in part due to higher pretest probabilities of disease. When evaluated for scores ⬎100, the two studies demonstrated nearly identical sensitivities and specificities
Budoff et al
Angiographic Disease by EBT
1795
TABLE 3. Post-Test Probabilities (%) for Obstructive CAD for a Variety of Pretest and Total EBT Scores in Men and Women Pre-Test Probability, %
Age 40
Age 50
Age 60
Score 0
Score 100*
Score 400*
Score 0
Score 100*
Score 400*
Score 0
Score 100*
Score 400*
10
1.7
25.9
68.1
1.2
23.0
61.0
0.9
20.4
53.3
20
3.7
44.1
82.8
2.8
40.2
77.9
2.0
36.5
72.0
30
6.2
57.4
89.2
4.6
53.6
85.8
3.5
49.7
81.5
40
9.3
67.7
92.8
7.0
64.2
90.4
5.3
60.5
87.3
50
13.3
72.9
93.4
10.2
72.9
93.4
7.7
69.7
91.1
60
18.7
82.5
96.7
14.5
80.2
95.5
11.1
77.5
93.3
70
26.4
88.0
97.8
20.9
86.3
97.0
16.3
84.3
96.0
Men
80
38.0
92.6
98.7
31.2
91.5
98.3
25.1
90.2
97.6
90
58.0
96.6
99.4
50.5
96.0
99.2
42.9
95.4
98.9
10
0.9
15.9
53.7
0.7
13.9
45.8
0.5
12.2
38.2
20
2.0
29.9
72.3
1.5
26.7
65.6
1.1
23.8
58.2
30
3.4
42.2
81.7
2.6
38.5
76.5
1.9
34.8
70.4
Women
40
5.2
53.2
87.4
3.9
49.3
83.5
2.9
45.4
78.8
50
7.7
63.0
91.2
5.8
59.3
88.4
4.3
55.5
84.8
60
11.1
71.9
94.0
8.4
68.6
91.9
6.4
65.1
89.3
70
16.2
79.9
96.1
12.5
77.3
94.7
9.6
74.4
92.8
80
24.9
87.2
97.7
19.7
85.4
96.8
15.3
83.3
95.7
90
42.8
93.9
98.9
35.6
92.9
98.6
29.0
91.8
98.0
*Total score based on equal distribution of calcification in each major epicardial artery. *Bold values represent 20% shift between pretest and post-test probability on the basis of EBT score.
for CAD as well as remarkably similar ROC values. Haberl et al24 demonstrated a sensitivity of 99% in men and 100% in women for detecting obstructive CAD, indicating that this test is an effective filter before angiography. EBT greatly enhances the clinician’s ability to predict angiographic disease, both prevalence and severity, and can be of aid in determining the next test or treatment. These observations should “travel” well from laboratory to laboratory despite local differences in population, as EBT is standardized, quantitative,8,11 and operator independent.11 These data will not be applicable to CAC scores obtained with multislice or spiral CT. Because this model is highly dependent on the absence of calcification (score⫽0), results of spiral CT should not be used. Calcium scores obtained with spiral CT are significantly different than EBT, with a lower sensitivity for calcification.25,26
Study Limitations Continuous variables of cardiac risk factors, such as serum lipoprotein levels, were not available for analysis. The results of this study are applicable to persons with suspected CAD, not the general asymptomatic population. We excluded severity of stress testing abnormality, since this must be used to estimate pretest probability. Similarly, the nature of chest pain would be used by the physician to derive a pretest probability of obstruction. A highly positive stress thallium examination in a patient with typical angina would greatly raise the pretest probability, not alter the age,
sex, or EBT results, which could then be used to generate a post-test probability.
Clinical Relevance of the Study Design We sought to determine the accuracy of EBT scanning in an environment that mirrored the realities of clinical practice. We gathered data from different geographic locales. The interpretation of each test, from risk factors to the angiogram, implied a variable amount of subjectivity and probable bias. This mirrors clinical reality, and is, in one sense, desirable. In summary, a practical, readily available program that integrates individual vessel calcium scores from EBT has been successfully used in the diagnosis and evaluation of CAD. EBT calcium scores are not a substitute for coronary angiography but can be used with other clinical and demographic markers to help predict angiographic stenosis severity in patients suspected of having obstructive CAD. As a noninvasive test for CAD, EBT has great appeal because it is fast (⬍10 minutes per scan), simple to use (no physiological or pharmacological stress required), and relatively inexpensive (estimated charge ⬍$400),12 with negligible interreader and intrareader variability.12 The model described can assist clinicians to develop a post-test probability that will be useful for establishing a high (75% to 92%) as opposed to a low (⬍10%) probability of angiographically significant CAD. It may facilitate individual decision-making concerning the use of coronary catheterization in symptomatic patients. Exclusion of CAC defines a substantial subgroup of patients, albeit
1796
Circulation
April 16, 2002
symptomatic, with a very low probability of significant stenosis. What differentiates this study is that it is the first to develop (1) a continuous model that is (2) adjusted for age and sex, (3) corrected for verification bias, and (4) independently validated in terms of its incremental diagnostic accuracy.
References 1. Kennedy J, Shavelle R, Wang S, et al. Coronary calcium and standard risk factors in symptomatic patients referred for coronary angiography. Am Heart J. 1998;135:696 –702. 2. Detrano R, Tzung H, Wang S, et al. Prognostic value of coronary calcification and angiographic stenoses in patients undergoing coronary angiography. J Am Coll Cardiol. 1996;27:285–290. 3. Arad Y, Sparado LA, Goodman K, et al. Prediction of coronary events with electron beam computed tomography. J Am Coll Cardiol. 2000;36: 1253–1260. 4. Raggi P, Callister TQ, Cooil B, et al. Identification of patients at increased risk of first unheralded acute myocardial infarction by electron beam computed tomography. Circulation. 2000;101:850 – 855. 5. Wong ND, Hsu JC, Detrano RC, et al. Coronary artery calcium evaluation by electron beam computed tomography and its relation to new cardiovascular events. Am J Cardiol. 2000;86:495– 498. 6. Rumberger JA, Simons DB, Fitzpatrick LA, et al. Coronary artery calcium areas by electron beam computed tomography and coronary atherosclerotic plaque area: a histopathologic correlative study. Circulation. 1995;92:2157–2162. 7. Mautner GC, Mautner SL, Froelich J, et al. Coronary artery calcification: assessment with electron beam CT and histomorphometric correlation. Radiology. 1994;192:619 – 623. 8. Budoff MJ, Georgiou D, Brody A, et al. Ultrafast computed tomography as a diagnostic modality in the detection of coronary artery disease: a multicenter study. Circulation. 1996;93:898 –904. 9. Wexler L, Brundage B, Crouse J, et al. Coronary artery calcification: pathophysiology, epidemiology, imaging methods, and clinical implications: a statement for health professionals from the American Heart Association. Circulation. 1996;94:1175–1192. 10. Guerci AD, Spadaro LA, Goodman KJ, et al. Comparison of electron beam computed tomography scanning and conventional risk factor assessment for the prediction of angiographic coronary artery disease. J Am Coll Cardiol. 1998;32:673– 679. 11. Agatston AS, Janowitz WR, Hildner FJ, et al. Quantification of coronary artery calcium using ultrafast computed tomography. J Am Coll Cardiol. 1990;15:827– 832. 12. Budoff MJ, Raggi P. Coronary artery disease progression assessed by electron-beam computed tomography. Am J Cardiol. 2001;88:46E-50E.
13. Becker C, Schatzl M, Feist H, et al. Assessment of the effective dose for routine protocols in conventional CT, electron beam CT and coronary angiography. Rofo Fortschr Geb Rontgenstr Neuen Bildgeb Verfahr. 1999;170:99 –104. 14. Scanlon PJ, Faxon DP, Audet AM, et al. ACC/AHA guidelines for coronary angiography: a report of the American College of Cardiology/ American Heart Association Task Force on practice guidelines (Committee on Coronary Angiography). J Am Coll Cardiol. 1999;33: 1756 –1824. 15. Begg CB, Greenes RA. Assessment of diagnostic tests when disease verification is subject to selection bias. Biometrics. 1983;39:207–215. 16. Roger VL, Pellikka PA, Bell MR, et al. Sex and test verification bias: impact on the diagnostic value of exercise echocardiography. Circulation. 1997;95:405– 410. 17. Vlietstra RE, Kronmal RA, Frye RL, et al. Factors affecting the extent and severity of coronary artery disease in patients enrolled in the Coronary Artery Surgery Study. Arteriosclerosis. 1982;2:208 –215. 18. O’Rourke RA, Brundage BH, Froelicher VF, et al. American College of Cardiology/American Heart Association expert consensus document on electron-beam computed tomography for the diagnosis and prognosis of coronary artery disease. Circulation. 2000;102:126 –140. 19. Nallamothu BK, Saint S, Bielak LF, et al. Electron-beam computed tomography in the diagnosis of coronary artery disease. Arch Intern Med. 2001;161:833– 838. 20. Schmermund A, Bailey KR, Rumberger JA, et al. An algorithm for noninvasive identification of angiographic three-vessel and/or left main coronary artery disease in symptomatic patients on the basis of cardiac risk and electron-beam computed tomographic calcium scores. J Am Coll Cardiol. 1999;33:444 – 452. 21. Caracciolo EA, Davis KB, Sopko G, et al. Comparison of surgical and medical group survival in patients with left main coronary artery disease: long-term CASS experience. Circulation. 1995;91:2325–2334. 22. Morise AP, Diamond GA, Detrano R, et al. The effect of diseaseprevalence adjustments on the accuracy of a logistic prediction model. Med Decis Making. 1996;16:133–142. 23. Bielak LF, Rumberger JA, Sheedy PF, et al. Probabilistic model for prediction of angiographically defined obstructive coronary artery disease using electron beam computed tomography calcium score strata. Circulation. 2000;102:380 –385. 24. Haberl R, Becker A, Leber A, et al. Correlation of coronary calcification and angiographically documented stenoses in patients with suspected coronary artery disease: results of 1,764 patients. J Am Coll Cardiol. 2001;37:451– 457. 25. Budoff MJ, Mao SS, Zalace CP, et al. Comparison of spiral and electron beam tomography in the evaluation of coronary calcification in asymptomatic persons. Int J Cardiol. 2001;77:181–188. 26. Goldin JG, Yoon HC, Greaser LE III, et al. Spiral versus electron-beam CT for coronary artery calcium scoring. Radiology. 2001;221:213–221.
