Development and implementation of a decision support system for ...

Development and Implementation of a Decision Support System for Carotid Artery Stenosis: The Carotid Ultrasound Report Enhancement (CURE) Brian F. Gage, MD, MSc, Gerald A. Banet, RN, MSN, Marion Goldstein, DDS, Walton Sumner, II, MD Division of General Medical Sciences, Washington University School of Medicine, St. Louis MO email gagebomsnotes.wust)l.edu and sumnerwsmsnotes.wustl.edu.

ABSTRACT

The management of carotid artery stenosis is an art evolving into a science, increasingly informed by clinical trials of medical management versus carotid endarterectomy (CEA). Ideas about optimal management depend on the confluence of patient-specific variables, surgical expertise, and the state of me/dical knowledge. In this complex and progressing setting, an up-to-date decision support system could help physicians apply the latest evidence to patient care. Carotid ultrasonography (US) studies provide an excellent opportunity to aid in the therapy of carotid stenosis. We developed a Carotid US Report Enhancement (CURE) to augment carotid US reports with treatment-specific prognostic information and patient-specific portions of the American Heart Association's 1998 guideline for the management of carotid artery stenosis. In the process of designing and implementing the CURE software, we encountered and eventually solved a variety of problems. The first problem was that US test was not always precise enough to distinguish between a moderate and mild carotid stenosis. Likewise, the standard US reports did not elucidate several technical problems that decreased the reliability of the US result. Third, although 17 of 18 physicians agreed to receive the CURE reports, they requested non-incriminating wording. Fourth, vascular surgeons supervising the US laboratories were reluctant to support the CURE report if they thought it would be construed as prompting self-referral. Finally, information about some comorbid conditions (e.g. a history of atrial fibrillation) could not be obtained reliably from the patients. The result of responding to these problems is a decision support program that is increasingly robust, able to detect many of its own limitations, and capable of integrating data from multiple sources. A randomized controlled trial now in progress will evaluate the clinical impact ofthe CUREprogram. INTRODUCTION Patients who have carotid artery stenosis may face a difficult choice between medical management and carotid endarterectomy (CEA). For asymptomatic patients, the

1067-5027/00/$5.00

2000 AMIA, Inc.

choice is difficult because the potential benefit of CEA is small-a reduction in the annual stroke rate by < 1%. Although the benefit of CEA is greater in symptomatic patients who have high-grade stenosis, the potential benefit is modest in symptomatic patients whose stenosis is < To further complicate the decision-making 80%32 process, the relative benefit of CEA depends on the skill of the operating surgeon and several patient-specific factorscomorbid conditions, age, and patient preferences. CEA guidelines3 could help guide therapy, but empiric evidence of adherence to similar guidelines demonstrates inconsistent application across physicians4' 5 Automated decision support could bring the evidencebased CEA guidelines promulgated by the American Heart Association, http: //circ .ahajournals . org/cgi/content /full/97/5/ 501 to the attention of physicians caring for patients who have a carotid artery stenosis. Without decision support, the complex choices between medical and surgical therapy depend upon providers' recall of the 12-page guideline. Decision support for carotid artery stenosis has potential to improve quality-adjusted survival and reduce medical expenditure: Ideal CEA candidates who forgo the procedure lose -11 months of qualityadjusted life-expectancy and have higher medical costs, on

average.6 Because the popularity of carotid duplex US screening and of CEA have made a dramatic comeback,7 an automated decision support system complementing carotid US tests has potential to help many Americans. Variability in the risks and benefits of CEA arises from variability at many levels: anatomy, accuracy of the US test, comorbid conditions, surgical expertise, and patient preferences.8 Although obtaining information on comorbidities is a potential obstacle to deploying a decision support system for carotid US tests, many US laboratories routinely collect this information to help them obtain accreditation by the Intersocietal Committee for the Accreditation of Vascular Laboratories. This information provides an opportunity for automated decision support to offer guidance customized to the clinical scenario as well as laboratory findings.

280

Because carotid duplex US is the most popular test for detecting carotid artery stenosis,9 US reports should be an ideal vehicle for providing decision support about the management of carotid stenosis. We sought to support decision making by doctors caring for patients who have carotid artery stenosis in two ways. First, we sought to disseminate pertinent portions of the American Heart Association (AHA) endarterectomy guideline through carotid US reports. Second, we sought to provide treatment-specific projections of health outcomes for individual patients. Our immediate goal was to test the potential to improve the care delivered after carotid US (US) by enhancing the US report with portions of the AHA guideline and prognostic information. Our long-term goal is to use enhanced US reports to educate referring physicians about carotid artery stenosis management, thus improving care for many patients. We describe the carotid US report enhancement (CURE) software and obstacles encountered during its development and implementation.

METHODS Overview The CURE software allows a technician to use a standalone computer to capture demographic, clinical, and US information about patients. The software produces a report describing the prognostic implications of the US results, based on a review of local surgical outcomes and patientlevel details. The report also notes relevant AHA recommendations, based on the same information, and reproduces portions of that guideline as an aid for the referring clinician. Software Development The CURE software package was written in Microsoft Excel and Visual Basic for Applications. Excel worksheets perform all calculations, while Visual Basic automates tasks such as randomization, restoring old records, and clearing input forms. An Excel worksheet serves as a flat database recording each study. Presently, CURE software runs on an independent computer system, but in the future it may be integrated with other software, including web browsers. The CURE program has 5 conceptual components. First, a subjective input component captures information from a questionnaire collected in the laboratory. Second, an objective input component captures the formal interpretation of the carotid US report. Both input components reflect their paper counterparts to facilitate data entry and eventual transition to an all-electronic process. The subjective component captures symptoms, cerebrovascular diagnoses, previous studies, objective findings (e.g. bruits), and comorbid conditions. The objective component records the study results for each

281

carotid artery. Third, the prediction component applies the prognostic algorithm (below) to estimate treatment-specific projections of stroke and death. Fourth, the guideline component uses the estimated internal carotid artery stenosis to navigate through the AHA guideline to identify a recommendation based on symptoms and ischemic events, US findings, and predicted peri-operative complications. Finally, the reporter component combines of the portions treatment-specific outcomes, recommendation, and the comorbidity list to assemble recommendations and caveats. This architecture should facilitate maintenance of CURE software as medical knowledge evolves, because it places each concept that is likely to change in a separate software component. Prognostic Algorithm Development

Stroke Incidence in the Symptomatic Carotid Artery For symptomatic patients undergoing CEA, we estimated the incidence of ipsilateral stroke from the two large clinical trials of CEA for symptomatic patients: The North American Symptomatic Carotid Endarterectomy Trial (NASCET)10 and the European Carotid Surgery Trial (ECST)2. We obtained a 5-year incidence of ipsilateral stroke of 11.5% by averaging the 5-year incidence of stroke from symptomatic NASCET participants (14.5%) with our 5-year extrapolation from symptomatic ECST participants (8.4%). Based on the results of the clinical trials, we used the same 11.5% 5-year stroke incidence regardless of the degree of stenosis. We used the same trials to estimate the ipsilateral stroke incidence for symptomatic patients who were treated with aspirin (ASA) alone. Because both NASCET and ECST found that the incidence of stroke increased in symptomatic participants who had higher degrees of stenosis, we combined these two trials using dosedependent meta-regression. First, we recalibrated the degree of stenosis in ECST to the same scale that was used in NASCET. Second, using ordinary least-squares regression for each individual trial, we derived equations predicting the 5-year incidence of stroke as a function of stenosis in the ipsilateral artery. By averaging results for these trials, we obtained our final equation: Stroke incidence = 0.221 *stenosis + 8.9%. For example, a patient with a 75% stenosis would have a 5-year projection of ipsilateral stroke incidence = 0.221*75% + 8.9% = 25.5%.

Stroke Incidence in the Asymptomatic Carotid Artery For asymptomatic patients considering CEA, we used the meta-analysis of Benavente and colleagues 11 to estimate the potential reduction in stroke from CEA. They found an odds ratio of stroke of 0.46 in asymptomatic participants in large trials of asymptomatic participants who underwent CEA.1 12 To calculate the 5-year incidence of stroke, we

multiplied 0.46 by our stroke projections in asymptomatic patients treated with ASA alone.

For projections in asymptomatic patients being treated with ASA alone, we estimated the incidence of stroke from the clinical trials of CEA for asymptomatic patients 1' 12 combined with two large observational studies of asymptomatic patients: FraminghamI3 and The Copenhagen City Heart Study (CCHS).14 These observational studies allowed us to incorporate comorbid conditions in our stroke projections and allowed us to have a consistent data source when projecting stroke rates in the contralateral carotid artery. For arteries without significant stenosis, we averaged the stroke projections from Framingham and the CCHS studies and divided by 2 (because we calculated stroke risk in each of the 2 arteries separately). For arteries in asymptomatic patients that had a significant stenosis, we increased the risk of stroke by a factor of 2.13 based on our analysis of ECST data.2 We defined, a significant stenosis as > 60% based on the results of the clinical trials.1' 2, 12 Because none of these trials found that the rate of stroke increased with tighter stenosis in asymptomatic patients, we used the same factor (2.13) for all patients having a stenosis in the range of 60-99%. Note that we did not project stroke rates for complete carotid occlusion because CEA is not done in this population. Peri-operative Complications The CURE software may use either generic or hospitalspecific incidence rates of adverse events. The generic incidence rates of stroke and death depend on the presence or absence of carotid artery symptoms and on patient age. For example, an asymptomatic middle-aged patient has a peri-operative adverse event rate of 3.0%, while higher rates are used in symptomatic or older populations. Hospital-specific incidence rates may be used to correct for local peri-operative adverse event rates. For example, vascular surgeons at Barnes-Jewish hospital operate using cervical block anesthesia, a technique that appears to have lower perioperative complications.

Generating guideline portions Although the AHA guideline is complex, the local adaptation to US findings is relatively simple. Surgery is recommended for symptomatic patients with > 50% internal carotid artery stenosis. Life expectancy is not considered explicitly, as only extremely ill patients do not benefit, on average, from avoiding the very high probability of stroke. Surgery is recommended for asymptomatic patients with'> 70% internal carotid artery stenosis and > 5 year life expectancy. The AHA guideline stipulates a 60% stenosis threshold, but Doppler US is literally incapable of differentiating 51% from 69%

282

stenoses. Local practice is to use blood velocity criteria to classify an internal carotid lesion as < 50%, 50 to 79%, or 80 to 99% stenosed. If the lesion is 50 to 79%, then the ratio of the internal and common carotid artery velocities is used to classify the lesion. Ratios > 4 are assumed to indicate > 70% stenosis in this subgroup, and a possible indication for surgery. Patients with lower ratios are managed medically.

All patients who qualify for cervical block anesthesia are expected to have surgical complication rates below the 3% threshold stipulated by the guideline for asymptomatic patients. However, the guideline provides a variety of situations in which higher perioperative risks are possibly acceptable. These were simply enumerated for all asymptomatic patients with > 50% stenosis.

Medical therapy reminders included antiplatelet therapy (suggesting aspirin unless the patient is allergic to it), smoking cessation for current smokers, blood pressure control for hypertensive patients, and warfarin for patients who have atrial fibrillation. Blood sugar control for diabetics is not included. Although diabetes is a major risk factor for carotid artery disease,15 tight control is not known to decrease the incidence of stroke. IMPLEMENTATION: PROBLEMS & SOLUTIONS

Early drafts of the CURE program were tested with patient data collected by a vascular laboratory. Subsequently, we began to enroll patients in a randomized controlled trial of report enhancements at the same laboratory. During its development and subsequent implentation, problems developed which required software revisions.

Subjective-Objective-Assessment-Reminder (SOAR) Historical information available to the vascular laboratory was of variable quality. For instance, a physician's office might schedule a study to evaluate a bruit, but the patient would arrive with no knowledge of this finding or where it was located. Atypical symptoms accounted for a large number of studies. These might involve a unilateral symptom that would initiate a cascade of assumptions about possible ischemic symptoms, changing decision thresholds, finally creating a reminder to consider CEA. Conversely, a conservative reminder could be inappropriately reassuring if the patient actually had an unsuspected symptom. To reduce misconceptions caused these problems, CURE produces a report enhancement in a SOAR format. The subjective section reiterates historical data offered to the staff at the vascular laboratory. The Objective section reiterates and begins to interpret US findings. The Assessment section presents the expected outcomes, or explains why it can not predict outcomes. The Reminder section presents the main surgical recommendation from the locally adapted AHA guideline,

followed by caveats that would negate the main conclusion. For instance, in an asymptomatic patient with a 50 to 70% lesion, the reminder section indicates that evaluation for surgery is not recommended, but that evaluation for surgery would be recommended if an ischemic symptom were discovered. Any report of a 50 to 79% stenosis includes the caveat, "ifthe history is correct." Malpractice concerns shaped the wording of the reminders, and even the choice of "Reminder" as the title of the section. "Recommendations" from the AHA were felt by several parties to be too condemning if a bad outcome occurred when the doctor did not follow the recommendation. "Reminders" were to "evaluate for surgery," meaning that anyone could do the evaluation, rather than "surgical evaluation," which seemed to suggest that a surgeon must evaluate the patient. Gracefully folding A peculiarity of Doppler US is that stenosis is inferred from velocity measures. Consequently, any process that alters velocities alters the reported stenosis. Contralateral stenosis predictably causes elevated velocities.16 17 Thus, if one carotid artery is occluded, the other artery may show elevated velocities and be reported asstenosed. When such patients have repeat studies, the doctor may be watching for a further increase in velocity, rather than interpreting the reported stenosis. A similar problem arises with arteries subjected to any surgery. Scarring or hypertrophy of vessel walls decreases the cross section of the lumen, increasing velocities and reported stenosis. Although cross sectional images might be expected to help, these are also quite unreliable. Interestingly, standard reports may ignore these nuances, reporting different degrees of stenosis calculated by Doppler and cross section in the presence of a known velocity-altering event. However, a decision support system should recognize when it is overwhelmed by circumstances, decline to extrapolate its prognostic algorithms, and add further caveats to the Objective and Reminder sections of the report.

Occasionally the prognostic algorithm does not support the AHA recommendation. That is, the AHA guideline may recommend surgery when our prognostic algorithm suggests that medical outcomes are equal or better at 5 years. As the prognostic algorithm has not been verified formally, we are currently holding these reports for further evaluation. A common cause of these discrepancies is that as compared to asymptomatic men, asymptomatic women have less benefit from CEA.12 Missing data Missing data are also common. The most important missing items have been the following: disease laterality, handedness, blood pressure, and the distinction between

atrial fibrillation versus other arrhythmias. Surprisingly, some patients believe that they have had strokes documented by imaging studies, but do not recall which side was affected, and cursory examination is not revealing. We would consider the affected side to be symptomatic, but are unable to clearly identify the correct stenosis threshold for each side. In these situations we navigate the AHA guideline assuming that both sides are symptomatic. The handedness of a patient is not always available, but may change a decision about which side to consider for CEA. Assuming right handedness is the best course, but could cause confusion if the patient is actually left handed. Blood pressure is one of the most important predictors of future stroke, but is not consistently measured in the vascular laboratory. In this case, information sufficient for accreditation, billing, and guideline purposes is inadequate for our prognostic algorithm. We simply assume a systolic blood pressure of 145, knowing that the actual blood pressure could be much higher. Atrial fibrillation dramatically increases a patient's risk of stroke, but has been an exclusion criterion for large trials. We assume that the risk of stroke attributable to atrial fibrillation is not reduced by CEA, and that atrial fibrillation increases the risk of stroke in a medically managed patient by a factor of 1.78.18 However, we again must make an estimate of the probability of atrial fibrillation when the vascular laboratory documents arrhythmia. Physician participation A collaborator knew the physicians who used the vascular laboratory most frequently. The collaborator left the institution as the trial was about to begin. Consequently, we needed to recruit physicians who did not know us to participate in a trial that some viewed as prescriptive interference. Three strategies were used to elicit the support of physicians referring patients for carotid ultrasounds. First, frequent referrers received letters explaining the CURE project and asking for their support and feedback. Second, all physicians received an "Invitation" CURE Report with a request to fax back their We telephoned the preference for participating. physician's office prior to faxing an invitation, offering a more complete explanation of the project and answering any questions. Third, two members of the team met with neurologists who care for the majority of the stroke patients referred to the hospital to solicit their support. Of the first 18 physicians contacted, one physician declined to participate, and one requested revisions in the CURE report prior to participating. Perceived Conflicts of Interest A major experimental problem has been that vascular surgeons supervising vascular laboratories are sensitive to charges of self-referral by their colleagues. We are primary care providers. We have offered the CURE

283

program to four vascular US providers. Two laboratories managed by neurologists or radiologists are interested in the program. One laboratory managed by vascular surgeons is cautiously cooperating, but with no commitment to continue the project. Another laboratory managed by vascular surgeons is unwilling to risk a political backlash from the primary care community it serves. DISCUSSION We are testing the CURE program in a randomnized, controlled trial. The main outcome variables will be treatment status at 3 months (surgical versus medical treatment) and health status (adverse event or not). Secondary analyses will include self-reported changes in medical therapy at 3 months. During the enrollment period 967 carotid ultrasound studies were identified, and 315 outpatients were enrolled in CURE. Cure subjects were predominantly white (88%), evenly divided between male and female, and their mean age was 69.2 years at the time of the US study. Over 36% of studies identified at least one internal carotid lesion with greater than or equal to 50% stenosis. Referring physicians were overwhelmingly local specialists in cardiovascular and neurologic medicine and surgery, and included fewer primary care physicians than originally expected. We hope that the CURE program will facilitate other studies, particularly studies of patient preferences for surgery versus medical therapy. We also expect that the problems encountered here will reappear as other test results are combined with patient-level information' to clarify a study-'s clinical implications. GRANT SUPPORT

The American Heart Association funded this project via a Patient Care and Outcomes Research Grant. REFERINCES 1. Hobson RW 2d, Weiss DG, Fields WS et al. Efficacy of carotid endarterectomy for asymptomatic carotid stenosis. The Veterans Affairs Cooperative Study Group. N EnglJMed 1993;328:221-7. 2. Randomised trial of endarterectomy for recently symptomatic carotid stenosis: final results of the MRC European Carotid Surgery Trial (ECST). Lancet

1998;351:1379-87. 3. Biller J, Feinberg WM, Castaldo JE et al. Guidelines for carotid endarterectomy: a statement for healthcare professionals from a special writing group of the Stroke Council, American Heart Association [see comments]. Stroke 1998;29:554-62. 4. Kutner M, Nixon G, Silverstone F. Physicians' attitudes toward oral anticoagulants and antiplatelet agents

for stroke prevention in elderly patients with atrial fibrillation. Arch Intern Med 1991; 151:1950-3. 5. Legorreta AP, Christian-Herman J, O'Connor RD, Hasan MM, Evans R, Leung KM. Compliance with national asthma management guidelines and specialty care: a health maintenance organization experience. Arch Intern Med 1998;158:457-64. 6. Nussbaum ES, Heros RC, Erickson DL. Costeffeptiveness of carotid endarterectomy. Neurosurgery 1996;38:237-44. 7. Tu JV, Hannan EL, Anderson GM ,et al. The fall and rise of carotid endarterectomy in the United States and Canada. NEnglJMed 1998;339:1441-7. 8. Gage BF, Cardinalli AB, Owens DK. The effect of stroke and stroke prophylaxis with aspirin or warfarin on quality of life. Arch Intern Med 1996;156:1829-36. 9. Lee TT, Solomon NA, Heidenreich PA, Oehlert J, Garber AM. Cost-effectiveness of screening for carotid stenosis in asymptomatic persons. Ann Intern Med 1997;126:337-46. 10. Barnett HJM, Taylor DW, Eliasziw M et al. Benefit of carotid endarterectomy in patients with symptomatic moderate or severe stenosis. North American Symptomatic Carotid Endarterectomy Trial Collaborators. N Engl J Med 1998;339: 1415-1425. 11. B¢navente 0, Moher D, Pham B. Carotid endarterectomy for asymptomatic carotid stenosis: a metaanalysis. BMJ 1998;317:1477-80. 12.. Executive Committee for the Asymptomatic Carotid Atherosclerosis Study. Endarterectomy for asymptomatic carotid artery stenosis. JAMA 1995;273: 1421-8. 13. D'Agostino RB, Wolf PA, Belanger AJ, Kannel WB. Stroke risk profile: adjustment for antihypertensive medication. The Framingham Study. Stroke 1994;25:40-3. 14. Truelsen T, LindenstromE, Boysen G. Comparison of probability of stroke between the Copenhagen City Heart Study and the Framingham Study. Stroke 1994;25:802-7. 15. Weinberger J, Biscarra V, Weisberg MK, Jacobson JH. Factors contributing to stroke in patients with atherosclerotic disease of the great vessels: the role of diabetes. Stroke 1983;14:709-12. 16. Horrow MM, Stassi J,' Shurman A, Brody JD, Kirby CL, Rosenberg HK. The limitations of carotid sonography: interpretive and technology- related errors. AJR Am J Roentgenol 2000;174: 189-94. 17. Hood DB, Mattos MA, Mansour A et al. Prospective evaluation of new duplex criteria to identify 70% internal carotid artery stenosis. J Vasc Surg 1996;23:254-61; discussion 261-2. 18. Lai SM, Alter M, Friday G, Sobel E. A multifactorial analysis of risk factors for recurrence of ischemic stroke. Stroke 1994;25:9158-62.

284