612
Wilber et al.
d
SCREENING TOOLS FOR COGNITIVE IMPAIRMENT
An Evaluation of Two Screening Tools for Cognitive Impairment in Older Emergency Department Patients Scott T. Wilber, MD, Samuel D. Lofgren, MD, Thomas G. Mager, MD, Michelle Blanda, MD, Lowell W. Gerson, PhD Abstract Objectives: Screening for cognitive impairment in older emergency department (ED) patients is recommended to ensure quality care. The Mini-Mental State Examination (MMSE) may be too long for routine ED use. Briefer alternatives include the Six-Item Screener (SIS) and the Mini-Cog. The objective of this study was to describe the test characteristics of the SIS and the Mini-Cog compared with the MMSE when administered to older ED patients. Methods: This institutional review board–approved, prospective, randomized study was performed in a universityaffiliated teaching hospital ED. Eligible patients were 65 years and older and able to communicate in English. Patients who were unable or unwilling to perform testing, who were medically unstable, or who received medications affecting their mental status were excluded. Patients were randomized to receive the SIS or the Mini-Cog by the treating emergency physician. Investigators administered
the MMSE 30 minutes later. An SIS score of #4, the MiniCog’s scoring algorithm, and an MMSE score of #23 defined cognitive impairment. Results: A total of 149 of 188 approached patients were enrolled; 74 received the SIS and 75 the Mini-Cog. Fifty-five percent were female, the average age was 75 years, and 23% had an MMSE score of #23. The SIS had a sensitivity of 94% (95% confidence interval [CI] = 73% to 100%) and a specificity of 86% (95% CI = 74% to 94%). The Mini-Cog had a sensitivity of 75% (95% CI = 48% to 93%) and a specificity of 85% (95% CI = 73% to 93%). Conclusions: The SIS, using a cutoff of #4 as impaired, is a promising test for ED use. It is short, easy to administer, and unobtrusive, allowing it to be easily incorporated into the initial assessment of older ED patients. Key words: cognitive impairment; emergency department; MiniMental State Examination; Six-Item Screener; Mini-Cog. ACADEMIC EMERGENCY MEDICINE 2005; 12:612–616.
The Society for Academic Emergency Medicine (SAEM) Geriatric Emergency Medicine Task Force care model included a recommendation that all older emergency department (ED) patients receive some assessment of their cognitive status.1 Emergency physicians often do not recognize cognitive impairment in older patients, despite a prevalence of 26%– 40%.2–6 Delirium (an acute mental status change considered a medical emergency due to its multiple serious underlying medical causes) and dementia are important causes of this cognitive impairment.2–6 Both affect patient care substantially. Failure to recognize dementia can lead to erroneous conclusions
about the reliability of the patient’s history and ability to comply with discharge instructions. Undiagnosed delirium can have serious consequences, including death.2 Improving emergency physicians’ recognition of cognitive impairment is essential. The task force recommended screening with threeitem recall and orientation, followed by the MiniMental State Examination (MMSE) if abnormalities are detected on the initial screen.1,7 The time it takes to administer the MMSE may make it impractical for routine ED use by emergency physicians. A sensitive test is needed that can be incorporated into routine history and examination without increasing the time needed to evaluate the patient.8 Two brief cognitive tests that meet these criteria are the Six-Item Screener (SIS) and the Mini-Cog. The SIS (available as an online Data Supplement at http:// www.aemj.org/cgi/content/full/12/7/612/DC1) consists of three questions on temporal orientation (day, month, and year) and three-item recall.9 The Mini-Cog (available as an online Data Supplement at http://www.aemj.org/cgi/content/full/12/7/612/ DC1) consists of three-item recall and a clock drawing test.10 Our objective was to determine the sensitivity and specificity of these two screening tools for cognitive impairment compared with the MMSE in older ED patients.
From the Department of Emergency Medicine, Summa Health System (STW, SDL, TGM, MB), Department of Emergency Medicine, Northeastern Ohio Universities College of Medicine (STW, SDL, TGM, MB), and Division of Community Health Sciences (LWG), Akron, OH. Received November 11, 2004; revisions received December 28, 2004, and January 11, 2005; accepted January 13, 2005. Presented at the SAEM annual meeting, Orlando, FL, May 2004. Supported by the Summa Foundation and a Dennis W. Jahnigen Career Development Scholars Award from the American Geriatrics Society (STW). Address for correspondence and reprints: Scott T. Wilber, MD, 41 Arch Street, Room 519, Akron, OH 44304. Fax: 330-375-7564; e-mail:
[email protected]. doi:10.1197/j.aem.2005.01.017
ACAD EMERG MED
d
July 2005, Vol. 12, No. 7
d
613
www.aemj.org
METHODS Study Design. We conducted a prospective, randomized, cross-sectional study. This study was approved by the institutional review board of Summa Health System. The institutional review board did not require informed consent, but we gave all patients a statement of research and obtained verbal consent before administration of the MMSE. Study Setting and Population. The study was conducted in the fall of 2003 during convenient times when investigators were available. During these times, we approached all patients 65 years or older who were able to communicate in English. We excluded patients unable or unwilling to perform testing, those who were medically unstable, and those who received medications during the study that could affect their mental status. The study was conducted in Summa Health System’s Akron City Hospital ED. This community teaching hospital had 72,500 ED visits in 2003; one fourth were by patients aged 65 years or older. Study Protocol. We used a computerized randomnumber generator to assign patients to receive the SIS or the Mini-Cog in four equal blocks. An investigator, who was not the treating physician, identified poten-
tially eligible patients and placed a data collection sheet corresponding to the random assignment on the chart. The treating physician (attending or senior emergency medicine resident) evaluated the patient’s level of consciousness and performed the SIS or the Mini-Cog. Level of consciousness was classified as alert (normal), vigilant (hyperalert), lethargic (drowsy, easily aroused), stuporous (difficult to arouse), or comatose (unarousable).11 The SIS and the Mini-Cog were performed and scored by the treating physician using standard instructions. All physicians underwent brief (10-minute) instruction in administering and scoring the screening tests before the start of patient enrollment. An investigator (SDL or TGM) administered the MMSE at least 30 minutes after the initial evaluation. The MMSE had a different three-item recall list than the initial screen. The time delay and alternate recall list were used to prevent a learning phenomenon. In our preliminary study, we used a 10-minute delay and the same recall list, and more than half the patients scored higher on the second three-item recall question.12 The MMSE was scored by investigators according to the standard instructions.7 Measurement. For the SIS, a score of #4 was used as the cutoff for impairment. This was selected based
Figure 1. Flow diagram of patient enrollment.
614
Wilber et al.
TABLE 1. Demographic Information Mean age 6 SD (yr) % Female % With an MMSE score #23 % With level of consciousness ‘‘awake and alert’’*
SIS
Mini-Cog
75.4 6 6.6 54
75.4 6 7.1 56
24
21
91
94
*Level of consciousness data were missing in four patients in the Mini-Cog group and six patients in the SIS group. SIS = Six-Item Screener; MMSE = Mini-Mental State Examination.
on our analysis of the original study’s results9 and a preliminary study at our institution in which this cutoff was found to provide the greatest diagnostic accuracy.12 The standard scoring algorithm was used to define impairment for the Mini-Cog.10 A score of #23 on the MMSE was used as the criterion standard for cognitive impairment.7 Data Analysis. All data were entered into a Microsoft Access database (Microsoft Corp., Redmond, WA) and converted for data analysis using Stata software (Stata Corp., College Station, TX). We present the sensitivity and specificity with 95% confidence intervals (CIs) and the raw agreement and k of the SIS or the Mini-Cog compared with the MMSE as the criterion standard. We constructed a receiver operating characteristic (ROC) curve for the SIS, because varying the cutoff for impairment on this test affects the sensitivity and specificity. An ROC was not done for the Mini-Cog, which has a dichotomous outcome. Sample Size. We anticipated that 33% of patients would be cognitively impaired2–6 and that the sensitivity of the screening tests would be 90%. A sample size of 75 per group would provide a lower bound of 70% for the 95% CI around 90%. We believed this was acceptable for this preliminary study, and doubling the sample size would increase this lower bound only to 80%.
RESULTS We approached 188 patients; 34 were excluded and four refused, leaving 150 patients for assignment (Figure 1). Twenty patients were excluded due to
d
SCREENING TOOLS FOR COGNITIVE IMPAIRMENT
inability to perform testing, nine were medically unstable, two were discharged before testing was completed, one had medication administration during the test, and one was already enrolled. One patient was listed as excluded, although no reason for the exclusion was recorded. Seventy-five patients received the SIS, and 75 received the Mini-Cog. One 64-year-old patient was inadvertently enrolled in the SIS group and was a protocol violation. This patient was not retained for data analysis, leaving 74 patients in the SIS group. The groups were similar in demographics and mental status (Table 1). Fifty-five percent of subjects were female, and the average (6SD) age was 75 (67) years. Thirty-four of 149 patients enrolled (23%; 95% CI = 16% to 30%) were cognitively impaired by the MMSE (18 in the SIS group and 16 in the Mini-Cog group). Seventeen attending physicians and 14 senior residents participated in the data collection. The test characteristics of the SIS and the Mini-Cog are shown in Tables 2 and 3, respectively. The SIS had a sensitivity of 94% (95% CI = 73% to 100%), and the Mini-Cog had a sensitivity of 75% (95% CI = 48% to 93%). The specificities were similar (86% [95% CI = 74% to 94%] and 85% [95% CI = 73% to 93%]). Negative predictive values were .90% for both the SIS (98%; 95% CI = 89% to 100%) and the Mini-Cog (93%; 95% CI = 82% to 98%), although positive predictive values were lower. The SIS agreed with the MMSE in 88% of the cases, with a k of 0.7 (substantial agreement). The Mini-Cog agreed with the MMSE in 83% of the cases, with a k of 0.5 (moderate agreement). The area under the ROC curve for the SIS shown in Figure 2 was 0.96 (95% CI = 0.92 to 1.0).
DISCUSSION We found the SIS to have an excellent sensitivity, specificity, and substantial overall agreement with the MMSE. The sensitivity and negative predictive value of the Mini-Cog were lower. We found in an earlier study that half of older patients completed the SIS in less than 1 minute (range, 0.5–3.5 minutes).12 The clock drawing test component of the Mini-Cog took a median of 1.5 minutes (range, 0.5–5 minutes) to complete.12 One practical lesson learned from that study was that older ED patients are often unable to write or draw.
TABLE 2. Test Characteristics for the SIS SIS positive SIS negative
MMSE Score #23
MMSE Score .23
17
8
1
48
Sensitivity = 17/18 (94%), 95% CI = 73%, 100%
Specificity = 48/56 (86%), 95% CI = 74%, 94%
SIS = Six-Item Screener; MMSE = Mini-Mental State Examination.
Positive predictive value = 17/25 (68%), 95% CI = 46%, 85% Negative predictive value = 48/49 (98%), 95% CI = 89%, 100%
ACAD EMERG MED
d
July 2005, Vol. 12, No. 7
d
615
www.aemj.org
TABLE 3. Test Characteristics for the Mini-Cog Mini-Cog positive Mini-Cog negative
MMSE Score #23
MMSE Score .23
12
9
4
50
Sensitivity = 12/16 (75%), 95% CI = 48%, 93%
Specificity = 50/59 (85%), 95% CI = 73%, 93%
Reasons for this include arm injuries, the presence of intravenous catheters in the arm, or weakness.12 Moreover, the Mini-Cog requires pencils, papers, and scoring sheets, while the SIS can be administered and scored without these. This further supports using the SIS in the ED. The MMSE was developed in 1975 and has since become a widely used and accepted test.7 The test assesses many cognitive functions, including orientation, registration, attention and calculation, recall, language, and the ability to follow simple commands.7 However, we have found the usefulness of the MMSE to be limited in the ED for several reasons. First, it takes a median of 5.5 minutes (range, 3.5–14 minutes) to complete. Additionally, it requires adequate vision and hearing, may be threatening and/or intimidating to some patients (which may alter effort level), and requires scoring sheets and pencils.12 Other short screening tests are available, including the Short Portable Mental Status Questionnaire13 and the Orientation Memory Concentration Test.14 Although briefer than the MMSE, we believed that these were more difficult to incorporate into the routine examination. The former consists of ten questions and the latter six, with scoring weights for each answer. The SAEM Task Force emphasized a ‘‘biopsychosocial’’ approach to the older patient, in which cognitive
Positive predictive value = 12/21 (57%), 95% CI = 34%, 78% Negative predictive value = 50/54 (93%), 95% CI = 82%, 98%
status, functional status, and psychosocial concerns are incorporated into patient care.15 We believe that knowledge about a patient’s mental status is important for emergency physicians because it has obvious repercussions for acute treatment, including the ability to give a reliable history or follow discharge instructions. Recent reports have shown increased mortality and morbidity in patients with delirium that was missed in the ED. Kakuma et al. found that patients with delirium not detected in the ED had a six-month mortality of 30.8%, more than double the rate of those with delirium detected in the ED (11.8%) and nondelirious patients (14.3%).2 Others have shown that the prevalence of delirium is approximately 10% in older ED patients, and it is detected by emergency physicians in only 17%–35% of cases.2–6 A hallmark of the diagnosis of delirium is recognizing that impairment of memory or orientation exists.16 A recent editorial emphasized the importance of missed delirium in older ED patients and argued that this represented an area of ‘‘medical errors, missed diagnoses, and quality-of-care concerns.’’17
LIMITATIONS Our study has several limitations. First, we did not blind the investigators performing the MMSE to the
Figure 2. ROC analysis for the SIS. Area under ROC curve = 0.9618.
616
Wilber et al.
results of the SIS or the Mini-Cog, so that the investigator could answer questions about administration or scoring of the screening test. This could introduce bias. Second, the 23% cognitive impairment seen in our study is slightly lower than in other ED studies.2–6 Some of these studies included patients with stupor or coma; we excluded these patients because they were unable to perform testing. Although the ED evaluation is similar, stupor and coma are independent diagnoses. We also excluded severely ill patients, and this could introduce spectrum bias. Third, while our results are compelling in support of the SIS, the CI for sensitivity is still broad due to the smaller number of patients with cognitive impairment. The study was not powered to detect a difference in sensitivity between the SIS and the Mini-Cog, and our results show no statistical significance for the difference in sensitivity. We did not enroll Spanish-speaking or other non– English-speaking patients, because our ED does not see a significant proportion of these patients. To our knowledge, the SIS has not been validated in this population. Finally, we used the standard cutoff for the MMSE rather than an age- or education-based cutoff. This may result in misclassification of some patients.
CONCLUSIONS The SIS, using a cutoff of #4 as impaired, is a promising test for ED use. It is a sensitive and accurate screening test when compared with the MMSE as a criterion standard. Its brevity and simplicity allow it to be easily incorporated into the initial assessment of older ED patients. The sensitivity and accuracy of the Mini-Cog are lower, and it requires writing ability and pencils, paper, and scoring sheets. We recommend that emergency physicians consider incorporating the SIS into their evaluation of older ED patients. Because both the SIS and the Mini-Cog are screening tools, abnormalities should be followed by more comprehensive mental status evaluation.
2.
3.
4.
5.
6.
7.
8. 9.
10.
11.
12.
13.
14.
15.
16.
References 1. Sanders AB, Witzke DB, Jones JS, Richmond K, Kidd P. Principles of care and application of the geriatric emergency care model. In: Sanders AB (ed). Emergency Care of the Elder
17.
d
SCREENING TOOLS FOR COGNITIVE IMPAIRMENT
Person. St. Louis, MO: Beverly Cracom Publications, 1996, pp 97–116. Kakuma R, du Fort GG, Arsenault L, et al. Delirium in older emergency department patients discharged home: effect on survival. J Am Geriatr Soc. 2003; 5:443–50. Naughton BJ, Moran M, Ghaly Y, Michalakes C. Computed tomography scanning and delirium in elder patients. Acad Emerg Med. 1997; 4:1107–10. Naughton BJ, Moran MB, Kadah H, Heman-Ackah Y, Longano J. Delirium and other cognitive impairment in older adults in an emergency department. Ann Emerg Med. 1995; 25:751–5. Hustey FM, Meldon SW. The prevalence and documentation of impaired mental status in elderly emergency department patients. Ann Emerg Med. 2002; 39:248–53. Elie M, Rousseau F, Cole M, Primeau F, McCusker J, Bellavance F. Prevalence and detection of delirium in elderly emergency department patients. CMAJ. 2000; 162: 977–81. Folstein MF, Folstein SE, McHugh PR. Mini-Mental State: a practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res. 1975; 12:189–98. Wilber ST, Gerson LW. A research agenda for geriatric emergency medicine. Acad Emerg Med. 2003; 10:251–60. Callahan CM, Unverzagt FW, Hui SL, Perkins AJ, Hendrie HC. Six-item Screener to identify cognitive impairment among potential subjects for clinical research. Med Care. 2002; 40: 771–81. Borson S, Scanlan J, Brush M, Vitaliano P, Dokmak A. The Mini-Cog: a cognitive ‘‘vital signs’’ measure for dementia screening in multi-lingual elderly. Int J Geriatr Psychiatry. 2000; 15:1021–7. Inouye SK, van Dyck CH, Alessi CA, et al. Clarifying confusion: the confusion assessment method. A new method for detection of delirium. Ann Intern Med. 1990; 113:941–8. Wilber ST, Blanda M, Gerson LW. Three screening tests for cognitive impairment in older emergency department patients [abstract]. Acad Emerg Med. 2004; 11:475. Pfeiffer E. A short portable mental status questionnaire for the assessment of organic brain deficit in elderly patients. J Am Geriatr Soc. 1975; 23:433–41. Katzman R, Brown T, Fuld P, Peck A, Schechter R, Schimmel H. Validation of a short Orientation-Memory-Concentration Test of cognitive impairment. Am J Psychiatry. 1983; 140: 734–9. Bernstein E. Functional assessment, mental status and case finding. In: Sanders AB (ed). Emergency Care of the Elder Person. St. Louis, MO: Beverly Cracom Publications, 1996, pp 97–116. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. ed 4, text revision. Washington, DC: American Psychiatric Association, 2000. Sanders AB. Missed delirium in older emergency department patients: a quality-of-care problem. Ann Emerg Med. 2002; 39: 338–41.
