C International Psychogeriatric Association 2011 International Psychogeriatrics (2011), 23:7, 1107–1115 doi:10.1017/S1041610210002450
Improving precision in the quantification of cognition using the Montreal Cognitive Assessment and the Mini-Mental State Examination .........................................................................................................................................................................................................................................................................................................................................................................
Lisa Koski,1,2,3 Haiqun Xie1 and Susanna Konsztowicz2 1
Division of Experimental Medicine, Department of Medicine, McGill University, Montreal, Quebec, Canada Department of Neurology and Neurosurgery, McGill University, Montreal, Quebec, Canada 3 Divisions of Geriatrics and of Clinical Epidemiology, Department of Medicine, McGill University and Research Institute of the McGill University Health Centre, Montreal, Quebec, Canada 2
ABSTRACT
Background: The Montreal Cognitive Assessment (MoCA) can be used to quantify cognitive ability in older persons undergoing screening for cognitive impairment. Although highly sensitive in detecting mild cognitive impairment, its measurement precision is weakest among persons with milder forms of impairment. We sought to overcome this limitation by integrating information from the Mini-Mental State Examination (MMSE) into the calculation of cognitive ability. Methods: Data from 185 geriatric outpatients screened for cognitive impairment with the MoCA and the MMSE were Rasch analyzed to evaluate the extent to which the MMSE items improved measurement precision in the upper ability ranges of the population. Results: Adding information from the MMSE resulted in a 13.8% (13.3–14.3%) reduction in measurement error, with significant improvements in all quartiles of patient ability. The addition of three-word repetition and recall, copy pentagons, repeat sentence, and write sentence improved measurement of cognition in the upper levels of ability. Conclusions: The algorithm presented here maximizes the yield of available clinical data while improving measurement of cognitive ability, which is particularly important for tracking changes over time in patients with milder levels of impairment. Key words: geriatric, cognitive impairment, dementia, measurement, Rasch, MoCA, MMSE
Introduction The Montreal Cognitive Assessment (MoCA) (Nasreddine et al., 2005) is a screening test for cognitive impairment that has been validated based on high accuracy for classifying older persons with dementia, mild cognitive impairment, or normal cognition (Lee et al., 2008; Wen et al., 2008; Duro et al., 2009; Fujiwara et al., 2010; Pendlebury et al., 2010). We recently demonstrated using Rasch analysis that the MoCA test items collectively assess cognitive ability as a unidimensional construct, such that the total score can be validly interpreted Correspondence should be addressed to: Dr. Lisa Koski, Royal Victoria Hospital, R4.74, 687 Pine Avenue West, Montreal, QC H3A 1A1, Canada. Phone: +1 514-934-1934, ext. 34420; Fax: +1 514-843-1734. Email:
[email protected]. Received 24 Sep 2010; revision requested 18 Nov 2010; revised version received 17 Dec 2010; accepted 19 Dec 2010. First published online 1 February 2011.
as a quantitative estimate of global cognitive ability (Koski et al., 2009). As a result, the MoCA test can be used not only for screening, but also for the purpose of tracking changes in global cognition over time within an individual or group. The results of our previous study (Koski et al., 2009) pointed to a potential limitation of the MoCA total score as an estimate of ability in persons within a certain range of ability: We refer to the relatively weaker measurement precision obtained for persons at the high end of the ability continuum. This limitation can be understood most clearly with reference to the graph shown in Figure 1a depicting the results of our previous study in 222 geriatric outpatients. The distribution of MoCA item difficulties are shown in the lower bars and the distribution of person abilities are shown in the upper bars. We note
1108
L. Koski et al.
Figure 1. (a) Distribution of items (bottom bars) and persons (top bars) by location on the cognitive ability scale measured by the MoCA alone in 222 patients (Koski et al., 2009). The lower bars in each graph show the difficulty of the items, which are centered on a mean of zero, with cognitive ability quantified in units of standard deviation from this mean. Easier items are to the left and harder items are to the right of the mean. The bottom y-axis indicates the number (left) and percentage (right) of items at each ability interval. The ability level of the patient sample is shown in the upper bars of each graph, with persons of lower ability to the left of the distribution and persons of higher ability to the right of the distribution. The top y-axis indicates the number (left) and percentage (right) of people located at each ability interval. The line plotted in the upper half of the figure is the information function, which is inversely related to the standard error and indicates precision of measurement obtained at each point along the cognitive ability continuum. Note that measurement precision is greatest between 0 and –1 logits, yet the average person targeted by the test is located at the opposite end of the distribution, at +1.2 logits. (b) Distribution of items (bottom bars) and persons (top bars) by location on the cognitive ability scale measured by the MoCA alone in the subset of 185 patients making up the current sample. (c) Distribution of items (bottom bars) and persons (top bars) by location on the cognitive ability scale measured by the MoCA and MMSE items combined in 185 patients. Diagonal-fill item bars represent items from the MoCA, and black item bars represent items from the MMSE. Note that this method of cognitive estimation allows for improved measurement precision and a wider range of possible scores, as detailed in the text.
Estimating cognitive ability in geriatric outpatients
that fewer items assess cognition at the extremes of the difficulty hierarchy, especially at the end representing higher cognitive ability (right side of graph). This decreases the capacity of the test to precisely quantify cognitive ability in these ranges. The information function (plotted line in upper part of graph) shows that measurement precision is greatest below the mean difficulty level of the items, dropping off precipitously in the range from 1 to 2 standard deviations above the mean difficulty level. This is precisely the midrange of the distribution of cognitive impairment in our patient sample, implying a limitation in the MoCA’s ability to detect change in the very people we would most like to target with cognitionmodifying interventions. Sensitivity to change is an essential attribute of an outcome measure when even modest improvements in cognition would be seen as a desirable outcome (e.g. brief interventions or clinical trials with short follow-up times). Improving the existing tools to increase precision in estimates of cognitive ability would produce better outcome measures with greater ability to detect change. Measurement precision is improved by adding to a test new items that fall within a difficulty range that is poorly covered by existing items. Unfortunately, adding to the length of a screening test is not an ideal approach due to time constraints. We noted that in the clinical context, the MiniMental State Examination (MMSE) and the MoCA are usually administered in series following the recommendations to administer a MoCA in a patient who presents with cognitive complaints but who obtains a score within normal limits on the less sensitive MMSE (Nasreddine et al., 2005). The MMSE as a whole is an easier test; however, it does contain some questions that we hypothesized would be helpful for filling gaps in the higher end of the cognitive difficulty spectrum, thus improving the precision with which we can measure cognition. Moreover, including MMSE responses that have already been collected in our estimate of cognitive ability would maximize the use of existing clinical data rather than adding to test burden. Thus the objective of the present study was to evaluate the extent to which precision in the quantification of cognition using the MoCA test would be improved by the addition of questions from the MMSE. Considering the desirability of minimizing assessment burden while optimizing measurement precision, we also evaluated a streamlined version of the dataset that included all of the items from the MoCA plus a select subset of items from the MMSE that best filled the gaps at the higher end of the cognitive continuum.
1109
Methods Design This study compared the measurement precision obtained by estimating global cognitive ability using (1) responses to the MoCA test alone (MoCA), (2) combined responses to both the MMSE and MoCA (MoCA+MMSE), and (3) responses to the MoCA plus a subset of responses to the MMSE (MoCA+5), within the same sample of geriatric outpatients. For the MoCA+5 dataset we added to the MoCA responses only the five MMSE items that contributed information at the high end of the ability spectrum. Sample and setting This study, approved by the McGill University Health Centre (MUHC) Research Ethics Board, was conducted as part of a clinical epidemiologic research project designed to characterize the nature and extent of cognitive impairment observed in individuals attending geriatric assessment clinics at the MUHC. Following standard practice guidelines for the outpatient clinics, the MMSE is administered by clinic nurses or geriatricians to all patients referred either to the specialty Geriatric Cognitive Disorders Clinics or for comprehensive geriatric assessment where cognitive impairment is suspected. As recommended (Nasreddine et al., 2005), those patients who score at least 20/30 on the MMSE are subsequently administered the MoCA in either English or French, depending on the patient’s mother tongue and preference. The MMSE allows the option of administering either the serial subtraction of 7s or the spelling of “world” backwards. Because of the potential overlap with the MoCA 7s subtraction item, our clinics administer “world” backwards on the MMSE. All data were coded in a clinical database. Data collection For this study, data were extracted from the clinical database for all patients to whom the MMSE and MoCA were administered on the same day between 1 February 2006 and 11 November 2007. Because of the additional requirement that both tests had to be administered during the same clinic visit, the resulting sample size for this study is 185 patients, representing a subset of the 222 patients analyzed in our previous study (Koski et al., 2009). The following information was extracted: clinic site (Royal Victoria Hospital or Montreal General Hospital), sex, years of education, language of test (English or French), date of birth, date of test, diagnosis at time of test, and itemized test scores for the MoCA and the MMSE. Years of education
1110
L. Koski et al.
were coded as low (elementary: 0–8 years), medium (secondary: 9–12 years), or high (post-secondary: 13+ years). Age was coded as ≤ 80 years and >81 years old. Each patient had a diagnosis recorded by the geriatrician at the time of the visit. The diagnosis was made according to DSM-IV criteria for dementia (Amercian Psychiatric Association, 2000) and revised criteria for mild cognitive impairment (MCI; Petersen, 2004) based on a clinical evaluation that included a full chart review, history, physical examination, assessment of basic and instrumental activities of daily living, blood tests, brain CT, as well as the results of the cognitive screening tests. The recorded diagnosis was classified for the purposes of this study as Dementia (any type), Mild Cognitive Impairment without dementia, and Normal (cognition within normal limits). Cases in which a definitive determination between MCI and dementia had not yet been made by the clinician were classified as Cognitive Impairment Unspecified. Cases in which cognitive impairment was attributed to a primary psychiatric impairment were classified as Psychiatric. The remaining cases were those for which no diagnostic information relevant to cognition was recorded. All of the individual items from the MoCA were scored dichotomously (0 or 1) except serial subtraction of 7s, which was coded as a single 4option item with a range of 0 to 3 points. Most of the MMSE items were scored dichotomously, excepting 3-word repetition, 3-word recall and 3step command (all 4 options, range: 0 to 3), and “world” backward (6 options, range: 0 to 5). The questions evaluating orientation to year, month, date, day of week, place, and city are identical for the two tests so only the results obtained during the MoCA assessment were retained in the dataset. Statistical analysis Frequency counts, means, standard deviations, and ranges were used to characterize the patient sample on demographic and clinical variables, with crosstabulation to assess the distribution of patients on demographic variables. The use of a total score to quantify cognitive ability requires that the scale is unidimensional. The validity of this assumption was verified for each of the three datasets using RUMM2020 software (RUMM Laboratory Pty. Ltd.), according to procedures described previously (Koski et al., 2009). To address our primary objective, we compared the distribution of items within the difficulty hierarchy for the MoCA dataset and the MoCA+MMSE dataset, identifying locations in the cognitive continuum for which the MMSE provided added precision. The five
MMSE items that contributed to the high end of the difficulty hierarchy were added to the MoCA data to build the MoCA+5 dataset. For both the MoCA+MMSE and the MoCA+5 datasets, the change in the standard error of the cognitive ability estimate compared with the MoCA dataset was calculated for each patient and averaged across the sample to yield an estimate of improvement in measurement precision. Paired t-tests were used to evaluate the significance of the change, and mean change scores are reported with 95% confidence intervals in parentheses. To evaluate the improvement in measurement precision obtained for different cognitive ability levels, the patient sample was divided into quartiles to evaluate the improvement in measurement precision obtained with the MoCA+MMSE dataset and with the MoCA+5 dataset. Finally, we also compared measurement precision obtained with the three methods in different diagnostic subgroups.
Results Description of the sample Of the 185 patients extracted from the database, 62% were more than 80 years of age and just over half the sample (55%) was composed of women. Educational level was less than 9 years for 17%, 9– 12 years for 30%, more than 12 years for 39%, and missing for 14% of the sample. Table 1 shows the distribution of sex by age and by education. Crosstabulation of sex by age and by education showed that women tended to be older (Pearson χ 2 = 8.8, df = 1, p = 0.003) and less well educated (χ 2 = 11.7, df = 3, p = 0.008) than the men. Women comprised 65% of the sample with an age of 80 or more years but only 41% of those below 80 years of age. Women represented 78% of those with fewer than 9 years of formal education, whereas the distribution of women and men was more equal for higher levels of education. The test language was English for 71% of the patients, with the remaining 29% tested in French. Test language showed a similar distribution across age, sex, and education. Table 1. Demographic frequency distributions of the patient sample WOMEN
MEN
T O TA L
.........................................................................................................................................................
Age (years) Education (years)
