Diagnostic Accuracy of the RBANS in Mild ... - Semantic Scholar

Archives of Clinical Neuropsychology 25 (2010) 429–441

Diagnostic Accuracy of the RBANS in Mild Cognitive Impairment: Limitations on Assessing Milder Impairments Kevin Duff 1,*, Valerie L. Hobson 2, Leigh J. Beglinger 3, Sid E. O’Bryant 4 1

Department of Neurology, University of Utah School of Medicine, Salt Lake City, UT, USA 2 Department of Psychology, Texas Tech University, Lubbock, TX, USA 3 Department of Psychiatry, University of Iowa, Iowa City, IA, USA 4 F. Marie Hall Institute for Rural and Community Health and Department of Neurology, Texas Tech University Health Sciences Center, Lubbock, TX, USA *Corresponding author at: Center for Alzheimer’s Care, Imaging and Research, Department of Neurology, University of Utah, 650 Komas Drive #106-A, Salt Lake City, UT 84108, USA. Tel: +1-801-585-9983; fax: +1-801-581-2483. E-mail address: [email protected] (K. Duff). Accepted 21 May 2010

Abstract The Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) has demonstrated adequate sensitivity in detecting cognitive impairment in a number of neuropsychiatric conditions, including Alzheimer’s disease. However, its ability to detect milder cognitive deficits in the elderly has not been examined. The current study examined the clinical utility of the RBANS by comparing two groups: Patients with Mild Cognitive Impairment (MCI; n ¼ 72) and cognitively intact peers (n ¼ 71). Significant differences were observed on the RBANS Total score, 3 of the 5 Indexes, and 6 of the 12 subtests, with individuals with MCI performing worse than the comparison participants. Specificity was very good, but sensitivity ranged from poor to moderate. Areas under the receiver operating characteristic curves for the RBANS Immediate and Delayed Memory Indexes and the Total Scale score were adequate. Although significant differences were observed between groups and the areas under the curves were adequate, the lower sensitivity values of the RBANS suggests that caution should be used when diagnosing conditions such as MCI. Keywords: Mild Cognitive Impairment; Diagnostic accuracy; Repeatable Battery for the Assessment of Neuropsychological Status

Introduction Mild Cognitive Impairment (MCI) is viewed as a transitional stage between healthy aging and dementia, and it is defined as cognitive decline greater than expected for an individual’s age and the education level but that does not notably interfere with activities of daily life (Petersen et al., 1999; Winblad et al., 2004). Individuals with this cognitive profile are at greater risk for converting to dementia across time than those without MCI (Petersen et al., 2001). Early detection of MCI may enable individuals to benefit from interventions that could potentially slow the course of the disease. Since there has been a dramatic rise in the number of clinical trials in MCI (Petrella et al., 2009; Raschetti, Albanese, Vanacore, & Maggini, 2007; Salloway et al., 2004; Saykin et al., 2004; Winblad et al., 2008), there is a growing need for measures that are both brief and sensitive in identifying this pattern of cognitive decline. One such brief battery that is receiving increased attention in both clinical and research settings is the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS; Randolph, 1998). The RBANS, which was initially developed as an assessment tool for dementia, has been validated in community-dwelling “normal” elderly samples (Duff et al., 2003, 2004, 2005; Patton et al., 2003) and in a number of studies of dementia and MCI (Juhasz, Kemeny, Linka, Santha, & Bartko, 2003; Kotani et al., 2006). The RBANS has also been related to functional limitations in patients with dementia and MCI (Badenes Guia, Casas Hernanz, Cejudo Bolivar, & Aguilar Barbera, 2008; Freilich & Hyer, 2007; Hobson, Hall, Humphreys-Clark, Schrimsher, & O’Bryant, 2010). More recently, the diagnostic accuracy of the RBANS has been shown to adequately detect cognitive impairment associated with Alzheimer’s disease (AD; Duff, # The Author 2010. Published by Oxford University Press. All rights reserved. For permissions, please e-mail: [email protected]. doi:10.1093/arclin/acq045 Advance Access publication on 21 June 2010

430

K. Duff et al. / Archives of Clinical Neuropsychology 25 (2010) 429–441

Humphreys Clark, et al., 2008). Although several studies have used the RBANS as a tool to examine cognitive dysfunction, there remains little information regarding the diagnostic accuracy of the RBANS and its ability to detect milder deficits in cognition in the elderly. According to the Standards for Reporting of Diagnostic Accuracy (STARD) work group (Bossuyt et al., 2003), “the amount of agreement between the results from the Index test and those from the reference standard” indicates the accuracy of that test. The STARD group identified several estimates of diagnostic accuracy, which include sensitivity, specificity, likelihood ratios, diagnostic odds ratios, and areas under receiver operating characteristic (ROC) curves. The areas under a curve (AUC) of an ROC represent the probability that normal and abnormal test scores will be correctly classified as the disease state (Hanley & McNeil, 1982). Further, ROC curves can provide a single estimate of diagnostic accuracy that does not require that ordinal and continuous test scores be simplified and therefore allow for the full range of cognitive scores to be examined (Lett, Hanley, & Smith, 1995). The purpose of the current study was to evaluate the diagnostic accuracy of the RBANS in detecting cognitive deficits associated with MCI. It was hypothesized that the MCI participants would perform more poorly than matched peers on most RBANS Index and subtest scores, especially on those that assess learning and memory, as our sample was primarily amnestic MCI (single- or multidomain). To provide information necessary for advancing clinical trials in MCI, sensitivity, specificity, odds ratios, and ROC data were calculated using various cutoff points (1, 1.5, and 2 SD below the mean) for RBANS scores. For example, 1 SD below the mean can represent the lower end of normal (16th percentile); 1.5 SD below the mean has been frequently cited in the literature identifying impairment in MCI (Petersen et al., 2001); and 2 SD below the mean more clearly falls into the impaired range (2nd percentile). Materials and Methods Participants One hundred and sixty-eight community-dwelling older adults participated in the current study, and these participants have been described previously (Duff, Beglinger, et al., 2008). Briefly, these individuals were recruited from senior centers and independent living facilities to prospectively study cognitive changes in older adults. Individuals were screened over the telephone with brief cognitive tasks to increase the chances that they would be classified as MCI or intact on in-person testing (Lines, McCarroll, Lipton, & Block, 2003). Their mean age was 78.7 (7.7) years and their mean education was 15.4 (2.5) years. Most were women (81%) and all were Caucasian. Premorbid intellect at baseline was average (Wide Range Achievement Test-3 [WRAT-3] Reading: M ¼ 107.8, SD ¼ 6.2). Procedures All participants provided informed consent prior to participation, and all procedures were approved by the local Institutional Review Board. During an in-person screening visit, all participants completed the WRAT-3 Reading subtest, 30-item Geriatric Depression Scale (GDS), and RBANS (Form A). During a subsequent baseline visit, all participants completed the Brief Visuospatial Memory Test-Revised (BVMT-R), Hopkins Verbal Learning Test-Revised (HVLT-R), Controlled Oral Word Association Test (COWAT), animal fluency, Trail Making Test Parts A and B (TMT-A and TMT-B), and Symbol Digit Modalities Test (SDMT). In 92% of all participants, the screening and baseline visits occurred on the same day. In the other cases, less than a week separate these two visits. Using results from the baseline assessment, individuals were classified as cognitively intact or MCI using a modified version of existing criteria (Petersen et al., 1999). These MCI criteria incorporate subjective and objective information. Subjectively, participants and/or a collateral source were asked if the participant had memory problems (i.e., endorsed as yes/no) or any functional impairments (e.g., assistance needed with managing money, taking medications, driving). Participants were also asked about exclusionary conditions, such as major neurological or psychiatric conditions, use of medications known to adversely affect cognition, or had uncontrolled medical conditions that would adversely affect cognition. Objective cognitive deficits were examined for four cognitive domains: (a) memory (mean age-corrected delayed recall trials of the HVLT-R and BVMT-R), (b) executive functioning (age- and education-corrected time to complete TMT-B), (c) language (age- and education-corrected score on animal fluency), and (d) attention/processing speed (age- and educationcorrected score on SDMT). An objective cognitive deficit was defined as a cognitive domain score that fell at or below the 7th percentile relative to a premorbid intellectual estimate (WRAT-3 Reading). Several recent studies have suggested that intellect (either current or premorbid) should be considered when assessing cognitive deficits, especially in older adults and those with suspected MCI or dementia (Brooks, Iverson, Feldman, & Holdnack, 2009; Brooks, Iverson, Holdnack, & Feldman,


431

2008; Horton, 1999; Steinberg, Bieliauskas, Smith, & Ivnik, 2005a, 2005b; Steinberg, Bieliauskas, Smith, Ivnik, & Malec, 2005; Steinberg, Bieliauskas, Smith, Langellotti, & Ivnik, 2005). When two scores were included (e.g., delayed recall trials on the HVLT-R and BVMT-R), in some cases, a “severely impaired score” (e.g., ,1st percentile) could be averaged with a “borderline score” (e.g., 9th percentile) to yield a mean score that fell below the cutoff (e.g., mean score ¼ 3rd percentile). Conversely, a “mildly impaired score” (e.g., 5th percentile) could be averaged with a “borderline score” (e.g., 12th percentile) to yield a mean score that fell above the cutoff (e.g., mean score ¼ 8th percentile). Unfortunately, two scores were only available for the memory domain, and all non-memory domain scores consisted of a single score. The cutoff of the 7th percentile is 1.5 SD below the mean, which is a typical demarcation point for cognitive deficits in MCI. All data were reviewed by two neuropsychologists (KD and LJB), and all participants were classified as intact or MCI. To be classified as intact, subjective memory problems may or may not have been reported, functional impairments were denied, and no objective cognitive deficits could be present. Seventy-one individuals were classified as intact. To be classified as MCI, participants and/or their collateral source had to report memory problems in the participant and deny functional impairments and objective cognitive deficits were present. Seventy-two individuals were classified as MCI. Consistent with recent views of MCI (Petersen et al., 2009), these participants were classified as having either single- or multidomain MCI, and the following subtypes were identified: Single domain amnestic (n ¼ 49), single domain executive functioning (n ¼ 15), single domain language (n ¼ 6), single domain attention/processing speed (n ¼ 2), multidomain amnestic and executive functioning (n ¼ 9), multidomain amnestic and language (n ¼ 4), multidomain amnestic, executive functioning, and language (n ¼ 2), multidomain amnestic, executive functioning, and attention (n ¼ 6), multidomain amnestic, executive functioning, language, and attention (n ¼ 2), and multidomain executive functioning and language (n ¼ 2). No one was classified as demented (i.e., both impaired memory and other cognitive domains and functional impairments). Given the sample sizes of the MCI subtypes and the relatively greater interest in the field in amnestic MCI, it was decided to combine all amnestic subtypes (single- and multidomains) into one group (n ¼ 72) and exclude the other MCI cases from further analyses (n ¼ 25). All classifications were made following the baseline visit, so examiners were “blinded” to classification at the screening and baseline visits. To avoid circularity, only baseline assessment results were used in the classification of participants, and these results were independent of the screening visit (i.e., RBANS). Demographic and baseline assessment scores for the intact and amnestic MCI cases are presented in Table 1. Measures The RBANS (Randolph, 1998) is a brief, individually administered test measuring attention, language, visuospatial/constructional abilities, and immediate and delayed memory. It consists of 12 subtests, which yield five Index scores and a Total Scale score. Normative data provided age- and education-corrected Index and subtest scores (Duff et al., 2003), and Table 1. Demographic and baseline descriptive data for patients with MCI and comparison participants

N Age (years) Gender (M/F) Education (years) WRAT-3 Reading GDS BVMT-R Total Recall BVMT-R Delayed Recall HVLT-R Total Recall HVLT-R Delayed Recall COWAT Animals TMT-A TMT-B SDMT

MCI

Cognitively Intact

F, p

72 82.1 (6.9) 17/55 15.1 (2.3) 109.0 (5.3) 4.6 (3.3) 72.1 (12.9) 69.2 (14.0) 90.7 (14.9) 78.9 (16.1) 10.5 (3.2) 11.0 (3.3) 9.4 (3.0) 9.2 (3.1) 10.1 (2.6)

71 76.3 (6.9) 9/62 15.2 (2.6) 105.9 (6.8) 3.6 (3.4) 93.1 (15.3) 99.2 (14.3) 107.9 (11.3) 103.5 (12.0) 10.0 (2.9) 12.9 (2.7) 10.8 (2.7) 10.6 (2.3) 11.0 (1.7)

t ¼ 25.0, ,.001 x 2 ¼ 2.9, .09 t ¼ 0.2, .81 t ¼ 23.1, .002 t ¼ 21.9, .06 55.5, ,.001 131.7, ,.001 43.1, ,.001 99.3, ,.001 0.1, .74 12.0, .001 5.4, .02 7.6, .007 5.7, .02

Notes: Means and standard deviations (in parentheses) are reported for all variables except Gender. WRAT-3, BVMT-R, and HVLT-R scores are age-corrected standard scores (M ¼ 100, SD ¼ 15). COWAT, Animals, TMT, and SDMT scores are age- and education-corrected scaled scores (M ¼ 10, SD ¼ 3). MCI ¼ Mild Cognitive Impairment; WRAT-3 ¼ Wide Range Achievement Test-3 Reading subtest; GDS ¼ Geriatric Depression Scale; BVMT-R ¼ Brief Visuospatial Memory Test-Revised; HVLT-R ¼ Hopkins Verbal Learning Test-Revised; COWAT ¼ Controlled Oral Word Association Test; TMT ¼ Trail Making Test; SDMT ¼ Symbol Digit Modalities Test. F-values come from the Between-Subjects univariate tests following an overall MANCOVA that covaries age and WRAT-3 scores.

432


these were used in the following analyses. Index scores have a mean of 100 and standard deviation of 15, and subtest scores have a mean of 10 and standard deviation of 3. All subtests were administered and scored as defined in the manual, with the exception of the Figure Copy and Figure Recall, which are more thoroughly described elsewhere (Duff et al., 2007). Briefly, the modified scoring criteria of the figure followed less stringent adherence to the manual’s scoring criteria. Examples of these modifications include: Less exact measurements, emphasizing the majority of correct elements, and discouraging the use of a ruler or protractor for measuring elements. Statistical Analyses Independent t-tests and x 2 analyses were calculated to compare the two groups (intact and MCI) on age, education, gender, GDS, and WRAT-3 Reading scores. If any of these demographic, depression, or premorbid intellect variables were significantly different between the groups, then they would be used as covariates in the following analyses. To begin testing the primary aims of the paper, an ANCOVA was used to compare the two groups on the RBANS Total Scale score; two MANCOVAs were conducted to compare scores from the two groups on the 5 Indexes and 12 subtests of the RBANS. An a level of 0.05 was maintained to test the three primary analyses. Sensitivity, specificity, positive predictive power, and negative predictive power at various cutoff points were calculated as outlined by Kraemer (1992). Diagnostic accuracy was estimated using ROC curves and the calculation of AUC via non-parametric analyses using SPSS 15.0. To equate these latter RBANS analyses to those presented in ANCOVA and MANCOVAs (i.e., correcting for WRAT-3 Reading scores), we corrected each RBANS score (Indexes and subtests) by each participant’s WRAT-3 Reading score before calculating sensitivity and specificity and ROC curves. Results As can be seen in Table 1, participants classified as amnestic MCI (single- or multidomain) and cognitively intact were comparable in education (p ¼ .81), gender (p ¼ .09), and GDS scores (p ¼ .06). They were different, however, in age (p , .001) and WRAT-3 Reading scores (p ¼ .002), so these variables were used as covariates in the remaining analyses. Baseline cognitive test scores (used in the classification of subjects) are also presented in Table 1. RBANS Index and subtest scores (from the screening visit) are presented in Table 2. Despite using age and WRAT-3 as covariates, there were significant differences between the groups on the RBANS Total Scale score—F(1,137) ¼ 24.88, Table 2. RBANS descriptive data for patients with MCI and comparison participants

RBANS Indexes Total Scale Immediate Memory Visuospatial Constructional Language Attention Delayed Memory RBANS subtests List Learning Story Memory Figure Copy Line Orientation Picture Naming Semantic Fluency Digit Span Coding List Recall List Recognition Story Recall Figure Recall

MCI

Cognitively intact

F-value, p-value

Partial h2

92.4 (9.1) 97.9 (10.8) 98.3 (12.4) 91.2 (9.0) 100.4 (11.6) 92.4 (9.5)

101.5 (10.2) 107.1 (12.8) 101.7 (10.9) 95.5 (8.8) 103.1 (14.2) 101.4 (9.2)

24.9, ,.001 16.9, ,.001 4.0, .05 8.0, .005 1.5, .23 22.4, ,.001

0.15 0.11 0.03 0.06 0.01 0.14

9.6 (1.6) 10.4 (2.4) 9.8 (2.6) 10.8 (2.5) 9.5 (1.7) 10.7 (2.8) 9.3 (1.9) 11.6 (2.7) 9.5 (1.9) 9.5 (2.5) 10.2 (2.0) 9.1 (2.3)

11.8 (2.5) 11.1 (2.6) 10.2 (2.4) 11.5 (2.5) 9.3 (1.4) 12.7 (2.6) 8.4 (2.3) 13.2 (2.5) 11.3 (2.4) 10.6 (1.7) 11.3 (2.2) 11.1 (2.2)

27.9, ,.001 2.9, .09 1.6, .21 3.5, .06 0.0, .87 13.2, ,.001 2.9, .09 9.1, .003 21.9, ,.001 2.6, .11 8.0, .006 18.6, ,.001

0.17 0.02 0.01 0.02 0.00 0.09 0.02 0.06 0.14 0.02 0.06 0.12

Notes: Means and standard deviations (in parentheses) are reported for all RBANS scores. RBANS Indexes are age- and education-corrected standard scores (M ¼ 100, SD ¼ 15), and subtests are age- and education-corrected scaled scores (M=10, SD=3). MCI ¼ Mild Cognitive Impairment; RBANS ¼ Repeatable Battery for the Assessment of Neuropsychological Status. Partial h2 are based on the results of ANCOVA and MANCOVAs, as described in the “Materials and Methods” section.


433

p , .001, partial h2 ¼ 0.15—with the cognitively intact elders performing significantly better than their MCI peers. The overall MANCOVA examining the five RBANS Indexes was also statistically significant, F(5,133) ¼ 5.37, p , .001, partial h2 ¼ 0.17. Follow-up univariate statistics revealed that three Indexes were significantly different between the groups (Immediate Memory, Language, and Delayed Memory). The overall MANCOVA examining the 12 RBANS subtests was also statistically significant—F(12,126) ¼ 4.43, p , .001, partial h2 ¼ 0.30—with the following subtests contributing to this effect: List Learning, Semantic Fluency, Coding, List Recall, Story Recall, and Figure Recall. Sensitivity and specificity at cutoff scores of 21.0, 21.5, and 22.0 SD below the mean of the cognitively intact comparison group for all Index and individual subtest scores are presented in Table 3. Additionally, positive and negative predictive powers for these same cutoff scores are presented in Table 4. These cutoff scores would be equivalent to standard scores (i.e., M ¼ 100, SD ¼ 15) of 85, 77, and 70, respectively. ROC curves for the two statistically significant Indexes (Immediate Memory and Delayed Memory) and the Total Scale score are presented in Fig. 1. The AUC for each of the Index scores and Total Score were as follows: Immediate Memory ¼ 0.76, Visuospatial Constructional ¼ 0.65, Language ¼ 0.71, Attention ¼ 0.62, Delayed Memory ¼ 0.78, and Total Score ¼ 0.78. Discussion The current study sought to evaluate the diagnostic accuracy of the RBANS in detecting milder cognitive deficits, such as those associated with amnestic MCI. The results of this study provide equivocal support for the RBANS in these mildly impaired individuals. On the one hand, older adults classified as amnestic MCI (either single- or multidomain) scored significantly below their cognitively intact peers on the Total score, 3 of the 5 Indexes, and 6 of the 12 subtests. Additionally, the AUC from the ROC analyses suggested adequate separation between the two groups in the current study on measures of learning and memory. Finally, specificity values for all memory-related subtests and Indexes were 0.82 or better and negative predictive power was similarly high. On the other hand, sensitivity values and positive predictive powers were quite poor for these memory subtests and Indexes on the RBANS (with the Delayed Memory Index and Total Scale having the best combination of sensitivity and specificity at the 21.0 SD cutoff). Although this is not an ideal situation, mixed results in assessing the diagnostic accuracy of a test is not uncommon in medicine. For example, in a study comparing several diagnostic criteria for dementia (including NINCDS-ADRDA criteria for AD) to neuropathology, the diagnostic criteria had low sensitivity and high specificity (Holmes, Cairns, Lantos, & Mann, 1999). Regardless, caution should be exercised when using the RBANS in cases of possible amnestic MCI. The RBANS has already demonstrated strong diagnostic accuracy in AD. Two studies (Duff, Humphreys Clark, et al., 2008; Randolph, Tierney, Mohr, & Chase, 1998) found significant differences between patients with AD and healthy elders with nearly 40 standard score points separating these two groups on the Delayed Memory Index. Since amnestic MCI is suspected to be the prodrome of AD, it was expected that the RBANS would again separate individuals with MCI from intact peers, at least on the memory Indexes of the RBANS. Smaller, but still statistically significant, differences were observed in the current study (e.g., 9.0 standard score points on the Delayed Memory Index). However, the sensitivity of the RBANS was very different between these two studies (Delayed Memory Index at 21.0 SD: Duff et al. ¼ 0.97, current study ¼ 0.56). Although there are similarities between Duff and colleagues and the current study, differences also exist. Inherently, the AD patients from Duff and colleagues were more impaired than the MCI patients in the present study (mean Total score: 64.5 vs. 92.4). The present MCI sample was larger, older, and had more women than Duff and colleagues’ AD sample. When examining the comparison group in these two studies, our study’s comparison group was larger, slightly younger, and had more women than Duff and colleagues. Although the demographic differences between the samples probably explains some of the differences in diagnostic accuracy, we suspect that the severity of cognitive impairments in these two samples explains most of the difference in diagnostic accuracy (i.e., very large RBANS differences between AD and controls lead to stronger diagnostic accuracy than the modest RBANS differences between MCI and controls). In the AD sample of Duff and colleagues (2008), the participants with dementia fell significantly below comparison subjects on all 5 Index scores and all 12 subtest scores. In the current study, significant differences were observed between patients diagnosed with amnestic MCI and comparison elders on only three Indexes (Immediate Memory, Language, and Delayed Memory) and only six subtests (List Learning, Semantic Fluency, Coding, List Recall, Story Recall, and Figure Recall). These differences are largely expected given the pathological conditions examined in each study. Since the current subjects were classified as amnestic MCI (i.e., prodrome AD), they should primarily have impairments of memory, which reflects 2 of the 5 Indexes and 6 of the 12 subtests (i.e., non-memory tasks should not necessarily be affected). However, since our MCI participants included multidomain subtypes (i.e., amnestic plus non-memory deficits), some non-memory differences were expected and found. The other identified cognitive differences in the MCI sample were on measures of semantic fluency and processing speed, and both of these types of tasks have been reported to fall below expectations in cases of

434


Table 3. Diagnostic utility information of RBANS Indexes and subtests

List Learning 21.0 SD 21.5 SD 22.0 SD Story Memory 21.0 SD 21.5 SD 22.0 SD Immediate Memory Index 21.0 SD 21.5 SD 22.0 SD Figure Copy 21.0 SD 21.5 SD 22.0 SD Line Orientation 21.0 SD 21.5 SD 22.0 SD Visuospatial Constructional Index 21.0 SD 21.5 SD 22.0 SD Picture Naming 21.0 SD 21.5 SD 22.0 SD Semantic Fluency 21.0 SD 21.5 SD 22.0 SD Language Index 21.0 SD 21.5 SD 22.0 SD Digit Span 21.0 SD 21.5 SD 22.0 SD Coding 21.0 SD 21.5 SD 22.0 SD Attention Index 21.0 SD 21.5 SD 22.0 SD List Recall 21.0 SD 21.5 SD 22.0 SD List Recognition 21.0 SD 21.5 SD 22.0 SD Story Recall 21.0 SD 21.5 SD 22.0 SD

SN (95% CI)

SP (95% CI)

0.333 (0.227– 0.454) 0.139 (0.069– 0.241) 0.042 (0.009– 0.117)

0.901 (0.807– 0.959) 0.944 (0.862– 0.984) 0.986 (0.924– 1.000)

0.250 (0.155– 0.366) 0.153 (0.079– 0.257) 0.014 (0.000– 0.075)

0.817 (0.707– 0.899) 0.958 (0.881– 0.991) 0.986 (0.924– 1.000)

0.347 (0.239– 0.469) 0.153 (0.079– 0.257) 0.069 (0.023– 0.155)

0.845 (0.740– 0.920) 0.916 (0.825– 0.968) 0.986 (0.924– 1.000)

0.409 (0.293– 0.532) 0.197 (0.112– 0.309) 0.099 (0.041– 0.193)

0.747 (0.629– 0.842) 0.887 (0.790– 0.950) 0.958 (0.881– 0.991)

0.236 (0.144– 0.351) 0.097 (0.040– 0.190) 0.028 (0.003– 0.097)

0.887 (0.790– 0.950) 0.930 (0.843– 0.977) 0.986 (0.924– 1.000)

0.451 (0.332– 0.573) 0.183 (0.101– 0.293) 0.056 (0.016– 0.138)

0.817 (0.707– 0.899) 0.930 (0.843– 0.977) 0.986 (0.924– 1.000)

0.389 (0.276– 0.511) 0.097 (0.040– 0.190) 0.056 (0.015– 0.136)

0.634 (0.511– 0.745) 0.916 (0.825– 0.968) 0.986 (0.924– 1.000)

0.222 (0.133– 0.336) 0.097 (0.040– 0.190) 0.056 (0.015– 0.136)

0.944 (0.862– 0.984) 0.972 (0.902– 1.000) 0.986 (0.924– 1.000)

0.694 (0.575– 0.798) 0.403 (0.289– 0.525) 0.097 (0.040– 0.190)

0.620 (0.497– 0.732) 0.873 (0.773– 0.940) 0.972 (0.902– 0.997)

0.417 (0.302– 0.539) 0.222 (0.133– 0.336) 0.069 (0.023– 0.155)

0.507 (0.386– 0.628) 0.775 (0.660– 0.865) 0.901 (0.807– 0.959)

0.153 (0.079– 0.257) 0.111 (0.049– 0.207) 0.028 (0.003– 0.097)

0.972 (0.902– 0.997) 1.000 (0.962– 1.000) 1.000 (0.962– 1.000)

0.264 (0.167– 0.381) 0.181 (0.100– 0.289) 0.014 (0.000– 0.075)

0.817 (0.707– 0.899) 0.873 (0.773– 0.940) 0.972 (0.902– 0.997)

0.403 (0.289– 0.525) 0.167 (0.089– 0.273) 0.042 (0.009– 0.117)

0.859 (0.756– 0.930) 0.944 (0.862– 0.984) 0.972 (0.902– 0.997)

0.417 (0.302– 0.539) 0.069 (0.023– 0.155) 0.014 (0.000– 0.075)

0.845 (0.740– 0.920) 0.990 (0.943– 1.000) 1.000 (0.962– 1.000)

0.264 (0.167– 0.381) 0.083 (0.031– 0.173) 0.028 (0.003– 0.097)

0.884 (0.802– 0.941) 0.947 (0.881– 0.983) 0.990 (0.943– 1.000) (continued on next page)


435

Table 3. (continued)

Figure Recall 21.0 SD 21.5 SD 22.0 SD Delayed Memory Index 21.0 SD 21.5 SD 22.0 SD Total Scale 21.0 SD 21.5 SD 22.0 SD

SN (95% CI)

SP (95% CI)

0.403 (0.289– 0.525) 0.236 (0.144– 0.351) 0.125 (0.059– 0.224)

0.875 (0.792– 0.934) 0.948 (0.883– 0.983) 0.990 (0.943– 1.000)

0.556 (0.434– 0.673) 0.292 (0.191– 0.411) 0.111 (0.049– 0.207)

0.829 (0.720– 0.908) 0.929 (0.841– 0.976) 0.986 (0.923– 1.000)

0.549 (0.427– 0.668) 0.268 (0.169– 0.386) 0.099 (0.041– 0.193)

0.800 (0.687– 0.886) 0.943 (0.860– 0.984) 0.968 (0.923– 1.000)

Notes: SN ¼ sensitivity; SP ¼ specificity. Standard scores equivalents (i.e., M ¼ 100, SD ¼ 15) of 21, 21.5, and 22 SD would be 85, 77, and 70, respectively. All scores are adjusted for age, education, and WRAT-3 Reading.

MCI (Cooper, Lacritz, Weiner, Rosenberg, & Cullum, 2004; Economou, Papageorgiou, Karageorgiou, & Vassilopoulos, 2007). In a related vein, the RBANS Indexes with the two best sensitivity values at the 21.0 SD cutoff in the current study were the Delayed Memory Index and the Language Index. Sensitivity, specificity, positive and negative predictive powers, ROC curves, and AUC estimates are routinely used in medicine to evaluate clinical measures (Nash et al., 2006; Schmidt et al., 2006; Stephan et al., 2006). Unfortunately, despite strong specificity, none of the RBANS Indexes or subtests achieved sensitivity that would be considered acceptable for clinical diagnostic purposes when either a 1, 1.5, or 2 SD cutoff was implemented. Sensitivity refers to the proportion of actual positive cases that are correctly identified as such (e.g., the percentage of MCI cases who are identified as having MCI). Specificity, however, refers to the proportion of negative cases that are correctly identified as such (e.g., the percentage of controls who are identified as not having MCI). Although an ideal diagnostic test would have an optimal balance of sensitivity and specificity, the current study did not find that balance in the RBANS. The high specificity values suggest that the RBANS can be used to identify negative cases (e.g., those without MCI), which still could be useful for clinical trials by excluding inappropriate subjects. However, the generally low sensitivity suggests that the RBANS does not accurately identify the cases of interest (e.g., those with MCI). Despite these less than optimal test characteristic values, there is some movement in them as the cutoff changes from 21.0 to 22.0 SD in Table 3. For example, as the cutoff on the Total Scale score shifts from 21.0 to 22.0 SD, sensitivity decreases (0.549 to 0.099) and specificity increases (0.800 to 0.968). Although these shifts are somewhat expected, they might provide avenues for fine tuning of the RBANS diagnostic accuracy. It is possible that the low sensitivity suggests that our cases of amnestic MCI do not really have this condition. In fact, the RBANS Immediate and Delayed Memory Indexes in this group averaged 97.9 and 92.4, respectively. Although these two Indexes do fall approximately 1 SD below premorbid intellect, these two Memory Indexes still fall in the average range. It should be reiterated that all subjects in the current study were classified by scores on two other memory tests, the BVMT-R and the HVLT-R, to avoid circularity with the RBANS. The scores from these two measures tended to be more impaired, especially for the delayed recall measures (BVMT-R: Total Recall ¼ 72.1, Delayed Recall ¼ 69.2; HVLT-R: Total Recall ¼ 90.7, Delayed Recall ¼ 78.9; effect sizes [Cohen’s d] between intact and MCI for Delayed Recall: BVMT-R ¼ 2.2, HVLT-R ¼ 1.5). Furthermore, although there were some statistical differences between the MCI and intact groups on nonmemory measures (e.g., COWAT, Animals, TMT, and SDMT), the MCI group generally performed in the average range on these measures (e.g., scores ranged from 39th to 63rd percentiles). On the basis of the results of these non-RBANS measures, our amnestic MCI subjects appear to have this condition, at least psychometrically. However, as noted in the “Materials and Methods” section, we did take some liberties with our application of the Petersen criteria for MCI (e.g., averaging two delayed recall measures, memory discrepancies from premorbid intellect, reliance on a single baseline assessment to determine MCI status), and these may have affected the classification of our sample, the resulting RBANS test characteristics, and the generalization of our findings to other studies. In one additional study that examined the RBANS in MCI, Hobson et al. (2010) found considerably lower scores on the Delayed Memory Index than in the current sample (77.0 vs. 92.8, respectively). However, there were notable differences between these two samples (e.g., Hobson’s sample was recruited from a Memory Disorder Clinic vs. community-dwelling sample; Hobson’s sample used age-corrected scores vs. age- and education-corrected scores; Hobson’s sample examined multiple subtypes of MCI vs. only amnestic MCI).

436


Table 4. Positive and negative predictive powers of RBANS Indexes and subtests Cutoff

Base rate 10% PPP

List Learning 21.0 SD 0.272 21.5 SD 0.216 22.0 SD 0.250 Story Memory 21.0 SD 0.132 21.5 SD 0.288 22.0 SD 0.100 Immediate Memory Index 21.0 SD 0.199 21.5 SD 0.168 22.0 SD 0.354 Figure Copy 21.0 SD 0.152 21.5 SD 0.168 22.0 SD 0.207 Line Orientation 21.0 SD 0.188 21.5 SD 0.133 22.0 SD 0.182 Visuospatial Constructional Index 21.0 SD 0.215 21.5 SD 0.225 22.0 SD 0.308 Picture Naming 21.0 SD 0.106 21.5 SD 0.114 22.0 SD 0.308 Semantic Fluency 21.0 SD 0.306 21.5 SD 0.278 22.0 SD 0.308 Language Index 21.0 SD 0.169 21.5 SD 0.261 22.0 SD 0.278 Digit span 21.0 SD 0.086 21.5 SD 0.099 22.0 SD 0.072 Coding 21.0 SD 0.378 21.5 SD 1.000 22.0 SD 1.000 Attention Index 21.0 SD 0.138 21.5 SD 0.137 22.0 SD 0.053 List recall 21.0 SD 0.138 21.5 SD 0.137 22.0 SD 0.053 List recognition 21.0 SD 0.230 21.5 SD 0.434 22.0 SD 1.000

15%

20%

NPP

PPP

NPP

PPP

NPP

0.924 0.908 0.903

0.372 0.305 0.346

0.884 0.861 0.854

0.457 0.383 0.429

0.844 0.814 0.805

0.907 0.911 0.900

0.194 0.391 0.150

0.861 0.865 0.850

0.255 0.477 0.200

0.813 0.819 0.800

0.921 0.907 0.905

0.283 0.243 0.465

0.880 0.860 0.857

0.359 0.313 0.552

0.838 0.812 0.809

0.919 0.907 0.905

0.222 0.243 0.294

0.877 0.860 0.858

0.288 0.313 0.371

0.835 0.812 0.810

0.913 0.903 0.901

0.269 0.196 0.261

0.868 0.854 0.852

0.343 0.257 0.333

0.823 0.805 0.802

0.930 0.911 0.904

0.303 0.316 0.414

0.894 0.866 0.855

0.381 0.395 0.500

0.856 0.820 0.807

0.903 0.901 0.904

0.158 0.169 0.414

0.855 0.852 0.855

0.210 0.224 0.500

0.806 0.802 0.807

0.916 0.906 0.904

0.412 0.379 0.414

0.873 0.859 0.855

0.498 0.464 0.500

0.829 0.811 0.807

0.948 0.929 0.906

0.244 0.359 0.379

0.920 0.892 0.859

0.313 0.442 0.464

0.890 0.854 0.811

0.887 0.900 0.897

0.130 0.148 0.109

0.831 0.849 0.846

0.175 0.148 0.148

0.777 0.849 0.795

0.912 0.910 0.902

0.491 1.000 1.000

0.867 0.864 0.856

0.577 1.000 1.000

0.821 0.818 0.804

0.909 0.906 0.899

0.203 0.201 0.081

0.863 0.858 0.848

0.265 0.263 0.111

0.816 0.810 0.798

0.909 0.906 0.899

0.203 0.201 0.081

0.863 0.858 0.848

0.265 0.263 0.111

0.816 0.810 0.798

0.929 0.905 0.901

0.322 0.549 1.000

0.891 0.858 0.852

0.402 0.633 1.000

0.853 0.810 0.802

(continued on next page)


437

Table 4. (continued) Cutoff

Base rate 10%

Story recall 21.0 SD 21.5 SD 22.0 SD Figure recall 21.0 SD 21.5 SD 22.0 SD Delayed Memory Index 21.0 SD 21.5 SD 22.0 SD Total Scale 21.0 SD 21.5 SD 22.0 SD

15%

20%

PPP

NPP

PPP

NPP

PPP

NPP

0.202 0.148 0.237

0.915 0.903 0.902

0.286 0.216 0.331

0.872 0.854 0.852

0.363 0.281 0.412

0.828 0.805 0.803

0.264 0.335 0.581

0.929 0.918 0.911

0.363 0.445 0.688

0.892 0.875 0.865

0.446 0.531 0.758

0.854 0.832 0.819

0.265 0.314 0.468

0.944 0.922 0.909

0.365 0.420 0.583

0.914 0.881 0.863

0.448 0.507 0.664

0.882 0.840 0.816

0.234 0.343 0.256

0.941 0.921 0.906

0.326 0.453 0.353

0.909 0.879 0.859

0.407 0.540 0.436

0.876 0.837 0.811

Notes: PPP ¼ positive predictive power; NPP ¼ negative predictive power. Standard scores equivalents (i.e., M ¼ 100, SD ¼ 15) of 21.0, 21.5, and 22.0 SD would be 85, 77, and 70, respectively. Base rates of 10%, 15%, and 20% are presented, but readers can contact the first author for additional base rate values and their corresponding PPP and NPP. All scores are adjusted for age, education, and WRAT-3 Reading.

Fig. 1. ROC curves for selected RBANS Indexes. All RBANS Index scores are adjusted for age, education, and WRAT-3 Reading scores.

As noted above, our method of classifying MCI required individuals to fall 1.5 SD below an estimate of premorbid intellect (i.e., WRAT-3 Reading). Some may view this approach as “unconventional,” as others in the field require individuals to fall 1.5 SD below the mean of normative data. However, the stricter criteria (i.e., 1.5 SD below the normative mean) might unfairly penalize individuals with relatively higher and lower intellectual functioning, as they have to present with more or less decline from premorbid levels before breaking the rigid cutoff, respectively. For example, an individual who is premorbidly in the high average range (e.g., 84th percentile) needs to decline by approximately 77 percentile points to break the 1.5 SD below the normative mean. Conversely, an individual who is premorbidly in the low average range (e.g., 16th percentile) only needs to

438


decline by approximately 9 percentile points to break this same diagnostic barrier. By using a more flexible and individualized barrier (i.e., 1.5 SD decline from your premorbid level), decline (and the resulting diagnostic decisions) can be determined more comparably across individuals. Our method of approximating the MCI barrier is quite consistent with the literature. For example, the initial studies of MCI from the Mayo clinic group used a threshold that was “generally 1.5 SDs below age- and education-matched control subjects” (Petersen et al., 1999, p. 307). Within this same article (p. 305), the authors present means and standard deviations for their MCI subjects on several memory measures. When these means are compared to MOANS normative data for 79-year olds, most fall at about 1.5 SD below the mean (e.g., Logical Memory II ¼ scaled score of 5, Visual Reproductions II ¼ scaled score of 7, RAVLT percent retention ¼ scale score of 6). However, these are “mean” scores, which suggests that some sizable minority of the sample had scores above this point. This trend of loosely defined MCI has carried throughout most of the Mayo clinic MCI papers. Other authors have also viewed the MCI criteria as flexible (e.g., Bennett et al., 2002, p. 199: “judged to have cognitive impairment by a neuropsychologist but did not meet accepted criteria for dementia”—additionally, presented Logical Memory II data for their MCI group fell at a MOANS scaled score of 7; Busse et al., 2003, p. 73: “more than one SD below age- and education-specific norms”; Farias et al., 2009, p. 1152: “fell approximately 1.5 SDs below age-corrected norms”; Fleischer et al., 2007, p. 2: “cutoff score approximately 1.5 to 2 SDs below the education adjusted norms”; Griffith et al., 2006, p. 168: “objective memory impairment falling approximately 1.5 standard deviations or more below”; Luis et al., 2004, p. 308: “cognitive impairment but of insufficient magnitude to negatively affect daily functioning”). Although these references do not encompass all MCI papers and their criteria for defining this state, they do suggest that there are many different definitions of MCI (both conventional and unconventional). So should one decide to use a rigid or flexible criterion for MCI? One opinion on this matter comes from Dr Ronald Petersen in his 2004 paper (p. 189): In the literature, the cutoff score of 1.5 SD below age norms has been suggested by some investigators. In the original description of the MCI cohort followed at the Mayo Clinic, the MCI group’s mean performance was 1.5 SD below their agemates. However, this was not a cutoff score, and of course, nearly half of the group had memory performance score falling somewhat ,1.5 SD below the mean. This criterion should be interpreted in conjunction with the first criterion. The memory complaint is meant to represent a change in function for the person. The second criterion corroborates the complaint by attesting to and an actual impairment in performance. The clinician may be challenged by persons who are of either high intellect whose performance is now in the statistically ‘normal’ range, but this level of performance represents a change for that person, and by the person with a low education whose lower cognitive performance may not represent a change.

There are numerous examples in the literature to suggest that correcting for premorbid intellect is appropriate and wise in neuropsychology, especially in evaluating milder cognitive deficits in older adults. For example, Brooks and colleagues have recently published several papers that review the relatively high base rates of “impaired” performances in healthy subjects. In one (Brooks et al., 2008), the authors specifically address the relevance to MCI, as they report that healthy individuals who are “premorbidly” low tend to achieve more impaired scores than healthy individuals who are “premorbidly” high functioning. One consequence of a rigid criterion for impairment is that low functioning individuals will be classified as MCI (or with other cognitive disorders) at a much higher rate. Another consequence is that high-functioning individuals will rarely be identified with cognitive impairments. For these latter individuals, there is the risk of missing an appropriate diagnosis because they have so much further to fall before passing a set threshold. In a related article (Brooks et al., 2009), these authors recommend that premorbid functioning be considered when assessing evidence of cognitive decline. They also provide information that suggests that clinicians evaluating higher functioning individuals might use a more lenient criterion (e.g., 16th percentile) to define objective evidence of cognitive impairment. These authors are not alone in suggesting that intelligence (current or premorbid) provides value in assessing cognitive decline. The classic normative data for older adults (MOANS) have recently been re-calculated to adjust for IQ scores of patients (Steinberg et al., 2005a, 2005b; Steinberg, Bieliauskas, Smith, Ivnik, et al., 2005; Steinberg, Bieliauskas, Smith, Langellotti, et al., 2005). Although any deviations from convention need to be supported with validation and longitudinal findings, these studies represent a growing trend within the field of neuropsychology to develop better methods for defining cognitive impairment, particularly in the elderly. For example, the ongoing revisions of the Diagnostic and Statistical Manual of Mental Disorders 5th Edition (www.dsm5.org) suggest that premorbid intellect be considered (along with age, education, gender, and cultural factors) when determining if there has been a significant decline in cognition to support a diagnosis of Major Neurocognitive Disorder (formerly Dementia). Given our sample of highly educated individuals, our methods appear appropriate for capturing mild impairments in high-functioning individuals. Nonetheless, we also re-ran all our analyses after classifying individuals based on a stricter criterion for MCI (i.e., 1.5 SD below the normative mean). The results were very similar to those presented above, and the interested reader can contact the first author for a copy of those results. Another explanation for the low sensitivity might be due to the clinical condition that we studied, as other studies comparing MCI to controls have generated similar results (De Jager, Hogervorst, Combrinck, & Budge, 2003). It should not be surprising


439

that a milder condition (e.g., MCI) separates less well from healthy controls than a more severe condition (e.g., AD). In clinical practice, it may be more feasible to tailor diagnostic decisions to the individual with some flexibility (e.g., weighting multiple sources of information and test data), whereas research requires more standardized cutoff scores that might somewhat arbitrarily separate a true continuum (e.g., cognitive functioning). The resulting mixed groups, when compared with distinct groups, could lead to lowered diagnostic accuracy. There are several important limitations of this study. First, the classification of the current subjects was based almost entirely on cognitive test scores. Future studies should utilize additional clinical information to make this diagnosis (e.g., thorough physical examination, neuroimaging, biomarkers). Second, the amnestic subtype of MCI (single- or multidomain) was the only subtype examined in the current study, and these diagnostic accuracy estimates might not apply to non-amnestic MCI subtypes. Similarly, the diagnostic accuracy of the RBANS for other neuropsychiatric conditions with milder cognitive impairments (e.g., depression and substance abuse) should not be inferred from the current findings. Although most cognitive tests were corrected for age and education, three were not (BVMT-R, HVLT-R, and WRAT-3 Reading). These three tests were correcting for the age of the participants using data from the test manuals. However, this inconsistency in the norming of the measures could create some anomalies in classification of the participants or possibly bias against the RBANS. Finally, the current sample was exclusively Caucasian and well-educated, so the generalizability of these findings to a more diverse sample is uncertain. Despite these limitations, the current study provides some information about the diagnostic accuracy of the RBANS in suspected MCI, although this information suggests caution when using this measure in patients with milder cognitive deficits, such as those seen in MCI. Funding The project described was supported a research grant (R03 AG025850-01; K23 AG028417-01A2) from the National Institute on Aging. Conflict of Interest None declared. Acknowledgements The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute on Aging or the National Institutes of Health. References Badenes Guia, D., Casas Hernanz, L., Cejudo Bolivar, J. C., & Aguilar Barbera, M. (2008). Evaluation of the capacity to drive in patients diagnosed of mild cognitive impairment and dementia. Neurologia, 23 (9), 575 –582. Bennett, D. A., Wilson, R. S., Schneider, J. A., Evans, D. A., Beckett, L. A., Aggarwal, N. T., et al. (2002). Natural history of mild cognitive impairment in older persons. Neurology, 59 (2), 198– 205. Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C. A., Glasziou, P. P., Irwig, L. M., et al. (2003). Towards complete and accurate reporting of studies of diagnostic accuracy: The STARD initiative. The Standards for Reporting of Diagnostic Accuracy Group. Croatian Medical Journal, 44 (5), 635– 638. Brooks, B. L., Iverson, G. L., Feldman, H. H., & Holdnack, J. A. (2009). Minimizing misdiagnosis: Psychometric criteria for possible or probable memory impairment. Dementia and geriatric cognitive disorders, 27 (5), 439–450. Brooks, B. L., Iverson, G. L., Holdnack, J. A., & Feldman, H. H. (2008). Potential for misclassification of mild cognitive impairment: A study of memory scores on the Wechsler Memory Scale-III in healthy older adults. Journal of the International Neuropsychological Society, 14 (3), 463– 478. Busse, A., Bischkopf, J., Riedel-Heller, S. G., & Angermeyer, M. C. (2003). Mild cognitive impairment: prevalence and predictive validity according to current approaches. Acta Neurologica Scandinavia, 108 (2), 71–81. Cooper, D. B., Lacritz, L. H., Weiner, M. F., Rosenberg, R. N., & Cullum, C. M. (2004). Category fluency in mild cognitive impairment: Reduced effect of practice in test– retest conditions. Alzheimer Disease and Associated Disorders, 18 (3), 120– 122. De Jager, C. A., Hogervorst, E., Combrinck, M., & Budge, M. M. (2003). Sensitivity and specificity of neuropsychological tests for mild cognitive impairment, vascular cognitive impairment and Alzheimer’s disease. Psychological Medicine, 33 (6), 1039– 1050. Duff, K., Beglinger, L., Schoenberg, M., Patton, D., Mold, J., Scott, J., et al. (2005). Test-retest stability and practice effects of the RBANS in a community dwelling elderly sample. Journal of Clinical and Experimental Neuropsychology, 27 (5), 565–575. Duff, K., Beglinger, L., Van Der Heiden, S., Moser, D., Arndt, S., Schultz, S., et al. (2008). Short-term practice effects in amnestic mild cognitive impairment: Implications for diagnosis and treatment. International Psychogeriatrics, 20 (5), 986– 999. Duff, K., Humphreys Clark, J., O’Bryant, S., Mold, J., Schiffer, R., & Sutker, P. (2008). Utility of the RBANS in detecting cognitive impairment associated with Alzheimer’s disease: Sensitivity, specificity, and positive and negative predictive powers. Archives of Clinical Neuropsychology, 23 (5), 603–612.

440


Duff, K., Leber, W., Patton, D., Schoenberg, M., Mold, J., Scott, J., et al. (2007). Modified Scoring Criteria for the RBANS Figures. Applied Neuropsychology, 14 (2), 73–83. Duff, K., Patton, D., Schoenberg, M., Mold, J., Scott, J., & Adams, R. (2003). Age- and education-corrected independent normative data for the RBANS in a community dwelling elderly sample. Clinical Neuropsychology, 17 (3), 351– 366. Duff, K., Schoenberg, M. R., Patton, D. E., Mold, J., Scott, J. G., & Adams, R. A. (2004). Predicting change with the RBANS in a community dwelling elderly sample. Journal of the International Neuropsychological Society, 10, 828–834. Economou, A., Papageorgiou, S. G., Karageorgiou, C., & Vassilopoulos, D. (2007). Nonepisodic memory deficits in amnestic MCI. Cognitive and Behavioral Neurology, 20 (2), 99–106. Farias, S. T., Mungas, D., Reed, B. R., Harvey, D., & DeCarli, C. (2009). Progression of mild cognitive impairment to dementia in clinic- vs. community-based cohorts. Archives of Neurology, 66 (9), 1151– 1157. Fleisher, A. S., Sowell, B. B., Taylor, C., Gamst, A. C., Petersen, R. C., & Thal, L. J. (2007). Clinical predictors of progression to Alzheimer disease in amnestic mild cognitive impairment. Neurology, 68 (19), 1588– 1595. Freilich, B. M., & Hyer, L. A. (2007). Relation of the Repeatable Battery for Assessment of Neuropsychological Status to measures of daily functioning in dementia. Psychological Reports, 101 (1), 119–129. Griffith, H. R., Netson, K. L., Harrell, L. E., Zamrini, E. Y., Brockington, J. C., & Marson, D. C. (2006). Amnestic mild cognitive impairment: Diagnostic outcomes and clinical prediction over a two-year time period. Journal of the International Neuropsychological Society, 12 (2), 166–175. Hanley, J. A., & McNeil, B. J. (1982). The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology, 143 (1), 29–36. Hobson, V. L., Hall, J. R., Humphreys-Clark, J. D., Schrimsher, G. W., & O’Bryant, S. E. (2010). Identifying functional impairment with scores from the repeatable battery for the assessment of neuropsychological status (RBANS). International Journal of Geriatric Psychiatry, 25 (2), 525– 530. Holmes, C., Cairns, N., Lantos, P., & Mann, A. (1999). Validity of current clinical criteria for Alzheimer’s disease, vascular dementia and dementia with Lewy bodies. British Journal of Psychiatry, 174, 45– 50. Horton, A. M., Jr (1999). Above-average intelligence and neuropsychological test score performance. International Journal of Neuroscience, 99 (1-4), 221 – 231. Juhasz, L. Z., Kemeny, K., Linka, E., Santha, J., & Bartko, G. (2003). The use of RBANS test (Repeatable Battery for the Assessment of Neuropsychological Status) in neurocognitive testing of patients suffering from schizophrenia and dementia. Ideggyo´gya´szati Szemle, 56 (9–10), 303–308. Kotani, S., Sakaguchi, E., Warashina, S., Matsukawa, N., Ishikura, Y., Kiso, Y., et al. (2006). Dietary supplementation of arachidonic and docosahexaenoic acids improves cognitive dysfunction. Neuroscience Research, 56 (2), 159– 164. Kraemer, H. C. (1992). Evaluating medical tests: Objective and quantitative guidelines. Newbury Park, CA: Sage. Lett, R. R., Hanley, J. A., & Smith, J. S. (1995). The comparison of injury severity instrument performance using likelihood ratio and ROC curve analyses. Journal of Trauma, 38 (1), 142–148. Lines, C. R., McCarroll, K. A., Lipton, R. B., & Block, G. A. (2003). Telephone screening for amnestic mild cognitive impairment. Neurology, 60 (2), 261 – 266. Luis, C. A., Barker, W. W., Loewenstein, D. A., Crum, T. A., Rogaeva, E., Kawarai, T., et al. (2004). Conversion to dementia among two groups with cognitive impairment. A preliminary report. Dementia and Geriatric Cognitive Disorders, 18 (3–4), 307–313. Nash, K., Rovet, J., Greenbaum, R., Fantus, E., Nulman, I., & Koren, G. (2006). Identifying the behavioural phenotype in Fetal Alcohol Spectrum Disorder: Sensitivity, specificity and screening potential. Archives of Women’s Mental Health, 9 (4), 181–186. Patton, D. E., Duff, K., Schoenberg, M. R., Mold, J., Scott, J. G., & Adams, R. L. (2003). Performance of cognitively normal African Americans on the RBANS in community dwelling older adults. Clinical Neuropsychology, 17 (4), 515– 530. Petersen, R. C. (2004). Mild cognitive impairment as a diagnostic entity. Journal of Internal Medicine, 256 (3), 183 –194. Petersen, R. C., Doody, R., Kurz, A., Mohs, R. C., Morris, J. C., Rabins, P. V., et al. (2001). Current concepts in mild cognitive impairment. Archives of Neurology, 58 (12), 1985–1992. Petersen, R. C., Roberts, R. O., Knopman, D. S., Boeve, B. F., Geda, Y. E., Ivnik, R. J., et al. (2009). Mild cognitive impairment: Ten years later. Archives of Neurology, 66 (12), 1447–1455. Petersen, R. C., Smith, G. E., Waring, S. C., Ivnik, R. J., Tangalos, E. G., & Kokmen, E. (1999). Mild cognitive impairment: Clinical characterization and outcome. Archives of Neurology, 56 (3), 303–308. Petrella, J. R., Prince, S. E., Krishnan, S., Husn, H., Kelley, L., & Doraiswamy, P. M. (2009). Effects of donepezil on cortical activation in mild cognitive impairment: A pilot double-blind placebo-controlled trial using functional MR imaging. AJNR. American Journal of Neuroradiology, 30 (2), 411–416. Randolph, C. (1998). Repeatable Battery for the Assessment of Neuropsychological Status. San Antonio, TX: The Psychological Corporation. Randolph, C., Tierney, M. C., Mohr, E., & Chase, T. N. (1998). The Repeatable Battery for the Assessment of Neuropsychological Status (RBANS): Preliminary clinical validity. Journal of Clinical and Experimental Neuropsychology, 20 (3), 310–319. Raschetti, R., Albanese, E., Vanacore, N., & Maggini, M. (2007). Cholinesterase inhibitors in mild cognitive impairment: A systematic review of randomised trials. PLoS Medicine, 4 (11), e338. Salloway, S., Ferris, S., Kluger, A., Goldman, R., Griesing, T., Kumar, D., et al. (2004). Efficacy of donepezil in mild cognitive impairment: A randomized placebo-controlled trial. Neurology, 63 (4), 651–657. Saykin, A. J., Wishart, H. A., Rabin, L. A., Flashman, L. A., McHugh, T. L., Mamourian, A. C., et al. (2004). Cholinergic enhancement of frontal lobe activity in mild cognitive impairment. Brain, 127 (Pt. 7), 1574– 1583. Schmidt, U., Fuessel, S., Koch, R., Baretton, G. B., Lohse, A., Tomasetti, S., et al. (2006). Quantitative multi-gene expression profiling of primary prostate cancer. Prostate, 66 (14), 1521– 1534. Steinberg, B. A., Bieliauskas, L. A., Smith, G. E., & Ivnik, R. J. (2005a). Mayo’s Older Americans Normative Studies: Age- and IQ-adjusted norms for the Trail-Making Test, the Stroop Test, and MAE Controlled Oral Word Association Test. Clinical Neuropsychology, 19 (3 –4), 329– 377. Steinberg, B. A., Bieliauskas, L. A., Smith, G. E., & Ivnik, R. J. (2005b). Mayo’s Older Americans Normative Studies: Age- and IQ-Adjusted Norms for the Wechsler Memory Scale–Revised. Clinical Neuropsychology, 19 (3 –4), 378 –463. Steinberg, B. A., Bieliauskas, L. A., Smith, G. E., Ivnik, R. J., & Malec, J. F. (2005). Mayo’s Older Americans Normative Studies: Age- and IQ-adjusted norms for the Auditory Verbal Learning Test and the Visual Spatial Learning Test. Clinical Neuropsychology, 19 (3– 4), 464– 523.


441

Steinberg, B. A., Bieliauskas, L. A., Smith, G. E., Langellotti, C., & Ivnik, R. J. (2005). Mayo’s Older Americans Normative Studies: Age- and IQ-adjusted norms for the Boston Naming Test, the MAE Token Test, and the Judgment of Line Orientation Test. Clinical Neuropsychology, 19 (3-4), 280 –328. Stephan, C., Meyer, H. A., Cammann, H., Nakamura, T., Diamandis, E. P., & Jung, K. (2006). Improved prostate cancer detection with a human kallikrein 11 and percentage free PSA-based artificial neural network. Biological Chemistry, 387 (6), 801–805. Winblad, B., Gauthier, S., Scinto, L., Feldman, H., Wilcock, G. K., Truyen, L., et al. (2008). Safety and efficacy of galantamine in subjects with mild cognitive impairment. Neurology, 70 (22), 2024–2035. Winblad, B., Palmer, K., Kivipelto, M., Jelic, V., Fratiglioni, L., Wahlund, L. O., et al. (2004). Mild cognitive impairment –beyond controversies, towards a consensus: Report of the International Working Group on Mild Cognitive Impairment. Journal of International Medicine, 256 (3), 240 –246.