The reliability, validity, sensitivity, specificity and

ORIGINAL ARTICLE

The reliability, validity, sensitivity, specificity and predictive values of the Chinese version of the Rowland Universal Dementia Assessment Scale Chia-Wei Chen, Hsin Chu, Chia-Fen Tsai, Hui-Ling Yang, Jui-Chen Tsai, Min-Huey Chung, Yuan-Mei Liao, Mei-ju Chi and Kuei-Ru Chou

Aims and objectives. The purpose of this study was to translate the Rowland Universal Dementia Assessment Scale into Chinese and to evaluate the psychometric properties (reliability and validity) and the diagnostic properties (sensitivity, specificity and predictive values) of the Chinese version of the Rowland Universal Dementia Assessment Scale. Background. The accurate detection of early dementia requires screening tools with favourable cross-cultural linguistic and appropriate sensitivity, specificity, and predictive values, particularly for Chinese-speaking populations. Design. This was a cross-sectional, descriptive study. Methods. Overall, 130 participants suspected to have cognitive impairment were enrolled in the study. A test-retest for determining reliability was scheduled four weeks after the initial test. Content validity was determined by five experts, whereas construct validity was established by using contrasted group technique. The participants’ clinical diagnoses were used as the standard in calculating the sensitivity, specificity, positive predictive value and negative predictive value. Results. The study revealed that the Chinese version of the Rowland Universal Dementia Assessment Scale exhibited a test-retest reliability of 090, an internal consistency reliability of 071, an inter-rater reliability (kappa value) of 088 and a content validity index of 097. Both the patients and healthy contrast group exhibited significant differences in their cognitive ability. The optimal cut-off points for the Chinese version of the Rowland Universal Dementia Assessment Scale in the test for mild cognitive impairment and dementia were 24 and 22,

What does this paper contribute to the wider global clinical community?

•

•

•

The Chinese version of the RUDAS (RUDAS-C) has several advantages over existing instruments; it comprises a moderate number of items, sixdimension structure, and exhibits good sensitivity, specificity, and high predictive values in screening mild cognitive impairment (MCI) and dementia. In this pioneer study, the sensitivity, specificity and predictive values of the RUDAS-C were developed as a tool for screening dementia in a Chinese-speaking population. Accurately detecting early dementia in the older population from culturally and linguistically diverse backgrounds requires screening tools with favourable cross-cultural linguistic and appropriate sensitivity and validity. The appropriate sensitivity, specificity and predictive values of the RUDASC show that it is a useful clinical tool for early screening and can be used as a standard measure of MCI in clinical settings. In addition, it is a fast, accurate and sensitive means for facilitating accurate diagnosis of dementia in clinical practice.

Authors: Chia-Wei Chen, MS, RN, Graduate student, Graduate Institute of Nursing, College of Nursing, Taipei Medical University; Hsin Chu, PhD, MD,

School of Gerontology Health Management, College of Nursing, Taipei Medical University and Assistant Professor, Master Program in Long-term Care,

Associate Professor, Institute of Aerospace and Undersea Medicine, School of Medicine, National Defense Medical Center and Doctor, Department of Neu-

College of Nursing, Taipei Medical University; Kuei-Ru Chou, PhD, RN, Professor, Graduate Institute of Nursing, College of Nursing, Taipei Medical

rology, Tri-Service General Hospital, National Defense Medical Center; ChiaFen Tsai, MD, Doctor, Department of Psychiatry, Taipei Veterans General

University, Professor, School of Gerontology Health Management, College of Nursing, Taipei Medical University, Professor, Psychiatric Research Center,

Hospital, Institute of Brain Science, National Yang-Ming University Schools of Medicine and Faculty of Medicine, National Yang-Ming University Schools

Taipei Medical University Hospital and Vice Director, Department of Nursing, Taipei Medical University-Shuang Ho Hospital, Taipei, Taiwan

of Medicine; Hui-Ling Yang, MS, RN, Doctoral student, Graduate Institute of Nursing, College of Nursing, Taipei Medical University; Jui-Chen Tsai,

Correspondence: Kuei-Ru Chou, Professor, Graduate Institute of Nursing, College of Nursing, Taipei Medical University, No. 250, Wu-Hsing Street, Taipei

MS, RN, Director, Department of Nursing, Taipei Medical University-Shuang Ho Hospital; Min-Huey Chung, PhD, RN, Associate Professor, Graduate

110, Taiwan. Telephone: +886 2 2736 1661 ext. E-mail: [email protected]

Institute of Nursing, College of Nursing, Taipei Medical University; Yuan-Mei Liao, PhD, RN, Associate Professor, Graduate Institute of Nursing, College of

Hsin Chu is equal contribution with first author.

Nursing, Taipei Medical University; Mei-ju Chi, PhD, Assistant Professor,

© 2015 John Wiley & Sons Ltd Journal of Clinical Nursing, 24, 3118–3128, doi: 10.1111/jocn.12941

3118

87

Original article

Psychometric properties of the RUDAS-C

respectively; moreover, for these two conditions, the sensitivities of the scale were 079 and 076, the specificities were 091 and 081, the areas under the curve were 085 and 078, the positive predictive values were 099 and 083 and the negative predictive values were 096 and 091 respectively. Conclusion. The Chinese version of the Rowland Universal Dementia Assessment Scale exhibited sound reliability, validity, sensitivity, specificity and predictive values. Relevance to clinical practice. This scale can help clinical staff members to quickly and accurately diagnose cognitive impairment and provide appropriate treatment as early as possible.

Key words: Chinese, Chinese version of the Rowland Universal Dementia Assessment Scale, dementia, reliability, sensitivity, specificity, validity Accepted for publication: 7 June 2015

Introduction Mild cognitive impairment (MCI) is a transitional state towards dementia along with the process of ageing. Persons with MCI have impairments limited to one aspect of cognitive function and persons with dementia have impairment in two or more cognitive functions (e.g. memory, judgment, reasoning, executive function) (Petersen 2011). In practice, MCI is often noticed as the preceding condition of dementia. The detection of MCI thereby can be seen as the essential strategy for the early prevention and treatment of dementia. According to the World Health Organization (WHO 2012), in 2012, 36 million patients worldwide had dementia, representing a prevalence rate of 47%. In North America, 435 million dementia patients have been reported, representing a prevalence rate of 69% (Prince et al. 2013). The prevalence rate for MCI in the United States of America is 222% (Plassman et al. 2008). Reports by the Taiwan Department of Health have indicated that 130,000 adults aged >65 years have dementia, representing a prevalence rate of 497% and that the prevalence of MCI is 1604% (Department of Health, Executive Yuan 2013). The World population is rapidly aging, and dementia and MCI are diseases of old age that can severely influence a patient’s capability for self-care and greatly increase the burden on a caretaker. Therefore, early screening and intervention are essential.

Background Early dementia may be difficult to diagnose in older population. This is an important health issue in many countries. The literature revealed that various dementia screening

tools used in Taiwan were influenced by educational level and cultural background and lacked complete measurement of visuospatial orientation in dementia screening (Guo et al. 1988). The most commonly used cognitive function screening tool is the Mini-Mental State Examination (MMSE), which has been translated into numerous languages and used extensively in clinical settings (Folstein et al. 1975). Although the MMSE can detect mild, moderate and severe dementia, it exhibits low sensitivity in detecting MCI, particularly frontal lobe dysfunction. In addition, its results are influenced by age, education, language and cultural background, potentially producing the ceiling effect (Tombaugh & Mcintyre 1992). The Fuld Object-Memory Evaluation was culturally fair but covers only a few cognitive dimensions (Mast et al. 2001). The Cognitive Abilities Screening Instrument has the significant effect for education in some cultural groups (Shadlen et al. 2001). The Elderly Cognitive Assessment Questionnaire (ECAQ) was established in a predominantly male population (Kua & Ko 1992). The abbreviated mental test (AMT, Hong Kong version) has also showed the advantage of simplicity and brevity to screen for impaired cognitive function of older adults in Chinese population (Chu et al. 1995, Lam et al. 2010). The AMT needs further studies to detect its use for screening MCI and dementia. The Rowland Universal Dementia Assessment Scale (RUDAS) is a screening tool that may be used alone and covers six aspects: memory, visuospatial orientation, praxis, visuoconstructional drawing, judgment and language (Storey et al. 2004). Table 1 lists previous studies evaluating the reliability, validity, sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) of the RUDAS. These studies have demonstrated that the RUDAS is not

© 2015 John Wiley & Sons Ltd Journal of Clinical Nursing, 24, 3118–3128

3119

88

C-W Chen et al. Table 1 Reliability and validity of the RUDAS n

Cronbach’s a

Test-retest

Limpawattana et al. (2012) Normal 89 – – MCI 22 Dementia 89 Basic et al. (2009) Normal 60 080 – CIND 33 Dementia 58 Iype et al. (2006) Normal 58 – – Dementia 58 Rowland et al. (2006) Normal 48 – – MCI 18 Dementia 63 Storey et al. (2004) Normal 45 – 098 Dementia 45

Inter-rater

Cut-off point

Sensitivity

Specificity

AUC

PPV

NPV

–

24

079

061

070

067

074

–

23

088

090

094

877 (Positive LR)

014 (Negative LR)

–

23

088

076

–

–

–

–

23

081

096

–

1943 (Positive LR)

020 (Negative LR)

099

23

089

098

–

–

–

MCI, mild cognitive impairment; CIND, cognitive impairment not dementia; AUC, area under curve; PPV, positive predictive value; NPV, negative predictive value; LR, likelihood ratio; RUDAS, Rowland Universal Dementia Assessment Scale.

exhibits good sensitivity, specificity and high predictive values in screening MCI and dementia. The aim of this study was to assess the psychometric properties (reliability and validity) and diagnostic properties (sensitivity, specificity, PPV and NPV) of the RUDAS-C scale for dementia in a Chinese-speaking population.

influenced by sex, educational level or language and exhibited a test-retest reliability of 098, an inter-rater reliability of 099, a total area under the curve (AUC) of 094 [95% confidence interval (CI): 087 098], an optimal cut-off point of 23, a sensitivity of 89% and a specificity of 98% (Storey et al. 2004). The Thai version of the RUDAS has been demonstrated to have high reliability and validity. It exhibited an internal consistency (Cronbach’s a) of 080 (Basic et al. 2009) and an inter-rater reliability of 099 (Storey et al. 2004). When the cut-off point was 23, the sensitivity ranged from 81–89% (Storey et al. 2004, Iype et al. 2006, Rowland et al. 2006, Basic et al. 2009), the specificity ranged from 760–958% (Storey et al. 2004, Iype et al. 2006, Rowland et al. 2006, Basic et al. 2009), the positive likelihood ratio ranged from 877–1943 (Rowland et al. 2006, Basic et al. 2009), and the negative likelihood ratio ranged from 014–020 (Rowland et al. 2006, Basic et al. 2009) (Table 1). Several items of the RUDAS address frontal lobe impairment, and the diverse response formats allow more comprehensive assessment of overall cognition. It covers multiple cognitive functions and has been proved to have high reliability and validity. In consideration of the aforementioned concerns, accurately detecting early dementia in older people from culturally and linguistically diverse backgrounds requires screening tools with favourable cross-cultural linguistic and appropriate sensitivity, specificity and predictive values. The Chinese version of the RUDAS (RUDAS-C) has several advantages over existing instruments; it comprises a moderate number of items, six-dimension structure and

Methods Participants The participants were selected from the outpatient clinics of the neurology, geriatric, and geriatric psychiatry departments of two medical centres in Taiwan. Participants who (1) were aged >65 years and reported deterioration in memory and (2) could read and write or understand the content of the questionnaire through verbal communication were eligible for participation in this study. Overall, 130 participants aged >65 years who reported deterioration in memory and activities of daily living (ADLs) were recruited between May 2012 and May 2013. After enrolment, the participants were clinically diagnosed with MCI and dementia according to the National Institute on Aging and Alzheimer’s Association Diagnostic Guidelines and the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision (DSM-IV-TR), which serve as the gold standard references and the RUDAS-C was administered to each participant by an independent and well-trained on-site neurologist for comprehensive diagnostic validation. The on-site neurologist was © 2015 John Wiley & Sons Ltd Journal of Clinical Nursing, 24, 3118–3128

3120

89

Original article


Chinese version was examined when the participants returned the questionnaires after four weeks to determine the stability of the scale; (2) Inter-rater reliability: In addition, the researcher and a psychiatrist evaluated 10 of the participants to confirm that the results were consistent and to determine the inter-rater reliability; (3) Internal consistency reliability: The intra-class correlation (ICC) was used to assess the internal consistency of the RUDAS-C. The validity was assessed as follows: (1) Content validity: The content validity was examined by two psychiatrists, one neurologist, one instrument development specialist and one psychiatric nurse practitioner, who evaluated the accuracy, content suitability and clarity of the translated text and amended it on the basis of their expert opinions. The content validity index (CVI), of which a value ≥80% was used as the assessment standard, was defined as the proportion in which ≥80% of the experts agreed with scores ≥3 points on the scale (Lynn 1986). The suggestions of the experts were considered in modifying the questionnaire when necessary; (2) Construct validity: Construct validity was established by using the contrast group technique. The RUDAS-C was determined by comparing the 130 participants with a contrast group of 50 healthy older participants from community health centre.

blinded to this evaluation. The clinical criteria for MCI, as proposed by the National Institute on Aging and the Alzheimer’s Association, were operationalised as follows: (1) concerns regarding changes in cognition: a self-report by the participant regarding changes in cognition compared with the participant’s prior level; (2) impairment in one or more cognitive domains (including memory, executive function, attention, language and visuospatial skills on the MMSE); (3) preservation of independence in functional abilities (selfreport of independent performance in all instrumental ADLs; and (4) absence of dementia [scores >24/30 on the MMSE and Clinical Dementia Rating (CDR) score < 1] (Albert et al. 2010). The DSM-IV-TR requires the presence of memory impairment in conjunction with deficits in one other cognitive domain (aphasia, apraxia, agnosia and executive dysfunctioning) (American Psychiatric Association 2000).

Procedure Translation Mere translation of study instruments is not enough to translate a questionnaire literally. The adaptation of the instrument in a culturally relevant and comprehensible form while maintaining the meaning and intent of the original items is especially important (Sperber 2004). The multiplestage cross-cultural cognitive examination was developed for use in cross-cultural epidemiological dementia (Glosser et al. 1993). Therefore, the RUDAS was translated using Brislin’s steps for translation (Brislin 1970). (1) We identified competent bilingual translators who were familiar with the content. (2) After practice, one translator was requested to translate the content from the source language (SL, English) to the target language (TL, Chinese); another translator was requested to blindly back translate the content from the TL to the SL. (3) Five raters examined the content in the original SL, the TL, and the back-translated version for meaning errors. The researcher then held a meeting with the translators to discuss the accuracy, equivalency, fluency and degree of interpretability of the Chinese version. The differences between the translation and the back-translation were rectified until the translation and the original were equivalent to ensure the accuracy of the research tool. (4) After the meaning errors were corrected, the target version was pretested in monolingual TL populations.

Sensitivity, specificity, PPV and NPV The sensitivity, specificity, PPV and NPV of the participants’ clinical diagnoses of MCI and dementia were determined according to the National Institute on Aging and Alzheimer’s Association Diagnostic Guidelines and the DSM-IV-TR, which serve as the gold standard references, and the RUDAS-C was administered to the participants from the outpatient clinics of the neurology, geriatric and geriatric psychiatry departments by a well-trained on-site neurologist for comprehensive diagnostic validation.

Instruments Gold Standard: The clinical criteria for MCI, as proposed by the National Institute on Aging and the Alzheimer’s Association, were operationalised as follows: (1) concerns regarding changes in cognition, (2) impairment in one or more cognitive domains (including memory, executive function, attention, language, and visuospatial skills on the MMSE), (3) preservation of independence in functional abilities, and (4) absence of dementia (scores > 24/30 on the MMSE and CDR score < 1) (Albert et al. 2010).

Reliability and validity Reliability was assessed as follows: (1) Test–retest reliability: The reliability of the scale was tested in 130 participants suspected to have cognitive impairment. The RUDAS questionnaire was administered, and the test-retest reliability of the

Mini-Mental State Examination The MMSE contains 11 questions with a maximum total score of 30. A higher score indicates higher cognitive


3121

90

C-W Chen et al.

et al. 2006). At least 65 participants were required to achieve 95% CI of 15%. In addition, the sample calculation used the standard error for the area under the receiver operating characteristic (ROC) curve. Accordingly, at least 130 participants were required to achieve an AUC of 08 in the RUDAS, and the standard error for this AUC was determined to be 005.

function (Folstein et al. 1975). The cut-off point for the MMSE scores was 23. A score 24 is considered normal (De Paulo & Folstein 1978). The test–retest reliability of the MMSE over a 24-hour period in the same participants was 089. A positive correlation between the MMSE and the Wechsler Adult Intelligence Scale scores of a participant (078 and 066 for verbal and performance sections respectively) indicated high validity of the MMSE (Folstein et al. 1975). The Chinese version of the MMSE was constructed and validated by Guo et al. (1988). The severity of dementia was categorised according to the MMSE score of the participants as mild (19–23), moderate (10–18), severe (1–9), and very severe (0) (Tiraboschi et al. 2000). The copyright of the MMSE and the Chinese version of the MMSE have been obtained from Psychological Assessment Resources.

Statistical analysis Statistical analysis was performed using SPSS version 21.0 (IBM Corporation, Armonk, New York, United States). The test-retest and inter-rater reliability were established using ICC and Cohen’s kappa respectively. The standard CVI was set at 80%, and the scale was amended according to expert opinion. The construct validity was determined using the contrasted group approach. Both the patient and healthy groups were compared using independent sample t-tests to determine any differences between the groups. Using the clinical diagnoses as the gold standard, we determined the ROC curves for the RUDAS scores, the maximum AUC, and the corresponding cut-off point. In addition, the sensitivity, specificity, PPV, and NPV were calculated using the following formulas: (1) Sensitivity of a clinical test represents test ability to correctly identify people with illness. Sensitivity (true positive rate, TPR) = [patients diagnosed with MCI (or dementia) by using one scale/patients diagnosed with MCI (or dementia) + patients with MCI [or dementia] but not diagnosed with MCI] 9 100%; (2) the specificity of a clinical test represents teat ability to correctly identify people without illness (Stojanovic et al. 2014). Specificity (true negative rate, TNR) = [patients diagnosed with MCI (or dementia) by using one scale/patients with MCI (or dementia) but not diagnosed with MCI + patients without MCI [or dementia)] 9 100%; (3) The PPV is defined as a proportion of people with a positive test result. PPV = [patients diagnosed with MCI (or dementia)/patients who were tested for MCI (or dementia)] 9 100%; and (4) The NPV is defined as a proportion of people with a negative test result. NPV = [patients determined not to have MCI (or dementia)/ patients who were tested for MCI (or dementia)] 9 100%.

Clinical dementia rating The CDR evaluates the severity of dementia and consists of six cognitive-functional domains: memory, orientation, judgment and problem solving, community affairs, home and hobbies and personal care (Morris 1993). Data derived from semi structured interviews with individual participants and their caregiver were reviewed to determine the functional impairment ratings of each cognitive category. These category scores (or box scores) were then analysed using scoring rules to determine a global CDR score from 0 to 3, where 0 indicated no cognitive impairment, 050 indicated MCI, and ≥1 indicated clear dementia (McCulla et al. 1989). Three neurologists and one psychologist (Choi et al. 2001) reported that the CDR has high inter-rater reliability (kappa = 086 100). Chinese version of the Rowland Universal Dementia Assessment Scale Index test: The RUDAS-C measures cognitive functions and comprises six categories of items. Completing the questionnaire requires 10 minutes. Of the 30 possible points, a score ≤23 indicates cognitive impairment. The test and interview must be administered by a physician, psychologist or other trained professional (Storey et al. 2004). Table 1 lists previous studies on the reliability, validity, sensitivity, specificity, PPVand NPV of the RUDAS.

Ethical considerations The study protocol was approved by the institutional review boards of the consenting hospitals for the protection of human participants (TGSHIRB: 1-101-05-016; VGHIRB: 2012-05-004A). Patients who met the inclusion criteria were informed of the study aims and duration as well as how the collected data were to be used. Moreover, they

Sample size The sample size was calculated by controlling the width of the 95% CI around an estimated correlation coefficient of 080 between the RUDAS and other instruments (Rowland


3122

91

Original article


were assured that the study would not interfere with the outcome of their medical treatment. The study was performed with the consent of the patients or their legal guardians. Throughout the study period, the participants could withdraw from the study at any time upon request, and this would not affect their existing treatment or health care. The participants’ background information was kept confidential and used only for academic purposes.

Table 2 Participants’ information n = 130 Variable

n

%

Gender Male Female

67 63

515 485 Mean

Age (years) Diagnosis Normal MCI Dementia Education Elementary school Junior high school Senior high University level MMSE ˃24 19–23 10–18 1–9 RUDAS ˃23 ≤23

Results This study assessed 130 participants who met the inclusion criteria and consented to participate in the study. The distributions of the basic descriptive statistical variables of the participants are described in the following section.

Participant characteristics Among the 130 participants, 22 exhibited normal cognitive functions, 55 had MCI and 53 had dementia. The sample contained 67 men (515%) and 63 women (485%) with an average age of 762 years. Regarding educational level, 362, 162, 192 and 284% of the participants had received elementary school, junior high school, senior high school, and university levels of education respectively. Moreover, 60 (462%) participants were patients of the neurology department, 46 (354%) were patients of the geriatric psychiatric department, and 24 (185%) were patients of the geriatric medical centre. Regarding educational level, 380, 160, 200, and 260% of the participants had received elementary school, junior high school, senior high school, and university levels of education respectively (Table 2).

7615

SD 8549

22 55 53

169 423 408

47

362

21 25 37

162 192 284

83 34 13 0

638 262 10 0

55 75

423 577

MCI, mild cognitive impairment; MMSE, Mini-Mental State Examination; RUDAS, The Rowland Universal Dementia Assessment Scale; SD, standard deviation.

participants, a Cohen’s kappa of 088 was obtained, confirming that the results of the scale were consistent with the diagnoses.

Reliability

Validity

The optimal interval between the two test-retest reliability measurements was approximately one to two weeks. According to Burns and Grove (2005), a test-retest reliability coefficient of 07 is acceptable, whereas a coefficient of 08 is the minimum requirement for a developing tool. The RUDAS-C was administered to all participants, and 30 participants were re-administered the RUDAS-C after four weeks to determine the test-retest reliability, revealing an r value of 090, which indicated high stability. Further analyses were conducted to determine the internal consistency and were found to have ICC coefficient with 071. The inter-rater reliability was determined using Cohen’s kappa. After the researcher and the specialist evaluated 10

The CVI was determined by five experts. The scale comprised 15 items, and only one of the experts provided a score of 2 for two items, whereas all remaining items received a score of 3 points. According to the formula, the CVI of the RUDAS-C was 097. The construct validity was determined using the contrasted group approach. Comparing the test results of the patient group (participants were from the neurology department, geriatric psychiatric department, and geriatric medical centre) and the contrast group (recruited from the community health centre) by using independent sample t-tests revealed significant differences in cognitive function between the two groups [t = 114 (p < 0001)].


3123

92

C-W Chen et al.

Discussion

Sensitivity, specificity, PPV and NPV The 130 participants were clinically diagnosed with MCI and dementia according to the clinical diagnostic criteria of the National Institute on Aging and Alzheimer’s Association Diagnostic Guidelines and the DSM-IV-TR, which were used as the gold standard to calculate the sensitivity, specificity, PPV, and NPV. Sensitivity and specificity were used to represent the validity of the screening tests. The AUC of the MCI assessment was 092 (95% CI: 086 098). The ROC curves presented in Fig. 1 and Table 3 show that, when a cut-off point of 24 was used, patients with MCI were accurately identified 79% of the time (sensitivity, TPR); moreover, the RUDAS-C exhibited a 99% chance of accurate identification (PPV), a 91% chance of accurately identifying patients without MCI (specificity, TNR), and a 96% chance that this identification is accurate (NPV). As shown in Fig. 2, the AUC for dementia assessment was 087 (95% CI: 080 093). Based on the ROC curve, Table 4 shows that, when a cut-off point of 22 was used, the RUDAS-C exhibited a 76% chance of detecting dementia (sensitivity, TPR) and an 83% chance that this identification is accurate (PPV). In addition, it exhibited an 81% chance of detecting patients without dementia (specificity, TNR) and a 91% chance that this identification is accurate (NPV).

Reliability This study enrolled adult participants aged >65 years, who were likely to cancel return visits for retests because of health conditions (sickness), circumstances (failure of caretaker to accompany) or weather (rain). Therefore, only 30 participants were included in the test-retest reliability analysis. The results showed that the RUDAS-C exhibited a test– retest reliability coefficient of 090, which was similar to Table 3 Chinese RUDAS ROC curve analysis (normal vs. MCI) n = 77 Cut-off point

Sensitivity

Specificity

AUC

PPV

NPV

22 23 24* 25 26 27

051 063 079* 085 096 098

096 096 091* 078 065 035

073 079 085* 082 081 067

1 098 099* 098 095 093

1 096 096* 091 078 065

AUC, area under curve; PPV, positive predictive value; NPV, negative predictive value; RUDAS, Rowland Universal Dementia Assessment Scale; ROC, receiver operating characteristic; MCI, mild cognitive impairment. *Optimal cut-off point and results.

Figure 1 Chinese RUDAS ROC curve and Chinese RUDAS assessment (normal vs. MCI). RUDAS, Rowland Universal Dementia Assessment Scale; ROC, receiver operating characteristic; MCI, mild cognitive impairment.


3124

923

Original article


Figure 2 Chinese RUDAS ROC curve and Chinese RUDAS assessment (normal vs. dementia). RUDAS, Rowland Universal Dementia Assessment Scale; ROC, receiver operating characteristic.

Table 4 Chinese RUDAS ROC curve analysis (normal vs. dementia)

easy to use clinically. The CVI of the original version was never reviewed, and thus, no comparison was available. The construct validity of the RUDAS-C was determined using the group contrast approach after the patients were compared to the healthy contrast groups. The results revealed significant differences between the groups and that the RUDAS-C exhibited construct validity.

n = 75 Cut-off point

Sensitivity

Specificity

AUC

PPV

NPV

20 21 22* 23 24 25

051 064 076* 083 091 094

099 091 081* 069 051 040

075 078 078* 076 071 067

1 096 083* 073 065 056

1 099 091* 081 069 051

Sensitivity, specificity, PPV and NPV According to Waller and Gotway (2004), the point of intersection of the two is selected as the standard in most cases and is also the point where the sensitivity and specificity are closest and interdependent. The ideal cut-off point should enable clinical differentiation, but a point with the smallest error is not necessarily the ideal cut-off point. The AUC is an index of the amount of information the test provides over its entire scoring range. At 14 points, the RUDAS-C exhibited a sensitivity of 002, a maximum specificity of 1, a PPV of 0 and an NPV of 0 in detecting MCI. When the scores were increased, the sensitivity increased, but the specificity decreased. According to the researcher’s estimation, at 24 points, the specificity (091) was lower than that at 23 points, but the sensitivity (079) was higher. When the score was >25 points, the specificity was too low. Therefore, the cut-off point was set at 24, where the scale exhibited a 79% chance of identifying patients with MCI and a 99% chance that this identification is accurate. In addition, the scale

AUC, area under curve; PPV, positive predictive value; NPV, negative predictive value; RUDAS, Rowland Universal Dementia Assessment Scale; ROC, receiver operating characteristic. *Optimal cut-off point and results.

the value (098) observed in 90 elderly participants from the same community, as determined by the original authors (Storey et al. 2004). The Kappa value of both scales was 088, whereas the intra-class coefficient of the original scale was 099 (Storey et al. 2004). The kappa value and ICC indicated appropriate inter-rater and internal reliability.

Validity The CVI of the RUDAS-C was 097, indicating that 97% of the content was deemed effective by the five experts. The items are clearly described, contextual relevant, specific to the concerned subject, include the necessary contents and © 2015 John Wiley & Sons Ltd Journal of Clinical Nursing, 24, 3118–3128

3125

94

C-W Chen et al.

reported that, in the early stages of dementia, a patient can suffer from orientation impairment that deteriorates severely during the course of the disease. Moreover, studies have shown that the early symptoms of an illness can be effectively detected by evaluating a participant’s visual and spatial perception (O’Brien et al. 2001, Kavcic & Duffy 2003, Mota Quental et al. 2009).

exhibited a 91% chance of identifying patients without MCI and a 96% chance that this identification is accurate. Therefore, the researcher believes that 24 is the optimal cutoff point for MCI assessment. At 14 points, the RUDAS-C had a sensitivity of 004, a maximum specificity of 1, a PPV of 0 and an NPV of 1 in detecting dementia. When the score was increased, the sensitivity increased, but the specificity decreased. According to the researcher’s estimation, at 22 points, the specificity (081) was lower than that at 20 and 21 points, but the sensitivity (076) was higher. When the score was >22 points, the specificity was too low. Therefore, the researcher set the cut-off point at 22, where the scale exhibited a 76% chance of identifying patients with dementia and an 83% chance that this identification is accurate. In addition, the scale exhibited an 81% chance of identifying patients with no dementia and a 91% chance that this identification is accurate. Therefore, the researcher believes that 22 is the optimal cut-off point for dementia assessment. The report on the original version of the RUDAS offered only an analysis of the sensitivity and specificity in dementia assessment. At 23 points, the original cut-off point recommended by the original author (Storey et al. 2004), the scale had a sensitivity and specificity of 89 and 98% respectively. Numerous studies have investigated the optimal cut-off points for the RUDAS (Table 1). For the Thai version, Limpawattana et al. (2012) set the cut-off point at 24 and observed a sensitivity and specificity of 79 and 61% respectively. When a cut-off point of 23 was set for the RUDAS-C, the specificity was low (69%). Therefore, 22 was determined to be the optimal cut-off point for assessing patients with a high risk of dementia; applying this cut-off point yielded a sensitivity and specificity of 76 and 81% respectively. Because participants with cognitive impairment often experience orientation impairment as well, items on orientation can be added to improve the sensitivity of the RUDAS-C. In addition, the four items involved in the memory tasks were tea leaves, vegetable oil, egg and soap, which are similar in nature. Therefore, including items that greatly vary in nature will increase the difficulty of the task and, thus, the sensitivity of the scale. This study revealed that the RUDAS-C has favourable sensitivity and specificity in MCI assessment. However, these aspects were not discussed in the original study. Our results revealed that at applying a cut-off point of 24 yielded a sensitivity and specificity of 079 and 091 in MCI assessment respectively. A cut-off point of 22 yielded moderate sensitivity (076) and specificity (081) in dementia assessment. The RUDAS-C is an accurate tool for detecting MCI. The RUDAS includes items on visuospatial orientation, which are uncommon in similar instruments. Certain studies have

Conclusion The RUDAS-C comprises six categories that cover multiple cognitive functions and can be completed in 10 minutes. It is easy to use and exhibits appropriate reliability, validity, sensitivity, specificity and predictive values. This is the first study to develop the reliability, validity, sensitivity, specificity and predictive values of the RUDAS-C as a screening tool for dementia in a Chinese-speaking population. Accurately detecting early dementia in older people from culturally and linguistically diverse backgrounds requires screening tools with favourable cross-cultural linguistic and appropriate sensitivity, specificity and predictive values. By using the scale, medical professionals can detect cognitive impairment during the prodromal phase or even earlier in various cultural settings and thus provide appropriate medication or cognitive training to slow the deterioration of cognitive ability and the progression of dementia. The major limitation to consider when using this instrument is the generalisability of the sample in which the measure was developed. The recommendations for future study include obtaining a large and representative sample for both dementia and non-dementia participants and conducting replication studies.

Relevance to clinical practice The appropriate sensitivity, specificity, and predictive values of the RUDAS-C indicate that this scale is a useful clinical tool for early screening and can be used as a standard measure for MCI detection and assessment in clinical settings. In addition, it provides a fast, accurate and sensitive means for facilitating the accurate diagnosis of dementia in clinical practice.

Disclosure The authors have confirmed that they meet the ICMJE criteria for authorship credit (www.icmje.org/ethical_1aut hor.html): (1) substantial contributions to conception and design or acquisition of data or analysis and interpretation of data, (2) drafting the article or revising it critically for important intellectual content, and (3) final approval of the version to be published. © 2015 John Wiley & Sons Ltd Journal of Clinical Nursing, 24, 3118–3128

3126

95

Original article

Funding The authors have no funding to disclose.


Conflicts of interest The authors declare no conflicts of interest.

References Albert MS, DeKosky ST, Dickson D, Dubois B, Feldman HH, Fox NC, Gamst A, Holtzman DM, Jagust WJ, Petersen RC, Snyder PJ, Carrillo MC, Thies B & Phelps CH (2010) The diagnosis of mild cognitive impairment due to Alzheimer’s disease: recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimer’s & Dementia 7, 270–279. American Psychiatric Association (2000) Diagnostic and Statistical Manual of Mental Disorders, 4th edn, Text Revision, Author, Washington, DC. Basic D, Khoo A, Conforti D, Rowland J, Vrantsidis F, Logiudice D, Hill K, Harry JAN, Lucero K & Prowse R (2009) Rowland Universal Dementia Assessment Scale, Mini-Mental State Examination and General Practitioner Assessment of Cognition in a multicultural cohort of community-dwelling older persons with early dementia. Australian Psychologist 44, 40–53. Brislin RW (1970) Back-translation for cross-cultural research. Journal of Cross-Cultural Psychology 1, 185–216. Burns N & Grove SK (2005) The concepts of measurement. In The Practices of Nursing Research: Conduct, Critique, & Utilization (Burns N & Grove SK eds). Saunders WB, Philadelphia, pp. 389–410. Choi SH, Na LD, Lee BH, Hahm DS, Jeong JH, Yoon SJ, Yoo KH, Ha CK, Han IW & Dementia Research Group (2001) Estimating the validity of the Korean version of expanded clinical dementia rating (CDR) scale. Journal of the Korean Neurological Association 19, 585–591. Chu LW, Pei CKW, Ho MH & Chan PT (1995) Validation of the abbreviated mental test (Hong Kong version) in the elderly medical patient. Hong Kong Medical Journal 1, 207–211. Ministry of Health and Welfare, Executive Yuan, Taiwan, R.O.C. (2013) 2013 statistics of causes of death. Available

at: http://www.mohw.gov.tw/EN/Ministry/Statistic_P.aspx?f_list_no=474&f od_list_no=5045&doc_no=45981 (accessed 29 June 2015). DePaulo JR Jr & Folstein MF (1978) Psychiatric disturbances in neurological patients: detection, recognition, and hospital course. Annals of Neurology 4, 225–228. Folstein MF, Folstein SE & McHugh PR (1975) “Mini-mental state”. A practical method for grading the cognitive state of patients for the clinician. Journal of Psychiatric Research 12, 189– 198. Glosser G, Wolfe N, Albert ML, Lavine L, Steele J, Caine D & Schoenberg B (1993) Cross-cultural cognitive examination: validation of a dementia screening instrument for neuroepidemiological research. Journal of the American Geriatrics Society 41, 931– 939. Guo NW, Liu HC, Wong PF, Liao KK, Yan SH, Lin KP, Chang CY & Hsu TC (1988) Chinese version and norms of the Mini-Mental State Examination. Taiwan Journal of Physical Medicine and Rehabilitation 16, 52–59. Iype T, Ajitha BK, Antony P, Ajeeth NB, Job S & Shaji KS (2006) Usefulness of the Rowland Universal Dementia Assessment scale in South India. Journal of Neurology, Neurosurgery & Psychiatry 77, 513–514. Kavcic V & Duffy CJ (2003) Attentional dynamics and visual perception: mechanisms of spatial disorientation in Alzheimer’s disease. Brain 126, 1173– 1181. Kua EH & Ko SM (1992) A questionnaire to screen for cognitive impairment among elderly people in developing countries. Acta Psychiatrica Scandinavica 85, 119–122. Lam SC, Wong YY & Woo J (2010) Reliability and validity of the abbreviated mental test (Hong Kong version) in residential care homes. Journal of the American Geriatrics Society 58, 2255– 2257.


Limpawattana P, Tiamkao S, Sawanyawisuth K & Thinkhamrop B (2012) Can Rowland Universal Dementia Assessment Scale (RUDAS) replace Minimental State Examination (MMSE) for dementia screening in a Thai geriatric outpatient setting? American Journal of Alzheimer’s Disease & Other Dementias 27, 254–259. Lynn MR (1986) Determination and quantification of content validity. Nursing Research 35, 382–385. Mast BT, Fitzgerald J, Steinberg J, MacNeill SE & Lichtenberg PA (2001) Effective screening for Alzheimer’s disease among older African Americans. The Clinical Neuropsychologist 15, 196– 202. McCulla MM, Coats M, Van Fleet N, Duchek J, Grant E & Morris JC (1989) Reliability of clinical nurse specialists in the staging of dementia. Archives of Neurology 46, 1210–1211. Morris JC (1993) The Clinical Dementia Rating (CDR): current version and scoring rules. Neurology 43, 2412– 2414. Mota Quental NB, Dozzi Brucki SM & Amodeo Bueno OF (2009) Visuospatial function in early alzheimer’s disease: preliminary study. Dementia & Neuropsychologia 3, 324–240. O’Brien HL, Tetewsky SJ, Avery LM, Cushman LA, Makous W & Duffy CJ (2001) Visual mechanisms of spatial disorientation in Alzheimer’s disease. Cerebral Cortex 11, 1083–1092. Petersen RC (2011) Mild cognitive impairment. The New England Journal of Medicine 364, 2227–2234. Plassman BL, Langa KM, Fisher GG, Heeringa SG, Weir DR, Ofstedal MB, Burke JR, Hurd MD, Potter GG, Rodgers WL, Steffens DC, McArdle JJ, Willis RJ & Wallace RB (2008) Prevalence of cognitive impairment without dementia in the United States. Annals of Internal Medicine 148, 427–434. Prince M, Bryce R, Albanese E, Wimo A, Ribeiro W & Ferri CP (2013) The

3127

96

C-W Chen et al. global prevalence of dementia: a systematic review and metaanalysis. Alzheimer’s & Dementia 9, 63–75. Rowland JT, Basic D, Storey JE & Conforti DA (2006) The Rowland Universal Dementia Assessment Scale (RUDAS) and the Folstein MMSE in a multicultural cohort of elderly persons. International Psychogeriatrics 18, 111– 120. Shadlen MF, Larson EB, Gibbons LE, Rice MM, McCormick WC, Bowen J, McCurry SM & Graves AM (2001) Ethnicity and cognitive performance among older African Americans, Japenese Americans, and Caucasians: the role of education. Journal of the American Geriatrics Society 49, 1371–1378.

Sperber AD (2004) Translation and validation of study instruments for cross-cultural research. Gastroenterology 126, S124–S128. Stojanovic M, Apostolovic M, Stojanovic D, Milosevic Z, Toplaovic A, Lakusic VM & Golubovic M (2014) Understanding sensitivity, specificity and predictive values. Vojnosanitetski Pregled 71, 1062–1065. Storey JE, Rowland JT, Basic D, Conforti DA & Dickson HG (2004) The Rowland Universal Dementia Assessment Scale (RUDAS): a multicultural cognitive assessment scale. International Psychogeriatrics 16, 13–31. Tiraboschi P, Hansen LA, Alford M, Masliah E, Thal LJ & Corey-Bloom J (2000) The decline in synapses and

cholinergic activity is asynchronous in Alzheimer’s disease. Neurology 55, 1278–1283. Tombaugh TN & Mcintyre NJ (1992) The Mini-Mental State Examination: a comprehensive review. Journal of the American Geriatrics Society 40, 922– 935. Waller LA & Gotway CA (2004) Applied Spatial Statistics for Public Health Data. John Wiley & Sons, Hoboken, NJ. World Health Organization (2012) Dementia Cases Set to Triple by 2050 but Still Largely Ignored. Available at: http://www.who.int/mediacentre/news/ releases/2012/dementia_20120411/en/ (accessed 26 January 2014).


3128

97