Redefining the Factor Structure of the Wechsler Memory Scale-III ...

Journal of Clinical and Experimental Neuropsychology 2002, Vol. 24, No. 5, pp. 574±585

1380-3395/02/2405-574$16.00 # Swets & Zeitlinger

Rede®ning the Factor Structure of the Wechsler Memory Scale-III: Con®rmatory Factor Analysis With Cross-Validation 1

Larry R. Price1, David Tulsky2, 3, Scott Millis2, 3, and Larry Weiss4

Southwest Texas State University, San Marcos, TX, USA, 2Kessler Rehabilitation Research and Education Corporation, West Orange, NJ, USA, 3University of Medicine and Dentistry of New Jersey, Newark, NJ, USA, and 4 The Psychological Corporation, San Antonio, TX, USA

ABSTRACT The purpose of this study was to revisit the underlying factor structure of the Wechsler Memory Scale-III. The WMS-III Technical Manual (Wechsler, 1997) presented ®ndings from con®rmatory factor analyses that support a three- and ®ve-factor solution with separate immediate and delayed memory factors. A rigorous structural equation modeling approach was used to examine the factor structure of the test. The results verify that a three-factor model composed of verbal (immediate and delayed), visual (immediate and delayed), and working memory factors accurately represents the factor structure of the WMS-III. Results were crossvalidated on an independent sample.

The Wechsler Memory Scale ± Third Edition (WMS-III; Wechsler, 1997) provides a means of assessing a broad range of cognitive abilities. The WMS-III was substantially revised from its predecessor ± the WMS±R, and some of the goals of the revision include improving the visual memory index, improving the measurement of delayed factors, and separating delayed recall from delayed recognition variables. A more complete review of the revision goals have been described in the WAIS-III±WMS-III Technical Manual (Wechsler, 1997) and will not be elaborated here. However, because of theoretical and structural changes that were advanced in the WMS-III, demonstration of the construct validity was an important task. Con®rmatory factor analyses were conducted to determine the latent factor structure of the test and to determine if the proposed structure was supported. Con®rmatory factor analyses presented in the WAIS-III±WMS-III Technical

Manual examined 5 measurement models with the entire WMS-III standardization sample (weighted N ˆ 1250) ± as well as running alternate analyses with the sample divided into three age bands (16±29, 30±64, and 65±89 years). The age ranges selected and used within each age band were based on sample size recommendations by Anderson and Gerbing (1984, 1988). Con®rmatory analyses using the three age bands were conducted in order to examine whether the factor structure changed as a function of age. The ®ve measurement models evaluated and reported in the WMS-III Technical Manual are: Model 1 (one-factor: general memory); Model 2 (two-factors: working memory and memory ± immediate and delayed); Model 3 (three-factors: working memory, immediate memory, and delayed memory); Model 4 (three factors: working memory, visual memory, and auditory memory); and Model 5 (®ve factors: working memory,

Address correspondence to: Larry R. Price, Ph.D., Dept. of EAPS, College of Education, #325 ASB South, Southwest Texas State University, San Marcos, TX 78666, USA. Tel.: ‡ 1-512-245-9654. Fax: ‡ 1-512-254-8872. E-mail: [email protected] Accepted for publication: February 6, 2002.

FACTOR STRUCTURE OF THE WMS-III

auditory immediate memory, visual immediate memory, auditory delayed memory, and visual delayed memory). The Technical Manual concluded that Model 4 best ®ts the data for the 16± 29 age group and Model 5 ®ts best for the remaining two age groups. These analyses were used as one piece of evidence to demonstrate the construct validity of the WMS-III factor structure. Millis, Malina, Bowers, and Ricker (1999) attempted to replicate the factor analyses using the correlation matrices presented in the manual. Unfortunately, there was not enough information provided in the WMS-III Technical Manual to replicate the analyses exactly and the authors pointed out that the variation in the degrees of freedom between the two sets of analyses indicate that the models used between the two sets of analyses were probably not identical. In fact, the analyses conducted by Millis et al. included very similar, but not exact combinations of subtests in the measurement models as those used in the original con®rmatory factor analytic studies of the WMS-III. Millis et al. also found that none of their analyses supported any models in which immediate and delayed memory subtests were speci®ed on separate factors. Millis et al. went on to explain their results in terms of multicollinearity between the immediate and delayed versions of the same subtest such that it would not be reasonable to expect factor analyses to support distinct immediate and delayed factors. In their conclusions, they suggest that the analyses be repeated replicating exactly the procedures and models that were used in the Technical Manual. They also point out that despite the negative ®ndings to support immediate and delayed constructs, it does not necessarily mean that this distinction should not be made. The authors go on to suggest repeating these analyses in clinical groups where immediate and delayed distinctions have proven to yield important diagnostic information. The purpose of this paper is to outline the procedure used in the original analyses so that the results could be replicated, to correct any errors that may have been made, to conduct more rigorous analyses, and to further understand the factor structure of the WMS-III.

575

Speci®cally, this study extends the work of the test developers, and of Millis et al. in three ways. First, we screened the WMS-III data for nonnormality and then adjusted if deemed necessary. This screening was not done in the previous factor analytic work on the WMS-III. Second, prior to analyzing the ®ve measurement models overall and by age band, the invariance of the factor structure was evaluated across the three age bands originally reported in the WMS-III Technical Manual. Third, cross-validation was conducted using the ®xed-parameter invariance method and the expected cross-validation index (ECVI) proposed by Browne and Cudeck (1989) for calibration and validation samples. Based on a recommendation by Browne and Cudeck (1989), ECVI point estimates and their corresponding 90% con®dence intervals were used in comparing the calibration and validation samples due to small sample sizes within age bands. METHOD Participants

Participants used in this study included the WMS-III standardization sample (weighted N ˆ 1250; The Psychological Corporation, 1997; Tulsky & Ledbetter, 2000). The original sample served as the calibration sample. This sample is the same one that was used for the majority of analyses in the WAIS-III±WMS-III Technical Manual (The Psychological Corporation, 1997) and was used in the original factor analyses.1 To further explore the hypothesized models in this study, an independent sample (N ˆ 858) was identi®ed and obtained from archived data and used for crossvalidation purposes. This second sample is composed of individuals who completed the standardization editions of the WAIS-III and WMS-III during the standardization process. However, their data was not used as part of the WMS-III standardization sample. Some of these individuals participated in validation

1

Tulsky and Ledbetter (2000) reported that a caseweighting methodology was used to ensure that the WMS-III sample was representative of the U.S. census population proportions. Such techniques are commonly used in test development. For a general description of caseweighting techniques and an example of how they might be used, the reader should review the WAIS-III Canadian Technical Manual (Gorsuch, 2000).

576

LARRY R. PRICE ET AL.

Table 1. Demographic Composition of the WMS-III Validation Sample (N ˆ 858). Age group

N

16±29 30±64 65±89

457 250 151

Total

858

Education (years) 8 or less 9±11 12 13±15 16 or more

N 245 184 169 117 143

Ethnicity African American Hispanic Other White

858

samples whereas other individuals did not meet the criteria speci®ed in the sampling matrix and served as extra ``oversample'' cases. The demographic characteristics of the validation sample are presented in Table 1.

Analytic Procedure A Priori Model Development and Speci®cation

Con®rmatory measurement model development for the calibration and validation samples involved using the total sample for each group and also by age bands. The following age bands were used: (a) 16±89 years, (b) 16±29 years, (c) 30±64 years, and (d) 65±89 years. Measurement models were constructed and analyzed using the WMS-III standardization sample. Subtest scaled scores were used, and included the primary subtests: Letter-Number Sequencing (LNS), Spatial Span (SS), Logical Memory Immediate (LM I Recall), Logical Memory Delayed (LM II Recall), Verbal Paired Associates Immediate (VPA I Recall), Verbal Paired Associates Delayed (VPA II Recall), Family Pictures Immediate (FP I Recall), Family Pictures Delayed (FP II Recall), Faces Immediate (FACE I Recall), Faces Delayed (FACE II Recall). The latent factors and error terms between the immediate and delayed measures on the same subtests were allowed to correlate. Next, the ®ve measurement models were analyzed to con®rm or dispute the results of the con®rmatory factor analyses previously conducted and reported in the WMSIII Technical Manual (p. 115) and by Millis et al. (1999). A competing models strategy was employed with the following models proposed: (1) a single-factor model (general factor; M1), (2) a two-factor oblique model (working memory & auditory memory; M2), a threefactor oblique model (immediate memory, delayed memory, & working memory; M3), a three-factor oblique model (auditory memory, visual memory, & working memory; M4), and a ®ve-factor oblique model (auditory immediate memory, auditory delayed memory, visual immediate memory, visual delayed memory, & working memory; M5). After analyses were conducted on the calibration sample, cross-validation validation was conducted on the sample (N ˆ 858).

N

Gender

N

191 124 61 482

Female Male

457 401

858

858

Model Estimation

Data on the one-factor, two-factor, three-factor, and ®ve-factor structure of the WMS-III were assessed using CFA, conducted with LISREL (Joreskog & Sorbom, 1999). As a crosscheck of the numerical estimation and statistical output generated by LISREL, AMOS (Arbuckle & Wothke, 1999), and the CALIS procedure (SAS Institute, 1989) were also used in follow-up analyses. The method of maximum likelihood was used to derive parameter estimates with all subsequent analyses performed on the covariance matrices. In order for models to be identi®ed, estimates on each latent variable were established at unity.

Assessment of Measurement Models

Assessment of model ®t is not a simple process and there exists no de®nitive way to assess how well a model ®ts the data (Bollen, 1989). Therefore, several ®t indices, beyond the overall chi-square test, were used as indicators of the goodness-of-®t of the measurement models. The ®t indices used are grouped into four categories: (1) absolute ®t measures; (2) incremental ®t measures; (3) parsimonious ®t measures; and (4) information-theoretic measures. The absolute measures of ®t used included: (a) the likelihood ratio chi-square statistic; (b) the rescaled (robust) chi-square statistic (2 ; Sartorra & Bentler, 1988 ); (c) adjusted goodnessof-®t index (AGFI); and (d) the root mean error square of approximation (RMSEA; Steiger, 1990). The incremental measures of ®t used was the nonnormed ®t index (NNFI; Bentler & Bonnet, 1980). The parsimonious measures of ®t included: (a) the comparative ®t index with an adjustment for model parsimony related to the degrees of freedom (CFI; Bentler, 1990), and (b) the relative chi-square (2 /df ; Wheaton, 1977). The information-theoretic measures of ®t included were the Akaike information criterion (AIC; Akaike, 1973, 1987), and the expected cross-validation index (ECVI). The rescaled (robust) chi-square statistic was used to correct for multivariate nonnormality in the data (see Table 2). The AGFI and CFI have values ranging from 0 to 1 with values above .90 indicating a good ®t of the empirical data to the implied model. The RMSEA

577

FACTOR STRUCTURE OF THE WMS-III

Table 2. Univariate Descriptive and Normality Statistics (Samplec ˆ 1250, Samplev ˆ 858). Subscale

Skewness Samplec

Lns Lm II recall Lm I Ss Vpa II recall Vpa I recall Fp II recall Face I recall Face II recall Fp I recall

Samplev

Kurtosis

Samplec

Samplev

Samplec

Samplev

M

SD

M

SD

Z-score

Z-score

Z-score

Z-score

10.01 10.03 10.02 10.00 10.02 9.99 10.03 9.99 10.02 10.00

3.09 3.00 2.99 2.98 2.99 3.02 3.04 3.01 2.98 3.01

9.29 9.24 9.11 9.73 9.54 9.48 9.67 9.79 9.80 9.60

3.24 3.35 3.40 3.10 3.10 3.15 3.17 3.12 3.01 3.05

ÿ2.49 ÿ2.35 ÿ3.39 ÿ4.16 ÿ6.71 ÿ0.80 ÿ1.81 3.15 1.49 0.14

ÿ0.19 ÿ2.59 ÿ3.04 ÿ4.04 ÿ8.18 ÿ1.19 ÿ1.77 3.78 2.03 ÿ0.01

ÿ0.02 ÿ0.22 ÿ0.11 0.00 ÿ0.43 ÿ0.47 ÿ0.34 ÿ0.24 0.04 ÿ0.42

ÿ0.16 ÿ1.25 ÿ1.93 0.35 ÿ0.80 ÿ1.37 ÿ3.94 ÿ0.49 ÿ0.77 ÿ4.27

12.19

7.65

6.18

6.62

Multivariate

Note. Lns ˆ Letter number sequencing, Lmd II recall ˆ Logical memory delayed, Lm I ˆ Logical memory immediate, Ss ˆ Spatial span, Vpa II recall ˆ Verbal paired associates delayed, Vpa I recall ˆ Verbal paired associates immediate, Fp II recall ˆ Family pictures delayed, Face I recall ˆ Faces immediate, Face II recall ˆ Faces delayed, Fp I recall ˆ Family pictures immediate. provides values that represent the goodness-of-®t of the model if it were estimated in the population. RMSEA values between .05 and .08 are viewed acceptable with lower values indicating a closer model ®t. The NNFI typically has values between 0 and 1, however NNFI indices are not limited to that range. NNFI values closer to 1 are considered to be indicative of a good model ®t to the data. Relative chi-square ratios of 2 to 1 or 3 to 1 are indicative of acceptable ®t between a hypothetical model and the sample data (Carmines & McIver, 1981). The Akaike information criterion is used for model comparison purposes. The AIC derives a composite measure of model ®t in consideration of the level of model complexity. The AIC assigns a greater penalty (larger value) to complex models. Competing models were compared and evaluated by examining differences in the rescaled chi-square, AGFI, NNFI, CFI, RMSEA, AIC, ECVI, and relative chi-square indices. Boundary solutions and positive-de®niteness of the model covariance matrices were examined in order to screen for inappropriate statistical solutions or model speci®cation errors. Finally, the t-values of individual parameter estimates and correlation values between latent factors were examined for inappropriate values.

Cross-Validation Methodology

The strategy adopted for model cross-validation was the invariance method and a comparison of ECVI point estimates with 90% con®dence intervals. The invariance method involves testing the invariance of factor loadings (M2), factor variance-covariance (M3), and

error uniqueness terms (M4). Comparison of model ®t was evaluated by examining the differences between the calibration and validation samples in relation to their respective absolute, incremental, parsimonious, and information theoretic ®t indexes. The ECVI point estimates and their 90% con®dence intervals were examined and compared for the calibration and validation samples. If the model cross-validates well, there should be little, if any, difference between ECVI point estimates for the calibration and validation samples.

RESULTS Univariate Descriptive Statistics Table 2 provides univariate descriptive statistics and multivariate normality values for the 10 WMS-III subtests. Statistics are provided for both the calibration and validation samples. Univariate skewness statistics indicated that the data were nonnormal (z values greater than 1:96) in 6 out of 10 subtests for both samples (see Table 2). Multivariate normality was assessed using Mardia's (1970) multivariate skewness and kurtosis statistic. Multivariate skewness was found to be statistically signi®cant (zˆ 12.19, p < :001) for the calibration sample, and (z ˆ 7.65, p < :001) for the validation sample. Multivariate kurtosis

578

LARRY R. PRICE ET AL.

was found to be statistically signi®cant (z ˆ 6.18, p < :01) for the calibration sample, and (z ˆ 6.62, p < :01) for the validation sample. Therefore, the assumption of multivariate and univariate normality was not tenable. Next, the invariance of the factor structure was evaluated across the three age bands originally reported in the WMS-III Technical Manual. The null hypothesis of parameter invariance was not rejected for the three age bands. The results indicated that the factor structure of the WMS-III was invariant across age bands. Tables 3 and 4 provide the goodness-of-®t statistics for the models estimated in this study using the calibration and validation samples overall and by age band. The overall chi-square statistic was statistically signi®cant … p < :001† for all models when using the total calibration sample (weighted N ˆ 1250) indicating that the data to model ®t was poor. Additionally, for the total calibration sample, the ratio of the chi-square statistic to the degrees of freedom exceeded the three to one ratio recommended by Wheaton et al. (1977). The three to one ratio was not exceeded when the models were ®tted by separate age bands. For the validation sample (N ˆ 858), the overall chi-square statistic was signi®cant ( p < :01) only for Model 1 in the ®rst, third, and overall age bands and for Model 2 in the total sample. Conversely, using the validation sample, the majority of the models were considered to have acceptable ®t to the data. Since the chi-square statistic is sensitive to sample size, examination of other indices was necessary to explore the ®t of the ®ve models to the data for both samples. Examination of the AGFI, NNFI, and CFI revealed an acceptable ®t to the data with values exceeding .92, but they provided little information for discerning which model best ®t the data. Next, using the RMSEA and the AIC provided additional information for clarifying which model best ®t the data. Across all age bands RMSEA and AIC values were lowest for Model 4 (three factors: auditory memory, visual memory, & working memory). The RMSEA values for Models 4 and 5 were in the acceptable range and were nearly identical. However, the AIC value for Model 4 was 14.63 points lower than the AIC value for Model 5, indicating that Model 4

was superior to Model 5. The ECVI point estimates and con®dence intervals for Models 4 and 5 were average (above .09) indicating moderate support for model cross-validation. In both samples, Models 3 and 5 were ¯agged for having nonpositive covariance matrices and boundary solution violations by LISREL. The same results were generated and con®rmed using AMOS and the SAS CALIS procedure. The boundary solution error message was triggered by correlation estimates of .99 or greater between the immediate and delayed factors in Model 3 overall and across all age bands. In the ®ve-factor model, using the calibration sample 30±64 age band, a boundary solution error message was triggered by a correlation estimate of 1.05 between the visual immediate and visual delayed factors and an estimate of 1.01 between the auditory immediate and auditory delayed factors. For the remaining age bands in the calibration sample, correlation estimates greater than .97 were observed between the visual immediate and visual delayed factors and auditory immediate and auditory delayed factors, triggering boundary solution errors. For the ®vefactor model using the validation sample age band 30±64, a correlation estimate of 1.00 between the visual immediate and visual delayed factors and an estimate of 1.00 between the auditory immediate and auditory delayed factors triggered a boundary error. For all remaining age bands in the validation sample, correlation estimates between the visual immediate and visual delayed factors and for the auditory immediate and auditory delayed factors were greater than .96. The correlation values greater than 1.00 caused all three programs to display nonpositive-de®nite covariance matrix messages. Boundary solution errors were generated by covariance matrix values than included values of zero, as was often the case when the correlation estimates between factors exceeded .98. The results of these analyses do not support the viability of Models 3 or 5. Although there are several possible explanations for inadmissible estimates, the results of this study posit that the problematic solutions emerged from the linear dependency among the immediate and delayed subtests and model speci®cation error.

Table 3. WMS-III Calibration Sample Goodness-of-Fit Statistics for Con®rmatory Factor Analyses. Age band by model

Chi-square

Chi-square (robust)

df

2 /df

p

AGFI

RMSEA NNFI

CFI

AIC

ECVI

90% ECVI

150.78 90.61 90.51 73.50 65.63

142.73 86.72 85.55 72.55 63.74

31.00 30.00 28.00 28.00 21.00

4.60 2.89 3.05 2.59 3.03

0.00 0.00 0.00 0.00 0.00

0.88 0.92 0.92 0.93 0.92

0.09 0.07 0.07 0.06 0.07

0.93 0.96 0.96 0.97 0.96

0.95 0.97 0.97 0.98 0.98

190.73 136.72 139.55 126.55 131.74

0.48 0.34 0.35 0.32 0.33

0.39±0.58 0.28±0.42 0.29±0.43 0.26±0.39 0.28±0.40

Ages 30±64 (N ˆ 400) 1 2 *3 4 *5

58.01 40.32 38.96 36.05 22.25

58.44 40.31 38.49 35.91 21.71

31.00 30.00 28.00 28.00 21.00

1.88 1.34 1.37 1.28 1.03

0.00 0.02 0.09 0.14 0.42

0.95 0.96 0.96 0.97 0.97

0.04 0.03 0.03 0.03 0.00

0.98 0.99 0.99 0.99 1.00

0.99 1.00 1.00 1.00 1.00

106.44 90.31 110.00 89.91 89.71

0.27 0.23 0.23 0.23 0.22

0.22±0.33 0.20±0.28 0.21±0.28 0.21±0.27 0.22±0.26

Ages 65±89 (N ˆ 450) 1 2 *3 4 *5

119.81 76.75 69.12 61.73 37.38

116.40 72.50 65.73 55.72 33.82

31.00 30.00 28.00 28.00 21.00

3.75 2.41 2.34 1.99 1.61

0.00 0.00 0.00 0.00 0.04

0.91 0.94 0.94 0.95 0.95

0.08 0.06 0.05 0.04 0.04

0.95 0.97 0.97 0.98 0.98

0.97 0.98 0.98 0.99 0.99

164.40 122.50 119.73 109.72 101.82

0.37 0.27 0.27 0.24 0.26

0.30±0.45 0.23±0.34 0.22±0.33 0.21±0.30 0.22±0.31

Ages 16±89 (N ˆ 1250) 1 2 *3 4 *5

243.74 126.63 123.54 93.80 64.54

238.98 124.68 121.18 90.16 61.53

31.00 30.00 28.00 28.00 21.00

7.70 4.15 4.32 3.22 2.93

0.00 0.00 0.00 0.00 0.00

0.93 0.96 0.96 0.97 0.97

0.07 0.05 0.05 0.04 0.03

0.96 0.98 0.98 0.99 0.99

0.97 0.99 0.99 0.99 0.99

286.98 174.68 175.18 144.16 129.53

0.23 0.14 0.14 0.12 0.10

0.19±0.27 0.12±0.17 0.12±0.17 0.09±0.14 0.08±0.13

FACTOR STRUCTURE OF THE WMS-III

Ages 16±29 (N ˆ 400) 1 2 *3 4 *5

Note. Model 1 ˆ one-factor; Model 2 ˆ two-factor oblique (auditory & working); Model 3 ˆ three-factor oblique model (immediate, delayed, & working with separate immediate and delayed subtests); Model 4 ˆ three-factor oblique model (auditory, visual, & working with combined immediate and delayed subtests); Model 5 ˆ ®ve-factor oblique model (auditory immediate, visual immediate, auditory delayed, visual delayed, working memory). *Inadmissible model estimates & violation of boundary solutions.

579

580

Table 4. WMS-III Validation Sample Goodness-of-Fit Statistics for Con®rmatory Factor Analyses. Age band by model

p

AGFI

31.00 30.00 28.00 28.00 21.00

2.61 1.57 1.51 1.54 1.11

0.00 0.03 0.04 0.03 0.33

0.94 0.96 0.96 0.96 0.97

0.06 0.03 0.03 0.03 0.01

38.71 23.91 19.91 21.03 16.70

31.00 30.00 28.00 28.00 21.00

1.25 0.79 0.71 0.75 0.80

0.16 0.78 0.87 0.82 0.73

0.94 0.96 0.97 0.97 0.96

74.95 54.28 50.78 39.28 30.22

70.88 47.10 43.50 35.99 27.39

31.00 30.00 28.00 28.00 21.00

2.28 1.57 1.55 1.28 1.30

0.00 0.02 0.03 0.14 0.16

122.02 126.63 44.63 38.23 19.70

120.13 124.68 41.44 36.69 18.45

31.00 30.00 28.00 28.00 21.00

3.87 4.15 1.48 1.31 0.89

0.00 0.00 0.05 0.13 0.62

Chi-square (robust)

Ages 16±29 (N ˆ 448) 1 2 *3 4 *5

84.64 49.02 44.32 44.72 24.79

80.93 47.00 42.37 43.10 23.36

Ages 30±64 (N ˆ 259) 1 2 *3 4 *5

39.95 25.57 21.92 22.67 18.85

Ages 65±89 (N ˆ 151) 1 2 *3 4 *5 Ages 16±89 (N ˆ 858) 1 2 *3 4 *5

df

RMSEA NNFI

CFI

AIC

ECVI

90% ECVI

0.98 0.99 0.99 0.99 1.00

0.98 0.99 0.99 0.99 1.00

128.93 97.00 96.37 97.10 91.36

0.27 0.20 0.20 0.20 0.19

0.22±0.33 0.17±0.25 0.17±0.24 0.17±0.25 0.18±0.22

0.03 0.00 0.00 0.00 0.00

0.99 1.00 1.01 1.01 1.00

0.99 1.00 1.00 1.00 1.00

86.71 73.91 73.91 75.03 84.70

0.34 0.31 0.33 0.32 0.34

0.31±0.41 0.31±0.34 0.33±0.35 0.32±0.34 0.34±0.38

0.84 0.88 0.88 0.90 0.90

0.09 0.06 0.06 0.04 0.04

0.93 0.96 0.96 0.98 0.98

0.95 0.97 0.98 0.99 0.99

118.88 97.10 97.50 89.99 95.39

0.79 0.65 0.65 0.60 0.64

0.65±0.98 0.55±0.80 0.56±0.80 0.55±0.73 0.59±0.75

0.95 0.96 0.98 0.98 0.99

0.06 0.05 0.02 0.02 0.00

0.98 0.98 1.00 1.00 1.00

0.98 0.99 1.00 1.00 1.00

168.13 174.68 95.44 90.69 86.45

0.20 0.14 0.11 0.11 0.10

0.16±0.24 0.12±0.17 0.09±0.14 0.09±0.13 0.10±0.12

Note. Model 1 ˆ one-factor; Model 2 ˆ two-factor oblique (auditory & working); Model 3 ˆ three-factor oblique model (immediate, delayed, & working with seperate immediate and delayed subtests); Model 4 ˆ three-factor oblique model (auditory, visual, & working with combined immediate and delayed subtests); Model 5 ˆ ®ve-factor oblique model (auditory immediate, visual immediate, auditory delayed, visual delayed, working memory). *Inadmissible model estimates & violation of boundary solutions.

LARRY R. PRICE ET AL.

2 /df

Chi-square

Table 5. Standardized Parameter Estimates for Model 4 (N ˆ 1250; N ˆ 858). WMS-III subtest

Visual memory Calibration sample

0.78 0.77 0.63 0.66

Working memory 0.84 0.55

0.79 0.37 0.39 0.75

R2 0.64 0.60 0.54 0.32 0.37 0.42 0.60 0.14 0.14 0.56

Auditory memory

Visual memory Validation sample

0.76 0.77 0.64 0.69

Working memory

R2

0.83

0.68 0.58 0.59 0.32 0.42 0.48 0.48 0.15 0.18 0.43

0.57 0.69 0.38 0.43 0.66

Note. Lns ˆ Letter number sequencing, Lm II recall ˆ Logical memory delayed, Lm I ˆ Logical memory immediate, Ss ˆ Spatial span, Vpa II recall ˆ Verbal paired associates delayed, Vpa I recall ˆ Verbal paired associates immediate, Fp II recall ˆ Family pictures delayed, Face I recall ˆ Faces immediate, Face II recall ˆ Faces delayed, Fp I recall ˆ Family pictures immediate.

FACTOR STRUCTURE OF THE WMS-III

Lns Lm II recall Lm I Ss Vpa II recall Vpa I recall Fp II recall Face I recall Face II recall Fp I recall

Auditory memory

581

582

LARRY R. PRICE ET AL.

Model speci®cation error was examined in the calibration and validation samples, overall and by age bands, by reviewing the correlation estimates between the subtests and then evaluated in relation to the structure of measurement Models 3 and 5. The results mirror those found by Millis et al. (1999, p. 90) in that the correlation between subtests of different factors were higher than those of the same factor signaling model speci®cation error. In consideration of all ®ve models, Model 4 based on separate constructs of auditory, visual, and working memory, was determined to be the best model. This determination was made based on review of ®t indices and parameter estimates derived from the calibration and validation samples. Table 5 includes the standardized parameter estimates by sample expressed as correlations and squared correlations (R2) between the latent factors and each subtest. The Faces and Family Picture subtests displayed low correlational statistics in both samples and thus only minimally contributed toward explaining the factor structure of the WMS-III. The between-factor correlation estimates are presented in Table 6. Cross-Validation of the WMS-III Table 7 provides the results of the cross-validation procedures conducted on Model 4. The ®rst model tested the amount of discrepancy between Model M1 based on the calibration sample and Model M2 based on the validation sample. A chi-square difference test supported the null hypothesis with model ®t measures being nearly identical. Next, several analyses examined the hypotheses of parameter and error invariance using Model 4 with the validation sample. Models M2 through M4 imposed increasing constraints ± forcing factor loadings, variance-covariance, and unique-

ness to be equal to calibration sample estimates. A comparison between the least stringent model (M2) and the next increasingly constrained model (M3), revealed a signi®cant chi-square difference test although ®t indices were nearly identical. A comparison between Model M3 and Model M4 also yielded a signi®cant result with similar ®t statistics. These results provide support for the hypothesis of equal factor loadings but not equality of error variances between the calibration and validation samples. To add additional support for model crossvalidation, ECVI point estimates and 90% con®dence intervals were calculated using the validation sample and compared to the calibration sample estimates for invariance Models M2 through M4. Minimal discrepancies were observed between ECVI point estimates and con®dence interval ranges derived from the calibration sample in comparison with ECVI estimates from invariance Models M2 through M4. The results of the ECVI comparisons lend further support to the accuracy of the cross-validation results. CONCLUSIONS The purpose of this study was to further investigate the recent ®nding that factor analyses of the WMS-III standardization sample do not support separate immediate and delayed memory factors (Millis et al., 1999). The current results support the ®ndings of Millis et al., by con®rming that a three-factor structure composed of auditory memory, visual memory, and working memory provides the best ®t to the data and indicate that the results previously reported by Wechsler (1997) were overstated. Most importantly, the analyses conducted here used the same data set

Table 6. Interfactor Correlations. Calibration sample Auditory memory Visual memory Working memory

Validation sample

1

2

3

1

2

3

1.00 0.74 0.65

± 1.00 0.49

± ± 1.00

1.00 0.82 0.72

± 1.00 0.57

± ± 1.00

Table 7. Model Cross-Validation Using Parameter Invariance Methods. Method Using validation sample M1 Free estimated model Invariance M2 Equality of factor loadings M3 Equality of factor loadings, factor variance/covariance M4 Equality of factor loadings, factor variance/covariance and error variances

2

df

Fa

AGFI

AIC

CFI

NNFI

RMSEA

ECVI

90% CI

90.16

28.00

0.07

0.97

144.16

0.99

0.99

0.04

0.12

0.09; 0.14

113.86 141.35

35.00 38.00

0.33 0.34

0.96 0.96

153.86 175.35

0.98 0.98

0.98 0.99

0.05 0.06

0.18 0.20

0.14; 0.22 0.16; 0.25

171.52

48.00

0.33

0.96

185.52

0.97

0.98

0.05

0.21

0.17; 0.27

FACTOR STRUCTURE OF THE WMS-III

Note. AGFI ˆ adjusted goodness of ®t index, AIC ˆ Akaike information criterion, CFI ˆ comparative ®t index, NNFI ˆ Nonnormed ®t index, ECVI ˆ expected cross-validation index, RMSEA ˆ root mean square error of approximation. a Maximum likelihood ®tting function.

583

584

LARRY R. PRICE ET AL.

and identical measurement model speci®cations as did the original analyses, and hence, the results provide evidence beyond that of Millis et al. that the results initially reported by Wechsler (1997) were untenable. Analyses with different samples (e.g., a group of individuals with neuropsychological damage), on alternative models, or with alternative subtests may yield different results. Also, consistent with the results reported by Millis et al., the current study identi®ed a nonpositive de®nite matrix when immediate and delayed factors were forced to separate. This is likely due to the high correlation between the immediate and delayed conditions of the same memory task observed in the nonclinical sample used in these studies. Given the intact neurocognitive functioning of this sample, immediate and delayed memory functions could be expected to signi®cantly covary in certain instances. Based, in part, upon these results, Tulsky, Ivnik, Price, and Wilkins (in press) have created new auditory and visual composite memory index scores for the WMS-III that do not separate the immediate and delayed tasks. Such scores may not have the same degree of clinical utility, however. In fact, distinctions between immediate and delayed memory factors have been useful in measuring individuals with neuropsychological, and in particular, cortical disorders (Baker, 1996; Cummings, 1986; Cummings & Benson, 1988; Paulsen, Salmon, & Monsch, 1995) and, hence, it is unclear if the Tulsky and Wilkins norms would have the same clinical utility. Moreover, it is also unclear if models that differentiate between immediate and delayed memory would not have the same speci®cation errors found here in a sample of individuals with cortical disorders. For this reason, it does not seem prudent to stop using the immediate and delayed composite scores in clinical interpretation of memory functioning. The original sampling protocol and development of the WMS-III produced limited data for persons with neurological disorders. Therefore, one area for future research on the factor structure of the WMS-III should include rigorous measurement modeling using clinical samples. In fact, the lack of discrimination between separate immediate and delayed constructs observed

here may have been the result of using the normal standardization sample. Separate immediate and delayed constructs may emerge in a clinical sample because the components of memory may vary as a function of cerebral compromise, there by changing the factor structure of the test. For example, patient groups with diffuse damage may demonstrate different performance patterns across tests. Similarly, patient groups with lateralized cerebral damage may exhibit slower performance in one domain as compared to another. Finally, the factor structure of the WMS-III may also be accurately represented by using entirely separate models of immediate and delayed memory (G. Larrabee, personal communication, August 2001). Larrabee argues that the immediate and delayed factors are so highly correlated, that including them in the same model causes an inadmissible method variance problem, and that this problem is manifested as model misspeci®cation. As of this writing, we are developing and testing separate immediate and delayed con®rmatory measurement models to explore their viability as competing models of the factor structure of the WMS-III (Price & Tulsky, unpublished data). The results of these analyses are forthcoming. REFERENCES Akaike, H. (1973). Information theory and an extension of the maximum lilihood principle. In B.N. Petrov & F. Csaki (Eds.), Proceedings of the 2nd International Symposium on Information Theory (pp. 267± 281). Budapest: Akademiai Kiado. Akaike, H. (1987). Factor analysis and AIC. Psychometrika, 52, 317±332. Anderson, J.C., & Gerbing, D.W. (1984). The effects of sampling error on convergence, improper solutions and goodness-of-®t indices for maximum likelihood con®rmatory factor analysis. Psychometrika, 49, 155±173. Anderson, J.C., & Gerbing, D.W. (1988). Structural equation modeling in practice: A review and recommended two-step approach. Psychological Bulletin, 103, 411±423. Arbuckle, J., & Wothke, W. (1999). AMOS 4 user's reference guide. Chicago: Smallwaters Corporation. Baker, J.G. (1996). Memory and emotion processing in cortical and subcortical dementia. Journal of General Psychology, 123, 185±191.

FACTOR STRUCTURE OF THE WMS-III

Bentler, P.M., & Bonnet, D.G. (1980). Signi®cance tests and goodness in the analysis of covariance structures. Psychological Bulletin, 88, 588±606. Bollen, K.A. (1989). Structural equations with latent variables. New York: Wiley. Browne, M.W., & Cudeck, R. (1989). Single sample cross-validation indices for covariance structures. Multivariate Behavioral Research, 24, 445±455. Browne, M., & Cudeck, R. (1993). Alternative ways of assessing model ®t. In K.A. Bollen & J.S. Long (Eds.), Testing Structural Equation Models (pp. 136±162). Newbury Park, CA: Sage. Burton, D.B., Mittenberg, W., & Burton.C.A. (1993). Con®rmatory factor analysis of the Wechsler Memory Scale ± Revised standard sample. Archives of Clinical Neuropsychology, 8, 467±475. Carmines, E.G., & McIver, J.P. (1981). Analyzing models with unobserved variables. In G.W. Borhrnstedt & E.F. Borgatta (Eds.), Social measurement: Current issues. Beverly Hills: Sage. Cudeck, R., & Browne, M.W. (1983). Cross-validating covariance structures. Multivariate Behavioral Research, 18, 147±167. Cummings, J.L. (1986). Subcortical dementia: Neuropsychology, neuropsychiatry, and pathophysiology. British Journal of Psychiatry, 149, 682±697. Cummings, J.L., & Benson, D.F. (1988). Psychological dysfunction accompanying subcortical dementias. Annual Review of Medicine, 39, 53±61. Gorsuch, R. (2000). In Wechsler, D., Wechsler Adult Intelligence Scale, Third Edition Canadian Technical Manual. San Antonio, TX: The Psychological Corporation. Joreskog, K.G., & Sorbom, D. (1999). LISREL 8 user's reference guide. Chicago: Scienti®c Software International. Mardia, K.V. (1970). Measures of multivariate skewness and kurtosis with applications. Biometrika, 57, 519±530. Millis, S.R., Malina, A.C., Bowers, D.A., & Ricker, J.H. (1999). Con®rmatory factor analysis of the

585

Wechsler Memory Scale-III. Journal of Clinical and Experimental Neuropsychology, 21, 87±93. Paulsen, J.S., Salmon, D.O., & Monsch, A.U. (1995). Discrimination of cortical from subcortical dementias on the basis of memory and problem-solving tests. Journal of Clinical Psychology, 51, 48±58. Price, L., & Tulsky, D. (2002). [Developing and testing measurement models for the WMS-III with separate immediate and delayed memory]. Unpublished raw data. SAS Institute. (1989). SAS/STAT user's guide (Vol. 1). Cary: SAS Institute. Sartorra, A., & Bentler, P.M. (1988). Scaling corrections for chi-square statistics in covariance structure analysis. Proceedings of the Business and Economic Statistics Section of the American Statistical Association (pp. 308±313). Alexandria, VA: American Statistical Association. Steiger, J.H. (1990). Structural model evaluation and modi®cation: An interval estimation approach. Multivariate Behavioral Research, 25, 173±180. Tulsky, D.S., Ivnik, R.J., Price, L.R., & Wilkins, C. (in press). A joint approach to interpreting the WAIS-III and WMS-III: The development of new index scores. In D.S. Tulsky (Ed.), Clinical interpretation of the WAIS-III and WMS-III. San Diego: Academic Press. Tulsky, D., & Ledbetter, M. (2000). Updating to the WAIS±III and WMS±III: Considerations for Research and Clinical Practice. Psychological Assessment, 12, 253±262. Wechsler, D. (1987). Wechsler Memory Scale ± Revised. San Antonio, TX: The Psychological Corporation. Wechsler, D. (1997). Wechsler Adult Intelligence ScaleIII/Wechsler Memory Scale ± Third Edition Technical Manual. San Antonio, TX: The Psychological Corporation. Wheaton, B., Muthen, B., Alwin, D.F., & Summers, G.F. (1977). Assessing reliability and stability in panel models. In D.R. Heise (Ed.), Sociology methodology 1977 (pp. 84±136). San Francisco: Jossey-Bass.