The Effect of Error Correlation on Interfactor

This article was downloaded by: [Texas Technology University], [Mr Roy Howell] On: 30 January 2012, At: 07:31 Publisher: Psychology Press Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Structural Equation Modeling: A Multidisciplinary Journal Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/hsem20

The Effect of Error Correlation on Interfactor Correlation in Psychometric Measurement a

a

Peter H. Westfall , Kevin S. S. Henning & Roy D. Howell

a

a

Jerry S. Rawls College of Business Administration, Texas Tech University Available online: 23 Jan 2012

To cite this article: Peter H. Westfall, Kevin S. S. Henning & Roy D. Howell (2012): The Effect of Error Correlation on Interfactor Correlation in Psychometric Measurement, Structural Equation Modeling: A Multidisciplinary Journal, 19:1, 99-117 To link to this article: http://dx.doi.org/10.1080/10705511.2012.634726

PLEASE SCROLL DOWN FOR ARTICLE Full terms and conditions of use: http://www.tandfonline.com/page/terms-and-conditions This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. The publisher does not give any warranty express or implied or make any representation that the contents will be complete or accurate or up to date. The accuracy of any instructions, formulae, and drug doses should be independently verified with primary sources. The publisher shall not be liable for any loss, actions, claims, proceedings, demand, or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with or arising out of the use of this material.

Downloaded by [Texas Technology University], [Mr Roy Howell] at 07:31 30 January 2012

Structural Equation Modeling, 19:99–117, 2012 Copyright © Taylor & Francis Group, LLC ISSN: 1070-5511 print/1532-8007 online DOI: 10.1080/10705511.2012.634726

The Effect of Error Correlation on Interfactor Correlation in Psychometric Measurement Peter H. Westfall, Kevin S. S. Henning, and Roy D. Howell Jerry S. Rawls College of Business Administration, Texas Tech University

This article shows how interfactor correlation is affected by error correlations. Theoretical and practical justifications for error correlations are given, and a new equivalence class of models is presented to explain the relationship between interfactor correlation and error correlations. The class allows simple, parsimonious modeling of error correlations via prespecifying reliabilities. Within the class, the correlation between latent factors can be as high as 1.0, and as low as the correlation between certain component scores. The models are indistinguishable in terms of parameter parsimony, identifiability, and fit statistics, implying that interfactor correlation is not identifiable within the class. The existence of the class is problematic for psychometric measurement, because estimates of interfactor correlation form the foundation of much of the literature. Keywords: confirmatory factor analysis, fit statistics, latent variable, measurement error, structural equation models

Psychometric measurement theory often seeks to describe the relationships among observable measures and unobservable theoretical (or latent) variables, and further to describe relationships between the latent variables themselves, often using structural equation models. It is commonly thought that use of such models corrects for bias of estimates of interfactor correlation that use manifest scores such as summates (Bollen, 1989, 2002; Kaplan, 2008). A fundamental problem that we highlight in this article is that the models cannot be assumed to correct for measurement error without making a fundamental, and untestable, assumption about the error correlation structure. Specifically, we show that the true interfactor correlation can be anything in the interval Œ¥L ; 1 for a simple, identifiable equivalence class of structural models with nonzero error correlations; the value of ¥L is the correlation between certain manifest scores. Model equivalence occurs when two or more distinct substantive relationships among the latent variables give the same indicator covariance matrix, which implies that models cannot be Correspondence should be addressed to Peter H. Westfall, Jerry S. Rawls College of Business Administration, Texas Tech University, Lubbock, TX 79409-3193, USA. E-mail: [email protected]

99

Downloaded by [Texas Technology University], [Mr Roy Howell] at 07:31 30 January 2012

100

WESTFALL, HENNING, HOWELL

distinguished statistically by the indicator covariances. A sizable proportion of the theoretical literature concerning structural equation models within the psychometric community has to do with this problem (e.g., Hershberger, 1994; Lee & Hershberger, 1990; MacCallum, Wegener, Uchino, & Fabrigar, 1993; McDonald, 2004; Raykov & Marcoulides, 2001; Raykov & Penev, 1999; Stanghellini, 1997; Stelzl, 1986). However, Steiger (2001) reported that discussion of the issue remains superficial or nonexistent in many popular structural equation modeling (SEM) textbooks. Our work concerns equivalent models that arise from allowing nonzero error correlations, a problem that has received much less attention than models that are equivalent because of reversed or alternative paths. The measurement model that we consider assumes that manifest measurements Y follow Y D ƒ˜ C ©

(1)

where ƒ is a .p  1/ vector of constants, ˜ is a random scalar (the latent factor) with unit variance, C ov.˜; ©/ D 0.1p/, and C ov.©/ D ‚, commonly assumed to be a diagonal matrix, although not in this article. Throughout this article all random variables are assumed, without loss of generality, to have zero means. Multivariate normality is also assumed, but only to make identifiability arguments tidy, as the distribution of the data is completely determined by C ov.Y / in this case. The assumption C ov.˜; ©/ D 0 is true when ƒ has the regression interpretation such that E.Y j˜/ D ƒ˜, and we make this assumption throughout (but see Grayson, 2006; Krijnen, 2006; Seddon, 1988, for consequences otherwise). On the other hand, it is not obviously true that errors should be uncorrelated, making ‚ diagonal. In this article we question the usual assumption that the errors are uncorrelated; we view it as a model choice of the researcher, rather than as a physical requirement of the measurement system. It is true that the covariance parameters are not identifiable unless constraints are imposed on ‚, so the choice of 0s for error correlations is practical. However, the researcher can just as easily choose some other number, such as 0:2, and retain identifiability. We ask and answer the question, “When correlations other than zero are chosen, what happens to the interfactor correlation?”

CORRELATED ERRORS Theoretical Considerations Error correlation is inextricably linked with the definition of ˜. We assume reflective models (Equation 1) with the regression interpretation of ƒ. The case where ˜ is assumed to preexist the measures Y is called the Platonic definition by Sobel (1994) and the entity realism definition by Borsboom, Mellenbergh, and van Heerden (2003). Here, ˜ is a numerical characteristic of the measured observational unit, one that preexists whatever measures Yi happen to be taken on that unit. In this case, there is no reason to assume uncorrelated errors. As noted by Bollen (2002), “technological or conceptual developments can occur that might make possible the measurement of variables that previously were treated as unmeasurable” (p. 608). Assuming such measurement has taken place on a sample of n subjects, the assumption of uncorrelated errors can be tested using the observed data fY i ; ˜i g; i D 1; : : : ; n, by computing the partial correlation matrix of Y j˜ and testing for its equality to the identity matrix. On its face the

Downloaded by [Texas Technology University], [Mr Roy Howell] at 07:31 30 January 2012

EFFECT OF ERROR CORRELATION ON INTERFACTOR CORRELATION

101

assumption is dubious because it is rare that correlation matrices (partial or otherwise) are precisely equal to identity matrices. This point of view was echoed by Sobel (1994), who noted that “with the Platonic conception of [˜], : : : the off-diagonal elements of [C ov.Y j˜/] are not a priori constrained to 0” (p. 25). Rae (2006) notes further examples of error correlation in the Platonic case. Further, the recent literature that is highly critical of null hypothesis significance testing also suggests that the assumption is almost certainly false: To assert that the partial correlation matrix is the identity matrix is a precise null hypothesis, one that is equally implausible as, for example, the precise null hypothesis H0 W Males D Females in a gender comparison study of salaries (see also Meehl, 1997; Nickerson, 2000; and references therein for discussions of the fallacy of testing precise null hypotheses). Finally, the lack of fit of the observed covariance matrix to the implied form ƒƒ0 C ‚, where ‚ is diagonal, is well acknowledged in the SEM literature (e.g., MacCallum, 2003). Achieving acceptable fit often suggests that errors should be correlated rather than uncorrelated. In the case where ˜ is defined by component analysis (McDonald, 1996; Schönemann & Steiger, 1976), ˜ is assumed to be a linear combination of the observed measures, ˜ D “0 Y , but the observed measures Y are still predicted in terms of this component as in Equation 1. Enforcing unit variance on ˜ implies that “0 †“ D 1, and by regression of Y on ˜ D “0 Y we have ƒ D C ov.Y ; “0 Y /fC ov.“0 Y /g 1 D .†“/.1/ 1 D †“, and hence © D Y †““0 Y . The covariance matrix of © is thus given by C ov.©/ D † †““0 †, which cannot be diagonal because there is a perfect linear dependency among the components of ©: Var .“0 ©/ D “0 .† †““0 †/“ D 1 1 D 0. As a generalization of component analysis, ˜ might be assumed to be a linear combination of the observed measures, but with some error, as in the representations of indeterminate factors (Steiger, 1994, Equation 4). Here we assume ˜ D “0 Y C •, where Var .•/ D ¢•2 , and where • is independent of Y . Still, the model in Equation 1 is employed, where the observed data Y are predicted in terms of such an ˜. Because Var .˜/ D 1, we must have “0 †“ C ¢•2 D 1. Regression again implies ƒ D †“, and the covariance matrix of © is again † †““0 †, which is nondiagonal in general. However, if † happens to admit the representation † D LL0 C ‰, for some L .p  1/ and some diagonal ‰, then C ov.©/ D † †““0 † is indeed diagonal (and equal to ‰), in the special case where “0 D L0 .LL0 C ‰/ 1 and ¢•2 D 1 L0 .LL0 C ‰/ 1 L. But there are two problems with this argument for the plausibility of a diagonal C ov.©/. The first is related to the hypothesis testing argument: The presumption that † happens to admit the representation † D LL0 C ‰ is an argument for a precise null hypothesis H0 W † D LL0 C ‰, which states that the p.p C 1/=2 free elements of the covariance matrix † happen to lie precisely in a lower dimensional subspace (assuming p > 3) having dimension 2p (the number of free elements in L and ‰ ). That the elements should lie precisely in a lower dimensional subspace is unlikely, as unlikely as it is that the elements .Males ; Females / of two-dimensional space should lie precisely in the one-dimensional subspace where Males D Females , as mentioned previously. But even supposing, for the sake of argument, that † happens to admit the representation † D LL0 C ‰, there is a second problem: The assertion that ˜ D “0 Y C •, with “0 D L0 .LL 0 C ‰/ 1 , does not logically follow from the statement † D LL0 C ‰. Other component representations using different “ are possible (infinitely many, in fact), and generally lead to nondiagonal C ov.©/ D † †““0 †, even when † D LL0 C ‰. For example, it is possible that “0 D .1=2/L0 .LL0 C ‰ / 1 , in which case C ov.©/ D .3=4/LL0 C ‰ , which is a nondiagonal matrix.

Downloaded by [Texas Technology University], [Mr Roy Howell] at 07:31 30 January 2012

102

WESTFALL, HENNING, HOWELL

Thus, whether one adopts the Platonic or component analysis standpoint, it seems mandatory to allow correlated errors if one wishes to work with realistic models. As is becoming the norm for null hypothesis significance testing in general, the question is not “Is the null hypothesis true?” but rather “How false is the null hypothesis, and how much does it matter?” In the specific context of covariance structure modeling considered in this article, the question is not “Are the error correlations zero?” but rather “How different are the error correlations from zero, and how much does it matter that the error correlations are nonzero?” We show that the question “How different are the error correlations from zero?” is unanswerable using standard covariance structure modeling. However, we do provide the answer to the question “How much does it matter that the error correlations are nonzero?” The answer is that it matters a great deal, because different error correlation structures imply dramatically different interfactor correlations. Although a common response to the possible problem of error correlation is to include more factors (e.g., Shevlin, Miles, Davies, & Walker, 2000), we show that this is not a general solution to the problem. For one thing, it might be impossible to fit the correlation structure using additional factors, either because of complex correlation structures, or because of uniformly negative correlations. More to the point, as we will show, the problem of nonidentifiability of interfactor correlation remains, no matter whether the model uses additional factors to model error correlations, or whether the error correlations are left intact. Practical Considerations That errors should be uncorrelated is often regarded as an axiom (“the axiom of local independence”) of latent variable modeling (e.g., Sobel, 1994). However, this is a questionable assumption in practice. Stout (1990), writing from an IRT perspective, argued that the usual assumption of unidimensionality should be “replaced by a weaker and arguably more appropriate statistical assumption of essential unidimensionality” (p. 293). Drawing on Humphreys’s (1984) insistence that all test items are “multiply determined,” Stout (1990) presented a model for the construction of essentially unidimensional tests in the presence of what he termed “the unavoidable empirical reality of multidimensional items” (p. 324). From a factor analytic perspective, McDonald (1981) argued for the existence of multiple “minor components” in factor analytic studies addressing unidimensionality issues. MacCallum and Tucker (1991) suggested that “there may be a large number of minor common factors influencing measured variables but which cannot be represented in a parsimonious model” (p. 503). Multidimensional items influenced by a number of common factors will result in correlated errors. This fact is made explicit in the “correlated uniqueness” factor model for the analysis of multitrait–multimethod data, where method effects are accounted for using correlated disturbance terms (Marsh, 1989). Context effects provide another practical source of correlated errors (Schuman & Presser, 1981). For example, Saris and Aalberts (2003) posited three models explaining correlated errors on the basis of relative answering: one in which the first item in a set of survey questions is an absolute answer, but the response to all subsequent items in the set is relative to the first; one in which the response to each item uses the previously answered item as a referent; and one in which the response to positive items is relative to the first positive item, whereas the response to negative items is relative to the first negative item. Sudman, Bradburn, and Schwarz (1996) discussed both cognitive and psychological explanations for context effects

Downloaded by [Texas Technology University], [Mr Roy Howell] at 07:31 30 January 2012

EFFECT OF ERROR CORRELATION ON INTERFACTOR CORRELATION

103

such as these (and others), and presented results from numerous studies documenting their presence and nonnegligible magnitude. Schwarz (2004) discussed “metacognitive” experiences of respondents as a further explanation of context effects. Correlated error terms can also be a consequence of acquiescence (Billiet & McClendon, 2000; Saris & Aalberts, 2003). As noted by Saris and Aalberts (2003), “acquiescence explains the correlated disturbance terms both within a battery [of items] and across batteries: monomethod and heteromethod error correlations” (p. 21). Further, individual survey respondents might exhibit variations in their response functions in that they differ in the way they answer questions. This means that respondents can have the same opinion on a stimulus but answer questions differently, leading to unobserved heterogeneity that can result in correlated errors (Saris & Aalberts, 2003). Additionally, MacCallum and Tucker (1991) suggested that nonlinear influences of factors on variables can result in correlated errors. Thus the large collection of very real situations in which the assumption of uncorrelated errors is violated makes according it the status of an axiom (a fundamental, self-evident statement) somewhat troubling. Furthermore, although each of these situations can potentially be modeled in isolation, it is likely that many of these sources of error correlation are present in any given application, such that modeling all possibilities is impractical at best. Therefore, it seems clear that there is a need to address the assumption of uncorrelated error terms, but this is seldom done. There are occasional exceptions: Cole, Ciesla, and Steiger (2007) discussed the importance of including correlated errors driven by design considerations such as longitudinal data, and correlated errors also have induced concern about data-snooping to improve fit (Bagozzi, 1984; Boomsma, 2000; Steiger, 1990). However, in the cases where correlated errors are allowed, the authors typically presume a “mostly zero error correlation” constraint for identifiability (Loehlin, 2004; Saris & Aalberts, 2003; Schumacker & Lomax, 2004). We remove this constraint, retain identifiability, and demonstrate a wide spectrum of conclusions that arise from equivalent models having differing degrees of error correlation.

RELIABILITY Our equivalence class of models having nonzero error correlations is conveniently and parsimoniously specified in terms of reliabilities. Reliability, a much-discussed concept in the theory of measurement, can be defined in different ways (e.g., Thompson & Vacha-Haase, 2000). In an SEM setting, reliability is used to gauge how well a manifest variable F measures a latent factor ˜. We adopt the traditional definition (Bollen, 2002; Lord & Novick, 1968): reliability D ¡2 .F; ˜/;

(2)

where ¡.
;
/ denotes ordinary Pearson correlation. When F D ˜C •, with ˜ and • uncorrelated, Equation 2 is equivalent to another common definition reliability D (Bentler, 2009).

Var .˜/ Var .F /

104

WESTFALL, HENNING, HOWELL

We can now calculate the reliability of the score F D a0 Y . From Equation 1, Y D ƒ˜ C ©, so F D a0 .ƒ˜ C ©/: Thus from Equation 2:

Downloaded by [Texas Technology University], [Mr Roy Howell] at 07:31 30 January 2012

¡2 D

a0 ƒƒ0 a a0 ƒƒ0 a a0 ƒƒ0 a D D : a0 †a a0 .ƒƒ0 C ‚/a a0 ƒƒ0 a C a0 ‚a

(3)

Although Cronbach’s (1951) coefficient ’ is frequently reported as a measure of reliability 0 of the summate F D 1 Y (Hogan, Benjamin, & Brezinski, 2000; Schmitt, 1996), it is more generally a lower bound to reliability, achieving equality only in the case of essential tauequivalence (Cortina, 1993; Kristof, 1974; Lord & Novick, 1968; Novick & Lewis, 1967; ten Berge & Zegers, 1978). Higher ’ values (standard advice suggests  :70, see Bagozzi & Yi, 1988; Lance, Butts, & Michels, 2006; Wallen & Fraenkel, 2001) are assumed to indicate more reliable measurements than lower ’ values. There is extensive literature that shows how reliability is affected by a priori assignment of error correlation (Green & Hershberger, 2000; Komaroff, 1997; Lucke, 2005; Rae, 2006; Raykov, 1998; Zimmerman & Williams, 1980). We take the converse position, showing that error correlation is affected by a priori assignment of reliability. An example of such a priori assignment can be found in Hayduk (1996, p. 25), where it is suggested that the reliability of a particular item (say Y1 without loss of generality) be prespecified; in this case a0 D .1 0 : : : 0/. Suppose the researcher wishes to set the reliability of a component F D a0 Y to be r 2 . The model in Equation 1 accomplishes this when ƒ D r †a=.a0 †a/1=2: plugging this into Equation 3 gives ¡2 D r 2 . Further, ‚ D † r 2 †aa0 †=a0 †a can shown to be nonnegative definite for all r 2 Œ0; 1 using the extended Cauchy–Schwarz inequality, implying that the model is valid. This analysis shows yet another way that error correlation is factual: For a fixed covariance matrix †, when the reliability of a component score is higher, error correlations are smaller (typically becoming negative at the extreme), and when the reliability of a component score is lower, then error correlations are higher (approaching correlations between items at the extreme). It is apparently not well known how error correlation affects interfactor correlation. We now show this relationship, using different reliability assignments to define equivalent models with different error correlation structures.

THE TWO LATENT VARIABLE MODEL Patterned structures for ƒ such as 

ƒx ƒD 0

0 ƒy



have zeros specified in fixed locations in many measurement models. Suppose for simplicity that there are two unidimensional latent variables, ˜ and Ÿ, with Y D ƒy ˜ C —, X D ƒx Ÿ C •,

Downloaded by [Texas Technology University], [Mr Roy Howell] at 07:31 30 January 2012

EFFECT OF ERROR CORRELATION ON INTERFACTOR CORRELATION

105

and with estimation of ¥ D C or r .˜; Ÿ/ the primary concern of the study. Usually, the primary concern is the estimation of a standardized path coefficient, but in this example the standardized path coefficient is equal to the correlation ¥. For modeling alternative error correlation structures, we provide in the following sections alternative models with    ƒx 0 ƒ D  ; 0 ƒy thus obeying the same structural measurement pattern, that are equivalent in the sense of having fit and identifiability properties identical to those of the original model. The equivalence class can result from either the platonic or the generalized component definitions of the latent variable, as defined earlier. A Toy Example To fix ideas, consider the following toy example. Two distinct models for the same covariance matrix are presented, with radically different ¥ D C or r .˜; Ÿ/. The example has contrived aspects to illustrate the points; a real example is presented later following the theory development of the next two sections. Suppose items Y1 ; : : : ; Y4 measure ˜, and items X1 ; : : : ; X4 measure Ÿ. The .8  8/ covariance matrix is composed of 2 3 2 3 1 .2 .5 .5 .2 .2 .2 .2 6 .2 1 .2 .5 7 6 .2 .2 .2 .2 7 7 6 7 †yy D †xx D 6 4 .5 .2 1 .2 5 ; and †xy D 4 .2 .2 .2 .2 5 : .5 .5 .2 1 .2 .2 .2 .2 Two models that reproduce the .8  8/ covariance matrix perfectly are shown in Figures 1 and 2. In the model of Figure 1, ¥ D 0:4; in Figure 2, ¥ D 1:0. Because both models reproduce the same covariance matrix, it is impossible to distinguish between a model with ¥ D 0:4 or ¥ D 1:0 using any fit criterion. This example also shows that the problem of correlated errors cannot simply be argued away using an additional factor, because an additional factor cannot fit the given residual covariance structures. The Two-Factor Parallel Model To provide theory for the equivalence issue shown in the previous section we start with the simpler case of a two-factor parallel model with Xi D œx Ÿ C •i , i D 1; : : : ; q; and Yi D œy ˜ C —i , i D 1; : : : ; p;

(4)

along with the usual condition that all latent variables are independent except for the two factors: Ÿ independent of f•i g; f—i g; ˜ independent of f•i g; f—i g; the f•i g independent of the f—i g; f•i g independent and f—i g independent. In addition, we assume Var .•i / D ¢•2 , Var .—i / D ¢—2 ,

Downloaded by [Texas Technology University], [Mr Roy Howell] at 07:31 30 January 2012

106

WESTFALL, HENNING, HOWELL

FIGURE 1 The model implied by this path diagram exactly reproduces the .8  8/ covariance matrix in the toy example.

FIGURE 2 This model also exactly reproduces the .8  8/ covariance matrix in the toy example, but leads to a substantively different conclusion than that of Figure 1.

EFFECT OF ERROR CORRELATION ON INTERFACTOR CORRELATION

107

Downloaded by [Texas Technology University], [Mr Roy Howell] at 07:31 30 January 2012

Var .Ÿ/ D Var .˜/ D 1, and C ov.Ÿ; ˜/ D ¥, with all random variables normally distributed (as noted earlier, the normality assumption serves only to make identifiability arguments tidy, but is not absolutely required for all results that follow). Here the parameter vector is ™ D Œœx ; ¢•2 ; œy ; ¢—2 ; ¥ and is identifiable if p; q > 1. The model for ŒX; Y is completely determined by † D C ov



X Y





† xx D †yx

  2 œ 1q 10 C ¢•2 I q † xy D x q ¥œy œx 1p 10q † yy

 ¥œx œy 1q 10p : œ2y 1p 10p C ¢—2 I p

Using Equation 3, the reliabilities of Fx D X1 C


C Xq and Fy D Y1 C


C Yp are respectively ¡2x D

qœ2x q 2 œ2x D 2 q 2 œ2x C q¢• qœ2x C ¢•2

and ¡2y D

p 2 œ2y p 2 œ2y C

p¢—2

D

pœ2y pœ2y C ¢—2

:

These results depend strongly on the a priori assumption of uncorrelated errors. Suppose that, instead of choosing C or r .•i ; •j / D 0 and C or r .—i ; —j / D 0 a priori, one instead chooses to allow them to be nonzero. This can be accomplished simply by prespecifying reliabilities rx2 2 .0; 1 and ry2 2 .0; 1 and then using the model Xi D œx Ÿ C •i ,

and i D 1; : : : ; qI Yi D œy ˜ C —i , i D 1; : : : ; p;

(5)

with Ÿ independent of f•i g; f—i g; ˜ independent of f•i g; f—i g; the f•i g independent of the f—i g; and all random variables normally distributed. The parameters of the new model (Equation 5) are related to those of the original model (Equation 4) as follows: œx D .rx =¡x /œx ; Var .•i / D œ2x C ¢•2

  2 œ2 x ; C ov.•i ; •j / D œx

œ2 x ,

(6)

œy D .ry =¡y /œy ; Var .—i / D œ2y C ¢—2

  2 œ2 y ; C ov.—i ; —j / D œy

œ2 y ,

(7)

and ¥ D C ov.Ÿ ; ˜ / D

¡x ¡y ¥: rx ry

(8)

As in the baseline model (Equation 4), it remains that the set of parameters is ™ D Œœx ; ¢•2 ; œy ; ¢—2 ; ¥ for the new model. There are no new error correlation parameters because the error correlations are defined explicitly in terms of the parameters of the original model, as shown in Equations 6, 7, and 8. This fact, coupled with the fact that the covariance matrix † is unchanged, implies that the new model retains identifiability characteristics for all parameters as in the original model. Hence the new model is indistinguishable from the baseline model as a probability model for ŒX; Y. The difference appears in the definitions of the latent variables (.Ÿ ; ˜ / versus .Ÿ; ˜/).

Downloaded by [Texas Technology University], [Mr Roy Howell] at 07:31 30 January 2012

108

WESTFALL, HENNING, HOWELL

FIGURE 3

Interfactor correlation ¥ as a function of r 2 .

p For example, suppose a baseline model has œx D œy D 1= 2, ¢•2 D ¢—2 D 1=2, ¥ D 1=2; and q D p D 5. Suppose also that rx2 D ry2 D r 2 . Figure 3 displays the interfactor correlation ¥ of the new model with a given r 2 . The following are notes on the models represented by Figure 3:  The lower bound ¥L D :347 is the correlation between component scores 10q X and 10p Y .  Values ¥ > 1 are obtained when r 2 < :347, but such ¥ are inconsistent with ¥ D C or r .Ÿ ; ˜ / and are therefore disallowed.  All models represented in Figure 3 (one for each r 2 ) provide the same probability distribution for the observable data and are therefore indistinguishable, regardless of sample size.  Although the special pattern of the error covariance matrix in the equivalent models (Equation 5) can be explained using an additional factor, the problem remains: The correlation between the two measured factors can lie anywhere in the range Œ:347; 1, with no way to distinguish which value is correct using the data. This example generalizes to more complex structural models with multiple latent variables, as we now show.

The General Structured Model The models of Figures 1 and 2 and the parallel models of the previous section are special cases of the more general case that is developed in this section. Suppose the baseline model is X D ƒx Ÿ C •,

Y D ƒy ˜ C —;

(9)

where Var .Ÿ/ D Var .˜/ D 1, C ov.Ÿ; ˜/ D ¥, C ov.Ÿ; •/ D 0.1q/ , C ov.˜;—/ D 0.1p/, C ov.•/ D ‚ • , C ov.—/ D ‚ — (‚ • , ‚ — possibly nondiagonal), C ov.˜; •/ D 0.1q/, C ov.Ÿ;—/ D 0.1p/, C ov.—; •/ D 0.pq/ , and where all random variables are normally distributed. Assume

EFFECT OF ERROR CORRELATION ON INTERFACTOR CORRELATION

ƒx , ƒy , ‚ • , ‚ — , ¥ are functions of ™, such that  ƒx ƒ0x C ‚ • †.™/ D ¥ƒy ƒ0x

¥ƒx ƒ0y ƒy ƒ0y C ‚ —



109

(10)

Downloaded by [Texas Technology University], [Mr Roy Howell] at 07:31 30 January 2012

is positive definite for all ™ 2 . The equivalence class is conveniently defined using factor scores FŸ D ƒ0x † xx1 X

and F˜ D ƒ0y †yy1 Y. 0

(11) 0

0

Using Equation 3, the reliability of FŸ D ƒ0x †xx1 X is ƒx † xx1 .ƒx ƒx /† xx1 ƒx =fƒx †xx1 .† xx / 0 † xx1 ƒx g D ƒx † xx1 ƒx ; hence ¡2x D ƒ0x † xx1 ƒx

and

¡2y D ƒ0y †yy1 ƒy .

(12)

To obtain the equivalence class, the analyst can again assign reliability values rx2 , ry2 2 .0; 1. A class of equivalent models, one model for each .ry2 ; rx2 /, is defined by X D ƒx Ÿ C •  ,

Y D ƒy ˜ C — ;

(13)

where Var .Ÿ / D Var .˜ / D 1, C ov.Ÿ ; ˜ / D ¥ , C ov.Ÿ ; • / D 0.1q/, C ov.˜ ;— / D 0.1p/ , C ov.˜ ; • / D 0.1q/, C ov.Ÿ ;— / D 0.1p/, C ov.—  ; •  / D 0.pq/ , and where all random variables are normally distributed. Here, ƒx D

ry rx ƒx , and ƒy D ƒy , ¡x ¡y

C ov.•  / D ‚ • D †xx

.rx2 =¡2x /ƒx ƒ0x , and

C ov.—  / D ‚ — D †yy

.ry2 =¡2y /ƒy ƒ0y :

(14) (15)

Using the generalized Cauchy–Schwarz inequality, it can be shown that ‚ • and ‚ — are nonnegative definite for all rx2 , ry2 2 Œ0; 1; hence the models are valid in the sense of having plausible error covariance matrices. Further, all models in the class produce identical † xx and † yy as produced by the baseline model (Equation 9). So that the models in the equivalence class produce the same cross-covariance matrix †xy as well, note that the model defined by Equations 13, 14, and 15 implies C ov.X ; Y/ D † xy D ¥

rx ry ƒx ƒ0y . ¡x ¡y

(16)

To match Equation 16 from the equivalent model with the corresponding ¥ƒx ƒ0y from the baseline model, we must have ¡x ¡y ¥ D ¥: (17) rx ry Thus the equivalence class of model is specified by Equations 13, 14, 15, and 17).

110

WESTFALL, HENNING, HOWELL

The following are notes on the equivalence class:

Downloaded by [Texas Technology University], [Mr Roy Howell] at 07:31 30 January 2012

 For ¥ of Equation 17 to be a correlation, the a priori values rx2 , ry2 must constrain ¥ to the interval Œ 1; 1. Some .rx2 ; ry2 / settings will produce invalid models; this problem can be circumvented by choosing nonnegative values a; b; c; d , and setting rx D j¥ja ¡cx ; ry D j¥jb ¡dy : Any settings with 0  a C b  1, 0  c; d  1 produce valid ¥ . When setting a D b D c D d D 0, the researcher obtains ¥ D ¥¡y ¡x , the minimum. The baseline model is obtained with a D b D 0, c D d D 1, and with a C b D 1, c D d D 1, the model implies j¥ j D 1. For example, if the researcher thinks both factor scores are highly reliable, he or she can choose a, b, c, and d at or near zero (.05, for example). If the researcher thinks both the factor scores lack reliability, he or she can choose a, b, c, and d so a C b, c, and d are at or near 1.0 (.95, for example). If the researcher thinks the X factor score is highly reliable and the Y factor score lacks reliability, he or she can choose a and c near 0 (.05, for example) and b and d near 1.0 (.95, for example), as long as a C b  1.  The minimum ¥L D ¥¡y ¡x occurs when ry D rx D 1, and the minimum value is ¥L D ¥¡x ¡y D ¥.ƒ0x † xx1 ƒx /1=2 .ƒ0y † yy1 ƒy /1=2 D C or r .FŸ ; F˜ /:

(18)

 For ¥ > 0, any ¥ 2 Œ¥L ; 1 is consistent with some model in the equivalence class. Because all models in the class provide the same distribution for the data, one can only identify the interfactor correlation to lie in the range Œ¥L ; 1; more precise identification is impossible, regardless of sample size.  The parameters of the new model are the same ™ 2  as in the original model, and the covariance matrix of the new model is identical to that of the baseline model; namely, †.™/. Hence all models in the equivalence class have identical identifiability properties as the baseline model.  Extending to m > 2 latent variables, Equation 17 becomes ¥ij D

¡i ¡j ¥ij ; 1  i; j  m; ri rj

and the model is valid provided f¥ij g is nonnegative definite. Example: Job Salience and Job Satisfaction For an example we consider the study of Dunham (1977), who provided seven measures of job satisfaction and five measures of job salience. We view Salience as predictor (x) and Satisfaction as response (y), and attempt to estimate the correlation ¥ between the two underlying latent variables. In so doing, we estimate ¥ to be either :554, :489, or :979 (depending on our choice of reliabilities), with identical fit statistics for all three models.

EFFECT OF ERROR CORRELATION ON INTERFACTOR CORRELATION

The relevant covariance matrices (of the standardized items) are, based on N D 784, 2

Downloaded by [Texas Technology University], [Mr Roy Howell] at 07:31 30 January 2012

S xx

2

S yy

6 6 6 6 D6 6 6 6 4

6 6 D6 6 4 1:0 :43 :27 :24 :34 :37 :40

1:0 :49 :53 :49 :51

1:0 :33 :26 :54 :32 :58

1:0 :57 :46 :53

3 1:0 :48 :57

1:0 :57

1:0

7 7 7; 7 5 3

1:0 :25 :46 :29 :45

1:0 :28 :30 :27

1:0 :35 :59

1:0 :31

:19 :08 :07 :19 :23

:30 :27 :24 :21 :32

:37 :35 :37 :29 :36

1:0

7 7 7 7 7; 7 7 7 5

and 2

S xy

:33 6:30 6 D6 6:31 4:24 :38

:32 :21 :23 :22 :32

:20 :16 :14 :12 :17

3 :21 :207 7 :187 7: :165 :27

A baseline model is shown in Figure 4; maximum likelihood estimates are 2

3 2 3 :706 :502 6:7187 6 0 7 :485 6 7 6 7 7; bx D 6:7577 ; ‚ b• D 6 0 ƒ 0 :427 6 7 6 7 4:6465 4 0 5 0 0 :583 :749 0 0 0 :086 :439 2 2 3 :587 :656 6 0 6:6777 6 6 7 6:5197 6 0 6 6 7 6 7 b b ƒy D 6:3887 ; ‚ — D 6 6 0 6:8087 6 :149 6 7 6 4:6835 4 0 :711 0

:541 0 :731 0 0 :850 0 0 0 :142 0 0 :098 0 0

3

:349 :207 0

:533 :169 :494

7 7 7 7 7; 7 7 7 5

111

Downloaded by [Texas Technology University], [Mr Roy Howell] at 07:31 30 January 2012

112

WESTFALL, HENNING, HOWELL

FIGURE 4 Baseline model for the satisfaction/work salience example.

and b ¥ D :554; the fitted covariance 2 1:0 6:507 6 b xx D 6:534 † 6 4:456 :529 2 1:0 6:397 6 6:305 6 b yy D 6:228 † 6 6:327 6 4:400 :418 2 :230 6:233 6 b xy D 6:246 and † 6 4:210 :244

matrices are 3

7 1:0 7 7; :543 1:0 7 5 :463 :489 1:0 :538 :567 :570 1:0 1:0 :352 :263 :549 :319 :580 :265 :269 :284 :242 :281

3

7 7 7 1:0 7 7; :201 1:0 7 7 :421 :314 1:0 7 5 :354 :264 :346 1:0 :369 :276 :577 :316 1:0 3 :203 :152 :317 :267 :278 :206 :154 :322 :271 :2837 7 :218 :163 :340 :286 :2987 7: :186 :139 :290 :244 :2555 :216 :161 :337 :283 :295

Fit statistics are root mean square residual D .0504, root mean square error of approximation D .0601, and Goodness-of-Fit D .9633 (the model fails according to the chi-square test, but this is irrelevant to the point of this article). The model implies that the reliability

EFFECT OF ERROR CORRELATION ON INTERFACTOR CORRELATION

113

of FŸ D ƒ0x † xx1 X is, from Equation 3, ƒ0x †xx1 .ƒx ƒ0x /† xx1 ƒx =fƒ0x † xx1 .† xx /† xx1 ƒx g D 0 ƒ0x † xx1 ƒx ; similarly, the reliability of F˜ D ƒ0y † yy1 Y is ƒy †yy1 ƒy . The estimated factor 0 b0 † b 1 D Œ:23 :24 :29 :14 :25 and b b0 † b 1 D Œ:16 score coefficient vectors are b a0 D ƒ a Dƒ

Downloaded by [Texas Technology University], [Mr Roy Howell] at 07:31 30 January 2012

x

x

xx

y

y

yy

b 0x † b xx1 ƒ b x / D 0:836 :14 :05 :03 :43 :35 :19, thus the estimated reliabilities for this model are .ƒ 0 b 1b b and .ƒy †yy ƒy / D 0:933. However, there are infinitely many equivalent models that cannot be distinguished from the baseline model. For example, if the researcher feels that the factor scores are perfectly reliable measures of the latent variables, then the estimated model shown next is obtained by specifying 1 0 1 reliabilities rx2 D 1 and ry2 D 1 for the factor scores ƒ0 x † xx X and ƒy † yy Y, respectively (or, equivalently, by Equation 14, for the factor scores from the baseline model). In this case, 2 3 2 3 :772 :404 6 :116 :384 7 6:7857 7 6 7 6   7 6 7; 6 b b ƒx D 6:8287 ; ‚ • D 6 :109 :079 :315 7 4:7065 4 :055 5 :094 :104 :501 :819 :122 :113 :108 :008 :329 3 2 2 3 :607 :631 6:7017 6 :004 7 :509 6 7 6 7 6:5377 6 :056 7 :047 :711 6 7 6 7 7; b  D 6:4017 ; ‚ b  D 6 :004 :021 :034 :839 ƒ y — 6 7 6 7 6:8377 6 :168 7 :046 :011 :056 :300 6 7 6 7 4:7065 4 :059 5 :175 :090 :016 :241 :501 :736 :047 :064 :054 :026 :026 :210 :458

and b ¥ D :489; the fitted covariance matrix and all fit statistics are identical to the baseline model. On the other hand, the researcher might feel that the measures are not as reliable as suggested by the baseline model, and instead specify reliabilities rx2 D :5 and ry2 D :5 for the factor scores 1 0 1 ƒ0 x † xx X and ƒy † yy Y (or again, for the factor scores from the baseline model). In this case 2 3 2 3 :546 :702 6:5557 6:187 :692 7 6 7 6 7 7; b  D 6:5857 ; ‚ b  D 6:211 :245 :658 ƒ x • 6 7 6 7 4:4995 4:218 :183 :188 :751 5 :579 :194 :209 :231 :281 :665 2 3 2 3 :430 :815 6:4967 6:217 :754 7 6 7 6 7 6:3807 6:107 :142 :856 7 6 7 6 7   b 6 7 b 6 7 ; and b ƒy D 6:2847 ; ‚ — D 6:118 :119 :142 :919 ¥ D :979I 7 6:5927 6:086 :247 :235 :122 :650 7 6 7 6 7 4:5005 4:155 :072 :100 :158 :054 :750 5 :521 :176 :322 :252 :122 :282 :050 :729 again the fitted covariance matrix and all fit statistics are identical to the baseline model.

114

WESTFALL, HENNING, HOWELL

CONCLUDING REMARKS

Downloaded by [Texas Technology University], [Mr Roy Howell] at 07:31 30 January 2012

Hayduk (1987) wrote: It is our view that measurement reliabilities should routinely be fixed rather than free. The researcher’s familiarity with the data collection procedures provides information about measurement quality that is lost unless the researcher takes the initiative and incorporates this information by specifying particular measurement reliabilities. (p. 119)

Although our purpose is not to agree or disagree with this position (although it is a position that has some support; see, e.g., Kaplan & Johnson, 2001), our equivalence class of models provides a simple way to fix reliabilities. Hayduk suggested fixing reliability of a single identified item, and earlier we showed that this can be accomplished without sacrificing fit, albeit by introducing error correlation. Further, we showed that if one fixes reliabilities of factor scores to (rx2 ; ry2) in the two-factor model, the resulting interfactor correlations are related via ¡x ¡y ¥ ¥ D rx ry as shown in Equation 17. This equation also shows that all correlations ¥ 2 Œ¥L ; 1 are equally supported by the data, and shows that setting rx D ry D 1 provides the lower bound ¥L D ¡x ¡y ¥: All else being equal, a theory that cannot be falsified using data is less tenable than a theory that can be falsified using data yet has withstood empirical attempts to do so. Nonidentifiability implies a lack of falsifiability: If two models implying radically different theories cannot be distinguished using the data, then one theory cannot be falsified in favor of the other. As we have shown, no one model in our equivalence class can be falsified in favor of any other. Because nonidentifiable models “represent a threat to the validity of substantive conclusions drawn from an entertained model” (Raykov & Penev, 1999, p. 200), the equivalence class that we present is very troublesome for psychometric measurement and theory testing. In the face of this problem, one might adopt a stance similar to that of Freedman (1987), who, when discussing path analysis, remarked that “repeating well-worn errors for lack of anything better to do can hardly be the right course of action : : : [I]t is better to abandon a faulty research paradigm and go back to the drawing board” (p. 102). A more positive suggestion comes from this research, however. Because all interfactor correlations ¥ 2 Œ¥L ; 1 are equally supported by the data, researchers might report the correlation ¥L in addition to the estimated correlation between latent variables, and comment that the true correlation between latent variables is not estimable, but known to be greater than the lower bound ¥L . ACKNOWLEDGMENT The late David Freedman of the University of California at Berkeley provided many insightful comments and suggestions on earlier versions of this article. We would also like to thank the reviewers, whose comments and suggestions have greatly improved this article.

EFFECT OF ERROR CORRELATION ON INTERFACTOR CORRELATION

115

Downloaded by [Texas Technology University], [Mr Roy Howell] at 07:31 30 January 2012

REFERENCES Bagozzi, R. P. (1984). A prospectus for theory construction in marketing. The Journal of Marketing, 48, 11–29. Bagozzi, R. P., & Yi, Y. (1988). On the evaluation of structural equation models. Journal of the Academy of Marketing Science, 16, 74–94. Bentler, P. M. (2009). Alpha, dimension-free, and model-based internal consistency reliability. Psychometrika, 74, 137–143. Billiet, J. B., & McClendon, M. J. (2000). Modeling acquiescence in measurement models for two balanced sets of items. Structural Equation Modeling, 7, 608–628. Bollen, K. A. (1989). Structural equations with latent variables. New York, NY: Wiley. Bollen, K. A. (2002). Latent variables in psychology and the social sciences. Annual Review of Psychology, 53, 605–634. Boomsma, A. (2000). Reporting analyses of covariance structures. Structural Equation Modeling, 7, 461–483. Borsboom, D., Mellenbergh, G. J., & van Heerden, J. (2003). The theoretical status of latent variables. Psychological Review, 110, 203–218. Cole, D. A., Ciesla, J. A., & Steiger, J. H. (2007). The insidious effects of failing to include design-driven correlated residuals in latent-variable covariance structure analysis. Psychological Methods, 12, 381–398. Cortina, J. M. (1993). What is coefficient alpha? An examination of theory and applications. Journal of Applied Psychology, 78, 98–104. Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297–334. Dunham, R. B. (1977). Reactions to job characteristics: Moderating effects of the organization. Academy of Management Journal, 20, 42–65. Freedman, D. A. (1987). As others see us: A case study in path analysis. Journal of Educational and Behavioral Statistics, 12, 101–128. Grayson, D. (2006). Might “unique” factors be “common”? On the possibility of indeterminate common–unique covariances. Psychometrika, 71, 521–528. Green, S. B., & Hershberger, S. L. (2000). Correlated errors in true score models and their effect on coefficient alpha. Structural Equation Modeling, 7, 251–270. Hayduk, L. A. (1987). Structural equation modeling with LISREL: Essentials and advances. Baltimore, MD: Johns Hopkins University Press. Hayduk, L. A. (1996). LISREL issues, debates, and strategies. Baltimore, MD: Johns Hopkins University Press. Hershberger, S. L. (1994). The specification of equivalent models before the collection of data. In A. von Eye & C. C. Clogg (Eds.), Latent variables analysis: Applications for developmental research (pp. 68–105). Thousand Oaks, CA: Sage. Hogan, T. P., Benjamin, A., & Brezinski, K. L. (2000). Reliability methods: A note on the frequency of use of various types. Educational and Psychological Measurement, 60, 523–531. Humphreys, L. (1984). A theoretical and empirical study of the psychometric assessment of psychological test dimensionality and bias (ONR Research Proposal). Office of Naval Research, Washington, DC. Kaplan, D. (2008). Structural equation modeling: Foundations and extensions. Thousand Oaks, CA: Sage. Kaplan, H. B., & Johnson, R. J. (2001). Social deviance: Testing a general theory. New York, NY: Kluwer Academic/Plenum. Komaroff, E. (1997). Effect of simultaneous violations of essential tau equivalence and uncorrelated error on coefficient alpha. Applied Psychological Measurement, 21, 337–348. Krijnen, W. P. (2006). Implications of indeterminate factor-error covariances for factor construction, prediction, and determinacy. Psychometrika, 71, 503–519. Kristof, W. (1974). Estimation of reliability and true score variance from a split of a test into three arbitrary parts. Psychometrika, 39, 491–499. Lance, C. E., Butts, M. M., & Michels, L. C. (2006). The sources of four commonly reported cutoff criteria: What did they really say? Organizational Research Methods, 9, 202–220. Lee, S., & Hershberger, S. (1990). A simple rule for generating equivalent models in covariance structure modeling. Multivariate Behavioral Research, 25, 313–334. Loehlin, J. C. (2004). Latent variable models: An introduction to factor, path, and structural equation analysis (4th ed.). Mahwah, NJ: Lawrence Erlbaum Associates, Inc.

Downloaded by [Texas Technology University], [Mr Roy Howell] at 07:31 30 January 2012

116

WESTFALL, HENNING, HOWELL

Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley. Lucke, J. F. (2005). “Rassling the hog”: The influence of correlated item error on internal consistency, classical reliability, and congeneric reliability. Applied Psychological Measurement, 29, 106–125. MacCallun, R. C. (2003). Working with imperfect models. Multivariate Behavioral Research, 38, 113–139. MacCallum, R. C., & Tucker, L. R. (1991). Representing sources of error in the common-factor model: Implications for theory and practice. Psychological Bulletin, 109, 502–511. MacCallum, R. C., Wegener, D. T., Uchino, B. N., & Fabrigar, L. R. (1993). The problem of equivalent models in applications of covariance structure analysis. Psychological Bulletin, 114, 185–199. Marsh, H. W. (1989). Confirmatory factor analyses of multitrait–multimethod data: Many problems and a few solutions. Applied Psychological Measurement, 13, 335–361. McDonald, R. P. (1981). The dimensionality of tests and items. British Journal of Mathematical and Statistical Psychology, 34, 100–117. McDonald, R. P. (1996). Path analysis with composite variables. Multivariate Behavioral Research, 31, 239–270. McDonald, R. P. (2004). The specific analysis of structural equation models. Multivariate Behavioral Research, 39, 687–713. Meehl, P. E. (1997). The problem is epistemology, not statistics: Replace significance tests by confidence intervals and quantify accuracy of risky numerical predictions. In L. L. Harlow, S. A. Mulaik, & J. H. Steiger (Eds.), What if there were no significance tests? (pp. 393–425). Mahwah, NJ: Lawrence Erlbaum Associates, Inc. Nickerson, R. S. (2000). Null hypothesis significance testing: A review of an old and continuing controversy. Psychological Methods, 5, 241–301. Novick, M. R., & Lewis, C. (1967). Coefficient alpha and the reliability of composite measurements. Psychometrika, 32, 1–13. Rae, G. (2006). Correcting coefficient alpha for correlated errors: Is ’K a lower bound to reliability? Applied Psychological Measurement, 30, 56–59. Raykov, T. (1998). Coefficient alpha and composite reliability with interrelated nonhomogeneous items. Applied Psychological Measurement, 22, 375–385. Raykov, T., & Marcoulides, G. A. (2001). Can there be infinitely many models equivalent to a given covariance structure model? Structural Equation Modeling, 8, 142–149. Raykov, T., & Penev, S. (1999). On structural equation model equivalence. Multivariate Behavioral Research, 34, 199–244. Saris, W. E., & Aalberts, C. (2003). Different explanations for correlated disturbance terms in MTMM studies. Structural Equation Modeling, 10, 193–213. Schmitt, N. (1996). Uses and abuses of coefficient alpha. Psychological Assessment, 8, 350–353. Schönemann, P. H., & Steiger, J. H. (1976). Regression component analysis. British Journal of Mathematical and Statistical Psychology, 29, 175–189. Schumacker, R. E., & Lomax, R. G. (2004). A beginner’s guide to structural equation modeling (2nd ed.). Mahwah, NJ: Lawrence Erlbaum Associates, Inc. Schuman, H., & Presser, S. (1981). Questions and answers in attitude surveys. New York, NY: Academic Press. Schwarz, N. (2004). Metacognitive experiences in consumer judgment and decision making. Journal of Consumer Psychology, 14, 332–348. Seddon, G. M. (1988). The validity of reliability measures. British Educational Research Journal, 14, 89–97. Shevlin, M., Miles, J. N. V., Davies, M. N. O., & Walker, S. (2000). Coefficient alpha: A useful indicator of reliability? Personality and Individual Differences, 28, 229–237. Sobel, M. E. (1994). Causal inference in latent variable models. In A. von Eye & C. Clogg (Eds.), Latent variables analysis: Applications for developmental research (pp. 3–35). Thousand Oaks, CA: Sage. Stanghellini, E. (1997). Identification of a single-factor model using graphical gaussian rules. Biometrika, 84, 241–244. Steiger, J. H. (1990). Structural model evaluation and modification: An interval estimation approach. Multivariate Behavioral Research, 25, 173–180. Steiger, J. H. (1994). Factor analysis in the 1980’s and the 1990’s: Some old debates and some new developments. In I. Borg & P. Mohler (Eds.), Trends and perspectives in empirical social research (pp. 201–224). New York, NY: Walter de Gruyter. Steiger, J. H. (2001). Driving fast in reverse. Journal of the American Statistical Association, 96, 331–338. Stelzl, I. (1986). Changing a causal hypothesis without changing the fit: Some rules for generating equivalent path models. Multivariate Behavioral Research, 21, 309–331.

Downloaded by [Texas Technology University], [Mr Roy Howell] at 07:31 30 January 2012

EFFECT OF ERROR CORRELATION ON INTERFACTOR CORRELATION

117

Stout, W. F. (1990). A new item response theory modeling approach with applications to unidimensionality assessment and ability estimation. Psychometrika, 55, 293–325. Sudman, S., Bradburn, N., & Schwarz, N. (1996). Thinking about answers: The application of cognitive processes to survey methodology. San Francisco, CA: Jossey-Bass. ten Berge, J. M. F., & Zegers, F. E. (1978). A series of lower bounds to the reliability of a test. Psychometrika, 43, 575–579. Thompson, B., & Vacha-Haase, T. (2000). Psychometrics is datametrics: The test is not reliable. Educational and Psychological Measurement, 60, 174–195. Wallen, N. E., & Fraenkel, J. R. (2001). Educational research: A guide to the process (2nd ed.). Mahwah, NJ: Lawrence Erlbaum Associates, Inc. Zimmerman, D. W., & Williams, R. H. (1980). Is classical test theory “robust” under violation of the assumption of uncorrelated errors? Canadian Journal of Psychology, 34, 227–237.