May 19, 2003 - Rasch models (Rasch, 1960; Fischer & Molenaar, 1995) are widely used for modelling of latent variables measured using ordinal scales.
SAS macros for Rasch based latent variable modelling Karl Bang Christensen National Institute of Occupational Health, Denmark and Department of Biostatistics, University of Copenhagen Jakob Bue Bjorner National Institute of Occupational Health, Denmark May 19, 2003 Abstract This paper describes SAS macros that can be used to estimate item parameters using CML, test the assumption of unidimensionality, and fit regression models where either outcome variables or covariates are latent variables measured using (log linear) Rasch models.
1
Introduction
Rasch models (Rasch, 1960; Fischer & Molenaar, 1995) are widely used for modelling of latent variables measured using ordinal scales. Traditionally, specialized software has been needed and this have limited the use of Rasch models. The traditional Rasch modelling framework consisting of a latent variable θ responsible for the correlation between the observed items x1 , . . . , xI has been extended in two ways: (i) to include relations between items and covariates (Kelderman, 1984, 1992; Kelderman & Rijkes, 1994) and (ii) to include relations between the latent variable and covariates (Andersen & Madsen, 1977; Zwinderman, 1991; Adams, Wilson, & Wu, 1997). A framework including both of these extensions has also been discussed (Kreiner & Christensen, 2002; Christensen, Bjorner, Kreiner, & Petersen, 2003). This paper describes SAS macros for CML estimation (Sections 2.1 and 2.2) and for regression models with latent variables as outcome variables (Section 4) or 1
covariates (Section 5). Tests of fit of the Rasch model are also discussed (Sections 3 and 2.2). All macros can be downloaded from http://www.biostat.ku.dk/rasch/macro.html. The use of the macros are illustrated using a data set test containing five items ITEM1, ITEM2, ITEM3, ITEM4, and ITEM5 scored 0, 1, 2 and binary covariates X, Y, and Z. The macros create output files in the form of SAS data sets and write results to the log file. The macros latreg, latreg 2d, and latreg st use output files created by the macros rasch and llrasch. An overview of input and output files is given in in Appendix A, p. 15.
2
CML estimation
An important feature of Rasch models is the sufficiency of the raw score making consistent estimation of item parameters without reference to the distribution of the latent variable in the population possible. Conditional maximum likelihood (CML) estimation has traditionally been computer intensive and available only in specialized IRT computer programs. This section describes SAS macros for CML estimation that are reliable and fast for up to 15 items (depending on the number of response categories). 2.1
CML estimation in Rasch models
Generalized linear models are standard statistical models (Nelder & Wedderburn, 1972; McCullagh & Nelder, 1989). CML estimation in the dichotomous and polytomous Rasch model can be done using standard software for fitting generalized linear models, (Tjur, 1982; Kelderman, 1984, 1992; Agresti, 1993). Assume that response categories are scored 0, 1, . . . , H and let a persons answers to the items i = 1, . . . , I be represented by the indicators x = (xih )i=1,... ,I,h=1,... ,H , 2
where xih = 1 if the person gives the answer h to item i and 0 otherwise. The conditional probability given the person parameter θ is
p(x|θ) =
exp([
P P P P xih h]θ + i h xih ηih ) i Q hP i l exp(lθ + ηil )
(1)
and the likelihood function is the product of these. The model is identified if
∀i : ηi0 = 0 ∧
X
ηim = 0.
(2)
i
These constraints are identical to the ones proposed for the Partial Credit model (Masters, 1982) using the so-called threshold parameters
βih = −(ηih − ηi,h−1 )
(3)
for i = 1, . . . , I and h = 1, . . . , H. The conditional likelihood function is the product of the conditional probabilities P P given the score t = i h xih h. These can be written
p(x|t) = where γt = γt (η) =
P(t) x
exp(
exp(
P P i
h
xih ηih )
γt
P i
ηixi ) are well-known gamma-polynomials, (
(4) P(t) x
is short notation for the sum over all response vectors x with sum t). To estimate the item parameters the data is transformed to a mk contingency table (Nx )x∈X , where X is the set of all possible responses. The expected numbers are ENx = Nt p(x|t) and this yields a generalized linear model
log(ENx ) =
Nt X X − xih ηih γt i h 3
(5)
it can be shown that the likelihood equations in this model are the CML equations (Agresti, 1993). The resulting parameter estimates must be standardized according P h to (2), i.e. ηih → ηih − km i ηim . The macro rasch that calculates these estimates is called by writing %rasch(data=test, items=ITEM1-ITEM3, max=2, out=RASCH, write=YES, fittest=YES); specifying that the data set test contains the items ITEM1, ITEM2, and ITEM3 scored 0, 1, 2. The sample size, the number of items and the number of response categories is written to the log file, followed by the estimated item parameters, estimated values of gamma polynomials, and the maximum value of the log likelihood function (this option can be switched off by specifying write=NO). The arguments are summarized in Table 1. Table 1: Arguments to the SAS macro rasch.
Argument Comments data=
Input data set containing the items
items=
Item names, written as V1 V2 V3 V4 (statements like V1-V4 can be used)
max=
Maximum value of items (minimum value is assumed to be 0, items assumed to have same number of response categories)
out=
First part of name given to output files (default value RASCH)
write=
If YES (the default) information is written to the log file
fittest=
If YES (the default) observed and expected values are plotted (see p. 5)
Three data sets RASCH par, RASCH logl and RASCH regr are created. The two 4
first contain estimated item parameters and maximum log likelihood value. The data set RASCH regr is a version of the input data set containing extra variables that can be used to fit regression models with latent outcomes (Section 4) P or latent covariates (Section 5). This data set contains the score t = i xi and the estimated values of log(γt ) for each possible score value t = 0, 1, . . . , T = I · H. Specifying fittest=YES creates plots of observed and expected item category frequencies stratified by the total score: Let Nt denote the number of persons with score t, and let Niht denote the number of these persons giving the answer h to item i. For for each combination (i, h) ∈ {1, . . . , I} × {1, . . . , H} the macro plots t 7→ p(xih = 1|t) and t 7→
Niht . Nt
These plots are closely related to plots of the Item
Characteristic Curves (or Item Response Functions) θ 7→ p(xih = 1|θ), because t is a sufficient statistic. Because the macro uses the contingency table of item responses no responses must be missing. It is assumed that all items use the same number of response categories. The macro fits a Poisson regression model to the observed counts of the contingency table, if the estimation procedure fails to converge a warning or error message is printed. 2.2
CML estimation in log linear Rasch models
When item parameters are allowed to depend on covariates y = (yl )l=1,... ,L in (1), i.e. when DIF (or item bias) is modelled, the model belong to the class of log linear Rasch models. These were first described in a series of papers by Kelderman (Kelderman, 1984, 1992; Kelderman & Rijkes, 1994) as conditional models without direct reference to the latent variable θ. In the following attention is restricted to the situation yl = 0, 1 (i.e. where categorical covariates are dummy coded). The following parameters are used
5
l ηih = ηih + Iil δih
(6)
where Iil is an indicator taking the value 1 if the item i is biased with respect to the covariate yl . The parameters can be estimated in a way similar to the estimation described in section 2.1: data is transformed to a mk · |y| contingency table (Nx,y )x∈X,y∈Y , where Y is the set of all possible values of covariates. The expected numbers are t,y ENx,y = Ny,t p(x|t) and the generalized linear model (9) is log(ENx,y ) = Nγt,y − P P P y i h y xihy ηih . The resulting parameter estimates must be standardized in a
way similar to (2), see (Kelderman, 1984) for details. The macro is called by writing %llrasch(data=test, items=ITEM1-ITEM3, max=2, dif=ITEM1*Y ITEM3*Z, out=LLRASCH); specifying that the data set test contains items ITEM1, ITEM2, ITEM3 (scored 0, 1, 2), that parameters for the item ITEM1 depend on Y and that parameters for the item ITEM3 depend on Z. Categorical covariates taking more than two values must be dummy coded. The arguments are summarized in Table 2. The macro creates data sets in a manner similar to the macro rasch (Section 2.1). Likelihood values computed by this macro can be compared to the one computed by the macro rasch yielding conditional likelihood ratio tests for DIF (not done automatically). Because the macro uses the contingency table of item responses no responses must be missing. It is assumed that all items use the same number of response categories. The macro fits a Poisson regression model to the observed counts of the contingency table, if the estimation procedure fails to converge a warning or error message is printed. Problems can occur for small data sets or very skewed items. 6
Table 2: Arguments to the SAS macro llrasch.
Argument Comments data=
input data set containing the items
items=
item names, written as V1 V2 V3 V4 (statements like V1-V4 can be used)
3
max=
maximum value of items (minimum value is assumed to be 0)
dif=
symbolic representation of item×exo interactions
out=
first part of name given to output files (default LLRASCH)
Testing unidimensionality
The Martin-L¨of test for unidimensionality, (Martin-L¨of, 1970; Gustafsson, 1980), is based on the so called extended likelihood function
L(x) = p(x|t)κt
(7)
where the probabilities of the marginal score distribution, κt , are unrestricted parameters estimated by the observed frequencies
Nt N
(Tjur, 1982; Christensen, Bjorner,
Kreiner, & Petersen, 2002). The probabilities (7) generalize to more than one latent variable where each item depend on only one of these. The assumption of unidimensionality can be tested using a likelihood ratio test based on extended likelihood functions: The Martin-L¨of test statistic. The macro to do this test is called by writing %pml(data=test, scale1=ITEM1-ITEM3, scale2=ITEM4 ITEM5, max=2, out=PML, bootstrap=NO, seed=0, nsimu=1000); specifying that the Rasch model where ITEM1, . . . , ITEM5 depends on θ is tested 7
against the two-dimensional alternative where ITEM1, ITEM2, and ITEM3 depend on θ1 and ITEM4 and ITEM5 depend on θ2 . The arguments are summarized in Table 3 Table 3: Arguments to the SAS macro pml.
Argument
Comments
data=
input data set containing the items
scale1=
names of items in first sub scale, written V1 V2 V3 V4 (statement like V1-V4 can be used)
scale2=
name of items in second sub scale, written V1 V2 V3 V4 (statement like V1-V4 can be used)
max=
maximum item score (minimum score assumed to be 0)
out=
name of output file (default PML)
bootstrap=
if YES a parametric bootstrap test (p. 8) is computed default is NO
nsimu=
number of simulations (default 1000)
Computing the test statistic implies estimating κ’s for each combination of sub scores. The difference in number of parameters is thus very large and a consequence of this the null distribution deviates strongly from the asymptotic chi-square distribution (Verhelst, 2001; Christensen et al., 2002). A simulation study (Christensen et al., 2002) showed that p-values underestimates significance. A significant result can in general be taken as evidence of multidimensionality, whereas a cautious approach to insignificant p-values is advisable. Evaluating the Martin-L¨of test statistic using the standard chi-square approximation thus yields a test with low power. This problem is dealt with by means of parametric bootstrap (Efron, 1979; Efron
8
& Tibshirani, 1993), by simulating nsimu (e.g. 1000 - the default) values of the test statistic under the null hypothesis of unidimensionality. This is done as follows: (i) using the observed frequencies
Nt N
simulate the score t for each of N persons, (ii)
simulate item response vectors x using the conditional probabilities 4, (iii) compute the test statistic. The difficult step is (ii) because the number of possible item response vectors x will be very large for some values of t. The solution to this problem is a recursive procedure sampling one item response at a time combining the probabilities p(xih = 1|t) (Andersen, 1995, (15.22)) with the recursion formulas used to compute the γ-values (Andersen, 1995, (15.26) and (15.27)). This bootstrap procedure is implemented in the macro pml and is performed if bootstrap=YES is specified. For each simulation the macro fits three Poisson regression models, if the estimation procedure fails to converge for one of these a warning or error message is printed (problems can occur for small data sets, very skewed items, and in situations where the fit of the Rasch model is poor for one or both sub scales). The entire simulation history is printed to the file out history results should of course be interpreted with caution if convergence problems have occurred for a large number of simulations.
4
Regression models with latent outcomes
In a (loglinear) Rasch model the score is a sufficient statistic for the latent variable θ. If the item parameters are known the distribution score depends only on θ and belongs to the class of one parameter exponential families. The distribution is given by
γt exp(tθ) . p(t|θ) = P t γt exp(tθ)
9
(8)
Imposing the structure θ = Zδ,
(9)
where Z is vector of covariates, thus specifies a generalized linear model. Extending (9) by modelling residual variation yields a generalized linear mixed model
ξ ∼ N (0, ω 2 ).
θ = Zδ + ξ
(10)
Latent regression models of this type (Christensen et al., 2003) can be fitted in PROC NLMIXED in SAS using an adaptive Gaussian quadrature. In order to do this estimated γ values are needed. These can be computed by the macro rasch (Section 2.1) in the case of a simple Rasch model or by the macro llrasch (Section 2.2) in the case of log linear Rasch model. The macro to fit the latent regression model (10) is called by writing %latreg(data=LLRASCH_regr, sumscore=t, lgamma=lg0-lg6, fixed=X Y, fittest=NO, group=NONE, out=LATREG); specifying that the output data set created by the macro llrasch is to be used, cf. p. 5. The model (10) is fitted using an intercept and the fixed effects X and Y. This means that four parameters βint , βX , βY , and σ are estimated. The macro can compute goodness of fit tests (Christensen & Kreiner, 2003) for the latent regression model: if fittest=YES the macro computes observed and expected score counts and creates data sets LATREG res with observed counts, expected counts, and standardized residuals and LATREG pearson with a Pearson test statistic based on these. The observed and expected score distributions are also compared through the skewness and kurtosis, in the file LATREG fit, this file contains observed values together with upper and lower confidence limits. 10
The option GROUP= can be used to extend these tests with a stratification variable. This variable must be numerical. The arguments to the macro latreg are summarized in Table 4. Table 4: Arguments to the SAS macro latreg.
Argument
Comments
data=
input data set containing sum score t =
P i
xi and
estimated values log(ˆ γt ) for t = 0, 1, . . . , T = I · H sumscore=
name of sum score variable (the default t, is the one used by the macros rasch and llrasch)
lgamma=
names of log(ˆ γ )-values (the names lg0-lgT are used by the macros rasch and llrasch)
fixed=
name of the (dummy coded) fixed effects default value is NONE, i.e. same mean
fittest=
If YES fit test statistics, cf. p. 10 are computed default value is NO
group=
name of stratification variable for fit tests, p. 11 default value is NONE, i.e. no stratification. This variable must be numerical
out=
5
first part of output file names (default LATREG)
Regression models with latent covariates
Latent variables are often of interest through their association with other(observed) variables. Section 4 described the situation where a latent variable is the outcome in a regression analysis. This section describes macros that include a latent variable as
11
a covariate in two standard regression models: logistic regression models and Poisson regression models. In what follows Y is assumed to be a dichotomous outcome variable, and a logistic regression model with random person effect
p(Y = 1) =
exp(η) exp(1 + η)
² ∼ N (0, σ 2 ),
η = Xβ + ²
is considered. The latent regression model given by (8) and (10) can be included in this regression model for the observed outcome variable Y in two ways: (i) through residual correlation corr(², ξ) = ρ and (ii) directly η = Xβ + γθ + ² (Christensen, Nielsen, & Smith-Hansen, 2003). The relationship between variables imposed by each of these two models is summarized in Figure 1. Figure 1: Overview of the relation between variables in the two regression models (i) two-dimensional regression model with correlated errors. µ
²H HH H j H -η X
(ii) regression model with structural relation. ²H HH H j H -η X
- y
- y
6
- θ * © © ©
Z R
- t
- θ * © © ©©
Z
© ξ ©
ξ ©
The regression model given by (i) is fitted with the statement %latreg_2d(data=LLRASCH_regr, outcomevar=Y, sumscore=t, lgamma=lg0-lg6, outcome_fixed=X, score_fixed=Z, type=LOGISTIC, out=LATREG_2D); and the regression model given by (ii) is fitted with the statement 12
- t
%latreg_st(data=LLRASCH_REGR, outcomevar=Y, sumscore=t, lgamma=lg0-lg6, outcome_fixed=X, score_fixed=Z, type=LOGISTIC, out=LATREG_ST); respectively. Specifying that the data set created by the macro llrasch be used, that the covariate X influences the binary outcome variable Y, and that the covariate Z influences the latent variable θ measured using the score t. The macros write information to the log file and create output files LATREG 2D par, LATREG 2D logl, LATREG ST par, and LATREG ST logl. The arguments to the macros latreg 2d and latreg st are summarized in Table 5.
13
Table 5: Arguments to the SAS macros latreg 2d and latreg st.
Argument
Comments
data=
input data set containing sum score t =
P i
xi and
estimated values log(ˆ γt ) for t = 0, 1, . . . , T = I · H. outcomevar=
name of outcome variable y
sumscore=
name of sum score variable (the default t, is the one used by the macros rasch and llrasch)
lgamma=
names of log(ˆ γ )-values (the names lg0-lgT are used by the macros rasch and llrasch)
outcome fixed=
name of (dummy coded) fixed effects for outcome regression, default is NONE, i.e. same mean
score fixed=
name of (dummy coded) fixed effects for latent regression, default is NONE, i.e. same mean
type=
type of regression model for outcome variable (LOGISTIC or POISSON)
out=
first part of output file names (default LATREG 2D and LATREG ST respectively)
14
A Input and output data sets used by the the SAS macros.
Macro
Input
Comments
Output
Comments
rasch
test
contains
out par
estimates βˆih , cf. (3)
items
out logl
log likelihood value
out regr
contains sum score t and log(ˆ γ )’s lg0-lgT
llrasch
test
contains
out par
l , cf. (6) estimates ηˆih
items
out logl
log likelihood value
out regr
contains sum score t and log(ˆ γ )’s lg0-lgT
pml
test
contains
out
items
test statistic, df, and p-value
out history
simulation history
(optional) latreg
latreg 2d
out regr
out regr
created by
LATREG par
ˆω estimates δ, ˆ , cf. (10)
rasch or
LATREG logl
log likelihood value
llrasch
LATREG res
observed and expected
(optional)
counts and residuals
LATREG pearson
Pearson test statistic,
(optional)
df, and p-value
LATREG fit
observed and expected
(optional)
skewness and kurtosis
LATREG 2D par
ˆ δ, ˆσ estimates β, ˆ, ω ˆ , ρˆ,
created by rasch or
latreg st
out regr
cf. (i), p. 12
llrasch
LATREG 2D logl
log likelihood value
created by
LATREG ST par
ˆ δ, ˆσ estimates β, ˆ, ω ˆ , γˆ ,
rasch or llrasch
cf. (ii), p. 12 LATREG ST logl
15
log likelihood value
References Adams, R. J., Wilson, M., & Wu, M. (1997). Multilevel item response models: An approach to errors in variables regression. Journal of Educational and Behavioral Statistics, 22 (1), 47–76. Agresti, A. (1993). Computing conditional maximum likelihood estimates for generalized Rasch models using simple loglinear models with diagonals parameters. Scandinavian Journal of Statistics, 20 (1), 63–71. Andersen, E. B. (1995). Polytomous Rasch models and their estimation. In G. H. Fischer & I. W. Molenaar (Eds.), Rasch models - foundations, recent developments, and applications (pp. 271–291). Springer-Verlag. Andersen, E. B., & Madsen, M. (1977). Estimating the parameters of the latent population distribution. Psychometrika, 42, 357–374. Christensen, K. B., Bjorner, J. B., Kreiner, S., & Petersen, J. H. (2002). Testing unidimensionality in polytomous Rasch models. Psychometrika, 67 (4), 563– 574. Christensen, K. B., Bjorner, J. B., Kreiner, S., & Petersen, J. H.
(2003).
Latent regression in loglinear Rasch models (Tech. Rep. No. 03/4). partment of Biostatistics, University of Copenhagen.
De-
(available from
http://www.biostat.ku.dk/publ-e.htm) Christensen, K. B., & Kreiner, S. (2003). Testing the fit of latent regression models (Tech. Rep. No. 03/3). Department of Biostatistics, University of Copenhagen. (available from http://www.biostat.ku.dk/publ-e.htm) Christensen, K. B., Nielsen, M. L., & Smith-Hansen, L. (2003). Latent covariates in generalized linear models (Tech. Rep. No. 03/12). Department of Biostatistics, 16
University of Copenhagen. (available from http://www.biostat.ku.dk/puble.htm) Efron, B. (1979). Bootstrap methods: Another look at the jackknife. Annals of Statistics, 7, 1–26. Efron, B., & Tibshirani, R. (1993). An introcuction to the bootstrap. London: Chapman and Hall. Fischer, G. H., & Molenaar, I. W. (1995). Rasch models - foundations, recent developments, and applications. Springer-Verlag. Gustafsson, J. E. (1980). Testing and obtaining fit of data to the Rasch model. British Journal of Mathematical and Statistical Psychology, 33, 205–233. Kelderman, H. (1984). Log linear Rasch model tests. Psychometrika, 49, 223–245. Kelderman, H. (1992). Computing maximum likelihood estimates of loglinear models from marginal sums with special attention to loglinear item response theory. Psychometrika, 57, 437–450. Kelderman, H., & Rijkes, C. P. M. (1994). Loglinear multidimensional IRT models for polytomously scored items. Psychometrika, 59 (2), 149–176. Kreiner, S., & Christensen, K. B. (2002). Graphical Rasch models. In M. Mesbah, M.-L. T. Lee, & B. F. Cole (Eds.), Statistical methods for quality of life studies: Design, measurements and analysis (pp. 187–203). Kluwer Academic Publishers. Martin-L¨of, P. (1970). Statistiska modeller. anteckninger fr˚ an seminarier l¨as˚ aret 1969-70 (statistical models. notes from the academic year 1969-70) (Tech. Rep.). Stockholm: Institut f¨or f¨ors¨akringsmatematik och matematisk statistik.
17
Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149–174. McCullagh, P., & Nelder, J. A. (1989). Generalized linear models (2 ed.). Chapman and Hall. Nelder, J. A., & Wedderburn, R. W. M. (1972). Generalized linear models. Journal of the Royal Statistical Society A, 135, 370–384. Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Chicago: University of Chicago Press; 1980. Tjur, T. (1982). A connection between Rasch’s item analysis model and a multiplicative poisson model. Scandinavian Journal of Statistics, 9 (1), 23–30. Verhelst, N. (2001). Testing the unidimensionality assumption of the rasch model. Methods of Psychological Research Online, 6 (3), 231–271. Zwinderman, A. H. (1991). A generalized Rasch model for manifest predictors. Psychometrika, 56, 589–600.
18
