Random-effect models for longitudinal data - Google Sites

Random-effect models for longitudinal data Nan M. Laird, James H. Ware Department of Biostatistics, harvard School of Public Health

Biometrics 38, December 1982

Laird, Ware (HSPH)

Random-effect models for longitudinal data

Biometrika, 1982

1 / 15

Outline

1

Introduction

2

Random effect model

3

Estimation and Inference

4

EM algorithm

5

Example

6

Discussion

Laird, Ware (HSPH)

Random-effect models for longitudinal data

Biometrika, 1982

2 / 15

Introduction

Two stage models First stage: probability distribution has the same form for each individual. But parameters vary over each individual (random effect). Second Stage: random effects constitutes the second stage of the model Example: Serial measurement of lung volume and cube of height. Stage 1: Linear regression model applies for each child Stage 2: regression parameter has special bivariate normal distribution; serial measurements have special covariate structure.

Laird, Ware (HSPH)

Random-effect models for longitudinal data

Biometrika, 1982

3 / 15

Introduction (continued)

Two stage models has several advantages: No requirement for balance in data (good for longitudinal study) Explicit analysis of between/within individual variation Facilitate exploratory analysis ... Major disadvantages by the publication of this paper (1982): Lack of unified fitting methodology Special form assumed for covariance structure

Laird, Ware (HSPH)

Random-effect models for longitudinal data

Biometrika, 1982

4 / 15

Comparison with multivariate model Definition (Multivariate model) y i = µi + e i

(1)

Here yi is a ni dimensional response vector with dispersion matrix Σ. ni is the number of observation for the ith individual (i = 1 . . . m). Multivariate model: requires that the design is balanced but permits missing data does not apply when measurements are made at unique/random times gives poor estimation when Σ is large does not permit estimation of individual characteristics However, two stage random effect model is able to overcome these problems. Laird, Ware (HSPH)

Random-effect models for longitudinal data

Biometrika, 1982

5 / 15

Random effect model Definition (Stage 1) For each individual i, we have yi = Xi a + Zi b + ei

(2)

where ei is distributed as N(0, Ri ). Here a is a p-dimensional vector of unknown population parameters. Xi is a known ni × p design matrix. bi is a k-dimensional vector of unknown individual effect. Zi is a known ni × k design matrix. Note here that Ri is dependent on i through its dimension ni . However, the set of unknown parameters for Ri is not dependent on i. This means that we can not simply estimate a using OLS and then estimate parameters for covariate structures separately.

Laird, Ware (HSPH)

Random-effect models for longitudinal data

Biometrika, 1982

6 / 15

Random effect model (continued)

Definition (Stage 2) bi are i.i.d. distributed as N(0, D), independent of ei . Then yi are independent normally distributed as N(Xi a, Ri + Zi DZiT ).

Definition (Parameters to estimate) a and θ, where θ is a q-vector of covariance parameters in Ri , i = 1, . . . , m. Then we have three estimation methods for (a, θ): Least square Maximum likelihood Empirical Bayesian

Laird, Ware (HSPH)

Random-effect models for longitudinal data

Biometrika, 1982

7 / 15

Known variance When var (yi ) = Vi = Ri + Zi RZiT is known, we have: !−1 m m X X T ˆa = Xi Wi Xi XiT Wi yi bˆi

=

i=1 DZiT Wi (yi

i=1

− Xi ˆa)

(3)

where Wi = Vi−1 . Note that ˆa is MLE and UMVUE. However, bˆ is not MLE but follows an extension of Gauss-Markov theorem and is also empirical Bayes. For the estimation variance, we have: !−1 m X var (ˆa) = XiT Wi Xi i=1

  var (bˆi ) = DZiT Wi − Wi Xi var (ˆa)XiT Wi Zi D

(4)

var (bˆi − bi ) = D − var (bˆi ) Laird, Ware (HSPH)

Random-effect models for longitudinal data

Biometrika, 1982

8 / 15

Unknown variance

When the covariance parameters are unknown, substituting Vi with ˆi = R ˆ i + Zi D ˆ i ZiT V

(5)

with MLE θˆM is straight forward. In this case ˆa(θˆm ) is still MLE for a. ˆ θˆM ) is still the empirical Bayesian estimate. b( However, the estimation of θ remains a problem. Two leading methods are maximum likelihood (ML), θˆM and restricted ML estimate (REML), θˆR . Both of them lead to unified estimation approach and EM algorithm.

Laird, Ware (HSPH)

Random-effect models for longitudinal data

Biometrika, 1982

9 / 15

Restricted maximum likelihood

Maximum likelihood estimates tend to give biased estimation of covariate structure. However, REML is able to give unbiased result. In REML, the estimation is not carried out for all y . Instead, it is carried out based on any full-rank set of error contrast µT y such that E(µT y ) = 0

(6)

which is equivalent to µT X = 0. Harville, David, 1977. Maximum likelihood approaches to variance component estimation and to related problems

Laird, Ware (HSPH)

Random-effect models for longitudinal data

Biometrika, 1982

10 / 15

EM algorithm for MLE EM algorithm is applied for the estimation of a, θ as a unified method. Starting with initial estimate of a0 , θ0 , and sufficient statistics t for θ, we have:

Definition (EM algorithm) M-step: θˆ = M(ˆt )

(7)

where M gives the MLE of θ given t. Then we have the E-step with ˆ as: estimation of ˆa(θ) ˆ θ) ˆ ˆt = E(t|y , ˆa(θ),

(8)

Then E and M steps are iterated until convergence criteria is satisfied. EM algorithm is also applied in other situations for filling missing data. Here we are treating θ as if they are missing. A Bayesian interpretation for REML was laid out. But the general steps are similar. Laird, Ware (HSPH)

Random-effect models for longitudinal data

Biometrika, 1982

11 / 15

Example for EM algorithm For illustration of EM algorithm in the estimation of (a, θ), consider situation when Ri P = σ 2 Ini and D isP unstructured. Define sufficient m T T ˆ θ, ˆ t1 statistics as t1 = i=1 ei ei , t2 = m a(θ), i=1 bi bi . When we have ˆ and t2 can be estimated as ˆt1

m X ˆ θ) ˆ = E( eiT ei |yi , ˆa(θ),

ˆt2

m X ˆ θ) ˆ = E( bi biT |yi , ˆa(θ),

i=1

i=1

ˆ = t2 /m, where ThenPwe have the estimation for θ as σ ˆ 2 = t1 /n, D n ˆ ˆ n = i=1 ni . Thus we can get updated (θ, ˆa(θ))

Laird, Ware (HSPH)

Random-effect models for longitudinal data

Biometrika, 1982

12 / 15

Example: Children FEV with air pollution

Data m = 200 school children, ni = 5 consecutive measurements for each child on and after the day of an air pollution incident. Objective identify sensitive subgroup or individual most severely affected by this pollution incident. Model I is a 5-dimensional identity matrix. a is a vector corresponding to the mean level of FEV of each measurement. biT = (b1i , b2i ), where b2i represents the change of each child compared with baseline. yi = Ia + Zi b + ei

(9)

So we have Zi with all elements equaling to 1 except the Zi (1, 2) = 0. Result 20 children identified with large b2i were selected for further study.

Laird, Ware (HSPH)

Random-effect models for longitudinal data

Biometrika, 1982

13 / 15

Discussion

The authors mentioned several points for further development: Unified approach for longitudinal study ˆ Estimation of var (ˆa) and var (b) Rate of convergence for EM

Laird, Ware (HSPH)

Random-effect models for longitudinal data

Biometrika, 1982

14 / 15

Thank you!

Laird, Ware (HSPH)

Random-effect models for longitudinal data

Biometrika, 1982

15 / 15