The nonlinear mixed effects model. The EM-type algorithms. Some pharmacokinetics examples. Estimation of the likelihood. Using MCMC for Maximum ...
The nonlinear mixed effects model The EM-type algorithms Some pharmacokinetics examples Estimation of the likelihood
Using MCMC for Maximum Likelihood Estimation in Non-Linear Mixed Effects Models
Marc Lavielle
Marc Lavielle
The MONOLIX software
The nonlinear mixed effects model The EM-type algorithms Some pharmacokinetics examples Estimation of the likelihood
A pharmacokinetics example : theophylline
Each individual curve is described by the same parametric model, with its own individual parameters. Marc Lavielle
The MONOLIX software
The nonlinear mixed effects model The EM-type algorithms Some pharmacokinetics examples Estimation of the likelihood
A pharmacokinetics example : theophylline
Each individual curve is described by the same parametric model, with its own individual parameters. Marc Lavielle
The MONOLIX software
The nonlinear mixed effects model The EM-type algorithms Some pharmacokinetics examples Estimation of the likelihood
The basic model The incomplete data model
The model
yij = f (xij , φi ) + εij , 1 ≤ i ≤ N , 1 ≤ j ≤ ni
yij ∈ R is the jth observation of subject i, N is the number of subjects ni is the number of observations of subject i. The regression variables, or design variables, (xij ) are known, The individual parameters (φi ) are unknown,
Marc Lavielle
The MONOLIX software
The nonlinear mixed effects model The EM-type algorithms Some pharmacokinetics examples Estimation of the likelihood
The basic model The incomplete data model
The model
yij = f (xij , φi ) + εij , 1 ≤ i ≤ N , 1 ≤ j ≤ ni
yij ∈ R is the jth observation of subject i, N is the number of subjects ni is the number of observations of subject i. The regression variables, or design variables, (xij ) are known, The individual parameters (φi ) are unknown,
Marc Lavielle
The MONOLIX software
The nonlinear mixed effects model The EM-type algorithms Some pharmacokinetics examples Estimation of the likelihood
The basic model The incomplete data model
The model
yij = f (xij , φi ) + εij , 1 ≤ i ≤ N , 1 ≤ j ≤ ni
yij ∈ R is the jth observation of subject i, N is the number of subjects ni is the number of observations of subject i. The regression variables, or design variables, (xij ) are known, The individual parameters (φi ) are unknown,
Marc Lavielle
The MONOLIX software
The nonlinear mixed effects model The EM-type algorithms Some pharmacokinetics examples Estimation of the likelihood
The basic model The incomplete data model
The model yij = f (xij , φi ) + εij , 1 ≤ i ≤ N , 1 ≤ j ≤ ni The vector φi of individual parameters is assumed to be Gaussian: φi = µ + ηi
with
ηi ∼i.i.d. N (0, Γ)
µ: unknown vector of population parameters (the fixed effects), (ηi ): unknown random vectors (the random effects). The sequence (εij ) is assumed to be Gaussian: εij ∼i.i.d. N (0, σ 2 ) Marc Lavielle
The MONOLIX software
The nonlinear mixed effects model The EM-type algorithms Some pharmacokinetics examples Estimation of the likelihood
The basic model The incomplete data model
The incomplete data model yij = f (xij , φi ) + εij , 1 ≤ i ≤ N , 1 ≤ j ≤ ni We are in a classical framework of “incomplete data”: the measurements y = (yij , 1 ≤ i ≤ N , 1 ≤ j ≤ ni ) are the “observed data” the individual random parameters φ = (φi , 1 ≤ i ≤ N), are the “non observed data”, the “complete data” of the model is (y , φ). (y , φ) ∼ p(y , φ; θ) θ = (µ, Γ, σ 2 ) is an unknown set of parameters Marc Lavielle
The MONOLIX software
The nonlinear mixed effects model The EM-type algorithms Some pharmacokinetics examples Estimation of the likelihood
The basic model The incomplete data model
Our objectives Estimate θ Estimate p(φ |y ; θ) Proposed approach: Define a sequence (θk ) that converges to the Maximum Likelihood Estimate of θ (a.s.) Run a non homogenous MCMC algorithm with stationary distributions sequence (p(φ |y ; θk )) that converges to p(φ |y ; θMLE ).
Marc Lavielle
The MONOLIX software
The nonlinear mixed effects model The EM-type algorithms Some pharmacokinetics examples Estimation of the likelihood
The basic model The incomplete data model
Our objectives Estimate θ Estimate p(φ |y ; θ) Proposed approach: Define a sequence (θk ) that converges to the Maximum Likelihood Estimate of θ (a.s.) Run a non homogenous MCMC algorithm with stationary distributions sequence (p(φ |y ; θk )) that converges to p(φ |y ; θMLE ).
Marc Lavielle
The MONOLIX software
The nonlinear mixed effects model The EM-type algorithms Some pharmacokinetics examples Estimation of the likelihood
The EM algorithm
The EM algorithm The SAEM algorithm The MCMC algorithm Simulated Annealing
(Expectation-Maximization)
(Dempster, Laird et Rubin, JRSSB, 1977)
Iteration k of the algorithm:
step E : evaluate the quantity Qk (θ) = E[log p(y , φ; θ)|y ; θk−1 ] step M : update the estimation of θ: θk = Argmax Qk (θ)
Marc Lavielle
The MONOLIX software
The nonlinear mixed effects model The EM-type algorithms Some pharmacokinetics examples Estimation of the likelihood
The SAEM algorithm
The EM algorithm The SAEM algorithm The MCMC algorithm Simulated Annealing
(Stochastic Approximation of EM)
Delyon, Lavielle and Moulines (the Annals of Statistics, 1999)
Iteration k of the algorithm: step E : Simulation: draw the non observed data φ(k) with the conditional distribution p(φ |y ; θk−1 ) Stochastic approximation: h i Qk (θ) = Qk−1 (θ) + γk log p(y , φ(k) ; θ) − Qk−1 (θ) P P 2 (γk ) is a decreasing sequence: γk = +∞, γk < +∞.
step M: update the estimation of θ: θk = Argmax Qk (θ) Marc Lavielle
The MONOLIX software
The nonlinear mixed effects model The EM-type algorithms Some pharmacokinetics examples Estimation of the likelihood
The SAEM algorithm
The EM algorithm The SAEM algorithm The MCMC algorithm Simulated Annealing
(Stochastic Approximation of EM)
Delyon, Lavielle and Moulines (the Annals of Statistics, 1999)
Iteration k of the algorithm: step E : Simulation: draw the non observed data φ(k) with the conditional distribution p(φ |y ; θk−1 ) Stochastic approximation: h i Qk (θ) = Qk−1 (θ) + γk log p(y , φ(k) ; θ) − Qk−1 (θ) P P 2 (γk ) is a decreasing sequence: γk = +∞, γk < +∞.
step M: update the estimation of θ: θk = Argmax Qk (θ) Marc Lavielle
The MONOLIX software
The nonlinear mixed effects model The EM-type algorithms Some pharmacokinetics examples Estimation of the likelihood
The SAEM algorithm
The EM algorithm The SAEM algorithm The MCMC algorithm Simulated Annealing
(Stochastic Approximation of EM)
Delyon, Lavielle and Moulines (the Annals of Statistics, 1999)
Iteration k of the algorithm: step E : Simulation: draw the non observed data φ(k) with the conditional distribution p(φ |y ; θk−1 ) Stochastic approximation: h i Qk (θ) = Qk−1 (θ) + γk log p(y , φ(k) ; θ) − Qk−1 (θ) P P 2 (γk ) is a decreasing sequence: γk = +∞, γk < +∞.
step M: update the estimation of θ: θk = Argmax Qk (θ) Marc Lavielle
The MONOLIX software
The nonlinear mixed effects model The EM-type algorithms Some pharmacokinetics examples Estimation of the likelihood
The EM algorithm The SAEM algorithm The MCMC algorithm Simulated Annealing
Coupling SAEM with MCMC Kuhn and Lavielle, ESAIM P&S, 2004
Let Πθ be the transition probability of an ergodic Markov Chain with limiting distribution pΦ|Y (·|y ; θ). Iteration k of the algorithm: Simulation : draw φ(k) according to the transition probability Πθk−1 (φ(k−1) , ·).
Marc Lavielle
The MONOLIX software
The nonlinear mixed effects model The EM-type algorithms Some pharmacokinetics examples Estimation of the likelihood
The EM algorithm The SAEM algorithm The MCMC algorithm Simulated Annealing
The main convergence Theorem Theorem Under very general technical conditions, the SAEM sequence (θk ) converges a.s. to some (local) maximum of the observed likelihood g (y ; θ). Proof. See 1. Delyon, Lavielle & Moulines The Annals of Statistics (1999) 2. Kuhn & Lavielle ESAIM P&S (2004)
Marc Lavielle
The MONOLIX software
The nonlinear mixed effects model The EM-type algorithms Some pharmacokinetics examples Estimation of the likelihood
The EM algorithm The SAEM algorithm The MCMC algorithm Simulated Annealing
MCMC (Markov Chain Monte Carlo) an iterative procedure for the simulation of p(φ|y ; θ)
At iteration ` 1 2
draw a new value φc with a proposal distribution q, accept this new value, that is set φ` = φc with probability α(φc ) =
q(φc , φ(`−1)) p(φc |y ; θ) q(φc , φ(`−1)) p(φ(`−1) |y ; θ)
In the model y = f (x; φ) + g (x; φ)ε, computing α(φc ) only requires to compute f (x, φc ) and g (x, φc ) but not the derivatives of f and g Marc Lavielle
The MONOLIX software
The nonlinear mixed effects model The EM-type algorithms Some pharmacokinetics examples Estimation of the likelihood
The EM algorithm The SAEM algorithm The MCMC algorithm Simulated Annealing
Some proposals used in Monolix Three following proposal kernels for 1 ≤ i ≤ N: (1)
1
qθk is the prior distribution of φi at iteration k, that is the Gaussian distribution N (Ai µk , Γk )
2
qθk is the multidimensional random walk N (φi , τk Γk ).
(2)
τk = τk−1 (1 + a(ρk−1 − ρ? )) 0 < a < 1; ρ? ≈ 0.4 ρk−1 : proportion of acceptation at iteration k − 1. 3
(3)
qθk is a succession of d unidimensional Gaussian random walks: each component of φi are successively updated.
Marc Lavielle
The MONOLIX software
The nonlinear mixed effects model The EM-type algorithms Some pharmacokinetics examples Estimation of the likelihood
The EM algorithm The SAEM algorithm The MCMC algorithm Simulated Annealing
Some proposals used in Monolix
Then, the simulation-step at iteration k consists in running (1)
1
m1 iterations of the Hasting-Metropolis with proposal qθk ,
2
m2 iterations with proposal qθk
3
m3 iterations with proposal qθk .
(2) (3)
Marc Lavielle
The MONOLIX software
The nonlinear mixed effects model The EM-type algorithms Some pharmacokinetics examples Estimation of the likelihood
The EM algorithm The SAEM algorithm The MCMC algorithm Simulated Annealing
Running several chains
Convergence of the algorithm can be improved by running L independent Markov Chain instead of only one. The simulation step requires to draw L sequences φ(k,1) , . . . , φ(k,L) at iteration k and to combine stochastic approximation and Monte Carlo in the approximation step: ! L 1X (k,`) log p(y , φ ; θ) − Qk−1 (θ) Qk (θ) = Qk−1 (θ) + γk L `=1
Marc Lavielle
The MONOLIX software
The nonlinear mixed effects model The EM-type algorithms Some pharmacokinetics examples Estimation of the likelihood
The EM algorithm The SAEM algorithm The MCMC algorithm Simulated Annealing
A Simulated Annealing version of SAEM Conditional distribution of φ: p( φ |y ; θ) = C (y ; θ)e −U(φ,y ;θ) Temperature parameter T : pT ( φ |y ; θ) = CT (y ; θ)e −
U(φ,y ;θ) T
Choose a decreasing Temperature sequence (Tk ) converging to 1. Then, at iteration k of SAEM, E-step: draw the non observed data φ(k) with the conditional distribution pTk ( · |y ; θk−1 ) M-step remains unchanged Marc Lavielle
The MONOLIX software
The nonlinear mixed effects model The EM-type algorithms Some pharmacokinetics examples Estimation of the likelihood
A pharmacokinetics example : theophylline
Each concentration curve is described by the same pharmacokinetic model, with its own individual pharmacokinetics parameters. Marc Lavielle
The MONOLIX software
The nonlinear mixed effects model The EM-type algorithms Some pharmacokinetics examples Estimation of the likelihood
oral administration, first-order absorbtion and elimination dose D at time t=0 absorption (rate ka ) → DRUG AMOUNT Q(t) → elimination (rate ke ) dQ (t) = ka Qa (t) − ke Q(t) dt dQa (t) = −ka Qa (t) dt Qa (t): amount at absorption site. ka Q(t) =D e −ke t − e −ka t C (t) = V V (ka − ke ) C (t) : concentration of the drug, V : volume of the compartment Marc Lavielle
The MONOLIX software
The nonlinear mixed effects model The EM-type algorithms Some pharmacokinetics examples Estimation of the likelihood
intravenous administration and first-order elimination
dose D (t=0) → DRUG AMOUNT Q(t) → elimination (rate ke )
dQ (t) = −kQ(t) dt Q(t) = De −kt C (t) =
Q(t) D = e −ke t V V
C (t) : concentration of the drug, V : volume of the compartment Marc Lavielle
The MONOLIX software
The nonlinear mixed effects model The EM-type algorithms Some pharmacokinetics examples Estimation of the likelihood
intravenous administration and nonlinear elimination
dose D (t=0) → DRUG AMOUNT Q(t) → nonlinear elimination dC Vm C =− dt V (Km + C ) C (t) : concentration of the drug, (Vm , Km ) : Michaelis-Menten elimination parameters, V : volume of the compartment.
Marc Lavielle
The MONOLIX software
The nonlinear mixed effects model The EM-type algorithms Some pharmacokinetics examples Estimation of the likelihood
Importance Sampling The Importance Sampling algorithm computes an estimate `M (y ) of the observed likelihood. Z `(y , θ) = p(y , φ)d φ Z = h(y |φ)π(φ)d φ Z π(φ) = h(y |φ) π ˜ (φ)d φ π ˜ (φ)
1 2
Draw φ(1) , φ(2) , . . . , φ(M) with the distribution π ˜, Let M 1 X π(φ(j) ) `M (y ) = h(y |φ(j) ) M π ˜ (φ(j) ) j=1
Marc Lavielle
The MONOLIX software
The nonlinear mixed effects model The EM-type algorithms Some pharmacokinetics examples Estimation of the likelihood
Importance Sampling
M 1 X π(φ(j) ) `M (y ) = h(y |φ(j) ) M π ˜ (φ(j) ) j=1
E`M (y ) = `(y ) and Var`M (y ) = O(1/M) The instrumental distribution used in Monolix : 1) Estimate the conditional mean and variance of φ, (using MCMC), 2) Use for π ˜ a Gaussian distribution with these parameters.
Marc Lavielle
The MONOLIX software
The nonlinear mixed effects model The EM-type algorithms Some pharmacokinetics examples Estimation of the likelihood
Marc Lavielle
The MONOLIX software
