The enlightening journey of three Data Generating Models Konstantinos Pateras Joint work with Stavros Nikolakopoulos and Kit CB Roes University Medical Center Utrecht
Sunday 19th July 93 A.Cox
Konstantinos Pateras (UCMU)
An enlightening journey
Sunday 19th July 93 A.Cox
1 / 11
Overview 1
Prologue
2
Chapter 1: The identification of three witnesses
3
Chapter 2: The DGMs testimonies Section 1, The pCfixed DGM Section 2, The pAverage DGM Section 3, The pRandom DGM
4
Chapter 3: The DGMs against each other
5
Chapter 4: The DGMs meet the three witnesses Section 1, 95% coverage Section 2, Power
6
Epilogue
Konstantinos Pateras (UCMU)
An enlightening journey
Sunday 19th July 93 A.Cox
2 / 11
Prologue Multilevel binomial models are usually compared through simulations (e.g Lambert 2005, Inthout 2014, Novianti 2014, Friede 2016). Different Data Generating Models (DGMs) are employed. Each DGM encompasses specific assumptions/restrictions. Results of individual studies risk to be inconsistent. Methodological systematic reviews avoid to report DGMs (Veroniki 2015, Langan 2015).
Problem Are we comparing apples with oranges? The journey of three DGMs against a two trial meta-analysis.
Konstantinos Pateras (UCMU)
An enlightening journey
Sunday 19th July 93 A.Cox
3 / 11
Chapter 1: The identification of three witnesses Three standard methods for meta-analysis
Inverse variance (IV) random-effects (DL) & fixed-effects model (FE) θˆ =
Pk
i=1 wi Yi /
Pk
i=1 wi ,
, i = 1, ..., k
τˆ can be the DerSimonial and Laird (DL) estimator. wi,RE = 1/(si2 + τˆ2 ) when τˆ > 0 or wi,FE = 1/si2 when τˆ = 0 p P σ ˆθ = 1/ wi θˆ ± σ ˆθ z(1−α/2)
IV RE model with Hartung and Knapp correction (HK) 1 P ˆ2 wi,RE (Yi − θ) k −1 √ θˆ ± qˆ σθ t(k−1);(1−α/2) q=
Konstantinos Pateras (UCMU)
An enlightening journey
Sunday 19th July 93 A.Cox
4 / 11
Chapter 2: Section 1, The pCfixed DGM
pCfixed Set θ,τ , a range for piC and a range for mi , i = 1, ..., k and j = (C )ontrol, (T )reatment nij = mi ∼ Uniform(mlo , mup ) θi ∼ Normal(θ, τ ) piC ∼ Uniform(α, β) piT = piC · exp(θi )/(1 − piC + piC · exp(θi )) rij ∼ Binomial(pij , nij ) i.e. Lambert 2005, Novianti 2014, Kuss 2015, IntHout 2016
Konstantinos Pateras (UCMU)
An enlightening journey
Sunday 19th July 93 A.Cox
5 / 11
Chapter 2: Section 2, The pAverage DGM pAverage Set θ,τ , a range for pi 0 and a range for mi . nij = mi ∼ Uniform(mlo , mup ) θi ∼ Normal(θ, τ ) pi 0 ∼ Uniform(α, β) P pi 0 = 2j=1 pij /2. (p ) · (1 − p ) iC iT . θi = log (piT ) · (1 − piC ) Solving (6) and (7) we acquire pij . rij ∼ Binomial(pij , nij ) i.e. Veroniki 2013, Inthout 2014
Konstantinos Pateras (UCMU)
An enlightening journey
Sunday 19th July 93 A.Cox
6 / 11
Chapter 2: Section 3, The pRandom DGM
pRandom Set θ, τ , piC ,Init and a range for mi . piT ,Init = piC ,Init · exp(θ)/(1 − piC ,Init + piC, Init · exp(θ)) nij = mi ∼ Uniform(mlo , mup ) µij = log (pij,Init /1 − pij,Init ) √ logitij ∼ Normal(µij , τ / 2) 1 pij = 1 + e −logitij rij ∼ Binomial(pij , nij ) i.e. Knapp 2001, Gonnerman 2015, Friede 2016
Konstantinos Pateras (UCMU)
An enlightening journey
Sunday 19th July 93 A.Cox
7 / 11
Chapter 3: The DGMs against each other Alternative hypothesis (θ = 1) 1.0
pRandom
0.5
0.8
0.8
1.0
Null hypothesis (θ = 0) pRandom
2
0.6
0.6
3
0.4
0.4
4.5
5
5
0.2
7
0.2
6 8
4
3.5
2.5 2
0.2
0.4
0.6
0.8
1.0
0.6
0.8
1.0
0.4
0.6
0.8
1.0
0.4
0.6
0.8
1.0
0.8
pAverage
0.4
14
0.2
8
6
5
0.2
0.0
4
0.0
6
7
9
1
0.0
10
8 5
3
4
0.4
0.6
0.8
1.0
pCfixed
pCfixed
3 4
2
0.8
5
0.4
0.6
0.8 0.4
0.6
2
0.2
3
2
4
2
0.0
1.0
4
0.2
0.2
6
8
8
9
0.0
0.0
14
0.0
7 8
12 10
0.2
0.4
0.6
0.8
Control event probability density
1.0
6
0.0
5
3
0.2
0.4
12
2
3
2
1.0
0.2
0.6
0.8 0.6 0.4
1
8
Konstantinos Pateras (UCMU)
1
0.0
1.0
pAverage
9
Treatment event probability density
1.0
0.0
1.5
1
4
0.0
0.0
3
0.2
Control event probability density
An enlightening journey
Sunday 19th July 93 A.Cox
8 / 11
Chapter 4: Section 1, 95% coverage pRandom
pAverage 0
DerSimonian & Laird
1
2
(MA of two small trials) pCfixed
3
4
Hartung & Knapp
Fixed−effect
1.00
τ = 0.001
0.95 0.90 0.85 0.80
Coverage of 95% CI
0.75
0
1
2
3
4
0
1
2
3
4
3
4
Overall treatment effect (log odds ratio) pRandom
pAverage 0
DerSimonian & Laird
1
2
pCfixed 3
4
Hartung & Knapp
Fixed−effect
1.00 0.95
τ=1
0.90 0.85 0.80 0.75
0
1
2
3
4
0
1
2
Overall treatment effect (log odds ratio)
Konstantinos Pateras (UCMU)
An enlightening journey
Sunday 19th July 93 A.Cox
9 / 11
Chapter 4: Section 2, Power pRandom
(MA of two small trials)
pAverage 0
DerSimonian & Laird
1
2
pCfixed 3
4
Hartung & Knapp
Fixed−effect
τ = 0.001
0.8 0.6 0.4
Empirical power
0.2
0
1
2
3
4
0
1
2
3
4
3
4
Overall treatment effect (log odds ratio) pRandom
pAverage 0
DerSimonian & Laird
1
2
pCfixed 3
4
Hartung & Knapp
Fixed−effect
τ=1
0.8 0.6 0.4 0.2
0
1
2
3
4
0
1
2
Overall treatment effect (log odds ratio)
Konstantinos Pateras (UCMU)
An enlightening journey
Sunday 19th July 93 A.Cox
10 / 11
Epilogue Always look where you want to go
Pateras, Konstantinos, Nikolakopoulos Stavros, and Kit Roes. ”Data-generating models of dichotomous outcomes: Heterogeneity in simulation studies for a random-effects meta-analysis.” Statistics in medicine 37.7 (2018): 1115-1124.
[email protected] This research was supported by the EU FP7 project Asterix
Konstantinos Pateras (UCMU)
An enlightening journey
Sunday 19th July 93 A.Cox
11 / 11
Chapter 4: Section 1, 95% coverage pRandom
pAverage 0
DerSimonian & Laird
1
2
(MA of two large trials) pCfixed
3
4
Hartung & Knapp
Fixed−effect
τ = 0.001
0.8
0.6
Coverage of 95% CI
0.4
0
1
2
3
4
0
1
2
3
4
3
4
Overall treatment effect (log odds ratio) pRandom
pAverage 0
DerSimonian & Laird
1
2
pCfixed 3
4
Hartung & Knapp
Fixed−effect
τ=1
0.8
0.6
0.4
0
1
2
3
4
0
1
2
Overall treatment effect (log odds ratio)
Konstantinos Pateras (UCMU)
An enlightening journey
Sunday 19th July 93 A.Cox
12 / 11
Chapter 4: Section 2, Power pRandom
(MA of two large trials)
pAverage 0
DerSimonian & Laird
1
2
pCfixed 3
4
Hartung & Knapp
Fixed−effect
τ = 0.001
0.8 0.6 0.4
Empirical power
0.2
0
1
2
3
4
0
1
2
3
4
3
4
Overall treatment effect (log odds ratio) pRandom
pAverage 0
DerSimonian & Laird
1
2
pCfixed 3
4
Hartung & Knapp
Fixed−effect
τ=1
0.8 0.6 0.4 0.2
0
1
2
3
4
0
1
2
Overall treatment effect (log odds ratio)
Konstantinos Pateras (UCMU)
An enlightening journey
Sunday 19th July 93 A.Cox
13 / 11
