Jul 27, 2012 - dependent function (VE = 0.78 Ã exp[â0.06t]) approximating the waning vaccine efficacy seen in the RV144 vaccine trial in Thailand (Fig. 1).
TUPE179
Differing models, same results: testing the consistency of seven HIV vaccine impact models across eight populations Kripke, Katharine1 and Hamilton, Matthew2 1Futures Institute and 2Futures Group
www.aids2012.org
Washington D.C., USA, 22-27 July 2012
In the September 2011 issue of the journal Vaccine, seven articles presented models of HIV vaccine impact in various populations in South Africa, Thailand, the U.S., and Australia. All models were based on a set of common parameters: • Models all employed the same timedependent function (VE = 0.78 × exp[−0.06t]) approximating the waning vaccine efficacy seen in the RV144 vaccine trial in Thailand (Fig. 1) • Models were to report fractions of infections averted over a 10-year follow-up period following a single round of mass vaccinations of 30% and 60% of sexually active adults
0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0
Rapidly waning vaccine efficacy used by modelers
0 6 12 18 24 30 36 42 48 54 60 66 72 78 84 90 96 102 108 114 120
People often question the usefulness of modeling given that differences in model structure and assumptions can potentially lead to widely different results. We took advantage of a series of HIV vaccine models to assess the impact of differences in model structure and population on the estimates of HIV vaccine impact.
vaccine efficacy
Introduction
months since vaccination
Figure 1. Relationship between HIV vaccine efficacy vs. time used in all models
www.aids2012.org
Washington D.C., USA, 22-27 July 2012
Methods
The seven articles were analyzed for differences in model architecture, the model populations, and the reported percentages of infections averted.
www.aids2012.org
Washington D.C., USA, 22-27 July 2012
Despite differences in model architecture and application to very different populations (concentrated and generalized epidemics, different parts of the world), the percent reduction in incidence over ten years ranged only between 3% and 7% for the 30% vaccination coverage scenario and between 6% and 14% for the 60% vaccination coverage scenario (Fig. 2). Vaccine impact was low across all models because of the rapidly waning vaccine efficacy and the standardization requiring a single vaccination campaign. Models differed with respect to overall type of model (continuous, discrete, compartmental, individual-based microsimulation), how the models dealt with risk groups, transmission, and ART, the specific populations modeled, the specific way in which the waning efficacy was incorporated into the model, and the specific scale-up strategy to reach 30% or 60% coverage.
percent reduction in vaccine incidence
Results 100% 90%
Projected vaccine impact varies little across models and populations
80% 70% 60% 50%
40% 30% coverage
30%
60% coverage
20% 10% 0% Nagelkerke Schneider Andersson Andersson Hontelez Andersson & Stover & Stover & Owens (South (Thailand) Africa)
Long
Gray
Figure 2. Percent reduction in vaccine incidence for two levels of vaccine coverage across the different models.
www.aids2012.org
Washington D.C., USA, 22-27 July 2012
Authors
Model Type
Description
Population
Software
Mechanism to Incorporate Waning Vaccine Efficacy
Scale-up Time to Target Coverage (30% or 60%)
% incidence reduction ART Included 30% coverage 60% coverage
deterministic Nagelkerke et al. continuous-time compartmental
32 compartments; 4 risk groups, no MSM or PWID; 2 treatment groups (none, ART)
transition rates from heterosexual, female ModelMake vaccinated to CSW and male client unvaccinated r 3.0.3 population in Thailand. compartments
instantaneous at start of 2016
yes
7%
14%
Schneider et al.
deterministic continuous-time compartmental
98 compartments; 7 risk groups; 3 treatment groups (none, 1st line, 2nd line ART);
heterosexual, PWID, MSM, female CSW and MATLAB male client population R2009a in Thailand
transition rates from vaccinated to unvaccinated compartments
instantaneous at start of first year
yes (1st and 2nd line treatment)
3%
7%
Andersson & Stover (South Africa)
deterministic discrete-time compartmental
demographic projection based; 80 compartments and 5 risk groups (not counting under-15)
heterosexual, CSW, MSM, and PWID population in South Africa (national level)
number of people in
1 year in 2020
yes
4%
8%
demographic projection based; 80 compartments and 5 risk groups (not counting under-15)
heterosexual, CSW, MSM, and PWID population in Thailand (national level)
1 year in 2020
yes
5%
10%
stochastic discretetime individualbased microsimulation
explicit demography; individual-based partnership formation, transmission and disease progression; 3 treatment groups;
rural South African subdistrict with wellfunctioning HIV Rx & care
yes (dependent on historical clinic-by-clinic rollout in Hlabisa subdistrict.)
5%
9%
deterministic Andersson & discrete-time Stover (Thailand) compartmental
Hontelez et al.
compartments in each Spectrum year is divided by the AIM Module duration of vaccine in years number of people in compartments in each Spectrum year is divided by the AIM Module duration of vaccine in years
STDSIM
unclear
6 months in 2015
deterministic discrete-time compartmental
12 compartments defined by 2 sex, 4 disease status, and 2 vax status (for 2 of Soweto, South Africa the 4 disease groups); no MSM, PWID, CSW or other high-risk group
Excel 2003
transition rates from HIVto asymptomatic HIV+ compartments depend on annualized expontential decay in efficacy
no (folded into 1 year in first year rates of disease progression.)
3%
6%
Long et al.
deterministic continuous-time compartmental
50 compartments; 6 risk groups including MSM/PWID intersection, but not CSW or clients; includes status-aware and status-unaware groups; 2 treatment groups (none and ART);
MATLAB R2010b
transition rates from HIVto asymptomatic HIV+ compartments
instantaneous at start of first year
yes
5%
10%
Gray et al.
stochastic discretetime individualbased microsimulation
individual-based partnership formation, MSM in New South sexual behavior (e.g. serosorting); transmission Wales, Australia and disease progression; 3 treatment groups;
MATLAB R2010b
per-act probabilities of infection depend on daily exponential decay in efficacy
1 year in 2010
yes
5%
10%
Andersson, Paltriel & Owens
United States
www.aids2012.org
Washington D.C., USA, 22-27 July 2012
Discussion Policymakers are often confounded by the wide variety of approaches to modeling and frequently wonder how to distinguish different modeling approaches in terms of their usefulness and appropriateness. This analysis should reassure end-users of modeling that at least in this one example, differences between these eight models had relatively small impact on the projected outcome. Despite differences in the model structures and populations, there was little variation in the projected incidence reduction across the different models. The variation was small, for example, when compared to the magnitude of confidence intervals acceptable for efficacy of licensed vaccines. This suggests that when models standardize the most important parameters (in this case, vaccine efficacy, vaccination scheme, coverage, and modeling question), the results can be robust to even basic differences in model design and population. While the accuracy of these model predictions cannot yet be determined and all models are sensitive to their assumptions, the fact that these models produced comparable results indicates an internal consistency which is critical for utilizing such models to predict the benefits of a vaccine in the future.
www.aids2012.org
Washington D.C., USA, 22-27 July 2012
Literature Cited • • • •
• • • •
Andersson, K.M. and Stover, J. 2011. The potential impact of a moderately effective HIV vaccine with rapidly waning protection in South Africa and Thailand. Vaccine 29:6092– 6099. Andersson, K.M., Paltiel, A.D., and Owens, D.K. 2011. The potential impact of an HIV vaccine with rapidly waning protection on the epidemic in Southern Africa: Examining the RV144 trial results. Vaccine 29:6107– 6112. Gray, R.T., Ghaus, M. H., Hoare, A., and Wilson, D.P. 2011. Expected epidemiological impact of the introduction of a partially effective HIV vaccine among men who have sex with men in Australia. Vaccine 29:6125– 6129. Hankins, C.A., Glasser, J.W., and Chen, R.T. 2011. Modeling the impact of RV144-like vaccines on HIV transmission. Vaccine 29:6069– 6071. Hontelez, J.A.C., Nagelkerke, N., Bärnighausen, T., Bakker, R., Tanser, F., Newell, M.-L., Lurie, M.N., Baltussen, R., and de Vlas, S.J. 2011. The potential impact of RV144-like vaccines in rural South Africa: A study using the STDSIM microsimulation model. Vaccine 29:6100– 6106. Long, E.F. and Owens, D.K. 2011. The cost-effectiveness of a modestly effective HIV vaccine in the United States. Vaccine 29: 6113– 6124. Nagelkerke, N.J.D., Hontelez, J.A.C., and de Vlas, S.J. 2011. The potential impact of an HIV vaccine with limited protection on HIV incidence in Thailand: A modeling study. Vaccine 29:6079– 6085. Schneider, K., Kerr, C.C., Hoare, A., and Wilson, D.P. 2011. Expected epidemiological impacts of introducing an HIV vaccine in Thailand: A model-based analysis. Vaccine 29:6086– 6091.
www.aids2012.org
Washington D.C., USA, 22-27 July 2012
Abbreviations
• • • • •
ART – antiretroviral therapy CSW – commercial sex workers HIV – Human Immunodeficiency Virus MSM – men who have sex with men PWID – people who inject drugs
www.aids2012.org
Washington D.C., USA, 22-27 July 2012
