TIDI - University of Leicester

Introduction to Transparent Interactive Decision Interrogator (TIDI) Sylwia Bujkiewicz,1 Hayley Jones,2 Monica Lai,1 Nicola Cooper,1 Neil Hawkins,3 Keith Abrams,1 David Spiegelhalter,4 Alex Sutton1 1 Department

of Health Sciences, University of Leicester, of Social Medicine, University of Bristol, 3 Oxford Outcomes, 4 Centre for Mathematical Sciences, University of Cambridge

2 Department

4 February 2011 S. Bujkiewicz et al. (University of Leicester)

TIDI

1 / 19

Acknowledgements

Acknowledgements Authors would like to thank Rebecca Turner for sharing her Stata code for performing the bias adjustments analysis Hazel Squires for sharing the anti-D decision model Mark Rodgers, David Epstein, Dawn Craig, Tiago Fonseca, Laura Bojke, Huiqin Yang, Mark Sculpher and Nerys Woolacott for their collaboration within the appraisal of TNF-alpha inhibitors in treatment of psoriatic arthritis Janet Robertson, Meindert Boysen and the NICE Technology Appraisal Committee A members for allowing us to pilot TIDI during their (real) committee meetings and for their invaluable comments.

S. Bujkiewicz et al. (University of Leicester)

TIDI

2 / 19

Introduction

Introduction

TIDI is an Excel-based user interface for facilitating transparent and efficient decision making in Health Technology Assessment (HTA) TIDI has been created as a concept of presenting results of decision models in HTA to decision makers in a way that enables them to critique evidence, uncertainty and assumptions that are the integral part of the model. TIDI will be presented here based on the anti-D example (decision model and meta-analysis)

S. Bujkiewicz et al. (University of Leicester)

TIDI

3 / 19

Background

Decisions @ NICE

Decision-making in health care Decision-making systems in health care are increasingly designed in such a way to ensure equity of access and to optimise the use of limited healthcare resources In England and Wales the decisions about reimbursement of the new healthcare technologies are conducted by the National Institute for Health and Clinical Excellence (NICE), and they are often based on complex health economic models. These usually have large number of inputs related to effectiveness and cost which are used to calculate cost-effectiveness. Health technology appraisal documents are produced both by independent academic teams and manufacturers, for consideration by NICE appraisal committees.

S. Bujkiewicz et al. (University of Leicester)

TIDI

4 / 19

Background

Decisions @ NICE

Challenges in the decision-making (1)

Decision makers make decisions based on their own informal judgments about evidence, uncertainty and the assumptions that are an integral part of decision models. Committee members frequently request further analysis to incorporate their opinions on numerous inputs and assumptions of the model, which delays decision (until updated analysis is completed and the committee reconvenes). There is a need in this decision process for a tool that would allow non-experts to be able to make those judgements more formally and also in real time in order to improve the transparency and efficiency of the process.

S. Bujkiewicz et al. (University of Leicester)

TIDI

5 / 19

Background

Decisions @ NICE

Challenges in the decision-making (2): software Health economic decision models for the NICE appraisals are designed using a number of different types of software. Excel is mostly known and accessible to a wider community, however models designed in Excel are incomprehensible and there have been reports of Excel built-in statistical functions being faulty. Models in R and WinBUGS are transparent in terms of the clarity of the code which makes them easy to follow. Use of WinBUGS has further advantages: it allows to carry out more complex (Bayesian) evidence syntheses that are not possible in more standard packages it allows for more integrated one-step approach to analytic model where preliminary analyses (e.g. meta-analysis) and the decision model are conducted within single analytical framework.

However models in R or WinBUGS are difficult to use or to be interrogated and fully appraised by non-technical decision-makers. S. Bujkiewicz et al. (University of Leicester)

TIDI

6 / 19

Background

Aims of TIDI

Aims of TIDI (1)

Transparent Interactive Decision Interrogator (TIDI) has been developed to aid the decisions made by NICE committees during their consultations It is a user interface which aims to combine the accessibility of Excel with the clarity and flexibility of R and WinBUGS models Interactive Excel interface will make it possible for the decision-makers (e.g. NICE committee) not only to run the models but also to have control over the model parameters and assumptions without the need of knowledge of how to construct the decision model or to use R or WinBUGS.

S. Bujkiewicz et al. (University of Leicester)

TIDI

7 / 19

Background

Aims of TIDI

Aims of TIDI (2)

TIDI aims to make the NICE decision process more transparent by making the decision models more accessible to critique by non-statisticians (all decision-makers and other stakeholders) and also by using flexible and clear model components developed in R and WinBUGS

High performance of R and WinBUGS models gives possibility of immediate control and execution of models in real time during the NICE committee meetings making the decision process more efficient.

S. Bujkiewicz et al. (University of Leicester)

TIDI

8 / 19

Software elements

How it works

All the model parameters and actions are controlled from the Excel interface programmed using Visual Basic (VBA) Data is passed form Excel worksheet to R workspace through RExcel where decision models, evidence syntheses are executed This can be taken further to transfer data to and process in WinBUGS (via R2WinBUGS) Results of the decision model are transferred from R back to Excel

S. Bujkiewicz et al. (University of Leicester)

TIDI

9 / 19

Example: anti-D

Aims

Anti-D model (1)

This interactive interface will be demonstrated using the example of a decision model which evaluates whether four anti-D treatments (D-Gam, Partobulin SDF, Rhophylac, and WinRho SDF) are cost-effective when used to reduce the incidence of sensitisation in RhD-negative women and hence of haemolytic disease of the newborn. Aim of the HTA was to compare 3 strategies: 1 2 3

Conventional management only (control) Routine AADP to RhD-negative primigravidae only Routine AADP all RhD-negative pregnant women

The efficacy of the 4 treatment was assumed equal (only varying costs).

S. Bujkiewicz et al. (University of Leicester)

TIDI

10 / 19

Example: anti-D

Model

Anti-D model (2) Forward sampling of outcomes based on large number of parameters, including Baseline sensitisation rate OR for anti-D from meta-analysis (Treatment efficacy) Various population assumptions, e.g. pregnancy rates Probability of fetal loss / disabilities, given affected fetus.

Costs associated with Treatment with anti-D Management of sensitisations Major / minor disabilities

Incremental Cost Effectiveness Ratios (ICERs): cost per Sensitisation avoided Case of haemolytic disease avoided Fetal loss avoided Life year gained Quality adjusted life year gained (QALY) S. Bujkiewicz et al. (University of Leicester)

TIDI

11 / 19

Interface simulation

S. Bujkiewicz et al. (University of Leicester)

TIDI

12 / 19

Interface simulation

S. Bujkiewicz et al. (University of Leicester)

TIDI

13 / 19

Interface simulation

S. Bujkiewicz et al. (University of Leicester)

TIDI

14 / 19

Interface simulation

S. Bujkiewicz et al. (University of Leicester)

TIDI

15 / 19

Interface simulation

S. Bujkiewicz et al. (University of Leicester)

TIDI

16 / 19

Discussion

Final comments Meta-analysis module can be used as a stand-alone software for use in any evidence-based decision making in healthcare. This concept could be taken further to be accessible online, and integrated into the systematic reviews within the Cochrane library. This would allow the reader to incorporate their beliefs, carry out sensitivity analysis or tailor the analysis for a particular patient population. TIDI has also been developed for an economic model and associated evidence syntheses commissioned by NICE assessing the cost-effectiveness of TNF-alpha inhibitors when used in treatment of psoriatic arthritis. This interface was used by the NICE appraisal committee during their meetings to support their decision making process.

S. Bujkiewicz et al. (University of Leicester)

TIDI

17 / 19

References

References

1

Bujkiewicz S, Jones H E, Lai M C W, Cooper N J, Hawkins N, Squires H, Abrams K R, Spiegelhalter D J, Sutton A J (2011), ”Development of a Transparent Interactive Decision Interrogator to facilitate the decision making process in health care”, Value in Health (in press).

2

Rodgers M, Epstein D, Bojke L, Yang H, Craig D, Fonseca T, Myers L, Bruce I, Chalmers R, Bujkiewicz S, Lai MCD, Cooper NJ, Abrams KR, Spiegelhalter DJ, Sutton AJ, Sculpher M, Woolacott N (2011), ”Etanercept, Infliximab and Adalimumab for the Treatment of Psoriatic Arthritis: a Systematic Review and Economic Evaluation”, Health Technology Assessment (in press).

S. Bujkiewicz et al. (University of Leicester)

TIDI

18 / 19

References

References 1

Turner R M, Spiegelhalter G J, Smith G C S, Thompson S G (2009): ”Bias modelling in evidence synthesis”, Journal of the Royal Statistical Society A 172, pp. 21-47.

2

Pilgrim H, Lloyd-Jones M, Rees A. ”Routine antenatal anti-D prophylaxis for RhD-negative women: a systematic review and economic evaluation”, Health Technol Assess 2009;13:number 10:0-126.

3

Baier T and Neuwirth E (2007), ”Excel :: COM :: R”, Computational Statistics 22/1, pp. 91-108

4

Sturtz S, Ligges U, Gelman A (2005): ”R2WinBUGS: A Package for Running WinBUGS from R.” Journal of Statistical Software, 12(3), 1-16.

5

Walkenbach J (2007): ”Excel 2007 Power Programming with VBA”, Wiley Publishing, Inc., Indianapolis, Indiana.

S. Bujkiewicz et al. (University of Leicester)

TIDI

19 / 19