Applying evidence from economic evaluations to ...

Applying evidence from economic evaluations to translate cancer survivorship research into care Janet S. de Moor, Catherine M. Alfano, Nancy Breen, Erin E. Kent & Julia Rowland Journal of Cancer Survivorship Research and Practice ISSN 1932-2259 J Cancer Surviv DOI 10.1007/s11764-015-0433-3

1 23

Your article is protected by copyright and all rights are held exclusively by Springer Science+Business Media New York (outside the USA) . This e-offprint is for personal use only and shall not be self-archived in electronic repositories. If you wish to self-archive your article, please use the accepted manuscript version for posting on your own website. You may further deposit the accepted manuscript version in any repository, provided it is only made publicly available 12 months after official publication or later and provided acknowledgement is given to the original source of publication and a link is inserted to the published article on Springer's website. The link must be accompanied by the following text: "The final publication is available at link.springer.com”.

1 23

Author's personal copy J Cancer Surviv DOI 10.1007/s11764-015-0433-3

Applying evidence from economic evaluations to translate cancer survivorship research into care Janet S. de Moor & Catherine M. Alfano & Nancy Breen & Erin E. Kent & Julia Rowland

Received: 24 November 2014 / Accepted: 27 January 2015 # Springer Science+Business Media New York (outside the USA) 2015

Abstract Purpose This paper summarizes recommendations stemming from the meeting, Applying Evidence from Economic Evaluations to Translate Cancer Survivorship Research into Care, hosted by the National Cancer Institute. Methods The meeting convened funded investigators, experts in cancer control, survivorship, health economics, and team science to identify the economic and health services data needed to facilitate the dissemination of cancer survivorship interventions into care and how survivorship and health economic investigators can successfully collaborate together and with other stakeholders. Results Recommendations from the meeting are as follows. First, investigators must engage key stakeholders early in the planning process to understand the outcomes and cost domains on which they base decisions. Second, evaluations of intervention efficacy and value should be conducted using standardized and comparable measures and analytic approaches to enable comparisons across studies. Finally, a J. S. de Moor (*) : N. Breen Health Services and Economics Branch, Applied Research Program, Division of Cancer Control and Population Sciences, National Cancer Institute, 9609 Medical Center Drive, 3E438, MSC 9764, Bethesda, MD 20892-9764, USA e-mail: [email protected] C. M. Alfano Health Behaviors Research Branch, Behavioral Research Program, Division of Cancer Control and Population Sciences, National Cancer Institute, Bethesda, MD, USA E. E. Kent Outcomes Research Branch, Applied Research Program, Division of Cancer Control and Population Sciences, National Cancer Institute, Bethesda, MD, USA J. Rowland Office of Cancer Survivorship, Division of Cancer Control and Population Sciences, National Cancer Institute, Bethesda, MD, USA

health economist should be included during the planning phase of the study so that the economic evaluation is pursued in concert with the survivorship intervention. Conclusions Economic analyses, from the perspective of key stakeholders, must be incorporated into survivorship intervention research. The results from these analyses should be disseminated in a manner that is transparent, accessible, and comparable across studies. Implications for Cancer Survivors To optimize cancer survivors’ health and quality of life, it is essential deliver highquality and high-value care. Incorporating economic analyses into survivorship intervention research can inform the translation of effective interventions into practice. Keywords Cancer survivorship . Cost-effectiveness . Value . Intervention . Dissemination

Introduction As of 2014, an estimated 14.5 million cancer survivors were living in the USA, a number that is projected to increase to approximately 19 million over the next decade [1]. Although advances in cancer therapy have contributed to improvements in long-term survival rates, the toxicities associated with cancer treatment can result in physical and psychosocial problems that require ongoing clinical and self-management [2–5]. Investigators have made great strides in documenting the long-term and late effects of cancer and treatment and in developing rigorous interventions to enhance post-treatment functioning and improve survivorship care delivery [6–9]. Despite these advances, few evidence-based cancer survivorship interventions are disseminated into practice or policy, resulting in an inefficient use of research dollars and, more importantly, constraining the potential impact of scientific advances on population health.

Author's personal copy J Cancer Surviv

Factors influencing dissemination and implementation are diverse and multifaceted; however, one fundamental barrier is the mismatch between the information generated by many randomized controlled trials and the types of data needed by potential adopters. Randomized clinical and behavioral trials are designed to maximize internal validity and typically include a resource-intensive protocol and specialized delivery requirements. Such strict requirements are impractical to implement in many practice settings. Actual clinical environments have competing demands, resource constraints, and patients who are diverse in terms of their health status and motivation or capacity for adherence to survivorship interventions [10, 11]. Of particular importance, most interventions fail to capture information about intervention cost and costeffectiveness or the impact on patients’ health service utilization, time, or employment outcomes [10–13]. These data provide much needed information for patients, providers, health-care decision makers, payers, and other stakeholders. Thus, to accelerate the translation of interventions for cancer survivors into care and optimize the quality of care being delivered, it is essential to collect data about economic impact as part of survivorship intervention research. Furthermore, to make survivorship interventions relevant and meaningful to key stakeholders, results of economic analyses need to be disseminated in a way that is transparent, accessible, and comparable across studies [12, 13]. To stimulate research partnerships between economic and survivorship investigators in order to successfully collect and analyze relevant cost data, the Office of Cancer Survivorship in collaboration with the Health Services and Economics Branch of the Division of Cancer Control and Population Sciences at the National Cancer Institute awarded administrative supplements from 2008 to 2009 to seven R01 investigators who were evaluating the efficacy of a survivorship intervention. The administrative supplements enabled these investigators to collect information about the economic and/or health services impact of their work, and projects were required to outline how the information collected was expected to affect planning and/or policy decision-making. On September 20, 2013, the seven R01 investigators and their health economist partners together with experts in cancer control, cancer survivorship, health economics, and team science participated in the 1-day meeting, Applying Evidence from Economic Evaluations to Translate Cancer Survivorship Research into Care. The objectives of this meeting were to identify: (1) the types of economic and health service data needed to facilitate the dissemination of cancer survivorship interventions into care and (2) how survivorship and health economic investigators can successfully collaborate together and with other stakeholders to integrate economic evaluation into survivorship research. The meeting comprised individual presentations from the funded investigators and presentations from invited experts interspersed with group discussion.

The purpose of this paper is to outline recommendations stemming from the meeting. Investigators conducting survivorship and economic evaluation research represent an array of disciplines. However, we will refer to Bsurvivorship interventionists^ and Bhealth economists^ for the purposes of this paper. Likewise, survivorship interventions comprise a heterogeneous range of therapies, programs, and supportive services; however, we will refer to these activities and approaches collectively as Bsurvivorship interventions.^ Recommendation 1: identify and engage stakeholders who have an interest in the outcomes of your research The primary question asked of any intervention is: BDoes it work?^ The answer is rarely straightforward. Rather, it depends on the perspective of the individual asking the question and the types of outcomes he or she considers relevant. To move effective interventions into practice, survivorship interventionists need to identify individuals and groups with a direct interest in the outcomes of their research and solicit input from these constituencies about the health outcomes, processes of intervention delivery, cost, and other information that are most important to their decision-making. Although the objectives of individual research studies may vary, important stakeholder groups for many survivorship interventions include survivors and their families, health-care providers working in settings where a given intervention will be delivered, clinic or hospital administrators who make decisions about survivorship care, payers responsible for covering the cost of medical care services, and employers of survivors and/or their caregivers. As summarized in Table 1, these stakeholders will care about different outcomes. For example, meeting participants observed that patients might prioritize out-of-pocket costs associated with the intervention, providers and healthcare administrators might prioritize information related to the opportunity cost1of using staff time and existing infrastructure to deliver an intervention, payers might prioritize information on the cost of covering the intervention and other medical costs per the conditions of enrollees’ health insurance policies, and employers might prioritize information about survivors’ and caregivers’ productivity and lost time from work. The list of stakeholders and costs in Table 1 is not intended to be exhaustive but rather illustrative of the types of information needed to evaluate the economic impact of new survivorship interventions. Many models of stakeholder engagement have been developed for use in government agencies, academia, business, and other industries. These may serve as useful tools for investigators who are thinking about how to involve stakeholders in 1 Opportunity cost: the money or other benefits lost when pursuing a particular course of action instead of a mutually exclusive alternative (opportunity cost. Dictionary.com. Dictionary.com Unabridged. Random House, Inc. http://dictionary.reference.com/browse/ opportunitycost) (accessed: November 10, 2014).

Author's personal copy J Cancer Surviv Table 1

Costs according to different stakeholder perspectives Stakeholder perspective

Intervention delivery costs –Personnel (i.e., salaries, training) –Space and supplies –Intervention materials/drug

Direct medical costs –Hospitalizations, surgery, physician visits, psychological treatment, rehabilitation, other medical care –Prescription medication –Over-the-counter medications and supplements Direct non-medical costs –Child or adult care –Transportation and accommodations –Patient and caregiver time spent pursuing care Indirect medical costs –Patient and caregiver days lost from work and level of productivity

Survivors and families

Health-care providers and administrators

Payers (health insurance)

Employers

Societal

Out-of-pocket expenses

Opportunity cost of using existing staff or infrastructure. Noncovered costs for new infrastructure or staff

Covered services

None

All costs

Out-of-pocket expenses

Reimbursement rate

Covered services

None

All costs

Out-of-pocket costs

None

None

None

All costs

Opportunity cost

None

None

Costs associated with lost productivity and time off from work

All costs

Adapted from [37]

a particular project [14–17]. Although existing models of stakeholder engagement were developed for different purposes, common elements include: (1) identification of key stakeholders and their domains of interest and (2) implementation of an iterative feedback process to ensure that stakeholder needs and preferences continue to guide intervention design and implementation. Recommendation 2: capture information about intervention costs needed to achieve health outcomes in order to assess the value of new interventions to key stakeholders It is critical to capture data about intervention cost and effectiveness from the perspective of patients and families, providers, health-care administrators, payers, and other stakeholders in order to determine whether health outcomes achieved from a given intervention are worth the resources required to deliver it. Examples of relevant costs are summarized in Table 1. The selection of an economic evaluation approach and the universe of costs considered will be driven by the research questions pursued and the type of information needed by key stakeholder groups. Because the information needs of

stakeholders will drive data collection, these individuals should be initially engaged during the planning stages of the project. In cancer survivorship intervention studies, a new intervention is typically compared to either usual care or a minimal intervention. The field of health economics offers a variety of techniques to compare the cost-outcome ratios of different interventions, which can facilitate comparisons of efficiency and value within and across studies [18]. Cost-effectiveness and cost-utility analyses are two widely used methods that are applicable to survivorship research. An important distinguishing feature of these two approaches is how the health outcomes associated with an intervention are measured. In cost-effectiveness analysis, health outcomes are expressed in natural units such as years of life gained or symptom-free days. Thus, results from a cost-effectiveness analysis could be used to determine the cost per life year gained or cost per day free of symptoms. Conversely, in cost-utility analysis, health is expressed as a generic index such as quality-adjusted life years (QALY), a measure of health improvement that is based on the number of years of life added by the intervention weighted (using health utilities) by the quality of life gained [18, 19]. Health utilities are preferences for certain states of


health and can be directly measured using methods such as time tradeoff and standard gamble or derived indirectly from preference-based measures of health that incorporate utilities for different health outcomes such as EuroQol-5 dimension (EQ-5D) and Short Form-6 dimension (SF-6D) [20, 21]. Thus, results from a cost-utility analysis could be used to determine the cost per QALY gained. Though either approach can be used to assess whether a new intervention represents good value relative to competing interventions, the choice will ultimately depend upon the research question and the data available. Meeting participants discussed the optimal timing of the initiation of an economic evaluation of a survivorship intervention. At issue was whether data about cost and resource demands are irrelevant when an intervention is not effective. Participants noted that waiting to conduct an economic evaluation until efficacy has been established could limit the information available or make data collection more expensive, whereas planning the economic evaluation up front allows for efficient data collection. For example, records to monitor intervention dose or fidelity to the protocol can be built in and efforts made to systematically capture the resources needed to deliver the intervention. Additionally, preference-based measures of health-related quality of life needed to calculate quality-adjusted life years can also be included in the assessment battery. In cases where the intervention is not effective, information about the resource demands of the different components can still be used to inform the development of future interventions. Data on health-care utilization, reimbursement for services, and other costs and outcomes can sometimes be obtained from administrative data sources after the conclusion of the study. However, information about patients’ direct nonmedical costs and indirect costs are not routinely available and could be subject to recall bias if gathered post hoc. Additionally, permission to access medical records and to obtain records from other providers may be difficult or impossible after completion of the study. In practice, most economic analyses are not conducted alongside clinical trials [22]. However, such analyses can still produce useful estimates of cost and cost-effectiveness, especially if they are based primarily on administrative data and/or statistical modeling. Simulation modeling is frequently used in cost-effectiveness studies when it is necessary to incorporate data that was not captured directly by the original trial [22]. For example, simulation modeling can be used to estimate the effect of an intervention on quality of life or years of life saved based on trial data combined with related findings from the scientific literature. Through modeling, it is also possible to evaluate quality of life and costs during a longer time period than what was captured during the trial and estimate how the intervention is likely to work in standard practice [23]. Results from economic evaluation can be an important aid to decisions about resource allocation and health-care delivery

[22]. However, decisions about adopting, supporting, and reimbursing a new intervention as part of routine survivorship care can be influenced by many other factors. Survivorship interventionists are encouraged to work with key stakeholders to understand the broader context surrounding decisions to adopt a new intervention. Forces such as patient demand and preferences, provider interests, the absence of comparable treatments or services, potential impact on the market share of an institution or insurer, and the concordance of an intervention with the mission of an organization have been shown to trump scientific evidence as prime drivers of adoption [24, 25]. Investigators need to understand and harness these broader forces to the extent, if possible. Recommendation 3: studies should be planned with comparability in mind Recommendations to adopt a new intervention, treatment, or test into practice are rarely based on a single study. To accelerate the translation of evidence-based survivorship research into practice, investigators should use well-validated measures of study endpoints along with standard approaches to economic evaluation to facilitate comparisons across studies. Using standardized measures and approaches also informs meta-analyses and systematic reviews on which treatment guidelines and health-care decisions are frequently based. Several initiatives at NIH have been designed to drive consensus around common data elements and to support the use of common data elements in research. Of particular relevance to survivorship are the Grid-Enabled Measures (GEM) database and the Patient Reported Outcomes Measurement Information System (PROMIS®). The GEM database contains behavioral and social science measures organized by theoretical construct (http://cancercontrol.cancer.gov/brp/gem.html). GEM enables researchers to access and use common measures of study endpoints with the objective of exchanging harmonized data. PROMIS comprises item banks of tailored measures of patient-reported outcomes across physical, mental, and social domains of health (http://www.nihpromis.org/default). PROMIS measures use common domains and metrics to allow for comparisons across health conditions. Additional linkages to common data elements that are supported by NIH can be accessed through the Common Data Element Resource Portal (http://www.nlm.nih.gov/cde/). Those charged with making decisions about health-care delivery are tasked with the responsibility of comparing available options and deciding which services best fit their needs. To promote comparability across the economic analyses used to inform the allocation of health-care resources, the Panel for Cost Effectiveness in Health and Medicine recommended that investigators include the reference case, which defines a standard set of methods and assumptions to measure costs, value health outcomes, estimate effectiveness, as well as other


analytic considerations [26, 27]. The standard reference case uses the societal perspective and thus includes all costs and health outcomes regardless of who experienced them. For example, an economic evaluation of a survivorship intervention, according to the reference case, would consider costs incurred by survivors and families, providers, health-care administrators, payers, and any other groups impacted by the intervention relative to the health outcomes achieved. The reference case was designed to be broadly applicable to different health conditions and interventions [26]. However, meeting participants noted that a reference case developed specifically for cancer survivorship could help establish a minimum standard for cost-effectiveness in the field and build a comparable evidence base on which health-care decisions affecting cancer survivors could be based. Of course, stakeholder support for an intervention is likely to be influenced by the costs that each stakeholder group bears. Thus, the research question and needs of key stakeholders will inform the research perspective for any particular cost study, and economic analyses based on the reference case would typically be complemented by analyses tailored to a specific audience. The International Society for Pharmacoeconomics and Outcomes Research (ISPOR) has also developed consensus guidelines for the design, conduct, and reporting of costeffectiveness analysis in clinical trials in order to improve the comparability of economic evaluations across studies [28]. Like the recommendations from the Panel for Cost Effectiveness in Health and Medicine, the ISPOR guidelines are relevant to a broad spectrum of clinical research, encompassing but not exclusive to survivorship. Recommendation 4: partner with a health economist from the study’s inception Productive collaboration between survivorship interventionists and health economists is critical if we are to successfully expand the focus of research beyond efficacy to capture economic and other health service outcomes that are salient to key stakeholders and incorporate these variables into an economic evaluation. The recommendations outlined below to achieve this goal are based on feedback from the survivorship interventionists and health economists who participated in the meeting. However, their feedback echoes findings from the field of team science, which provides guidance for building and maintaining successful transdisciplinary teams that are applicable to many different types of collaborations [29]. Individuals working in large academic medical centers or university settings are likely to have access to colleagues working at their institution who have training or experience in health services and health economics. In contrast, those working in smaller hospitals or community settings may need to look outside their institution to a collaborator who brings economic expertise to the project. Su rvivorship

interventionists and health economists will differ in their theoretical training and orientation, methodologies that are typically applied in research, vernacular, and norms around collaborative work and authorship. Thus, deliberate efforts are often needed to bridge the fields by formally establishing common expectations and articulating a shared vision for collaboration and science. Communication needs to occur about roles and responsibilities of all team members throughout the study. To maximize the potential for efficient and effective studies, these partnerships should be formed when the research is being conceptualized. Typically, the involvement of a health economist fluctuates over the course of the study with involvement being most intensive in the planning stages of the project, including writing the grant and developing the study protocol and at the end of the study when data are being analyzed, interpreted, and disseminated. A major barrier to productive collaboration and high-quality economic evaluation is involving the health economist too late in the planning process and compensating them for minimal effort. This limits the contribution that person can bring, diminishes her or his importance on the team, and constrains the quality and impact of the research. The survivorship interventionist should work with the health economist to create a shared vision for the project and communicate to the research team the goal of the study. Developing a shared vision is dependent on translating discipline-specific constructs, terminology and methods, and educating the team about the goals of the intervention and economic evaluation component. Productive collaboration is supported when the study team is willing to learn from each other and try new approaches as well as when relationships are built on trust and mutual respect, including recognition of the importance of the economic evaluation. Regular team meetings can facilitate the development of a shared vision for the study and a common language and provide an opportunity for open communication and shared learning. Regular meetings can be a venue for addressing questions about study protocol, data collection, analytic approach, and plans to disseminate results as well as for managing emergent issues or challenges. Although somewhat outside the control of individual investigators, institutional culture can foster transdisciplinary collaboration by rewarding collaborative work during the tenure and promotion process and by providing financial incentives for cross-departmental work. Adequate funding can provide a vital catalyst for partnerships between survivorship interventionists and health economists and, ultimately, for advancing the science of economic evaluation as part of survivorship research. Economic analyses can be included as part of survivorship research submitted to standing funding opportunity announcements (FOAs) such as the NIH Research Project Grant Program (R01), NCI Exploratory/Developmental Research Grant Program (NCI Omnibus R21), and NCI


Small Grants Program for Cancer Research (NCI Omnibus R03) as well as to more targeted FOAs such as the Dissemination and Implementation Research in Health grant program (e.g., PAR-13-055), which specifically call for applications that address issues of resources expended, program costs, cost-effectiveness, or other economic outcomes as well as stakeholder engagement. Additionally, the multiple PI option available through many NIH grant mechanisms may be a useful tool for certain types of research to support partnerships among survivorship interventionists and health economists, particularly as it ensures funding for an economist across the entire project. Of relevance across funding opportunities, meeting participants noted the importance of thoughtfully integrating the economic evaluation into the aims of the study. Applications for which the economic evaluation reads as superfluous or projects that do not include an investigator with health economics expertise tend not to fare well in review.

Discussion As the population of cancer survivors increases, so will the need for survivorship care. The low rate at which new interventions are implemented into care coupled with the projected growth in the cost of cancer care underscores the need for an evidence base of high-quality and highvalue interventions [30]. To build this evidence base, investigators must identify and engage relevant stakeholders to identify the health and quality of life outcomes and cost domains on which they base decisions. The evaluation of intervention efficacy and value should be conducted using comparable measures of study endpoints and analytic approaches to enable comparisons across studies and to inform decision-making and treatment guidelines. Importantly, the design and conduct of an economic evaluation should be pursued in partnership with a health economist who brings expertise in valuing costs and outcomes and modeling cost-effectiveness under different scenarios. Although the recommendations presented in this paper were discussed through a survivorship lens, they are consistent and complementary to discussions occurring across medicine and public health. There is growing recognition across disciplines of the need to improve the rate at which scientific discoveries are translated into practice [31–34]. The cost of delivering a new intervention, whether that intervention is a new drug, medical device, procedure, or program, is an important consideration for stakeholders and potential adopters and is integral to evidence-based decision-making [22, 35, 36].

Acknowledgments The authors gratefully acknowledge the involvement and contribution of everyone who participated in the meeting, Applying Evidence from Economic Evaluations to Translate Cancer Survivorship Research into Care. Compliance with ethical standards Conflict of interest The authors declare that they have no conflict of interest. Disclaimer Findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the National Cancer Institute.

References 1. American Cancer Society. Cancer treatment survivorship facts figures, 2014–2015. Atlanta: American Cancer Society; 2014. 2. Alfano CM, Rowland JH. Recovery issues in cancer survivorship: a new challenge for supportive care. Cancer J. 2006;12(5):432–43. 3. Keating NL, Norredam M, Landrum MB, Huskamp HA, Meara E. Physical and mental health status of older long-term cancer survivors. J Am Geriatr Soc. 2005;53(12):2145–52. doi:10.1111/j.1532-5415. 2005.00507.x. 4. Stein KD, Syrjala KL, Andrykowski MA. Physical and psychological long-term and late effects of cancer. Cancer. 2008;112(11 Suppl): 2577–92. doi:10.1002/cncr.23448. 5. Weaver KE, Forsythe LP, Reeve BB, Alfano CM, Rodriguez JL, Sabatino SA, et al. Mental and physical health-related quality of life among U.S. cancer survivors: population estimates from the 2010 National Health Interview Survey. Cancer Epidemiol Biomarkers Prev. 2012;21(11):2108–17. doi:10.1158/1055-9965.EPI-12-0740. 6. Stanton AL. What happens now? Psychosocial care for cancer survivors after medical treatment completion. J Clin Oncol. 2012;30(11): 1215–20. doi:10.1200/JCO.2011.39.7406. 7. Scott DA, Mills M, Black A, Cantwell M, Campbell A, Cardwell CR, et al. Multidimensional rehabilitation programmes for adult cancer survivors. Cochrane Database Syst Rev. 2013;3, CD007730. doi:10. 1002/14651858.CD007730.pub2. 8. Johannsen M, Farver I, Beck N, Zachariae R. The efficacy of psychosocial intervention for pain in breast cancer patients and survivors: a systematic review and meta-analysis. Breast Cancer Res Treat. 2013;138(3):675–90. doi:10.1007/s10549-013-2503-4. 9. Mishra SI, Scherer RW, Geigle PM, Berlanstein DR, Topaloglu O, Gotay CC, et al. Exercise interventions on health-related quality of life for cancer survivors. Cochrane Database Syst Rev. 2012;8, CD007566. doi:10.1002/14651858.CD007566.pub2. 10. Glasgow RE, Emmons KM. How can we increase translation of research into practice? Types of evidence needed. Annu Rev Public Health. 2007;28:413–33. doi:10.1146/annurev.publhealth.28. 021406.144145. 11. Glasgow RE, Klesges LM, Dzewaltowski DA, Bull SS, Estabrooks P. The future of health behavior change research: what is needed to improve translation of research into health promotion practice? Ann Behav Med. 2004;27(1):3–12. doi:10.1207/s15324796abm2701_2. 12. Gordon LG, Beesley VL, Scuffham PA. Evidence on the economic value of psychosocial interventions to alleviate anxiety and depression among cancer survivors: a systematic review. Asia-PacJ Clin Oncol. 2011;7(2):96–105. doi:10.1111/j.1743-7563.2011.01395.x. 13. Mewes JC, Steuten LM, Ijzerman MJ, van Harten WH. Effectiveness of multidimensional cancer survivor rehabilitation and costeffectiveness of cancer rehabilitation in general: a systematic review.

Author's personal copy J Cancer Surviv Oncologist. 2012;17(12):1581–93. doi:10.1634/theoncologist. 20120151. 14. Deverka PA, Lavallee DC, Desai PJ, Esmail LC, Ramsey SD, Veenstra DL, et al. Stakeholder participation in comparative effectiveness research: defining a framework for effective engagement. J comp Eff Res. 2012;1(2):181–94. doi:10.2217/cer.12.7. 15. AA1000 Stakeholder Engagement Standard 2011: Final Exposure Draft. London, UK: AccountAbility 2011. 16. Concannon TW, Meissner P, Grunbaum JA, McElwee N, Guise JM, Santa J, et al. A new taxonomy for stakeholder engagement in patient-centered outcomes research. J Gen Intern Med. 2012;27(8): 985–91. doi:10.1007/s11606-012-2037-1. 17. Hartzler A, McCarty CA, Rasmussen LV, Williams MS, Brilliant M, Bowton EA, et al. Stakeholder engagement: a key component of integrating genomic information into electronic health records. Genet Med: Off J Am Coll Med Genet. 2013;15(10):792–801. doi: 10.1038/gim.2013.127. 18. Shiell A, Donaldson C, Mitton C, Currie G. Health economic evaluation. J Epidemiol Community Health. 2002;56(2):85–8. 19. Weinstein MC, Torrance G, McGuire A. QALYs: the basics. Value Health. 2009;12 Suppl 1:S5–9. doi:10.1111/j.1524-4733.2009. 00515.x. 20. EuroQol G. EuroQol—a new facility for the measurement of healthrelated quality of life. Health Policy. 1990;16(3):199–208. 21. Brazier J, Roberts J, Deverill M. The estimation of a preference-based measure of health from the SF-36. J Health Econ. 2002;21(2):271– 92. 22. Greenberg D, Earle C, Fang CH, Eldar-Lissai A, Neumann PJ. When is cancer care cost-effective? A systematic overview of cost-utility analyses in oncology. J Natl Cancer Inst. 2010;102(2):82–8. doi:10. 1093/jnci/djp472. 23. Ramsey SD, McIntosh M, Etzioni R, Urban N. Simulation modeling of outcomes and cost effectiveness. Hematol Oncol Clin North Am. 2000;14(4):925–38. 24. Parry C, Kent EE, Forsythe LP, Alfano CM, Rowland JH. Can’t see the forest for the care plan: a call to revisit the context of care planning. J Clin Oncol. 2013;31(21):2651–3. doi:10.1200/JCO.2012.48.4618. 25. Lomas J, Culyer T, McCutcheon C, McAuley L, Law S. Conceptualizing and combining evidence for health system guidance. Ott, Ont: Cana Health Services Res Found. 2005 May;2005. 26. Russell LB, Gold MR, Siegel JE, Daniels N, Weinstein MC. The role of cost-effectiveness analysis in health and medicine. Panel Cost-Eff Health Med JAMA. 1996;276(14):1172–7.

27. Weinstein MC, Siegel JE, Gold MR, Kamlet MS, Russell LB. Recommendations of the panel on cost-effectiveness in health and medicine. JAMA. 1996;276(15):1253–8. 28. Ramsey S, Willke R, Briggs A, Brown R, Buxton M, Chawla A, et al. Good research practices for cost-effectiveness analysis alongside clinical trials: the ISPOR RCT-CEA Task Force report. Value Health. 2005;8(5):521–33. doi:10.1111/j.1524-4733.2005.00045.x. 29. Stokols D, Misra S, Moser RP, Hall KL, Taylor BK. The ecology of team science: understanding contextual influences on transdisciplinary collaboration. Am J Prev Med. 2008;35(2 Suppl):S96–115. doi: 10.1016/j.amepre.2008.05.003. 30. Mariotto AB, Yabroff KR, Shao Y, Feuer EJ, Brown ML. Projections of the cost of cancer care in the United States: 2010–2020. J Natl Cancer Inst. 2011;103(2):117–28. doi:10.1093/jnci/djq495. 31. Khoury MJ, Gwinn M, Yoon PW, Dowling N, Moore CA, Bradley L. The continuum of translation research in genomic medicine: how can we accelerate the appropriate integration of human genome discoveries into health care and disease prevention? Genet Med: Off J Am Coll Med Genet. 2007;9(10):665–74. doi:10.1097/GIM. 0b013e31815699d0. 32. Brownson RC, Jacobs JA, Tabak RG, Hoehner CM, Stamatakis KA. Designing for dissemination among public health researchers: findings from a national survey in the United States. Am J Public Health. 2013;103(9):1693–9. doi:10.2105/AJPH.2012.301165. 33. Balas EA, Boren SA. Managing clinical knowledge for health care improvement. In: Bemmel J, McCrat AT, editors. Yearbook of medical informatics 2000: patient-centered systems. Stuttgart: Schattauer Verlagsgesellschaft mbH; 2000. p. 60–75. 34. McGlynn EA, Asch SM, Adams J, Keesey J, Hicks J, DeCristofaro A, et al. The quality of health care delivered to adults in the United States. N Engl J Med. 2003;348(26):2635–45. doi:10.1056/ NEJMsa022615. 35. Helfand M. Incorporating information about cost-effectiveness into evidence-based decision-making: the evidence-based practice center (EPC) model. Med Care. 2005;43(7 Suppl):33–43. 36. Russell LB. The methodologic partnership of effectiveness reviews and cost-effectiveness analysis. Am J Prev Med. 2001;20(3 Suppl): 10–2. 37. Luce BR, Manning WG, Siegel JE, Lipscomb J. Estimating costs in cost-effectiveness analysis In: Gold MR, Siegel JE, Russell LB, Weinstein MC, editors. Cost-effectiveness in health and medicine. New York: Oxford University Press; 1996. p 176–213.