Oncology training programs: are we doing ... - Future Medicine

PERSPECTIVE For reprint orders, please contact: reprints@futuremedicine.com

Oncology training programs: are we doing comparative effectiveness research justice? Comparative effectiveness research (CER) is defined as an assessment of all available options for a specific medical condition, with intent to estimate effectiveness in specific subpopulations. Medical professionals must receive training in CER, including its general goals, the ‘toolbox’ necessary to perform CER and its design. Oncologic training programs are currently not doing justice to CER: a rationale for conducting CER has been proposed, funding from the US government is available, but encouragement from oncologic residencies and fellowships is minimal. Encouragement to train oncologic physicians in CER is limited by a few factors, including inadequate emphasis on evidence weighing, and no explicit mention of factors key to CER in the Accreditation Council for Graduate Medical Education guidelines. As residency program requirements transition to milestone-based curricula and evaluations, explicit recommendations regarding CER and patient-centered outcomes research should be implemented for all programs. Upper level trainees who have reached milestones related to CER will be competitive applicants for research funding.

Nicholas G Zaorsky1 & Timothy N Showalter*2 Department of Radiation Oncology, Fox Chase Cancer Center, Philadelphia, PA, USA 2 Department of Radiation Oncology, University of Virginia School of Medicine, 1215 Lee Street, Charlottesville, VA 22908, USA *Author for correspondence: Tel.: +1 434 924 5191 Fax: +1 434 982 6282 tns3b@virginia.edu 1

KEYWORDS: comparative effectiveness research n education n graduate n medical n standards

Comparative effectiveness research (CER) attempts to frame the delivery of healthcare by comparing the benefit and harm of available diagnostic, prognostic and therapeutic options to optimize the risk–benefit ratio and improve cost–effectiveness. A formal definition of CER has been proposed (Box 1) [1,2]. Informally, CER is defined as an assessment of all available options for a specific medical condition, with the intent to estimate their effectiveness in specific subpopulations or for individuals [3,4]. It is essential that medical professionals must receive training in CER, including its general goals, the ‘toolbox’ necessary to perform CER, weighing evidence and CER design (Box 2). In this perspective article, we review the impetus for CER and its relationship to oncologic training; the training program goals by the Accreditation Council for Graduate Medical Education (ACGME) and their relationship to CER; and how CER will be integrated in oncologic training programs as the ACGME phases out core competencies and integrates milestones for trainees. Importance of CER in oncologic training

The need for CER is particularly compelling in oncology, where subpopulations or individuals can be defined based upon a broad range of specific factors that may affect decisions: from molecular markers, as in personalized medicine to match the appropriate targeted therapy to a tumor marker [5]; to more humanistic considerations such as patient preferences, as in preference-sensitive treatment decisions in prostate cancer [6]. There are currently more than 1200 genomic companion diagnostic tests available to detect biomarkers in clinical practice for over 2500 conditions [7]. Approximately two to three genomic tests are created each week, and 65% of them are specific to oncology [8]. Biomarker-specific therapeutic interventions are

10.2217/CER.13.71 © 2013 Future Medicine Ltd

2(6), 573–582 (2013)

part of

ISSN 2042-6305

573

perspective

Zaorsky & Showalter

Box 1. Formal definition of comparative effectiveness research. Comparative effectiveness research is the generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat, and monitor or improve the delivery of care. The purpose of comparative effectiveness research is to assist consumers, clinicians, purchasers and policy-makers to make informed decisions that will improve healthcare at both the individual and population levels. Data taken from [1,2].

combined with companion diagnostics to target patient subpopulations for treatment. Furthermore, the number of treatment options available for cancer patients is also increasing. For example, for men with highrisk/locally advanced prostate cancer, the three common treatment options in the 1990s typically included radical prostatectomy, external beam radiation therapy and androgen deprivation therapy. Since 2000, combination therapies including novel surgical methods (e.g., robotic approaches) and radiation modalities (e.g., protons, brachytherapy and cryotherapy) have become acceptable treatment options in the multimodal management of certain patients [9]. Additionally, numerous pharmacotherapies exist, and over 30 novel drugs are in Phase I–III trials [10]. The combination of companion diagnostics, biomarkers and targeted therapies is the sine qua non of personalized medicine, and these considerations are increasingly informing the decisions of individuals and subgroups – hence, the connection with CER in oncology. Furthermore, the development of novel diagnostic and therapeutic technologies must be considered as a potential influence on rising healthcare costs. In the USA, healthcare expenditures have continued to rise at a rate higher than that of other countries; by contrast, life expectancy rates have fallen behind those of most other developed countries [101]. Thus, it is necessary to identify, acquire, evaluate and synthesize all relevant information related to the effectiveness, safety and overall value of medical options to influence clinical decision-making [11]. Impetus for CER in oncologic training

As the federal government has placed such a strong emphasis on CER, evidenced by the provision for funding of CER in American Recovery and Reinvestment Act legislation and the increased focus on CER in the portfolio of National Cancer Institute (NCI)-funded research, it suggests that familiarity with CER principles and methods will become essential for oncologists. In the quest to better inform

574

J. Compar. Effect. Res. (2013) 2(6)

diagnostic and therapeutic decisions, and to compare the effectiveness and value of interventions in a real-world and policy-relevant format, it is anticipated that CER will become progressively more important and central to evidence evaluation. Although some significant steps have been taken to educate the oncology community regarding CER, including a special issue of the Journal of Clinical Oncology dedicated to this topic, it is clear that some leading oncologists do not understand the need and underlying principles of CER [12,13]. There is a need to incorporate CER into cancer research approaches in order to provide important data that can complement traditional methods such as randomized controlled trials and fills current evidence gaps that will enrich clinical decision-making for individual patients and their physicians. Trainees in oncology have additional financial stimuli to embrace CER in their future careers, should they choose to pursue a research component to their career. In recognition of the value in CER, the American Recovery and Reinvestment Act of 2009 allocated US$1.1 billion toward CER and the Patient Protection and Affordable Care Act of 2010 created the Patient Centered Outcomes Research Institute (PCORI), with an estimated US$600 million of yearly funding, to pursue CER [14]. Both PCORI and the Agency for Healthcare Research and Quality (AHRQ ) define CER as comparing “drugs, medical devices, tests, surgeries or ways to deliver healthcare” [102]; unofficially, CER also includes analyses of the economic impact of potential interventions with cost-effective and cost–benefit analyses [15,16]. Between 2009 and 2011, the NCI Division of Cancer Control and Population Science funded 103 CER-related projects [17]. The themes of these projects were diverse and nonexclusive, ranging from organizational change interventions (54%), multilevel behavioral changes (53%), prevention (39%), economic modeling (35%) and simulation modeling (11%). The NCI funded CER for numerous cancer types, and breast (22%), colorectal (18%) and prostate

future science group

Comparative effectiveness research in oncology training

(13%) were the most frequent. Thus, recent cancer CER has spread across multiple dimensions, and trainees in oncology may find funding for their proposals. Encouraging oncologic trainees to further participate in CER may also be promoted by training programs. CER in oncology training programs: current status ■■ Evidence weighing

In making healthcare management decisions, patients and clinicians must know the advantages, disadvantages and confidence intervals of available treatment strategies [18]. Expert clinicians and organizations offering

perspective

recommendations to the clinical community have erred as a result of not accounting for the quality of evidence in their recommendations. For example, in oncology, although some have noted that post hoc analyses should never trump the results of carefully conducted clinical trials [12], others note that placing clinical trials at the top of the research hierarchy is often inappropriate because the methodology (e.g., accrual, study design, confounders, powering and selection of outcomes) of clinical trials may make them as biased as observational studies [13]. The use of components affecting the quality of evidence, balance between health benefits and harms, incorporation of health benefits and costs

Box 2. Components of comparative effectiveness research. General goals [1–4,21] ■■ Influence clinical decision-making from a patient perspective or health decision policy/population perspective ■■ Compare the most effective healthcare options where clear options exist (e.g., best/most commonly used alternatives; not necessarily placebo) ■■ Support the development of personalized or stratified medicine by examining the racial, ethnic, socioeconomic and geographic variations in care that affect health outcomes ■■ Predict which intervention would be most successful in individual patients by finding particular clinical characteristics ■■ Integrate specific outcome measures, including benefits, harms, patient-reported QOL and costs to patients and providers ■■ Link data from public and private entities CER toolbox† [12,21] ■■ RCTs ■■ Systematic reviews and meta-analyses ■■ Other comparative clinical trials ■■ Population studies including registries, administrative and claims data ■■ Prognostic, predictive association studies ■■ QOL studies with PROs [22] ■■ Clinical decision models including cost and utility ■■ Technology assessments ■■ Better understanding of stakeholder engagement ■■ Attaining skills to use shared decision-making Specific CER design [23] ■■ Priority setting: multiple dimensions (e.g., disease burden, cost and information gap) representing perspectives of various stakeholders ■■ Study designs: retrospective or prospective ■■ Comparisons: direct, head-to-head of new therapy with ‘usual care’ ■■ Study questions: broad ■■ Perspectives: multiple (e.g., patients, providers and payers) ■■ Study populations, setting: representative of clinical practice, with efforts to evaluate subpopulations to maximize effectiveness in target population ■■ Funding: multistakeholder ‘coverage with evidence’ programs, public–private partnerships ■■ End points: effectiveness, efficiency, function/disability, absenteeism/productivity, impact on caregivers and patient satisfaction †

Use of GRADE to rate the quality of evidence and strength of recommendations [18].

CER: Comparative effectiveness research; GRADE: Grading of recommendations assessment, development and evaluation; PRO: Patient reported outcome; RCT: Randomized controlled trial; QOL: Quality of life.


www.futuremedicine.com

575

perspective

Zaorsky & Showalter

are typically not considered in many evidence weighing systems [19]; moreover, there is no standardized teaching of evidence weighing to oncologic trainees. The Grading of Recommendations Assessment, Development and Evaluation (GRADE) system was developed internationally to address the deficits of other systems. The advantages of the GRADE system [18] in comparison to other systems are its: international development; clear separation between quality of evidence and strength of recommendations; explicit evaluation of importance of outcomes of alternative management strategies; explicit comprehensive criteria for up-/down-grading evidence quality; transparent process of moving from evidence to recommendations; explicit acknowledgement of values and preferences; clear, pragmatic interpretation of strong versus weak recommendations for clinicians, patients and policy-makers; and usefulness for systematic reviews, technology assessments and guidelines. The GRADE system incorporates many aspects key to CER, and it should be integrated into specific oncologic program requirements and ACGME core competencies to assess evidence. ■■ Oncology-specific competencies & ACGME core competencies

As of 2013, three oncologist-track training programs exist for fellow and resident physicians: medical oncology, surgical oncology and radiation oncology. Each training program has specific requirements as defined by the ACGME. The current CER-related goals of training programs are listed in Table 1 [103–105]. In terms of general research goals, all three tracks for training oncologists encourage “publication of original research or review articles” [103–105]. Medical and radiation oncology residency programs specifically stress the incorporation of cost awareness and risk–benefit analysis in patient–patient and population-based medicine [103,105]. Moreover, programs stress the use of certain tools that are important to CER, including biostatics [103–105], implementation of prospective databases [104] and the use of information technology [103,105]. Unfortunately, all three oncologist training tracks have limitations in their requirements in promoting CER. In 2013, a survey was sent to radiation oncology program directors (n = 85), chief residents (n = 98), medical oncology program directors (n = 99) and fellows (n = 160). From a combined response rate of 15% (68 out

576


of 442), more than half of the respondents did not ‘agree’ that: they have a clear definition of CER; their programs encourage CER; or their programs include a course on CER [20]. Additionally, although 86% of respondents believe that retrospective cohort studies are easy to perform at their institutions; only 45% of these practitioners believe the same of CER [20]. The lack of an emphasis on CER in oncology training is reflected in ACGME guidelines of oncologic training programs. The ACGME guidelines do not mention key terms essential to CER, including ‘comparative effectiveness research’; ‘clinical decision-making’; ‘personalized medicine’; ‘quality of life’; and of differentiation among ‘efficacy’, ‘effectiveness’ and ‘efficiency’. The guidelines do not yet promote research that would predict the success of an intervention, or the linking of private and public entities. With respect to tools key to CER [11,21], guidelines also do not explicitly encourage the use of clinical decision models that include cost and utility, technology assessments, or analysis of registries, administrative data and claims data. The design for resident and fellow research studies is largely undefined, and guidelines do not explicitly promote ‘head-to-head’ comparisons, patient-centered outcomes or patient reported outcomes [22,23]. The possible study designs and populations encouraged include patients, but do not explicitly mention providers or payers. Although weighing evidence is important [12,13,18], none of the oncologic training programs incorporate the GRADE system in its requirements. Finally, ACGME guidelines do not suggest resources for research funding, including the AHRQ or PCORI. The program goals of oncologic training programs are influenced in part by the ACGME core competencies. The core competencies are currently split among six categories [106] and, similar to the oncology-specific goals, do not comprehensively include all aspects key to CER (Table 2). Notably, among both the oncologyspecific programs goals and the global goals, there is not an explicit discussion of CER methods and studies. Similar to the oncology-specific competencies, the ACGME core competencies do not mention patient quality of life or patentrelated outcomes [22], the use of evidence-based weighing systems [18], or the toolbox necessary to perform CER [11,21]. Thus, oncologic training programs are currently not doing justice to CER: a rationale



perspective

Table 1. Comparative effectiveness research in Accreditation Council for Graduate Medical Education guidelines of medical oncology, surgical oncology and radiation oncology residency/fellowship training programs. Components of CER Current program goals relating to CER

Deficiencies/factors not mentioned

General goals

“Publication of original research or review articles” [103–105] ‘Personalized medicine’ “Advocate for quality patient care and optimal patient care Prediction of intervention success systems” [103,105] Linking public, private entities “Incorporate considerations of cost awareness and risk–benefit analysis in patient and/or population-based care as appropriate” [103,105] “Locate, appraise, assimilate evidence from scientific studies related to patients’ health problems” [104,105]

CER toolbox

Support epidemiology and biostatistics [103–105] “Design, implement prospective database, especially prospective clinical trials; conduct clinical cancer research; use statistical methods to properly evaluate results of published research studies” [104] “Biological laboratory research, clinical research, translational research, medical physics research or other research approved by program director” [105] “Use information technology to optimize learning” [103,105]

‘CER’ QOL studies Registries, administrative, claims data Predictive, prognostic studies Clinical decision models including cost and utility Technology assessments Evidence weighing systems [18]

Specific CER design

Study designs: prospective studies encouraged [104] Study populations: cost awareness; risk–benefit analysis; patient-/population-based care [103,105]

Study designs: undefined [104,105] Comparisons: undefined [103–105] Study questions: undefined [103–105] Perspectives: undefined [103–105] Funding: undefined [103–105] End points: undefined [103–105] Evidence weighing systems [18]

CER: Comparative effectiveness research; QOL: Quality of life.

for conducting CER has been proposed, funding from the US government is available, but encouragement from oncologic residencies and fellowships is minimal. Encouragement to train oncologic physicians in CER is limited by a few factors, including inadequate emphasis on evidence weighing and no explicit mention of factors key to CER in the ACGME guidelines. Future perspective

As residency program requirements transition to milestone-based curricula and evaluations, explicit recommendations regarding CER and patient-centered outcomes research should be implemented for all programs. Programs should incorporate educational content on basic CER concepts, perhaps integrated with training in statistics and/or existing journal club offerings, with the explicit intent of ensuring that graduates can evaluate and incorporate CER-generated evidence into their future clinical practice. This step would satisfy the primary need for CER in residency and fellowship training, since it would provide trainees with the basic knowledge to understand CER


studies that they may encounter during their clinical career. However, many training programs, particularly those with a mission to develop future researchers, may wish to extend CER-related didactic content to a more advanced level that includes scholarly contributions. These programs may choose to encourage residents and fellows to engage in CER projects that are designed to impact clinical decision-making, to promote personalized or stratified medicine for particular patient subpopulations, and to compare multiple treatment options. It is anticipated that oncology trainees who wish to pursue careers as CER investigators will need to seek out advanced training, such as complementary degrees (e.g., Master of Public Health degree) or research fellowships, and oncology training programs should seek to provide mentorship and support for such individuals. In this way, oncology training programs can contribute to the development of future leaders in oncology CER. It is currently difficult to outline a curriculum on CER to be used in all programs. The optimal CER curriculum will probably include successful

www.futuremedicine.com

577

perspective

Zaorsky & Showalter

Table 2. Current Accreditation Council for Graduate Medical Education core competencies and sites of integration of comparative effectiveness research. Core competency

Specific competency currently graded

CER components that may be further emphasized in milestone-based system

Patient care

Identify, respect and care about patients’ differences, values, preferences and expressed needs Listen to, clearly inform, communicate with and educate patients Share decision-making and management Continuously advocate disease prevention, wellness and promotion of healthy lifestyles, including a focus on population health Make informed diagnostic, therapeutic decisions Demonstrate appropriate procedural skills

Incorporate patient QOL and PRO [22] Make decisions based on evidence weighing systems [18]

Medical knowledge

Demonstrate knowledge of evolving biomedical, clinical and cognate (e.g., epidemiological and social behavioral) sciences Apply knowledge and analytical thinking to address clinical questions

Incorporate CER toolbox [11,21] Make decisions based on evidence weighing systems [18]

Practice-based learning and improvement

Investigation and evaluation of one’s own patient care, appraisal and assimilation of scientific evidence, and improvements in patient care of self and others

Incorporate CER toolbox [11,21] Make decisions based on evidence weighing systems [18] Understanding of PCORI statutory criteria for national priorities in research [26]

Interpersonal and communication skills

Exchange information with team, patients, their families and other health professionals

Incorporate patient QOL and PRO [22]

Professionalism

Demonstrate integrity, ethical behavior; accepts responsibilities and follows through on tasks Practice within scope of own abilities Demonstrate care and concern for patients and families regardless of age, gender, ethnicity and sexual orientation; respond to unique patient needs

Systems-based practice

Provide cost-conscious, effective medical care Work to promote patient safety Coordinate with other healthcare providers

Link data from public and private entities

CER: Comparative effectiveness research; PCORI: Patient Centered Outcomes Research Institute; PRO: Patient-reported outcomes; QOL: Quality of life. Data on current Accreditation Council for Graduate Medical Education core competencies taken from [106].

aspects of established CER curricula. The optimal CER curriculum will need to define: time commitment from trainees; time commitment from faculty at a trainee’s institution; time commitment from faculty outside of the institution; infrastructure support; administrative assistance; and processes to help the institutional review board understand CER. Additionally, it will be necessary to hold programs accountable for providing CER training; for example, by surveys, tracking of projects, funding, publications and future roles of trainees when they complete training. CER by oncologist trainees will be further supported by available funding to investigators, and these opportunities may be more explicitly defined in ACGME guidelines (Table 2). For example, the American Society for Radiation Oncology (ASTRO)/Radiation Oncology Institute CER award provides US$50,000 per year for

578


2 years for radiation oncologists involved in CER [107]. The American Society of Clinical Oncology (ASCO) young investigator award provides US$50,000 for 1 year, and may be applied toward CER-related projects [108]. PCORI grants provide up to US$250,000 per year for the development or enhancement of CER and PCOR [109]. AHRQ R13, R18 and F32 grants also provide funding for a wide range of organizations, inducing institutions, nonprofits, public organizations and commercial organizations (Table 3) [110]. Finally, the annual ASTRO and ASCO meetings will continue to emphasize themes with concepts that are central to CER. For example, the priorities of ASTRO explicitly include an emphasis on patient communication strategies, quality indicators, development of a national registry, value assessment and CER studies [24]; similarly, the ASCO guidance statement argues that physicians have a societal



responsibility to integrate cost implications into treatment considerations and communicate with their patients about out-of-pocket expenses [25]. ACGME training guidelines will also have to make trainees competitive applicants for this research funding because applicants will be instructed to explain how their proposed research aligns with statutory criteria, how patent-related outcomes will be incorporated into research and describe how patients and other stakeholders engaged will benefit from research. Concurrently, PCORI will convene workshops, multistakeholder conferences and advisory panels to consider whether specific research areas are aligned with PCORI criteria and deserve more funding [26]. Thus, PCORI funding will be directed toward trainees with high-priority research questions and areas. Upper level trainees who have reached milestones related to CER will be competitive applicants for research funding. It is unclear how funding CER among oncologic trainees will change. One issue that may be faced by oncologic trainees is difficulty in getting grant support compared with more senior scientists, because there is often a preference to fund senior physician scientists since they have the “years of experience to marshal the resources to design and carry out novel clinical experiments that use strong inference” [27]. This is especially worrisome for trainees, since senior faculty are

perspective

currently having a difficult time getting grant support. One potential option to support the development of CER investigators among oncology trainees is to incorporate CER training into currently funded institutional training programs (e.g., T32 programs). However, methods of cancer research have changed drastically over the past 200 years, from the use of microscopes to detect cancer in the 1860s to proving the efficacy of surgical procedures and chemotherapies of today [28]. Novel diagnostic and therapeutic procedures are being introduced into oncology at a rate faster than ever before [8,9]. The principles of CER for these diagnostic and therapeutic procedures have not been on the list of major medical discoveries [28]. Moreover, there are an increasing number of elderly cancer patients, who are often under-represented in clinical trials. CER studies may contribute to available evidence by evaluating the benefits and harms of interventions observed in cohorts of elderly patients. Senior researchers may be at a disadvantage in obtaining funding for CER because they tend to get attached to their hypotheses and design studies to prove it, rather than to disprove it [27]. For example, in urologic oncology, the ability to deliver extremely high doses of radiation in few fractions with stereotactic body radiation therapy has led to the widespread use of stereotactic

Table 3. Available funding for comparative effectiveness research in oncology training programs. Name

Award description

Eligibility/stipulation

ASTRO/ROI CER award

US$50,000 per year, for 2 years Exclusive to CER; >40% of recipient’s professional effort devoted to goals of award

Board-certified or -eligible radiation oncologist Committed to academic radiation oncology

ASCO young investigator award

US$50,000 for 1 year Nonexclusive to CER

Resident or fellow in oncology training program

PCORI grant