COMMENTARY
Executive Summary: Education Research in Emergency Medicine—Opportunities, Challenges, and Strategies for Success Joseph LaMantia, MD, Nicole M. Deiorio, MD, and Lalena M. Yarris, MD, MCR
Abstract The 2012 Academic Emergency Medicine (AEM) consensus conference “Education Research In Emergency Medicine: Opportunities, Challenges, and Strategies for Success” convened a diverse group of stakeholders in medical education to target gaps in emergency medicine (EM) education research and identify priorities for future study. A total of 175 registrants collaborated in preparatory and conferenceday activities to explore subtopics in educational interventions, learner assessment, faculty development, and research funding and infrastructure. The conference was punctuated by didactic sessions led by key international medical education experts and ended with consensus formation in many domains. This issue of AEM presents the exciting results of this process. ACADEMIC EMERGENCY MEDICINE 2012; 19:1319–1322 © 2012 by the Society for Academic Emergency Medicine
A
divide has traditionally existed in academic medicine between the educator and the researcher. The goal of this conference was to bridge this gap by exploring the principles and practices to help the teacher and the researcher create a unified focus on education research science that will benefit our learners, our educators, our investigators, and most importantly, our patients. Emergency medicine (EM) educators rely on theory and tradition for most educational problems. Despite the growing numbers of journals and conferences dedicated to education research in the greater community of medical education, there are still few published studies
From the Department of Emergency Medicine, North Shore University Hospital (JL), Manhasset, NY; and the Department of Emergency Medicine, Oregon Health & Science University (NMD, LMY), Portland, OR. Received July 2, 2012; accepted July 3, 2012. The authors thank our conference benefactors, Society for Academic Medicine (SAEM), American College of Emergency Physicians (ACEP), Council of Emergency Medicine Residency Directors (CORD), and Emergency Medicine Residents’ Association (EMRA), as well as the departments and institutions at the Chair’s Circle and Patrons contribution levels who generously funded our activities for a successful conference effort. The authors have no relevant financial information or potential conflicts of interest to disclose. Supervising Editor: Terry Kowalenko, MD. Address for correspondence and reprints: Joseph LaMantia, MD; e-mail:
[email protected].
© 2012 by the Society for Academic Emergency Medicine doi: 10.1111/acem.12027
in the EM literature each year that address education research questions.1–3 Studies are needed to identify effective teaching methods, accurate methods of learner assessment, and curricula and programs; determine how to measure the effect of educational interventions and how to prevent cognitive errors; and promote patient safety. Efforts to address these challenges have begun; however, the historical use of suboptimal study designs, subjective outcomes, small samples sizes, and lack of expertise in methods useful in other domains can limit the success of education research studies. We identified a need for a coordinated agenda for EM education research to address these gaps and launch a collaborative research effort among leaders in EM education. In addition to educator needs for evidenced-based instruction and assessment guidelines, directors of training programs also need to demonstrate achievement of educational objectives. The Accreditation Council for Graduate Medical Education (ACGME) requires that training programs demonstrate the effectiveness of educational interventions and show evidence of trainee aptitude and achievement in the core competencies.4 With the implementation of the ACGME Next Accreditation System (NAS) in 2013, programs will need to implement continuous processes for assessing learners’ progress through educational milestones and programs’ incorporation of feedback from these processes to promote longitudinal program growth and maintain accreditation.5 The American Board of Emergency Medicine (ABEM) now requires its diplomates to provide evidence of assessment of practice performance to
ISSN 1069-6563 PII ISSN 1069-6563583
1319 1319
1320
receive continuous certification.6 These and other requirements highlight the current paucity of available evidence to inform our instruction and evaluation of emergency physicians (EPs) and call for our field to engage in high-quality education research. A systematic approach to education research in EM is essential for the continued improvement of clinical emergency care, even for providers beyond residency training. In the decade since the Institute of Medicine’s 2001 “Crossing the Quality Chasm” identified the failure of health care environments to consistently deliver evidence-based care,7 the increased emphasis on translational research and patient safety has identified even broader needs for education-based research. Without well-designed studies to investigate the most effective methods to teach and evaluate EPs, scientific discoveries cannot be effectively disseminated to physicians in training or in practice, nor can the benefits be fully realized by our patients. This consensus conference on education research in EM aimed to create a solid foundation upon which EM education researchers can build interdisciplinary scholarship, networks of expertise, discussion forums, multicenter collaborations, evidence-based publications, and enhanced learner education. Such efforts will enable us to make significant contributions to the state of knowledge in medical education and, ultimately, to improve patient care. MISSION The mission of the 2012 Academic Emergency Medicine (AEM) consensus conference on education research was to stimulate the development of a coordinated initiative within EM to inform a research agenda in education science for the next decade, to advance effective and validated methods of teaching and assessment at all levels. The specific objectives of the consensus conference on education research in EM were to: 1. Review the current state of research in EM instructional methods and identify the most critical agenda areas within each specific education research domain. 2. Review the current state of research in assessment methods in EM and identify the most critical agenda areas within each core competency. 3. Identify common barriers that educators face in conducting well-powered, rigorous education research and develop recommendations for overcoming these barriers. 4. Develop a national agenda for strengthening the infrastructure around education research in EM, including: A. Development of postgraduate and longitudinal professional development opportunities for education researchers; B. Constructing a framework for an EM Education Research Consortium designed to foster multiinstitutional and multidisciplinary collaboration; C. Propose a coordinated agenda to promote grantfunded education research in EM.
LaMantia et al. • EXECUTIVE SUMMARY
Specific objectives for each breakout session are listed in Table 1. CONFERENCE DETAILS Participants were solicited from diverse organizations to attract a broad mix of educators and key stakeholders. Invited groups included EM and non-EM physician societies, nonphysician professional groups, and education and leadership organizations within medical education. A total of 175 participants attended the conference. At the beginning of the day, participants congregated for a welcome and orientation session. Dr. Stanley Hamstra presented the keynote address, an overview of the state of education research in EM,8 and Dr. James Gordon discussed the principles of consensus building.9 The conference participants then divided into four tracks (with a morning and afternoon breakout session for each track) according to their interests and involvement in preparatory activities. The use of tracks allowed an in-depth exploration of multiple related themes concurrently. After exploring the different components of each track, the participants regrouped to present their findings, identify related and conforming themes, and propose a research agenda that stemmed from these breakout session consensus proposals. A keynote panel discussion aimed to explore barriers that educators face in conducting quality education research, propose an approach to overcome these barriers, and discuss examples of published studies that effectively answer relevant education research questions.10 Dr. David Cook moderated the panel of expert discussants, including Drs. K. Anders Ericsson, Larry Gruppen, and Stanley Hamstra. This conference employed a group decision-making process that sought the input of all participants and agreement of the majority, as well as the resolution or mitigation of minority objections. The participants were given information on this process and the available tracks prior to conference day, as well as during the orientation. Participants were then given the opportunity to help build consensus through online discussions prior to the conference. During the conference activities, consensus-building sessions were structured in a similar fashion: participants discussed an item, formed a proposal and recommendations, modified or amended these based on the consensus opinion, and ratified a final version. Several groups also proposed and discussed additional, broader agenda items. Breakout session leaders presented their recommendations to the entire participant group at a final closing session, and large group consensus on key items was collected through the use of an audience response system. The individual breakout session consensus proposals and recommendations follow in the papers included in this journal issue. RESULTS OF THE CONFERENCE Several broad themes and areas of priority regarding education research in EM emerged. First, there is a critical need to focus on the scholarship of education
ACADEMIC EMERGENCY MEDICINE • December 2012, Vol. 19, No. 12 • www.aemj.org
1321
Table 1 Consensus Conference Breakout Sessions and Objectives Breakout Session Session 1a: Developing and assessing initiatives designed to improve clinical teaching performance
Session 1b: Evaluating curricula and didactic teaching
Session 2a: Assessing learner performance
Session 2b: Assessing diagnostic reasoning
Session 3a: Defining a core curriculum for education research fellowships
Session 3b: Promoting faculty development in education research
Session 4a: An agenda for increasing grant-funded education research in EM
Session 4b: Developing an education research consortium for EM
Session Objectives 1. Define clinical teaching performance. 2. Review existing strategies for developing teaching skills and assessing teaching performance. 3. Refine and prioritize research priorities, including: A. Linking faculty development programs to learner outcomes (ACGME core competencies), results on Kirkpatrick’s levels; B. Collaboration across programs and disciplines; C. Maintenance of change over time; D. Process-oriented studies comparing different faculty development strategies; E. Define ideal teaching methods for an emergency physician to possess; F. Models for obtaining mastery in teaching. 1. Review the literature regarding effectiveness of several types of educational modalities: A. Didactic curricula, B. Longitudinal curricula, C. Simulation-based learning, D. Small group learning, E. Web 2.0 as an educational modality, F. Web-based learning/asynchronous curricula. 2. Reach consensus regarding gaps in the current literature and future research directions for evaluation of different teaching methods in EM education. 1. Have working knowledge of the current tools for core competency assessment and the limitations of these tools. 2. Have the opportunity to provide feedback on a proposed plan for future research into core competency assessment tools. 3. Have the opportunity to indicate an interest in participating in future research on core competency assessment tools, if applicable. 1. Have a working knowledge of the current theories of diagnostic reasoning, as well as tools that have been used to assess these skills in both clinical and nonclinical settings. 2. Have the opportunity to provide feedback on suggested areas for future research in diagnostic reasoning assessment. 3. Have the opportunity to indicate an interest in participating in future research on diagnostic reasoning assessment tools, if applicable. 1. Review existing postgraduate fellowship structures. 2. Match unique needs of an education research fellowship to a conducive structure. 3. Identify core content for a postgraduate education research fellowship. 4. Propose an ideal structure for the fellowship, including duration, formal training opportunities, fellow role, integration into department, and funding logistics. 1. Outline best strategies to translate education methods supported by research into teach-the-teacher curricula. 2. Examine current available opportunities for postgraduate education research training in EM and other fields (such as education research fellowship, MERC, etc.); construct core content to guide development of new training opportunities. 3. Commit to alliances with education scientists across a broad range of clinical specialties and outside of medicine, with the goal of creating a common research agenda and shared resources. 4. Develop strategies for recruiting and mentoring future EM education researchers. 1. Review successful past funding strategies for education research, exploring strategies that have been effective in clinical research to assess for applicability. 2. Initiate a shared database of funding opportunities and administrative support for education researchers in EM. 3. Create an action plan for increasing opportunities for EM education research funding. 1. Explore options for different types of networks and how they function. 2. Select an education research network structure to match resources, funding, and goals. 3. Develop the basic framework for a research consortium. 4. Identify an ideal structure for leadership, decision-making, and publication.
ACGME = Accreditation Council for Graduate Medical Education; MERC = Medical Education Research Certificate.
and to align the culture and the values of the institution to accommodate this perspective. Second, research in medical education must attend particularly to clinically relevant outcomes, and better outcome assessment tools must be developed and studied in methodologically rigorous ways. Third, there is a need for systems that promote mentoring and collaboration across insti-
tutions, disciplines, and levels of education, including in the domains of education research funding and infrastructure support. Successful collaboration also mitigates duplication of efforts, an important consideration in this time of declining resources. As a major focus for this coordination and partnership, the framework for a central education research network in EM was
1322
developed. Finally, there is a need for a strong focus on professional development for the education researcher, including rigorous postgraduate fellowship models, programs for ongoing maintenance and further refinement of skills, and rigorous evaluation of the effectiveness of such programs in enhancing the educational mission. CONCLUSIONS A diverse group of EM educators, researchers, and medical education experts collaborated to plan and implement the 2012 AEM consensus conference on education research in EM. This issue of Academic Emergency Medicine is dedicated to disseminating the proceedings from the sessions to the EM education research community. These proceeding papers present a literature-based consensus on the current state of EM education research and propose key research priorities to guide our field in the next decade. However, the value of the consensus process relates not only to the work of the day, but even more importantly to the momentum formed by relationships that are built and to the networking and collaboration that ensue. The research questions raised in these proceedings are best addressed by collaborative, creative efforts with and beyond our community to advance the science of training physicians, promote the professional development of education researchers, and improve patient care. The conference co-chairs acknowledge the contributions of our steering committee members (Esther Choo, MD, MPH, Jonathan Ilgen, MD, MCR, and Chad Kessler, MD, MHPE) and planning committee (Jill Castaneda, BS, David C. Cone, MD, Sandra K. Arjona, Kathleen G. Seal, Maryanne Greketis, and Melissa McMillian). We also acknowledge our invited experts David A. Cook, MD, MHPE (Mayo Clinic), K. Anders Ericsson, PhD (Florida State University), James A. Gordon, MD, MPA (Harvard Medical School), Larry D. Gruppen, PhD (University of Michigan), Stanley J. Hamstra, PhD (University of Ottowa), William McGaghie, PhD (Northwestern University), Geoff Norman, PhD, MA (McMaster University), Janet Riddle, MD (University of Illinois-Chicago), Nancy S. Searle, EdD
LaMantia et al. • EXECUTIVE SUMMARY (Baylor College of Medicine), and Jonathan Sherbino, MD, MEd (McMaster University).
References 1. Farrell SE, Coates WC, Kuhn GJ, Fisher J, Shayne P, Lin M. Highlights in emergency medicine medical education research: 2008. Acad Emerg Med. 2009; 16:1318–24. 2. Kuhn GJ, Shayne P, Coates WC, et al. Critical appraisal of emergency medicine educational research: the best publications of 2009. Acad Emerg Med. 2010; 17(Suppl 2):S16–25. 3. Shayne P, Coates WC, Farrell SE, et al. Critical appraisal of emergency medicine educational research: the best publications of 2010. Acad Emerg Med. 2011; 18:1081–9. 4. The ACGME Outcomes Project. Available at: http:// www.acgme.org/outcome/about/aboutHome.asp. Accessed Oct 1, 2011. 5. Nasca TJ, Philibert I, Brigham T, Flynn TC. The next ACGME accreditation system–rationale and benefits. NEJM. 2012; 366:1051–6. 6. The American Board of Emergency Medicine. Homepage. Available at: http://www.abem.org. Accessed Oct 11, 2011. 7. Institute of Medicine, Committee on Health Care in America. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: The National Academies Press, 2001. 8. Hamstra S. Keynote address: the focus on competencies and individual learner assessment as emerging themes in medical education research. Acad Emerg Med. 2012; 19:1336–43. 9. Gordon JA. Reflections on the consensus process: a leadership role for emergency medicine in educational scholarship and practice across health care. Acad Emerg Med. 2012; 19:1333–35. 10. Yarris LM, Gruppen LD, Hamstra SJ, Ericsson A, Cook D. Overcoming barriers to addressing education problems with research design: a panel discussion. Acad Emerg Med. 2012; 19:1344–49.
