Advances in Health Sciences Education (2006) 11: 33–39 DOI: 10.1007/s10459-004-7346-7
Ó Springer 2006
A Randomized Controlled Trial of SimulationBased Teaching versus Traditional Instruction in Medicine: A Pilot Study among Clinical Medical Students JAMES A. GORDON1,2,3,*, DAVID W. SHAFFER4, DANIEL B. RAEMER1,5, JOHN PAWLOWSKI1,2,6, WILLIAM E. HURFORD1,5,7, JEFFREY B. COOPER1,5 1
Center for Medical Simulation, Boston; 2G.S. Beckwith Gilbert and Katharine S. Gilbert Medical Education Program in Medical Simulation, Harvard Medical School; 3Department of Emergency Medicine, Massachusetts General Hospital; 4Department of Educational Psychology, University of Wisconsin-Madison; 5Department of Anaesthesia and Critical Care, Massachusetts General Hospital; 6Department of Anesthesia and Critical Care, Beth Israel Deaconess Medical Center; 7 Department of Anesthesia, University of Cincinnati (*Corresponding author: Department of Emergency Medicine, Massachusetts General Hospital, 55 Fruit Street, Clinics 115, Boston, MA 02114; Phone: 617-726-7622; Fax: 617-724-0917; e-mail:
[email protected]) (Received 25 March 2004; accepted 7 December 2004)
Abstract. Objective: To compare simulator-based teaching with traditional instruction among clinical medical students. Methods: Randomized controlled trial with written pre-post testing. Third-year medical students (n ¼ 38) received either a myocardial infarction (MI) simulation followed by a reactive airways disease (RAD) lecture, or a RAD simulation followed by an MI lecture. Results: Mean pre-post test score improvement was seen across teaching modalities (overall change score [simulation]=8.8 [95% CI ¼ 2.3–15.3], pretest [62.7]; change score [lecture] ¼ 11.3 [95% CI ¼ 5.7–16.9], pretest [59.7]). However, no significant differences were observed between simulator-based teaching and lecture, in either subject domain. Conclusions: After a single instructional session for clinical medical students, differences between simulatorbased teaching and lecture could not be established by the written test protocols used in this pilot. Future studies should consider the effects of iterative exposure assessed by clinical performance measures across multiple centers.
Introduction Experiential learning, or ‘‘learning by doing,’’ (Dewey, 1916) has long dominated the culture of professional apprenticeships. In medical training, however, this powerful pedagogical technique carries an inherent risk to patients. ‘‘See one, do one, teach one’’ has long been a guiding principle of medical trainees, yet leaves precious little room for error (Freidrich, 2002). While ‘‘safe’’ modalities of lecture and small-group teaching are essential to
34
JAMES A. GORDON ET AL.
the educational process, they cannot replace the value of clinical experience. The advent of realistic medical simulation, however, promises to enhance the apprenticeship of young physicians without any risk to patients. These sophisticated robot-mannequins can replicate the clinical encounter with such a degree of realism that trainees can intensively ‘‘practice’’ medicine in a safe environment in preparation for actual patient care (Gaba et al., 2001; Gordon et al., 2004). Despite its inherent appeal, however, objective evidence for the efficacy of simulator-based medical education remains limited (Issenberg et al., 1999; Morgan et al., 2002). We conducted this study to compare simulator-based teaching with traditional instruction in critical care and emergency medicine among clinical medical students. Our hypothesis was that a single exposure to realistic patient simulation would help students to integrate basic and clinical science in a way that enhances trainee understanding compared with traditional teaching.
Methods STUDY
DESIGN
We conducted a randomized controlled trial of simulation based-teaching versus traditional instruction among clinical medical students, using pre-post written evaluations (n ¼ 38). Two subject domains (reactive airways disease [RAD] and myocardial infarction [MI]) were ‘‘crossed over’’ in the intervention phase, producing two experimental cohorts: (1) students receiving a respiratory simulation followed by a cardiac lecture [n ¼ 17], and (2) students receiving a cardiac simulation followed by a respiratory lecture [n ¼ 21]. We hypothesized that for each subject domain, students who received simulation training would outperform students who received lecture training, across cohorts. SETTING
AND POPULATION
A convenience sample of all 3rd and 4th year Harvard medical students rotating through a surgery clerkship at Beth Israel Deaconess Medical Center were enrolled in the study during the winter–spring of 2000 (n ¼ 38). Each study day accommodated a single group of 3 students, who were randomized as a team to a particular simulation/lecture cohort based on their date of participation. A single instructor (emergency medicine physician) conducted all simulation exercises (both subject domains) while two different faculty gave the lectures (anesthesiologist gave all respiratory lectures; emergency physician
A RANDOMIZED CONTROLLED TRIAL OF SIMULATION-BASED TEACHING
35
gave all cardiac lectures). While both lectures were considered ‘‘traditional teaching’’ for study purposes, the cardiac lecture was a more of a traditional didactic (as if given in an auditorium full of students), while the respiratory lecture became more of a tutorial discussion. All study sessions were conducted at the Center for Medical Simulation in Boston, and used the Eagle high-fidelity patient simulator (MedSim-Eagle Simulation, Inc.). PROTOCOL/INTERVENTION On the appointed day, each group of 3 subjects presented to the simulation center for, in order: (1) a 30-minute written pretest [both subject domains]; (2) a 90-minute simulation exercise [single subject domain]; (3) a 90-minute lecture [complementary subject domain]; and (4) 30-minute post-test [both subject domains]. An instructional ‘‘challenge model’’ was designed to compare the two teaching modalities. All instructors were given only the following instructions to guide their instructional plan: ‘‘You will have 90 minutes to teach a group of 3 students about either reactive airways disease or myocardial infarction. After each session the student should: (1) understand the clinical care of the relevant disease process, (2) understand the basic physiology and pharmacology of the care process, (3) appreciate the complications of the disease and care process, and (4) communicate effectively about the encounter.’’ No further guidance was given. Instructors were told to use their respective modalities (simulation vs. lecture) as they saw fit, to achieve the outlined objectives for the assigned subject domain. All instructors were blinded to the other’s specific teaching materials and lesson plan; they were also blinded to the content of the written tests (see below). MEASUREMENTS Based only on the global learning objectives, two equivalent written tests in both subjects (cardiac version A or B; respiratory version A or B) were written and graded by an expert blinded to the specific instructional plans (intensivist-anesthesiologist). For each study day, the versions A and B (for both subject domains) were randomized as pre-test or post-test. Each test version comprised 11 or 12 short-answer/multi-part questions, and was worth 100 points. The tests were piloted among students and refined in a pre-study phase based on the same protocol. A typical exam might begin with a physical diagnosis question: ‘‘Describe the physical signs and symptoms that may be suggestive of an acute myocardial infarction (or asthma attack)’’ [10 points]. It might then probe
36
JAMES A. GORDON ET AL.
Table I. Changes in pre-post written test performance (mean score) for simulation-based teaching versus traditional instruction among clinical medical students, stratified by subject domain (reactive airways disease [RAD], myocardial infarction [MI])
Subject Simulation domain N Pre (SD)
Lecture1 Post (SD) Change (SD) N Pre (SD)
RAD 17 63.4 (14.7) 69.8 (16.4) 6.5 (15.9) MI 21 62.2 (17.9) 73.0 (10.8) 10.8 (22.6) Overall 38 62.7 (16.3) 71.6 (13.5) 8.8 (19.8) 1
Post (SD) Change (SD)
21 61.0 (17.8) 71.5 (14.4) 10.5 (15.9) 17 58.2 (15.7) 70.5 (16.3) 12.2 (18.6) 38 59.7 (16.7) 71.0 (15.1) 11.3 (16.9)
The RAD ‘‘lecture’’ evolved into a small group tutorial session during the study.
understanding of management and complications: ‘‘When would you consider endotracheal intubation and mechanical ventilation (in a refractory asthmatic)?’’ [5 points]; or, ‘‘When would you not want to use thrombolytic therapy in a patient with an acute MI?’’[5 points]. The test might conclude: ‘‘After successful treatment, the patient and family ask how they might reduce risk for a subsequent event – what would you tell them?’’ [10 points]. DATA
ANALYSIS
For each subject domain, data were analyzed by analysis of covariance (ANCOVA). Post-test score was designated as the dependent outcome variable, controlled by pre-test score, intervention format (lecture vs. simulation), pre-post test sequence (version A then B vs. version B then A), and gender. Significance was set at
