Using Second Life Virtual Simulation Environment for Mock Oral Emergency Medicine Examination Jillian Schwaab, MD, Nicholas Kman, MD, Rollin Nagel, PhD, David Bahner, MD, Daniel R. Martin, MD, Sorabh Khandelwal MD, John Vozenilek, MD, Douglas R. Danforth, PhD, and Richard Nelson, MD
Abstract Objectives: Oral examination is a method used to evaluate emergency medicine (EM) residents and is a requirement for board certification of emergency physicians. Second Life (SL) is a virtual three-dimensional (3-D) immersive learning environment that has been used for medical education. In this study we explore the use of SL virtual simulation technology to administer mock oral examinations to EM residents. Methods: This was a prospective observational study of EM residents who had previously completed mock oral examinations, participating in a similar mock oral examination case scenario conducted via SL. EM residents in this training program completed mock oral examinations in a traditional format, conducted face to face with a faculty examiner. All current residents were invited to participate in a similar case scenario conducted via SL for this study. The examinee managed the case while acting as the physician avatar and communicated via headset and microphone from a remote computer with a faculty examiner who acted as the patient avatar. Participants were surveyed regarding their experience with the traditional and virtual formats using a Likert scale. Results: Twenty-seven EM residents participated in the virtual oral examination. None of the examinees had used SL previously. SL proved easy for examinees to log into (92.6%) and navigate (96.3%). All felt comfortable communicating with the examiner via remote computer. Most examinees thought the SL encounter was realistic (92.6%), and many found it more realistic than the traditional format (70.3%). All examinees felt that the virtual examination was fair, objective, and conducted efficiently. A majority preferred to take oral examinations via SL over the traditional format and expressed interest in using SL for other educational experiences (66.6 and 92.6%, respectively). Conclusions: Application of SL virtual simulation technology is a potential alternative to traditional mock oral examinations for EM residents. ACADEMIC EMERGENCY MEDICINE 2011; 18:559–562 ª 2011 by the Society for Academic Emergency Medicine
From the Department of Emergency Medicine (JS, NK, DB, DRM, SK, RNe), the Center for Education and Scholarship (RNa), and the Department of Obstetrics and Gynecology (DRD), The Ohio State University, Columbus, OH; and the Department of Emergency Medicine, Feinberg School of Medicine, Northwestern University (JV), Chicago, IL. Received August 5, 2010; revisions received October 8 and November 15, 2010; accepted November 20, 2010. Presented at the Society for Academic Emergency Medicine annual meeting, Phoenix, AZ, June 2010. Funded by the Department of Emergency Medicine Strategic Initiatives Grant, The Ohio State University Medical Center. The authors have no financial disclosures or affiliation with Second Life. Supervising Editor: James Miner, MD. Address for correspondence and reprints: Jillian Schwaab, MD; e-mail:
[email protected].
ª 2011 by the Society for Academic Emergency Medicine doi: 10.1111/j.1553-2712.2011.01064.x
S
imulation-based education and assessment strategies have been increasingly applied in medical education.1 Advances in computer processing and internet connectivity have created opportunities for remote simulated virtual experiences. Second Life (SL) is an Internet-based, virtual three-dimensional (3-D) world (Web 2.0 ⁄ Web 3-D dynamic computer-based application produced by Linden Labs, Inc., Atlanta, GA) in which individuals from anywhere in the real world can interact.2,3 This immersive learning environment has been used successfully for medical and public health education.3 SL has been used to model doctor–patient relationships, teach clinical diagnosis, train for disasters, and even virtually tour the human anatomy.4–6 However, educational research on 3-D virtual worlds and their effect on outcomes is limited.2
ISSN 1069-6563 PII ISSN 1069-6563583
559
560
Figure 1. The Second Life examination room: second author’s avatar, Doctor Inkpen, interviews first author’s patient avatar.
In SL, users are represented in a virtual world by their avatars. An avatar is an online, self-created, animated characterization of the user that can act in any role (doctor, patient, nurse, or teacher) and perform programmed tasks (Figure 1). An oral board examination offered semiannually by the American Board of Emergency Medicine is the second step in the board certification process for residency-trained emergency physicians. In the current format, examiners and candidates must travel to Chicago, and patient cases are administered face to face to assess candidates’ competencies in the practice of emergency medicine (EM). Residents frequently prepare during residency by practicing ‘‘mock’’ oral examinations. Residents at our academic EM residency program participate in annual mock oral examinations conducted in the traditional format (face-to-face interaction with proctor) in preparation for the oral certification examination. In this study we explore the use of SL virtual simulation technology to administer mock oral examinations to EM residents. METHODS Study Design This study was a prospective, observational study that was granted institutional review board exemption from informed consent requirements. Study Setting and Population The study setting was a university-based, academic EM residency training program. All current EM residents (n = 36) were invited to participate in an additional oral examination case conducted via SL virtual interface. Study Protocol The case scenario was a patient with ectopic pregnancy (identical to a case that was administered in the traditional format within 1 month prior to the SL examination). The examinee managed the patient case while acting as the physician avatar in SL. The faculty proctor acted as the patient avatar. The examinee and proctor
Schwaab et al.
•
AVATARS
were in remote physical locations and communicated via headset and computer within SL. The SL program was downloaded to the computers designated for use in this study free of charge.7 Two EM faculty proctors administered all SL examinations after limited training (2 hours of self-directed use and 1 hour of case rehearsal). Faculty proctors used a generic login for the patient avatar. The examinee was provided instructions with generic login information (screen name and password for the physician avatar) and instructions for navigation and communication in the SL hospital and examination room. Examinees (physician avatar) verbally interviewed the proctor (patient avatar) for historical details and physical examination findings. A collection of pertinent diagnostic data was created in PowerPoint with individual slides subsequently loaded onto an image viewer in SL (Figure 1). The image viewer was controlled by the faculty proctor, allowing diagnostic data (initial and repeat vital signs, laboratory reports, and diagnostic imaging including ultrasound) to be displayed in the virtual examination room when requested by the examinee or at appropriate times during the case. Outcomes Participants were surveyed regarding their experience with the traditional and virtual oral examinations using a five-point Likert scale (Table 1). Additional questions following the survey addressed the students’ perceptions of the utility of SL in medical education and elicited subjective comments from participants. Data Analysis Descriptive data are reported as percentages. RESULTS Twenty-seven EM residents participated in the virtual examination (results in Table 1). None of the examinees had used SL previously. Examinees found SL to be user-friendly and easy to log into. Examinees were able to navigate within SL using the physician avatar and locate the examination room with the instructions provided. All examinees felt comfortable communicating with the faculty proctor via remote computer. Nearly all examinees felt comfortable communicating with the faculty proctor via microphone and headset. Most examinees found the SL encounter to be realistic, and many felt that the virtual format was more realistic than the traditional format. Participants commented that they ‘‘were more able to use visual clues (as in real life),’’ ‘‘can see what has been done (i.e., there are my fluids, there’s the monitor),’’and ‘‘like seeing the labs, images, and data on the screen.’’ Very few examinees found the virtual format intimidating, and a majority found it less intimidating than the traditional format. Specific comments were ‘‘less intimidating when the proctor is not so close, able to gather your thoughts easier and stay organized’’ and ‘‘not face-to-face with stranger you don’t know and awkward.’’ All examinees expressed that the virtual examination was fair and objective and was conducted efficiently. A majority of participants expressed a preference
ACAD EMERG MED • May 2011, Vol. 18, No. 5
•
www.aemj.org
561
Table 1 Results of SL Resident User Survey
It was easy to login to SL using the instructions provided. It was easy to use my avatar to navigate in SL. It was easy to locate the virtual examination room and patient. I felt comfortable using the computer to communicate during the SL mock examination. I felt comfortable using the microphone to communicate during the SL mock examination. SL provided a realistic patient encounter for a mock examination. SL provided a more realistic patient encounter than traditional mock oral examinations. SL mock examination was intimidating. SL provided a less intimidating experience than traditional mock oral examinations. SL mock examination was fair and objective. SL mock examination provides a more fair and objective evaluation of my patient management than traditional oral examinations. SL examination is an efficient way to complete mock examinations. I would prefer to complete more of my oral examination requirements using a virtual patient and interface than a traditional format. SL should be used for other medical education experiences.
% Agree or Strongly Agree
% Neutral
% Disagree or Strongly Disagree
92.6 96.3 96.3
7.4 3.6 3.7
0 0 0
0
0
92.6
7.4
0
92.6
7.4
0
70.4
25.9
3.7
11.1 63.0
7.4 37.0
81.5 0
100.0 40.7
0 59.3
0 0
100.0
0
0
66.7
33.3
0
92.6
7.4
0
100.0
SL = Second Life.
for practicing oral examinations via SL, and residents were interested in using SL for other educational experiences, such as mock codes and case conferences. When queried regarding potential disadvantages, some participants suggested drawbacks to a lack of face-to-face interaction. Expressed concerns were that they might receive ‘‘no cues from the examiner (nonverbal as feedback)’’ or it is ‘‘perhaps harder to assess physician’s interpersonal skills.’’ DISCUSSION Concerns about the process of traditional oral examination have been cited by EM residents and recent graduates preparing for this examination.8 Attempts are made to replicate a patient encounter in the ED; however, the traditional oral examination process remains somewhat artificial. The examinee is unable to visualize the patient and is only able to view paper copies of physical examination findings and diagnostic imaging. Another concern is related to the intimidation or anxiety that may occur when examinees are queried face to face by examiners with whom they are not familiar. Further, despite formalized scoring systems, there is an element of subjectivity when an examiner assesses examinees in person.9 Literature suggests a possible bias for candidates with good interpersonal skills, good communication skills, and those who are physically attractive.10 Other criticisms are of a logistical nature, such as the expense of preparation and practice or travel to the testing site in Chicago.9 If virtual examinations are found to be comparable to traditional oral examinations, there may be many
potential benefits; such advantages may be economic savings, time conservation, and possible limitation of subjectivity in EM residency and medical student oral examinations. Although this study does not evaluate whether SL is equivalent to the traditional oral board examination in assessing candidates, one can envision the application of advanced virtual simulation technology as a way to alleviate some of the barriers encountered in the current process. In addition to reported ease of use and perception by many that this was a more realistic experience, the SL virtual examination format is adaptable. Using SL, a mock oral examination could be administered from any remote location with computer access and at any time of day. Thus, oral examinations could be completed while on away rotations, while travelling, at home rather than in an office setting, or at a remote testing site. Many aspects of testing via virtual simulation require exploration before such technology can be appropriately implemented for general use in oral board examination. Faculty perceptions and experiences need to be evaluated. Additionally, this format must be assessed for limitations regarding reliability, interobserver agreement, and general outcomes in formative and evaluative settings. Resident feedback regarding further use of SL in EM education was overwhelmingly positive. Many examinees desired the capability of more advanced interaction with the virtual patient and within the examination room. With currently available animation and programming capabilities, items in the room could be more interactive in future examinations. The examinee might click on the patient avatar’s body to perform physical
562
examination skills or the IV pole to order IV fluids and instruct a nurse avatar to perform programmed tasks. Transition to an automated SL scenario, without a reallife proctor, could be achieved through application of artificial intelligence.4 Further, this could be easily expanded to a multicase format, requiring concurrent care of multiple patients and the physician avatar to transition between patient rooms. LIMITATIONS This study design measures feasibility of administering a virtual examination; however, it does not evaluate the effectiveness of a virtual examination in the assessment of examinees relative to a standard oral examination. This study was conducted at a single academic training site. Examinees experienced the same simulated patient case in the traditional format used previously, and case familiarity may have contributed to a more positive experience during the subsequent virtual examination. Three examinees reported feedback or echoing in the headset, and one experienced a shutdown of the computer system requiring a restart of the examination. Such impediments can likely be eliminated completely with upgraded quality of microphone and headset and system capabilities. CONCLUSIONS Application of Second Life virtual technology is a potential alternative to traditional mock oral examinations for EM residents. Residents found Second Life to be a user-friendly, realistic venue for simulated patient encounters. Research is necessary to characterize this immersive learning environment for use in resident education and assessment. References 1. Vozenilek J, Huff JS, Reznek M, Gordon JA. See one, do one, teach one: advanced technology in medical education. Acad Emerg Med. 2004; 11:1149– 54. 2. Hansen M. Versatile, immersive, creative and dynamic virtual 3-D healthcare learning environments: a review of the literature. J Med Internet Res. 2008; 10:e26.
Schwaab et al.
•
AVATARS
3. Boulos Maged , Kamel N, Hetherington L, Wheeler S. Second Life: an overview of the potential of 3-D virtual worlds in medical and health education. Health Info Libr J. 2007; 24:233–45. 4. Danforth D, Procter M, Heller R, Chen R, Johnson M. Development of virtual patient simulations for medical education. J Virt Worlds Res. Available at: https://journals.tdl.org/jvwr/article/view/707/503. Accessed Feb 28, 2011. 5. Simon S. Avatar II: the hospital. Wall Street J. Available at: http://online.wsj.com/article/SB10001424 052748703909804575124470868041204.html. Accessed Feb 28, 2011. 6. Yellowlees P, Cook J, Marks S, Wolfe D, Mangin E. Can virtual reality be used to conduct mass prophylaxis clinic training? A pilot program. Biosecur Bioterror. 2008; 6:36–44. 7. Linden Research, Inc. Download Second Life Viewer 2. San Francisco: March 31, 2010 Version 2.0.1 Available at: http://secondlife.com/support/ downloads/?lang=en-US. Accessed Feb 28, 2011. 8. Bianchi L, Gallagher EJ, Horte R, Hazen PH. Interexaminer Agreement on the American Board of Emergency Medicine oral certification examination. Ann Emerg Med. 2003; 41:859–64. 9. Platts-Mills TF, Lewin MR, Madsen T. The oral certification examination. Ann Emerg Med. 2006; 47: 278–82. 10. Lunz ME, Bashook PG. Relationship between candidate communication ability and oral certification examination scores. Med Educ. 2008; 42:1227–33.
Supporting Information The following supporting information is available in the online version of this paper: Data Supplement S1. AVATAR Video for CES. The document is in PDF format. Please note: Wiley Periodicals Inc. is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing material) should be directed to the corresponding author for the article.
