Leveraging Quick Response Code Technology to

Empirical Investigations

Leveraging Quick Response Code Technology to Facilitate Simulation-Based Leaderboard Competition Todd P. Chang, MD, MAcM; Cara B. Doughty, MD, MEd; Diana Mitchell, MD; Chrystal Rutledge, MD; Marc A. Auerbach, MD, MS; Karin Frisell, MD; Priti Jani, MD; David O. Kessler, MD, MSc; Heather Wolfe, MD; Ralph J. MacKinnon, MBChB; Maya Dewan, MD; Jonathan Pirie, MD; Daniel Lemke, MD;

Introduction: Leaderboards provide feedback on relative performance and a competitive atmosphere for both self-guided improvement and social comparison. Because simulation can provide substantial quantitative participant feedback, leaderboards can be used, not only locally but also in a multidepartment, multicenter fashion. Quick Response (QR) codes can be integrated to allow participants to access and upload data. We present the development, implementation, and initial evaluation of an online leaderboard employing principles of gamification using points, badges, and leaderboards designed to enhance competition among healthcare providers. Method: This article details the fundamentals behind the development and implementation of a user-friendly, online, multinational leaderboard that employs principles of gamification to enhance competition and integrates a QR code system to promote both self-reporting of performance data and data integrity. An open-ended survey was administered to capture perceptions of leaderboard implementation. Results: Conceptual step-by-step instructions detailing how to apply the QR code system to any leaderboard using simulated or real performance metrics are outlined using an illustrative example of a leaderboard that employed simulated cardiopulmonary resuscitation performance scores to compare participants across 17 hospitals in 4 countries for 16 months. The following three major descriptive categories that captured perceptions of leaderboard implementation emerged from initial evaluation data from 10 sites: (1) competition, (2) longevity, and (3) perceived deficits. Conclusions: A well-designed leaderboard should be user-friendly and encompass best practices in gamification principles while collecting and storing data for research analyses. Easy storage and export of data allow for longitudinal record keeping that can be leveraged both to track compliance and to enable social competition. (Sim Healthcare 13:00–00, 2018)

Key Words: Simulation, resuscitation, games, experimental, competitive behavior

Mona Khattab, MD; Nancy Tofil, MD, PhD; Chenthila Nagamuthu, MPH; Catharine M. Walsh, MD, MEd, PhD; on behalf of INSPIRE In-Hospital QCPR Leaderboard Investigators

Q

uick Response (QR) codes are two-dimensional, square bar codes that can be scanned on any smartphone device using a simple and often free application. They encode either an

From the Children's Hospital Los Angeles and University of Southern California, Los Angeles, CA (T.P.C); Texas Children's Hospital and Baylor College of Medicine, Houston, TX (C.B.D, D.L., M.K.); University of Chicago Comer Children's Hospital, Chicago, IL (D.M., P.J.); University of Alabama Birmingham and Children's Hospital Alabama, Birmingham, AL (C.R., N.T.); Yale University and Yale New-Haven Health System, New Haven, CT (M.A.); Mälarsjukhuset, Eskilstuna, Sweden and Karolinska Institutet, Solna, Sweden (K.F.); Children's Hospital of New York and Columbia University, New York, NY (D.K.); Children's Hospital of Philadelphia and University of Pennsylvania, Philadelphia, PA (H.W.); Royal Manchester Children's Hospital, Manchester, UK and Karolinska Institutet, Solna, Sweden (R.M.); Cincinnati Children's Hospital & Medical Center and University of Cincinnati, Cincinnati, OH (M.D); Hospital for Sick Children and University of Toronto, Ontario, Canada (J.P., C.N., C.W.).

e-mail address or a specific Web site URL, thus enabling smartphone users to directly access Web sites that are encoded by QR codes without typing, copying, or memorizing the Web site address, using their own device. The use of QR codes within healthcare informational technology systems has previously Reprints: Todd P. Chang, MD MAcM, Children's Hospital Los Angeles, Division of Emergency Medicine & Transport, 4650 Sunset Blvd, Mailstop 113, Los Angeles, CA 90027 (e‐mail: [email protected]). Supported by the American Heart Association Western States Grant-in-Aid Fund 15GRNT25030000 (TPC) and a National Health Services Grant (RJM). Presented as a poster presentation at the International Pediatric Simulation Symposium & Workshops, Glasgow, UK, May 7, 2016. Copyright © 2018 Society for Simulation in Healthcare DOI: 10.1097/SIH.0000000000000281

Vol. 13, Number 1, February 2018

Copyright © 2018 by the Society for Simulation in Healthcare. Unauthorized reproduction of this article is prohibited.

1

been described as mobile technologies have improved.1,2 Descriptions of their use among trainees and healthcare providers suggest that QR codes allow for a more convenient method of data entry that improves compliance.1 The intent is to reduce barriers to traditional data collection by leveraging a technology that is nearly ubiquitous. Gamification is a complex term that frequently describes the addition or integration of traditional game-like elements into a nongame activity, such as simulation-based healthcare professionals training.3 Leaderboards are a simple gamification tool that rank participants based on a quantitative score such as a point value or time. Leaderboards have been used not only in simulation-based cardiopulmonary resuscitation (CPR) training but also in other health professions education domains, including general internal medicine knowledge,4 quality improvement principles,5 and tumor board presentations2. In addition, leaderboards are often combined with “achievement badges,” which are visual signifiers of some predefined achievement, milestone, or competency that further decorate the distinctions of the participants. Novel CPR mannequins with appropriate sensors and feedback software are capable of providing quantifiable scores that reflect adherence to the American Heart Association or European Resuscitation Council guidelines. This feedback allows for scores to be displayed on a competitive leaderboard. MacKinnon et al6 have shown that use of a leaderboard can trigger self-motivated practice of CPR skills in a pediatric postanesthesia recovery unit. Their methodology used a simple pen-and-paper leaderboard that worked well in a single unit but would not be feasible in a larger, multicenter setting. A rapid, real-time, and crowd-sourced method to populate an online leaderboard was required to expand the reach beyond hospital wings and institutions. There is minimal literature on the combination of an online leaderboard and a QR code system that transcends the limitations of a pen-and-paper leaderboard. The purpose of this article is to describe the development, implementation, and initial evaluation of a QR code system integrated within an online leaderboard developed specifically to post-CPR performance scores within a collaborative of institutions in the International Network for Simulation-based Pediatric Innovation, Research, and Education (INSPIRE). The leaderboard was used across 17 hospitals in 4 countries for 16 months resulting in more than 1000 uploads. The leaderboard was specifically developed to use participants' own smartphones to enable self-reporting of CPR data. This concept, technology, and methodology can be widely adapted to any institution, multicampus system, or across regional, national, or international settings for a variety of simulation-based endeavors.

THEORETICAL FRAMEWORK Self-guided learning is the framework within which learners can generate their own feedback and critique for certain skills.7 Using this framework, the process of reporting one's performance prompts self-reflection on skills and leads one to incorporate the simulator-enabled feedback to motivate further practice and achievement. Therefore, scores that are used to build the leaderboard rankings facilitate self-guided learning.7,8 2

QR Codes for Simulation Leaderboard Competition

Social comparison theory provides an additional framework for the development of the leaderboard.9 Social comparison theory postulates that individuals compare themselves to others to evaluate or enhance some aspect of the self. The leaderboard facilitates comparison among individuals by making performance scores visible, thus acting to motivate performance and enhance self-improvement—both competition within oneself and among other competitors. The precedent for using selfie photographs to spur competition and challenge is supported by the recent success of the Ice Bucket Challenge10 and the high prevalence of “selfie” sharing. The technological innovations augment the possibility of comparative evaluation among regional, national, and even international cohorts. In our study, we sought to examine CPR skills, which are known to be poor among healthcare providers11 and prone to skill decay over time.12 There is also a gap between the taskvalue of CPR—conceivably the most dramatic, lifesaving skill possible—and the motivation to practice it, which may not be as strong.13 The leaderboard was developed with these gaps in mind, using elements from the two previously mentioned learning theories.

LEADERBOARD FUNCTIONALITY NEEDS The leaderboard was intended to be used for a multinational network of institutions to provide real-time updates on CPR performance scores for a research study and for subsequent permanent use. At the time of this writing, no method existed to allow for multiple institutions worldwide to share CPR performance data from the Laerdal ResusciAnne and ResusciBaby (Stavenger, Norway) simulators to a central online leaderboard. There was also no capacity to share photographs. The features required for an online leaderboard were ascertained for a 1-month period through a small group discussion process involving several authors (TPC, DOK, MAA) within INSPIRE and in separate consultation with a programmer fluent in Web design and PHP language programming. The solution of a QR code and leaderboard system was developed such that its conceptual framework could be used for any activity that allowed for quantitative scoring and photo sharing, regardless of the activity—be it simulation based, clinical, or even outside of healthcare entirely. The leaderboard needed to feature the following: • Accessibility. The ability to view the leaderboard at any time on multiple device types in space was important. Any healthcare staff from any institution within INSPIRE across the world should be able to both view the available data and to submit scores, regardless of time or type of device. Although applications could have been developed to meet this need, to enhance feasibility and compliance, we opted for a Web site that did not require participants to download an application to their devices to view the leaderboard. • Real-time updates. Updates for a new high score needed to reflect the change in leaderboard status quickly after submission to provide instant feedback. This was particularly important with many participants working evenings, nights, and across multiple time zones. • Possibility of additional gamification principles. Gamification is the addition of game-like elements into real-life training or learning.3 The three most common gamification elements used in education include points, badges, and leaderboards.Because the leaderboard was designed to collect CPR performance scores, it fulfilled both points and the leaderboard. During construction of the leaderboard, we were mindful of the future presence of Simulation in Healthcare


•

•

•

•

achievement badges that are particularly relevant to longevity of competition. Images, particularly selfies. In 2014 when the leaderboard logistics were being developed, we bore witness to the Ice Bucket Challenge, a highly successful social media funding campaign by the Amyotrophic Lateral Sclerosis Association.10 Much of its viral success lay in its use of the selfie photograph to provoke peer-to-peer competition. In the study by MacKinnon et al,6 participants were observed to spur each other on to improve on each score, quickly taking to social media despite social media was not a part of the research protocol. Therefore, the leaderboard development team added a selfie upload feature in hopes of fostering a similar competitive atmosphere. Interface to add/subtract institutions. We projected that the leaderboard would attract further participation from additional institutions worldwide in a rolling fashion. Furthermore, the ability to remove an institution or a cluster of participants quickly without manually deleting data was important to facilitate randomized control studies using the leaderboard. Self-reporting. There was a conscious effort by the development team to create a platform that allows participants to upload scores independently using their own smartphone devices. This feature permitted use of a self-directed learning framework within the research design and enables healthcare staff to practice CPR on a simulator at their convenience and report their scores without assistance. We capitalized on the widespread availability of smartphones to enable self-reporting and image or selfie uploading, although the convenience of using one's own device also was a limitation as we discuss hereinafter. Downloading. For data analysis purposes, the leaderboard required the ability to export leaderboard data from the server in a usable format (eg, *.csv file).

DEVELOPMENT For this leaderboard's ranking system, we used the summative scores generated from a Laerdal SkillReporter or SkillsGuide connected to a QCPR simulator (infant or adult). Either of the devices calculated a simple score from 0% to 100% that was used by the leaderboard to rank participants. The development and usage cycles are summarized in a flow diagram in Figure 1. To plan the concept and development, multiple face-toface meetings were held between the author (TPC) and programmer for 5 months to ascertain the capacity of the PHP language to accommodate all the needs, as listed previously, with limited funding. PHP is a coding language that is used to manage complex databases and retrieval of data. Later meetings were used to troubleshoot issues that were noted in the pilot phases. Server space and several simple domain names were

purchased to begin the leaderboard, the cost of which was estimated at less than US $100 annually. An administrator portal to the leaderboard was developed for research staff leading the CPR study. This portal allowed researchers to assign numbers to code a variety of demographic information to be displayed on the leaderboard, including the type of mannequin, participant's department, and the participant's institution. Geographical location was defined by the institution's city and country. These variables were chosen only for the specific study to assign each potential user a nine-digit number. This was an identification number whose individual digits represented an element of the various demographic information displayed on the leaderboard. For example, the first digits signify the type of simulator mannequin, and the next digits signify the institution code and so on. The final four digits were reserved to identify individual participants per institution, leaving up to 10,000 participants per department per institution (0000 through 9999). Figure 2 illustrates the filtering system of the nine-digit number used for this leaderboard. The purpose of this identification system was to allow the leaderboard programming to quickly filter and sort large blocks of data according to the variables we chose for this study, for example, adult/child mannequin scores were all between 120000000 to 129999999. Each nine-digit number was then turned into a unique upload Web site URL. An example screenshot is shown in Figure 3. This was done by using the domain name and adding the nine-digit number as a subfolder (eg, http://www. exampleleaderboard.com/1201109999). The database was configured to aggregate data within each of these URLs, which were identifiable by the nine-digit number. That is, the numbering-URL system established a many-to-one relationship database such that each participant had a unique and identifiable URL. Web URL addresses can be translated into QR codes. Small batches of QR codes were printed on sticker paper using a batch QR code generation service (http://www. qrstuff.com/). These were distributed to the institutions included in the study, and representatives at each participating institution assigned each participant a unique QR code. After simulated CPR practice, the participant was instructed to scan his or her QR code. After scanning, the smartphone immediately went to the participant's unique upload Web site (each participant had a unique Web site encoded by the QR

FIGURE 1. Development and use flow diagram. Vol. 13, Number 1, February 2018

© 2018 Society for Simulation in Healthcare


3

FIGURE 2. The nine-digit numbering system for the CPR study used in the leaderboard development.

code) where the participant was asked to type in their CPR performance percentage score. The Web site asked the participant to submit an optional photo using the camera within their device; for this study, it was a selfie photograph with the actual CPR score displayed on the reporting device. Participants were encouraged to, but not required to, be in the photograph as a true selfie and be visible on the leaderboard. All data were aggregated to the central database. When the uploading process was completed, the leaderboard was updated based on the new score assigned to the unique participant. The leaderboard filters the nine-digit number into demographic information that is displayed on the leaderboard and ranks the new score as compared with scores already on the leaderboard. The photograph is saved onto the server and accessible as a thumbnail and full *.jpg file (standard photo file). To enable achievement badges, *.png files (standard icon or vector image file) of badges were developed concurrently by the research team to reflect best practices in simulation-based CPR training. These included achievements in high scores and score improvement, longevity of practice (eg, multiple uploads over time), department and institutional averages, and regimentation (eg, a noted set schedule of CPR practice, such as every 2 weeks). Requirements to earn badges were converted to mathematical algorithms and provided to the leaderboard programmer to embed into the

PHP code. PHP is a coding language primarily designed for dynamic Web sites that use a server-based database. As an example, an institutional badge would be awarded to the highest score within study IDs 120100000 through 120199999, which corresponds to the highest score within the institution depicted in Figure 4. We planned on the addition of new badges over time with corresponding algorithms. To decrease potential barriers to accessibility, we provided participants with guidance on optimal QR code reader applications (Fig. 5) and made the Web site public. Each institution was responsible for notifying participants of the public nature of voluntary selfies during the informed consent process, before distributing the QR codes. The QR codes were printed on sticker paper for the study. These were placed on a convenient surface (eg, back of identification badge, notebook next to mannequin) depending on institutional preference.

EVALUATION In addition to ongoing quantitative data collection, site directors were asked to complete an open-ended questionnaire designed to elicit feedback and observations on how people were responding to the social and technical aspects of the leaderboard, including usage patterns, barriers to adoption, and recommended changes. Open-ended survey questions were posed in e-mail form from the principal investigator (TPC)

FIGURE 3. The unique URL web address for the nine-digit number corresponding to a unique participant. This has one score entry and a photograph upload button (iOS). 4


Simulation in Healthcare


FIGURE 4. The online leaderboard.

to the site directors: 1. What advantages have you seen with this QR code and leaderboard system? 2. What difficulties have you encountered with this system? 3. How would you modify this system for other leaderboards/selfies in other healthcare simulation activities? Responses were deidentified and compiled into Dedoose qualitative analytic software (SocioCultural Research Consultants, LLC, Los Angeles, CA) for coding. All respondents were site directors with professional ties to the principal investigator. Codes were generated using a rough phenomenology framework; we sought to investigate the subjective experiences of a larger technological innovation because it applied to a specific institution through the site directors' viewpoint.14 Multiple coders, agreement through consensus, and member checking were strategies employed to ensure trustworthiness of the findings. The inductive coding process began with initial coding, in which all data were labeled according to their topic on a line-by-line basis. This was completed independently by two investigators (TPC and CMW).

Subsequently, the investigators met to compare interpretations and review codes, reconciling different viewpoints through discussion. Member checking of the codes (ie, checking with participants that the codes are in accord with their experiences) was employed as a validation strategy.15 This was accomplished by resending the entire deidentified data and codes to the 10 survey respondents, who were in consensus with the coding strategy. Thereafter, three main descriptive categories were derived and defined by two authors (TPC and CMW) from all available codes. All survey respondents were asked to further refine the definitions of the categories. Institutional review board approvals were obtained at all participating institutions.

RESULTS The online leaderboard is depicted in Figure 3. This is a fully accessible online leaderboard that has filters for the top 5, 10, and 25 CPR performance scores. Scores remain on the leaderboard

FIGURE 5. Suggested QR code reader applications. Vol. 13, Number 1, February 2018

© 2018 Society for Simulation in Healthcare


5

for only 2 weeks and thereafter drop from the visible leaderboard; this was designed to encourage repetitive practice. All uploaded data—visible or not—, however, are stored within the database and are only accessible via a separate login. This allows for CPR training or administrative staff to use the performance and data histories for both assessment and tracking purposes. Evaluation Based on evaluation data from 10 sites, three major descriptive categories emerged from the qualitative analysis that captured perceptions of leaderboard implementation. Competition A majority of the 10 site leaders responding to the threequestion survey noted the leaderboard's utility in motivating highly competitive staff members—both with internal competition within a site and external competition between sites. However, the competitive nature of the leaderboard was viewed as divisive; while more competitive staff were encouraged to train, others were put off by the competition. From one observer, “The use of competition engaged people in a way that went beyond just training. The competitive nature of some providers results in repeated practice and aim to “win”; however, others did not get as enthusiastic about competing.” In three of the sites, the novelty of the game mechanics was viewed as a frequent motivator, and a “chain reaction” was also observed, in which one participant inspires competitive play within a cluster of staff within minutes of each other. The immediacy of CPR performance feedback and ability to compare scores across centers were viewed as factors, which fostered competition. Longevity and Catalysts Several sites noted that the leaderboard was better suited for more short-term motivation. Long-term motivation, progressing for months, was more difficult. All sites noted a subjective decrease in motivation as the leaderboard's novelty wore off. Barriers to longevity included technical difficulties at many of the sites, particularly with the QR code system. In addition, although the QR code system was meant to make the uploading and photography process simpler, participants who self-identified with being “tech averse” were intimidated by the system that depended on coordinating reliable Wi-Fi and QR code reader application software. Some participants self-identified as tech averse and others were identified anecdotally by site directors or research assistants. No formal data collection was performed on the technological aptitude of participants being tech averse to the QR code system was not relegated to a particular demographic group (age, sex, etc). Most participants used available facilitators or research assistants to upload data successfully. What was perceived as an advantageself-motivated freedom to trainwas at times a disadvantage; there was not always someone available to facilitate the data uploading process. It was noted that presence of a local champion encouraged longevity, and facilitation with trained simulation specialists or research assistants dramatically reduced the barriers for participation in all sites. Perceived Deficits (Barriers) Anonymity, lack of automatic data upload, and access were perceived as deficits of the leaderboard. Some emphasized 6


that using identifiers such as names or discipline to label the leaderboard and make it more targeted or personal would improve uptake and foster competition. Alternatively, others preferred the anonymous nature and noted a general reluctance to take selfie photographs. Another site director noted, “I was surprised by the number of people who opted out of a selfie photograph, even though they were ok with submitting data—these folks had supervised CPR through research assistants and I believe they did not practice on their own. It dawned on me that the benefits of public competition is [sic] not universally felt for a group of participants.” Because the QR code system and software were used as a workaround system, many identified the current system's inability to automatically upload simulator performance scores as a deterrent to participation. Finally, it was noted by some that the leaderboard would function more effectively if always visible to participants, rather than having find the Web site each time. DISCUSSION We present the development, implementation, and initial evaluation of an online leaderboard employing principles of gamification using points, badges, and leaderboards designed to enhance competition among healthcare providers. The leaderboard is currently being used for a randomized control crossover study, which hypothesizes that institutional exposure to a competitive leaderboard improves self-motivated simulated CPR practice. However, this design can be adapted to any simulation-based activity that produces quantitative performance metrics. The innovative aspect of using QR codes and a smartphone device to allow self-reporting of scores is integrated into this leaderboard. This data entry system has several advantages. Firstly, the QR code system provides a linear method of uploading scores using devices readily accessible to all subjects and research staff worldwide. This reduces barriers to traditional data collection—it eliminates memorizing study IDs, losing paper forms, or mislabeling data with the wrong study ID. Modern data upload methods such as logins, passwords, and multiple steps to upload data can severely hamper compliance and participation. The QR code system, although imperfect, improves the efficiency within which a Web site progresses from the entry to the actual transaction, termed conversion efficiency.16 Improving conversion efficiency is a common business tactic for commerce Web sites but is equally applicable to Web sites or applications collecting research data. Here, we should note a limitation of the QR code system within the study. Laerdal currently has a system, Resuscitation Quality Improvement that displays CPR performance scores from its connected simulators onto a local leaderboard; their system currently does not allow for multicenter data sharing or photography. Therefore, any leaderboard within the Resuscitation Quality Improvement system is only local. It uses a traditional login system to a computer connected to the mannequins. Although our system is advantageous in that it allows for a multicenter and selfie-friendly approach, it adds the step of the QR code scanning and data entry. Even though these two transactions are relatively quick, it does require carrying Simulation in Healthcare


a smartphone, having working mobile data connectivity or Wi-Fi, and installing the appropriate QR code reader application. In this sense, the process of using a QR code system to push data to the leaderboard is less efficient than the leaderboard pulling the data from the mannequin automatically. Data collection, photography, and QR codes require use of a personal smartphone device, a purposeful design feature that encourages participants to donate their own data. In addition, some study sites elected to leave a tablet device with the CPR mannequin to facilitate entry and uploading of data to the leaderboard, although no obvious difference was observed by any site director. Participants in the research study are responsible for and encouraged to perform data collection and entry on their own, particularly with voluntary selfie photographs. The QR code structure allows for active participation and crowd sourcing the data collection for simulation performance metrics. This means that data collection is done by the actual participants and the responsibilities spread out, rather than through a centralized collection process, such as through a research assistant. An additional advantage to the system outlined is that much of the data acquisition is automated, which includes a time and date stamp that can be used to graph upload frequency. For studies tracking usage patterns, having a QR code system with automated date and time stamps makes frequency data simpler to capture. Quick Response codes have not been explored extensively in the context of medical education. Articles on mobile resources often feature QR codes printed in the article itself,17 and QR codes have been used for case log management among residents.1 In the study by Avidan et al, 1 QR code implementation increased compliance with case log reporting by anesthesiology residents from 46% to 96% after 6 months. The ready accessibility of a QR code removes significant barriers to compliance in uploading data when it replaces a more cumbersome login-password or paper system. Allowing each subject to upload their own data independently contributes to the sense of self-efficacy and achievement, in line with the competitive behavior elicited by the leaderboard. Consistent with adult learning theory,18 the ease of use of the QR codes facilitates self-directed learning of CPR skills because it allows learners to independently plan, carry out, and evaluate their own learning experiences, using the leaderboard to help benchmark their performance against their peers to facilitate generation of future learning goals to promote iterative improvement. In addition, as individuals achieve proficiency in CPR skills with deliberate practice, it is likely to promote feelings of competency, which, in line with self-determination theory, have been shown to augment internal motivation and further improve learning.19 Furthermore, both the leaderboard and the selfie photograph facilitate social comparison among peers, because participants likely feel pressure to “measure up” to their peers, a factor that may enhance learning.9 Another advantage to the QR code and leaderboard system outlined in this article is that it is generic. In contrast, a proprietary leaderboard is fixed to work only with a particular product, mannequin, or a mannequin setting. Using a generic QR code system allows for a leaderboard that is agnostic to the skill or knowledge being measured. Vol. 13, Number 1, February 2018

Finally, the photograph attenuates one of the most common disadvantages of crowd sourcing data collection: falsified or inaccurate entries. This assists in maintaining data integrity. Initially, the photograph and selfie were included to inspire peer-to-peer competition in a manner similar to the Ice Bucket Challenge and other social media campaigns. For privacy reasons, the study protocol did not mandate a selfie but required that all photographs feature the actual score displayed on the reporting device. This requirement means that the data entered by the participant can be corroborated by the score generated by the simulator software. Future technologies may enable automatic data entry and eliminate the need for manual input by participants. CONCLUSIONS We present the fundamentals behind the development and implementation of an online, multisite leaderboard that allows for competition using gamification principles, which was created for a specific CPR performance study. This leaderboard uses QR codes to reduce barriers to participation and data collection with distinct advantages. The leaderboard and QR code concept can be adapted for other simulationbased studies within which feedback and scores are generated or simulation-based training that produces performance metrics. This concept, though, is a simple adjunct that draws on competitive learning, and additional self-directed learning methods are likely to synergize with leaderboards and QR codes. Further inquiry into best practices of implementation and mitigating short-term popularity is warranted when these gamification elements are more broadly applied to simulationbased education. REFERENCES 1. Avidan A, Weissman C, Levin PD. Integration of QR codes into an anesthesia information management system for resident case log management. Int J Med Inform 2015;84:271–276. 2. Siderits R, Yates S, Rodriguez A, Lee T, Rimmer C, Roche M. Embedding QR codes in tumor board presentations, enhancing educational content for oncology information management. J Registry Manag 2011;38:209–211. 3. Hamari J, Koivisto J, Sarsa H. Does Gamification Work? A Literature Review of Empirical Studies on Gamification. In: Proceedings of the 47th Hawaii International Conference on System Sciences. January 6–9, 2014, Hawaii, USA System Sciences. IEEE,2014. 4. Nevin C, Westfall AO, Rodriguez JM, et al. Gamification as a tool for enhancing graduate medical education. Postgrad Med J 2014;90: 685–693. 5. Scales CD Jr., Moin T, Fink A, et al. A randomized, controlled trial of team-based competition to increase learner participation in quality-improvement education. Int J Qual Health Care 2016;28: 227–232. 6. MacKinnon RJ, Stoeter R, Doherty C, et al. Self-motivated learning with gamification improves infant CPR performance, a randomised controlled trial. BMJ Simul Technol Enhanced Learn 2015. 7. Brydges R, Carnahan H, Safir O, Dubrowski A. How effective is self-guided learning of clinical technical skills? It's all about process. Med Educ 2009;43:507–515. 8. Brydges R, Dubrowski A, Regehr G. A new concept of unsupervised learning: directed self-guided learning in the health professions. Acad Med 2010;85:S49–S55. 9. Raat J, Kuks J, Cohen-Schotanus J. Learning in clinical practice: Stimulating and discouraging response to social comparison. Med Teach 2010;32:899–904. © 2018 Society for Simulation in Healthcare


7

10. Landen R. Ice Bucket Challenge offers invigorating fundraising lessons. Mod Healthc 2014;44:12.

15. Mays N, Pope C. Qualitative research in health care. Assessing quality in qualitative research. BMJ 2000;320:50–52.

11. Cheng A, Brown LL, Duff JP, Davidson J, et al. Nadkarni VM for the INSPIRE CPR Investigators. Improving cardiopulmonary resuscitation with a CPR feedback device and refresher simulations (CPR CARES Study): a randomized clinical trial. JAMA Pediatr 2015;169:137–144.

16. Spiliopoulou M, Pohle C. Data mining for measuring and improving the success of web sites. Data Mining Knowledge Discovery 2001;5:85–114.

12. Sutton R, Niles D, Meaney PA, et al. Low-dose, high-frequency CPR training improves skill retention of in-hospital pediatric providers. Pediatrics 2011;128:e145–e151. 13. Hopstock LA. Motivation and adult learning: a survey among hospital personnel attending a CPR course. Resuscitation 2008;76:425–430. 14. Johnston CM, Wallis M, Oprescu FI, Gray M. Methodological considerations related to nurse researchers using their own experience of a phenomenon within phenomenology. J Adv Nurs 2017;73:574–584.

8


17. Baumgart DC. Smartphones in clinical practice, medical education, and research. Arch Intern Med 2011;171:1294–1296. 18. Knowles M, Holton EF, Swanson RA. The Adult Learner: The Definitive Classic in Adult and Human Resource Development. 7th ed. Elsevier Inc: Oxford, England; 2011. 19. Ten Cate TJ, Kusurkar RA, Williams GC. How self-determination theory can assist our understanding of the teaching and learning processes in medical education. AMEE guide No. 59. Med Teach 2011;33:961–973.

Simulation in Healthcare
