Development and Evaluation of an Interactive Dental Video Game to

Development and Evaluation of an Interactive Dental Video Game to Teach Dentin Bonding Rafat S. Amer, D.D.S., M.S.; Gerald E. Denehy, D.D.S., M.S.; Deborah S. Cobb, D.D.S., M.S.; Deborah V. Dawson, Sc.M., Ph.D.; Marsha A. Cunningham-Ford, B.S., M.S.; Cathia Bergeron, D.D.S., M.S. Abstract: Written and clinical tests compared the change in clinical knowledge and practical clinical skill of first-year dental students watching a clinical video recording of the three-step etch-and-rinse resin bonding system to those using an interactive dental video game teaching the same procedure. The research design was a randomized controlled trial with eighty first-year dental students enrolled in the preclinical operative dentistry course. Students’ change in knowledge was measured through written examination using a pre-test and a post-test, as well as clinical tests in the form of a benchtop shear bond strength test. There was no statistically significant difference between teaching methods in regards to change in either knowledge or clinical skills, with one minor exception relating to the wetness of dentin following etching. Students expressed their preference for an interactive self-paced method of teaching. Dr. Amer is Assistant Professor, Division of Restorative and Prosthetic Dentistry, College of Dentistry, The Ohio State University; Dr. Denehy is Professor and Head, Department of Operative Dentistry, College of Dentistry, University of Iowa; Dr. Cobb is Associate Professor, Department of Operative Dentistry, College of Dentistry, University of Iowa; Dr. Dawson is Professor and Director of Biostatistics, College of Dentistry, University of Iowa; Ms. Cunningham-Ford is Associate Professor, Department of Preventive and Community Dentistry, University of Iowa, College of Dentistry; and Dr. Bergeron is Associate Professor, Department of Operative Dentistry, College of Dentistry, University of Iowa. Direct correspondence and requests for reprints to Dr. Rafat Amer, College of Dentistry, The Ohio State University, 305 West 12th Ave., Columbus, OH 43210-1267; 614-292-1472 phone; [email protected]. Keywords: operative dentistry, interactive lecture, computer simulations, video games, virtual systems, dental education, computer-assisted instruction Submitted for publication 7/19/10; accepted 12/29/10

P

rojections suggest the demand for qualified dentists will continue to grow1 as many dentists retire and more people are retaining their natural teeth, thus requiring more dental care.2 Although new dental colleges are opening their doors to meet this increased demand for dental care,3,4 the shortage in the number of qualified dental faculty members is at a new peak.5,6 The dental faculty shortage adds to the already high workload of existing educators,7 so more effective teaching techniques need to be developed.8 In addition to making dental teaching less faculty-intensive, innovations in dental educational technology should address ways in which current and future dental students learn. Dental students have long practiced clinical dental procedures on dental simulators, which traditionally were mannequins that could be positioned like patients. Some recently introduced more advanced simulators provide computer feedback to students on their progress.9,10 These virtual reality physical simulators, however, require

June 2011 ■ Journal of Dental Education

highly specialized custom-made equipment that is usually very expensive.11 Some authorities have argued that simulators do not need to replicate the real physical experience to be successful.12 An effective computerized training program should faithfully represent the cognitive mental demands that happen in the real world. For simulators to be effective, mental training is more important than physical training.12,13 In addition to the argument that a physical simulator is not a necessity for a successful dental simulator, not all dental clinical procedures require refined psychomotor skill such as that required to prepare a cavity. Some procedures, like applying a resin bonding system to a tooth, require following specific steps and timing each step. For these procedures, a simpler, less expensive computer interactive program, in the form of a video game, may provide the information and training needed. An interactive dental video game (IDVG) provides many advantages over a more traditional, passive (e.g., lecture) way of teaching this kind of

823

procedure. An IDVG can provide anonymous feedback, allowing students to learn from their mistakes. In addition, an IDVG allows unlimited repetition and practice of the procedure without incurring the cost of involved materials. Moreover, an IDVG can be used in large classrooms as well as in smaller ones.14 Perhaps most importantly, students can use this game module without supervision to learn the proper sequencing procedures at any time on or off campus. In this way, the use of such technology will free up faculty time and allow increased student learning. It should be noted that dental students of today belong to a much different generation compared to those of years ago. Much has been said about these “Generation Y” computer-savvy individuals who spend more time playing video games than reading15 and who learn by trying things out rather than being told. IDVG modules as previously described lend themselves well to a format familiar to the current generation of dental students. However, little is known of the educational impact of IDVG or of students’ acceptance of such a teaching method. The purpose of our study was to compare performance on both written and practical skill evaluation of firstyear dental students receiving a conventional passive clinical video instruction versus an interactive learning module teaching the same skills set. A survey questionnaire was also administered to determine attitudes towards the IDVG style of learning.

Methods and Materials This study compared knowledge and clinical skills obtained by first-year dental students regarding a three-step resin bonding procedure using a randomized controlled trial design. Specifically, knowledge and skills were compared between first-year students completing an interactive video game teaching module and those completing a more traditional video instruction module. Because an interactive dental module teaching the three-step resin bonding procedure was not available on the market in a video game format at the time of this study, it was necessary to develop an interactive dental video game to teach the sequencing steps of applying a three-step resin bonding system as used in the operative dentistry curriculum at the University of Iowa College of Dentistry. At the time of this experiment, the components of this procedure included an Ultra-Etch tooth conditioner (Ultradent Products, Inc., South Jordan, UT), along with Opti Bond FL dentin primer and

824

Opti Bond FL light-cured adhesive (Sybron Dental Specialties, Orange, CA). It was decided that the IDVG interface should look like a video game, but that the students would need to read the instructions before being able to perform the virtual procedure. Thus, the program was developed to allow advancement to the next step only if the current step was performed adequately. The interactive module was designed in the form of a video game using Adobe Flash CS3 software (Adobe Systems Incorporated, San Jose, CA). Creating interactive content using Adobe Flash creates a very compact file size that can run on most computers, regardless of operating system. The study was conducted with approval of the University of Iowa Human Subjects Office. The subjects were first-year dental students (2008–09 academic year) at the University of Iowa College of Dentistry enrolled in the Preclinical Operative Dentistry Techniques course, an introductory course for dental students conducted in the college’s simulation clinic during the fall semester of the first year. These students were introduced to the research and were asked to become volunteers for the study. All consented to participate. The class was randomized into two groups: an experimental group that played the computer game, and a control group that watched a clinical video of the same procedure. All participants in the study had attended the same lecture on resin bonding together prior to their involvement in the study (Figure 1). Following the scheduled lecture on bonding, a tenquestion pre-test was given prior to conducting the experiment for students in both groups to assess each student’s existing knowledge of the bonding procedure. These questions were in multiple-choice format, with only one possible correct answer for each question (Figure 2). Specifically, each participant in the experimental group sat in front of a computer and used the IDVG independently for up to twenty minutes (Figures 3–6). Two faculty members provided technical assistance to the participants in case of computer or software malfunction, but did not provide any instructions about the resin bonding procedure. Students in the control group watched a three-minute clinical video depicting the steps of the resin bonding system narrated by an experienced native English-speaking educator. The students watched the video twice on individual monitors before answering the post-test questionnaire. A separate questionnaire was completed by students in the IDVG group to assess their attitudes

Journal of Dental Education ■ Volume 75, Number 6

towards the IDVG format. The questionnaire asked them to compare their views on its effectiveness and clarity to those of traditional instruction. The questionnaire consisted of nine Likert-scale questions, measured on a five-point ordinal scale. The possible responses were strongly disagree, disagree, neutral, agree, and strongly agree. In addition to the nine Likert-scale questions that evaluated students’ experience with IDVG, two questions were asked to collect data regarding the relationship between the students and their video game playing experiences and types of popular technology they use on a regular basis. The students were also asked to write features of the game they liked the least and any changes they would like to see in the final version of the game. After completing watching the video or using the IDVG, all students completed a practical examination that consisted of a bonding exercise. This exercise required the students to bond a composite resin button to the dentin of an extracted human tooth embedded in an acrylic cylinder (Figure 7). Students’ performance on this practical exercise was determined by assessing shear bond strength as determined using a shear bond tester (T-63010 Bisco, Schaumburg, IL) (Figure 8). Wilcoxon rank-sum tests were used to compare the two groups with respect to student knowledge and performance.

Results A total of eighty students between the ages of twenty-one and thirty-nine participated in this study. There were thirty-one females and forty-nine males in this class. Students were originally divided randomly into two groups, with forty-two in the control group (Group I) and thirty-eight in the study group (Group II). There were only thirty-eight computer stations available at the time of this experiment, resulting in this disparity in group size. At the time of the experiment, however, one student from the experimental group accidentally went with the control group instead of his designated group, resulting in the experimental group being reduced to thirty-seven students. For this reason, the student was reassigned from Group II to Group I, resulting in forty-three students in this group. Due to IRB requirements, participation in this experiment was voluntary and anonymous. No personal identifiable information was gathered, and participation in this experiment had no effect on course grade. One student from the control group did


Figure 1. Study design

not return the immediate pre-test questionnaire, and another student from the same group did not return the immediate post-test questionnaire to the research team. Data from both students were eliminated from the study and final analysis. As a result, the study ended with forty-one students in Group I and thirtyseven students in Group II.

Survey Responses The nonparametric Wilcoxon rank-sum (Wilcoxon-Mann-Whitney) procedure was used to compare the responses of the two study groups for the total score based on the ten questions on the pre-test and then on the post-test questionnaires. There was no difference between groups in prior knowledge, as measured by the pre-test total score (p=0.81), with a median score of 7 in the control group and 6 in the IDVG group. Similarly the two groups did not appear to differ with respect to post-test total scores (p=0.27). Both groups had a median post-test score of 8 (Table 1). Comparison of the distribution difference score (total post-test score minus total-pre-test score) also yielded no evidence of group differences (p=0.13). There was no difference between the two groups when considering individual questions

825

4. When applying the primer on the conditioned dentin, the dentin surface should be: a. Wet b. Damp c. Dry 5. Following the application of a primer, air should be blown on the surface to: a. Produce a dry surface b. Produce a very thin layer of the primer c. Remove the solvent from the primer d. Remove the water from the tooth 6. Which component of the three-step etch-and-rinse bonding system should be applied as several coats and then gently blown with air to evaporate the solvent? a. The primer b. The acid etch c. The adhesive d. None of the above 7. When a conditioner is applied to the tooth, which structure is it applied to first? a. Enamel b. Dentin c. It is applied to both simultaneously 8. After rinsing the conditioner from dentin, we should remove water from the cavity to the extent that the surface should be: a. Wet b. Damp c. Dry 9. After applying the resin adhesive to the preparation, the dentist should: a. Air dry to evaporate the solvent b. Run it in by agitating with the brush for thirty seconds c. Light cure

Figure 2. Sample of questions from the pre-test questionnaire

(p>0.05 in all instances, Fisher’s exact test). There was a suggestive finding (p=0.0011, Fisher’s exact test) that the control group was more likely to answer post-test question 4 correctly (32/41, 78.1 percent) than were the members of IDVG group (15/37, 40.5 percent). Similarly, responses to question 8 were somewhat suggestive (p=0.075, Fisher’s exact test), with twenty-four out of forty-one (58.5 percent) controls answering correctly and fourteen out of thirty-seven (37.8 percent) of the IDVG group subjects answering correctly. Both question 4 and 8 asked about the wetness of the cavity surface after etching and before applying the primer. The fact that dampness of the cavity is hard to simulate on screen might have contributed to this somewhat suggestive difference. No other group comparisons were statistically significant (p>0.05) for any of the other post-test questions.

826

Only students in the experimental group were given an attitude questionnaire to collect feedback on their experience with the module. Two of the thirty-seven students in this group returned the questionnaire unanswered and were not included in these analyses. Students in the IDVG group were asked to fill out the attitude questionnaire after completing the test and before receiving any feedback on their performance on the written or practical tests. The attitude questionnaire was intended to reflect the personal opinions of the students regarding a different from usual teaching approach. Responses tended to be very favorable to the use of the interactive video game, with most falling in the agree zone of the Likert scale (Table 2). The question that generated the most negative feedback with a shift in responses towards the disagree zone was question 9. This question


Figure 3. IDVG opening screen

Figure 4. IDVG, applying acid to the virtual tooth surface

Figure 5. IDVG, applying the primer to the virtual tooth surface

Figure 6. IDVG, light curing after applying the adhesive layer

asked students what they thought of an interactive module completely replacing the lecture in teaching this topic. Of the students in this study, 11 percent strongly disagreed with this concept. None of the other eight questions on this survey received any “strongly disagree” response. Students’ response to question 10 addressed the video game playing habits of the dental students in this experiment, and 65.71 percent reported that they liked to play video games in some capacity. Responses to question 11 revealed that these firstyear dental students use personal computers and cell phones on a regular basis (94 percent and 86 percent, respectively); however, only a few reported using Smart Phones or handheld PCs (31 percent and

9 percent, respectively). This is an important finding that will guide the future development of the IDVG.


Shear Bond Strength Testing The shear bond strength testing exercise was conducted immediately after all participants in this study completed their post-test questionnaire. All students were asked to bond a composite resin stub to the prepared dentin of a flattened tooth surface. The force required to break this bonded composite button was measured for each specimen in Megapascals (MPa) by using a shear bond tester. The highest score recorded was 37.20 MPa. Forty-two results out of forty-three were recorded for Group I, and thirty-six results out of thirty-seven were recorded

827

Figure 7. Bonding a composite button to a prepared tooth surface using the Ultradent jig

Figure 8. The bonded composite sample is placed in the shear bond measuring device, and the force needed to break the composite button is recorded

for Group II. The results from the two students who were previously eliminated due to nonresponse were not included in the final analysis of the shear bond strength, resulting in the final control group sample size of forty-one. The nonparametric Wilcoxon rank-sum test was used to compare the results of the shear bond strength testing between the two study groups. There was no difference between the two groups in the distribution of shear bond strength measures from the practicum (p=0.97), with a median shear bond strength of 12.8 MPa in the control group and 12.9 MPa in the IDVG groups.

clinical steps of the three-step etch and rinse bonding system. Properly performing dental bonding requires knowledge of repetitive sequential steps. Although specific timing and proper sequencing are crucial to successful bonding, this procedure does not require highly developed psychomotor skills. Unlike studies that have assessed the impact of computer simulators on the psychomotor skills of students,16-18 our study was designed from the beginning to test clinical procedures that require mastering sequencing and handling concepts rather than a refined psychomotor skill. Some researchers have suggested that interactive software should be evaluated for many important technical criteria, including the quality of the end user interface, which affects the user’s perception of the product; the level of engagement, which is the motivation to work with the medium; and the interactivity and customization of the software, which allows users to configure the program settings to meet individual needs.19 It is very time- and cost-intensive to evaluate any software according to the previously mentioned criteria. This explains why evaluation of dental and medical educational software is mostly reported in the literature in the form of quantitative or qualitative evaluations.20-23 Some experts have argued that conducting randomized trials to compare a study group using a computer-based module to a control group using a traditional module is a flawed endeavor.19,24,25

Discussion This study compared the clinical knowledge and performance of first-year dental students receiving two differing educational approaches: one passive and one interactive. A conventional video demonstration was compared to a computer interactive video game teaching the three-step etch and rinse bonding system. Some may argue that a clinical video recording is not a conventional teaching method, but for the students in this study, clinical video clips are used on a regular basis as a passive part of the lecture. To the best of our knowledge this was the first attempt to evaluate the efficacy of an interactive module to teach dental students the

828


Table 1. Results for Wilcoxon rank-sum test comparing the IDVG group to the control group Group

N

Control

41

IDVG

37

Variable

Mean

Standard Deviation

Median

Minimum

Maximum

Pre-test Post-test Change Shear bond strength test (MPa)

6.27 8.34 2.07 13.45

1.48 1.46 1.74 9.14

7.0 8.0 2.0 12.6

3.0 4.0 -3.0 0.0

9.0 10.0 7.0 35.6

Pre-test Post-test Change Shear bond strength test (MPa)

6.49 8.03 1.54 13.41

1.50 1.38 1.54 9.20

6.0 8.0 2.0 12.9

4.0 4.0 -2.0 0.0

10.0 10.0 4.0 37.2

Table 2. Students’ responses (N=35) on the IDVG attitude questionnaire, by percentage of total respondents

Strongly Agree

1. The instructions and prompts were clear and self-explanatory. 2. I enjoyed using the interactive dental video game (IDVG). 3. Using the IDVG helped me understand the proper way of using the three-step bonding system. 4. IDVG helped me identify my strengths and weaknesses in understanding the three-step bonding system. 5. Using the IDVG helped me maintain interest in the topic. 6. After using the IDVG I feel more confident for the upcoming practical shear bond strength testing exercise. 7. I consider using the IDVG more informative than attending a conventional lecture. 8. I prefer being taught the three-step resin bonding system using IDVG rather than a lecture. 9. I believe that the IDVG should completely replace a lecture in teaching the three-step bonding system.

The research question of whether a video game results in better learning outcomes does not usually distinguish between two important confounding effects, that of the educational media and that of the educational method. Our research attempted to address this reservation by delivering the passive educational format as a clinical video recording viewed on a computer monitor screen instead of comparing the interactive module to a live demonstration.

Limitations of the Study The timing of this experiment was the first experience of the students in handling composite resin, resulting in improper condensation of the composite resin into the Ultradent jig well (see Figure 7). Some of the low bond strengths recorded in this exercise might be due to improper handling of the composite


Agree

Neutral

Disagree

Strongly Disagree

69% 60% 46%

31% 37% 49%

0 3% 5%

0 0 0

0 0 0

26%

48%

23%

3%

0

46% 31%

49% 49%

5% 14%

0 6%

0 0

43%

34%

23%

0

0

49%

37%

14%

0

0

9%

17%

32%

31%

11%

when preparing the samples (which our teaching module never addressed), rather than improper handling of the resin bonding system. Also, we were unable to find a validated evaluation instrument in the literature to measure dental students’ attitude towards interactive educational modules. This meant that we had to develop our own instrument. Due to the limited window of opportunity to run our experiment we were unable to evaluate our questionnaire for reliability (reproducibility of measurements) or validity (how well it measures the variables of interest) before administering it. For the most part, the pre-test and post-test to evaluate the change in knowledge regarding the resin bonding procedure did not show any statistically detectable difference. For future research, it would be much better to compare knowledge retention after a longer period of time.

829

Figure 9. Amount of time dental students in our study reported playing video games

Questionnaire Responses Responses to the IDVG evaluation questionnaire were favorable. Most students preferred being taught the three-step resin bonding system by a computer game rather than a lecture. Due to the lack of a crossover design in our study, the experimental group was not exposed to the clinical video demonstration that the control group had. We were therefore unable to assess whether the students would rather have a video recording or an interactive module to learn the bonding procedure. No evaluation questionnaire was administered to the control group to assess their preference for a clinical video. The researchers considered the clinical video to be part of the traditional passive teaching module. The biggest negative response from the students (43 percent either strongly disagreed or disagreed) came in response to question number nine. This question asked whether students believed IDVG should completely replace a lecture. It could be argued by reading this result differently, however, that 57 percent of the students would like the IDVG to replace a lecture or would not care if it did. Regarding the relationship between dental students and video game playing, our questionnaire found that 66 percent of the dental students like to play video games in some capacity (Figure 9). This is very close to the data reported for the general public by the Entertainment Software Association (ESA),

830

which found that 65 percent of American households play computer or video games.26 The ESA also reported that 38 percent of homes in America have a video game console, which is higher than what our study found for dental students. In conclusion, our findings indicate that results using an interactive dental video game are as good as a passive, noninteractive way of teaching. They also show dental students preferring this method of teaching to a lecture. Further development and research are needed to fully utilize the potential of this technology.

Acknowledgments

Gratitude is expressed to Dr. Steve Armstrong, Department of Operative Dentistry, and Dr. John Warren, Department of Community Dentistry, University of Iowa College of Dentistry.

REFERENCES 1. Collier R. United States faces dentist shortage. CMAJ 2009;181(11):E253. 2. Douglass CW. Patients’ expectations for oral health care in the 21st century. J Am Dent Assoc 2000;131(1):3S. 3. Fox K. ECU, Western U move forward with plans for dental schools. ADA News, March 2009. 4. Fox K. Reversing a trend? Five new dental schools under consideration. ADA News, March 2009. 5. Walker MP, Duley SI, Beach MM, Deem L, Pileggi R, Samet N, et al. Dental education economics: challenges and innovative strategies. J Dent Educ 2008;72(12):1440–9.


6. Chmar JE, Weaver RG, Valachovic RW. Dental school vacant budgeted faculty positions, academic years 2005–06 and 2006–07. J Dent Educ 2008;72(3):370–85. 7. Froeschle ML, Sinkford JC. Full-time dental faculty perceptions of satisfaction with the academic work environment. J Dent Educ 2009;73(10):1153–70. 8. Pyle M. The case for change in dental education. J Dent Educ 2006;70(9):921–4. 9. Buchanan JA. Experience with virtual reality-based technology in teaching restorative dental procedures. J Dent Educ 2004;68(12):1258–65. 10. Jasinevicius TR, Landers M, Nelson S, Urbankova A. An evaluation of two dental simulation systems: virtual reality versus contemporary non-computer-assisted. J Dent Educ 2004;68(11):1151–62. 11. Welk A. German dental faculty attitudes towards computer-assisted simulation systems correlated with personal and professional profiles. Eur J Dent Educ 2006;10(2): 87–95. 12. Fernandez A. Cognitive training for basketball game intelligence: interview with Prof. Daniel Gopher, November 2006. At: www.sharpbrains.com/blog/2006/11/02/ cognitive-simulations-for-basketball-game-intelligenceinterview-with-prof-daniel-gopher/. Accessed: May 8, 2011. 13. Gopher D. Transfer of skill from a computer game trainer to flight. Hum Factors 1994;36(3):387–405. 14. Mayo MJ. Games for science and engineering education. Communications Assoc Comput Machinery 2007;50(7): 30–5. 15. Prensky M. Don’t bother me, mom—I’m learning. St. Paul, MN: Paragon House Publishers, 2006. 16. Jasinevicius TR, Landers M, Nelson S, Urbankova A. An evaluation of two dental simulation systems: virtual reality versus contemporary non-computer-assisted. J Dent Educ 2004;68(11):1151–62.


17. Wierinck ER, Puttemans V, Swinnen SP, van Steenberghe D. Expert performance on a virtual reality simulation system. J Dent Educ 2007;71(6):759–64. 18. Von Sternberg N, Bartsch MS, Petersik A, Wiltfang J, Sibbersen W, Grindel T, et al. Learning by doing virtually. Int J Oral Maxillofac Surg 2007;36(5):386–90. 19. Geissinger H. Educational software: criteria for evaluation. Melbourne: Australian Society for Computers in Learning in Tertiary Education, 1997. 20. Howerton W. Interactive computer-assisted instruction vs. lecture format in dental education. J Dent Hyg 2004; 78(4):10–3. 21. Lechner SK, Lechner KM, Thomas GA. Evaluation of a computer-aided learning program in removable partial denture framework designing. J Prosthodont 1999;8(2):100–5. 22. Roubidoux MA, Chapman CM, Piontek ME. Development and evaluation of an interactive web-based breast imaging game for medical students. Acad Radiol 2002;9(10): 1169–78. 23. Sanders C, Kleinert HL, Boyd SE, Herren C, Theiss L, Mink J. Virtual patient instruction for dental students: can it improve dental care access for persons with special needs? Chicago: American Dental Association, 2008. 24. Schleyer T, Johnson LA. Evaluation of educational software. J Dent Educ 2003;67(11):1221–8. 25. Ehrmann SC. Asking the right questions: what does research tell us about technology and higher learning? Washington, DC: Heldref Publications, 1995. 26. Entertainment Software Association annual report, 2008. Washington, DC: Entertaiment Software Association, 2008.

831