A Comparison of Student Performance in a Simulation Clinic and a Traditional Laboratory Environment: Three-Year Results James M.S. Clancy, D.D.S., M.S.; Terry J. Lindquist, D.D.S., M.S.; Joyce F. Palik, D.D.S., M.S.; Lynn A. Johnson, M.S., Ph.D. Abstract: With simulation clinics, dental schools have improved their preclinical laboratories to provide a more realistic clinical teaching environment. However, there is very little data to support the assumption that these facilities actually improve student performance of technical skills. This study compared the scores of two fixed preparations for full cast crowns by third-year dental students. One of the preparations was made in the simulation clinic manikin, and the other was prepared on the bench top. Three prosthodontic faculty members scored the preparations in the areas of occlusal reduction, axial reduction, resistance and retention, and margination. The study also compared the performance of three classes of dental students: one class with no experience in the simulation clinic, one with one year of experience, and one with two years of experience. The amount of time since completing the fixed prosthodontics course among the students was also evaluated. This was done because the third-year students at the University of Iowa rotate through a series of ten-week clerkships rather than a comprehensive care model. (Therefore, not all students start clinical prosthodontics at the same time.) In addition, all student participants completed a questionnaire that addressed their perception of their clinical readiness prior to treating their first fixed prosthodontic patient. When we compared the classes of years 1, 2, and 3 by average preparation score, we found a significant difference among the scores for teeth prepared on the bench top (p = 0.0001) but not for the teeth prepared in the mannequin (p = 0.1176). For Year 1 (no simulation clinic experience), the amount of elapsed time following completion of the fixed prosthodontic course was not significant for the tooth prepared on the bench top or in the manikin (p = 0.57113 and 0.0661). For Year 2 (one year of simulation clinic experience), the elapsed time following completion of the fixed prosthodontic course was significant for the tooth prepared on the bench top (p = 0.0482), but it was not significant for the tooth prepared in the manikin (p = 0.2968). For Year 3 (two years of simulation clinic experience), the amount of elapsed time following completion of the fixed prosthodontic course was not significant for the tooth prepared on the bench top or in the manikin (p = 0.7275 and 0.6007). The questionnaire revealed that, in general, the majority of the students perceived their clinical readiness as more than adequate. These results are mixed in that students with more bench top experience scored better on the bench top, and students with more manikin experience scored equally in both environments. Dr. Clancy is Associate Professor, Department of Prosthodontics; Dr. Lindquist is Associate Professor, Department of Prosthodontics; Dr. Palik is Assistant Professor, Department of Prosthodontics; and Dr. Johnson is Associate Professor, Department of Oral Pathology, Radiology, and Medicine, all at the College of Dentistry, University of Iowa. Direct correspondence to Dr. James Clancy, Department of Prosthodontics, College of Dentistry, University of Iowa, Iowa City, IA 52242; 319-335-7258;
[email protected]. Key words: fixed prosthodontics, manikins, preparations, simulation Submitted for publication 6/4/02; accepted 10/9/02
T
he University of Iowa College of Dentistry opened a new teaching facility, the Simulation Clinic, in fall 1998. This facility was designed to improve and facilitate the teaching of preclinical dental students. Prior to the construction of this facility, preclinical dental students were taught in a traditional bench-type laboratory. It was thought that preclinical dental students would learn dental procedures and develop hand skills better and faster in this new type of facility. However, there is very little research to prove that this assumption is true. In the past decade, several dental simulators with lifelike manikins have been developed through the cooperative efforts of dental supply companies
December 2002
■
Journal of Dental Education
and academic institutions. In a realistic operatory setting, novice dental students use these manikins, which have synthetic teeth and cheeks, to learn dentistry’s technical skills prior to treating actual patients. The manikins and associated operatory stations are designed to reproduce the key aspects of the dental patient in the physical environment of a dental operatory. It was anticipated that these facilities would be particularly effective in teaching physical tasks, such as mirror positioning, that ideally should be fully understood and performed automatically prior to patient treatment. When such physical skills are automated, students are less distracted by these subskills and are able to concentrate on mas-
1331
tering more complex mental skills.1 The new manikins and stations, which provide a more realistic setting for learning the practice of dentistry than the bench environment, are believed to increase the transfer of learning.2-4 Dental simulators installed in teaching institutions over the past few years are generally of two basic types. The first (KaVo, Adec, or Nevin) is a system that utilizes a realistic manikin and dentiform (KaVo, Colombia, Kilgore, or Frasaco) surrounded by a simulated dental treatment operatory. The second type of simulator (DentSim by DenX, Inc.5) is similar but also utilizes computer-assisted tracking to follow a special dental handpiece while it is being used by a student and provides immediate feedback for teaching and evaluation. Buchanan et al.6 have studied this more complex system at the University of Pennsylvania and found that it significantly decreased the amount of time necessary for students to achieve an acceptable performance level. The use of simulators to facilitate teaching technical skills is not new. The military and the airline industry, for instance, have used sophisticated mechanisms and strategies for training and maintaining technical skills for many years.4,7,8 More recently, medicine has begun utilizing computer-assisted simulators to teach technical skills associated with complex surgery.9 With appropriate demonstration and training, simulators can help students become competent in, or at least familiar with, technical skills through repeated exposure to consistent stimuli over time. This repetition develops automaticity, which is the ability to perform a task so well that it no longer requires the deliberate attention of the learner. In dental education, it is highly desirable for students to develop some degree of automaticity with certain clinical skills (positioning the dental mirror, manipulating the dental handpiece, patient positioning, familiarity with infection control practices, etc.) prior to treating patients. In this way, students can concentrate on the more complex, non-ideal variables presented by live patients during treatment. Although there is abundant literature describing the benefits of cognitive simulations in dentistry, few studies have explored the use of dental simulators for the development of technical skills. This is not surprising, as sophisticated dental simulators have only recently been used in dental schools. Chan et al.10 studied the pre- and postsimulation performance of dental students at the University of Georgia where a simulation laboratory has been in use since 1995. When the operative dentistry course
1332
was reviewed, it was found that the percentage of As given to students decreased from 22.7 percent to 4.5 percent, while the percentage of Bs and Cs rose significantly. These changes were attributed to the increased difficulty and realism of performing on a dental simulator as opposed to the previous method. In addition, because of the increased efficiency of the new facility, it was possible to increase the faculty-to-student ratio in the course and decrease the number of required procedures. In a formative study, Johnson et al.11 described the development of a prototype dental surgical simulator designed to teach tactile diagnosis of dental caries using a joystick and a dental explorer. Twelve experienced practitioners probed two virtual teeth using a joystick or an explorer attached to a force feedback device. The forces were recorded, and the participants filled out a standardized questionnaire. The initial findings supported the strategy of teaching novice students with lower fidelity experiences followed by increasingly more difficult experiences. One of the main arguments for building a simulation clinic at the University of Iowa was the need for a more realistic environment for preclinical students during their initial experiences with preparing teeth for dental restorations. Prior to construction of the clinic, the student’s first tooth preparations for fixed partial dentures were on a dentiform mounted on an articulator. Today, the students use the simulation clinic extensively in their first two years in several preclinical courses. In their first year, they use the clinic in Dental Anatomy, Principles of Occlusion, Operative Dentistry I, Fixed Prosthodontics Laboratory, Fundamentals of Clinical Dentistry, Periodontic Methods, and Fundamentals of Oral Radiology. In their second year, they utilize the clinic in the following courses: Fixed Prosthodontics Laboratory II, Periodontic Methods II, Operative Dentistry II, Esthetic Dentistry, Endodontic Preclinical Laboratory, Removable Prosthodontic Laboratory, and Orthodontic Laboratory. In both the Fixed and Removable Prosthodontic courses, the students spend at least 50 percent of their laboratory class time (six hours/week) in the simulation clinic practicing clinical procedures on the simulator manikins. The simulation clinic provides the opportunity to simultaneously learn several skills such as tooth preparations, patient positioning, and use of the dental mirror. This study was designed to investigate the impact on students’ technical skills and their perceptions of those skills of preparing dentiform teeth on a manikin in the simulation clinic versus preparing them on an articulator.
Journal of Dental Education ■ Volume 66, No. 12
Material and Methods Third-year dental students were asked to participate in this study at the beginning of their prosthodontics clerkship, which was prior to any experience with fixed prosthodontic preparations on live patients. At the University of Iowa College of Dentistry, the third-year students are rotated through a system of four clerkships rather than a comprehensive care model. Each clerkship takes place over a ten-week period and is based in one or two departments. For example, periodontics and endodontics are paired together, as are operative dentistry and pediatric dentistry. The prosthodontics clerkship is not paired with another department, which allows the students a full ten weeks of concentrated exposure to clinical prosthodontics. Because of this rotating system, some of the students start their third year in prosthodontics, and some do not treat any prosthodontic patients until the last ten weeks. This temporal difference was tracked and analyzed within this investigation. During their fourth year, students
follow a more conventional comprehensive care model in the Department of Family Dentistry. Prior to starting this investigation, the protocol was presented to the human subjects committee and approved. All students who chose to participate in the study (99 percent of the students) were given a general oral description of the study. In addition, all were required to read and sign a detailed series of informed consent documents. The first class of students (Iowa second-year dental class of 1998) to participate in the study had no previous experience with the simulation clinic. The first clerkship of that class was not able to participate in the study because of logistical problems. However, the remaining three clerkships did participate (n = 44). The second class of students (Iowa second-year dental class of 1999 [n = 69]) had one year of preclinical teaching in the simulation clinic. The third class of students (Iowa second-year dental class of 2000 [n = 68]) had two years of preclinical teaching in the simulation clinic. This study was a crossover research design (see Figure 1). Participants were asked to prepare tooth
Figure 1. Research design
December 2002
■
Journal of Dental Education
1333
#3 for a full metal crown in the KaVo dentiform on the benchtop. They were also asked to prepare tooth #14 in the same dentiform, but they prepared this tooth while the dentiform was mounted in the manikin in the simulation clinic. To avoid an order effect, the tooth prepared first alternated from clerkship to clerkship. All participants were given one and onehalf hours to prepare both teeth. At the time of tooth preparation, the students were also directed to fill out a questionnaire. The investigators were interested in measuring the students’ perceptions of the effectiveness of the simulation clinic in preparing them for patient care. The students were asked to fill out a questionnaire that incorporated a Likert scale (see Table 1). The eight questions covered clinical topics that the students were taught in both lecture and the laboratory (tooth preparation, temporary fabrication, etc.), as well as subjects described only in lecture (tissue management, retraction). After the students finished preparing teeth, the dentiforms were collected, and three full-time fac-
ulty members of the Department of Prosthodontics evaluated each preparation before the next clerkship started. The same evaluators were used for all three years of the study. Neither the identity of the students nor the preparation environment (bench top or manikin) was known by any of the evaluators. The criteria for evaluation were the same as that used for the Preclinical Fixed Prosthodontics course. The variables evaluated were: occlusal reduction (clearance, cusp orientation, and form/contour); axial reduction (dimension, orientation/draw, form/contour, and adjacent teeth); retention and resistance (taper/parallel and undercut); and margination (location, definition/ form, continuity, and finish/integrity). Each variable had individual criteria noted in parentheses that were used to evaluate each preparation. The grading scale for each variable was a score of 4, 3, or 2 (equally weighted), representing clinically excellent, clinically acceptable, and clinically unacceptable. The grading form was the same computer “bubble sheet” used in the Preclinical Fixed Prosthodontics course. Each evaluator graded all variables for all preparations and
Table 1. Questionnaire on perception of clinical readiness The number indicates how many students responded in each area each year. Strongly Question Disagree Disagree Neither 4-9 percent 2-3 percent 5-7 percent
4-9 percent 2-3 percent 4-6 percent
Agree
Strongly Agree
Year*
31-71 percent 51-74 percent 49-72 percent
5-11 percent 13-19 percent 9-13 percent
1 2 3
25-57 percent 38-55 percent 42-62 percent
5-11 percent 9-13 percent 3-4 percent
1 2 3
1. Do you feel that you are adequately prepared to begin preparing teeth for fixed restorations on actual patients?
0-0 percent 1-1 percent 1-2 percent
2. Do you feel confident in using a mirror on patients while preparing teeth?
0-0 percent 0-0 percent 2-3 percent
3. Do you feel confident in positioning a patient appropriately for preparing teeth?
0-0 percent 1-2 percent 1-3 percent
3-7 percent 2-3 percent 2-3 percent
5-11 percent 12-17 percent 3-5 percent
28-64 percent 45-65 percent 51-75 percent
8-18 percent 9-13 percent 11-16 percent
1 2 3
4. Do you feel confident in managing the water spray from the handpiece in terms of visibility while preparing teeth?
1-2 percent 3-4 percent 2-3 percent
10-23 percent 16-23 percent 15-22 percent
6-14 percent 15-22 percent 15-22 percent
26-59 percent 31-45 percent 28-41 percent
1-2 percent 4-6 percent 8-12 percent
1 2 3
5. Do you feel adequately prepared to manage soft tissue retraction for a final impression for a fixed partial denture?
0-0 percent 1-2 percent 3-4 percent
3-7 percent 6-9 percent 9-13 percent
6-14 percent 16-23 percent 16-24 percent
33-75 percent 41-41 percent 35-51 percent
2-4 percent 5-7 percent 5-7 percent
1 2 3
6. Do you feel adequately prepared to make a final impression for a fixed partial denture?
1-2 percent 0-0 percent 1-1 percent
3-7 percent 7-10 percent 3-5 percent
7-16 percent 5-7 percent 13-19 percent
29-66 percent 53-77 percent 44-65 percent
4-9 percent 4-6 percent 7-10 percent
1 2 3
7. Do you feel adequately prepared to make a provisional restoration for a fixed partial denture?
1-2 percent 2-3 percent 4-6 percent
6-14 percent 12-27 percent 17-24 percent 20-29 percent 16-23 percent 23-34 percent
23-53 percent 29-42 percent 22-32 percent
2-4 percent 1-2 percent 3-5 percent
1 2 3
8. Do you feel adequately prepared to fit and cement a finished fixed partial denture?
0-0 percent 0-0 percent 1-1 percent
1-2 percent 8-12 percent 9-13 percent
29-66 percent 38-55 percent 37-54 percent
5-11 percent 11-16 percent 10-15 percent
1 2 3
7-16 percent 7-16 percent 9-13 percent 13-19 percent 13-19 percent 8-12 percent
9-21 percent 12-17 percent 11-16 percent
*Year 1, no simulation (N = 44); Year 2, one year of simulation (N = 69); Year 3, two years of simulation (N = 68)
1334
Journal of Dental Education ■ Volume 66, No. 12
submitted the grading forms for analysis. The scoring of the individual evaluators was tracked to analyze interrater reliability. The data was entered into an SAS program for data analysis. The overall average score for each preparation was compared as well as the individual variable scores (occlusal reduction, axial reduction, retention and resistance, and margination) for each preparation using ANOVA (p
