Educational Methodologies
PBL Core Skills Faculty Development Workshop 1: An Experiential Exercise with the PBL Process Kirsten R. Dalrymple, Ph.D.; Carol Wuenschell, Ph.D.; Alvin Rosenblum, D.D.S.; Michael Paine, B.D.S., Ph.D.; David Crowe, D.D.S., D.M.Sc.; Hsing Chi von Bergmann, Ph.D.; Shirley Wong, D.D.S.; Marian Said Bradford, D.D.S.; Charles F. Shuler, D.M.D., Ph.D. Abstract: This report describes the first in a series of foundation-building faculty development workshops focused on the instructional methodology of problem-based learning (PBL). The PBL Process workshop reported here introduced the learning theory topics supporting PBL and utilized an extended roleplay method to provide participants with personal experience with the PBL learning cycle. Overall, participants were satisfied with the methods and content of the workshop. A majority of survey respondents indicated that simulating a complete iteration of the PBL process was an effective way to learn about PBL. Participants expressed relatively greater difficulty understanding and performing activities related to qualitative assessment of learning processes. The workshop was developed to align with adult learning principles, and continued refinement of the workshop has enhanced the learning theory components underpinning PBL as well as the experiential aspects. These dual goals have resulted in blending the existing experiential workshop with an online distance-learning component addressing the learning theory topics relevant to PBL pedagogy. Dr. Dalrymple is Assistant Professor, Division of Diagnostic Sciences, University of Southern California School of Dentistry; Dr. Wuenschell is Assistant Professor, Division of Craniofacial Sciences and Therapeutics, University of Southern California School of Dentistry; Dr. Rosenblum is Professor, Clinical Dentistry Division of Health Promotion, Disease Prevention, and Epidemiology, University of Southern California School of Dentistry; Dr. Paine is Assistant Professor, Division of Surgical, Therapeutic, and Bioengineering Sciences, University of Southern California School of Dentistry; Dr. Crowe is Associate Professor, University of Illinois at Chicago School of Dentistry; Dr. von Bergmann is Assistant Professor, University of Calgary; Dr. Wong is Assistant Professor of Clinical Dentistry, Division of Diagnostic Sciences, University of Southern California School of Dentistry; Dr. Bradford is Assistant Professor of Clinical Dentistry, Division of Diagnostic Sciences, University of Southern California School of Dentistry; and Dr. Shuler is Director of the Center for Craniofacial Molecular Biology and George and Mary Lou Boone Chair of Craniofacial Molecular Biology, University of Southern California School of Dentistry. Direct correspondence and requests for reprints to Dr. Charles F. Shuler, Center for Craniofacial Molecular Biology, 2250 Alcazar St., CSA 103, Los Angeles, CA 90033; 323-442-3170 phone; 323-442-2981 fax;
[email protected]. Key words: faculty development, problem-based learning, adult learning theory Submitted for publication 7/10/06; accepted 9/28/06
P
roblem-based learning (PBL) is an educational methodology found in many different learning environments. It has become a common pedagogy in medical education, and the use of PBL in dental education is growing worldwide although incorporation of PBL into U.S. dental curricula has been limited to date to those dental schools linked to medical schools by a common basic science curriculum that is completed jointly by students in both schools.1 PBL requires a shift to a learner-centered, inquiry-based environment for the students.2,3 Small student groups work cooperatively to analyze a well-structured, clinically relevant patient case. The process of investigating this “PBL case” deliber-
February 2007 ■ Journal of Dental Education
ately places the students in a constructive learning environment. In so doing, the students expose and elaborate upon their prior knowledge.4 Through this analysis of the case, the students identify gaps in their knowledge that must be addressed to understand the patient’s signs and symptoms. They identify and collect resources to achieve the necessary learning outcomes and apply the new information to better understand the signs and symptoms associated with the case under investigation. Throughout the case analysis, the student groups are charged with regularly reflecting on their learning process. Together, these processes comprise the PBL pedagogy, a pedagogy underpinned by the learning theory principles
249
of constructive, contextualized, collaborative, and self-directed learning.4 The faculty member serves as a facilitator of this learning process rather than a primary disseminator of specific content.2,3 A change to a predominantly PBL structure in a dental school curriculum necessitates that faculty be provided support so that they can become familiar with the rationale underpinning this type of pedagogy and experience teaching and learning strategies associated with effective PBL implementation. Two specific sets of factors must be considered in the development of a faculty development program to support the implementation of PBL. First, the sequence, methods, and topics to be included in a PBL faculty development curriculum need to be identified to allow faculty to function as effective facilitators. Second, and perhaps more important, specific obstacles to change must be appreciated.5-8 The transition from the traditional lecturer role to that of a facilitator of learning is difficult for many faculty members.6,7 Pedagogical concerns tend to focus on whether students have sufficient ability or should be trusted to learn more independently. Some faculty express concern that learners are not knowledgeable enough to discuss a case without having been given background information. Others express concern that students will not be able to define what they need to know, find reliable information, or appreciate the nuances of complicated information. As a consequence, many faculty members anticipate that PBL will result in “gaps in knowledge” for the students.9-12 The faculty development program must address both the theory and practice of the PBL pedagogy and faculty concerns about this approach to learning. Previously described PBL faculty development programs have focused primarily on promoting understanding of the rationale for PBL and on learning the PBL process before embarking on training for facilitation-specific roles and responsibilities. The former has typically been achieved via observation or practice of the stages of PBL.6,7,13,14 Focusing on the steps of the PBL process, while important, arguably puts less emphasis on the learning, reflection, and revision taking place in the small-group setting. By charging participants with undertaking a full cycle of learning via the small-group PBL method, the faculty development workshop described in this article emphasized both the steps in PBL learning and the theory supporting the pedagogy. Deliberate linkages were made between the faculty members’ experience of learning via PBL and the underlying
250
educational rationale for its organization. Specifically, these linkages included a purposeful attention to elaborating and refining prior knowledge, engaging in self-directed learning, and reflecting regularly on practice.4 These linkages were introduced not only to put PBL into context as a theory-driven learning methodology but also to introduce educational theory concepts that may not be familiar to individuals in dental education.15 Few dental schools use PBL as a primary pedagogy. Consequently, dental educators have less experience implementing PBL faculty development programs. The purpose of this article is to discuss the approach and participant perceptions of a foundationbuilding faculty development workshop devoted to understanding the rationale and procedures of PBL by allowing faculty to personally experience a full iteration of a PBL learning cycle. The PBL Process workshop described here is the first in a series of connected PBL core skills workshops designed to build a scaffold for faculty development and was followed by workshops focusing on development of facilitation skills and tutorial assessment skills. The overall aims and structure of the comprehensive PBL core skills program were developed to provide faculty at the University of Southern California School of Dentistry (USCSD) with the core skills necessary to facilitate small-group learning in the PBL pedagogy. Implementation of the program was coincident with the phase-in of the PBL pedagogy and the need to have trained facilitators to support the curriculum. Nearly 70 percent of USCSD faculty members have fulfilled their responsibilities in participating in the program workshops. The USCSD framework for development and implementation of its PBL core skills faculty development program along with the implementation outcomes is reported elsewhere.16
Methods: PBL Process Workshop Development Reporting on PBL programs, including the faculty development related to them, has typically been inadequate in that the descriptions have not been sufficiently detailed. Consensus on best practice has thus been difficult to achieve.17,18 In light of this, a thorough description of the topics and methods utilized in our faculty development program will be provided. The Faculty Development, Mentoring, and Evaluation (FDME) Subcommittee of the USCSD Journal of Dental Education ■ Volume 71, Number 2
Curriculum Committee was responsible for devising the PBL core skills program. The FDME identified specific learning outcomes for the first workshop in the series, PBL Process, which are listed in Table 1. These learning outcomes were determined to be necessary to achieve the goals of the workshop; to cover the specific learning theory linked to the use of PBL; to provide a participatory experience using PBL as a pedagogy; and to permit reflection on learning in this mode. Table 2 provides a map for the workshop linking activities, time frame allotted to each activity, learning outcomes linked to specific activities, and
parties principally involved in the activity. In keeping with adult education principles, learning objectives were set for the workshop and were shared with participants at the commencement of the workshop along with introductory information and methods to be utilized during the course of the workshop.19 The PBL Process workshop was advertised to the dental faculty. The USCSD administration required participation in the core skills program for individuals involved with the education of the dental students. A total of 114 individuals completed this workshop. The workshop was conducted ten times between July 2003 and December 2004 and included
Table 1. PBL Process workshop objectives Number
Learning Objective
This workshop helped me …
1
To increase participant understanding of how facts are distinguished, ideas based on facts are generated, and appropriately defined learning issues are generated by learners going through a PBL case.
2
To increase participant understanding of how students utilize resources to generate “Learning Needs” that are relevant to the case and their group’s ideas.
3
To increase participant awareness of how students gauge their limit of understanding from a PBL case.
4
To increase participant understanding of the importance of reviewing case outcomes and defining unresolved learning issues.
5
To increase participant understanding of the importance of conducting self- and peer review.
6
To increase participant understanding of the inquiry-based, student-centered learning approach utilized in the problem-based learning program.
Table 2. Workshop program, resources, and learning objectives Activity
Time Allotted
Parties Primarily Involved
Supporting Materials
Relevant Learning Outcome*
Introduction to PBL process and group dynamics
45’
Workshop leader
PowerPoint slides
1-6
Small-group analysis of case
60’
Facilitated group(s)
Short, biomedical case, flip charts
1, 6
Independent research of learning need topics
60’
Individual work, with facilitator available
Selection of reference textbooks, paper
2, 6
Application and discussion of learning need topics to case
60’
Facilitated group(s)
Flip charts
Case closing activities
15’
Facilitated group(s)
Case closing forms (case outcomes, lingering questions forms)
Self/peer assessment
45’
Facilitated group(s)
Self-assessment form, flip charts
Workshop wrap-up and evaluation
45’
Workshop leader and participants
Survey, PBL journal articles
2, 3, 6 3, 6
5 1-6
*Workshop learning outcomes 1-6 are listed in Table 1.
February 2007 ■ Journal of Dental Education
251
dental hygiene faculty, postdoctoral students, and junior and senior clinical and basic science faculty. The workshop leader was responsible for coordinating the overall program, delivering background information, and primary fielding of questions. Experienced facilitators for the workshop were required for each group of faculty participants. The workshop leader also served as a facilitator for the small-group learning components. Roles for participants, the workshop facilitator, and workshop leader were defined for the participants. Groups of eight to ten participants were constituted. The participants were asked to play the role of students for the day and were encouraged to limit themselves to asking clarifying questions about their task and role during the activity. More global questions about PBL, the role of the facilitator, and assessment were deferred to specific question and answer periods or to the subsequent workshops in the PBL core skills program. A short, biomedical case (Figure 1) was selected for the exercise to ensure the group would have some but not necessarily extensive prior knowledge about the subject area. Each small group of workshop participants was asked to analyze the case following the same process used by a typical student group in PBL—namely, first defining the facts; then, generating ideas based on the facts to explain the patient’s condition; and finally, defining areas for new learning required to evaluate the ideas generated. An experienced PBL facilitator
worked with the small group of faculty in a manner identical to that used with small groups of students. Faculty participants then individually completed a period of resource identification and investigation of the learning topics identified. Individuals were allowed to conduct research at a location of their choosing. The facilitator was available to answer clarifying questions if required. Each faculty member produced information related to the specific learning need that could be shared with the entire group and serve as the basis for the application of these new facts to better understanding the patient’s problem. To allow completion of an entire PBL learning cycle in a single day, two specific concessions were made to the PBL process that differ from the standard student learning experience in which all activities associated with case analysis may extend over several days. First, a collection of content resources was provided to the participants to be used to generate the learning packet. Second, participants were instructed to assemble a brief, unstructured learning packet highlighting key points most applicable to the case. Participants were briefed on the significance of these concessions to the learning process—namely, that depth of learning and discussion would be more constrained and that the practice of independently identifying appropriate resources would not occur. Subsequent to the assembly of the learning packets, participants were tasked to discuss their materials with the group. The facilitator would then
The Promotion A 30-year-old man presented to the emergency clinic of the local hospital with his wife. He reported that two days earlier he and some friends did some excessive “boozing” to celebrate his recent job promotion. He also reported that when he left his friends he ordered a taxi home, but all details of the night beyond that point are vague. After the night of excessive drinking, he had a generalized headache that he said was “worse near the back of his head.” Ibuprofen did not ease the headache. He also reported vomiting and double vision. His wife noted that these symptoms had become more noticeable over the past day.
Figure 1. PBL case utilized in workshop
252
Journal of Dental Education ■ Volume 71, Number 2
rejoin the discussion, posing further questions to the group that facilitated examination of the group’s application of learning to the previously generated ideas. Unanswered questions related to understanding the case were recorded and identified as further topics for research as they represented the limits of learning achieved by the group. The participant group closed the case by reflecting on their learning achievement and conducting a self- and peer evaluation. Documentation used in the PBL dental curriculum was also used in the workshop to increase familiarity with the structure and function of the forms. The day was concluded with a wrap-up discussion. Handouts and selected literature on PBL were provided. A timetable indicating the amount of time dedicated to various components of the PBL case analysis is detailed in Table 2. The role of the small-group facilitator throughout the day was to encourage the group to model the appropriate student behavior. At appropriate times during the course of the activities, the facilitator identified links between the PBL learning process and elements of learning theory. For example, when a participant would state an idea based on previous knowledge or experience, the facilitator would note the importance of this in terms of the learning theory on knowledge construction. On completion of the workshop, a survey on the effectiveness of the PBL Process workshop was distributed to the participants to complete (Figure 2). Coding of surveys was done to remove any personal identifiers from the analysis. A four-point Likert scale was used to query level of participant agreement with achievement of the learning objectives set at the beginning of the day. Mean values and standard deviations were calculated for each survey item. Participants were also asked to describe effective and ineffective methods used for the workshops and were solicited for suggestions for improvements in future workshops. Like responses were grouped into themes and frequencies tallied.
Results Of the 114 surveys distributed at the PBL Process workshops between July 2003 and December 2004, a total of ninety-nine were completed (87.8 percent response rate). Valid responses were available for between ninety-five and ninety-seven participants. On average, participants agreed that all of the objectives set for the PBL Process workshop
February 2007 ■ Journal of Dental Education
had been met (Table 3). Mean participant agreement ranged between a minimum of 3.12 (SD ±0.62) for objective 3 and a maximum of 3.4 (SD ±0.55) for objective 6 with a score of 3=Agree and 4=Strongly Agree (Table 2). The highest-ranked objective on the survey queried whether participants perceived they understood “the inquiry-based, student-centered learning approach utilized in the problem-based learning program.” The middle-ranked objectives (objectives 1, 2, 4, and 5) related to specific steps and process outcomes of PBL. The lowest-rated objective (objective 3) referred to the participant’s understanding of “how students gauge their limit of understanding from a PBL case” and could be potentially perceived as an objective related to the faculty member’s ability to assess student knowledge. Open-ended comments were grouped based on clearly recognizable similarities and then were tallied and ranked. Eighty-five individuals offered comments when asked what elements of the workshop were most effective. Numerous individuals offered multiple comments. Fourteen individuals offered no response. Responses could be grouped into two major categories: those related directly to the PBL process and those related to interpersonal workshop dimensions (Table 4). Effective workshop elements related to the PBL process and their frequencies were as follows: simulation of a case (n=55), researching and producing learning materials (n=14), applying learning and subsequent identification of follow-up research topics (n=8), and self- and peer assessment (n=10). Though specific comments varied considerably, satisfaction with simulating a PBL learning experience was expressed clearly by a majority. One participant described the experience with the workshop as having “‘Demystified’ the PBL pedagogy. [It was] . . . more structured than anticipated. Doing an actual case with a group was extremely helpful.” Those participants who focused on their experience with specific elements of the PBL process described their satisfaction with having learned “to find resources and how to stay focused on the subject.” One participant who found the self- and peer assessment process effective expressed that it helped him or her to “confront my shortcomings and areas I need to improve.” Within the interpersonal dimension category, participants identified the group learning environment (n=20), the workshop leader and/or facilitator(s) (n=17), and the ability to work with peers (n=4) as the most effective elements of the workshop. As ex-
253
Survey for the Improvement of Faculty Development Workshop This survey is to collect information to help the Education and Learning Council’s FDME (Faculty Development, Mentoring, and Evaluation) Subcommittee improve its programs. Name (optional):
Date:
Professional Development Workshops Evaluation In the following questions, please circle the answer that best describes your experience in the Faculty Development Workshop. Today’s program has helped me…. 1=Strongly disagree, 2=Disagree, 3=Agree, 4=Strongly Agree 1. To increase my understanding of how facts are distinguished, ideas based on facts are generated, and appropriately defined learning issues are defined by learners going through a PBL case. 1 2 3 4 2. To increase my understanding of how students utilize resources to generate “Learning Needs” that are relevant to the case and their group’s ideas. 1 2 3 4 3. To increase my awareness of how students gauge their limit of understanding from a PBL case. 1 2 3 4 4. To increase my understanding of the importance of reviewing case outcomes and defining unresolved learning issues. 1 2 3 4 5. To increase my understanding of the importance of conducting self- and peer assessment. 1 2 3 4 6. To increase my understanding of the inquiry-based, student-centered learning approach utilized in the problem-based learning program. 1 2 3 4 Please describe what you felt were the most effective elements of the workshop(s) you attended today. Please describe what you felt were the least effective elements of the workshop(s) you attended today. In what ways do you think the workshop(s) could be improved to enhance your understanding of these topics? Thank you for your time and input! Figure 2. Post-workshop survey
254
Journal of Dental Education ■ Volume 71, Number 2
Table 3. Participant perceptions of PBL Process workshop effectiveness Number
Learning Objective
This workshop helped me to understand….
1 2 3 4 5 6
…how facts are distinguished, ideas based on facts are generated, and appropriately defined learning issues are generated by learners going through a PBL case. …how students utilize resources to generate “Learning Needs” that are relevant to the case and their group’s ideas. …how students gauge their limit of understanding from a PBL case. …the importance of reviewing case outcomes and defining unresolved learning issues. …the importance of conducting self- and peer review. …the inquiry-based, student-centered learning approach utilized in the problem-based learning program.
pressed by one participant, the small-group setting was effective because of “the comfort/ease between all the members of the group. [I] felt there was good teamwork and a very capable facilitator who was clear.” Though facilitation was not a focus of the workshop, numerous participants commented that one aspect of the workshop facilitator’s effectiveness related to role-modeling “appropriate” facilitator behavior. As one participant commented, “The facilitator was very helpful in showing by her actions what a facilitator does.” When queried about least effective elements of the workshop, considerably more varied responses were offered (Table 5). Forty-five participants offered comments, from which fifty-two separate items were tallied. Fifty-nine participants offered no comment. Four individuals remarked that all aspects of the workshop were effective. The remaining comments about the least effective workshop components could be grouped into roughly four categories: 1) self- and peer assessment, 2) researching learning topics, 3) time management, and 4) alignment between workshop goals and participant needs. The respondents’ most frequently expressed concerns related to researching learning topics were insufficient time allocation to conduct the research and inadequate resources being available. One participant explained, “Our ability to pursue and synthesize learning resources was limited by the time available. This is not meant to be a criticism as much as an observation of why this important part of the student experience was not realized to a higher degree.” Another pointed out, “There was too little time, too few materials available. Time frame of workshop does not allow [one] to understand a full case.”
February 2007 ■ Journal of Dental Education
N
Mean
SD
Rank
97
3.36
0.54
3
97 97
3.34 3.12
0.58 0.62
4 6
96 96
3.38 3.33
0.60 0.69
2 5
95
3.40
0.55
1
Table 4. Effective workshop elements Response
Frequency
PBL Process Simulating the case 55 Researching and producing learning need packets 14 Self- and peer assessment 10 Applying learning and identifying follow-up topics 8 Interpersonal Group learning environment 20 Workshop leader 17 Working with peers 4
Rank 1 4 5 6 2 3 7
Self- and peer assessment was the other workshop component most frequently mentioned as being ineffective. Here respondents noted that the self- and peer assessment was difficult to understand and challenging to do for the first time. Time management was identified as problematic for some participants who cited that various segments of the workshop had been allotted either too much or too little time. Misalignment between the workshop goals and individual participant interests was evidenced by a few disparate comments noting that the particular case utilized the exclusive focus on PBL Process at the expense of addressing facilitations or assessment skills and an audience with mixed PBL experience levels was not ideal. When participants were asked to offer suggestions on how to improve the PBL Process workshop, they offered a wide range of comments that could be arranged around three themes: 1) enhancing the exist-
255
haps, called for practicing PBL with additional cases, “going Topics Comment Frequency through actual cases assigned to students and doing what we Research-related did today to better prepare for Insufficient time to conduct research 11 our role as facilitator.” Another Insufficient references available 10 participant expressed an interSelf- and peer assessment 12 est in learning how students Alignment between workshop and participant needs 6 select resources and format Time management 12 learning packets by stating, “[I] would like a little more guidance on the resources. How ing workshop, 2) addressing additional PBL-related [they] should be presented. The format that is used content areas, and 3) reinforcing the place of the PBL to give information to the peers.” Other individuals Process workshop in context of the PBL core skills commented that more direction from the workshop training program (Table 6). Suggestions were offered leader (n=1), reinforcement of learning objectives on by forty-five different participants, with some parconclusion of the workshop (n=1), and decreased use ticipants offering multiple suggestions. Some merely of “educational jargon” (n=1) would serve to improve affirmed that, for its purpose, the workshop “seems the workshop. to work, so stay with it.” Sixty-three participants ofParticipant comments viewed as addressing fered no feedback. additional PBL-related content areas included introSuggestions for enhancing the methods and ducing role-play of facilitation with feedback (n=2) content of the PBL Process workshop included and discussion of common student problems and replacing the PBL case used for the exercise (n=7); solutions. Five participants requested that additional introducing observation of students doing PBL, eiworkshops on PBL be offered. ther live or in a video, along with discussions (n=5); The final theme that drew a number of particiand providing additional background literature or pant comments related to the fit of the PBL Process pre-reading materials on PBL (n=3). Individual sugworkshop in the overall scheme of facilitator traingestions, representing personal learning interests pering. The comment most frequently noted among participants was the suggestion to enforce sequencing of the PBL core skills workshop proTable 6. Suggestions for workshop improvement gram (n=6), so that participants Topic Comment Frequency would take the PBL Process Changes to existing workshop content and methods workshop first followed by • modify case used for exercise 7 the Facilitation and Assess • demonstrate PBL with students and discussion 5 ment workshops respectively. • more background literature or pre-reading 3 Though participants had been • increase direction from workshop leader 1 advised to attend the PBL core • reinforce learning outcomes for workshop at end 1 skills workshops in a specific • decrease educational jargon 1 sequence, some had opted to • resource selection and learning need packet construction process 1 attend in a different order, • practice doing PBL with additional cases 1 commenting later that, “I took Additional workshop content and methods it out of sequence and [am hav • offer additional workshops 5 ing] difficulty putting it in per • add role-play of facilitation with feedback 2 spective.” Further respondents • discuss problems associated with doing PBL and remedies 2 suggested that the PBL core skills program be linked to an PBL core skills and PBL Process workshop in context observation of a real PBL case • enforce sequencing of core skills workshops 6 (n=3) and completed prior to • link workshops to observation of a real case 3 becoming a facilitator (n=3). • ensure core skills workshops taken before facilitating 3 Table 5. Least effective workshop elements
256
Journal of Dental Education ■ Volume 71, Number 2
Discussion A faculty development workshop based on experiencing a full iteration of the PBL process was designed and provided to the faculty of USCSD as foundation training for facilitators. Among the provisions of this workshop were an increased emphasis on self-assessment skills and a contextualized view of the learning theory underpinning the design of PBL, whereby relevant principles were linked to specific stages of the PBL process as the experience unfolded for the participants. The strategy of explicitly describing to participants, for example, how and why prior knowledge should be drawn out and built upon as the participants were actually experiencing that principle was used to align learning theory with practice. The approach is, likewise, consistent with adult learning concepts advocated by organizers of faculty development programs in both dentistry and medicine.7,15,20 When reviewing the perceptions of faculty development organizers and participants from the PBL Process workshop, three major themes emerged that are applicable to future faculty development efforts. These themes relate to 1) the general success of employing an extended role-play activity to understand the structure and rationale of PBL, including its various reflective components; 2) the notion that particular aspects of the PBL process are more difficult to access than others and therefore may require additional or modified training approaches; and 3) that dissatisfaction with particular approaches used within the workshop may be related to restrictions on autonomy placed on the participants as learners. While many organizers of PBL training advocate and indeed incorporate experiential methods into their programs, faculty development devoted specifically to the learning process of PBL typically focuses on elements that can be completed in a shorter time interval with a greater relative emphasis being put on discussing comparisons between PBL and traditional programs and roles and responsibilities of the facilitator.6,8,13,14 Such workshops have typically stopped short of experiencing a full PBL learning cycle.5,8 Role-playing a full iteration of the PBL process allows faculty members to experience the phases of the PBL process with which they have noted concerns—namely, how students build information literacy skills, acquire and apply new information, and correct previously misunderstood concepts. A key component of PBL—regular reflection on per-
February 2007 ■ Journal of Dental Education
sonal and group performance—has been lacking in descriptions of PBL facilitator training. It is perhaps not surprising that working facilitators interviewed about their concept of PBL tend to neglect this aspect of PBL program design almost entirely.9 These latter stages of the PBL process link closely with the principles for learning associated with the design of PBL—namely, constructive, cooperative, contextualized, and self-directed approaches to learning.4 Thus we feel the inclusion of the reflective elements of PBL in the training program seems justified and represents a general improvement in faculty development for PBL in health professions education. A key measure of success of the approaches taken in the PBL Process workshop is evidenced by the positive perceptions of the participants. When surveyed as to whether they felt the learning objectives for the workshop had been met, the majority of respondents agreed. The methodological vehicle utilized to access the content for these objectives—that is, the use of the extended role-play—was highlighted by the participants as being the most effective aspect of the workshop. In accordance with adult learning principles16 and findings from faculty development literature,14,21 the majority of our workshop participants expressed satisfaction with the experiential approaches utilized in the workshop, from active simulation of the PBL Process to researching learning topics to the active discussion of how research materials are applied to patient problems. High levels of participant satisfaction were also associated with working in a group environment, reinforcing the general notion that cooperative learning environments are associated with learner satisfaction.22 Specifically, it lends further support to the finding that faculty development efforts are more greatly appreciated by participants when the opportunity to work with peers is incorporated.21,23,24 While participant perceptions of faculty development workshops are valuable, they represent only one contributor to determining overall program success. Future evaluation of the USCSD faculty development program will attempt an analysis of whether participation in faculty development workshops is related to student perceptions of facilitator effectiveness and student achievement.25 Though overall satisfaction with the PBL Process workshop was high, the areas associated with lower relative scores represent potential areas for future improvement. For example, the PBL stages associated with assessment skills, both in how students’ judge their level of understanding of
257
learning materials and in the processes of self- and peer assessment, were met with a greater degree of dissatisfaction by the workshop participants than the other components of the workshop. This was noted both in the relative responses to the learning objectives statements and in the open response data provided by participants. The relatively higher levels of dissatisfaction expressed here may indicate that a greater emphasis or different methods should be employed when addressing processes related to selfand peer assessment. Identification of the reasons for the relatively higher dissatisfaction in this area may assist in determining more appropriate methods for approaching teaching and learning of this important professional skill. The learning objective with the lowest average agreement levels was linked to objective three, “increasing awareness of how students gauge their limit of understanding from a PBL case.” Previous studies have shown that faculty who had recently received PBL facilitator training struggled to understand the appropriate depth and breadth of knowledge expected of students.8 While this finding may be more closely related to an inability to link the level of student knowledge to desired performance levels expected from the case curriculum, it may also point to underlying difficulties with the nature of qualitative assessment of students in PBL in particular. Assessment relies on the assessor’s ability to view student performance in various conditions, for sufficiently lengthy periods of time, and utilizing agreed upon and understood criteria by faculty, typically gained via training.26 Such qualitative assessment of student performance in PBL—and, in our situation, the faculty member’s initial ability to assess the student’s level of understanding—may be influenced by the difficulty inherent in separating learning processes from unfamiliar content being discussed in the case. Additional attention may need to be committed to this aspect of the PBL process within the workshop or addressed with different methods whereby relevant examples can be carefully analyzed. Alternatively, it may be that the ability to observe and thereby encourage this behavior in students is only possible after one has spent a significant amount of time observing students working in PBL. Though most participants agreed that the learning objective related specifically to the process of self- and peer assessment (learning objective 5) was met, mixed reviews were noted in the open-response data, with roughly equal numbers of participants
258
finding self- and peer assessment of learning skills effective and ineffective. Those who found the activity ineffective specified that self- and peer assessment was difficult to understand and to conduct for the first time. Self- and peer assessment of performance in the tutorial group has been identified as a difficult skill to perform based on numerous factors, not least on the fact that individuals report a reluctance to assess their peers in a public venue, especially when they feel they have not had sufficient time to observe individual performance.27 This issue may be further complicated by dental faculty attitudes toward the scope of dental education, where some may view with skepticism teaching and learning approaches that go beyond the primary goal of establishing procedural competency in dental students.28,29 Again, more time or diversified methods may need to be dedicated to the subject area of self- and peer assessment. Despite indications that the self- and peer assessment activity will require revision, we feel its incorporation into the workshop has, at the least, raised awareness of its integral place in the PBL process. Other areas from the workshop that could be conceived as needing improvement were related to dissatisfaction with the time allotted for conducting research and the availability of sufficient resources to assemble learning materials for the exercise. In planning the workshop, it had been decided that concessions would be made to this aspect of the self-directed learning component of PBL. In attempting to accommodate the program into a single day, our control of resource availability and truncation of study time may have placed uncomfortable restrictions on the participants as individual adult learners.19 Restricting the participants’ ability to direct their own learning process may have served as a source of frustration for some. These findings, as well as anecdotal comments from faculty about committing a full day to workshop attendance, have led to the development of a blended online and faceto-face workshop on PBL Process at USCSD. Specific content areas have been augmented and moved into the online environment while still maintaining the role-play experience of going through an entire learning cycle via PBL. The face-to-face meetings have been structured so as to allow for two shorter meetings, between which participants may spend as much time as they wish selecting resources and assembling the materials for the remainder of the exercise.
Journal of Dental Education ■ Volume 71, Number 2
Conclusion This article reports a foundation-building faculty development workshop to introduce the instructional methodology of PBL. An extended role-play method was adopted for the workshop in an effort to provide participants with greater exposure to the PBL learning cycle and, in turn, to address stated areas of faculty concern with the PBL methodology. Overall, participants were satisfied with the experiential, collaborative methods and content of the workshop. Participant satisfaction with the methodological choices is consistent with previous reports in the faculty development literature for medical education and demonstrate that adult learning theory principles are relevant to faculty development in dental education. Participant and program organizer input has been utilized to make specific improvements to the workshop, both enhancing weaker content areas and modifying workshop methods to increase alignment with adult learning principles.
Acknowledgments
The authors would like to thank the subcommittee members, past and present, of the Curriculum Subcommittee on Faculty Development, Mentoring, and Evaluation for their efforts in creating, organizing, and implementing the PBL core skills program.
REFERENCES 1. Kassebaum DK, Hendricson WD, Taft T, Haden NK. The dental curriculum at North American dental institutions in 2002-03: a survey of current structure, recent innovations, and planned changes. J Dent Educ 2004;68(9):914-31. 2. Barrows HS. The essentials of problem-based learning. J Dent Educ 1998;62(9):630-3. 3. Wood DF. Problem-based learning. BMJ 2003;326(7384): 328-30. 4. Dolmans DH, De Grave W, Wolfhagen IH, van der Vleuten CP. Problem-based learning: future challenges for educational practice and research. Med Educ 2005;39(7): 732-41. 5. Irby DM. Models of faculty development for problembased learning. Adv Health Sci Educ Theory Pract 1996;1:69-81. 6. Hitchcock MA, Mylona ZE. Teaching faculty to conduct problem-based learning. Teach Learn Med 2000;12(1): 52-7. 7. Farmer EA. Faculty development for problem-based learning. Eur J Dent Educ 2004;8:59-66. 8. Olmesdahl PJ, Manning DM. Impact of training on PBL facilitators. Med Educ 1999;33(10):753-5.
February 2007 ■ Journal of Dental Education
9. Maudsley G. Making sense of trying not to teach: an interview study of tutors’ ideas of problem-based learning. Acad Med 2002;77(2):162-72. 10. Kaufman DM, Holmes DB. Tutoring in problem-based learning: perceptions of teachers and students. Med Educ 1996;30(5):371-7. 11. Vernon DT, Hosokawa MC. Faculty attitudes and opinions about problem-based learning. Acad Med 1996;71(11): 1233-8. 12. Vernon DT. Attitudes and opinions of faculty tutors about problem-based learning. Acad Med 1995;70(3):216-23. 13. Vanhanen H, Pitkala K, Puolakkainen P, Strandberg T, Lonka K, et al. The PBL tutorial laboratory: a method for training medical teachers. Med Teach 2001;23:99-101. 14. Des Marchais JE, Chaput M. A comprehensive continuous preclinical PBL tutor training system. Teach Learn Med 1997; 9(1):66-72. 15. Masella RS, Thompson TJ. Dental education and evidencebased educational best practices: bridging the great divide. J Dent Educ 2004;68(12):1266-71. 16. Dalrymple KR, Wuenschell C, Shuler CF. Development and implementation of a comprehensive faculty development program in PBL core skills. J Dent Educ 2006;70(9):948-55. 17. Maudsley G. Do we all mean the same thing by “problembased learning”? A review of the concepts and a formulation of the ground rules. Acad Med 1999;74(2):178-85. 18. A pilot systematic review and meta-analysis on the effectiveness of problem-based learning. Newcastle upon Tyne, UK: Higher Education Academy Subject Centre for Medicine, Dentistry, and Veterinary Medicine, 2003. 19. Knowles MS, Holton EF, Swanson RA. The adult learner. 5th ed. Woburn, MA: Butterworth-Heinemann, 1998. 20. Pololi L, Clay MC, Lipkin M Jr., Hewson M, Kaplan C, Frankel RM. Reflections on integrating theories of adult education into a medical school faculty development course. Med Teach 2001;23(3):276-83. 21. Cole KA, Barker LR, Kolodner K, Williamson P, Wright SM, Kern DE. Faculty development in teaching skills: an intensive longitudinal model. Acad Med 2004;79(5): 469-80. 22. Johnson RT, Johnson DW. An overview of cooperative learning. In: Thousand J, Villa A, Nevin A, eds. Creativity and collaborative learning. Baltimore, MD: Brookes Press, 1994. 23. Wilkerson L, Irby DM. Strategies for improving teaching practices: a comprehensive approach to faculty development. Acad Med 1998;73:387-96. 24. Bland CJ, Seaquist E, Pacala JT, Center B, Finstad D. One school’s strategy to assess and improve the vitality of its faculty. Acad Med 2002;77(5):368-76. 25. Wilkes M, Bligh J. Evaluating educational interventions. BMJ 1999;318(7193):1269-72. 26. Shumway JM, Harden RM. AMEE guide no. 25: the assessment of learning outcomes for the competent and reflective physician. Med Teach 2003;25(6):569-84. 27. Eva KW. Assessing tutorial-based assessment. Adv Health Sci Educ Theory Pract 2001;6(3):243-57. 28. Masella RS. Confronting shibboleths of dental education. J Dent Educ 2005;69(10):1089-94. 29. Hendricson WD, Cohen PA. Oral health care in the 21st century: implications for dental and medical education. Acad Med 2001;76(12):1181-206.
259
