Incorporating Problem-Based Learning Concepts ... - Semantic Scholar

Incorporating Problem-Based Learning Concepts into a Lecture-based Pharmacoeconomics Course Ya-Chen Tina Shin and Teresa L. Kauf School of Pharmacy, University of North Carolina, Chapel Hill NC 27599-7360

Andrea K. Biddle School of Public Health, University of North Carolina, Chapel Hill NC 27599-7400

Kit N. Simpson College of Pharmacy, Medical University of South Carolina, Charleston SC 29425 In recent years, pharmacoeconomics has increased its presence in the curricula of professional Doctor of Pharmacy programs as well as traditional post-graduate programs in schools of pharmacy. Due to the unique and varied nature of the course material, students may have difficulties linking important concepts and methods to their own projects under a lecture-based, didactic teaching approach. We felt that this problem could not be solved simply be reorganizing the lecture material within the didactic model. Instead, we developed an innovative teaching method that applies the concepts of problem-based learning (PBL) within the context of the standard lecture-based learning format in a graduate level pharmacoeconomics course. In this article, we introduce this novel approach and share our teaching experiences. Results of our course evaluations show that more than 84 percent of the students found the new approach to be an interesting, useful, and stimulating way to learn pharmacoeconomics.

INTRODUCTION The recent increase in the volume of published literature on pharmacoeconomics, both methodological and applied, speaks to the increasing importance of this subject in academic and professional environments. As pharmacy schools continue to adopt the Doctor of Pharmacy degree as their entry-level program, students are beginning to explore pharmacoeconomics as one of many alternative career paths. Graduate students in traditional masters and doctoral programs also are interested in developing pharmacoeconomic skills. In response to the demands of students and the industry, pharmacy schools and other health-related programs have begun to formally incorporate pharmacoeconomics concepts and courses into their curricula. Pharmacoeconomics, as part of a broader field sometimes referred to as technology assessment, is a difficult subject to master for several reasons. First, continuous advances in the theory and methodology of pharmacoeconomics require students to master an ever-increasing base of knowledge in order to become competent in the field. Second, many students, especially those without any previous knowledge of economics, have difficulty applying these concepts to specific problems. Lastly, the broad scope of pharmacoeconomics necessitates the ability to think both in very concrete and abstract terms. These problems were reflected in our own efforts to coordinate a course in pharmacoeconomics and policy analysis at the graduate (MS and PhD) level. Through several offerings of the course, the lecture material was synthesized and readings were narrowed to include only the most critical elements of pharmacoeconomic analysis. Our philosophy for the course was that it 152

was more important for students to master a few, basic concepts rather than give them a cursory knowledge of a broader set of topics. However, even with this “paring-down” of the material, we remained frustrated by the students’ lack of ability to apply the information taught in classroom to individual research problems. Of course, the students experienced this same frustration. We began to realize that the critical thinking skills mentioned above are particularly difficult to develop in the context of a didactic, lecture-based learning (LBL) approach. Instead, we began to investigate problem-based learning (PBL) approaches to teaching and applied several of those concepts to the course while maintaining a lecture-based foundation. In this article, we discuss how we adapted PBL to a graduatelevel course in pharmacoeconomics and policy analysis. The next section reviews the key elements of PBL, and we then explain our rationale for incorporating some of these elements into the course. The structure and outline of the course, including a summary of learning objectives and assignments, and the use of student peer evaluations are provided as an illustration. The last section discusses students’ evaluations of the course and presents concluding thoughts. PROBLEM-BASED LEARNING (PBL) Problem-based learning has been defined as “the learning that results from the process of working toward the understanding or resolution of a problem. The problem is encountered first in the learning process(1).” Though there are a variety of PBL teaching methods, in general, the key elements of a PBL curriculum include the following(2,3):

American Journal of Pharmaceutical Education Vol. 63, Summer 1999

• • • • • • • • •

“cases” are assigned to students at the beginning of the course; students are active participants in learning, not passive recipient of information; the learning process is lead by students, with faculty as facilitators; students are organized into small groups, with five to six persons in each groups; problems (or “cases”) are the focus and motivation for learning; promotion of problem-solving skills; new information is acquired through self-directed learning; students develop and enhance their interpersonal skills; and students are encouraged to become life-long, independent learners.

In essence, PBL focuses on “how to learn”, rather than “what to learn.” The PBL teaching approach is supported by cognitive psychology, which states that competence is fostered not primarily by teaching to deliver knowledge, but through teaching to engender specific kinds of cognitive activity(4). PBL is a teaching approach reflecting these conditions of human cognition and learning. The PBL approach, in contrast to the traditional lecturebased learning approach, has several advantages(5-8). The education literature has documented these as: (i) PBL provides an interesting, stimulating and enjoyable learning environment in which students take responsibility for their own learning; (ii) PBL provides a context for interdisciplinary learning; and (iii) PBL enables students to develop the skills necessary for selfdirected, life-long learning(2). In addition, PBL encourages student leadership and participation. PBL students were found to have more interactions with faculty and performed as well or better than LBL students on broad examinations(3). Moreover, medical students receiving instruction under PBL curricula have shown to perform better in the retention of knowledge, general problem-solving skills, ability to integrate basic science concepts into clinical problems, self-directed learning skills, and intrinsic interest in subject manner(9). PBL has been used in medical education for several decades. McMaster University incorporated PBL as part of its medical school curriculum in 1974(10). The University of New Mexico became the first medical school in the United States to include PBL in its curriculum in 1979. By 1991, more than 100 medical schools have employed some form of PBL as part of medical education(3). The application of PBL is not limited to medical schools, however. PBL applications can be found in a number of professional curricula, including pharmacy, dentistry, nursing, occupational therapy, engineering, and business(2,11-15). PBL IN A PHARMACOECONOMICS COURSE We considered revising the course syllabus for this course as a challenge. Having taught the class several times in the past, we were familiar with its strong points and wanted to overcome its pitfalls and felt that PBL could help us do that. However, we did not want to convert the class to the “traditional” PBL model. Due to the technical nature of many of the concepts that must be covered in the course, we felt that it was important to provide basic and/or especially difficult material in a lecture format. For example, the economic theory behind costing often is confusing to students, and the presentation of this material as a didactic lecture is relatively straightforward. It would be impractical, however, to attempt to cover the wide variety of practical applications of the theory as a lecture or series of lec

tures in a single semester course. We wanted to provide students with an alternative arena for discussing key concepts and learning to apply them under many different scenarios. The case-driven, student-centered learning process was the major PBL feature included in the course. However, unlike the standard problem-based learning method in which all students are assigned to the same cases at the beginning of the course and are required to use these cases for discussions throughout the semester, we asked each student to identify his/her individual research topic by the end of the second week. Each student’s selected research topic served as his/her case for the semester. That is, instead of having the same case for all students to discuss in the group sessions and having them review several cases in a semester, each of our student groups had five to six “cases” for the entire semester, one for each group member. Following the recommendations found in the literature, each group was composed of not more than six students. For this class, we had 26 students in six groups. At our institution, this course is taken by students in programs that span the spectrum of health professions—pharmacy, nursing, medicine, and public health. Within these programs are students with a wide variety of professional and academic experience, goals, and perspectives. We wanted to make sure that each group contained as many students with different backgrounds as possible. On the first day of class, we asked students to identify their academic programs and any coursework or other experiences that may be relevant to the course. We then placed the students into groups, making sure to distribute student types as evenly as possible. For example, one group contained a physician, a pharmacist, a public health student, and a student with a relatively strong background in economics and statistics. When we have asked students to work in groups in the past, students with similar academic backgrounds tended to group together, thus limiting the breadth of their discussions. We hope that by carefully assigning students to groups that the level of discussion and the amount of learning engendered will be maximized. Several class meetings were dedicated as PBL sessions, one following each section or topic. For example, following the lectures on “Decision Analysis” and “Demonstration of Decision Analysis Software”, each student was asked to structure his/her research topic in a decision tree framework and to share the decision tree with group members in a PBL session. The group then gave feedback on each member’s decision tree and discussed other possible tree structures for members’ research topics. The faculty and teaching assistant observed and facilitated, but did not lead, the group discussions. Because each student was required to submit a manuscript at the end of the course, he/she was motivated to apply a variety of economic evaluation methods, such as cost-effectiveness, costbenefit, or cost-utility analysis, to the research topic and investigate which method was more appropriate for the particular research question. In addition, since each student was required to submit a draft of his/her research topic proposal early in the semester, each student was able to relate every lecture to his/her “case” throughout the semester, and each PBL session helped the student prepare necessary parts of the manuscript. The following example demonstrates the differences between the standard PBL approach and the teaching approach used in our course. A PBL-based pharmacoeconomics course might assign “EPO versus transfusion in the treatment of anemic end-stage renal disease patients” as the case for all students to use in discussing cost-effectiveness analysis; “medical versus surgical treatment of workers with lower back pain” could be used as the case for discussing the cost-benefit

American Journal of Pharmaceutical Education Vol. 63, Summer 1999

153

approach. In our class, if student A chooses “EPO versus transfusion in the treatment of anemia” as his research topic, then this will be the case he discusses with his group members in every discussion session. Therefore, when discussing “costs”, student A would share with his group members what would be the “costs” associated with EPO versus blood transfusion. As the topic moves on to “quality of life”, student A then would examine the quality of life issues associated with EPO treatment versus transfusion. This particular case would be used repeatedly by this student throughout the semester. Likewise, student B would use “medical versus surgical treatment of workers with lower back pain” as her case for each PBL session throughout the semester. COURSE STRUCTURE AND OUTLINE The title of our pharmacoeconomics course is, “Economic Evaluation of Health Care Technology.” We define health care technology broadly as: “programs, processes, procedures, drugs or other interventions aimed at improving health, preventing disease, alleviating symptoms or postponing death of human populations.” The course is based on the premise that we currently know how to treat or prevent more disease than we are willing to pay for. Given the available technologies/policies/treatments, the allocation of scarce resources to the most efficient competing technologies and equity in access to cost-effective technologies are essential health policy issues that should be informed, but not dictated by, qualitative and quantitative policy analyses. The course is an investigation of the theory, methods, and application of technology assessment to health care and covers basic methods used to identify policy issues, structure a technology assessment study, measure and summarize health outcomes, estimate their value to patients or the public, identify the resources used, estimate cost of health resources, and construct and test mathematical models to make population and/or temporal predictions of outcomes and costs from limited data. Applications are drawn from a variety of health care settings and include the evaluation of Pharmaceuticals. Listed in Appendix A are the learning objectives for this course; upon successful completion of this course, students should be able to perform each item listed. The items are grouped under the three broad categories of policy analysis, techniques for economic evaluation of health care, and reporting results of economic analyses. The sequencing of lectures and problem-based learning sessions in this course corresponds closely to the objectives listed under techniques for economic evaluation. Throughout the course, we examined the policy analysis objective. Reporting of results in terms of the preparation of a poster presentation and a final manuscript were discussed near the completion of the course. In addition, we used periodic assignments (described below) to illustrate accepted reporting techniques for individual sections of an economic evaluation. To provide an interactive and stimulating learning environment and to enhance students’ ability to link economic evaluation methods with their research questions, seven problembased learning sessions complemented the lecture material. Figure 1 illustrates the organizational structure of the course. The rectangular boxes summarize the pharmacoeconomics topics to be included in the lecture format, with the sequence of the lectures listed from top to the bottom. Note that each box may contain more than one lecture on the specified topic, depending on the complexity of that topic. For example, two one-and-one-half hour lectures were devoted to the topic of 154

Fig. 1. Organizational flowchart of the course

“Health-Related Quality of Life.” Included in the oval-shaped boxes are the objectives of the PBL session corresponding to each topic. As shown in Figure 1, each of these sessions is designed to help students connect topics covered in the previous lectures to their own research projects. Seven assignments were given in this course; one following each PBL session. For each assignment, we provided students with specific instructions for preparing a periodic report (or update) of their research topics, corresponding to the discussion in the previous PBL session. Appendix B provides a brief description of each PBL session and the content included in the subsequent assignment. The purpose of the research updates is two-fold. First, they serve as a mechanism for students to prepare each section of their final papers throughout the semester. Second, they allow the faculty to supervise the progress of each student. By reviewing these reports, faculty were able to identify any concepts which remained unclear to students and which student (or students) had difficulties completing his/her project(s). The faculty were able to clarify these concepts in class and bring the students up to speed in the course. Additionally, we were able to identify exemplary projects and to further nurture and encourage the student’s performance. The final products for this course are a poster presentation and a manuscript prepared in a “ready-to-submit” format. Near the end of the semester, each student was required to summarize his/her project in a poster and present the poster to the course faculty, other students, and invited guests. The poster presentation provided an opportunity for each student to get feedback from a broader audience. That is, in addition to feedback given by course faculty and his/her PBL group members throughout the semester, each student also received comments

American Journal of Pharmaceutical Education Vol. 63, Summer 1999

from students in other PBL groups as well as from faculty who had expertise in a particular clinical or pharmacoeconomic area, but who did not participate in teaching the course. Students also were asked to identify target journals for submission of their research papers, then prepare their manuscripts following the “Instructions to Authors” for their specific journals. Grading of the final paper (manuscript) was based on the quality of research and reporting, taking into account whether the paper complied with journal submission instructions. PEER EVALUATIONS Two types of student peer evaluations were used in this course. One was the “PBL session evaluation” and the other was the “summary peer evaluation.” The PBL session evaluation was administered at the end of each PBL session and served as an instrument for continuous quality improvement for group interactions. The purpose of this evaluation was to give each PBL group an opportunity to talk about its group dynamics, identify any problems that might exist, and make any adjustments necessary to insure a successful discussion the next time. Each group was asked to answer the same set of questions. These were reviewed by the faculty and returned to the groups so they could track their development and progress as a group over time. Questions addressed at each PBL session evaluation included: (i) did all members of the group have an equal opportunity to present their cases; (ii) was an equal opportunity provided for all members to provide feedback; (iii) was feedback given appropriately; (iv) were cases presented in manner that facilitated feedback; and (v) should faculty and/or the teaching assistant provide more help or less help. Groups also were asked to identify any problems or issues that they were not able to resolve during the session, and these were addressed by faculty at the beginning of the next class. These evaluations also allowed the faculty to be aware of problems that may inhibit the interactions of certain groups in the PBL sessions. The summary peer evaluations were administered at the end of this course. In a traditional PBL course, group members would work on a common research project and each member would be expected to make a contribution to that project. Thus, the work of the students is very interdependent; in this context, peer evaluations are typically used to determine the relative effort that each student made toward the group project. In our case, group members did not share the workload for other members’ projects and took sole responsibility for their own projects. The degree of interdependence among group members therefore was much less. However, we still wanted to use an instrument to assess student participation in the PBL sessions in a systematic way. In consultation with the Center for Teaching and Learning at the University of North Carolina, we developed a summary peer evaluation form for use in this class. Each student was asked to complete an evaluation of his/her fellow group members, but the evaluation focused on the adequacy of students’ contributions to the group discussions overall as opposed to a single project. Students were asked to make their judgments based on a scale of 1 to 10, with 10 being excellent. Items rated include preparedness, quality of feedback, and various items that affect the functioning of the group, such as awareness of and respect for others’ opinions and ideas. These rankings were averaged and added to the student’s total grade points. The group participation component of the course was worth ten points out of a total of 100 points. COURSE EVALUATION A course evaluation form was distributed at the end of the

semester to evaluate this new approach to the course. In designing the survey, several PBL-related survey questionnaires from the literature were referenced and revised to reflect the course(16,17). A Likert scale was used to categorize responses. The evaluation form contained thirty-four questions, followed by several open-ended questions. To evaluate different aspects of this course, the evaluation instrument was divided into six components: comparison of the course to lecture-based learning, small group process, learning resources, tutor effectiveness, academic support, and an overall evaluation. The first component included four questions about students’ general attitudes toward the teaching approach used in the course. The second component, “Small Group Process”, was designed to understand how students felt about the small group discussions and possible obstacles hindering students’ learning. The “Learning Resources” component focused on the functions and timing of various learning media, including the individual project, homework assignments, and reading materials. Questions in the “Tutor Effectiveness” component were used to examine the role of course faculty and the teaching assistant in their role as PBL facilitators throughout the semester. “Academic Support” inquired about factors enhancing students’ learning and understanding of pharmacoeconomics. The overall evaluation measured students’ overall satisfaction with various aspects of the course. Table I summarizes students’ responses to the 34-item questionnaire. Out of the 26 students enrolled in this course, 25 students completed the course evaluation survey. In general, the majority of students (more than 80 percent) found this new teaching approach to be an interesting, useful, and helpful way to learn both the theory and applications of pharmacoeconomics. Compared to a purely lecture-based approach, approximately 25 percent of the students showed some reservations about this new teaching approach. Only 12 percent of the students indicated that they would have preferred a purely problem-based approach. These results suggest that most students would not be comfortable with a pharmacoeconomics curriculum taught exclusively under a PBL approach. However, our hybrid approach appeared to be well-accepted by most students. In terms of the small group process, more than 90 percent of the students felt that they were able to contribute to their small group discussions, and 72 percent felt they were able to learn from the other group members. On the other hand, eight out of 25 students felt that their learning was hindered by their own lack of preparedness, while two out of 25 attributed to the lack of preparedness of other group members. Nevertheless, 84 percent of the students felt they could identify gaps in their knowledge base and address them as learning issues that throughout the semester. These results indicate that students seemed to have a strong sense of responsibility as to their roles in the small groups. 76 percent of the students either agreed or strongly agreed that the small group discussions had increased their understanding of pharmacoeconomics and enhanced their ability to complete their projects. Moreover, more than 70 percent of the students felt that the length of each PBL session (75 minutes) was sufficient to accomplish the specified goals of each session. Also, only one out of 25 students indicated that insufficient guidance was provided for the PBL sessions. Overall, 84 percent of the students thought their small groups worked effectively. In general, students were clear about their roles and responsibilities in their groups and were positive about their small group experiences. Students were asked to evaluate three learning resources in this class: the individual project, homework assignments, and

American Journal of Pharmaceutical Education Vol. 63, Summer 1999

155

Table I. Course evaluation Comparison with Lecture-based Teaching I found this new teaching approach to be an interesting and useful method to learn the theory of Pharmacoeconomics. I found this new teaching approach helpful in understanding different applications of Pharmacoeconomics concepts. I probably learned more with this new approach than if the course was purely lecture-based. I probably learned more with this new approach than if the course was purely problem-based Small Group Process I was able to learn from the other group members during the PBL sessions. I was able to contribute to the small group discussions. Group discussion increased my understanding of the course material. Group discussion increased my ability to apply course theory and methods to my project. The length of the PBL sessions was sufficient to accomplish the goals of each session. The lack of preparedness on the part of other group members hindered my learning. My own lack of preparedness hindered my learning. Sufficient guidance on students’ roles and responsibilities under this new approach was provided. Overall, my group worked effectively. Learning Resources - Throughout the semester, I found that... Working on my own project increased my understanding of the theory underlying pharmacoeconomics. Working on my own project increased my ability to apply pharmacoeconomic concepts and methods. Choosing my own research topic early in the semester enhanced my learning. The homework expanded my understanding of course material and methods beyondthe particular requirement of my project. The homework assignments enhanced my ability to develop and complete my project. I could identify gaps in my knowledge base and address them as learning issues. Reading the articles and textbooks increased my understanding of the course materials. Tutor Effectiveness The faculty and teaching assistant helped me integrate the course material into my project. The faculty and teaching assistant encouraged small group learning through questions, challenges, and critiques. The faculty and teaching assistant promoted a comfortable group environment. Overall, the faculty and teaching assistant were effective small group facilitators. Academic Support - In this course, I found that I increased my understanding of pharmacoeconomics by ... Working with faculty during office hours. Working with the teaching assistant during office hours. Attending computer laboratory sessions for software demonstration. Working with my small Group. Overall Evaluation The ability to get personal help in the course when needed wasb The overall quality of the course syllabus wasb The clarity and relationship of materials presented in the course to the stated goals and objectives wasb The integration of the subject matter in this course (logical sequencing of material, “flow” from session to session) wasb The extent to which this course has increased my knowledge and/or competence in pharmacoeconomics wasc I would rate the overall quality of this course asb

SDa

D

N

S

A

Mean (SE)

0

2

1

14

8

4.12(0.17)

0

2

3

12

8

4.04(0.18)

1

2

4

8

10

3.96 (0.23)

1

1

1

11

11

4.20 (0.20)

2 1 1

11 15 12

1 0 1

4 1 4

7 8 7

3.76 (0.23) 4.20(0.14) 3.80 (0.23)

2

2

2

10

9

3.88 (0.25)

0

2

5

13

5

3.84(0.17)

6 6

13 9

4 2

2 8

0

0 0

1

4 3

14 12

6

0

2

2

5

16

4.40(0.19)

0 3

1

2 6

6 6

16 8

4.48(0.16) 3.56 (0.27)

0

2

5

9

9

4.00(0.19)

1 0

2

3 3

11 16

8

1

1

4

12

7

3.92 (0.20)

1

0

3

11

10

4.16(0.19)

1 0 2

0

5 1 2

8 11 8

1

1 0

12 13

4.12(0.20) 4.36().15) 4.20 (0.23)

1 2 2 3

3 1 1 1

2 4 2 4

10 9 10 9

9 9 10 8

3.92 (0.23) 3.88 (0.24) 4.00 (0.24) 3.72 (0.26)

0 0

0 0

3 3

11 11

11 11

4.32(0.14) 4.32(0.14)

1

2

1

0

2.08(0.17) 2.48 (0.24)

9

4.00(0.15) 4.16(0.16)

5

3.92 (0.22) 4.00(0.14)

0

1

2

13

9

4.20(0.15)

1

0

5

12

7

3.96(0.19)

0 0

0 1

4 3

8 9

13 12

4.36(0.15) 4.28(0.17)

a

SD (strongly disagree), D (disagree), N (neither disagree nor agree), A (agree), SA (strongly agree). Five categories used are: P (poor), F (fair), G (good), VG (very good), and E (excellent). c Five categories used in this question are: 1 (not at all), 2 (slight), 3 (modest), 4 (significant) and 5 (very significant). b

reading materials. More than 60 percent of the students strongly agreed that working on their own projects has increased their 156

understanding of both the theory and application of pharmacoeconomics. However, students did show concern about the

American Journal of Pharmaceutical Education Vol. 63, Summer 1999

early due date assigned to their project proposals. Eleven out of 25 students did not agree that choosing a topic early in the semester enhanced their learning. A slightly lower percentage (76 percent) of students agreed that the reading materials increased their understanding of the course. Among the learning resources, our survey results indicate that students ranked the individual project as the most useful learning tool, followed by homework assignments and reading materials. Tutor effectiveness was included in the questionnaire to examine whether ineffective tutoring may have hampered the implementation of this new teaching approach. Our survey indicated that the majority of students (approximately 80 percent) considered the faculty and teaching assistant, acting as PBL-session facilitators, to be helpful, encouraging, and effective. Among the four types of academic support—faculty office hours, teaching assistant office hours, computer laboratory sessions, and small groups—students’ responses suggest that the software demonstration in the computer sessions was considered most helpful, followed by faculty office hours, TA office hours, and, lastly, small groups. This finding indicates that although students generally felt their small groups functioned effectively, their learning still relied heavily on more traditional media. However, the differences in mean scores between each type of support were not significant. Responses from the overall evaluation show that students appeared to be pleased with the ability to get personal help, the overall quality of the syllabus, and the clarity and relationship of course materials and the stated objectives. One student felt that the integration of the course subjects was poor; however, the rest of the students ranked this item as good [5] very good [12], or excellent [7]. In addition, more than 50 percent of the students thought this course had increased their knowledge and/or competence in pharmacoeconomics very significantly and another 32 percent ranked this increase as significant. Overall, 84 percent of the students rated the quality of the course as very good or excellent. In total, the results from our evaluation of the course suggest that the majority of students enjoyed this new teaching approach and felt that their learning, understanding, and knowledge of pharmacoeconomics were significantly improved through participation in this course. DISCUSSION Instructors who have had experiences using PBL in their courses commonly perceive several negative aspects of PBL, including: (i) PBL is an inefficient learning method; (ii) there are difficulties in evaluating students; and (iii) the time demand placed on faculty members is large(18). Experts in PBL note that this teaching approach works best within a well-planned curriculum with clear learning objectives determined for each stage(19). In designing our course, we have attempted to avoid these problems by carefully planning and reviewing the course structure and the syllabus. We included the teaching assistant in the planning process as early as possible to receive her input and so that she would be familiar with the new course and her role and responsibilities. An equal share of course responsibilities (beyond the teaching assistant’s duties) has been allocated to each faculty member. As mentioned above, we also sought professional advice from the Center of Teaching and Learning at our campus. This was particularly helpful in the design of a variety of instruments to supervise individual progress of students, the PBL group dynamics, and the overall evaluation of the course, such as homework assignments, PBL session evaluations, and summary peer evaluations. The biggest difficulty in planning the course was organiz-

ing the process by which students would choose their research projects. In order to take advantage of the PBL components of the course, students needed to have their case ready very early in the semester, much earlier than has been the case in previous offerings of the course. For most doctoral students, this is fairly straightforward since they often want to incorporate some element of technology assessment into their dissertations. However, for others, it can be a confusing and intimidating process. Therefore, we wanted to make sure that we would be able to provide students feedback so that they could find a feasible research project idea. In order to accomplish this, we provided the students with guidelines for writing a research topic proposal on the first day of class. They were required to submit this brief summary (two to three pages) one week later. The faculty reviewed the proposals, made comments, and returned them to the students for the next class period that week. At that time, we also distributed instructions for updating and refining proposals; research topics had to be finalized and approved by the following week’s class. Faculty then worked with students who had problems developing a suitable research idea and provided them with a topic, if necessary. Thus, by the third week of classes and prior to the first PBL session, the students were ready to begin discussing their research projects with their groups. For this process to run smoothly, both the faculty and the students must be committed to the class and the concept of PBL. Our experience with the course, in comparison to lecturebased teaching, was that it required a much larger time commitment from both the faculty and teaching assistant. We agree with the literature that a well-planned curriculum with clear learning objectives is the key for this new teaching approach to work. We also feel that a moderate class size (not more than 30 students) with at least two course faculty and one teaching assistant is another important factor for this approach to succeed. Throughout the semester, we gradually deviated from the more traditional PBL approach by playing more active roles in the PBL sessions than we had originally intended. Students appeared uneasy with the level of autonomy given in these session discussions and requested more help and problem-solving from the faculty and the teaching assistant. Though tutors in a traditional PBL session should not actively participate in students’ discussions and not “give answers” to questions raised during the discussion, we found that due to the complexity of the subject matter, it was very difficult for students to resolve their questions through discussion. Confirmation and reassurance from the faculty and teaching assistant appeared to have created a more positive and comfortable environment for the group discussions. CONCLUSION Pharmacoeconomics is an exciting area of study and is increasing in popularity. However, its representation in professional and graduate curricula in schools of pharmacy and other programs is still low. This may be due, in part, to the difficulties in presenting the material in a practical and meaningful way. To assist others in their efforts to develop or improve offerings in pharmacoeconomics, we have presented an outline and description of a novel approach to teaching which includes several components of PBL. In particular, we have utilized the concept of group learning and modified the traditional “case study” approach to the particular needs of the course. We also incorporated two methods of student peer evaluations as a “continuous quality improvement” tool. Results of our course evaluation show that more than 84 percent of the students found this novel teaching approach to be an interesting, useful, and stimulating way to learn pharmacoeconomics.

American Journal of Pharmaceutical Education Vol. 63, Summer 1999

157

This course has been developed specifically for the graduate level. Those who may wish to incorporate PBL into similar courses for advanced professional students should take care to insure that the approach will be appropriate in that setting. We know of no reason why this structure could not be used at either level. However, it may not be a suitable method to teach pharmacoeconomics at the undergraduate level or for the first one or two years of a professional program, both because the needs of these students are different and because they have limited experience in identifying and developing research topics. In that case, faculty may consider assigning cases to students instead of soliciting research topics from them.

the popular media and a systematic review of the scientific literature, group discussion, and/or expert interviews. Specific objectives for this component include: • • • • • •

Identify issues that are important to a given policy analysis. Organize issues relevant to an analysis into meaningful categories. Rank issues in order of their importance to a policy debate. Identify the ethical implications involved with specific emphasis on issues of equity and justice. Understand the bias inherent in policy analysis. Recognize how the process of policy analysis changes perceptions of issues and highlights subjective opinions of the importance of issues. Use the results of a policy analysis to help specify the analytical model used for cost-effectiveness, cost-utility, and cost-benefit analysis.

Acknowledgement. The authors would like to acknowledge the contribution of Zhimei Liu, teaching assistant.

•

Am. J. Pharm. Educ., 63, 152-159(1999) received 8/18/98, accepted 2/5/99.

Techniques for Economic Evaluation of Health Care. Construct models of relationships between outcomes and cost measures, construct cost measures that reflect the perspective of the analysis, and identify epidemiological data and adjust these for biases and for temporal changes as needed to reflect effectiveness or outcomes. Specific objectives for this component include:

References (1) Barrows, H.S. and Tamblyn, R.M., Problem-Based Learning: An Approach to Medical Education, Springer, New York, NY (1980). (2) Young, N.C., “Problem-based learning: Using cases to drive the learning process,” J. Dental Educ. 62, 235-241(1998). (3) Pringle, R.K., “Problem-based learning: Definitions, uses, and concerns,” J. Manipulative Physiol.Therap., 21, 122-125(1998). (4) Glaser, R., “The maturing of the relationship between the science of learning and cognition and educational practice,” Learn. Instruct., 1, 129-144(1991). (5) Neame, R.L.B., “Academic roles and satisfaction in a problem-based medical curriculum,” Studies Higher Educ., 7, 141-151(1982). (6) Walton, H.J. and Matthews, M.B., “Essentials of problem-based learning,” Med. Educ., 23, 542-558(1989). (7) Schwartz, P., “Persevering with problem-based learning,” in The Challenge of Problem-Based Learning, (edits. Boud D. and Feletti G.) Kogan Page, London (1991) pp. 65-71. (8) Norman, G.R. and Schmidt, H.G., “The psychological basis of problembased learning: a review of the evidence,” Acad. Med., 67, 557565(1992). (9) Dolmans, D. and Schmidt, H., “The advantages of problem-based curricula,” Postgrad. Med. J., 72(851), 535-538(September 1996). (10) Neufeld, V.R. and Barrows, J.S., “The ‘McMaster philosophy’: An approach to medical education,” J. Med. Educ., 49, 1040-1050(1974). (11) Culbertson, V.L., Kale, M. and Eric, J.J., “Problem-based learning: A tutorial model incorporating pharmaceutical diagnosis,” Am. J. Pharm Educ., 61, 18-26(1997). (12) Strand, L., Morley, P.C. and Cipolle, R.J., “Problem-based student-centered approach to pharmacy education,” ibid., 51, 75-79 (1987). (13) Amos, E. and White, M.J., “Problem-based learning,” Nurse Educator, 23(2), 11-14(1998). (14) Bowen, D.M., “Integrating case-based instruction into dental hygiene curricula,” J. Dental Educ., 62, 253-256(1998). (15) Saarinen-Rahiika, H. and Binkley, J.M., “Problem-based learning in physical therapy: A review of the literature and overview of the McMaster University experience,” Physical Therapy, 78, 195-207(1998). (16) O’Hanlon, A., Winefield H., Hejka E. and Chur-Hansen A., “Initial response of first-year medical students to problem-based learning in a behavioural science course: role of language background and course content,” Med. Educ., 29, 198-204(1995). (17) Kalaian H.A. and Mullan P.B., “Exploratory factor analysis of students’ ratings of a problem-based learning curriculum,” Acad. Med., 71, 390392(1996). (18) Kaufman, D.M. and Holmes, D.B., “Tutoring in problem-based learning: perceptions of teachers and students,” Med. Educ., 30, 371-377(1996). (19) Bligh, J., “Problem-based learning in medicine: an introduction,” Postgrad. Med. J., 71(836), 323-326 (June 1995).

APPENDIX A. LEARNING OBJECTIVES FOR THE COURSE Policy Analysis. Identify policy issues that may be expected to influence the use of one or more health technologies based on items from 158

• • • • • • • •

• • •

Specify an analytical model (cost-effectiveness/cost-utility/costbenefit/Markov). Identify the strengths and weaknesses of different evaluation designs. Understand the importance of specification/selection of perspectives to be included in the analysis. Identify measures of direct and indirect costs based on data on charges, expenditures, and treatment algorithms. Determine and use standard costs in economic evaluations. Identify measures of outcomes and understand the appropriate use of HRQOL, QALY, and utility measures. Understand the difference between efficacy and effectiveness data. Understand the implications of choice of endpoints for the analysis, including the use of (a) intermediate outcomes measures, (b) utilities and quality of life measures, (c) projected final outcomes based on trial data, and (d) summary of findings by meta-analysis. Discount costs and benefits appropriately. Understand sensitivity analysis, including choice of variables and one- and two-way analysis. Construct a simple cost-effectiveness/cost-utility/cost-benefit analysis model using Excel or TreeAge® software.

Reporting Results. Understand the requirements for reporting of policy analysis and technology assessment results in a clear, simple, parsimonious manner in text and poster presentations. • • • •

Outline a manuscript based on instructions for authors from an appropriate scientific journal. Write the draft manuscript based on the outline. Produce a final written manuscript and abstract. Develop and present a poster presentation of the study results.

APPENDIX B. PBL SESSION DESCRIPTIONS AND ASSIGNMENT SUMMARIES #1: Research Topic Proposal

During this session, each student will present the research topic or problem that he or she has selected for the semester’s project. Group discussion will focus on the magnitude of the problem, the policy/technology/treatment alternatives to be considered in addressing the problem, background information regarding each alternative,

American Journal of Pharmaceutical Education Vol. 63, Summer 1999

and the significance of the study. The student will also listen to each group members’ research topic, asking questions and providing feedback for clarification. The goals of this session are to (1) orient all discussants to the research topics that will be discussed during the remaining PBL sessions and (2) help each individual develop and refine his or her research topic. Assignment: The purpose of this homework assignment is for students write a complete research topic proposal (three to five pages, double-spaced). Students should include the following items in the proposal: (a) description of the research topic and policy/technology/treatment alternatives to be examined, (b) background information for each alternative and rationales for inclusion, (c) the hypothesis and significance of the study, (d) the perspective to be taken, and (f) a discussion of relevant policy implications. You may also wish to discuss possible sources of data and choice of analytical model (e.g., costeffectiveness or cost-benefit analysis). In addition, this proposal revision should reflect comments from faculty and the group members. #2: Decision Trees

In this session, each student will apply decision analysis to his/her research topic and conceptualize the topic under the framework of a decision tree. Elements to be included in the tree are: what are the policy/technology/treatment alternatives and progressions following each alternative, what are the possible outcomes associated with each alternative, and what is the decision criteria for choosing one alternative over another (or others)? As there are a variety of ways to model a research question, students should be able to illustrate their topics by more than one decision tree. Each student should discuss these possible tree structures with his/her group members and determine which one is a better representation of the research question. Assignment: Use TreeAge®(DATA) software or EXCEL to draw the decision tree(s) that illustrate(s) your research project. Describe the tree(s) in text and state possible data sources for the payoffs and probabilities in the decision tree(s). Specify your decision criteria and the time frame of your study. Note that it is expected that your hypothesis, research methods, and/or data sources may change later in the course. #3: Costing and Discounting

In this session, students will review the all resource components that must be costed out for their analyses, depending on the perspective that will be taken. Issues to discuss include the type of costing approach to be used (micro or gross costing), items that need to be discounted and the appropriate discount rate, and whether an adjustment for inflation is needed. Students should be prepared to discuss sources of cost information. Assignment: Consistent with the perspective of the study, provide a concise description of the resource (incl. fixed, variable, marginal), the unit of measure, and the source of cost information for each resource component. A table format is useful for this purpose. Be sure to note whether the costing approach is at the micro or gross level. Indicate which items, if any, need to be discounted and/or inflated and indicate the rate(s) to be used. Lastly, provide a justification for the final cost of each item and any adjustments made to the cost. For example, if a DRG reimbursement rate is used as a proxy for the cost of hospitalization for a particular condition, indicate why this measure is being used in lieu of other alternatives. #4:Health-Related Quality of Life (HRQOL) and Ratios

Students should be prepared to discuss quality-of-life issues related to their projects. Study questions should be formulated as a cost-utility analysis (if not already in this format). Topics to be discussed in this session include relevant quality of life issues, measurement of HRQOL, incorporation of QOL information in outcome measures, and determination of items that belong in the numerator and denominator of the cost-effectiveness or cost-utility ratio.

Assignment: Construct your research question in the form of a costutility analysis. Indicate the relationship between the intervention, the patient’s QOL, and the perspective of the study, how QOL will be determined (e.g., survey approach, published literature), the outcome measure that will be used, and the construction of the ratio. Be sure to list which items will be included in the numerator and denominator of the ratio. #5: Markov Models

During this session, students will describe the structured of their research questions in terms of a Markov model, including the selection of transition probabilities and analysis of the resulting model. Assumptions made and difficulties encountered also are important issues that should be part of the discussion. Assignment: Formulate your research question in the form of a Markov chain or Markov model to estimate the long-term health consequences and costs associated with your problem. First, define the health or disease states associated with your chosen problem, describe these states, and justify the clinical and/or economic significance of these states. Identify transient and absorbing health states. Next, draw a state transition diagram with all health states, transitions, and transition probabilities shown and integrate this into your decision analytic model. Present your transition probabilities in a matrix and describe the sources of these probabilities and any calculations you may have made to derive them. Identify cycle length, the total number of cycles, and whether you will employ a half-cycle correction in your model. Justify your “stopping” rule. Develop estimates of the health-statespecific costs and outcomes, describing data sources and any assumptions required. Finally, use either cohort or individual simulation to analyze your model and to produce estimates of the health outcomes, costs, and incremental cost-effectiveness or cost-utility ratio(s). #6:

Cost-Benefit Analysis (CBA)

Students will describe the benefits or outcomes of their projects in monetary terms and determine how these will be measured. The costs and benefit information will be combined to calculate net present value and net benefit for each intervention alternative. Those students who will not be using a CBA approach in their final papers should reexamine their research topics from this perspective. This should include any changes to the analysis that would need to be made in order to conduct a CBA approach and a justification of why their chosen approach is best to address their research questions. Assignment: Construct your research question as a CBA. Describe the perspective of the study, relevant policy/technology/treatment alternatives, and how costs and benefits will be determined. For example, if the human capital approach is used to measure benefits, the student should explain why this approach has been chosen and what kind of information will be required. If the willingness to pay (or accept) approach is chosen, provide a rationale for choosing this approach and a list of questions to be used for the targeted population. Lastly, explain how net benefit will be calculated and determine how the best alternative will be chosen. #7: Sensitivity Analysis and Statistical Issues

In this session, students will review the assumptions made in their research projects, discuss sources of uncertainties underlying each assumption, and employ sensitivity analysis to test the robustness of research findings. At a minimum, one-way and two-way sensitivity analysis should be planned and discussed. Assignment: List all the assumptions of your research project. For each assumption, discuss other possible scenarios and use sensitivity analysis to re-evaluate the research question under these scenarios to see if the conclusions are robust. Results of the sensitivity analysis should be presented in tables and/or graphs.

American Journal of Pharmaceutical Education Vol. 63, Summer 1999

159