Research - CiteSeerX

JOURNAL OF MICROBIOLOGY & BIOLOGY EDUCATION, May 2012, p. 2-10 Copyright © 2012 American Society for Microbiology DOI: http://dx.doi.org/10.1128/jmbe.v13i1.389

Research

The Use of Open-Ended Problem-Based Learning Scenarios in an Interdisciplinary Biotechnology Class: Evaluation of a Problem-Based Learning Course Across Three Years † Todd R. Steck1*, Warren DiBiase2 , Chuang Wang3, and Anatoli Boukhtiarov2 1Department of Biology, 2Department of Middle, Secondary and K-12 Education, 3Department of Education Leadership, University of North Carolina at Charlotte, Charlotte, NC 28223 Use of open-ended Problem-Based Learning (PBL) in biology classrooms has been limited by the difficulty in designing problem scenarios such that the content learned in a course can be predicted and controlled, the lack of familiarity of this method of instruction by faculty, and the difficulty in assessment. Here we present the results of a study in which we developed a team-based interdisciplinary course that combined the fields of biology and civil engineering across three years. We used PBL scenarios as the only learning tool, wrote the problem scenarios, and developed the means to assess these courses and the results of that assessment. Our data indicates that PBL changed students’ perception of their learning in content knowledge and promoted a change in students’ learning styles. Although no statistically significant improvement in problem-solving skills and critical thinking skills was observed, students reported substantial changes in their problem-based learning strategies and critical thinking skills.

INTRODUCTION After the first introduction of the Problem-Based Learning (PBL) instructional model by McMaster Medical School, Hamilton, Ontario, Canada in the mid sixties, this particular teaching methodology has been steadily gaining interest here in the United States and abroad. Some studies suggest that PBL is an effective pedagogy to help students become self-regulated learners and develop problem-solving skills (5, 12, 13). Other studies, however, noted some weakness of PBL. Nuy (9) and Patel et al. (10) posit that as far as content knowledge is concerned, the traditional methods teach basic science in a more coherent way than does PBL, even though the traditional approach may lack motivation. Problem scenarios used in PBL can be either closed, in which there is a correct answer to the stated problem, or open ended, in which multiple solution strategies are possible. Although there has not been a direct comparison of the effectiveness of open versus closed PBL in achieving any of multiple educational outcomes, open PBL provides students with greater flexibility in developing solution strategies, and it better mimics the type of problems students will encounter outside the classroom. However, closed problembased learning is more common. This is likely due to closed PBL scenarios being easier to develop, since knowing what the correct answer is to a problem scenario makes it easier to predict what content will need to be covered and what *Corresponding author. Mailing address: 9201 University City Blvd, The University of North Carolina at Charlotte, Charlotte, NC 28223. Phone: 704-687-8534. Fax: 704-687-3128. E-mail: [email protected]. †Supplementary materials available at http://jmbe.asm.org 2

resources the students will need. Closed PBL is used in medical schools to teach students how to diagnose a patient having a specific illness, and in microbiology laboratories when students are tasked with identifying an unknown, yet specific, microbe. One of the challenges the PBL methodology has been facing is assessment; the assessment of PBL has never reached a consensus. Even though evaluating PBL effectiveness by comparing such courses with traditional courses may be valid and useful from the accreditation standpoint, it may not provide the necessary objective data. PBL as a learning method dramatically shifts the emphasis of the goals of learning; critical thinking and self-directed learning become more emphasized than a pure accumulation and recall of information. Conceptualization becomes a bottomup process when confronting a problem; students critically assess it, brainstorm possible solutions, and only then come up with a generalized solution. Norton (7) cautioned that making assessment criteria explicit to students may lead those students to develop strategies that comply with the assessment instruments. The author emphasized the need for formative assessments that support reflective thinking and cooperation between students. However, Gijbels et al. (4) maintained that changes in the assessment could produce changes in students’ learning outcomes. In addition to content learning, PBL claims to develop such important skills as critical thinking, problem solving strategies, self-regulated learning, and collaborative learning in teams—the skills which are not always assessed in traditional, lecture-based classrooms. There has been an argument in the PBL literature that learning in the PBL environment needs to be assessed differently, namely as a process and a qualitative change in the cognition of the learner. Since the PBL approach is based on self-directed experiential

Journal of Microbiology & Biology Education

Volume 13, Number 1

STECK et al.: Open-Ended Problem-Based Learning Scenarios

learning, the assessment needs to somehow reflect it (SavinBaden (11)). Savin-Baden found that much of the meaningful learning took place during group work. However, this kind of learning was seldom rewarded academically; therefore, students experienced a conflict between what they valued in the course and what was valued in the assessment. On the other hand, assessment determines not only how the students perceive learning goals, but also what strategies these students choose in order to achieve these goals (6). Nuy (9) suggested that students need to reflect on their attitudes toward PBL and how PBL was related to their motivation to learn during group discussions. Chung and Chow (1) incorporated concepts of learning in their PBL course, and students commented that the course was much better structured, that the workload and assessments were more in step with the course, and that their motivation was higher, which lead to better learning outcomes. Concept mapping was an alternative form of assessment. Novak and Gowin (8) maintained that meaningful learning occurred more easily when new concepts were presented in the context of broader and more familiar concepts, thus emphasizing the role of prior knowledge in learning. The authors saw concept mapping as both an instructional tool and an assessment mechanism that reflected changes in critical thinking of the students. At UNC Charlotte, The Fund for the Improvement in Postsecondary Education (FIPSE) supported the development of a series of courses in which biology and civil engineering students would study together and be equally exposed to the both disciplines. One course in the series used student-centered instruction and introduced problembased learning (PBL) via scenarios that focused on development of microbiology-based biotechnology solutions to real-world civil engineering problems. The goals of this course were to: (1) enhance student conceptual knowledge and understanding; (2) enhance students’ critical thinking skills; (3) enhance students’ appreciation for collaborative learning; (4) enhance faculty receptiveness to the use of PBL strategies in classroom practice; and (5) create a PBL model that could be useful in other settings/disciplines. To encourage interdisciplinary student interactions, there were no prerequisites for this course; any senior or graduate civil engineering or biology student could enroll. In this paper, the course is described and the effectiveness of the course in accomplishing the stated goals is evaluated. In our case, internal evaluation seemed to be more appropriate for two reasons. First, the courses themselves combined two disciplines and, thus, to a degree, created a novel discipline which would make it difficult to assess a course against its constituting components. Second, there had been no similar courses that used a more traditional method of instruction that could be used as a “norm.” In designing the courses, the goals and objectives were radically modified and moved away from a data accumulation approach. In other words, content knowledge in PBL courses was not presented in a well-structured and sequential Volume 13, Number 1

format. It was acquired as a by-product of PBL experience instead. The research questions that were answered include: 1. Is the series of PBL courses effective in improving students’ content knowledge, problem solving strategies, and student perceptions of critical thinking skills? 2. Do students’ learning styles change after taking the PBL courses? 3. What are students’ perspectives toward taking PBL courses? 4. What are professors’ perspectives toward teaching PBL courses?

METHODS Student population Upon approval from the Institutional Review Board at the University of North Carolina at Charlotte (UNC Charlotte), 58 senior undergraduates and first-year Master’s students and two faculty members at the University of North Carolina at Charlotte participated in this three-year project. No changes were made to the course during the three years to follow the approved plan of the intervention funded by FIPSE. Pretests of the combination of student self-reported content knowledge, problem-solving strategy, and critical thinking skills were not statistically significant different across three years: Wilk’s λ = 0.84, F (6, 106) = 1.63, p = 0.15. Univariate tests of between-subjects also failed to see a statistically significant differences of student groups across three years: F (2, 55) = 2.74, p = 0.07 for content knowledge; F (2, 55) = 1.46, p = 0.24 for problem-solving strategies, and F (2, 55) = 3.00, p = 0.06 for critical thinking skills. As a result, student cohorts of three years were treated as a single group for data analysis. Of these students, 26 (45%) were female and 32 (55%) were male; 20 (35%) were African American, 36 (62%) were Caucasian, and 2 (3%) were either Asian or Hispanic. Both faculty members were Caucasian, one is male from the biology department, while the other is female from the civil engineering department. The problem scenarios Each course consisted of student teams developing solution strategies to three problem scenarios. The first “mini” scenario was designed to introduce the students to the PBL concept; that is, the nontraditional means of learning, working as a member of an interdisciplinary group, and the means of student performance evaluation. This first scenario (Appendix 1) had as its task deciding which of three detection methods should be chosen to monitor a genetically modified bacterium in the environment. Students spent the first 3–4 weeks of the course on this scenario. The second and third scenarios (examples are given in Appendices 2 and 3) required the student teams to develop a solution


3


strategy to a real-world problem, using microbiology-based biotechnology. Each scenario was developed based on identifying a current problem from the scientific literature of a conceptual and content level that would be appropriate for both civil engineering and biology students. Students spent 5–6 weeks on each of these scenarios. Each scenario was 2–4 pages in length and consisted of: a title, specific goal, background on the topic, a scenario in which the problem is stated, the student team’s task, an action plan (what should be contained in a report containing their solution strategy), and a list of resources. Because the course content stemmed directly from the background and tasks given in each scenario, care was taken in writing the scenarios to ensure that core concepts would be learned and that each team could individualize their solution strategy but within content and technique constraints which would be based on the background of the students in each course. Pre- and postcourse assessments The following instruments were administered to all 58 students twice during the course, at the beginning and in the end of the course, to collect data. Classroom Strategy and Problem Solving Skills Survey (CSPSSS) This is a 20-item survey (Appendix 4) that was used to measure the extent to which students thought the specific classroom activities would assist their learning before the course, and did assist their learning during and after the course, as well as students’ self-rating of their problemsolving strategies. Items 1–12 were to measure students’ opinions of the strategies (Strategy) used in the class, and items 13–20 were to measure students’ opinion about how the course impacted their problem-solving skills (Skills). All items used the 5-level Likert-scale, which ranged from 1 (Strongly Disagree) to 5 (Strongly Agree). Student Content Knowledge Survey (SCKS) This is a 7-item survey (Appendix 5) that was used to measure students’ self-perceptions of their current level of knowledge to accomplish objectives of the course (Knowledge). All items used the 5-level Likert-scale, which ranged from 1 (Not Knowledgeable) to 5 (Extremely Knowledgeable). Index of Learning Styles Questionnaire (ILSQ) This is a 44-item survey (available on-line at http:// www.ncsu.edu/felder-public/ILSpage.html) developed by Felder and Soloman (3) to assess students’ learning styles. Students were asked to select one of the two options for each question. Based upon their choices, students were identified along four dimensions: active/reflective, sensing/ intuitive, visual/verbal, and sequential/global. This instrument was used to measure the impact of the course on the change of students’ learning styles. 4

Focus groups Students attending the course were interviewed in focus groups by a graduate research assistant, who was not involved in the process of teaching or assessment. Although the focus groups were conducted before the release of final grades, the graduate research assistant was from the evaluation team at the college of education and did not share any common interests with the faculties teaching the course or the students interviewed. Following the university policy regarding human subject research, the research assistant had to replace student names with pseudonyms and was required to keep the identification of the participants confidential. All students participated in the focus groups. Questions asked included: “What teaching strategies did the professors primarily use?” “Do you think some of the teaching strategies should change in future classes?” “What were positive aspects of teaching strategies they used?” “What were the negative aspects of teaching strategies used in class?” “Do you have any comments or suggestions?” Faculty interviews Two 30-minute interviews (Appendix 6) were conducted each year with the two faculty members, respectively, at the beginning and end of the PBL course. Faculties were asked about their concerns regarding the implementation of PBL in the course and experiences and challenges they met teaching the course. Student journal Every student kept a journal which was submitted for grading after completion of each course. The journal was written weekly and served as a record of students’ individual efforts in the course. The activities included meetings with the team members, performing class-related tasks, ideas and factual information contributed to group discussions, and any other relevant activity occurred outside of class time. In addition to the instruments above, a graduate research assistant also sat in the course every time the course was taught during the whole year, and took detailed field notes of the classroom activities and how students actually approached the problem solving in class. Statistical analysis A mixed method was used to analyze the quantitative and qualitative data, respectively. The quantitative data include student responses to the surveys (CSPSSS, SCKS, and ILSQ), while the qualitative data include student and faculty responses to focus group and individual interviews, as well as the student journals. Doubly multivariate analysis of variance (MANOVA) was used to examine changes of students’ perceptions of their content knowledge, problemsolving strategies, and critical-thinking skills after taking the PBL courses. Chi-square tests were employed to see if the categories of student learning styles changed. Constant comparison method (codes were developed and modified


Volume 13, Number 1


during the data analysis process) was employed to identify emerging themes from observation field notes, transcribed interviews, and student journals.

RESULTS AND DISCUSSION PBL effect on student content knowledge Means and standard deviations of students’ self-report of content knowledge are reported in Table 1, and for problemsolving strategies and critical-thinking skills before and after taking this series of courses in Table 2. Comparison of mean differences using MANOVA resulted statistically significant differences on the combination of the three measures, F (3, 40) = 10.51, p < .001, η2 = 0.44, for within subjects effects. Follow-up univariate tests showed that students improved significantly with respect to content knowledge, F (1, 42) = 32.28, p < .001, η2 = 0.43, but not with respect to problemsolving strategies, F (1, 42) = 0.58, p = .45, η2 = 0.02, or critical-thinking skills, F (1, 42) = 0.30, p = 0.59, η2 = 0.01. According to Cohen (2), the effect size of the multivariate and that of the univariate for content knowledge were large, and those of the univariate for problem-solving strategies and critical-thinking skills were small. These data indicate that students’ mean report of their content knowledge improved after taking this series of courses, but problem-solving strategies and critical-thinking skills did not statistically improve. Item-level analysis of these surveys suggested that students reported gains in knowledge in all aspects measured by SCKS, but not in problem-solving strategies or critical-thinking skills (Table 1). PBL effect on student learning style Students’ learning styles were classified into four categories: (1) active or reflective; (2) sensing or intuitive; (3) visual or verbal; and (4) sequential or global (3). Chi-square

tests (two degrees of freedom) failed to show statistically significant difference of students’ learning styles before and after taking the course for active or reflective, χ2 = 1.69, p = 0.43; sensing or intuitive, χ2 = 0.16, p = 0.92; or sequential or global, χ2 = 1.38, p = 0.50. For student learning styles for visual or verbal, however, chi-square tests showed a significant change, χ2 = 7.47, p = 0.02. Four students changed from a neutral standing to verbal, and four other students changed from a neutral standing to visual. These data suggest that verbal–visual learning is supported by PBL. In the PBL process, the students make observations, pose questions, communicate both orally and verbally, and seek solutions to posed problems. The students then use their observations (including data) to construct an understanding of the underlying concepts and content. This may well explain the increase in a visual–verbal learning style. We are not aware of other studies documenting PBL promoting a change in student learning styles. Pearson correlation coefficients of the inter-relationships between students’ self-reported content knowledge, PBL strategies, critical thinking skills, and their final grades in the PBL course are presented in Table 3. It is interesting to note that only students’ critical-thinking skills at the beginning, rather than at the end, of the course was statistically significantly related to their final grades, r = .44, p = 0.01. It is also worthy to note that student’s self-report of their content knowledge, PBL strategies, and critical-thinking skills were not so strongly related to each other at the beginning of the course, .41, .41, and .11, but were strongly related, .71, .81, and .76, respectively, at the end of the PBL course (See Table 3 for specific correlation coefficients). This indicates that the PBL course influenced the student’s approaches during the course, and that students became more confident in applying PBL toward the end of the course. The students’ critical-thinking skills were greatly enhanced facilitated by the professors. Classroom observation field notes indicated that professors showed high enthusiasm

Table 1. Descriptive statistics and differences between pre- and post-tests for the content knowledge. Pre (n = 52) 1. Name and describe the principles behind a variety of biotechnology methods.

Post (n=47)

M

SD

M

SD

3.33

0.95

3.82

0.70

Difference t

d

4.08 0.60

2. List examples of molecular biology applications.

3.67

0.85

4.15

0.64

4.36 0.64

3. L ist examples of molecular biology applications specific to environmental engineering.

3.17

0.87

3.88

0.69

6.45 0.94

4. Name some potential future applications of the methods.

3.33

1.03

4.25

0.93

6.57 0.94

5. Explain the basic of bioprocess engineering.

2.58

1.24

3.76

0.95

7.38 1.07

6. Describe the ethical issues and arguments associated with generic engineering.

3.46

0.85

4.06

0.71

5.45 0.77

7. D escribe the advantages and disadvantages of biotechnology methods relative to conventional methods.

3.50

0.90

4.24

0.72

5.62 0.85

Content Knowledge (Items 1–7)

3.29

0.96

4.02

0.76

5.68 0.84

Note. All t-values are statistically significantly different from zero at an alpha level of .001.

Volume 13, Number 1


5

STECK et al.: Open-Ended Problem-Based Learning Scenarios Table 2. Descriptive statistics for the problem-solving strategy and critical-thinking skills at item and construct levels. Pre (n=52)

Post (n=47)

M

SD

M

SD

4.24

0.60

4.16

0.81

2. Working in groups.

3.94

0.55

3.85

0.83

3. Communicating about environmental biotechnology with group members.

4.09

0.33

4.06

0.71

4. Peers as teachers.

3.53

0.26

3.42

0.67

5. Working individually on assignments.

3.76

0.28

3.68

0.72

6. Class discussions led by the professor.

4.32

0.29

3.97

0.85

1. The use of problems.

7. Class discussions led by classmates.

3.47

0.39

3.45

0.58

8. Lectures by the professor.

4.12

0.31

3.99

0.61

9. The coursepack of readings.

3.65

0.24

3.63

0.74

10. The use of electronic resources, primarily the internet, to find information.

4.47

0.31

4.43

0.64

11. Library resources, other than electronic ones.

3.53

0.29

3.49

0.52

12. The use of computers as an investigative tool.

4.45

0.25

4.38

0.70

Problem-Solving Strategy (Items 1—12)

3.96

0.34

3.88

0.70

13. Communicating literature and/or research results.

4.06

0.49

3.91

0.94

14. Participating in discussions.

4.21

0.47

4.19

0.93

15. Writing about environmental biotechnology.

4.14

0.48

3.97

0.78

16. Working collaboratively with classmates

4.09

0.55

3.98

0.84

17. Finding relevant information.

4.35

0.46

4.24

0.86

18. Analyzing and synthesizing information.

4.29

0.42

4.14

0.82

19. Using computers for information retrieval and data analysis.

4.18

0.51

4.16

0.92

20. Thinking critically about environmental biotechnology issues.

4.50

0.56

4.42

0.73

Critical Thinking Skills (Items 13—20)

4.25

0.49

4.16

0.84

Table 3. Correlation coefficients between variables of interest.

a b c

Knowledge Pre (a)

Knowledge Post (b)

Strategy Pre (c)

Strategy Post (d)

Skills Pre (e)

Skills Post (f)

Final Grade (g)

1

.49*

.41*

.12

.11

-.03

.12

1

.12

.71**

.07

.76**

.05

1

.38*

.41*

.26

.22

1

.30*

.81**

.03

1

.30*

.44*

1

-.04

d e f

Notes: (1) * p < .05, ** p < .01, two-tailed; (2) Knowledge is measured by the student content knowledge survey; Strategy is measured by Items 1–12 of the Classroom Strategy and Problem Solving Skills Survey; Skills is measured by Items 13–20 of the Classroom Strategy and Problem Solving Skills Survey; Final Grade is measured by the instructors of the course.

in teaching and consistently encouraged student collaboration by redirecting questions back to the students. They were friendly, caring, and encouraging toward students. Professors offered ideas at times, but encouraged students to build the infrastructure. When confused, students were encouraged to collaborate among themselves. The profes6

sors assigned students specific roles to be accountable, so everyone had a role. Oftentimes, the students were not used to working in groups. To help with this fact, class time at the beginning of the semester was devoted to discussing successful groups, and a handout describing best group practices was shared with the students. This handout suggests


Volume 13, Number 1


various roles and responsibilities that members of a group can take. The students in each group were then encouraged to make these assignments in their respective group, which the groups did, resulting in each member of every group having an assigned task. The ability to effectively work in groups is necessary for PBL. The professors presented PBL with real scenarios students might encounter on the job someday, and assigned articles to read before next class for critical analysis. All students agreed that they learned a lot by actually solving the problem with the guidance from the professors (See Table 4 for themes and quotes from focus group interviews). The professors challenged students’ critical thinking

skills by “not directly giving us the answer but let us brainstorming on our own.” Another student commented, “It was collaborative with them (the professors) throwing questions back at us so we could answer them among ourselves. In the lab, there was a discussion with a problem scenario.” Students’ perspectives toward PBL course From the students’ perception of their learning experience, the implementation of PBL was a success. There were challenges of the logistics of the teamwork. A common problem with the collaborative work is equal shares, as complained by a student: “A lot of times everybody didn’t

Table 4. Themes and representative quotes from focus group interviews. Themes Students learned a lot from this class.

Quotes • Effective, the material was made simple and I retained most of the material months after the completion of the class. • Very effective. I was able to directly apply what was learned in lecture to take home problems and tests. • They really addressed any problems that we are having specifically with the different problem scenarios and helped us kind of work those out.

Teaching strategies were effective.

• Concepts were taught and explained very clearly. Professor obviously knows and enjoys what he teaches. • There was a good bit of lecture and kind of very helpful one-on-one group based problem discussion. • The lecture method of teaching is a highly effective strategy for the presentation of newer information, while the ‘group inquisition’ is a very effective strategy for reviewing material.

Group works are great.

• I learned more when doing the group activity and didn’t obtain much of the information from the lecture. • I really enjoyed the class activity (groups) and the opportunity to “teach” the class for even a few minutes. • The strategies were useful because there were a lot of partner and group activities that we were involved in.

Group works are difficult.

• The group based, a lot of is finding the time that was the major negative aspect, because all this is outside the class. You also rely on other people. It is kind of hard sometimes. • Often group activities have one or two people contributing and the rest piggy backing off the others.

Lectures are effective.

• Lectures are presented in a very straightforward way with hints and additional information provided to help expedite comprehension of key concepts.

Lectures move too fast.

• Sometimes, unfortunately, the material is presented in such a quick manner. It is sometimes difficult to take adequate notes, and it can be difficult to relate material to previous ideas presented in lecture due to the rapid pace of class.

Suggestions

• I would like though once you introduce the new problem scenario, maybe to give us a little lecture on the new material so we know what type of questions whatever we need to ask about. • I would enjoy more linear learning methods instead of jumping around to different subjects. • More interaction with students, more retention of information. Study guides/outlines of information provide guidance on how and what to study.

Volume 13, Number 1


7


do equal shares with the group work.” This lack of cooperation within the group and the lack of leadership skills of the group leader were noticed by the professor at the beginning of the course. One instructor commented: “I don’t think the leader ever stepped forward to assume responsibility to make sure they’d arrange meeting times or to assume responsibility as they were preparing a report for the class, causing stress and frustration for students and for us as well.” Another instructor supported this: “Each week there was a reason that their meeting had gotten garbled. They were very resistant to meeting. We struggled with that and so we probably spend half the semester to get them to the point where they had a solution strategy that was viable.” Most students, however, enjoyed group work and found it very beneficial to their content learning and the development of their critical-thinking skills. Many also saw group work as a good practice for their future work in the field. However, the enrollment in higher-level courses was not as high as in lower-level courses, which may lead to the conclusion that some students tried to avoid a PBL course. Another major concern among students was that they found that the PBL courses require much more time than conventional courses. Results of students’ interviews and field notes show that at the very beginning, when presented with the PBL instructional format, students experienced somewhat of a cultural shock and tended to try to redirect the instruction to a more familiar traditional, lecture-based format. The students had to go through a transitional period of two weeks or so before beginning to accept the PBL as an instructional format and fully cooperate. The biggest nonpedagogical improvements in the third year over the previous years are the class size and the classroom setting. Smaller classes with seminar-like roundtable arrangements were seen as very inclusive by the students where they perceived that the power was equally distributed among all the participants of the discussion, including the instructors. Students seemed to be very comfortable in generating ideas, questioning each other, and providing explanations to each other. The instructors usually sit in the back, allowing students to lead the class and provide updates. The instructors’ roles became more of coordinators and facilitators. Lecture format was applied very seldom by the instructors and only for clarifications of some major concepts. Faculty perspective toward teaching using PBL Both faculty members reported that they understood the principles of PBL “fairly well” and were positive about the impact of PBL on students’ critical-thinking skills. The following excerpt is an example. Interviewer: H ow well does PBL enhance critical thinking skills? 8

Instructor A: F airly well, because we can finesse it better than we did. Regardless of content, it was hard to be patient while the students probed one another. Instructor B: Very well. Students didn’t get much out of it (lecture). There’s nothing else in the biology department even close. Biology instructor is filling in the blank questions, multiple choices, so nothing like this where they take more than 1 step in the critical thinking process. Let’s see where it leads. I don’t think they even got the minimum content, because we dealt with unanticipated issues during class time. Although the instructors recognized the value of PBL, they had concerns about PBL being used by their colleagues. Some of these concerns centered on the challenges in implementing PBL, especially in science departments. They recognize it would be difficult to convince instructors who were taught using the traditional approach, and who have been comfortable teaching with that same approach, to change to a new teaching style. And even for those faculty members willing to try this approach, it is difficult to adopt PBL without training or mentoring, and many departments lack these resources. Separate from implementation challenges are those inherent to using open-ended PBL. For example, in science courses using a traditional lecture-based format, the instructor has complete control over the content of lectures delivered in class. In open-ended PBL, some content control is relinquished in order to allow students to explore their particular solution strategy. Student freedom in pursuing strategies need not be absolute; the problem scenario is critical to providing a guide to the students. An additional challenge arose due to the interdisciplinary nature of the course. Civil engineering graduate students with little biology knowledge necessary to develop a solution strategy were teamed with biology students with little knowledge of civil engineering necessary to understand the environmental problem to be solved using biotechnology. A core set of biotechnology-related topics were developed (Table 5) that were introduced at the beginning of the semester. The goal was to distill biological concepts down to those components needed to use biotechnology tools. For example, in order to use PCR, it is not necessary to understand the mechanism of DNA replication, but it is necessary to understand the complementary nature of double-stranded DNA. Because there are multiple acceptable solution strategies with an open-ended PBL design, the importance of the problem scenarios cannot be overestimated. Through the details given in the background and the specific tasks assigned, the instructor can influence the direction the students take in developing solution strategies and tailor the scenario to each student group. These instructors felt, based upon their combined 25 years of college-level teaching experience and their


Volume 13, Number 1

STECK et al.: Open-Ended Problem-Based Learning Scenarios Table 5. Biotechnology-related core topics and concepts contained in the course. Biology Topic

Relevant Concepts

Bacterial Classification

Microbial diversity; DNA as means to identify taxa

Central Dogma

Connection of genotype to phenotype

Nucleic Acid properties

Importance of complementary nature of nucleic acids in biotechniques

Gene/Operon

Recognition of factors involved when choosing a gene cloning strategy

Biotechnology Techniques

Relevant Concepts

DNA Hybridization

Investigator controls factors that influence single vs. doublestrandedness of nucleic acids; melting temperature

PCR

Amplification; contamination; sensitivity vs. specificity in assays

Transformation (natural, artificial)

Recognition of pros and cons associated with experimental methods

Cloning: - Vectors (plasmids, viruses) - Procedural steps - Enzymes used

Investigators decision in choosing from range of experimental designs

Environmental/Civil Engineering

Relevant Concepts

Microbiology - Monitoring and Detection Biosensors

For all of these topics the common relevant concept is to allow biology majors to appreciate the range of biotechnology applications beyond medicine, and for civil engineers to become familiar with techniques that are expected to soon impact their field.

Role of microbes in: - removing pollution - causing problems - reflecting environmental changes Biological Treatment Agents / Bioremediation Ethics

Relevant Concepts

Specific issues varied with environmental problem contained within problem scenario (e.g, use/release of a GMO to use as a tracer to identify pollution source).

Understanding of ethics attached to any solution strategy that uses biotechnology to solve an environmental problem.

many interactions with each of the students in these courses, that although they were not able to know at the beginning of the semester what content was going to be covered during the semester, they were assured that the students learned relevant material while improving their critical-thinking, writing, and team interaction skills.

CONCLUSION This study examined the effectiveness of PBL courses that combined the fields of biology and civil engineering across three years. Large amount of both quantitative and qualitative data suggested that PBL significantly promoted students’ perception of their learning in content knowledge, problem-solving skills, and critical-thinking skills. In addition, students shifted from neutral to visual–verbal learning. The time devoted to help the students, including the assigning of roles, made for a more effective use of the PBL process. These conclusions may be useful for future researchers and evaluators. For instance, we noted that more structure, consistency, and transparency may be required in order Volume 13, Number 1

to assess the effects of PBL. Our comprehensive method provides resources to evaluate PBL courses.

SUPPLEMENTAL MATERIALS Appendix 1: Module #1 – Choosing the Best GMO Detection Method Appendix 2: Module #2 – Genetic Testing Appendix 3: Module #3 - Bioremediation of Oil Using a Suicidal GMO Appendix 4: Classroom Strategy and Problem Solving Skills Survey (CSPSSS) Appendix 5: Student Content Knowledge Survey (SCKS) Appendix 6: Faculty Perceptions about PBL in Class (FPC)

ACKNOWLEDGMENTS The following people contributed to the development, implementation or assessment of this project: Dr. Helene


9


Hilger, Dr. Dawson Hancock, and Ms. Jeanie Marklin. This work was supported by a US Dept. of Education Award No. P116B040349 given to the authors. The opinions expressed are those of the authors and do not reflect a position, policy, or endorsement from U.S. Department of Education. The authors declare that there are no conflicts of interest.

REFERENCES 1. Chung, J., and S. Chow. 2004. Promoting student learning through a student-centered problem-based learning subject curriculum. Innovations in Education and Teaching International 41:157–168. 2. Cohen, J. 1988. Statistical power analysis for the behavioral sciences. Lawrence Erlbaum Associates, Hillsdale, NJ. 3. Felder, R. M., and B. A. Solomon. 1996. Index of learning styles. http://www.ncsu.edu/felder-public/ILSpage.html. 4. Gijbels, D., G. van de Watering, and F. Dochy. 2005. Integrating assessment tasks in a problem-based learning environment. Assessment & Evaluation in Higher Education 30:73–86. 5. Hmelo, C. 1998. Problem-based learning: effects on the early acquisition of cognitive skill in medicine. J. Learning Sciences. 7:173–208. 6. Kelson, A. C., and L . Distlehorst. 2000. Groups in problem-based learning (PBL): essential elements in

10

theory and practice. In D. H. Evensen & C. E. Hmelo (ed.), Problem-based learning: a research perspective on learning interactions. Lawrence Erlbaum Associates, Mahwah, NJ. p. 167–184. 7. Norton, L. 2004. Using assessment criteria as learning criteria: Case study in psychology. Assessment & Evaluation in Higher Education 29:687–702. 8. Novak, J., and D. Gowin. 1984. Learning how to learn. Cambridge University Press, Cambridge, U.K. 9. Nuy, H. 1999. Interactions of study orientation and students’ appreciation of structure in their educational environment. Higher Education 22:267–274. 10. Patel, V., G. Groen, and G. Norman. 1993. Reasoning and instruction in medical curricula. Cognition and instruction. 10:335–378 11. Savin- Baden, M. 2004. Understanding the impact of assessment on students in problem-based learning. Innovations in Education and Teaching International 41:224– 233. 12. Sobral, D. 1995. The problem-based learning approach as an enhancement factor of personal meaningfulness of learning. Higher Education 29:93–101. 13. Yeung, E., S. Au-Yeung, Th. Chiu, N. Mok, and P. Lai. 2003. Problem design in problem-based learning: evaluating students’ learning and self-directed learning practices. Innovations in Education and Teaching International 40:237–244.


Volume 13, Number 1