Effectiveness of Problem Based Learning Implementation Savitri Bevinakoppa, Biplob Ray, Fariza Sabrina
[email protected],
[email protected]*,
[email protected] Melbourne Institute of Technology Central Queensland University* 284-294 La Trobe Street, Melbourne Victoria 3000 Australia Cairns, Queensland Australia Abstract: Problem based learning (PBL) exercises have been proven to be an effective teaching method for preparing the students as work ready graduates. Students work on a real world industry based problem of relevant discipline. It also help students to improve their capability of critical analysis. PBL has been adopted in educational practise in many disciplines. In this paper, we explore the various existing PBL practices, develop, implement a model and analyse the effectiveness of the implementation. This paper explains few exemplars of PBL, their implementation method and analysis of students’ feedback. Paper concludes with the direction of our future work. Keywords: Problem Based learning, real world case study, critical thinking, collaborative learning, learning strategies 1.
Introduction
Problem Based Learning (PBL) mainly has been used in the medical science area, since its development by Barrows and Tamblyn at McMaster University in 1980 (Major & Palmer, 2001). It is viewed as a successful strategy to align university courses/units with the real life case studies and students be able to ready as work ready graduates (Biggs & Tang, 2007; Pawson, Fournier, Haigh, Muniz, Trafford & Vajoczki, 2006). It was since introduced in educational practice in all of the areas such as health sciences, engineering, business, science, education etc. courses/units in many universities worldwide. Over the last 15 years, many Australian universities have also adopted PBL as an instructional approach either in a 'pure' or a 'hybrid' form. In the pure form of PBL, the entire course is delivered using the PBL instructional approach, with no traditional lectures involved. As an example Monash University offers Business strategy courses in Peninsula campus using pure PBL instructional approach (Problem-based learning, n.d.). In the 'hybrid' form, PBL approach is blended with tradition instructional approach such as in lectures, laboratories, tutorials etc. For example, computer science courses of University of Sydney (Barg, Fekete, Greening, & Hollands, 1999). Based on the nature of students enrolment in networking and engineering courses in Melbourne Institute of Technology (MIT), we practice 'hybrid' form of PBL. PBL is one of the teaching and learning strategies, which is an open-ended and ill structured problem. There are many definitions in the literature to define PBL. None-the-less all definitions might have slight variation based on the context PBL is implemented, the main concept of all those definitions attempts to frame same view. A working definition of PBL which will be suitable for the 'hybrid' approach used at MIT is: “Problem-based learning use stimulus material to engage students in considering a problem which, as far as possible, is presented in the same context as they would find it in ‘real life;’ this often means that it crosses traditional disciplinary boundaries. Information on how to tackle the problem is not given, although resources are available to assist the students to clarify what the ‘problem’ consists of and how they might deal with it. Students work cooperatively in a group or team with access to a tutor who can facilitate the learning process.” The above definition is adopted from (Boud & Feletti, 1998) and modified to fit in our context. This paper explores the various existing PBL practices, develop a model to satisfy MIT courses and benchmark with practices at other higher education institutions. This will also contribute to standardising MIT PBL practice across all schools (IT & Engineering and Business). 2.
Background
Problem Based Learning (PBL) is an learning approach, that is characterised by flexibility and diversity that can be implemented in a various ways in diverse contexts' to improve on students’ deep learning capability. Researchers from within Australia have commented on the lack of a deep approach to learning and thus
1
employers’ dissatisfaction with graduates (Cope, Staehr & Horan, 2002; Tan, 2008). The framework for success based on promoting a deep learning approach includes: • Assessment tasks that demonstrate conceptual understanding; • The emphasis is on depth of learning rather than breadth of coverage; • Tackle real world problems/scenario. PBL can be a combination of cognitive and social constructivist theories, as developed by Piaget and Vygotsky, respectively (Ozer, 2004). Both cognitive and social constructivist theories state the importance of deep learning using unstructured problem context that closely resemble future professional problems. According to SavinBaden (Savin-Baden, 2001), PBL is characterized by several pedagogical approach including: • • • • •
An acknowledgment of learners’ experience An emphasis on students taking responsibility for their own learning An intertwining of theory and practice A focus on the processes of knowledge acquisition rather than the products of such processes A focus on communication and interpersonal skills so that students understand, relate their knowledge, and require skills to communicate with others
Well designed PBL exercise can make students a creative thinker and achieve deep learning in an industry situation, where learning starts by defining the problem which always unstructured. Coral Pepper from University of Western Australia (Pepper, 2009) stated benefits of PBL in business and Science courses as below: • students deciding on the information and skills they need to investigate issues, while building on their current knowledge to synthesise than integrate a new information • students taking responsibility for the learning that occurs within their group, while instructors monitor and facilitate student learning • students engaging with the learning experience. Although there are differences at the concrete model level in achieving learning objectives of collaborative learning and lifelong learning, there are common learning and teaching principles cross PBL models and can be categorised in three approaches: learning, contents and social (Xiangyun, Graaff, & Kolmos, 2009) (Etherington, 2011). Students learn by working on real world problems using a combination of self-directed, peer, and collegial learning approaches that are designed to meet the demands of the discipline and the profession. After reviewing many literatures, we came up with the PBL approach written in section 3. We followed the following sequential process to conduct this project: 1. Review of the current literature regarding standard PBL practices. 2. Review of the current PBL conducting process and practices at other higher education institutions. 3. Analyse current MIT PBL practices and benchmark with other higher education institutions based on their PBL process and practices. 4. Develop a standard PBL model to satisfy MIT units and prepare exemplars. 5. Conduct seminar and workshops for all staff. Provide feedback to Unit Coordinators (UC) based on their current PBL problem, process and practices for improvement. 6. Work along with UC to ensure that improved PBL problem, process and practices are aligned with newly developed model. 7. Finally, analysed feedback from UC to verify that new PBL problem, process and practices are achieving its objective to preparing work ready graduates. 3.
Problem Based Learning approach
The PBL has three main elements: the problem, process and reflection (reflective journal with plan for improvement) which determine the overall outcome of the learning. Carefully designed problem, proven process and continuous improvement plan will achieve optimal outcome of PBL exercise in a hybrid implementation at MIT. There are three characters (unit coordinators, students and tutors) involved in PBL exercise, the discussion below detail their roles and responsibilities to achieve learning outcome of the PBL. Normally unit coordinators
2
design and develop PBL exercise, students solve the problem and tutors give constructive feedback to students with valuable suggestions or relevant solutions. 3.1 The problem: Design and Develop PBL exercise (for Unit Coordinators) Designing and developing a problem is not easy. Instructors need to understand carefully on various parameters, such as students’ prior knowledge, course/program level (such as Under Graduate or Post Graduate), students’ work experience, ability to solve problems etc. Problem based learning is 'problem first learning' (Spencer & Jordan, 1999) because it is the problem which defines the learning. Instructors design problems to represent authentic, real world situations where small group of students work on problem to resolve. To enhance the understanding of problems, a number of central features for problem construction in PBL have been proposed (Dolmans, Snellen-Balendong, Wolfhagen, & Van der Vleuten, 1997). University of Wollongong’s approach (Loyens, Kirschner, & Paas, 2011) explains the following parameters to design a problem: prior knowledge, elicit discussion, stimulate self-directed learning, and encourage knowledge integration and transfer. Based on this approach, we came up with five “rules” for effective problem design as ‘PREES’: 1.
Prior (P) knowledge: Students need to have at least prior knowledge of basic concept necessary to understand the problem. Piaget’s research on cognitive constructivist theory has shown that prior knowledge influences the quantity and quality of new knowledge acquired (Ozer, 2004).
2.
Relevance (R): To motivate students’ learning, PBL problems should be relevant for students’ future profession (Loyens, Magda, & Rikers, 2008). It stimulates their cognitive and social learning aspiration. This also covers the gap between learning theories in traditional class and using it in professional context.
3.
Elicit (E) discussion: In addition to linking of the problem to prior knowledge, problem must have scope and encourage elicit decision making. Problems can elicit discussion with some self-directed clues such as opposing viewpoints, allowing students to generate arguments within peers and discuss optimal point of view among group. Unstructured problems can lead to confusion and on the other hand, too well-structured problems that lead to structured answer without critical analysis part.
4.
Encourage (E): Problems need to inspire students to apply their developed knowledge with existing schemas, so that they can apply information in subsequent new situations. In other words, a problem needs to stimulate knowledge integration and transfer. To accomplish this, information needs to be presented in a broad context, so that students can better understand the purpose of the problem (Loyens, Kirschner, & Paas, 2011).
5.
Stimulate (S): PBL problems need to be constructed that learners determine themselves what they find relevant for their learning. Same time, they are not diverted from their learning issue, such as learning objectives of the unit. The problem should stimulate student’s cognitive conflict and further Self Directed Learning (SDL), because it can involve multiple solutions which generate discussions within the group.
It is not necessary to have a separate problem for each tutorial session. As an example, in a Web Technology unit, the tutor might give a problem to design an e-market site that is better than ebay. In one tutorial, students might investigate the features of ebay, in another they might try to discover deficiencies in the ebay operation, in another tutorial they might take one of those deficiencies and work out how to rectify it, in the next tutorial they might critically analyse the strengths and weaknesses of their solution strategy etc. 3.2 Process or approach: Student role to solve PBL exercise The process of conducting PBL in business, ICT and engineering context has been discussed in some models (Karami, Karami & Attaran, 2013; The Guide offers illustrative examples of key variables to consider when contemplating the Design, Implementation, Evaluation or Revision, of undergraduate medical curricula, n.d.; Graff & Kolmos, 2003). These models are designed to keep in mind the pure PBL learning approach. Karami et al (Karami, Karami, & Attaran, 2013) have broadly introduced a model to engage ICT solution on PBL. It has four main stages: presenting issues related to the content knowledge and unit objectives, grouping, data collection and involving groups in problem to conduct PBL. In University of Newcastle (The Guide which
3
follows offers illustrative examples of key variables to consider when contemplating the Design, Implemention, Evaluation or Revision, of undergraduate medical curricula, n.d.), PBL study guide state the process as “Selfdirected study groups discuss and analyse selected cases. The typical study group (8 to12 students) meets once or twice a week. Each individual student in the study group presents his/her work. It is then discussed and the group decides who will continue with what tasks. Often students organise their work in such a way that their individual work supplements the work of the group, enabling them to develop a broader perspective of the related themes. The role of the teacher who attends the meetings is primarily to facilitate the learning process i.e. to facilitate the group's work and internal communication (Graaff & Kolmos, 2003). Trinity Western University proposed by Etherington in Australian Journal of Teacher Education (Etherington, 2011) is shown in Figure 1. Figure 1 has four stages to achieve PBL learning outcome and designed for pure PBL approaches. Four stages are mainly categorised as: Problem, Analyses, Proposal of solution and Report. Student analyse the problem Real life scenario or problem encountered
What do i know about this probelm? What do i need to know about this problem?
Probelm confirmed Conduct Research
Problem solved/Report result
Solution Proposed Figure 1: PBL learning process model in (Etherington, 2011) To develop an appropriate process for conducting hybrid PBL in MIT, this paper adapted PBL learning approach from (Etherington, 2011) with a number of modifications. Modifications are based on various case study presented on hybrid PBL implementations within Australian institutions, such as The University of Sydney (Barg, Fekete, Greening, & Hollands, 1999), University of Western Australia (Pepper, 2009) and Monash University (Edwards & Hammer, 2007). MIT learning process model (“RAID”) is shown in Figure 2, where few elements are modified such as students analyses in a group of 3-4, students final outcome is their reflection on learning outcome, etc. Main four stages are Real (R) life problem, Analyse (A), Identify (I), Discuss (D) and last stage is Reflective Journal that can be combined as in the stage ‘discuss’. The final outcome is students’ proposed solution to the problem and then a reflection of the process and outcomes. Real life scenario or problem mapped to unit Learning Outcome (LO) Write Discussion outcome (Reflective journal)
Discuss
Analyse the problem
Identify the requirements
Figure 2: The RAID - PBL learning process for MIT
4
3.3 Guide: Instructors/tutors Role Traditionally tutors play as a source of information and lead students to reach a pre-determined goals. However, to achieve learning benefits offered by PBL method, tutors require to take the role of facilitator, which is different to traditional tutor’s role (Spronken-Smith & Harland, 2009). In the literature (Loyens, Kirschner & Paas, 2011; Spronken-Smith & Harland, 2009), authors identified various reasons of instructor’s weaknesses, such as lack of understanding about PBL process, fear of getting negative feedback from students and lack of broader view of problem knowledge within the domain. 4.
Delivery of PBL exercises
MIT offers combined Undergraduate (UG) and Postgraduate (PG) lecture and laboratory classes. PG units have an extra one hour tutorial, where we deliver PBL tutorial. The Undergraduate (UG) students can also learn from PBL as long as it is completely built on their appropriate level of knowledge and their interest domain. PBLs are included in UG lectures and discussed during class. As an example, units might have one short problem during the lecture which is related to the lecture content. PBL problem is discussed not more than 10 minutes of the lecture time. These PBLs were common to UG and PG students, however, the PBLs are mainly focused to PG students. For an effective participation of students, we have allocated responsibilities as follows: -
PBL preparation and marking guide – Unit Coordinator (UC) responsibility Solving problem – Students’ responsibility Facilitating – Tutors’ responsibility
Tutors facilitate PBL tutorials, currently there is one hour allocated for this session. Tutors follow the following steps as shown in figure 3: • • • •
5 minutes introduction of the problem by tutor 20-30 minutes students discussion among the group 10-15 minutes students write their individual discussion outcome 10-15 minutes feedback to their outcome by tutor
Figure 3: PBL tutorial session process Since PBL is a new concept for students, it is important to explain students in the very first session regarding PBL and its nature. Lecturer or UC chooses an appropriate problem which is learners’ interest domain, appropriate for level of learners and relevant to their future profession to overcome leaners’ dissatisfaction, confusion and uncertainty. As an example, if we choose the problem from a newspaper which created great public interest on the relevant domain, students will be very much interested to brainstorm on it. Then tutors facilitate the discussion. 5.
PBL Exemplars
Master of Networking course has 12 units. There are 3 units per trimester. MN501 Network Management in Organisatios unit is in first trimester, and MN505 Local Area Network Management unit is a second trimester unit, where students have a background of Networking concepts. MN501 unit topic covers mainly professional practice: One of the PBL exercises for MN501 and MN505 are given below. MN501 Network Management in Organisations PBL exercise exemplar ABC Pty Ltd is a sales company, which sells networking devices such as routers, switches, cables, compact flashes, wireless dongles etc. Ross is a long serving employee of ABC Pty ltd. He has contacts with more than 100 companies/institutes who buys networking items through him. He is friendly and knowledgeable person. Because of his friendliness, one of his buyer gifts a bottle of wine and hamper for Christmas.
5
a. b. c. d.
Do you think Ross will accept gifts? Explain your answer with justification. What is bribery? Do you think gift is a genuine business purpose? How can you find out ethics policy of the organisation?
Above example can be discussed within group of 2-3 students in a class in one week PBL class duration. MN505 Local Area Network PBL exercise exemplar XYZ IT company is situated within city area. This company building has 6 floors, with each floor accommodate approximately 40 employees. First floor is a security area where guard will be looking after the area with CCTV cameras monitoring. Second floor to 6th floor, most of the employees need specialised software to work on IT projects. Server room with IT manager is situated on 3rd floor. Each floor has approximately 10 rooms. Design a network for this IT company. Your network design should consider the following criteria: a. Type of network cable, length of the cable b. Type of networking devices c. Specification of each networking device with justification d. Type of server with specification e. Network diagram f. Budget to build LAN or cost analysis g. Configuration of networking devices h. Performance analysis of the network in terms of speed, efficiency etc. The above PBL exercise can be for 5 weeks covering 2-3 points in each week. Such as week 1, students can think about points a-c before coming to week 2 class. Group of 2-3 students discuss their finding and come to a conclusion of the outcome for each point a- c. Similarly week 3 covers points d-e, week 4 covers f-g and week 5 covers point h. Each week students are asked to write reflective journal on their finding before the class, outcome of the group discussion and plan for upcoming week based on interactive discussion with whole class and instructor.
6. Student survey Surveys have been conducted in both Melbourne and Sydney campuses where PBL exercises are same. Following number of surveys have been received by students in Melbourne campus: MN501 = 15, MN503 =
37, MN505 = 26, MN506 = 17, MN601 = 25, and MN603 = 9. MN 501, MN503 are first trimester units, MN505, MN601 are second trimester units and MN506, MN603 are third trimester units were chosen for surveys.
6.1 Survey questions Following points were given on the students’ survey form. Option are ‘Strongly agree’, ‘Agree’, ‘Neutral’, ‘Disagree’, ‘Strongly disagree’ and comments column to write additional feedback. 1. The PBL problems are based on real world case studies. 2. The PBL problems are appropriate for this Unit. 3. The PBL problems are interesting. 4. The PBL problems are based on unit learning outcomes. 5. The PBL problems gave me an exposure to the industry based work. 6. The PBL task helped me in improving my critical thinking ability. 7. The PBL task helped me understanding the topic better. 8. The brainstorming sessions were very effective. 9. The discussion among the students and with the tutor was very beneficial for my learning. 10. The duration of the PBL session was appropriate. 6.2 Survey feedbacks Survey results are shown in figures 4-9. X axis shows question number, y axis show percent of students, bars show percentage of options ‘Strongly agree’, ‘Agree’, ‘Neutral’, ‘Disagree’, and ‘Strongly disagree’.
6
120 100
%
80 60 40 20 0
1
2
3
4
5
6
7
8
9
10
Survey questions Strongly Agree
Agree
Neutral
Disagree Strongly
Disagree
Figure 4: MN501 Students’ feedback graph 120 100
%
80 60 40 20 0
1
2
3
4
5
6
7
8
9
10
Survey questions Strongly Agree
Agree
Neutral
Disagree
Strongly Disagree
Figure 5: MN503 Students’ feedback graph 120 100 80 60 40 20 0
1
2
3
Strongly Agree
4 Agree
5 Neutral
6
7
Disagree
8
9
10
Strongly Disagree
Figure 6: MN505 Students’ feedback graph 7
120 100 80 60 40 20 0
1
2
3
Strongly Agree
4
5
Agree
6
Neutral
7
8
Disagree
9
10
Strongly Disagree
Figure 7: MN506 Students’ feedback graph 120 100
%
80 60 40 20 0
1
2
3
4
5
6
7
8
9
10
Survey questions Strongly Agree
Agree
Neutral
Disagree
Strongly Disagree
Figure 8: MN601 Students’ feedback graph 120 100 80 60 40 20 0
1
2
3
Strongly Agree
4 Agree
5 Neutral
6
7
Disagree
8
9
10
Strongly Disagree
Figure 9: MN603 Students’ feedback graph 6.3 Survey analysis Figure 4 and 5 show feedback graph of first trimester units MN501 and MN503. Percentage for option ‘agree’ has more than other 4 units shown in figures 6-9. It indicates that students agree that PBL exercises are useful and gives them opportunity to explore real world scenario in depth. Once students understand the expectation of teh PBL, student ‘strongly agree’ that PBLs are very useful, except in MN506. After analysing MN506 PBL exercises, we found that most of the questions are direct questions on theory covered in lecture class. Students are not applying their theory knowledge to explore in real world case studies.
8
7 Conclusion Qualitative survey was conducted among unit coordinators and students. Survey results show that Unit Coordinators were happy to receive feedback on their PBL exercises and did constructively updated PBL based on critical comments. On the other hand, student often express their dissatisfaction regarding PBL as they experience confusion and uncertainty (what was required from them) because of the responsibility and control given to them. Some students experienced this differently as stimulating and challenging, while others felt seriously hindered in their learning process to the extent of developing feelings of anxiety (Duke, Forbes, Hunter, & Prosser, 1998). It can be challenging to facilitate a group of students who are facing uncertainty because of the above mentioned reasons. This situations can be avoided by explaining students in the very first session regarding PBL and its nature. It is also very important to choose an appropriate problem which is in learner’s interest domain, appropriate for level of learners and relevant to their future profession to overcome learners’ dissatisfaction, confusion and uncertainty. As an example, if we choose the problem from a newspaper which created great public interest on the relevant domain, students will be very interested to brainstorm on it. Our future work will focus on training students and tutors to handle PBL process efficiently. Acknowledgement Authors would like to acknowledge Melbourne Institute of Technology (MIT) for providing funds to carry out this research project.
References Barg, M., Fekete, A., Greening, T., & Hollands, O. (1999). Problem-Based Learning for Foundation Computer Science Courses. Retrieved February 27, 2015, from www.cs.usyd.edu.au/~judy/Teach/cse_pbl99.pdf Biggs , J., & Tang, C. (2007). Teaching for quality learning at university. Berkshire, UK: Open University Press. Boud, D., & Feletti, G. (1998). The Challenge of Problem Based Learning. Kogan Page. Cope, C., Staehr, L., & Horan, P. (2002). Towards Establishing the Best Ways to Teach and Learn about IT. In Challenges of Information Technology Education in the 21st Century (pp. 57-84). PA: Idea Group Publishing. Dolmans, D. H., Snellen-Balendong, H., Wolfhagen, I. H., & Van der Vleuten, C. P. (1997). Seven principles of effective case design for a problem-based curriculum. Medical Teacher(19), 185-189. Duke, M., Forbes, H., Hunter, S., & Prosser, M. (1998). Problem-based learning: conceptions and approaches of undergraduate students of nursing. , 3, 59-70. Advances in Health Sciences Education. Edwards, S., & Hammer, M. (2007). Problem-based Learning in Early Childhood and Primary PreService Teacher Education: Identifying the Issues and Examining the Benefits. Australian Journal of Teacher Education, 32(2). Etherington, M. B. (2011). Investigative primary science: A problem-based learning approach. Australian Journal of Teacher Education, 36(9), 36-57. Graaff, E. D., & Kolmos, A. (2003). Characteristics of Problem- Based Learning. International Journal of Engineering Education, 19(5), 657-662.
9
Karami, M., Karami, Z., & Attaran, M. (2013). Integrating problem-based learning with ICT for developing trainee teacher’ content knowledge and teaching skill. International Journal of Education and Development using Information and Communication Technology, 9(1), 36-49. Loyens, S. M., Kirschner, P., & Paas, F. (2011). Problem-based Learning . American Psychological Association, 2. Loyens, S. M., Magda, J., & Rikers, R. M. (2008). Self-directed learning in problem-based learning and its relationships with self-regulated learning. Educational Psychology Review(20), 411-427. Major, C., & Palmer, B. (2001). Assessing the effectiveness of Problem Based Learning in Higher Education: Lessons from the literature. Academic Exchange Quarterly, 5(4), pp. 4-11. Ozer, O. (2004). CONSTRUCTIVISM in Piaget and Vygotsky. Retrieved from http://www.psychology.org.au/publication/statements/substance/. Retrieved February 27, 2015, from http://www.psychology.org.au/publication/statements/substance/ Pawson, E., Fournier, E., Haigh, M., Muniz, O., Trafford, J., & Vajoczki, S. (2006). Problem based learning in Geography: Towards a critical assessment of its purposes, benefits and risks. Journal of Geography in Higher Education, 30(1), 103-116. Pepper, C. (2009). Problem-based learning in science. Educational Research, 19(2), 128-141. Problem-based learning. (n.d.). Retrieved February 27, 2015, from Problem-based http://www.buseco.monash.edu.au/futurestudents/undergraduate/studentexperience/problem-based-learning.html Savin-Baden, M. (2001). Problem Based Learning in Higher Education: Untold stories. Buckingham: Open University Press. Spencer, J. A., & Jordan, R. K. (1999). Learner centred approaches in medical education. British Medical Journal, 318, 1280-1283. Spronken-Smith, R., & Harland, T. (2009). Learning to teach with problem-based learning. Active Learning in Higher Education(10), 138-153. Stark, J. (2013). Optus rolls out new 4G network. Retrieved February 27, 2015, from http://www.smh.com.au/digital-life/mobiles/optus-rolls-out-new-4g-network-201309132to95.html Tan, O. S. (2008). Problem-Based Learning and Creativity,. Cengage Learning Asia. The Guide which follows offers illustrative examples of key variables to consider when contemplating the Design, Implemention, Evaluation or Revision, of undergraduate medical curricula. (n.d.). Retrieved February 27, 2015, from http://www.newcastle.edu.au/__data/assets/pdf Xiangyun, D., Graaff, E. D., & Kolmos, A. (2009). Research on PBL practice in engineering education. Sense Publishers, 2009.
10
