Trialling a Problem-Based Learning Approach to First Year Computer Science Tony Greening, Judy Kay, and Jeffrey H. Kingston Basser Department of Computer Science The University of Sydney { tony, judy, jeff }@cs.usyd.edu.au Kathryn Crawford SMITE Research Unit Faculty of Education The University of Sydney
[email protected] Abstract This paper describes the trial of a radically new first year Computer Science course using problem-based learning. It presents the rationale for the new course, a description of a successful trial undertaken in 1996, an analysis of the outcomes which compares the trial group to another group taught using a conventional lecture-based approach, and a few remarks on the first full implementation in 1997. 1. Introduction First year Computer Science at the University of Sydney used to be a fairly standard mixture of lectures and closed laboratories. After six contact hours per week for two semesters, the students had mastered Pascal to the extent of being able to write small programs. They had also learnt the basics of some Computer Science topics, mainly program correctness and efficiency, and a simplified assembly language. Dissatisfaction with this course and others like it in many Computer Science departments has several sources. Pascal is obsolete now and most would agree that a change to a procedural object-oriented language is urgent. But beyond this, staff in subsequent years are concerned that they are seeing students who lack such generic skills as independence and initiative, problem solving ability, the ability to apply theory to practice, and critical thinking, as well as more technical skills such as the ability to test and debug programs effectively. Employers are asking for students who are good communicators and able to work in a team. These are goals that were often not considered relevant to first year Computer Science in the past;but attitudes are changing. Our experiences with third-year project-based courses demonstrated that these desirable skills could not be merely appended to prior learning; they needed to be integrated into the coursework at an earlier stage. These problems are compounded by the monolithic structure of many first year courses. With many students (over 600 startersin our case), many classes,and several staff members taking
-2responsibility for different parts of one structure, it is very difficult to make changes, or to offer variants of the course tailored to students’varying backgrounds, abilities, and interests. With the growth of computing courses in high schools we are seeing an increasing number of students with some programming and other Computer Science knowledge, and such students can only mark time while the others catch up under the monolithic structure. Further evolutionary change, for example the replacement of Pascal by an object-oriented language without other changes, offers no prospect of solving all of these problems. Accordingly, we devoted a year to researching and developing a radical proposal for a new first year Computer Science course, in which the change to an object-oriented language would be accompanied by a new style of teaching: problem-based learning. The following extracts from a recent book [3] will serve for a definition of problem-based learning: Traditional (predominantly lecture-based) courses in higher education for the professions have been criticized for their lack of attention to such issues as the relevance of subjects, for placing little emphasis on encouraging teamwork, for poor attempts at developing skills of enquiry in students,and for inadequate protrayal of the context of major issues and problems. Problem-based learning addresses these criticisms headon and uses such deficiencies as the foundation of its approach… The principal idea behind problem-based learning is that the starting point for learning should be a problem, a query or a puzzle that the learner wishes to solve… Problem-based courses use stimulus material to engage students in considering a problem which, as far as possible, is presented in the same context as they would find it in ‘real life;’ this often means that it crosses traditional disciplinary boundaries. Information on how to tackle the problem is not given,although resources are available to assist the students to clarify what the ‘problem’ consists of and how they might deal with it. Students work cooperatively in a group or team with access to a tutor who is often not an expert in the field of the particular problem presented, but someone who can facilitate the learning process. This approach clearly has the potential to develop the generic skills of independence and initiative, problem solving, critical thinking, communication (with fellow group members, for example), life-long learning, and group work. The literature states that these skills are in fact found in the graduates of problem-based learning courses [1]. Group work in particular should reinforce the lessons of object-oriented programming by bringing to the students’ attention the software engineering issues that object-oriented programming addresses, such as the importance of loose coupling and clear interfaces. Problem-based learning is quite widespread in professional degrees, notably Medicine and Nursing. One influential author works in Chemical Engineering [12, 13, 14]. Papers describing work in professional degrees often emphasize the importance of integrating the entire degree programme by using problems which require students to draw together knowledge from many sub-specialties of their professional discipline. This seems less applicable in our case, since our course is taken by many students undertaking many courses of study, all of which are beyond our influence.
-3We are aware of a few other attempts to use problem-based learning in Computer Science. One is the Bachelor of Informatics degree at Griffith University, created about ten years ago [9]. Its approach seems very different from ours. For example, it emphasised the social context of computing as much as its technical core; and in first year, only one-third of the course was problem-based. Another is a Computer Science course taught in an Aeronautical Engineering department [6], very successfully according to the authors. Artificial Intelligence has also been taught using problem-based learning [10]. Several other recent initiatives in Computer Science education [2, 4, 5, 7, 11]contain elements of problem-based learning. To our knowledge,however, we are the first group to create an entirely problem-based first year Computer Science course. Section 2 describes our trial implementation of the PBL course, and Section 3 presents our results. These show that PBL students were as successful academically as their main group counterparts, while at the same time gaining valuable generic skills, and feeling more positive about the experience than main group students. Section 4 concludes with a few brief remarks about the first full implementation of PBL in 1997. 2. A Trial Implementation A trial implementation of the new structure ran in our Department during the 1996 academic year. This ‘PBL stream’contained about 40 students chosen randomly from a pool of volunteers; it was staffed by one of us and one regular CS1 TA. Instead of the three hours of lectures, one hour of recitation,and one two-hour closed laboratory per week taken by the main group, the PBL group took one one-hour seminar, one two-hour recitation, and one three-hour closed laboratory per week. To ensure that staff workload was not increased, the three-hour laboratory was staffed for only one hour except during the first few weeks of the year. During the first three weeks of the first semester,enrolment is too volatile to allow permanent groups to form, and students mainly need to familiarise themselves with the Department’s equipment and software. Two small problems were set for this early period: creating a home page, and developing a minimalist Unix manual, providing the sort of Unix starter kit that students perceived new first-year students as wanting, including a section on how to use the Pascal compiler and decode some of its error messages. Weeks 4 to 11 formed the main problem phase of first semester, in which students formed small stable groups and chose a problem to work on. Two problems were offered. One was an exercise in ‘classical cryptography,’ with a scenario that the Internet was being used to make available controversial information in the form of breakable codes which hid the information from casual users but made it accessible to those with the inclination to break the ciphers. Their task was to develop a code-breaking toolkit in response to this, plus create a cipher of their own design using the principles they discovered. Some encrypted files were provided as a starting point. The other problem was to develop a program which could be used in video shops to allow customers to get information about videos. A database of over 6,600 videos with substantial information on each was made available. Weeks 12 to 14 formed the demonstration and final reflection period, in which the groups were asked to present their work to others, and to review the learning process. Such ‘reflective’ experiences are considered important to deep learning and are a standard part of the PBL approach.
-4In second semester,the time was divided into two equal task periods, with students able to reform groups for each. In the first, students were encouraged to encounter advanced programming areas in Pascal, including the use of pointers and linked data structures. Problems from which students could choose included the development of an anagram machine, an intelligent checkers machine, a database system (of their choosing), a queueing simulator for supermarket checkouts, and a library for infinite precision integer routines. Alternatively, a topic could be negotiated. It was expected that students would encounter design difficulties in each of these problems, once they had been explored beyond the superficial level. Students were encouraged to explore those difficulties and learn from them, even where they were not fully overcome. Group reports and presentations were used in part to expound on potential improvements to their project in the light of their experiences. The second task period was used to explore the workings of an artificial machine architecture and instruction set as required by the content of the conventional course. This also introduced students to basic operating systems concepts. In order to preserve the student ownership of the task in this limited scenario, a problem was set which aimed to provoke some of the stimulation which came from assembly programming of early computers. Using the simulated machine with its scarce instruction set, very little memory, and its ability to use only integers, students were invited to implement an algorithm of their choice. Significance of the task was an important factor in the assessment of the project. Many of the tasks far exceeded expectations, including factorial algorithms, bubble sorts, and simple matrix manipulations! In addition to this, students were encouraged to also submit an optional project which could provide bonus marks to those received for the assembly language one. In this project, students were asked to choose any two CPU scheduling algorithms and produce code which would implement them and allow them to be compared. A presentation to the class would describe the workings of the algorithms chosen, the tests used to compare them, and their findings. Thus, students in the class were able to benefit from the inherent diversity of the task. Student ownership of the tasks in this second period in particular produced experiences with the prescribed content which far exceeded those imposed on students in the conventional curriculum. 3. Results Our trial implementation was complemented by an extensive evaluation undertaken by staff with professional expertise in the evaluation of higher education courses. Here we address three issues. First, we wanted some assurance that the PBL students were not less competent at programming than main group students. Second, we wanted evidence that the additional generic skills that PBL is designed for were in fact being imparted. Third, we wanted to know about students’ attitudes and perceptions towards PBL, since they can have a marked effect on learning. For ethical reasons it was necessary to select the PBL stream from volunteers, so we cannot rule out self-selection bias altogether. However, we can say that the PBL group is typical of the main group in ability, as measured by the Tertiary Entrance Rank (TER), a single number summarizing the performance of each student on the university entrance examinations. For the main group the average TER was 80.9 with standard deviation 9.2. For the PBL group the average TER was 81.2 with standard deviation 9.6, a statistically insignificant difference at the 5% level.
-5It was necessary to use Pascal in the trial since the students needed competence in it when entering second year, so the expected synergy between problem-based learning and object-oriented programming could not be assessed. On the other hand, students in both groups sat the same examinations, and these examination results provide an excellent basis for an objective comparison of the outcomes of the two groups. Our analysis of the results of the first semester examination found no significant difference between the performance of the PBL group and either a group from the main CS1 class matched on the background variables of academic achievement and previous computing background, or the main class as a whole. When the results of individual examination questions were analysed, one question was answered significantly better by main group students (not surprisingly, since that question related to a specific format used in lectures to show the contents of a data structure, which was not used in the PBL stream) and one question was answered significantly better by PBL students (for no clear reason). These results effectively answer concerns that eliminating lectures will reduce student learning. We have not analysed the second semester written examination results, but we have looked at the results of the second semester practical examination. This was a three-hour examination during which the students worked individually at a computer terminal on a previously unseen programming problem. Students who failed the examination had an opportunity to try again later. The percentage of students passing the first sitting of the practical examination was 48.5 for the PBL group (with average marks of 55% and standard deviation of 40.5%), and 48.1 for the main group (with average marks of 58% and standard deviation of 38%). The differences are not statistically significant at the 5% level. On the second issue, the learning of generic skills, we do not have quantative comparisons of the main and PBL groups; but in many cases there is no basis for such comparisons. PBL students were required to design, plan, implement, test, manage, and report on a large group software project. No comparable demands were made of students in the main group. This for us is the crucial advantage of PBL: that without losing any of the technical skills, it makes room in the course for activities that encourage generic skills. To address the third issue, student perceptions and attitudes, we conducted an open-ended questionnaire at the end of the first semester, in which we asked students to complete the statement ‘After one semester of computer science my attitude to this course is …’. Answers were coded on two scales: how the students felt they were engaged in meaningful learning, and their emotional response. The answers were analysed by a graduate student from the Faculty of Education at our university, under the direction of the author from that faculty. The PBL students were compared against the matched group from the main CS1 class. To analyse the students’responses from the point of view of learning attitudes, the responses were coded in the following classes:
-6Class
Example
A. No meaningful learning
“I feel I have great difficulties in learning this course. I consider the speed of everything too fast. I might drop out.”
B. As A but hopeful
“The work load is very heavy and I feel disheartened but I suppose if I kept at it, I’d find it a lot easier to cope.”
C. As A but improving
“It is pretty good. I find it difficult sometimes but once I have worked it out it is enjoyable.”
D. Meaningful
“Learning all the time! The best and most enjoyable practical course I have enrolled in so far!”
Conclusions must be cautious because of the high percentage of students who failed to respond. However, as indicated by the graph below, the largest proportion of PBL students who answered were in the meaningful learner category: No Data Code D Code C % of CS1-matched % of PBL
Code B Code A 0
20 40 Percentage of class
60
The second analysis coded the responses for their emotional content: Class
Example
A. Despair
“I feel very depressed and anxious about this course.” “I’m sure I’m going to fail and there isn’t anything I can do about it.”
B. Dislike
“I do not like computer science.” “Disheartened, it wasn’t what I expected and I can’t imagine working with computers any more.”
C. Bored
“It is very boring.” “This course is very slow and dull.”
D. Hopeful
“Sometimes I feel overwhelmed by the work, but I feel better when something goes right.”
E. Positive
“Positive, I think I’m going to make it.” “It’s good overall, I feel better when everything is going well.”
F. Great
“I love this course, it is my favourite.” “This course is far more varied and enjoyable than I would have imagined.”
The results are
-7No Data Code F Code E Code D Code C % of CS1-matched % of PBL
Code B Code A 0
20 40 Percentage of class
60
Once again, we need to note the large number of non-responses. For the others, we can see that the more positive emotions are expressed by larger proportions of the PBL group. Similarly, smaller proportions of the PBL students expressed negative emotional responses. In interpreting these comparisons it must be remembered that the PBL course was a first prototype. It suffered from several teething troubles that did not afflict the mature, established course. On the other hand, staff enthusiasm for the new course was very high. 4. Conclusion This paper has described the successful trial of a problem-based first year Computer Science course. In essence, problem-based learning •
Discards lectures in favour of learning in the context of solving problems with a ‘real life’ flavour;
•
Cuts across traditional disciplinary boundaries to integrate knowledge from many sources;
•
Emphasises generic skills such as group work, independence and initiative, planning, and problem-solving.
Excepting the second point, which is beyond our reach, our course does capture the spirit of problem-based learning. In late 1996 our Department judged the trial to be successful and resolved to adopt PBL for the main class in 1997, along with the Blue object-oriented programming language [8]. Although some minor adjustments were made (and continue to be made) with various aspects of the course, the 1997 offering is substantially the same as that described above for the 1996 trial. At the time of writing (three-quarters of the way through the 1997 academic year) the new course has proved very successful and has been accepted for 1998 and beyond. We have had to contend with a few problems in 1997: no Blue text book could be created in time, the Blue programming environment does not yet handle multiple simultaneous users of one project, and student numbers jumped 30% to 800 starters. It would have been very difficult to cope with these extra students under the old structure, which assumed one student per terminal during
-8scheduled laboratories. Acknowledgements. This project was supported by a Teaching Development Grant from the Faculty of Science at the University of Sydney. The results reported in this paper were obtained with the assistance of Sharon Dowsett and Kirsten Hogg. References [1] M. A. Albanese and S. A. Mitchell. Problem based learning: a review of literature on its outcomes and implementation issues. Academic Medicine 68, 52–81. [2] Catherine C. Bareiss. A semester project for CS1. In Proceedings 27th SIGCSE Technical Symposium on Computer Science Education, pages 310–314, Philadelphia, Pennsylvania, 1996. [3] D. Boud and G. Feletti. The Challenge of Problem Based Learning. Kogan Page, 1991. [4] Curtis A. Carver, Richard A. Howard, and William D. Lane. A methodology for active, student controlled learning:motivating our weakest students. In Proceedings 27th SIGCSE Technical Symposium on Computer Science Education, pages 195–199, Philadelphia, Pennsylvania, 1996. [5] Suzanne W. Dietrich and Susan D. Urban. Database theory in practice: learning from cooperative group projects. In Proceedings 27th SIGCSE Technical Symposium on Computer Science Education, pages 112–116, Philadelphia, Pennsylvania, 1996. [6] I. Hirmanpour, T. Hilburn, and A. Kornecki. A domain centered curriculum: an alternative approach to computing curriculum. In Proceedings of the 26th SIGCSE Technical Symposium, 1995. [7] Raymond P. Kirsch. Teaching OLE automation: a problem based learning approach. In Proceedings 27th SIGCSE Technical Symposium on Computer Science Education, pages 68–72, Philadelphia, Pennsylvania, 1996. [8] M. Kölling and J. Rosenberg. Blue – a language for teaching object-oriented programming. In SIGCSE Technical Symposium on Computer Science Education, 1996. [9] S. E. Little and D. Margetson. A project-based approach to information systems design for undergraduates. Australian Computer Journal 21, 130–138 (1989). [10] L. Cavedon, J. Harland and L. Padgham. Problem based learning with technological support in an AI subject: description and evaluation. In Proceedings of the Second Australian Conference on Computer Science Education Conference, pages 191–200, Melbourne, Australia, 1997. [11] Massood Towhidnejad and James R. Aman. Software engineering emphasis in advanced classes. In Proceedings 27th SIGCSE Technical Symposium on Computer Science Education, pages 210–213, Philadelphia, Pennsylvania, 1996.
-9[12] D. R. Woods, J. D. Wright,T.W. Hoffman, R. K. Swartman,and I. D. Doig. Teaching problem solving skills. Engineering Education 66, 238–243 (1975). [13] D. R. Woods and R. J. Sawchuk. Fundamentals of chemical engineering education. Chemical Engineering Education, 80–85 (1993). [14] D. R. Woods. Problem-Based Learning: How to Gain the Most from PBL. McMaster University Bookshop, Hamilton, Ontario, 1994.
