Experience with a Project–Based Approach to Teaching ... - CiteSeerX

Experience with a Project–Based Approach to Teaching Software Engineering M.J. Oudshoorn and K.J. Maciunas Department of Computer Science University of Adelaide Adelaide SA 5005 AUSTRALIA Section 3. The feedback from employers and from past graduates is somewhat limited at this point in time given the relatively small period of time that the course has been offered, but what feedback is available is discussed in Section 4. Section 5 discusses potential changes to the course. Some conclusions are provided in Section 6.

Abstract This paper reviews the experiences of teaching software engineering at the University of Adelaide using a group-based project over the course of one semester (15 weeks). Students are required to design, prototype, implement, maintain and document the software to satisfy the requirements defined by the lecturing staff who endeavour to simulate a customer as closely as possible. This paper focuses on the project and the benefits it offers rather than the content of the lectures.

1

2

Software Engineering and Project course structure

The subject consists of some 24 lectures covering topics such as group dynamics, project management, verification and validation, design, quality assurance, modern software engineering environments, safety-critical and real-time software engineering. The lectures are accompanied by tutorials which are used to guide the students through the project. The programming project forms the major learning experience of the subject. It is used to teach software engineering principles and to get students to experience software engineering issues associated with real-life software projects. Lectures are used to guide the students through the project and to consider software engineering topics that may not be directly relevant to the project. The subject represents a 25% workload for a single semester. One of the objectives of the course is to ready the students for the workplace and so great care is taken to simulate the work environment as closely as possible within the confines of a university department. Students are randomly assigned to groups to force them to work with students other than their friends and hence experience group dynamics [3,5]. Students are free to adopt any group structure they feel will work best for them. Over the course of the semester they informally relate their difficulties and successes to their fellow students and so information is shared in a manner that is far more effective than lectures. Students are also encouraged to help each other (even their friends who are in other groups) so as to broaden their experiences – interestingly, this does not affect the competitive nature of the project as each group still aims to win the Rational Prize. The project has a number of phases that require students to hand up requirements analysis, design documentation, user documentation, demonstrate their prototype and final

Introduction

Software engineering is an integral part of the computer science course at the University of Adelaide. Software engineering-in-the-small is emphasised in a variety of subjects throughout the undergraduate curriculum, however software engineering-in-the-large was only introduced into the undergraduate curriculum in 1992 by way of a distinct final year subject. This subject, Software Engineering and Project, is a one semester course (15 weeks) and it focuses on the issues associated with the lifecycle of large software systems. Students are required to work together in groups of 5–6 to develop a moderately complex software system (about 15,000 lines of code or 450 man-hours); students must understand the problem domain, design a solution, build a prototype, undertake some maintenance (due to a late change in the requirements), and provide appropriate documentation. This project forms a major part of the assessment for the subject. Rational Corporation1 have kindly donated a prize which is awarded annually to the best student project; this ensures the students interest and provides a competitive atmosphere in the class. This paper examines the project component of the course, Software Engineering and Project, looking at its structure in Section 2, and the benefits that it provides the students from the perspective of both staff and students. in 1

The University of Adelaide thanks Rational Corporation for generously providing the Rational Prize in Software Engineering which is awarded annually to the students in the group which produce the best project.

1

forced to draw on their experiences in other subjects. Projects are also sufficiently large and open-ended that it is unreasonable to undertake them on an individual basis. Projects are selected to encourage students to learn new things for themselves; they should realise that Computer Science is a discipline that interacts with many other disciplines. Projects have included a referencing system similar to EndNote [2] (1992), software to simulate a range of personal organisers [4] (1993), and a building management system responsible for the control of air-conditioning systems and fire response systems2 (1994). All projects are written in Ada, on unix platforms with the X windows graphical user interface.

software product. As a result, there is significant contact between staff and students. The lecturing staff involved in the subject (typically there are 3 or 4), also act as the customer. If a student has a query, they are encouraged to seek an answer from staff members. Staff deliberately do not discuss with each other how they perceive the student project and, hence, students are provided conflicting demands and advice — further simulating the real-world. Students find this behaviour frustrating initially, but soon learn to cope. Staff also change the requirements placed on the software after the prototype has been demonstrated. Students are told at the outset that the requirements will be changed to encourage them to carefully consider the design of their software, although they have no idea what changes will be required. The changes are not minor and are requested some two weeks before the final deadline. All deadlines are specified at the outset and students are informed that they are non-negotiable. CASE tools are not used throughout the project and discussion of such tools is deferred to the final lectures. This is done for pedagogical reasons so students are forced to deal with the issues that CASE tools address and hence gain an appreciation of what a CASE tool can, and can not, do to assist in the development of software. Students are, however, strongly encouraged to utilise support tools which will assist them in managing their projects. Tools, such as RCS [6], are considered essential and students are expected to make use of such tools. The use of support tools is encouraged because students have already encountered the difficulties that these tools address and appreciate the benefits of using them. It is also important that students realise that tools are an integral and important part of software engineering. The conflicting desires of wanting the students to understand the difficulties addressed by a tool, and realising that tools are essential when developing large complex systems, is difficult to address. A decision was made to class the tools into two different categories, namely support tools and CASE tools, and encourage use of the former while delaying discussion of the latter. This strategy works well and assists students in the transition from programming–in–the–small to programming–in–the–large. There is a deliberate attempt to select projects that are realistic, both from the real-world perspective and the abilities of the students. Time constraints (15 weeks are available for teaching in one semester) mean that the projects can not be excessively large, but it is essential that the project have sufficient complexity to force the students to design solutions and think about what they are trying to achieve. One way of identifying real-life projects for the students is to invite local industry to participate and provide projects [1]. The nature and aims of the subject tend to make selection of industrial software projects somewhat difficult, as a result projects are determined by the lecturing staff and vary each year. Care is taken to select projects that are real to the students and contain sufficient complexity that they are

3

Evaluation and expectation

3.1 Student expectations Students come into the subject expecting to have a significant programming assignment. They have heard from students of the previous year that the course is interesting and challenging; they are consequently eager to start. Once the project details are provided to the students and they start to define the project for themselves, some realise that there is more involved in this subject than just a programming assignment: this excites some students while others are not prepared to work through all of the aspects of the project and withdraw from the subject. At this point groups may need to be reorganised giving some students the opportunity to welcome a new member to their group and forcing them to explain their designs, and so forth, to the new member. Once students realise that there is more to software engineering than mere coding, most recognize the value of the project, the amount of work involved, and the benefits that the course offers them. Many students enter the course believing that they are capable of winning the Rational Prize in Software Engineering. Students are keen to win the prize ($1,000) not for its monetary value, but because the university records such official prizes on the students academic transcript. Most students see this as a mechanism to secure an interesting job. As the project gets underway, some student groups design overly ambitious solutions which they have little chance in completing over the semester. Once they realise that the prize will be awarded for overall performance in all aspects of the project, including quality of documentation, testing, adherence to design and so forth, they tend to revise their designs to be more realistic and put significant effort into the design, testing and documentation. The result has been that the aspects of software engineering that students tended to ignore 2

Our thanks to the Adelaide Metropolitan Fire Service for helpful advice and information regarding the design and layout of modern office buildings and their fire systems.

2

Staff involved in lecturing the course have given careful consideration to the above complaints. One of the aims of the course, namely preparing students for the workplace, necessitate the need to apply pressure to the students. This is somewhat difficult to do with a smaller project. Students who have completed the course have indicated that they are pleased that the pressure is applied during the course as it has eased the transition from tertiary study to a place in the workforce.

or believe were unimportant, becomes a focus over the course of the project. Since the project runs over the length of a semester, the design, testing and documentation phase of building a software product becomes a habit and slowly the students become software engineers rather than just programmers. After the course, the students design before coding, test more thoroughly than ever and document their programs – all things that are encouraged in other courses (but because of the small exercises, students are able to get away with a half hearted attempt). Since these are not new concepts to the students, they look back on the project as a significant programming exercise with more emphasis than usual on these features. Most students cite that the principle benefit of the course is learning to interact and manage people. Even students that have failed the subject have commented that they have learned a great deal by having attempted Software Engineering and Project. The students, although appreciative of the course once it is completed, are not without complaints and criticisms. The major source of complaint is the fact that the requirements are changed toward the end of the semester. Although the students are told that this will happen, and it is emphasised that they must “design for change”, there is a general feeling of annoyance and anger when the change is announced. For some groups, the change is a trivial affair and once this is realised, the anger quickly subsides. For other groups the change signifies a major rewrite of parts of their system — these are the groups that did not design for change. Most students, including those in the latter groups, reflect on the course on completion and appreciate the fact that the requirements are changed. Another source of complaint is the timetable set for the submission of milestones in the project (requirements analysis, functional specification, prototype, documentation, final program and demonstration). Students are informed of the timetable at the beginning of the semester and told that the submission dates are non-negotiable. As the semester draws to a close, students find that, through mismanagement of their time, they have several assignments due in a short space of time. This is a direct result of ignoring advice offered to them in all their undergraduate courses — start early. It is a complaint met with little sympathy. The final complaint is related to the above: the amount of work involved is excessive. This again stems from the fact that many students delay any work on the project until the last possible minute and then they overestimate their own abilities and underestimate the amount of time it takes to complete a task. Accurate time estimation is a difficult skill to learn, and a difficult one to teach. Students’ past experience with shorter assignments does not lend itself to well to time estimation for a larger, more complex project. Once again, advice offered at the beginning of the course is ignored.

3.2 Staff observations Staff expected students to find the course difficult and challenging in many ways. Students have little experience with group projects so difficulties managing themselves into an efficient work unit was expected. Many students overestimated their own abilities and found difficulties in adhering to deadlines or realistically estimating the amount of time required to complete a certain task. Weaker students tend to shy away from writing code and volunteer for tasks perceived as less technical such as writing the user manual. The predominant problem encountered in each of the groups was a tension between the student who assumed the role of group leader and the remainder of the group. Group leaders generally complained of poor performance from some group members, and group members often complained of poor leadership, or unreasonable expectations. Students are not permitted to change groups and are forced to address and deal with these problems. The projects are chosen so that a group of 5–6 students should be able to complete the work in approximately 75–90 hours each. Students claim that the actual time required (they are asked to keep timesheets) is in the order of 100–170 hours each over the course of the semester. Students have several deadlines during the course of the semester and their progress is paced by these deadlines. Examination of the students timesheets has shown that significant time is wasted in meetings and in attempts to merge together work completed by different people. It has also been observed that as the course progresses students become less inclined to leave the bulk of the work until the last minute. Submitted work is marked and returned to the students within a week so that they have the chance to take any corrective action deemed necessary. Despite the problems that the students encounter, each year they have risen to the challenge and have surpassed staff expectations of their performance. The quality of the submitted work (programs and documentation) has been of a high professional standard. In many instances, the user documentation submitted has been of comparable, or better, quality than the documentation accompanying many commercial software products. The principle difficulty for staff is dealing with the number of students involved. The subject typically has 120–160 students enrolled in it – this represents some 30

3

milestones in terms of dates by which a requirements analysis must be submitted, a prototype demonstrated, and so forth. This provides no suggestion or hints as to what software architecture the students should adopt. The assumption made by staff was that students would be able to take concepts learned in these courses and apply them in different domains and situations. It was found that several students were unable to take what they had learned in the previous courses and apply this knowledge in a different context. It was clear that the students knew about the data structures, but they seemed to lack a sufficiently complete understanding of the data structures to see how their use was applicable in the domain of the software engineering project. This seems to suggest that the students gain insufficient practice in using the data structures. In particular, students seem to require more practice in choosing an appropriate data structure when writing a program. The lack of real-world experience was expected; it is, after all, what the project is attempting to address. All projects selected to date have been chosen on the grounds that they are within the comfort zone of all students — they have all written reports and had to cite papers (referencing system, 1992), they know what a diary is used for (personal organiser, 1993), and they all inhabit buildings with firedoors and smoke detectors (building management system, 1994). In each case, the students were expected to think about the problem and the problem domains contain sufficient complexity to be interesting. In the years when a working sample solution was made available, students had a firm grasp of what was wanted and what was considered important. When not provided with a working solution, some student groups tended to focus on irrelevant details. It seemed that they were unable to distinguish between the essential aspects and the gratuitous features of a system that just make it nicer or easier to use. Several groups seemed to fall into the trap of concentrating on the “bells and whistles”. Since the software engineering project is the first large and complex software system3 written by the students, they naturally tend to treat it in the same manner in which they have treated previous programming assignments; namely, they focus on issues relating to programming–in–the–small. This is despite the fact the the initial lectures focus on the distinction between programming–in–the–small and programming–in–the–large. When faced with greater design responsibility, students tended to ignore advice from lectures and did not step back to “look at the big picture”. Consequently, poor design decisions were made which were ex-

student groups. This is a significant load because of the high level of student contact in this course. We have found that 3–4 staff is optimal for dealing with such class sizes. It ensures that everyone is (very) busy and provides the students with a choice of “customers” to interview. It also ensures that the necessary marking can take place in a timely fashion.

3.3 General observations Each year that the course has been offered, a different project has been chosen. The projects initially chosen (referencing system and a personal organiser) were selected because existing software was available to use as an example. Students were provided with access to the documentation and the software and encouraged to experiment with it. The software acted as a demonstration of the system similar to what was wanted by the customer. This avoided the need for the lecturing staff to be interviewed by each student group to explain what was required. It was stressed that the demonstration software was only a guide to what was wanted. Students were encouraged to expand on this theme or focus on particular aspects as they felt fit. This allowed each group to develop a different solution and, hence, prevented the students from plagiarising a solution from another group. It also means that the students are free to develop a solution in ways that they find interesting; this motivates them to put in the necessary work to produce a complete solution to what they perceive to be the problem. In 1994, the selected problem (building management system) did not have a pre-existing solution. A vague document was produced and provided to the students which outlined the problem. Discussions with staff helped the students refine the problem. This departure from previous practice gave the students much greater freedom and placed greater emphasis on the engineering and design aspect of the project since the problem specification was not as tight as it had been in the past. The additional control and creative input available to the students served to highlight a number of problems with the undergraduate teaching in other subjects. These problems had been masked in the years where a representative solution was made available to the students to experiment with. The difficulties highlighted involved the students’ understanding of data structures, their lack of real-world experience, their focus on programming–in– the–small, and an excessive confidence in their own ability. All students must have completed the second year course on data structures and it is assumed that they complete the course of advanced programming techniques before attempting Software Engineering and Project. In order to ensure some consistency in the programming exercises in these subjects, students are often provided with a program skeleton, or a guide as to what the architectural structure of the solution should be. This is often implicit, rather than explicit, and achieved by setting milestones which students must achieve. The software engineering project only sets

3

Most students undertake the subject Compiler Construction and Project before undertaking Software Engineering and Project. Compiler Construction and Project involves the development of a complete compiler, but students are provided with a skeleton program and significant hints on the architectural structure.

4

more pre-defined functions to the building definition language. For those groups following the advice offered, this simply meant writing five more small procedures; for those groups who insisted on writing their own parsers, this represented a significant workload. There is a suspicion that part of the difficulty here is that the students feel a pressing need to start coding as soon as possible. In order to achieve this goal, minimal thought is given to the consequences of their actions. Those groups which start coding the earliest, tend to be the groups that develop the code with the greatest resistance to change and produce the greatest volume of code to achieve even the simplest of tasks.

tremely difficult for lecturing staff to detect when reviewing the requirements analysis, functional specifications and so on. This resulted in some groups being able to barely piece together a prototype to demonstrate and, later, being unable to demonstrate a complete working solution. The major difficulty experienced here was an inability to integrate the various software components built by different group members. The groups which suffered from this problem typically changed the structure of a software module (and its interface) after the functional specifications were handed up and failed to discuss the changes with other group members affected by the change. Many student groups refused to seek significant help when they recognised that things had going wrong. This appears to be related to a belief that seeking large amounts of help would adversely affect their grades. The hint to lecturing staff that such a problem exists is the focussing of student questions on minute, often irrelevant, details of the implementation aspects of the project. The final major problem noted is the existence of the “not–invented–here” and “not–needed–here” syndromes. There seemed to be a need for some students to exhibit a “macho” image by avoiding the use of support tools, such as RCS, and a refusal to write self-checking code. When challenged about the lack of self-checking code, some students indicated that they “knew their code was right”, the student wrote their own code after all! In some cases, this meant that bugs remained dormant and unnoticed until demonstration of the final product, in other cases, this meant that some groups spent inordinate amounts of time tracking down bugs that “should not have been there”. The refusal to listen to advice offered in lectures early in the course proved costly for these groups — although the students involved now appreciate self-checking code more than ever. The avoidance of tools such as RCS is a temporary problem which is rectified as soon as the first student deletes a file and the systems programs refuse to restore the file from the backup tapes. This horror story travels through the class quickly and RCS usage increases accordingly. Students are also reluctant to use tools in novel ways: this was highlighted in 1994 when a declarative Ada–like building description language was specified for the project (this as so staff could design test data for projects on final submission). It was suggested to students that they might like to use the Ada compiler to parse the building definition language. This simply involved generating an Ada procedure by writing a wrapper for the building definition program. Using this approach, pre-defined procedures of the building definition language could be implemented as external Ada procedures. Execution of the generated Ada procedure needed to produce a data structure which represented the building and its layout (location of walls, sensors, firedoors, etc). Many students ignored this advice, preferring instead to develop a complete parser to process the building definition language and generate the appropriate data structure. The change to the requirements in 1994 was to add five

4

Employer and graduate feedback

The subject has only been offered for a relatively short period of time and, hence, only limited data is available. Generally speaking, employers are impressed with the work that the students are asked to undertake in the project and have commented that the documentation that the students are showing at job interviews is impressive and covers all aspects of the software lifecycle. Employers are actively seeking students who have completed the subject when they are recruiting graduates. Several recent graduates have commented that the subject was instrumental in securing employment. Students have pointed out that their experiences in the course of the project are very similar to the experiences that they are experiencing in the workforce. The majority of students are pleased to have undertaken the subject, even though it has a heavy workload, because they believe that it provides them with all the necessary skills required in the workplace. At present, only graduates who have found employment in Adelaide have come forward with their views on the subject. It is not yet clear how interstate employers have reacted to the subject, or how helpful the subject has been to students seeking employment interstate or overseas.

5

Future changes

Projects used in 1992 and 1993 involved fairly straight-forward coding. The 1994 project dealing with building management involved aspects safety-critical programming, and to a lesser extent, real-time programming. The lecturing staff would like to emphasise these aspects further and have secured a University of Adelaide grant4 to purchase a computer controlled train set. The 1995 project is likely to involve students managing a number of trains circulating the track in such a way that collisions must be avoided and trains must travel on designated routes and adhere to some timetable. Students will need to interact with the control unit which interfaces to the train set to pass instructions to 4 Grant through the Quality Audit Funds Teaching and Learning Initiatives Scheme held by A. Brown and M.J. Oudshoorn.

5

opinions on issues, serves to guide the students into asking much more refined and detailed questions. It is clear that the course as currently presented is not perfect, but despite the difficulties it suffers from, it is a course that must be designated as successful. Students enter the course with expectations that completion of the course will assist them in finding a job and they are better prepared for the workforce. Staff teach the course hoping to pass on basic software engineering principles and prepare students for employment. Students exit the course with basic software engineering expertise and experience; they are more mature and better prepared for the workforce, having at least some idea of what life outside university might be like. Students are also more aware of the amount they do not know and have a greater awareness of their own limitations based on their successes and failures during the project. The general feeling of graduates, current students and staff is that the course is successful in meeting its objectives. It is a highly regarded subject and considered essential by students seriously wishing to work in the information technology industry. Despite the fact that no modern CASE tools are used by the students to develop their projects for pedagogical reasons, students are also able to quickly grasp the impact and utility of these tools when they are demonstrated and discussed at the end of the course. This course amply demonstrates the academic benefit of a large (carefully chosen and controlled) project in the teaching of this subject.

the trains and signals, identify which sections of track are occupied and by which train, and so forth. Such a project provides the increased focus on safety-critical and real-time programming. Students would need to simulate the train system while developing the software and would only gain access to the train set when they were convinced they had a working system. This emphasises the fact that their software may one day control potentially life-threatening devices and they must take responsibility for software failure. Another advantage of a train-based project is that it involves interaction with physical devices and helps unify the material taught in different computer science subjects. Any project involving the train set is likely to be perceived by the students as “real” and, hence, stimulate their interest. Australian National Railways may be able to provide examples of real-life problems for the students to tackle, or provide a guest lecture to give students a deeper insight into workings of a rail system.

6

Conclusions

The subject as described is very successful both from the perspective of educative outcomes and popularity and this success stems from the project component. In addition to the very desireable educative outcome, students are also well prepared psychologically for the job market place – many students have commented that the subject not only teaches them something, but that they feel that they really know what it has taught them and they felt much more “mature”. Students have found the subject to be beneficial to them in putting their computer science knowledge in perspective rather than considering the subjects they have studied as disjoint. Students also use the documentation produced in the course of the project (requirements analysis, design documents, user manuals and software maintenance documentation) to convince employers that they are employable and competent. The subject is highly regarded by both the students and the employers in this respect. The lectures in the course are structured towards the project and serve to guide students through the project and to amplify some aspects of the text book [3]. The system in which the lecturing staff take the roles of both customer and mentor appears to function well and, by giving differing

7

References

[1] P.J. Knoke, “Medium Size Project Model: Variations on a Theme”. In J.E. Tomayko (Ed.), Software Engineering Education, SEI Conference 1991, Lecture Notes in Computer Science, Volume 536, 1991, pp. 5–24. [2] Niles and Associates, EndNote Plus, 1991. [3] R.S. Pressman, Software Engineering. A Practitioner’s Approach, Third Edition, McGraw–Hill, 1992. [4] Rupp Corporation, Organizer Link II Software, 1992. [5] S.R. Schach, Software Engineering, Second Edition, Aksen Associates, 1993. [6] W.F. Tichy, RCS – a System for Version Control, Software–Practice and Experience, Volume 15, Number 7, July 1985, pages 637–654.

6