Reporting about Industrial Strength Software Engineering Courses for Undergraduates Matthias Gehrke, Holger Giese, Ulrich A. Nickel, Jörg Niere, Matthias Tichy and Jörg P. Wadsack
Albert Zündorf
Software Engineering Group Department of Mathematics and Computer Science University of Paderborn Warburger Straße 100, D-33098 Paderborn, Germany
Institute Of Software Technical University of Braunschweig Gaussstr. 11, D-38023 Braunschweig Germany
[mgehrke, hg, duke, nierej, mtt, maroc]@uni-paderborn.de
[email protected]
ABSTRACT
the trainees fulfil only a limited role in an already running project.
How do you organize an “industrial strength”, one semester educational programming project for up to 200 second year students? This paper reports on four years of experience with such projects at University of Paderborn and University of Braunschweig. Key properties of our project design are: starting with an existing large application, regular hard deadlines with peer reviews and presentations to a large audience, working in groups applying project and configuration management tools, a standard system architecture with interchangeable components and competing software agents, quality assurance and standard conformance test through final overall system integration spanning all groups, and exposure to real-world project threats.
To provide reasonable practical project experience covering all project phases and all kinds of software development activities, an educational software development project for undergraduate students has been integrated in the curriculum of the University of Paderborn and the University of Braunschweig. Such a project runs in the second year of education before students have to perform an additional industrial placement. It practices main software engineering concepts and prepares students for their industrial placements and for their first jobs.
Categories and Subject Descriptors K.3.2 Computer and Information Science Education General Terms Education, Experimentation Keywords Software engineering education, educational programming projects, real world requirements, large project size, hard deadlines, exposure to real-world projects threats
1. MOTIVATION Within the growing IT market, applications become larger and the complexity increases dramatically. This forces the need for more and well trained software engineers and computer specialists. In addition to profound knowledge of software processes, design languages, etc., for software engineers a high demand on practical experience emerges. Typically, practical experiences are gained by industrial placements. However, those hand-on training practices are only a part of a software development process because usually
These goals create several teaching challenges. How do you organize an one semester educational software development project for some hundreds of second year students? How do you achieve maximal practical experiences for the students? How do you enforce certain software engineering and quality standards? And, last but not least, how do you motivate students to write umpteen documents and to deliver high quality software? Some years ago, our group became responsible for organizing the mandatory second year students programming project at the University of Paderborn. Our goal was to redesign the existing small programming projects towards more realistic “industrial strength” projects preparing our students for their future jobs and teaching them main software engineering principles. Usually, second year students have experience only with little programming exercises such as linked lists, binary trees or sorting algorithms starting from scratch without teamwork requirements and without explicit analysis and design phases or documents. Within the first two years, software engineering education is restricted to design languages such as UML and to software development processes such as the V-Model or the Unified Process. The students lack experiences with larger projects, team work, project management and the application of software engineering techniques. To provide maximal practical software engineering experience within a four month educational programming project, we have developed a new project design employing a number of innovative concepts that turned this course into a valuable exercise for our students. Until now, this new project design has been applied for four years at the University of Paderborn and for one year at the University of Braunschweig with an increasing number of up to 200 students per year and University. In the next year we expect about 350 students for this course. The new project design has been a great success in motivating students as well as in the application
2
of software engineering principles in large projects. Thus, over the years it has become mature enough to be adopted by other sites. As stated in various other publications [MR99, Bot01, SSvdW99] the key achievements addressed by an educational software engineering programming project are experience with “large” projects exposing students to the corresponding project complexity. Students should work in a team to train social skills and learn about project management, the importance of a common, well-understood design and about coordination efforts. All students should participate in all project-phases. Each of them should have the responsibility to clarify a certain part of the requirements, for the corresponding analysis phase, for the design of the corresponding functionalists and for the implementation of them. The project should not start from-scratch but as in most industrial projects the project should start with an existing system that needs to be extended or modified. Instead of a stand-alone system, the resulting application should require the integration with several other applications or system parts to simulate projects involving multiple departments or enterprises. Students should use state-of-the-art CASE tools and programming environments as well as groupware and versioning tools. In addition, the development process should have hard deadlines with well-defined deliveries in order to experience budget and time restrictions. State-of-the-art quality assurance techniques such as a welldefined repeatable process, peer reviews and external reviews of all deliveries and the deployment of test (coverage) tools have to be achieved. We extended this list with some real-life experience, usually not part of artificial programming projects, like changing customers/ supervisors, changing requirements, changing team organisation etc. Students should learn to deal with customers in terms of acquisition, contract negotiations, cost estimations, presentations at different project stages, product delivery, etc. Other more technical issues are harddisc crashes, corrupted repositories or server shutdowns close to delivery dates or shortly announced additional project presentations. We were inspired by [Daw00] which presents “Twenty Dirty Tricks to Train Software Engineers” in order to simulate a ‘real-world’ environment. Besides all this, we have to monitor the contribution of each student and to measure their individual performance with respect to a comprehensible grading-scheme.
the corresponding complexity problems. Next, many authors emphasize the importance of real-world applications that are used by real customers in order to motivate students [MR99, AL00, Bot01, Vau00, SSvdW99]. Third, like most industrial projects we should not start from scratch but with some existing system to train reverse engineering techniques and programming comprehension. Starting with existing systems also facilitates to reach a reasonable project size. Fourth, special challenges are created by projects involving multiple parties, e.g. multiple departments of one enterprise or even multiple enterprises. Such projects rely on welldefined interface definitions and include extra challenges for system integration that should be experienced by students. On the other hand, this is a four month project and the students have additional courses on other topics and the students have limited programming and barely project experience. Thus, the project must be executable within this restrictions by students showing varying performance. In addition, it must allow to grade the performance of each student. During the last four years we have experimented with different applications for our educational programming project. Some of the most successful applications are now described and their properties with respect to our teaching goals are discussed. Kitchen Designer One of our first applications was the Kitchen Designer. The aim of the Kitchen Designer project is to provide a software product to salesmen, which allow them to present virtual kitchens to their customers. This software product has to enable the salesman to design new kitchens or change existing ones. The possibility to add new furniture or to change the colour or the shape of it has to be provided. Furthermore the kitchen has to be presented from different view angels including shadows to present a realistic image of the kitchen. The Kitchen Designer has two main parts, a GUI and a rendering (raytracer) algorithm. The basis for our first project in 1998 was a Kitchen Designer where the GUI was implemented in Java and the application logic in C++. The exercise for each student group was to redesign the raytracer algorithm and to implement it in Java, to obtain a pure Java software product. Furthermore they had to add some new predefined furniture to the GUI.
In the next section we introduce the architecture of the applications we use in our educational projects. This is continued by a discussion of project phases and the project repository in section 3. Section 4 reports on how to motivate students, how to achieve high quality work, how to control student’s performance and how to grade students. Section 5 provides some feedback from our students. A comparison with other educational software development projects and additional related work is provided by section 6. We conclude with some lessons learned in section 7.
2. CHOOSING AN APPLICATION The first problem in setting up an educational programming project is to choose an appropriate application. Due to our experience this choice is crucial with respect to many of our teaching goals. First of all the application must be large enough to expose students to
Figure 1. Rendered Kitchen Scene
3
The project in 1999 was based on the results of one group in 1998. The application was the same but we added new major requirements. This time the groups had to add a macro functionality to the kitchen designer, i.e. the possibility to group objects such that they can be handled as one object. Furthermore they had to implement two other functions such as editing (undo/ redo) functionality, hiding objects, TCP/IP connection to distribute the rendering algorithm, help-items, simulation features, progressive image rendering, etc. Some groups chose other functions, e.g., visualisation with Java3D or vector graphic representation. Figure 1 shows a kitchen scene rendered during this project. The kitchen designer project has several properties making it appropriate for educational programming projects. First of all, the application is very motivating for students. They get direct visual feedback of their implementation. In addition, graphical rendering may be improved within several steps. First objects are just drawn under different view points. Next, light and shadow may be added. Then different kinds of reflecting or absorbing surfaces may be introduced. Finally, challenges such as partly transparent objects with different light refraction or water surfaces may be added. Each of these improvements can be easily seen in different kitchen scenes. There are numerous pitfalls that are easily uncovered by customer/supervisor provided kitchen scenes. Thus, it was easy to set up a competition between different student groups and to evaluate the quality of the developed system. In addition, the application is scalable for student groups with varying performance. Poor groups implement only a basic renderer, good groups implement more sophisticated features. This mitigates the risk of unfinished projects causing whole groups of students to fail the course. Another important point is, that graphical rendering needs some theoretical background. Students can not just sit down and start hacking. They have to study the concepts of graphical rendering and to do some analysis and design before they start with the implementation. Thereby, students learn about the value of an explicit analysis and design phase.
or after a pre-defined time limit. Example of arenas are shown in Figure 2 and 3. In the first year (2000) the interface between the robots and the arena was pre-defined and the same for all groups. Each group had to implement an arena and a robot in Java which runs in each other arena. A number of restrictions were made for the arena and the robot to achieve a fair competition at the project’s end.
Figure 2. Sample Arena from 2000 In the second year (2001) the Realtime Card-Game was based on the results of one group of the first Realtime Card-Game project. The exercise was changed such that the arena should be based on honeycombs instead of squares like in the previous project. Furthermore, as explicit requirement we added that the GUI has to be concise. Moreover, the interface was defined by the students themselves in a moderated interface group, which contains a delegate of each group.
The main problem with the Kitchen Designer is, that in later project phases the main development work focuses on the graphical rendering part. In this phase, usually only one or two students do the whole implementation work while the other group members just attend the meetings and prepare the presentations and documentation. Thus, we were looking for a more component oriented application that facilitates the splitting of work while maintaining the motivating effects of visual feedback and evaluation. In addition we wanted to address the system integration aspect, which is one of our teaching goals. Realtime Card-Game After two years, we changed the project to achieve a more balanced splitting of work within the student groups and an integration between the different student group results. Based on the opensource game Realtimebattle, we provided a core implementation of a Realtime Card-Game. The main idea is that robots (players) search cards in an arena composed of squares. The goal is to get five cards such that their combined value is maximal when the game finishes. The game ends when a robot takes the exit
Figure 3. Sample Arena from 2001 RoboRally In 2001, one of our staff members moved to the University of Braunschweig and had to organize a similar educational software development project for 190 second year students. There, several different applications were used by different groups of students.
4
We used one very successful project for 5 groups of about 6 students per group. Again, a robot game called RoboRally. Multiple robots have to make their way through an arena with various kinds of obstructions such as walls, holes in the floor, conveyer belts, one-way fields, etc. Each robot gets a number of randomly selected cards representing different move operations. Robots have to play their cards and to move accordingly in order to reach a certain target field. They have to develop a travel strategy to circumvent the obstructions and to deal with their competitors. A basic user interface was provided as a starting point. The students had to extend this user interface and to develop automatic agents to run the robots.
for storing and loading arenas, the agent strategy itself, etc. In addition, not all components are mandatory for a running prototype. At a first shot, a fixed simple arena a basic game engine and agent management component and a very simple agent suffice to fulfil the formal course requirements. Based on such a simple prototype additional components may be added and the agent strategy may be improved. Like for the Kitchen Designers this allows to scale the project for student groups showing varying performance and mitigates the risk of unfinished projects. Very important with respect to our teaching goals is that our example frameworks employ multiple competing agents. Each student group has to realize the same framework and (at least) one agent. This allows to run agents from different groups within one framework. Besides the mentioned motivating effects, this creates the necessity that all groups actually create compatible frameworks and agents. This requires a well defined interface between framework and agents and creates the challenge of integrating software from different student groups simulating similar integration problems of industrial multi party projects. We take this idea to the extreme by organizing the final project integration test phase as a tournament where each group’s framework works as competition area for all groups’ agents. We measure the performance of each agent and the most successful agent wins some certificate. However, the agent performance does not contribute to the students grade. Only the robustness and reliability of the framework and of the agent is of importance. Crashes of agents or frameworks contribute to the grades of the corresponding group. The idea is to grade quality properties higher and additional quick-hacked functionality lower. The tournament is very motivating for students: no one wants to have a lousy agent showing a poor performance and each framework needs to work reliably with a dozen agents programmed by foreign groups. Within the tournament, robustness and reliability of all components is reasonably tested and all groups are forced to meet certain implementation quality standards.
Figure 4. RoboRally sample Appropriate Application Structure The common property of the Realtime Card-Game and the Robo Rally application is a large component oriented framework employing multiple competing agents of similar kinds. In addition, both applications employ a direct visualisation of the system state and agent performance. As discussed for the Kitchen Designer, this motivates the students and facilitates the evaluation of the product quality. The component oriented framework architecture with competing agents has several additional advantages for an educational programming project. First, competitions between agents developed by different groups create additional motivation for the students. This will be discussed in more detail in the next section. Second, the component oriented framework allows to split work between group members more easily. Different group members may be responsible for different components such as the graphical user interface, the logical game engine, the management of the agents, an editor for different arenas, a persistency concept
As discussed, educational projects should not start from scratch but with an existing application. The project should involve major changes or extensions of that application. Within our game architecture, this is easily achieved by, e.g., taking a good result from one year as starting point for the next year and just to change several rules of the game and several other requirements as we did for the Realtime Card-Game. It turned out that the structure of a general component oriented framework employing multiple competing agents is very generic and that the setup of new projects based on previous projects becomes easy. Application Size One advantage of starting with an existing application is, that larger applications become tractable. Table 1 gives the average sizes of the initial legacy software and of the finally delivered software per group. We think that reasonable application sizes are achieved. The measurement of individual student’s contribution is discussed in another section. ‘PB xxxx’ stands for the projects at the University of Paderborn from 1998 to 2001 and ‘BS 2001’
5
denotes the project at the University of Braunschweig.
files / classes / # LOC provided
files / classes / # LOC delivered (average)
PB 1998
C++: 9/19/4133 Java: 66/118/10429
C++: 0/0/0 Java: 84/133/16101
PB 1999
85/134/16250
91/147/19606
PB 2000
15/15/1759
55/117/13117
PB 2001
51/145/23001
74/118/18298
BS 2001
11/14/1944
68/136/14909
Smaller groups minimize coordination overhead and more students take over central responsibilities and become major contributors. However, smaller groups are vulnerable by drop-outs of one or two group members and smaller groups imply generally smaller projects with less software engineering complexity. Table 3 shows the groups employed in our projects so far. We will discuss our choice in section 4.
# Students
# Groups
Students/ Group
PB 1998
55
7
7,8
PB 1999
53
8
6,6
PB 2000
83
9
9,2
3. PROJECT MANAGEMENT
PB 2001
102
12
8,5
Table 2 shows the project plan from the Realtime Card-Game project in 2001. A similar plan has been used for all projects at Braunschweig University. The weeks listed in Table 2 are the deadlines where the results are presented internally in each group.
BS 2001
194
34
5,7
Table 1: Software sizes
Week
Phases
1
Organisation/Orientation
2&3
Reverse engineering
4
Integration with new requirements
5
(Re)Design
6
Specification
7
Test presentation
8
Customer presentation
9, 10 & 11
Implementation
12
Code Review & Internal Test
13
Integration Test
14
Tournament
15 to18
Documentation Table 2: Project Plan (PB 2001)
Project initialisation In the first week the students are split in groups, a short, imprecise requirements description and the “legacy” application are given to them and other organisational stuff such as the document and source code repository initialization are done. Due to our experience, a good teamwork is crucial for a groups’ success. Some small percentage of groups drop-out of the course due to group problems. To minimize this risk, forming groups is mainly done by the students themselves. This facilitates the teamwork and collaboration within the groups. In the literature, different group sizes are discussed. Larger groups increase the demand for good project management and the learning curve for social skills, groupware and coordination efforts. However larger groups inhibit the risk of nominal group members that just benefit from the work of some active team members.
Table 3: Group sizes Reverse engineering phase From an educational point of view, starting with an existing application is more realistic since most industrial projects do so (cf. [Bot01], [SSvdW99]). The existing application provides a working framework that may serve as test bed for changes and new features. Most students do not appreciate these ideas. The majority of the students complains about the huge amount of code and about missing or poor documentation. (To achieve a realistic setting, we do not hand documentation to the students). Furthermore, the students complain about the poor reverse engineering capabilities of provided CASE tools like Rational Rose [Ros], TogetherJ [Tog] or Fujaba 1 [FNTZ98, NNZ00]. Mostly, they are unsatisfied with the results of those tools. The biggest problem for the students is to differentiate between the important and the obsolete parts of the code to reverse engineer. As delivery each group has to provide an architectural overview of the existing system using class diagrams. Furthermore, they have to outline the main methods and algorithms with UML activity, sequence or collaboration diagrams. At this point many problems occur. Students usually have difficulties to identify the main methods and represent the most useful parts of the legacy application. In addition, for the first time they have to use some UML and CASE tools. We just provide some templates for what we expect within a redesign document and give some recommendations for existing tools. The students have to evaluate and to choose the tool they want to use by themselves. The main utilized tools are nearly the same as in the industry: Rational Rose, TogetherJ, Fujaba, Visio, Word, LaTex, PowerPoint, etc. Students complain a lot about the learning overhead and the poor quality of existing tools. Requirements, analysis and (re)design After the reverse engineering phase one iteration of the Unified Process is applied. First the new requirements have to be integrated with the recovered information. This part of the exercise is 1. From UML to Java And Back Again, see http://www.fujaba.de
6
managed quite easy by the most groups. We only have to take care that they do not define to many new requirements. Second, the students have to analyse and (re)design the project. Third, they have to write a specification. The (re)design and specification phases are significantly more difficult for the students. With limited programming experiences and barely design experiences, students have severe problems to focus on the important topics and to develop ideas for core concepts and to write these concepts down. Students frequently complain that only after the implementation phase they got an idea of what needs to be done in the analysis and design phase and how this could be done and written down. On the other hand, a good design and elaborated core realization concepts are mandatory for a successful implementation phase. This topic is revisited in the conclusions section. Implementation phase Next, the phase is the one where the students are waiting for and have the most fun: the implementation. Depending on the quality of the groups design and realization concepts, different groups show quite different performance and organization within this phase. The delivery of this phase is on the one hand a running prototype showing the core functionality and, on the other hand, an updated design and specification document. A major challenge for most students is the use of a library in the implementation phase. For the most of them it is the first time and many of them did not even got an introduction in library use. The use of a library is a difficult task for undergraduate students, but we consider this practice as an important topic for a programming course. Usually we use the Swing library [ELW98] to produce the graphical user interface. Reviews and internal tests In the first years we employed an automatic, mail based code reviewing system. After a check-in to the group’s source code repository, a robot computed all modified code fragments and these fragments were sent to a round-robin list of reviewers together with a guide line, how to review the corresponding code. While this approach works reasonably well within our research projects, the students tend to ignore their reviewing duties. If they are forced to reply to their review mails they frequently just bounce the mail with a short “everything is fine” remark. It seems that the implementation phase is just too short in order to establish a reasonable reviewing culture. For these reasons we introduced an explicit code reviewing phase after the implementation phase in parallel to internal testing. Two students of each group have to review the code of another group and have to report the results to the other group. This review is an explicit assignment with a required delivery. The results are presented within a group meeting. This facilitates to motivate the students to do a reasonable review and to control the quality of the review. We plan to introduce similar external reviews for the requirements, analysis, design and specification documents, too. If all students participate in reviews, the review quality may contribute to the grades of students. In addition to the reviews the remaining students have to do internal testing. Testing is very unpopular and students try to circumvent this effort. They rather try to use this phase as an
extended implementation phase and to add new functionality to their application and especially to improve their software agents. However, if the internal testing is not done thoroughly, severe problems occur in the integration phase where each group’s agent has to work within each framework. To improve our control within the internal testing phase, we plan to employ automatic test suites and test coverage measurement tools. Integration The integration test is a first highlight in the project. For the first time the results of 12 weeks work can be evaluated and each group can compare its framework and its software agent with the results of the other groups. This also creates a lot of quality feedback, errors passing the internal tests show up quickly. In addition, this phase is very motivating: robots that exhibit poor performance, that do not work at all or that produce many runtime exceptions provide feedback that the efforts spent into the own has been worth it. Robots running stable and showing a good performance and successful agent strategies motivate to invest more work into the quality and performance of the own robot in order to have good chances within the final tournament. Interface standardisation group The interface between the framework and the software agents has to be well defined. Conformance to such an interface is the crucial issue for integrating all groups’ agents into all groups’ frameworks. In the first years, the interface definition was managed by some of the supervisors. In the last two years we created a special interface definition group stuffed by one member of each student group. This turned out to be very successful. First, this interface definition group was a great relief for our supervisors since they did not longer need to settle all the change and explanation requests. Second, the students learned how hard interface definition is and how hard it is to stick with and conform to a given interface. The main observation is that the interface definition group did not manage to focus on important topics and created an over-engineered interface definition that was very hard to implement. Thus, they had to strip-down and adjust their interface definition several times, even during the implementation phase. Surprisingly, the application development groups had little difficulties to deal with the various interface adjustments. They focused on their core functionality and adapted the interface changes in a lazy fashion until it stabilized. Tournament To achieve individual grades there is a final exam at the end of the semester, that evaluates their knowledge based on questions about the project. The grades for this test together with the grades of the overall group performance are used to determine the final individual grades. After the one hour exam, we start the finishing tournament where each robot has to run in each arena. Depending on the final ranking of each arena, the robots get points. This ensures that we will have an overall champion at the end of the tournament. This overall competition between the groups lets the lecture hall become a sports-arena where each group roots for their own robot. In addition, the tournament is a very emotional event and is much fun for the students.
7
Groupware During the whole project we employ a central repository holding all documents and all source code of all groups. Each group gets its own directory subtree with a pre-defined initial structure and templates for the deliveries. This directory subtree serves as a project handbook for each group. Access to the central document repository is controlled by using a configuration management system, the Concurrent Versioning System (CVS) [CVS], with optimistic locking concept. Each student employs his/her personnel workspace with copies of all documents and files she/he uses. For binary documents explicit write locks are maintained preventing concurrent modifications. For ASCII files, each student may have full read and write access. Concurrent modifications are automatically merged by the CVS system. In addition, the CVS system provides a mail based notification mechanism informing all group members (and the supervisors) when a modification is checked into the repository. Working with versioning tools is a major learning step for our students. Due to our experience, it is hard to teach the benefits of a versioning system lectures. Students must use a versioning tool in a real group project in order to understand its values. When introducing the use of a versioning tool, students have many objections and doubts that versioning works and delivers valuable team support. The first steps with the versioning tool are often confusing and considered as overhead. However, after one or two weeks, students would refuse to work without a versioning system. Many students report that they cannot imagine how to do group work without such a system and students with experience in group work not employing a versioning system are totally enthusiastic about the great relief from coordination efforts. They usually start to promote versioning systems to other project and even to enterprises where they are jobbing.
4. EDUCATIONAL VIEW This chapter discusses some additional properties of our application architecture, our process and our project repository with respect to our educational goals. LOC counting The central repository and the versioning tool proved to be very important for the whole course design. The pre-defined structure and the templates provided by the initial repository help the students to get started and to find all the documents they need. In turn, the uniform directory structures help us to find the students’ contributions and deliveries. The notification mechanism of the versioning system helps us to monitor the performance of the different groups. In addition, for ASCII files the versioning tool protocols which student contributes which line at which check-in. This allows us to compute the exact contribution of each student. Thus, each student can be graded for his overall contribution to the implementation. In terms of lines of code (LOC) individual student contributions vary from some hundred LOC to 10 000 and more LOC. Usually, two or three group members contribute the majority of the implementation and the other team members have only up to 1000
LOC. In 2001 we removed the best contributors from all groups and formed a new group out of these students. This new group got a special assignment. The observation was, that within the old groups some other student jumped in and became a major contributor and that within the best-of group again only 3 students did the major implementation work while the other became minor contributors. The latter may stem from the fact that the new assignment (a statistical evaluation application for our tournament) was less component oriented and splitting forces may have been harder. We will continue to investigate this phenomenon in the next years. One conclusion that we already have drawn at the University of Braunschweig is to build smaller groups (about 5 students compared to about 8 students in Paderborn) such that more students gain large implementation experience. On the other hand, smaller groups inhibit a higher risk of failure if some student drops-out and smaller groups require smaller applications and smaller applications imply less experiences with the complexity of large applications. The counting of individual contributions to the project implementation also allows to detect nominal group members that do not contribute to the implementation or even to the project at all. In the early phases, usually only the whole delivery is graded and thus nominal group members get enough credits to pass the course without actually contributing to the group’s results. This remains undetected since students solidarity requires that the active group members do not blame the nominal members in front of their supervisor. Therefore, at the University of Braunschweig, we introduced a minimal limit of 300 LOC contribution to the final application. For active group members this is a ridiculous low border. However, to meet this limit, students have to participate actively in the analysis, design and implementation work. Nominal group members fail to contribute this small amount of code since they just do not know how the system works and how and where some functionality could be added. In addition, while active students would not blame inactive group members in front of their supervisors they would also not donate some of their code to such nominal members that never show up on group meetings and did not do a single piece of work. Due to our experiences, about 5% of the students are such nominal participants and the 300 LOC requirement worked perfectly to identify them. Discussions with active group members usually proved our suspicions and active students reported that they were upset about these lazy group members and that they were happy that we took actions on them. Publishing the current project state at the web Many motivating properties of our project design have already been discussed. First, each delivery has to be presented in front of a larger group with the supervisor playing the role of a customer. It is a difference to deliver a lousy document or to blame oneself in front of a larger audience. Second, the final tournament and the competition between the different agents and thereby the competition between the different groups is highly motivating. We also utilize the competition between groups for quality insurance and for enforcing delivery dead lines. During the whole project we maintain an up-to-date copy of the whole project repository on our departments web server. Thus, the current
8
version of each document of each group is world-wide visible. Each student or each group may directly compare quality, size and content of its deliveries with the other groups. Any poor, incomplete, missing or late document is observed by all other groups. At the University of Braunscheig we even employ a central colour coded project performance table showing which deliveries are in time, which deliveries are late and which deliveries are not yet accepted as sufficient by the supervisors. In addition, late deliveries are punished by some extra assignments which is flagged in the overview table. The web copy of the project repository and the project performance table creates a high transparency for the whole project. Missing or incomplete or poor documents are immediately visible. Each group can judge its own work in comparison with the overall project. Due to our experiences, all groups try to avoid to be blamed by the project performance table. Thus, most groups deliver documents of reasonable quality in time. Groups that fail to do so, usually have severe internal problems. With the help of the project performance table these problems become obvious and we become able to react directly. We usually have a meeting with the corresponding group and reorganize it, appropriately. Another effect of our transparent technical project management is that students get a lot of hints and ideas of how to improve their own work. From another point of view, students could steal design or code from each other. This is usually a major concern of the well-performing groups. However, due to our experience, direct plagiarism does not occur. Typically, the main contributions are delivered close to the dead lines. Thus, there is just not enough time to steal ideas within one project phase. Direct copying would be detected due to the transparency of the project repository. After the first deliveries, each group has already developed so many own concepts and ideas that the direct copying of foreign ideas becomes complicated. It is only possible to get inspirations from other groups’ concepts. From an educational point of view, the latter is actually desired and students learn a lot from studying other solutions of the same problem. Still there is the problem that a successful agent concept of one group may be adapted by other groups and the students have objections to publish their concepts. However, we tell them, that they get their grades only for the quality and functionality of their application and not for the performance of their agents. Thus, being copied does not actually harm them. So far our students have accepted this. Project organization We assign specific responsibilities to each team-member to ensure that, for activities such as requirement engineering, reengineering, design, testing, installation, project documentation and revision control. Each person coordinates one of these activities. That the students learn the toughness of a project manager and how difficult it is to coordinate the different activities of the team members. To ensure reasonable results, all team-members have to deliver a significant amount of work in every phase of the project and understand the intention and context of each activity. At the end of each phase, the assigned, liable persons have to present the results in the weekly meeting of the groups and have to deliver an adequate document. Timely delivery of these phase results is then reported on a common project progress web-page. Late deliveries conduct to students’ grades.
Assigning responsibilities per phase creates a problem for the individual grading of students. Although one student is a phase’s manager, the phase result is usually a team result and giving good or bad grades just to the phase manager is not appropriate. Thus, in addition to round-robin phase management responsibilities, we plan to introduce vertical responsibilities. Each student becomes responsible for a certain group of requirements which ideally belong to a certain system component. Each student takes responsibility for the sections of the requirements, analysis, design and specification documents that correspond to his or her group of requirements. Within the deliveries, the corresponding sections show the names of the responsible team member. Thus, we become able to grade students individually depending on their contribution to the different project phases.
5. STUDENTS REACTIONS About hard deadlines Another problem is the short time and the deadlines, which are hard and hard to respect. The reactions to the advises of the teaching assistants covers the hole spectrum from: “I manage all that in the implementation phase!” to “We will try to fulfil the deadlines a week in advance to have more time at the end.” But in most cases it is much effort to persuade the student that not respecting the deadlines will end up in chaos. In contrast to that, it is to note that the students want and require the weekly meetings and meet themselves more often. Another (surprising) fact is, that in general the students accept small adjustments (refinements) of the requirements, from which we had a lot in the last two projects. Actually, once they realise that their documentation is not false but has only to be updated, they deal with the adjustments as it is usual business. About group reorganisation In the last project in 2001 we incorporated two ‘dirty tricks’ [Daw00] for the first time. We took one member out of the groups largest, to form a new group with a new task. The reactions of the students were rather different. One student who had to change the group stated: “It is impossible that I leave the group, because otherwise the group will not succeed in this exercise.” One group only stated: “It is a pity that X has to leave us.” Other reactions were: “I can leave the group instead of Y, I do not mind.” or “You cannot do that, that has nothing to do with simulating industry. Changing the requirements is realistic, but not this scenario.” About system failures Only some groups were affected by this situation, therefore we planed another ‘dirty trick’ for the other groups. For them we simulated a harddisc crash, with the consequence that the data could only be restored from two days before. Overall we have expected their reactions, because most of them were prepared. The resulting problems were mostly of technical nature, i.e. merging versions and update the repository with the right version. Usually the majority of check-ins to our central project repository is done close to the delivery deadlines. At the University of Braunschweig our CVS server became a bottle neck in these days and two times the server even dropped out for several hours.
9
Accordingly, many groups had problems to submit their latest versions within the deadline. First, we tried to blame the students for such failures arguing: “your customer won’t accept that excuse, either”. However, after vehement students’ protests we had to extend the deadlines in order to keep the students motivated. Still, we consider these technical problems as a valuable experience for our students and we plan to simulate such system failures intentionally in future courses. About team presentations Furthermore, each student has to present the results he/she is responsible for internally to the group with the tutor playing the customer role. It is remarkable how hard students question and judge the work of other group members. In contrast to that they are a team when they have public presentations. About information broadcasting The last experience we want to present is the handling of getting and providing information using mails, news groups or the world wide web. Most of the students do not get, do not manage to get or do not try to get all necessary information. We provide all information on special web pages and mailing groups. Each group has his own mailing group it can use. In the last year, we further installed a news group. But students always come in our offices or state that they did not get all information. When we refer to the web pages, mails, etc. they always have excuses. The favourite excuse is that they receive too many mails and do not have time to read them all! Or even worse: “I do not read mails with more than 10 lines content.” Looking regularly at the web pages also seems to be an hard task to most of the students. And about move to practice One mail we received a month after the end of the course at University of Paderborn in 2001 was: “[...] I had a lot of fun during the educational course and the work with you. The experience I currently made during the industrial placement shows me that the educational course had quite industrial strength. All in all, the educational course helped me a lot to accustom me to the environment and functioning of a software (computer science) enterprise. [...] But afterwards it is always easy to say so.”
6. SUMMARY AND DISCUSSION In this paper, we present our experience in teaching students in educational programming projects over the last four years. It is beyond controversy, that a more sophisticated educational concept in software engineering should serve the purpose of practical teaching. The most crucial aspects to achieve this are the choice of an appropriate application and a project execution that is as near to practice as possible. We decided to employ existing applications, which the students had to modify according to some new requirements. This is the common practice in industry, where in a first step the software engineer has to understand the functionality of the software. Due to the fact, that no documentation was available and the projects exceeded a complexity where the functionality can be revealed by just looking at the code, the application of reengineering techniques became reasonable.
To achieve a practical project execution, we defined a project plan with milestones and deadlines. The grading of the students directly depends on meeting these deadlines to underline the practical importance. The project plan includes all classical phases of software engineering. At the end of each phase, the students officially have to present their work, which results in an unexpected increase of motivation. Industrial strength university projects vs. projects in industry? While a number of „industrial strength“ student project descriptions and proposals have been published which either simulate industrial practice [AL00,Bot01,Fau00,Woh97, Vau00, SSvdW99] or taking place in industry itself [May97], the outlined restrictions for a 4th semester undergraduate project prevent from apply these approaches directly. Besides this fact that we have to somehow adjust our project requirements and plan to achieve our goals to the described more restricted context, we make the observation that some „myth“ of industrial strength software education do not hold for our setting. While direct practical evidence is missing some of these observations may also hold for more advanced student projects. Industrial strength in an academic environment? Besides proposals for projects within industry [May97] often the simulation of industrial conditions is proposed [AL00, Bot01, Daw00]. Simulating reality is always somehow artificial and therefore having a university student project and an industrial project afterwards seems to be the better solution. However, pure industrial practice often can only provide very specific experience and therefore fails to deliver the additional required overview. While the problems of large groups and project management should be made, students in industrial settings are usually integrated into an overall software process in form of a specific role rather than experience its management. Therefore, our university projects should prepare them to ensure that they can understand their embedding in the broader context of the overall project. When our student project should focus on teaching software engineering culture and „industrial strength“ rather than pure practice the question naturally arises why we did not use a lecture rather than a project to teach these contents. We think, that the experiences required for industrial strength such as large projects, project management, etc. can only be made in a project. Weekly exercises are not an appropriate context to teach them and only the combination of practical experience and software engineering techniques can deliver what is required here. Another significant difference with respect to most proposed industrial strength student projects is, that also activities such as customer contact and presentations have been simulated. While real customers are in fact the most realistic environment, our 4th semester students are not prepared to face real customers. The same holds often even for more experienced students. Therefore, such projects are often externally managed by the university staff which is also responsible for establishing and maintaining the real customer relationship. It is, therefore, from our point of view more appropriate to first train students with simulated customers and confront them with „real“ industrial settings later on.
10
The duration of student projects in industrial setting or university projects is another serious restriction. How should we teach iterative software processes in only a few weeks? In fact, this time constraints prevent from cycling through phases multiple times and thus the state-of-the-art using iterative software processes can hardly be realized. In industrial projects, therefore, often each student is working only on a specific task and thus only gains experience on one phase at all. University projects tend to use the classical waterfall software process [Cus00] and therefore exclude changes in the requirements. Instead we use reengineering to ensure the embedding of the single student project into an overall cyclic process built by combining the projects of two consecutive years. Therefore, the student experience the more iterative nature of this overall process. Additionally, changed requirements and reassign group memberships have been employed to somehow simulate some iterative elements in the software process. However, time constraints prevent from larger changes. Should we let students develop to be used products? Often projects emphasise that industrial strength, to a great extend, depends on the fact that the final project outcome is somehow required later on. Our observation is, however, that such projects tend to be oriented towards the developed application rather than the educational aspect. The tutors and other staff members often do not stay in their required pure advisory role and instead try to „rescue“ the project when it is in danger. Either in industrial context or when applications for later use at the university such as tools are built, such effects can be observed. For our projects we instead focus on education rather than productivity and therefore use the parallel development of the same component by multiple groups to avoid widespread effects when a single group failure might effect others. Thus, if a specific group performance is not acceptable and attempts to help the students fixing their organizational problems fail due to missing engagement of the students, we are in a position where we can accept that their work will be useless in the end. In practice, however, instead of such extreme project failures, the independence of the tutors and staff has proved to be essential to ensure that our support concentrates on teaching rather than helping the students producing the application we require. Another fact is that the appropriate group size varies whether student productivity or education is the main goal. We are very well aware of the fact that larger groups tend to result in more organizational overhead, however, students have to experience this and find out how to minimize these overhead. Other projects propagate in contrast industrial strength for 2 person projects [Sli97] and therefore cannot address the practices required for larger groups. Another source of project overhead and project management problems is integration. While in a number of earlier projects we avoid this complication to ensure that each student group can realize their task at hand independently, in this years’ project an interface group has been established to let them experience the problems of coordinating multiple groups. While some delays occur the overall evaluation indicates that students should also be confronted with this problem.
REFERENCES [AL00] J. H. Andrews and H. L. Lutfiyya. Experience Report: A th Software Maintenace Project Course. In Proc. of the 13 Conference on Software Engineering Education and Training, Austin, Texas USA. IEEE Computer Society Press, March 2000. [Bot01] K. Bothe. Reverse Engineering: the Challenge of Largeth Scale Real-World Educational Projects. In Proc. of the 14 Conference on Software Engineering Education and Training, Charlotte, North Carolina USA. IEEE Computer Society Press, February 2001. [Cus00] J. Cusick. Lessons Learned from Teaching Software Engith neering to Adult Students. In Proc. of the 13 Conference on Software Engineering Education and Training, Austin, Texas USA. IEEE Computer Society Press, March 2000. [CVS] CVS. Concurrent Versions System - The open standard for version control. http://www.cvshome.org/. [Daw00] R. Dawson. Twenty Dirty Tricks to Train Software Engith neers. In Proc. of the 22 International Conference on Software Engineering (ICSE), Limerick, Irland, pages 209–218. ACM Press, 2000. [ELW98] R. Eckstein, M. Loy, and D. Wood, editors. Java Swing. O’Reilly, 1998. [Fau00] S. R. Faulk. Achieving Industrial Relevance with Academic Excellence: Lessons from the Oregon Master of Software th Engineering. In Proc. of the 22 International Conference on Software Engineering (ICSE), Limerick, Irland, pages 293– 302. ACM Press, May 2000. [FNTZ98] T. Fischer, J. Niere, L. Torunski, and A. Zündorf. Story Diagrams: A new Graph Rewrite Language based on the Unified Modeling Language. In G. Engels and G.Rozenberg, edith tors, Proc. of the 6 International Workshop on Theory and Application of Graph Transformation (TAGT), Paderborn, Germany, LNCS 1764. Springer Verlag, 1998. [May97] H. Mayr. Teaching Software Engineering by Means of a th "Vitual Enterprise". In Proc. of the 10 Conference on Software Engineering Education and Training, Virgnia Beach, Virginia USA, pages 176–184. IEEE Computer Society Press, April 1997. [MR99] W.W. McMillan and S. Rajaprabhakaran. What Leading Practitioners Say Should Be Emphasized in Students’ Softth ware Engineering Projects. In Proc. of the 12 Conference on Software Engineering Education and Training, New Orleans, Louisana USA, pages 177–185. IEEE Computer Society Press, March 1999. [NNZ00] U. Nickel, J. Niere, and A. Zündorf. Tool demonstration: th The FUJABA environment. In Proc. of the 22 International Conference on Software Engineering (ICSE), Limerick, Irland, pages 742–745. ACM Press, 2000. [Ros] Rational. Rose, the Rational Rose case tool. Online at http:/ /www.rational.com.
11
[Sli97] J. Slimick. An Undergraduate Course in Software Mainteth nance and Enhancement. In Proc. of the 10 Conference on Software Engineering Education and Training, Virgnia Beach, Virginia USA, pages 61–73. IEEE Computer Society Press, April 1997.
[Vau00] Rayford B. Vaughn. A Report on Transfer of Software Enth gineering to the Classroom Environment. In Proc. of the 13 Conference on Software Engineering Education and Training, Austin, Texas USA. IEEE Computer Society Press, March 2000.
[SSvdW99] Klaas Sikkel, Ton A. M. Spil, and Rob L. W. van de Weg. Replacing a Hospital Information System: an Example th of a Real-World Case Study. In Proc. of the 12 Conference on Software Engineering Education and Training, New Orleans, Louisana USA. IEEE Computer Society Press, March 1999.
[Woh97] C. Wohlin. Meeting the Challenge of Large-Scale Software Development in an Educational Environment. In Proc. of th the 10 Conference on Software Engineering Education and Training, Virgnia Beach, Virginia USA, pages 40–52. IEEE Computer Society Press, April 1997.
[Tog] Object International. TogetherJ, the TogetherJ case tool. Online at http://www.togethersoft.com.
