State of the Survey on Team-based Software Engineering Project Course Atsuo Hazeyama Department of Information Science Tokyo Gakugei University 4-1-1 Nukuikita-machi, Koganei-shi, Tokyo 184-8501, JAPAN E-mail:
[email protected]
Abstract It is recognized importance of team-based software engineering education in these days, and lots of case studies have been reported. This paper describes a comparative study on team-based software engineering education. We propose three perspectives (team selection, assessment method for grading, and computer-supported environments) for comparison at first and compare cases based on the perspectives.
et al. [12], and Scott et al. [23]. We describe a brief overview for each study as follows. And then based on both the results and my original perspective, we propose the perspectives for comparative study.
2.1
2.2
1. Introduction Importance of team-based software engineering education is recognized in these days. A lot of case studies have been reported, for example, [1, 2, 4, 5, 6, 7, 8, 9, 10, 16, 17, 21]. The goal of such type of courses is for the students to nourish problem resolution abilities through collaboration in the form of group learning as well as technical knowledge and skills for software development by actual experience of the software life cycle. On the other hand, in the field of learning theory, paradigm shift emerges from teacher-centered to learner-centered [15]. Team-based software engineering education corresponds with this trend. A lot of case studies have been published thus far, however, few survey papers exist which systematize practice on team-based software engineering education. This paper aims at a comparative study on team-based software engineering education. We propose three perspectives (team selection, assessment method for grading, and computer supported environments) for comparison at first and compare cases based on the perspectives. This paper is organized as follows: Section 2 introduces brief descriptions of some survey papers for software engineering education. Then we propose a framework for comparative study of team-based software engineering project course. Section 3 gives descriptions for some cases and Section 4 compares according to the proposed framework.
2. Perspectives for Comparison Some survey papers were published for software engineering education, such as by Daniel et al. [9], Hayes
Daniels et al.
Daniels et al. proposed the following items to compare their course with others; duration, simultaneous activities, scope and type of project, cohort composition, team size and composition, method of selecting and managing the teams, modular credit, and position within the program [9].
Scott et al.
Scott et al. proposed a classification scheme for team selection dynamics, project specification, team implementation dynamics, and team evaluation dynamics in student software engineering projects [23]. * For team selection dynamics, (a) random student placement on teams (b) teacher chosen teams, designed to equalize the differing teams’ capabilities (c) student chosen teams (d) selection of a team using one of the methods described above, then moving students from team to team if necessary * For project specification, (a) teams identify and specify their own projects (b) the professor specifies all aspects of the project (c) the specification is provided by an outside agency, working closely with the professor * For team implementation dynamics (a) each team implements the entire project (b) each team implements and tests a selected number of modules (c) each team implements and tests only its own project (d) each team implements another team’s specification, and tests another team’s implementations
2.3 Hayes et al. Haynes et al. discussed the criteria for good grading schemes [12]. They also present five schemes for grading individual effort within teams as follows: (1) the team mark is everybody's mark. (2) everybody reports what they personally did, and separate marks are given to those components by the grader. (3) other team members report (confidentially or openly) the relative contributions of other team members to
allow for an adjustment of the final grade. (4) quizzes in class to ensure that students know the intimate details of the project. (5) cross-validating with the results of individual work. Hayes et al. concluded no single scheme meets all the grading criteria and using a combination of these schemes is the best approach for achieving adequate criteria coverage. 2.4 Wilkins et al. Wilkins et al. proposed an assessment method of individuals in team projects and the data for assessment [26]. They proposed three types of categories for assignment: Personal Characteristics, People/Team Competencies, and Problem Solving Skills. They defined sixteen, seventeen, and nineteen attributes for each category. They also proposed five types of assessment methods: ”Scale”, ”Ranking”, ”Matching”, ”Sentence Completion”, and “Short answer”. They only proposed a framework and did not apply the proposal to case studies.
2.5 Proposal of a framework for comparative study Although the abovementioned studies are useful, they focus on limited aspects of team projects. They do not focus on an aspect of a computer-supported environment either. This paper proposes a framework for comparative study on software engineering project courses.
team members
software dev. process
product
project specification
support system
grade
Figure 1 Conceptual model for team-based software engineering project course Figure 1 shows my conceptual model for a team-based software engineering project course according to the notation of SADT [22]. This figure means software development is conducted by inputting the project specification, and produce a software product. It also means a system supports the software development process and controls the process. From the figure, support systems and team selection scheme, namely, how team members are assigned, affect the software development process. The grade of students will be determined based on both the process and product. Therefore I regard team selection, assessment method for grading, and support environments as major aspects for comparison. I also pick up course overview as an aspect for
comparison. I adopt a proposal by Daniels as that for the course overview. I basically adopt a proposal by Scott as that for the team selection. I add the objective of team selection for the scheme and the data for selection. I adopt a proposal by Hayes et al. as the scheme for assessment method for grading. I also adopt a proposal by Wilkins as the data items for assessment. Comparative schemes for the supporting environments are not published. I therefore propose this aspect for comparison. (1) Course overview I utilize the data items proposed by Daniels et al. * Year: * Duration: * Number of students * Team size (No. of students per team): * Project specification (2) Team selection Students have to be assigned to teams in team projects. Abilities of software development differ significantly among individuals [3]. On the other hand, team selection should be fair from the perspectives of grading. The formation of teams is one important issue in successful teamwork [19]. Scott et al. proposed four patterns as a team selection method. However they do not present the data for team selection. I also deal with the data for team selection. *Objective of team selection * Team selection method * Data for selection This paper investigates who organizes teams by what types of data and what ways. (3) Assessment method for grading The final grade must be given for individuals. However grading is difficult because all students do not tackle the same tasks in this type of education. Different problems may be selected among teams. Even within a team a task is divided into sub-tasks, which are not uniform, and they are assigned to team members. Furthermore a lot of activities are done outside of the classroom, so it is not easy to ascertain the activities of students from the teaching staff. We propose the following items for comparison: * Assessment pattern adopted * Data for assessment (4) Computer supported environments There are two types of approaches for building a support environment. One is to apply general purpose tools. Another is to develop systems based on their own requirements and apply them. We proposed the major functions for software engineering education support systems as follows: * Functions for students: - Artifacts creation and information sharing - Communication - Process support (ex. inspection, problem management,
configuration management, etc.) - Project management (metrics) * Functions for the teaching staff - Team selection (including data collection for team selection) - Progress monitoring - Mentoring - Data collection and analysis for evaluation Software is generally provided for use via analysis, design, coding, and testing phases. Each phase creates its associated artifacts. The artifacts are foundations for common recognition among team members and the inputs to the next phases [18]. To clarify the items to be described in each artifact and to structure them as templates enable to reduce missing to describe important information. It is therefore necessary for a system to support artifacts creation and to share them among members. In the software development process, various types of communications occur such as questions and answers, requests, notifications, discussions and so on, within a team. However, it is difficult for the members to meet altogether because of their own schedules. Therefore a mechanism to support communication under distributed environments is required. In the software process, some typical processes exist. It is necessary to support them efficiently by a computer, for example, association the processes with artifacts and/or the status management. From the nature of education, existing courses regard the teacher and teaching assistants as mentor. It is necessary to support communication between the students teams and the teaching staff.
3. Overview of Cases A lot of case studies for software engineering education were reported in some conferences and/or journals such as ACM CSE Symposium and /or IEEE CSEET Conference. In some of those, in spite of excellent practice they do not show the data for comparison, for example [1, 2, 8, 17, 18]. I introduce five cases that include almost all data items for comparison.
3.1
Rein
Rein reported their course in [20, 21]. (1) Course overview * Year: undergraduate business students * Duration: eight weeks * Number of students: 60-70 students * Team size: 12 - 14 students per team (No. of teams is fixed) * Project specification: ideas for the project are proposed by the team and subject to approval by the teacher. (2) Team selection * Objective of team selection: skill-balanced teams * Team selection method: student chosen teams * Data for selection: first of all, the instructor talks about
high performance team in her class. Her approach seems to let the leaders have a strong power. Then, leaders are decided by candidacy. The remaining students are instructed to organize themselves into five skill teams (speakers, programmers, writers, researchers, and process experts). The leaders visit the teams to interview people for their teams. (3) Assessment method for grading * Assessment pattern: combination from (1) to (4). * Data for assessment: Rein adopted Criterion-referenced grading (mastery grade) [20]. Each student’s grade is interpreted relative to the achievement of certain objectives. - Quizzes (students retake until they pass the specified standard level) - A team project: two conditions must be met: 1) the project must be evaluated as “meeting” or “exceeding” the standards specified for that project, and 2) each student must be a “significant” contributor to the group project. Each student’s contribution to a team project is evaluated as two types of ranks by (peer evaluations and the teacher’s impressions). A project is evaluated as three types ranks by comparing the project artifacts to the standards specified for that project. (4) Computer supported environments * Tool: Net news, E-mail, spreadsheet * Functions: Information sharing, Asynchronous communication, Schedule management
3.2
Brown
Brown et al. reported their course in [4, 5, 6]. (1) Course overview * Year: third year undergraduate students * Duration: 12 weeks * Number of students: Around twenty-five * Team size: 5 - 6 students per team * Project specification: students select from a list of projects. The projects usually require students to develop software which is useful to someone in the authors’ school. (2) Team selection * Objective of team selection: Not Clarified * Team selection method: teacher chosen teams * Data for selection: two types of questionnaires, “team formation (TF)” and “compatibility point (CP)”. The most significant questions for team selection were who they would like to be in a team with, their background, and the projects they preferred to do. The TF questionnaire also includes seven items, ex. courses the students have studied previously, whether they are willing to serve as a team leader. The CP questionnaire includes eleven items, ex. what grade they expect to receive in the course, how heavy their schedule is. (3) Assessment method for grading * Assessment pattern: combination from (1) to (3).
* Data for assessment: 70% of the grade of the course is reflected by the grade of the project. 30% is evaluation for the team, namely all members will get the same score. 40% is assigned from the perspective of contribution of individual [5]. (4) Computer supported environments * Tool: UNIX groups，mailing lists，Web pages * Functions: Information sharing ， Asynchronous communication.
3.3 Drummond et al. Drummond reported their course in [10, 11]. (1) Course overview * Year: second year undergraduate computer science students * Duration: fifteen weeks * Number of students: Not Clarified * Team size: five to six students * Project specification: the same task is assigned to all teams. (2) Team selection * Objective of team selection: Not Clarified * Team selection method: teacher chosen teams * Data for team selection: Not Clarified (3) Assessment method for grading: * Assessment method for grading: (1). However it is not clear whether other methods are used or not * Data for assessment: team assessment has been based on the delivery of a system and interim reports. [10] (4) Computer supported environments * Tool: original application development with the BSCW, and a video conference * Functions: communication, Artifact sharing
3.4 Matsuura Matsuura reported her course in [16]. (1) Course overview * Year: third year undergraduate computer science * Duration：15 weeks * Number of students: 70-120 students * Team size: 11～16 students per team * Project specification: teacher presents two tasks and then the students select one of the two. (2) Team selection * Objective of team selection: Not clarified * Team selection method: random * Data for selection: by lot (3) Assessment method for grading: * Assessment method for grading: combination from (1) to (2) * Data for assessment: the team mark is assigned by peer evaluation (all students of the course). In the final class, all teams give presentation of the system they developed. They evaluated from the presentation. It is difficult to ascertain individual contribution in the project work. So
she created a fine-grained questionnaire (137 items： understanding of technologies，attitude of tackling， planning, team work, self evaluation, development method). She evaluated individual mark from the response. (4) Computer supported environments * Tool: proprietary Web application * Functions: communication, Planning and progress reporting
3.5 Hazeyama Hazeyama reported his course in [13, 14]. (1) Course overview * Year: third year undergraduate informatics education students * Duration: fifteen weeks * Number of students: around 25 * Team size: three to five * Project specification: Teacher presents two or tasks and then each team selects one of them. (2) Team selection * Objective of team selection: fairness of capabilities among teams * Team selection method: teacher chosen teams * Data for team selection: - The grade of the “Intro. to SE” - Intention to be a leader - Job request after graduation - Programming skill - Document writing skill (3) Assessment method for grading * Assessment pattern: (1), (2), and (4) * Data for assessment: based on the following three items: - Score of the final examination after the projects have finished. - Evaluation for team work (from the perspective of processes and products). The teacher specified the artifacts which should be created as a team (ex. development plan, team progress report, meeting minutes, system analysis document, user interface design, database specification, system test specification, and user manual, etc.) - Evaluation for individuals by the teacher. The teacher specified the artifacts which should be created as an individual (personal progress report, his/her assigned tasks, unit test specification). The teacher also evaluated contribution from team communication logs. In assessment, the author utilizes data stored in the supporting system. (4) Computer supported environments * Tool: proprietary Web application * Functions: team selection, planning, progress reporting, Bulletin Board System-based asynchronous communication, meeting minutes creation, inspection
support, test specification, bug tracking, artifact sharing, awareness support
4. Comparative Study This section gives some comparison from three perspectives (team selection, assessment, and computersupported environments) for the abovementioned cases. 4.1 Team selection In many case studies teacher organizes teams. However the reason varies: Daniels et al. said “Pre-selecting the teams also corresponds to the normal arrangement in industry and commerce where one cannot usually select ones team mates.” Redmond stressed on importance of heterogeneity of members in collaborative learning [19]. On the other hand, Brown especially emphasizes on the data “who they would like to be in a team with”, “their background” [4]. It will aim at homogeneity. Hazeyama emphasizes fairness of capabilities among teams. Many studies use the data of questionnaire for team selection, however they do not clarify the mechanism on how the data are dealt with. Under such a situation, Redmond developed a team selection support system [19]. The system used results of questionnaire as the input data. The items are computer-related jobs, project preference, and possible time slot. The system selects teams by a heuristic algorithm, which especially put emphasis on possible time slot of the students, because they are part time students. Hazeyama adopted Genetic Algorithm (GA) for team selection engine [14]. 4.2 Assessment method for grading Table 1 shows the scheme of assessment method for grading of some case studies according to the patterns by Hayes et al [12]. It shows that as Hayes et al. suggest, most mixed some patterns. Especially many studies pay attention to ascertain individual contribution in a project as well as the team mark.
5. Conclusion
Rein
Brown
Matsuura
Hazeyama
*
*
*
*
This paper has proposed a framework to compare the cases of a team-based software engineering education course. It has aggregated items from existing survey studies, added an attribute for team selection, and appended an item on computer-supported environments. According to the perspectives, we performed a comparative study. As for team selection, in most cases teacher organizes teams. Many cases use the data of questionnaire for team selection, however they do not clarify the mechanism on how the data are dealt with. As for grade assessment, in all cases except that by Drummond et al. (I omitted the case because they do not present data for assessment fully), more than two items are adopted from five which were proposed by Hayes et al. For grading a student, both the assessment result for a team and individual contribution were taken into consideration. As for computer-supported environments, few discuss them. I also found that current computer supported environments support very limited aspects of the software process, so no integrated environments exist which provide functions for data collection for team selection, team selection, development process support, and data collection for evaluation. In recent years, research activities, which mine a software repository are paid attention to in the software engineering field. These technologies should be embedded into the software engineering education environment. We have to tackle to constructing an integrated software engineering education.
*
*
*
*
Acknowledgements
*
*
Table 1: Comparison of assessment method for grading Team mark is everybody’s mark. Individuals reporting for their assignment Evaluation from other team members Quizzes
The environment should support not only development work by students but also the teaching staff who support development teams, prepare projects and assess students grade. I found the instructors collected various types of data for assessment. It will take a lot of efforts to collect the data for assessment manually. It is therefore desirable to collect them from the supporting environment. It is also difficult to analyze the data from multiple perspectives. Computer support will contribute to these aspects.
*
Cross-validation
4.3 The computer-supported environments The workload of software engineering project course for both the students and the teaching staff is high [6]. Surprisingly few discuss computer-supported environments for team projects [25]. In such a situation, almost all cases provide facilities for asynchronous communication and information sharing. These support student developers only in parts.
I would like to thank anonymous reviewers for their comments to improve this paper.
References [1] E. J. Adams, A Project-Intensive Software Design Course, Proceedings of the twenty-fourth SIGCSE technical symposium on Computer Science Education, pp. 112-116, ACM Press, 1993. [2] M. I. Alfonso and F. Mora, Learning Software Engineering with Group Work, Proceedings of the 16th Conference on Software Engineering Education and Training (CSEET2003), IEEE Computer Society Press, 2003. [3] B. Boehm, Software Engineering Economics,
Prentice-Hall, 1981. [4] J. Brown and G. Dobbie, Software Engineers Aren't Born in Teams: Supporting Team Processes in Software Engineering Project Courses, Proceedings for Software Engineering Education and Practice (SEEP'98), pp 42-49, IEEE Computer Society Press, 1998. [5] J. Brown and G. Dobbie, Supporting and Evaluating Team Dynamics in Group Projects, Proceedings of SIGCSE'99, pp 281-285, ACM Press, 1999. [6] J. Brown, Bloodshot Eyes: Workload issues in Computer Science Project Courses, Proceedings of the 7th Asia-Pacific Software Engineering Conference (APSEC2000), pp. 46-53, IEEE Computer Society Press, 2000. [7] C. Chrisman, and B. Beccue, Evaluating students in systems development group projects, Proceedings of the eighteenth SIGCSE Technical Symposium on Computer Science Education, pp. 366-373, ACM Press, 1987. [8] I. Crnkovic, M. Larsson, and F. Luders, Implementation of a Software Engineering Course for Computer Science Students, Proceedings of the 7th Asia-Pacific Software Engineering Conference (APSEC2000), pp. 397-401, IEEE Computer Society Press, 2000. [9] M. Daniels, X. Faulkner, and I. Newman, Open Ended Group Projects, Motivating Students and Preparing them for the "Real World", Proceedings of the 15th Conference on Software Engineering Education and Training (CSEE&T 2002), pp. 128-139, IEEE Computer Society Press, 2002. [10] S. A. Drummond, C. Boldyreff and M. Munro, Software Engineering Group Project Work: Past, Present and Future, paper presented at SIGToSE, London, March 1997, http://www.dur.ac.uk/~dcs1sad/papers/sigtose/sigtose.htm. [11] S. A. Drummond, and C. Boldyreff, The Development and Trial of SEGWorld: A Virtual Environment for Software Engineering Student Group Work, Proceedings of the 13th Conference on Software Engineering Education and Training (CSEE&T 2000), pp. 87-97, IEEE Computer Society Press, 2000. [12] J. H. Hayes, T. C. Lethbridge, and D. Port, Evaluating Individual Contribution Toward Group Software Engineering Projects, Proceedings of the 25th International Conference on Software Engineering, pp.622-627, IEEE Computer Society Press, 2003. [13] A. Hazeyama, K. Osada, Y. Miyadera, and S. Yokoyama, An Education Support System of Information System Design and Implementation, Proceedings of the 7th Asia-Pacific Software Engineering Conference (APSEC2000), pp. 393-396, IEEE Computer Society Press, 2000.
[14] A. Hazeyama, N. Sawabe, and S. Komiya, Group Organization System for Software Engineering Group Learning with Genetic Algorithm, IEICE Transactions on Information and Systems, Vol. E85-D, No. 4, pp. 666 - 673, 2002. [15] T. Koschmann (editor), CSCL: Theory and Practice of an Emerging Paradigm, Mahwah, JJ: USA: Lawrence Erlbaum Associates, 1996. [16] S. Matsuura and R. Aiba，A Software Engineering Education by Experimental Software Development Group Work，SIGNotes of IPSJ, CE68-1, pp.1-8, Feb. 2003 (In Japanese). [17] L. M. Northrop, Success with the Project-intensive Model for an Undergraduate Software Engineering Course, Proceedings of the twentieth SIGCSE Technical Symposium on Computer Science Education, pp. 151-155, ACM Press, 1989. [18] H. Pournaghshband, The students' problems in courses with team projects, Proceedings of the twenty-first SIGCSE Technical Symposium on Computer Science Education, pp. 44-47, ACM Press, 1990. [19] M. A. Redmond, A computer program to aid assignment of student project groups, Proceedings of the thirty-second SIGCSE Technical Symposium on Computer Science Education, pp. 134-138, ACM Press, 2001. [20] G.L. Rein, Grades That Motivate, Proceedings of the Conference on Information Systems and Global Competitiveness, pp. 355-362, 1995. [21] G.L. Rein, Teaching IS Design and Development in a Group Learning Setting, Proceedings of Computer Supported Collaborative Learning (CSCL95), 1995. [22] D. T. Ross, Structured Analysis (SA) : A Language for Communicationg Ideas, IEEE Transaction on Software Engineering, Vol.3, (1977), pp.16-34. [23] T. J. Scott, L. H. Tichenor, R. B. island, Jr., James H. Cross II, Team Dynamics in Student Programming Projects, Proceedings of the twenty-fifth SIGCSE Technical Symposium on Computer Science Education, pp. 111-115, ACM Press, 1994. [24] M. Shaw, and J. E. Tomayko, Models for Undergraduate Project Courses in Software Engineering, CMU/SEI-91-TR-10, ESD-91-TR-10, 1991. [25] S. Shoenig, Supporting a Software Engineering Course with Lotus Note, Proceedings of the International Conference on Software Engineering Education and Practice (SEEP1998), IEEE Computer Society Press，1998. [26] D. E. Wilkins, and P. B. Lawhead, Evaluating Individuals in Team Projects, Proceedings of the thirty-first SIGCSE Technical Symposium on Computer Science Education, pp. 172-175, ACM Press.
