Session T4J
Work in Progress - Does Maintenance First Improve Student's Understanding and Appreciation of Clean Code and Documentation Maureen Doyle, Brooke Buckley, Wei Hao and James Walden Northern Kentucky University,
[email protected] ,
[email protected] ,
[email protected],
[email protected]
Abstract - The ACM’s “Computer Science Curriculum 2008: An Interim Revision of CS 2001” suggests 31 core hours of Software Engineering covering the standard phases of software development. At many universities, including ours, Software Engineering is a capstone course with a semester-long team project. While the course covers the software development lifecycle there is not enough time in a single semester for students to gain an appreciation for all phases, especially the importance of testing and documentation to the maintenance phase which is the longest phase in successful software projects. To address this, a new course, Software Maintenance and Testing, was added as a pre-requisite to Software Engineering. The course will be implemented in fall, 2011. Students will be surveyed to determine if their appreciation for different development models, documentation, recording of design decisions, and clean code are improved by maintaining code prior to developing a brand-new or greenfield project. A Likert scale instrument will be develop to evaluate students’ attitudes. The research instrument will be tuned until validity and reliability are achieved. Preliminary and baseline results will be presented. Index Terms – software documentation, Engineering, software testing, student attitudes
ACM guidelines, but with only a cursory introduction to software testing due to the compressed time-scale of a semester project. In addition, students develop a greenfield project and provide documentation required for a good grade, without experiencing enough time maintaining the project to understand the value documentation of requirements, design, software and testing has in maintenance and testing. The ACM computer science curriculum guidelines[1] contains an “Industrial perspectives,” listing technology areas industry wants our graduates to possess. These areas include testing, debugging and bug tracking; code readability; documentation; and code reviews. In addition, industry also noted a desire for graduates to possess basic release management principles and code archeology skills. A new course, Software Maintenance and Testing was created to address industry concerns and topic-shortcomings observed in our classrooms. It will be first offered in fall, 2011. This course will be required for students graduating with a BS/CS and, with minor additions, offered as an elective for Masters’ students. The course is a prerequisite for students’ Software Engineering capstone.
Software
INTRODUCTION
ACM Computer Science Curriculum 2008 [1] recommends 31 core hours of Software Engineering for a student graduating with a bachelor of science in Computer Science. The 31 hours is divided among eight core topics including traditional software phases of requirements, design, verification and validation. Software Verification and Validation has a minimum coverage time of three hours. Software Evolution, which includes maintenance as a topic, also has a minimum coverage time of three hours. The Software Engineering course taught at NKU is a capstone course involving the introduction of Software Engineering along with the development of a software product [2]. This is not unlike many Software Engineering courses [3] and it has been our experience that students leave the course satisfying the topic coverage defined by the
At the minimum, students will explore and practice software maintenance and testing on open-source brownfield projects. Students will work in teams, in a manner similar to standard Software Engineering courses, with instructions on how to test and validate maintenance code. GOALS Positioning Software Maintenance and Testing prior to Software Engineering was done for four reasons. First, many of our students work in co-ops or internships parttime during their junior or senior years. This course prepares them for these experiences since students are rarely assigned brand new, or greenfield, projects. Most often students’ initial work is maintenance, documentation or testing. The second reason is the focus of this research paper. Our working hypothesis is that students will obtain a greater appreciation of documentation throughout the software cycle. In this new course, we will ask students to
978-1-61284-469-5/11/$26.00 ©2011 IEEE October 12 - 15, 2011, Rapid City, SD 41st ASEE/IEEE Frontiers in Education Conference T4J-1
Session T4J perform inspection, testing, and code maintenance on existing open source projects. Students will be formed in teams. Each team member will be required to document their bug fixes, enhancements, and validation steps. The team member will pass their knowledge on to other team member through documents. The third reason is software testing can help students build quality awareness. It can help students make a better software design. In a test-driven development (TDD) process, engineers need to think about testing first. The fourth reason is that experience with performance and stress testing can help students better understand nonfunctional requirements, such as scalability reliability, and availability. Learning testing first can help student truly understand the differences between good software and bad software. It definitely will help students better analyze software requirements and design software. EVALUATION PROCESS Evaluation of our hypothesis will be accomplished in two different ways. The first evaluation technique will compare a software project developed by students who have only taken Software Engineering with a project, with the same requirements, developed by students who have taken the Software Maintenance and Testing course. Comparisons will include quantitative metrics such as object-oriented design metrics[4], documentation size, and scores on acceptance tests developed by the authors. Results from this study will not be available before 2013. The second evaluation technique is a 5-point Likert-Scale Software Documentation Attitude survey consisting of statements of attitudes towards the documentation component of software development. The survey is constructed in such a way that four main subscales are addressed: requirements, design, implementation and testing. Questions solicit both positive and negative responses (e.g., Software requirements should be documented whether the customer requires them or not versus software requirements will change and should only be documented in the code). This attitude survey that we developed uses the techniques developed for the standard Computer Attitude Survey [5]. The survey, and initial results, will be presented at the conference. In addition, the survey’s reliability and validity analysis will be presented. Requirements questions focus on the need to and benefit of documenting requirements. Design questions focus on the need of documenting design justifications and alternatives. Implementation questions focus on code style and comments, while testing questions focus on documentation of unit testing.
addition to the survey questions, students will be asked to record their prior software development experience and whether they have had the Software Testing and Maintenance course. Validation of the survey will occur during the Summer of 2011. Using the data collected, reliability will be assessed using Cronbach’s alpha testing for internal consistency. A factor analysis with a four factor solution will also be employed in order to assess if the factor loadings support the a-priori grouping of the survey questions with respect to the four subscales. THREATS TO VALIDITY
There are a number of internal and external threats to validity. The main threat to external validity occurs because the samples are not random. Students from one regionally comprehensive Midwest school will be regularly taking the survey and therefore the results may not generalize. There is a threat to internal validity since survey reliability and validity will be done using survey results from BSCS students who graduated in the past two years. CONCLUSIONS While software engineering curricula traditionally cover the phases of the software development lifecycle in chronological order, we hypothesize that there are pedagogical advantages to starting with testing and maintenance. We describe a plan to investigate the effects of a maintenance and testing focused software engineering course taught before a traditional software engineering course on the quality of software developed by students and on student perception of the importance of documentation and testing activities. REFERENCES [1] ACM. "Computer Science CS2008 Curriculum Update: An Interim Revision of CS 2001." 2008. [2] Hao W., Doyle M., Fu J. "Preparing Students for New Programming Paradigms: Integrating a Mobile-Cloud Project into the Software Engineering Course." SmartPhones in the Curriculum Workshop (SMACK). Honolulu, Hawaii: IEEE, 2011. [3] Pressmen, R. S. Software Engineering: A Pracitioner's Approach 7th edition. New York: McGraw Hill Higher Education, 2010. [4] Chidamber S.R., Kemerer C.F. "A Metrics Suite for Object-Oriented Design." IEEE Transactions Software Engineering 20, no. 6 (June 1994): 476-493. [5] Loyd B., Gressard C. "Reliability and Factorial Validity of Computer Attitude Scales." Educational and Psychological Measurement 44 (1984): 501-505.
The software documentation attitude survey will initially be given to students enrolled in Software Engineering for spring, 2011. The survey will then be administered at the beginning of each Software Testing and Maintenance course and at the end of each offering of Software Engineering. In 978-1-61284-469-5/11/$26.00 ©2011 IEEE October 12 - 15, 2011, Rapid City, SD 41st ASEE/IEEE Frontiers in Education Conference T4J-2
