Integrated Online Courseware for Computer Science Courses Michael T. Helmick Miami University Department of Computer Science and Systems Analysis Oxford, Ohio USA
[email protected] ABSTRACT Computer Science CourseWare (CSCW) is a Web-based integrated online courseware system for the management of computer science courses developed by the Miami University Department of Computer Science and Systems Analysis. The system is designed to support a unified student experience for content delivery and programming (and general) assignment management. CSCW promotes incremental development through rapid feedback for students with its integrated Java style checking (using PMD [2]) and automatic grading capabilities. CSCW integrates with existing version control infrastructure (using Subversion [4]) and authentication infrastructure (LDAP or custom plug-in) to minimize administration. Paperless handling of programming assignments is encouraged with the ability for precise feedback to be entered by instructors for student consumption. CSCW focuses on the needs of courses using computer programs as a primary assessment measure. CSCW is actively being used at Miami University.
Categories and Subject Descriptors K.3.1 [Computer Uses in Education]: Computerassisted instruction; K.3.2 [Computer and Information Science Education]: Computer science education
General Terms Human Factors, Management
Keywords Education, Courseware, Automatic assessment
1.
INTRODUCTION
Managing the amount of information generated in a course is a difficult proposition. Categories of information include items distributed by the instructor (document, assignments,
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. ITiCSE’07, June 25–27, 2007, Dundee, Scotland, United Kingdom. Copyright 2007 ACM 978-1-59593-610-3/07/0006 ...$5.00.
notes, announcements), information generated by the students (homework, computer programs, papers), and the accompanying information associated with the grading of this student work. Complexity of course management only increases as courses become larger, have multiple sections, or have multiple roles for participants: teaching assistant, guest, instructor, or student. Computer science courses, and programming-centric courses in particular, have some unique challenges. Programming assignments generate a lot of paper (when printouts are used to turn in), take a long time to grade, and are difficult to grade consistently. For a reasonably sized course it can take a considerable period of time for programs to be graded and feedback to be released to students. These factors lead to the desire for a centralized collection point for student work, and the addition of automated grading ability to assist in rapid feedback for students. Existing courseware and course management systems, including Blackboard [5], WebCT [8], and Moodle [1], fulfill the needs of the larger academic community, but do not address the needs of the computer science curriculum in relation to management of programming assignments. Since 2005, we have been evolving parts of a new course management system geared towards courses where computer programming is an integral part of the course, while keeping the system sufficiently general for the larger academic community. This effort includes the development of an automated functional grading package for assignments using Java, the use of process journals [15] during programming assignments, and the integration of the Subversion version control system [4] for both observational [12] and backup purposes. These efforts evolved into a more comprehensive and cohesive system in which we look to solve specific design goals and address the needs of computer science courses: • Programming Assignments: Provide system functionality aimed at the delivery and support of computer programming assignments. • Paperless System: One motivation of this research is to move the handling of computer programming assignments to a totally paperless system and still provide a way for instructors to comment on student code. • Automation: Bring automation into the process as much as possible for enrolling students, managing source code repositories, testing, and grading of code. • Accessibility: The courseware system must be acces-
sible to students utilizing any operating system platform using a modern Web browser. • Security: Since the data involved in managing a course contains confidential information (i.e. student identifiers, grades) the system must be capable of adequately protecting this information without restricting the functionality of the system. This paper describes the integrated online Computer Science CourseWare (CSCW) system developed developed and used at Miami University in computer science courses to assign, collect, and grade computer programming assignments. Section 2 gives an overview of some existing courseware systems as well as motivations for features included in CSCW. CSCW is capable of replacing the core functionality of existing course management systems, such as [5, 8, 1], wile providing additional benefits for programming assignments. Section 3 describes the design, implementation, and features of CSCW.
2.
BACKGROUND
Several systems are in existence with the aim of solving the same problem that we are solving in relation to the management of computer science courses. CMS [10] is a system that is most comparable to what is presented here. CMS offers a workflow management for the handling of computer programming assignments and re-grade requests, but does not present an interface for grading programs within the system itself. The CMS design focuses on workflow within a large course in terms of students as well as instructors / teaching assistants. CMS allows for role-based access to parts of the system allowing for an instructor to delegate selective responsibilities. Other systems do not provide additional functionality for computer programming assignments [14, 16, 5, 8, 1], and therefore do not meet the goals that we have set for a courseware system. These systems do remind us that we do not want to have a course supported through disparate systems, but rather create a single centralized learning environment for the students [11]. There is a compelling case for collection of student process journals [15] to furnish insight into a student’s software development thought process. Collecting process journals offers a look at the student development process that is not seen when merely looking at the finished product. They help instructors understand how students are thinking and help students understand their own thinking and modify their software process on future assignments. Process journals have been used in our data structures course since August 2005, first on paper and then online. The online process journaling system has been rolled into the full courseware system and use has been expanded to multiple courses in the department. Version control systems are a valuable asset to computer science education for several reasons. At a minimal level, a properly administered server acts as a backup in the event of a student losing work due to deletion or disk failure. We have used individualized Subversion [4] repositories to manage and monitor the student development process. This has helped to identify procrastinating or struggling students and provide encouragement and advice. The use of Subversion also contributes to the effective use of office hours giving the instructor the ability to access the student’s code and
eliminating the need for students to bring their code with with them.
3.
COMPUTER SCIENCE COURSEWARE
The Computer Science CourseWare (CSCW) system is intended to be deployed and used with relatively little administrative intervention. CSCW makes use of authentication plugins, allowing the system to be easily reconfigured to integrate with a university’s existing authentication infrastructure. At Miami University, we use the built-in LDAP authentication module to authenticate users using the same sign on credentials used for all university computing resources. This module can be easily reconfigured to function with LDAP authentication frameworks of other universities, or customized further to integrate with any authentication scheme. Registration information presented in an individual user’s LDAP entry enables automatic enrollment for CSCW managed courses. When instructors log in, the courses they are teaching are automatically created in CSCW, a feature requiring customization for other institutions. Students logging in are automatically enrolled in any courses that are in the system. This seamless integration saves time for the instructor in that they do not need to manually enroll students as in CMS [10]. A basic authentication module is also built in, allowing an installation to be self-sufficient in managing user identities and passwords, but this requires more extensive administrative effort and disables the auto-enrollment feature. CSCW is implemented using the Ruby [7] programming language and Ruby on Rails [3] Web framework. The flexibility and simplicity of these tools allow for rapid development, testing, and deployment. Our installation is deployed on a Linux server and uses the MySQL database (InnoDB engine) for storage. The application itself is flexible and is configurable for any Ruby on Rails supported database and deployable on any Unix-like platform. CSCW is designed for horizontal scalability and allows for server clustering if the need arises. This system is in production use for multiple courses in our department. Within the first year of use, our CSCW system reached over 200 students and is used to manage all aspects of several courses. After our first complete semester, we received positive feedback from both students and instructors, and suggestions for features and enhancements which we incorporate into future enhancements.
3.1
Student Perspective
CSCW is designed and built with the student in mind, so we present the student perspective on the software first. When first accessing a course, the student is presented with a reverse chronologically ordered list of the most recent activities that have occurred in the course. This same overview information is also available via an XML RSS feed [6] allowing students to receive course updates without having to visit the Web site. From there, the student can navigate the six modules of CSCW: course blog, documents, assignments, forums, grades, and attendance. The six course modules are meant to provide everything needed for course content delivery and assignment collection, allowing the student to have a single source for all course information and activities. Except for the assignments module, the others can be disabled if an instructor
prefers to provide information in another fashion such as Blackboard [5] or a custom Web site. The Course Blog is simply a forum for announcements from the instructor(s) of a course with the optional ability to allow students to comment on the posts. Documents are course documents unrelated to individual assignments that are available to students for download, we use this section to distribution things like the course syllabus and lectures. Grades shows an individualized grade book for each student, including categorized weighting of grades. The attendance module allows for attendance tracking in a course. For attendance tracking the instructor generates a 6 letter code, and has students enter this code during the class period. In order to enable interactive communication, the Forums provide discussion boards similar to those seen on many Web sites. The inclusion of online forums allows students and instructions to interact with each other in a rapid fashion and promotes students learning from each other. For all of the modules in the system, they are designed with simplicity and usability in mind. This simplicity stems from a consistent user interface across the system and a commitment to convention over configuration, or the use of sensible well thought-out defaults instead of extensive configuration and options. The assignment module is central to the operation of CSCW as it is the portion that is tailored to computer programming assignments (although these features can easily be disabled for a non-programming course). Figure 1 shows a student’s view of a programming assignment where Subversion is enabled as the turn-in method. The system also allows an instructor to configure an assignment to accept files via upload from a browser, the preferred method with introductory courses. This page is divided into three sections: assignment information, subversion control, and assignment process journals. We designed this page to be as complete as possible without being overwhelming to a student. Assignment due dates are provided in a friendly manner, ‘due in 7 days’, as well as a precise date and time, with each assignment page providing the exact, current server time. A three month calendar is displayed down the right side of the page showing the current date, as well as highlighting all upcoming due dates in this course. Since Subversion (or version control systems in general) can be difficult to deal with, especially for students who do not have experience with a version control system, CSCW provides easy to use Web-based controls for their repository. We set up projects so that each student has a development and a release path in their repository. This allows the student, at any point in time, to turn in their development files by copying them to the release area and continue to work on the assignment. For introductory level courses, we forgo the use of version control in order to simplify the process for students new to programming. We also further simplify by using a lightweight development environment in CS1, Dr. Java [9], in order to reduce the context bewilderment for students. In this case, students upload their files to the CWCS Web site to submit their assignments. At the beginning of an assignment using Subversion, a student simply goes to the assignment page and executes the “Create Development Directory” command. This translates the assignment settings into the appropriate Subversion commands and creates the directory structure on the
server for the student. In previous courses not using this system, we found that students did not uniformly structure their repository causing delays for an instructor collecting their work. By enabling these controls through a common system, all student repositories will have a consistent structure. This approach integrates well with popular development environments, including the IDE we use in most courses, Eclipse. Once the development repository is in place, a student then “checks out” their files to their development machine within Eclipse. This provides a consistent method for checking out code from and committing code to the student’s Subversion repository and allows a student to work on their personal computer or in a campus lab without having to worry about transporting files. When the student is finished with the programming assignment and have checked in all their files to Subversion, they return to the CSCW page for the assignment. From here, they select the “List Development Directory” which will review the files that are currently checked in. This helps a student to identify missing and/or incorrect versions of files before copying them to the release area. After verification, the student runs the “Release (and Turn in!) Development Directory” command which copies their entire development directory (including history) to the release area and registers each file with the CSCW system. At this point their turn-in set is finalized, a process that can be manually performed for assignments not using version control. We currently allow students to finalize up to three turn-in sets per day. During the finalization process, all Java files are checked and marked up for stylistic and common programming mistakes using PMD [2]. PMD checks for mistakes such as empty catch blocks and short variable names. Automatic grading then takes place using one of two methods: 1) examining a diff (difference) report with expected output and 2) unit testing using our AutoGrader framework [13], with both of these methods described in the following section. A student will receive immediate feedback on their code in regards to both style and functionality, allowing the student to proceed with greater confidence during the development cycle. After the assignment has been graded (both automatically by the server and manually by an instructor), students return to the assignment page on the CSCW site. From here they can view their grade on the assignment, general instructor feedback, as well as all of their source code for the project with syntax highlighted and instructor comments displayed throughout the file that are attached to specific lines of code. All of this is displayed on the same page, giving students an easily printable or savable record of their work on the assignment.
3.2
Instructor Perspective
While CSCW is designed with the student in mind, there are many features that make managing a course easier on an instructor. The feature set of the system is meant to be as complete as possible to support a single point of management for instructors. This includes the ability to support course content delivery for the wider academic community. The course blog facilitates information communication with students; the documents section allows for distribution of items from syllabi to lecture notes, course forums provide interaction, and fully functional grade book is available. Each
Figure 1: Student’s view of an assignment. module of CSCW is designed to be intuitive and easy to navigate with all course functions quickly available from an instructor’s overview page. We now examine the instructor tasks.
3.2.1 Enrollment The automatic enrollment available in CSCW makes managing student lists a trivial task. This is because we are able to read university enrollment data from both student and faculty LDAP entries upon their initial authentication. In addition to the role of student, an instructor can add users to the roles of TA and Guest. This allows for limited access to content for guests and the ability to delegate select responsibilities to a teaching assistant.
3.2.2
Course Settings The course settings pane allows for fine grained control over the configuration of a course. An instructor may disable entire modules (blog, grade book, documents, attendance, forums) or the course RSS feed. This page also contains
individual tasks to which access can be granted for teaching assistants, limiting them to only grading of assignments or granting wider control of the course.
3.2.3
Creating Assignments The assignment creation (and maintenance) functionality is meant to be flexible to provide for handling the widest range of assignments possible, yet simple enough that assignments can be configured in only a few minutes. Instructors can specify the following information for an assignment (1) title, description, and files; (2) date the assignment is visible to students; (3) a due date for the assignment; (4) an additional date indicating the limit for late submissions; (5) the submission method: Web turn-in, Subversion (and associated settings), or external; (6) whether to require process journals and the fields to capture for the process journals; and (7) automatic grading settings. If the grade book module is enabled, the instructor may enter a score, immediately creating an entry in the grade book. When automatic grading is enabled, the instructor must
configure the tests, and there are three possibilities here. Currently all automatic grading is restricted to the Java programming language. When using PMD, we have specified a sensible default configuration and given instructors the ability to enable or disable individual rules. The second and third configuration points relate to automated execution and examination of code. Input/Output based tests are specified by providing an input file and expected output file. When executed, the input file is provided as standard input for the program and the output is compared to the expected output. Students can instantly view a diff report and see where their solution doesn’t match. An instructor can specify as many different input/output file pairings as they like and choose whether or not to show the associated input to students for individual tests. Automatic grading can also be configured using the AutoGrader framework that we have created. This option requires an instructor to write unit tests capable of exercising student code.
5.
This research is funded by a grant from Miami University’s Center for the Enhancement of Learning and Teaching, http://www.units.muohio.edu/celt/.
6. [1] [2] [3] [4] [5] [6] [7] [8] [9]
3.2.4 Grading Assignments Throughout the duration of the assignment, instructors have access to the current state of all student progress, including the ability to access their Subversion repositories to monitor progress, turned in files, process journals, and automatic grading results. We have found it helpful to read process journals during the project as it helps identify successes, failures, and frustrations of the students which, if addressed, can foster student success. When grading the assignment, an instructor views individual source code files with syntax highlighting and line numbers. Instructors have the ability to leave general comments at the assignment level, or specific comments on individual lines within a file. For any line in the file, the instructor simply clicks “new comment” and is presented with a text box to enter free form comments. If automatic grading is enabled, an instructor will see the results of these tests including PMD style annotations and resets from the test runs. After marking up student source code and assigning grades for all students, the comments can be released for students to retrieve their grade and instructor feedback. Submitted files are also available to download so that they can be examined, compiled, and executed. The time needed for an instructor to assess student work can be greatly reduced since execution of the code happens when the files are submitted and suggestions from PMD cover some of the more rudimentary problems occurring in code. This allows an instructor to spend their time commenting on larger design issues.
[10]
[11]
[12]
[13]
[14]
4.
CONCLUSIONS AND FUTURE WORK
The Computer Science CourseWare (CSCW) system developed and deployed at Miami University provides a fully integrated online courseware environment for the management of courses, especially those utilizing computer programming. Our experience with CSCW has shown that the system is stable, scaleable, and easy to use for both students and instructors. We are, however, constantly improving the system and implementing new features as requested by instructors and/or students. The complete source code for CSCW system, including the AutoGrader framework [13], are available to the academic community from a link at our CSCW installation at https://my.csi.muohio.edu/.
ACKNOWLEDGMENTS
[15]
[16]
REFERENCES http://moodle.org/, Jan 15, 2007. http://pmd.sourceforge.net/, Jan 15, 2007. http://rubyonrails.com/, Jan 15, 2007. http://subversion.tigris.org/, Jan 15, 2007. http://www.blackboard.com/us/index.aspx, Jan 15, 2007. http://www.feedburner.com/fb/a/aboutrss, Jan 15, 2007. http://www.ruby-lang.org/en/, Jan 15, 2007. http://www.webct.com/, Jan 15, 2007. E. Allen, R. Cartwright, and B. Stoler. Drjava: a lightweight pedagogic environment for java. In SIGCSE ’02: Proceedings of the 33rd SIGCSE technical symposium on Computer science education, pages 137–141, New York, NY, USA, 2002. ACM Press. C. Botev, H. Chao, T. Chao, Y. Cheng, R. Doyle, S. Grankin, J. Guarino, S. Guha, P.-C. Lee, D. Perry, C. Re, I. Rifkin, T. Yuan, D. Abdullah, K. Carpenter, D. Gries, D. Kozen, A. Myers, D. Schwartz, and J. Shanmugasundaram. Supporting workflow in a course management system. In SIGCSE ’05: Proceedings of the 36th SIGCSE technical symposium on Computer science education, pages 262–266, New York, NY, USA, 2005. ACM Press. J. Ekman. A web-based information centre to provide help, guidance and support for students. In ITICSE ’06: Proceedings of the 11th annual SIGCSE conference on Innovation and technology in computer science education, pages 363–364, New York, NY, USA, 2006. ACM Press. L. Glassy. Using version control to observe student software development processes. J. Comput. Small Coll., 21(3):99–106, 2006. M. Helmick. Interface-based Programming Assignments and Automatic Grading of Java Programs. ITiCSE ’07: Proceedings of the 12th annual conference on Innovation and technology in computer science education, Dundee, Scotland, 2007. ACM Press. C. M. Kapp. Implementation & evolution of a course management system. In SIGUCCS ’02: Proceedings of the 30th annual ACM SIGUCCS conference on User services, pages 264–266, New York, NY, USA, 2002. ACM Press. G. Lewandowski. Using process journals to gain qualitative understanding of beginning programmers. J. Comput. Small Coll., 19(1):299–310, 2003. C. Meinel, H. Sack, and V. Schillings. Course management in the twinkle of an eye - lcms: a professional course management system. In SIGUCCS ’02: Proceedings of the 30th annual ACM SIGUCCS conference on User services, pages 281–283, New York, NY, USA, 2002. ACM Press.
