Coupling Pair Programming and Writing: Learning About Students’ Perceptions and Processes Tammy VanDeGrift University of Washington Seattle, WA 98195-2350
[email protected] ABSTRACT
work, diminishing the learning experience for the weaker partner. McDowell et al. studied pairings and found that pairers had a higher average score on the final exam than those who worked individually, so the pairings benefited both students [4]. In order to ensure that both partners take responsibility for the work, we had students complete individual written reports about the pair programming projects. Our goals were threefold in assigning written reports: To give students the opportunity to improve their writing skills and master technical vocabulary, to have a mechanism for accountability of both partners, and to learn about the processes students use when creating project solutions. Curriculum 2001 stresses the importance of communication skills in Computer Science graduates, with students being able to “communicate ideas effectively in written form” [1]. Project reports give students the opportunity to practice communicating technical ideas. Others have documented how they integrate writing into the curriculum [3] and tools for scaffolding writing for engineers [6]. Here, we focus on the integration of writing with pair programming, students’ perceptions of this integration, and students’ processes as they complete pair programming projects. We investigated the following research questions: How do students perceive pair programming and individual reports as beneficial to their learning? What processes do students use, what challenges do they face, and where do they go for supportive resources? We combined qualitative and quantitative methods to answer these questions. We analyzed survey responses to open-ended and ranking questions and analyzed a sample of written reports, looking at processes, challenges, and supporting resources. We provide background of the course context in the following section. Sections 3 and 4 explain our methods, data collection and analysis, and the results. We conclude with implications for instruction and learning.
Pair Programming has been successful in many introductory computer science courses. Its success has manifested in increased retention rates, better performance on programming tasks, increased confidence, and decreased frustration. This paper highlights experiences using pair programming coupled with individual written reports in a large, introductory computer science course. Through analysis of survey data and written reports, our study shows that students perceive benefits in pair programming, such as those documented earlier (social structure, peer help, less frustration, reduced workload). Regarding the written reports, students cited benefits such as a chance to summarize their project, to reflect on the code, and to use new vocabulary. We discuss what we learn about students’ processes, their challenges, and the resources they use for supporting their learning.
Categories and Subject Descriptors K.3 [Computers and Education]: Computers and Information Science Education - Computer Science Education
General Terms Documentation
Keywords Pair Programming, Writing, Process
1.
INTRODUCTION
Previous research has shown the effectiveness of pair programming in introductory Computer Science courses [5, 4, 7]. In pair programming students sit side-by-side at one computer to complete a task together, taking turns “driving” and “navigating”. Students who used pair programming performed better on programming projects and exams [5, 4]. Retention rates increased when students pair programmed [7, 4]. However, one concern when using pair programming is that the stronger student will do most of the
2. STUDY BACKGROUND We summarize the course structure and materials pertinent to this study. The study context was a ten week introductory programming course at a large university.1
2.1 Course Structure
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. SIGCSE’04, March 3–7, 2004, Norfolk, Virginia, USA. Copyright 2004 ACM 1-58113-798-2/04/0003 ...$5.00.
The introductory course used an objects early approach in teaching the Java programming language. The course included typical introductory content: objects, classes, meth1
More information about the course can be found at
http://www.cs.washington.edu/education/courses/cse142/03wi/.
2
tem), process (how the team designed and implemented the solution), system description (internals of the system, descriptions of key algorithms and data structures), testing and evaluation (evidence of test cases, evaluation of the solution), and a conclusion (learning experiences). Each project (solution and report) was worth 16 points. The project solution totalled 8 points, with 4 points awarded for the quality of the design, implementation, and testing and another 4 points awarded for solution functionality. Both members of a pair received the same grade for the 8 solution points. The individual reports were worth 8 points, with 4 points awarded for the quality of the writing and 4 points awarded for the correct usage of technical vocabulary. Students received individual grades on the project reports.
ods, types, loops, conditionals, scope, arrays, searching, sorting, iterators, static variables and methods, inheritance, abstract classes, interfaces, and recursion. The course also emphasized software engineering practices regarding testing, debugging, and documentation. Students attended lectures2 three times per week for 50 minutes and met in small sections, led by teaching assistants, once per week for 50 minutes. Two instructors, including the author, led the lectures. Our study differs from most pair programming studies, since students did not meet in computer laboratories as part of the course. A dedicated introductory computing laboratory was available for use outside class sessions, with access to undergraduate tutors.
2.2 Student Population Since the course is required for all engineering programs, most students studied disciplines other than Computer Science and Computer Engineering. Of the approximately 550 students in the course, 24% stated they intended to major in Computer Science or Computer Engineering by the end of the course. 53% were freshmen, 27% were sophomores, 15% were juniors, 4% were seniors, and 1% were graduate students. 20% were females and 80% were males.
3. STUDY METHODS We were interested in how students value pair programming and the written reports in addition to the processes students use when undertaking programming tasks.
3.1 Data Collection Instruments and Analysis Two main instruments served as our data collection devices. First, a survey about students’ perceptions of pair programming and written reports was administered in class during the final week of the course. Students signed consent forms to allow research use of their survey data. Since the author was one of the instructors, a third party collected the surveys and consent forms and kept them until final grades were submitted. Since completing the survey was voluntary, this presents a source of bias in the study. We collected 293 surveys from the 546 students enrolled at the end of the term.3 The distribution of surveys returned from freshmen, sophomores, juniors, seniors, and graduate students resembled the enrollment in the course. The survey consisted of questions related to the students’ background in computing, questions for ranking statements about pair programming and the written reports, and free-response questions. We asked students for their name (so we could relate their responses to year and gender), their programming experience prior to taking the course, and their intended major. Students ranked the following statements with strongly disagree, disagree, neutral, agree, and strongly agree:
2.3 Programming Projects and Reports This study focuses on pair programming and individual project reports. Other assessments included exams, individual homework assignments, and weekly quizzes. To introduce the pair programming concept, students read the paper, “All I Really Need to Know About Pair Programming I Learned in Kindergarten” [8]. Additionally, the instructors demonstrated pair programming, followed by a debriefing session about the roles of drivers and navigators. Students completed three pair programming projects, with each project spanning two weeks. Project 1 was a tiling application, where students created kitchen floor tiles and patterns. Students created methods for displaying tiles and patterns (of their own design), using a locally built graphics library. Programming concepts included methods, classes, loops, and conditionals. Project 2 was a graphical elevator simulation. Students implemented at least two elevator algorithms of their own design and wrote methods to gather statistics on waiting times for the people in the simulation. They also wrote short analysis reports in pairs, based on their experimental results. Programming concepts included algorithms, classes, methods, testing, analysis, ArrayLists, interfaces, and searching. Project 3 was a graphical application showing digital camera filtering algorithms. Students implemented color interpolation algorithms for pixels and simple image processing algorithms, such as inversion and rotation. Programming concepts included two-dimensional arrays, array processing, classes, methods, interfaces, and algorithms. Students were divided into pairs within discussion sections (randomly for project 1, comparable skill levels for project 2, comparable effort levels for project 3). Each student wrote individual reports for each project. These reports served as an accountability mechanism, so that students had to understand the project solution well enough to write about it. Project report guidelines were given to the students: an introduction (purpose of the application), system use (how a user interacts with the sys2
• (Q1) I enjoyed working with a partner on the pair programming projects (Tiling project, Elevator project, Digital Camera project) in this course. • (Q2) I had more confidence in my solutions to the pair programming projects than the individual homework assignments in this course. • (Q3) I learned about concepts covered in the course by working with a partner on the projects. • (Q4) I gained more understanding of concepts in the course by explaining them to my project partners. • (Q5) I was more efficient in debugging my code while working with a partner on the projects versus working individually on the homework assignments. 3
The response rate was affected by the attendance rate. Approximately 400 students attended class the day the survey was administered.
Two separate lectures with 275 students each
3
Population All Students (N=293) Students with prior programming experience (N=113) Students without prior programming experience (N=169) Women (N=64) Men (N=206) Freshmen (N=144) Sophomores (N=72) Juniors (N=39) Seniors (N=10)
• (Q9) My confidence in the knowledge of project solutions (the code and process of creating the solution) increased through the writing of the project reports. • (Q10) My understanding of the technical vocabulary in computer science increased through the writing of the project reports. • (Q11) The written reports helped me reflect on the process used to create a solution. • (Q12) The written reports helped me reflect on the quality of my final solutions. Questions 6 - 8 and 13 - 15 were free-response questions. Question 6 asked what they liked most about the pair programming projects, question 7 asked what they liked least about the pair programming projects, and question 8 asked them to comment on their ratings for questions 1 - 5. Questions 13 - 15 were analogous to the pair programming questions 6 - 8, with students commenting on the written reports. We categorized the free response answers using emergent categories in the content analysis methodology [2]. Each answer was broken into distinct ideas and each idea was categorized separately. After an initial categorization was complete, the categories were established. The author recategorized the ideas into the established categories, since an independent rater was not available. The second data collection source included sample written reports for project 1. We chose to analyze reports from project 1, since students had not yet seen a model report; these are presumed to be more candid accounts of their project experiences. Of the approximately 550 students in the course, 212 students gave their consent for us to use their work as research data. This presents a source of bias, since we cannot look at the entire population of the students; however, we can still learn about students’ processes from this sample. Thirty reports were randomly selected for in-depth analysis regarding students’ processes. Excerpts from the written reports were marked as representing emergent categories related to processes, challenges, and resources used.
4.
Q1 3.35 3.24
Q2 3.33 3.24
Q3 3.21 3.04
Q4 3.52 3.50
Q5 3.58 3.45
3.46
3.40
3.34
3.53
3.67
3.42 3.37 3.25 3.54 3.59 3.80
3.50 3.27 3.19 3.50 3.59 3.67
3.28 3.20 3.05 3.39 3.59 3.50
3.42 3.58 3.51 3.54 3.67 3.90
3.97 3.46 3.46 3.68 3.95 3.70
Table 1: Average ratings for survey questions about pair programming. Students who answered anonymously are not included in population breakdowns, for lack of this information. Based on the ranking results, students perceived benefits in pair programming. The median ranking for questions 1, 4, and 5 was 4 (meaning students agreed with the statements) and for 2 and 3 it was 3. Students perceived having more confidence in their solutions, having more understanding of the project solution, and being more efficient in debugging. The breakdowns among students with and without prior programming experience, gender, and year in school show slight differences among groups. Those without prior experience felt that pair programming had more value. Men and women seem to value the experience almost equally. Students more advanced in their studies tended to value the experience more than the freshmen students. The free-response questions give us more precise results in terms of what students value in the pair programming experience. Table 2 shows the categories and percentage of students who stated benefits and disadvantages. Students felt they benefitted from having a partner for programming projects, since partners helped answer questions, brought complementary sets of ideas and skills to the group, and helped with debugging and problem solving. Students felt that finding time for meeting with their partners posed the biggest burden. This is not surprising, since the course did not include computer laboratory time; all project work had to be done outside class. Many students at this institution do not live on campus and had to fit working with partners into their commuting schedules. The next major complaint was working with people with different personalities and different skill levels. Even though students felt this was a disadvantage, only 6.14% mentioned that they would rather choose their own partners for completing the projects. Overall, we see that students felt working in a social setting had both positives and negatives.
RESULTS
Here we describe the results from analyzing the end-ofterm surveys and the sample project 1 written reports. First, to provide some background on students’ performance on these tasks, students generally did very well on the projects and written reports. Average grades for students who submitted projects4 were 7.75, 7.34, and 7.31 of 8 points for projects 1, 2, and 3, respectively. Average scores for the written reports were 7.59, 7.47, and 7.71 of 8 possible points. Students’ success on the projects demonstrates that pair programming can be successfully implemented in a large lecture course with no structured laboratory time.
4.1 Perceptions of Pair Programming
4.2 Perceptions of Written Reports
We analyzed the survey responses to the ranking questions by simply calculating the averages and medians for each question. We further divided responses according to programming experience, gender, and year in school. Our results are presented, so that 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, and 5 = strongly agree.
Students were more neutral about the written reports with regard to questions 9 - 12 listed in Section 3. All questions had a median response of 3 and average responses for these questions are listed in Table 3. Students’ responses about what they liked most and least about the written reports are categorized in Table 4. Students valued the chance to explain the project, describe
4 The number of students submitting solutions for projects 1, 2, and 3 were 554, 529, and 526, respectively.
4
Benefit Partner answers questions, complementary knowledge Multiple brains for problem solving and debugging Social nature of work More efficient Reduced workload Motivation to stay on task Less frustrating Real-life projects Disadvantage Scheduling time for meetings Partner personality differences Partner skill level differences Not choosing own partner Equal grade for unequal effort Difficulty of project Not enough time given for project Desk space in lab not adequate Inconvenient to work on campus Group of three difficult to work in
Population All Students (N=293) Students with prior programming experience (N=112) Students without prior programming experience (N=169) Women (N=64) Men (N=205) Freshmen (N=143) Sophomores (N=72) Juniors (N=39) Seniors (N=10)
Percentage 30.03 29.35 10.58 6.14 4.44 3.41 2.05 1.7 Percentage 47.78 25.60 22.87 6.14 4.78 4.44 4.44 1.02 0.68 0.34
Q9 2.83 2.91
Q10 3.10 3.19
Q11 3.14 3.29
Q12 3.16 3.23
2.78
3.04
3.04
3.11
3.27 2.72 2.76 2.87 2.85 3.60
3.31 3.05 3.26 2.96 2.90 3.30
3.39 3.10 3.15 3.00 3.28 3.90
3.23 3.15 3.24 3.00 3.10 3.30
Table 3: Average ratings for survey questions about written reports. Students who answered anonymously are not included in the population breakdowns, for lack of information. Benefit Chance to explain project Chance to summarize Grasped code, gained knowledge Easy to write Chance to use vocabulary Learned technical writing Disadvantage Took too much time Repetitive after finished with project “Busy”work Did not learn anything new Unsure of guidelines Do not enjoy writing Did not like to include test cases Due after project code
Table 2: Perceived benefits and disadvantages of pair programming. Percentages indicate the number of students who responded with a response fitting the category to survey questions about what they liked most and least about pair programming. problems they encountered, and write a summary of the project. However, they felt that writing reports took too much of their time, writing was repetitive after finishing the project, and it felt like “busy”work. The written reports served as a mechanism for individual assessment for the paired programming projects. It seems that students felt the written reports provided a forum to explain their personal view of the project, which may have led to the small number of students (4.78%) who felt their pair programming partners were earning “free” grades. Based on students’ responses, it seems that the written reports curbed the “free” grade issue associated with pair programming.
Percentage 20.82 16.04 11.26 8.19 4.44 3.75 Percentage 26.62 25.60 15.36 15.02 9.56 7.51 1.02 0.34
Table 4: Perceived benefits and disadvantages of written reports. Percentages indicate number of students who responded with the category to survey question about what they liked most and least about the project reports.
4.3 Insights Into Students’ Processes We were also interested in the processes students used to complete the pair programming projects. The written reports serve as artifacts for analysis. This presents a source of bias in that these are records produced by students, and these records were evaluated for a grade. However, the records do give us concrete evidence of students’ processes written from students’ perspectives. Written reports from 31 randomly selected students who gave us consent to use their reports were used for analysis. One person selected in the random sample did not submit a project 1 report, so we analyzed a total of thirty reports. Because 218 of 546 students gave consent to use their work, the sample chosen is probably not truly representative of the entire class. In our sample of 31 students, their average final grade was 3.38 whereas the average grade of the class overall was 2.88.5
Report excerpts were tagged according to the following categories: challenges students faced, supportive resources, and the processes students used. When extracting excepts about processes, only processes other than implementation were considered, since the bulk of the reports refer to implementation. Also tagged were excepts related to students’ perceptions of pair programming. Students’ comments about pair programming in the reports were similar to the survey responses, so we do not repeat those excerpts here. Table 5 shows the emergent categories produced when the report excerpts were analyzed. Also shown are the total number of excerpts falling into each category and the number of distinct students with excerpts falling into each category. Here we give examples of excerpts that demonstrate students’ processes in completing the project. For example, one student mentioned planning in the report: “The checkerboard pattern we first worked out on paper, realizing that the row pattern repeated every two rows and a tile was added
5 This is most likely due to the fact that students who gave consent attended class the day the form was collected, and students who attended class generally performed better in the class.
5
Categories Processes Testing Evaluating Solution Iterating/Refining Reading/Gathering information Planning Challenges Java language Translating ideas to code Using graphics Understanding libraries Understanding existing code Partner in pair programming Understanding requirements Designing own floor patterns Supportive Resources Partner in pair programming The project itself Course resources The planning stages Libraries/documentation Person (other than partner) Visual cues Creativity Persistence Online meeting technology
Total
Distinct
27 12 12 9 9
20 10 8 9 8
8 6 5 4 4 3 1 1
7 6 5 3 3 3 1 1
14 7 6 5 3 3 2 2 1 1
13 7 6 4 3 3 2 2 1 1
(Boolean operators, loops, parameters) and translating their own ideas into functioning code. Course resources, such as previous homework assignments, lecture notes, and the class discussion board helped students overcome challenges.
5. CONCLUSION This study demonstrates that pair programming was successfully implemented in a large course with no formal laboratory time. Students perceived pair programming as valuable to their learning. In fact, paired partners served a large support resource for students. Coupled with the pair programming projects, written reports served as accountability mechanisms to ensure both partners understood project solutions. The written reports benefitted students by giving them a forum to explain their solutions and helping them understand the final solution. The reports also served as windows to gain understanding of students’ processes, their challenges, and their supportive resources. By understanding more about students’ processes, challenges, and where they go for help, educators can build better support mechanisms for their students. Acknowledgments: Thanks to Hal Perkins (an instructor during the study), the teaching assistants, the students who participated in the study, and Richard Anderson, Sarah Schwarm, and the reviewers for making paper suggestions.
6. REFERENCES [1] CC2001 Task Force. Computing Curricula 2001: Final Draft. http://www.computer.org/education/cc2001/ final/index.htm. [2] O. R. Holsti. Content Analysis for the Social Sciences and Humanities. Addison-Wesley Publishing Company, Reading, MA, 1969. [3] L. C. Kaczmarczyk. A Technical Writing Class for Computer Science Majors: Measuring Student Perceptions of Learning. In Proceedings of SIGCSE, pages 341 – 345, 2003. [4] C. McDowell, L. Werner, H. E. Bullock, and J. Fernald. The Impact of Pair Programming on Student Performance, Perception, and Persistence. In Proceedings of the International Conference on Software Engineering, pages 602 – 607, 2003. [5] N. Nagappan, L. Williams, M. Ferzli, E. Wiebe, K. Yang, C. Miller, and S. Balik. Improving the CS1 Experience with Pair Programming. In Proceedings of SIGCSE, pages 359 – 362, 2003. [6] J. Turns, W. Newstetter, J. K. Allen, and F. Mistree. Learning Essays and the Reflective Learner: Supporting Reflection in Engineering Design Education. In Proceedings of the ASEE Annual Conference, 1997. Session 2230. [7] L. Williams, K. Yang, E. Wiebe, M. Ferzli, and C. Miller. Pair Programming in an Introductory Computer Science Course: Initial Results and Recommendations. OOPSLA Educator’s Symposium, pages 20 – 26, 2002. [8] L. A. Williams and R. R. Kessler. All I Really Need to Know About Pair Programming I Learned in Kindergarten. Communications of the ACM, 45(5):108 – 114, 2000.
Table 5: Shows the emergent categories through analysis of project 1 written reports of 30 random students. This table also shows the number of total excerpts falling into each category and the number of distinct students with excerpts in each category. Implementing was not included, since the bulk of the written reports focused on implementing code. to every other square in a particular row.” Another student mentioned the importance of planning: “The interaction of two or more classes on a larger scale requires more planning and forethought.” Another student wrote about reading as a start to the project: “We started the process by looking over the code originally provided to us for the project.” Since evidence of testing was required of the students, many students wrote about their approach to testing. One student commented on their test cases: “We also made sure to use patterns with both odd and even row and column sizes, as well as sizes divisible and not divisible by three.” Another student mentioned a patterned approach to testing: “Every time we adjusted the code (by implementing a method, etc.) we tested to see whether [or] not it worked. Thus at every phase of the project we were always certain which method needed improvements, rather than implementing in general and having to guess which method was executing properly.” One student described a change, an iteration, made in the implementation: “Eventually we saw that it would be easier and more efficient to just call the private method, display, for each shape in order to achieve the same purpose.” Along with process, written reports gave us insight into the challenges faced and resources used by introductory students. Pair programming, while creating challenges in a few cases, generally created a useful and valuable resource for students. Students struggled with using the Java language
6
