spring 2007. COP 2510 Programming Concepts is a first-time programming course for Information Technology, Computer. Science and Computer Engineering ...
Restoring “Coding with Intention” in Introductory Programming Courses Alessio Gaspar, Sarah Langevin University of South Florida 3433 Winter Lake Road, Lakeland, FL, 33803 USA [Alessio | Sarah] @ softice.lakeland.usf.edu ABSTRACT This descriptive study discusses two conceptual difficulties encountered by students in introductory programming courses regardless of the chosen language or pedagogical approach (e.g. objects, classes or fundamentals first). Firstly, students tend to learn programming by memorizing correct code examples instead of acquiring the programming thought process. Secondly, they tend to read code by “flying” over it at a comfortable altitude while thinking to its assumed intent. While relaxing, this practice fails to train students to develop the rigor to catch bugs in others’ or their own code. Both trends result in an almost complete loss of intentionality in the programming activity; un-innovative code is generated by analogy with (or cut and paste from) existing solutions and is then almost randomly modified until “it fits” the minimal tests requirements without real analysis of its flaws. We review and evaluate pedagogical strategies which can be leveraged by instructors to address the above mentioned issues. Namely, we discuss the benefits of various forms of “Live coding” and test-driven pair programming active learning practices.
Categories and Subject Descriptors K.3.2 [Computer and Information Science Education]: Computer Science Education / Information Systems Education
General Terms Design, Human Factors, Languages
Keywords CS-1, Introductory Programming Courses, live coding, active learning, test harness, constructive alignment theory, evolutionary computation, evolutionary epistemology
1. INTRODUCTION 1.1 Problem Statement Regardless of the chosen language (e.g. flowchart interpreters [3], Java, VB) and pedagogy (e.g. Objects / Classes [1], Fundamentals [2] first), introductory programming courses are still challenging Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. SIGITE 2007, October 18-20, 2007, Sandestin, FL, USA Copyright 2004 ACM 1-58113-000-0/00/0004…$5.00.
for both students and instructors. Even though this concern has been the focus of the computing education community for a very long time, no silver bullet has been revealed yet. Reports of the successful applications of a wide array of, sometimes diametrically opposed, pedagogies are continuously published. What can we deduce from this? The first obvious conclusion is that each approach has its own niche of effectiveness. However, the problem is not only a matter of matching teaching and learning styles; our student population is evolving. The problems encountered today with “random programming” and “cut and paste oriented programming” were not as dominant a decade ago. The pedagogy of programming is therefore not a problem to be solved once and for all, but rather a continuous effort to connect the new generations’ cognitive characteristics, with the learning outcome we need.
1.2 Context of this study The problems and solutions discussed in this paper are therefore to be considered in context. We have evaluated the pedagogical practices herein discussed in two programming courses taught in spring 2007. COP 2510 Programming Concepts is a first-time programming course for Information Technology, Computer Science and Computer Engineering majors. It uses the Java language along with the BlueJ environment [1]. COP 3515 Program Design is meant as a follow-up on the latter and was taught to IT majors only. The C language was used for this course to focus on strengthening students’ skills and exposing low-level concepts (program stack, heap…) to prepare them for systemoriented senior-level courses (e.g. operating systems).
1.3 Paper organization The remainder of this paper discusses two conceptual difficulties observed in our students and suggests new solutions that were deployed and evaluated in the two above mentioned courses. Section 2 focuses on the tendency of students to only employ analogical thinking when trying to solve new problems. It investigates how “live coding” practices can help break this habit. Section #3 discusses the need to teach students debugging reflexes that they can internalize in their programming practice. Once again, a variant of “live coding” active learning strategy is evaluated, and the possibilities to employ competitive assignments inspired by test-driven programming methodologies is discussed. Section 4 concludes this paper by aligning the above issues in the perspective of a loss of intentionality in the students programming thought process which leads them to employ something we have termed “random programming” reflexes.
2. “Cut & Paste Oriented Programming” 2.1 Problem at Hand We coined this term to capture an increasingly popular cognitive approach to programming which our students have been exhibiting over the past 5 years. The key characteristics of this trend are that students (ab)use analogical thinking when solving new programming problems and end up entirely replacing problem-solving skills by pattern matching mechanics. When confronted with new assignments, some students start by identifying keywords in the program specification which they can relate to previous exercises, for which they have the solutions in their notes. The origins of this tendency are difficult to identify but a causal link can definitively be established with at least two characteristics of modern programming teaching: (1) The pedagogy used in many programming textbooks focuses on teaching programming through “complete this program” exercises. These often require students to cut and paste working code which they only slightly modify afterward. This practice is based on the idea of acquainting students with the programming language before to require them to actually write their own code from scratch. This is very close to what is done by some linguistic methods when teaching natural languages; informal immersion in the language prior to learning its grammatical structures. Our objective is not to discredit such approaches but rather to underline that, no matter how efficient they are as a starter, they need to be followed by a more formal training involving designing code from scratch to solve a given problem. Without such follow up, students are “familiar” with the programming concepts but might still not have acquired problem solving, and therefore programming, skills. (2) The ubiquitous availability of technical information. Students “Google” anything, sometimes even forgetting the need to reference their sources and falling into plagiarism. How does this trend, clearly identified by instructors when grading essay-like assignments, applies to programming? The emergence of search engines which specialize in technical contents (e.g. programs [14]) is reinforcing our students’ tendency to look for pre-existing solutions to solve any new problems they are faced with. Now that we have identified this first issue, we can examine how it impacts both students’ learning and our instructional methods.
2.2 Impact on teaching / learning process When taken to its limits, cut and paste oriented programming can lead students to interesting misconceptions about what programming is, but also about what is expected from them in a programming course. The first obvious problem is that of plagiarism. Students are not necessarily trained to clearly identify their personal contributions from what they have “borrowed” from other public sources. This requires instructors to be extremely clear on what constitutes good professional ethics practices from the beginning. “Borrowing” from other students is also likely to be claimed as a practice that was “thought acceptable” by students. It is however generally a sign of poor ethics and prepares students for failure in higher level courses since their understanding of fundamentals was, indeed,
“borrowed” along with the assignments. A positive way to envision this issue is to see plagiarism as a symptom of a pedagogical problem: the failure to teach design and programming skills vs. only pattern-matching in search of existing solutions. The most extreme illustration of this pedagogical issue was brought to us by a student who was failing an intermediate programming class, using the C language. That student had a taste for technology but was unable to acquire fundamental programming reflexes. He ended up demanding to be provided with a book containing all exercises and solutions which were likely to be on the exam. While this request often causes instructors to smile, it is worth wondering what motivates such requests and how we can handle them. Two pedagogical theoretical frameworks can help us in this endeavor. Firstly, learner-type categorization tests such as the Learning Connections Inventory (LCI) [6] can help identify students’ cognitive preferences. The need for explicit, step by step instructions on performing a given task and, by extension, the need to know exactly “what is on the exam” can be characterized by high scores in precision and sequential thinking. Constructive alignment theory [4] also sheds some light on this kind of behavior from the motivational perspective. The “perfect learner” student is curious, motivated to understand the material while other students are indeed only there to pass the class. Constructive alignment consists of aligning the examination methods with the expected learning outcomes in order to lead non-motivated students to focus their efforts on preparing the exact skills we intended originally to teach them for the exam. Before we discuss teaching practices which can help changing students’ attitude toward programming, let us open a pluridisciplinary parenthesis and relate these problems to the field of artificial intelligence. AI-related disciplines also focus on learning dynamics, albeit in the context of artificial systems instead of natural ones. While the difference is significant, it shouldn’t prevent researchers from both fields from having meaningful exchanges. For instance, Searle’s philosophical position on AI’s limits led him to propose a thought exercise called the Chinese room [5]. In this experience, a person is locked in a room with a book containing all possible Chinese questions and their answers. The person doesn’t speak Chinese but receives written letters (in Chinese) through a mail slot. He matches the symbols on the message to those in the book and copies the response which is then sent out of the room. To an external observer, sending in and receiving responses from the room, it seems that a conversation is being held. However, no real understanding of the dialogue occurs for the person inside the room. This argument has been the focus of numerous controversies about the capability of symbol manipulating machinery to achieve intelligent understanding. By analogy, if our students are to approach problem solving as a pattern matching of existing solutions, what level of understanding can this lead to?
2.3 Instructor-led “live coding” As we discussed in the previous subsection, the way we teach early computing courses might significantly impact how our students perceive the programming process. By nature, textbooks introduce students to programming problems in a two step process: (1) state the problem in plain English and (2) present
students with a complete functional solution. While students can indeed learn something about programming with this type of approach, it does not expose the process of coming up with the solution. Its suitability for programming courses beyond CS-0 is therefore highly discussable. Unfortunately, most texts use the same approach in subsequent programming courses thus reducing the chances for students to work on the very skills they ought to acquire. Most instructors would agree that textbooks are not the best medium for engaging students in active learning. However, it is also true that a significant proportion of instructors use the textbook, and its associated pedagogy, as their primary teaching tool. It is interesting to link this situation to cut and paste oriented programming practices. By presenting students with problemsolution pairs, we indeed reinforce their tendency to work by analogy and match new problems to existing solutions. Whether or not this is the right way to introduce students to programming, it is clearly not sufficient to teach them how to program. To avoid having students work only by recycling old solutions into new ones, we propose to leverage an active learning practice for which the term “live coding” was coined. This practice has been used increasingly by instructors but, besides some discussions on SIGCSE, has not yet been extensively published. The closest practices are revolving around the idea of apprenticeship based teaching [7]. The idea is simple; instead of focusing on providing complete solutions to students, we expose them to how they are built step by step. Making the programming thought process explicit can be achieved very simply by having an instructor write code in front of the students as a way to correct a given exercise. We refer to this pedagogical practice as instructor-led live coding in order to differentiate it from a student-focused variant which we are going to discuss in the next section. As has already been argued by proponents of apprenticeship-based teaching, the errors and hesitations of the instructor are also invaluable in teaching students where they will “get stuck” and how to think their way around these kinds of situations. We also implemented a variant approach which is more suitable when class time is limited or for online-based courses. This variant is based on the use of screen casting software, such as Camtasia, which allows to record narrated video screen captures. We used this tool to make even more time available during class to focus on helping students find their way through exercises and activities. Instead of taking time to develop the solution during class, the entire time slot was devoted to scaffold students into coming up with their own solution. The Camtasia video was then made available on the course Content Management System (e.g. blackboard) before the next session for students to review. This allowed the instructor to also summarize and address the various solution (s)he helped students come up with by themselves during class time. Such videos make a great resource for students to refer to as a summary of an active learning session when preparing for exams.
2.4 Preliminary Evaluation Student feedback on such videos indicated that they particularly appreciated the ability to rewind, repeat, or simply play them when they are more likely to learn. A survey in the two courses mentioned in 1.2 incorporated the two following questions: (1) Seeing the instructor code "live" solutions was more useful than seeing the final solutions directly
Level of Agreement Strongly disagree
Course
Disagree
Neutral
cop 3515 cop 2510
30% (3)
Agree
Strongly agree
40% (2)
60% (3)
40%(4)
30%(3)
(2) The CAMTASIA videos of instructor-led live coding complemented usefully the students-led live coding sessions in class cop 3515 cop 2510
10%(1)
40% (2)
40% (2)
20%(1)
40% (4)
30%(3)
20%(2)
These results are not based on a sufficiently large sample to establish significance. However, we include them in this paper to gain insight, as part of a descriptive study, on how our pedagogical efforts were perceived by students. In this context, the above information indicates that instructor-led live coding is positively perceived by students. It is reasonable to suspect it is less positively perceived by those who have already had programming experience. On the other hand, the situation is less clear concerning Camtasia live coding sessions. While students are less enthusiastic about these, it is difficult to assess how much this is due to a natural tendency to find it more motivating to watch a live performance rather than a video recording. The above Likert scale questions were completed by an open ended one: “Provide positive or negative feedback on the "Live Coding" idea”. The latter led to the following comments in COP 2510. Please note that the comments from COP 3515 will be discussed later in this paper since they were all focused on a student-led live coding variant which we will introduce in the next section. 1)
Having the instructor code was very helpful. [… rest of the comment is on student-led live coding variant…]
2)
I learn best seeing it done and then doing it myself.
3)
I think this was useful for me because I could see how we needed to work through the problem rather than just seeing the end result
4)
The video was helpful as I was able to stop and go back to repeat what I was not sure about. I was also able to repeat the videos to try to reinforce the concepts that were being used. The examples are much more helpful than just reading it in the book without reinforcement of examples. The answers you put for us was also helpful so that I knew if I was headed in the correct direction.
5)
Really good idea as it does allow to pause and understand what is happening.
These comments can be seen as derived from several categories of feedback; general appreciation with no specifics (#1), appreciation of the flexibility offered by video recording of live coding (#4, #5), appreciation of the way live coding makes the programming thought process explicit (#3) and disagreement with the usefulness of this feature (#2). Interestingly, despite a small population, we obtained a wide spectrum of attitudes. One of the most thought-provocative feedbacks is #2. At this stage, we can only conjecture the reasons for such a rejection of the live-coding. Most likely, this particular student already understood the programming process and is at a learning stage in which (s)he seeks more examples to practice autonomously. Such responses
could also come from students who prefer to see programming as a pattern matching of solutions rather than having to design solutions from scratch. This would be consistent with Bloom’s Taxonomy [15] in so far that sheer memorization is a much easier objective than application or creation of original knowledge.
2.5 Going Further: Toward grammars-early? Live coding is a practice which changes the way content is introduced to students during lecture or when concluding an exercise session. We want to conclude this section by discussing alternative changes to our instructional approach which involve introducing new contents. Most programming instructors would readily agree that it is not useful to overwhelm beginners with the intricacies of formal grammars. However, another pedagogical error would be to leave them out of the picture altogether. We mentioned earlier that high sequential scores on learner type analysis instruments such as the LCI [6] can help predict which students will be more likely to base their learning only on the exam contents. These students need clear rules to play by and this can translate into their need of knowing what they are allowed to do or not when programming. This can be true at the design level, but also at the code writing level. Students often express difficulties related to the syntax of the language they are using. A classical standpoint on this issue has been to focus on concepts and problem solving skills while relegating syntactical concerns as unimportant. While it is true that developing students’ programming skills is foremost a matter of teaching them how to program in a syntax-agnostic matter, we question whether this very endeavor can be achieved by totally ignoring the syntactical aspects. More specifically, we believe that syntactical difficulties can reveal a lot about students learning styles and, if addressed properly, can help steer them back into good learning practices. Previous studies [16][17] have pointed out that novice programmers already have problem solving skills which are applicable to computing (e.g. sorting problems). However, as they acquire the capability to more rigorously formalize these algorithms during CS-1, they also lose their ability to correctly solve problems. These results are consistent with our own classroom observations and students’ claims that, while they feel they can solve the problems at hand, they are unable to formalize a solution in a computer language. In [16][17] students were asked to write down in plain English how they would sort an array of integers. Some of the solutions underlined that the student did not see how the solution could be formalized at all (“Just put the things in order”), [16] or could not write a general solution which would sort any integer array. This leads us to believe that the students’ claims that their difficulties are solely syntactical might in fact be hiding deeply rooted difficulties in grasping the nature of programming. After all, employing formalisms, as opposed to coding with a specific syntax, is an important part of engineering, math and computing disciplines. While there is a tendency to overlook syntactical details to favor concepts, we need to be aware that struggling with syntax is often a sign of a serious lack of programming rigor and discipline. From this perspective, syntactical aspects are no longer useless or secondary but rather a first barrier to learning to program which deserves our attention as instructors. This observation led us to introduce, very informally, the concept of programming languages’ grammars in our introductory programming courses. This was intended to (1) help some
students with basic syntactical issues and (2) provide the students with a high sequential learning profile with “rules to play by” when they are writing code. This approach significantly differs from the traditional way students are expected to learn a programming language’s grammar simply by being repeatedly exposed to correct code examples. This was motivated by seeing how our students were hesitant when forced to write code from scratch instead of modifying existing programs. Also, this links the current issue with the cut and paste oriented programming techniques from the previous section. If asked to write code from scratch, students will have tremendous difficulties to do so since their learning was mostly by-example. They will therefore revert back to an analogy-based thought pattern, trying to generate a solution using what they were shown: solutions. To discourage our students from just re-using complete solutions, we need to expose the programming process (c.f. live coding). However, it is also important to provide them with the tools they need to write their own programs. These tools include building blocks (e.g. fundamental iterative and conditional patterns) and syntactical rules. So, assuming that an informal description of a language will be helpful to our students, the question remains on how it can be best introduced. We chose to introduce the grammatical structures of the language being used very informally, as “recipes” or “rules” to be followed, e.g. “a java file will start with a comment, followed by a class declaration. The class declaration will have a visibility modifier, for now we always use public, followed by…”. These recipes are often presented verbally and with hand drawn diagrams on the board. Later, once students have developed a certain familiarity with these structures, they can be pointed to resources such as [11] to get a glimpse of the overall formalism. The intent is not to teach them formal grammars and not to have them memorize the BNF of any given language but rather to change their attitude when writing code. Instead of randomly moving statements around because they have no formal understanding of the rules to follow, we aim at teaching them formally how they can combine statements in a syntactically meaningful manner.
3. “Random Programming” The previous section focused on difficulties encountered by students when programming. This section investigates another common learning barrier which prevents students from reading, understanding and debugging already written code, including their own.
3.1 Problem at hand In-class exercises or active learning sessions are great opportunities to guide students through the development of their own solutions. When engaged in such sessions, it is helpful to have students stuck on a bug or design problem do a narrated code walkthrough to explain their code We observed that many students “fly over their code” and not only ignore details but seem to force their own assumptions on what it is supposed to do. For instance, students will read through a function declaring “it computes the square root” instead of actually looking at what each statement does and therefore catching algorithmic flaws. This distancing from the “details” of the program is confused with abstraction since it doesn’t allow decisions about a higher level of design. Instead, it is which disables students from actually gaining
not to be educated a shortcut a deeper
understanding of what they are writing and finding errors when the program does not perform as expected. To a certain extent, this issue is not only characteristic of a lack of technical knowledge but also of a lack of critical thinking. Our first instinct was to justify this kind of barrier by the algorithmic complexity of the code with which students were working. We also considered the fact that it had been written by others, assuming novice programmers would encounter more problems understanding code they did not write. However, over the past 5 years, a portion of our students exhibited difficulty reading and understanding code regardless of whether it was their own or not. Such a strong confusion about programming indicates that these students have no understanding of the code they wrote or cut and pasted from existing examples. This is where the pedagogical issue discussed in previous section comes in to play and enables students to come up with “almost functional” code if the question they are working on is close enough to a previous solved exercise from the text or slides. We observed students piece together a program from code fragment from the slides, modify it until it compiles but then were unable to assess why it was not working. We termed this trend “Random Programming” due to the fact that when trying to “fit” the code to the assignments specifications, some students almost randomly move around, add or remove blocks of code. All the while, they are unable to formulate in plain English what the code is doing at a statement per statement level but default back to high level explanations.
3.2 Impact on teaching / learning process Novice programmers’ attitudes toward the concept of program correctness has already been the focus of other studies, [18] which revealed that instructors and students have very different standards on what constitute a correct program. Our observations might shed some light on the reasons for such a discrepancy. In introductory programming courses, we have observed for the past 5 years at USF that students who are having the most difficulty use a combination of cut and paste of existing code, followed by almost random modification of statements. They compile and run the program to see how much closer these modifications bring them to the assignment specifications. Being unable to understand written code prevents students from also developing the debugging skills which they will need when tackling non-trivial problems or during their professional life. How can we address these issues? Let’s revisit the practices we recommended in the previous section. Live coding sessions are useful to constantly remind students that correct programs are never written in one shot. Instead, programming involves thinking about design, writing code, testing it in your mind as you go, thinking about potential problems it can induce on other parts of the code, etc. By engaging students in live coding, we develop some of the skills they’ll need to comprehend the code being written. On the other hand, the grammatical structures of the language might not be nearly as useful for this particular issue. Students will overlook common syntactical traps (= vs. ==) and logical errors due to their way of reading code. Grammatical structures do not help students acquire the debugging skills they need in order to (1) understand code and (2) write it as an expression of the design they have in mind. We refer to the latter capability as
“coding with intention”. This term was coined as an opposite to the observed "random programming” practices. In order to restore the intentionality (idea first, express it in a formalism) in these students programming practices, we describe several active learning approaches in the following sub-sections.
3.3 Student-led “live coding” One of the main limitations of instructor-led live coding is that we do not commit the same errors students generally do. As we discussed in the introduction, there is no set list of common student misconceptions which will hold true for the next decade. Instead, it is part of our instructor role to adapt to new misconceptions and learning issues and solve them generation of students after generation of students. When adopting such a perspective, it becomes evident that relying solely on the instructor to expose learning problems is bound to be inefficient. Instead, we suggest leveraging a variant of live coding which brings the focus back to student learning and elevates the idea of live coding to an authentic active learning practice. We termed this approach student-led live coding, and is based on a SIGCSE post. [8] It indicated the success encountered in programming courses by engaging students to write code “live” on the data projector PC using a wireless keyboard. The core of this type of activity consists in having a single student work on the PC connected to the data projector from her/his seat using a wireless keyboard [8]. Alternative technological solutions exist (Video over network switches, software switches) but have not yet been comparatively evaluated (c.f. evaluation of wireless keyboard as a tool to implement this type of activity below). When first confronted to this idea, it is natural to compare it to the student submissions features of other tools such as Classroom Presenter [19], Ubiquitous Presenter [20], and Message Grid [9]. All these tools allow an instructor to collect students’ submissions to display and discuss them in class. However, their focus is on the complete final solution rather than showing the process by which they were achieved. Student-led live coding allows the instructor to share with the whole class the thought process used by a student when developing code. Many technological limitations still need to be overcome by developing specially suited software such as the CLUE IDE [24]. Right now, the wireless keyboard already allows for instructors to make this transition from live coding as a traditional (instructor-led) teaching strategy exposing the right contents (programming process vs. only solution) to live coding as a constructivist / active learning approach (student-led). Preliminary work has already reported that such activities can be especially efficient when dealing with assignments which can feature a wide array of different yet all valid solutions [10].
3.4 Preliminary Evaluation We used Likert-scale questions to get an idea of how students perceived the student-led live coding activities as opposed to the instructor-led ones. The following details the responses obtained in COP 3515 which was the only course in which the student-led version was experimented. (1) Seeing classmates code "live" solutions was more useful than seeing the final solutions directly Level of Agreement Course
Strongly disagree
Disagree
Neutral
Agree
Strongly agree
cop 3515
40% (2)
60%(3)
(2) Seeing classmates code "live" solutions was more useful than seeing the instructor do so cop 3515
100% (5)
These results indicate that the same way instructor-led live coding was appreciated over just getting the solution, the student-led variant was also perceived as an improvement. The levels of agreement are exactly the same. Due to the small population, it is difficult to extrapolate anything from this specific observation but it will be interesting, as more data is collected, to keep an eye on this trend. The second question was meant to specifically prompt students for a preference between the two. All respondents felt that the student-led live coding activities were more useful thus indicating a slight preference. This is consistent with the fact that the student-led variant is much more engaging for students due to its active learning nature. We also used the same open ended question discussed in subsection 2.4. Below are the responses which were relevant to the student-led live coding activities. 6)
It made us participate in the class and help each other code.
7)
When the students typed we often ran into problems and the instructor could explain everything as we went it, if we didn’t catch it before hand, we would try to compile, and run into problems and fix them from there, where when the code is already finished I didn’t feel like I learned as much, what to look for, common problems, etc.
8)
The problem with live coding is the time spent on each program. If we could limit the amount time spent on each program and just come up with the solution after a certain time frame. That way the whole class does not get stuck on one portion of the program and be able to focus on the solution of the entire program.
9)
This was by far the most helpful to me. It helped me get involved. It was also helpful to see others make errors, and then for us to find out what was wrong with the code they typed.
The improvement of students’ engagement has been clearly identified (#6, #7, #9) along with the improved learning resulting from observing others while they code (#7, #9). The downside expressed is that it can be frustrating for students to have to “wait” while the others are catching up on the example. This results from the use of a single wireless keyboard, and the fact that the code review is still guided by the instructor. This student’s critic also underlines that for this approach to be scalable to large classes the centralization of the activity must be addressed. These issues are the focus of our current work with Dr. Simon’s Ubiquitous Presenter Workshop group.
3.5 Going Further: co-evolutionary arm races What if students still look at their code and can’t find the error? Student-led live coding is good at bringing the attention of the whole class to a single student’s work. It compares to other student submissions-based systems [9][19][20] but adds a “live” dimension by allowing students to see their classmates coming up with the solution. Regardless, this approach still focuses on a
single student at a time and might be of little help when others are persuaded that their wrong solution is correct [18]. What is an instructor to do when facing such a situation? Simply showing a correct solution to the student is not the best pedagogical approach in most cases. It is not uncommon for the student to still think their solution is an alternative correct one and be unable to arbitrarily drop it in favor of the instructor’s without proper understanding of their error(s). Debugging the student's code is generally a much more pedagogically satisfying approach; instead of offering a whole solution to the student, the instructor offers alternative, smaller, code fragments to replace the parts of the student’s implementation which are incorrect. By doing so, the student can be more easily convinced that the current program is incorrect and a scaffold to the correct solution is offered instead of an arbitrary acceptation. This is more likely to ignite a learning process, and in our experience we have found that similar results could be achieved without the instructor even providing fragments of the solution. The idea is to present the student with a counter example, a data set for them to apply their algorithm and which will cause it to reveal its flaws. This circumvents the issues of students who are persuaded they have found an alternative valid solution and who are not interested in simply accepting blindly the one offered by the instructor. This approach was mainly tested with sorting algorithms for which it is particularly easy to design data sets (e.g. an array of integers) on which a faulty algorithm will fail. In this context, a student can be guided to correct her own program without ever having to impose the entire solution or correct code fragments. This pedagogical strategy is a better fit in a constructivist perspective of learning since it allows us to stay away from instructionism even when correcting students. We tried to develop a variant of this pedagogical approach more focused on the principle of peer to peer learning [25] rather than requiring the instructor to serve as a “discussion hub”. This improvement spells the difference between being able to deploy such pedagogical strategy in large classes or being restricted to use it only in small courses. We designed in-class activities in which students are paired. However, unlike pair programming practices [22], each student is working on solving the problem individually. Once both students have a solution they deem acceptable, they exchange their code and start developing a testharness for each other’s code. This activity has been inspired by Agile and extreme programming principles and, even though it is clearly not an instance of test-driven development [21], it clearly puts the emphasis on teaching students critical thinking skills. Our focus is to present this activity as competitive and antagonistic in nature; each student is in charge to invalidate the classmate’s code with legitimate test cases. Unlike other programming teaching activities which make students compete by designing programs which will compete against one another [23], this approach has the advantage of keeping the focus of the students’ efforts on what we want to teach them. Let us explain this statement, in most game-based pedagogical approaches to teaching programming, students write programs which will compete against one another. What interests us in such practices is the motivation and engagement brought by these competitive settings. However, when the criterion used to decide which program is superior is unrelated to code quality, we can observe that students write quick and dirty code which does the job well. Beyond the boundaries of a programming course, an AI course would suffer
from the same “what matters is to win” syndrome. Instead of developing intelligent game agents featuring complex adaptive algorithms, students would often find a smart and dumb algorithm which would still manage to exploit a loophole in the game design and overtake other approaches. The constructive alignment theory tells us that if we want rewards to encourage students to learn what it is that we are teaching them, we need to align those rewards with what really matters in the course [4]. Clearly, this is not the case in such competitive games; the reward comes from implementing a tank able to blast all other tanks quicker in an arena, regardless of the quality of the algorithms and code employed. This phenomenon is however common to industry settings where developers are pressured into meeting unachievable deadlines which cause them to develop to an unhealthy level the art of cutting corners. This results in unpleasant long-term situations for software development companies, as is elegantly put by the designer of the Scum software engineering methodology in [27]. Programming competitions of all sorts therefore seem to be aligned on industrial practices but not those that anyone would be pleased to introduce to our students. What is the relation to the pedagogical strategies we propose? Having students find loopholes in each other’s code and develop test harnesses to point them out allows them to compete, thus leveraging the motivational and engagement components of such activities, on the very things we value: code correctness and code quality. While engaging in such exercises, they implement the very same instructional method that we discussed earlier in this section: they find suitable data sets to prove implementations wrong without providing solutions directly. This requires them to understand each other’s code and apply critical thinking in evaluating how and when the code will fail. Once these skills are acquired, it is our hypothesis that they will be used subconsciously by students when they write their own code. As one student of the pair develops new test cases which fail the other student’s code, the latter will improve the program. Then, new test cases will be found to lead to another series of improvements. This iterative improvement is not unfamiliar to researchers from the field of neural networks and evolutionary computation. Leveraging co-evolutionary dynamics, that is dynamics in which two populations are evolving while selecting each other through a predator-prey selection scheme, is still a fruitful research topic which led to significant improvement in the learning capabilities of artificial systems such as neural networks [26]. In such works, neural networks are created almost randomly to constitute a “population” of learning algorithms. They are particularly suited to problems in which an input data set has to be mapped to an output data set (e.g. classifications tasks). The most basic learning algorithms require such neural nets to be presented with a training data set, examples of good input / output pairs from which the neural network will learn and generalize. Such pairs are also evolved in another population. The two populations undergo mutation / recombination / selection scheme which is common to any evolutionary computation approach [17]. The major difference is that the quality of a neural network is quantified as the number of data sets it can successfully solve. Inversely, the quality of a data set is measured by the number of neural networks it can cause to produce a wrong error. By improving the quality of one of the two populations, we improve
the quality of the other, thus leading to a so-called coevolutionary scheme. This core dynamic, albeit in a very different context, is present in the peer to peer exercises we discussed in this section. While quantifying the impact on student learning will be the object of another publication, trying to better understand how and why such an approach is beneficial to students is a worthy endeavor in itself. Being open to sharing across disciplines revolving around the idea of learning in natural or artificial systems can also lead to significant insights. For instance, it is easy to measure the impact of a certain training data set on learning when dealing with neural networks. In such settings, the more disruptive the data set is, the larger the impact on learning. It is interesting to compare this to what happens when an instructor tries to educate a student on why their code is wrong. The more disruptive the counter example is, the more likely we are to convince the student of the errors in their solution and point them to their attention. Although the work of Piaget doesn’t apply very well to college students, it is interesting to also note the similarities between this phenomenon and the concept of cognitive dissonance which suggests that learning is best achieved through cognitively “ground shaking” steps.
4. Summary & Discussion This paper discussed solutions to two learning issues which are encountered by programming students regardless of the chosen language and pedagogical approach (e.g. object, class, fundamentals first). Many students nowadays assume that, because of the readily available information at their fingertips, solving a problem is a matter of finding the solution somewhere else. We have proposed several pedagogical strategies, some of which were evaluated in spring 2007 at USFL in two programming courses, which are meant to address these issues. We can summarize their intent along two conceptual lines:
4.1 Limits of learning by example The ability to learn from examples is hallmark to human psyche, however, as any other educational strategy it can be misused and is not a silver bullet. Our live coding activities underlined the details of the programming process instead of providing students with the complete solution. This is still learning by example, and there is nothing wrong with that, but the examples we provide our students need to be aligned with what we want them to learn. Providing solutions teaches them to fall back on those. Providing them with examples of solutions being developed teaches them the thought patterns to develop their own code. Karl Popper’s evolutionary epistemology [12] is an invaluable resource in understand this distinction. Finding your way through a maze does not imply knowing the structure of the maze. To a certain extent, it only allows you to teach others how to follow your exact steps through the maze. On the other hand, providing someone with a maze map would allow them to find their own path through it, maybe even a better path. When teaching programming, only presenting solutions to exercises is akin to giving students a single way out of each maze they encounter. At the end of the day, they are still mostly unable to generate their own solutions from scratch and refer back to these solutions as some student refer back to mathematical demonstrations without being able to construct one of their own.
4.2 Coding with intention The two learning issues we discussed can also be viewed as two different aspects of a new student perspective on programming; Design is a matter of recycling correct code which partially matches the problem description; Implementing is a matter of adjusting this code (which is still not understood) more or less randomly until it compiles. The use of test harnesses might even make things worse; while it encourages students to not consider that their program are correct once they compile, it gives them yet another task to complete almost mechanically. Programs are further randomly adjusted until they pass the tests on which they are being graded. In all these scenarii, what is missing is intentionality. Students do not generate solutions with intent and, when questioned about why they wrote a specific segment of code, are often unable to provide a reason. Programming needs to be taught in a way which prevents students from seeing it as anything less than a creative endeavor. Many argued that Knuth’s titling of his book “the art of computer programming” captured the essence of this intellectual endeavor. If we fail to pass this important perspective to our students, we might end up training the next generation of programmers to act like a human-powered version of genetic programming algorithms [17].
5. REFERENCES [1] M. Kolling, B. Quig, A. Patterson, J. Rosenberg, (2003), “The BlueJ system and its pedagogy”, Journal of Computer Science Education, special issue on learning and teaching object technology, vol 13, no 4 [2] Y. D. Liang, Introduction to Java Programming: Fundamentals first, 6/e, Prentice Hall, 2007 [3] J.C. Giordano, M. Carlisle, “Tools and systems: Toward a more effective visualization tool to teach novice programmers, October 2006, Proceedings of the 7th conference on Information technology education SIGITE Constructive alignment theory [4] J. Biggs, “Teaching for Quality Learning at University”, Buckingham: Open University Press/McGraw Hill Educational, 1999, 2003 [5] Searle, John R. (1984) Minds, brains, and science. Cambridge, Mass.: Harvard University Press. [6] Johnston, C. (1996). Unlocking the will to learn. Thousand Oaks, CA: Corwin Press [7] M. Kolling, D.J. Barnes, “Enhancing apprentice-based learning of Java”, 35th SIGCSE technical symposium on computer science education, 2004, pp. 286-290
[11] JAVA BNF grammar diagrams, http://cui.unige.ch/dbresearch/Enseignement/analyseinfo/JAVA/BNFindex.html [12] Karl R. Popper. Objective Knowledge, An Evolutionary Approach. Oxford University Press, 1972. [13] Bruner, J., The Process of Education. Cambridge, Massachusetts: Harvard University Press, 1960 [14] Online resource, http://Krugle.com, last accessed 6/19/2007 [15] Bloom, B. S., ed., Taxonomy of educational objectives: The classification of educational goals. Handbook I, cognitive domain, Longmans, 1956 [16] Tzu-Yi Chen, Gary Lewandowski, Robert McCartney, Kate Sanders, Beth Simon, Common sense Computing: Using student sorting abilities to improve instruction. In Proceedings of SIGCSE, March 2007. [17] John R. Koza, Genetic programming: on the programming of computers by means of natural selection, MIT press, 1992 [18] Yifat Ben-David Kolikant, “Students' alternative standards for correctness”, Proceedings of the 2005 international workshop on Computing education research ICER '05 [19] R. Anderson et al., Supporting active learning and example based instruction with classroom technology, ACM special interest group on computer science education conference, SIGCSE 2007 [20] R. Anderson, R.Anderson, O. Chung, K. M. Davis, P. Davis, C. Prince, V. Razmov and B. Simon, “Classroom Presenter – A classroom interaction system for active and collaborative learning, workshop on the impact of pen technologies on education”, 2006 [21] J. Langr, Agile Java: Crafting code with Test Driven Development, 2005, Pearson [22] C. McDowell, B. Hanks, L. Werner, Experimenting with pair programming in the classroom, June 2003, ACM SIGCSE Bulletin , Proceedings of the 8th annual conference on Innovation and technology in computer science education ITiCSE '03, Volume 35 Issue 3 [23] Kevin Bierre, Phil Ventura, Andrew Phelps, Christopher Egert, "Motivating OOP by blowing things up: an exercise in cooperation and competition in an introductory java programming course”, March 2006, SIGCSE [24] A. Gaspar, N. Boyer. CLUE, C Learning Undergraduate Environment, http://clue-ide.sourceforge.net/, last accessed 8/6/2007
[8] M. Hailperin, SIGCSE-members mailing list, posts #34 and #37, July 10th 2006, accessed on 2/22.2007 at http://listserv.acm.org/archives/sigcse-members.html
[25] C.E. Willis, D. Finkel, M.A. Gennet, M.O. Ward, Peer learning in an introductory computer science course, ACM special interest group in computer science education conference, SIGCSE 1994
[9] R.P. Pargas, Reducing lecture and increasing student activity in large computer science courses, ACM’ Special Interest Group in Computer Science Education Conference, 2006
[26] H.A. Mayer, Symbiotic coevolution of artificial neural networks and training data sets, Lecture Notes in Computer Science, Springer, vol 1498, 1998
[10] A. Gaspar, S. Langevin, Active learning in introductory programming courses through student-led “live coding” and test-driven pair programming, accepted to EISTA 2007, Education and Information Systems, Technologies and Applications, July 12-15, Orlando, FL
[27] Schwaber, K. (2006), Online resource, Google Tech Talk webinar, Scrum et al., 9/5/2006, available at http://video.google.com/videoplay?docid=7230144396191025011&q=ken+schwaber+google+tech+tal ks&total=6&start=0&num=10&so=0&type=search&plindex =0, last accessed 7/27/2007
