Methodologies for teaching new programming languages: A case study teaching LISP. A. E. Nicholson Department of Computer Science Monash University Clayton, Victoria, 3168 E-mail
[email protected]
Abstract A primary aim of computing degrees is to produce graduates with general programming skills, the ability to program in several programming languages, and the ability to learn new programming languages. In this paper, two approaches to teaching a programming language are identi ed: \start small", where students begin writing small pieces of code from scratch; and \codeprovided", where students are given existing code to modify and extend. Each approach has been used for teaching introductory programming, and for teaching subsequent additional programming languages. This paper reports on a case study involving the teaching of LISP programming. The class was divided into two groups and given di erent assignments, in accordance with the two approaches, for their rst two assignments. A common third assignment, and a LISP programming question on the exam, were used to compare the impact of the assignment-based teaching methodology on student programming skills in the new programming language. The teaching methodology was found to have no signi cant e ect on student programming outcomes, with performance in a core prerequisite programming subject being the only predictor. In addition to the presentation of the details of the study and the results, comments from student surveys, and ethical and logistical issues in undertaking such a study, are presented and discussed.
K. M. Fraser Centre for Higher Education and Development Monash University Clayton, Victoria, 3168 E-mail
[email protected]
1 Introduction
Computing degrees aim to prepare students for entry into the computing profession [1, p.7]. A critical component of many jobs in the computing profession is pro ciency in programming which is de ned as \the entire collection of activities that surround the description, development, and e ective implementation of algorithmic solutions to well-speci ed problems" [1, p.16]. Programming occurs in all subject areas in the discipline of computing [1, p.17]. While there are literally hundreds of programming languages, at any given time several programming languages will dominate in the commercial environment, with the particular languages changing over time. Examples of such changes are the shifts from C to the object-oriented version C++ within the last ve years, and, within the last two years, the Internet motivated emergence of Java. As the dominance of programming languages continually changes, there is a clear need for computing degrees to ensure that students have the necessary skills to learn a new programming language when required1 . In order to achieve this, computing degrees invariably require that students learn several programming languages, preferably from a range of di erent programming paradigms2. When learning any programming language, students learn basic programming skills which can be transferred to other languages, as well as uency in the speci c language being studied. Regardless of the language rst studied, students learn a core of programming knowledge and skills which can then be used in the learning of new languages. This core of knowledge and skills is then used by the student to write 'good' programs, which Weinberg [9, pp. 15-25] de nes in terms of the following four essential criteria:
Correctness: that the program meets it speci cations, and produces the correct output for all possible inputs;
1 Weinberg [9, Chapter 10] discusses the on-going learning required of a programming practitioner; he is particularly scathing about formal education in programming, although he doesn't identify explicitly the teaching methods used. 2 One classi cation of the major programming paradigms is: imperative, declarative (logic and functional), object-oriented.
Eciency: the program uses acceptable time and space resources; Schedule: software is produced within the production time requirements3 . Adaptability: able to be maintained over its lifespan. To construct ecient and adaptable programs, students need to write well-structured and documented code. However when learning a new programming language, students have diculties producing wellstructured, well documented and correct code. The authors of this paper investigated two di erent ways of teaching students to program in a new programming language. The e ect on student programming skills of the two di erent teaching approaches were compared. The role of programming assignments in teaching programming languages, and two general approaches to setting them, are described in Section 2. The experimental study is described in Section 3, with the results of the study given in Section 4. The nal sections discuss the outcomes of the study and outline possible directions for future research.
2 Approaches to Setting Programming Assignments 2.1 Current practice for teaching programming languages
Programming languages are typically taught as an individual subject, or as part of a subject in a particular computing eld. The teaching structure takes the following format: the syntax and semantic of the language are presented in lectures, together with code examples; one or more texts on the programming language is prescribed for the subject; students undertake laboratory work4 where they design, implement and run programs in the programming language. Assessment is some combination of programming assignments, a practical programming examination, and traditional written examination. It is generally agreed that uency in a new programming language can only be attained by actually programming. While lectures are used to demonstrate examples of good programming in new language5, the actual learning of programming skills occurs through the completion of assignments. The teaching methodology used to teach programming skills is primarily the setting of assignments. This paper reports on the impact on student programming skills of two di erent approaches used to set assignments. 3 This falls more into the realm of software engineering than in a subject teaching uency in a programming language. 4 This may be in supervised class situation or on their own. 5 East [4] believes that \students may not develop into good programmers unless they see good programs" and that it is the responsibility of teachers to ensure that this occurs, however he does not explicitly distinguish between presenting such examples in lectures, or in the actual programming environment.
2.2 The traditional, \start-small", approach. One approach used to teach students to program is to ask them to write very small, simple programs. Students are then gradually required to write code for increasingly more complex and structured programs. Pedagogically this approach provides students with the basic syntax and semantics of the language, and the intention is that the student will be able to follow the execution of pieces of code. This approach has been used to teach students to program since at least the early 1980s and is still used today, as evidenced by programming text books [5, 3]. The approach is used to teach both rst and subsequent programming languages. One of the limitations of this approach is that students do not work with more complex, sophisticated examples of the code and as a result, students often produce \working" code that is poorly structured or unsophisticated. Such code may use inappropriate language structures, may use a limited range of the languages' features, and may not be suciently modular. In particular, students lack a framework for producing larger, structured code. Another limitation of the approach is that the problems that can be solved with small pieces of code tend to be simplistic, contrived and mathematical rather than related to \real world" problems. The resultant assignments tend to be uninteresting to students and that can make it dicult for the lecturer to motivate an interest in learning the programming language 6 .
2.3 An alternative, \code-provided" approach. Another teaching approach used to introduce students to programming is to provide them with existing code for the core of the programming assignment. Students are then required to write additional pieces of code for the program and/or modify the supplied code to make it work or extend its functionality. This approach has been used to teach basic programming skills in a rst programming course 7, and subsequent programming languages. The pedagogical rationale for this approach is that the existing code models desirable code structures and provides example code in the context of the problem set for the students. Advocates of this approach also claim that it re ects the experience of programming practitioners in the work environment when they are required to learn a new programming language, typically by maintaining 6 \With beginning programmers, one major obstacle is lack of motivation needed to carry students through early mechanical problems" [8]. 7 For example, Brown University, CS015 Introduction to Object-Oriented Programming and Computer Science.
or modifying existing software8 . One criticism of this approach is that while students learn to recognise good program structure and style, they do not learn how to design and code such programs from scratch.
2.4 Which approach?
A formal evaluation of the e ect of the \codeprovided" teaching approach on student programming skills, in comparison to the traditional \start small" teaching approach has not been previously undertaken. This paper describes a research project that investigated and compared the e ects of the two di erent approaches to teaching on the skills of students learning a new programming language at the third year university level.
3 Experimental Study 3.1 The Case Study Subject
The test subject was a third year Computer Science subject at Monash University, CSC3091 Arti cial Intelligence. The subject introduces students to a new programminglanguage and paradigm, LISP and prior to 1996 the language was taught using the traditional \start small" approach. Although students are senior students who have already learnt at least two programming languages, LISP has a completely different syntax from the other languages and is based on a di erent programming paradigm (the functional programming paradigm). In practice the students
oundered when learning this language. Their diculties were compounded by the fact that there were no laboratories or tutorials for the subject. Prior to 1995 the programming assignments formed only 20% of the course and were not compulsory. 10 - 20% of students did not complete the assignments. In 1995 the programming assignments were made a hurdle for the subject. Although this led to greater e ort on the part of the students, much of it was ill-directed and much of the code written for the second assignment of two assignments, while satisfying the basic requirements of solving the AI problem, was poorly structured, used limited features of the language, used very strange combinations of programming constructs and was, in many cases, unintelligible.
3.2 Methodology
An experimental methodology was chosen in which the 124 students in the subject were divided into two, equivalent groups based upon their success in the second year Science of Programming subject CSC2040 (refer to Table 2). Both groups attended the same lectures (four lectures on the syntax and semantics of LISP), sat the same end of semester examination and completed the same programming 8 C. Bellamy, Dean of the Monash University Faculty of Computing and Information Technology, personal communication, December 20, 1995
assignment just prior to the exam. As in previous years, students were shown small code examples in lectures while slightly larger code examples were available online. A language text [10] was their main resource. The di erence in treatment for the two groups centred on the requirements of their rst and second programming assignments. While students in both groups were required to demonstrate the same programming skills and were assessed using identical criteria, the tasks they were given di ered. The control group was asked to write code using the \start small" approach while the experimental group was asked to write code associated with the provided code. The assignments for both groups were designed to require the same time commitment and to be of a similar level of diculty. This was con rmed by having a graduate student complete assignments one and two for both groups.
3.3 Assignments
The assessment criteria for both groups for all assignments was identical: correctness 80%, style, structure and documentation 20%. For both groups, correctness was determined only by the output of their code given a test set of inputs. Students' understanding of the code was assessed by their documentation.
3.3.1 Assignment 1 \Introductory LISP programming"
Students were required to write and execute LISP code. Control group students were provided with an English language description of the functionality required of the code. These students were required to produce and document ve small LISP functions, each of which had a straightforward solution which required less than ten lines of code. Experimental group students were provided with an English language description of the functionality required of the code and were also given existing code. They were required to add code to the existing code, change nonfunctioning code, lay out nonformatted code and document the code produced. Both groups were assessed for the production of working, well structured code (refer to Appendix 1 for examples of questions set for Assignment 1).
3.3.2 Assignment 2 \Arti cial Intelligence programming"
Students were given the option of working in pairs with someone from the same group. Control group students were provided with an English language description of the functionality required of the code. They were required to implement an Arti cial Intelligence algorithm presented in the lecture, the alphabeta game playing algorithm for tic-tac-toe. Experimental group students were provided with working functions which contained some internal doc-
umentation for the problem of simulating a vacuum cleaning agent[7]. Students were asked to run each function. They were required to add functions to the existing code in order to achieve extra functionality (di erent types of agents). The amount of code provided, in terms of numbers of lines, is given in Table 1. Table 1: Lines of code provided to experimental group for Assignment 2 on a vacuum cleaning agent simulation (output from UNIX command wc -l *.lisp agents/* utilities/*). Lines 191 387 10 6 161 405 16 96 357 73 130 52 554 2438
Files aima.lisp dirt-world.lisp load les.lisp agents/README agents/basic-env.lisp agents/grid-env.lisp agents/test.lisp agents/vacuum.lisp utilities/binary-tree.lisp utilities/cltl2.lisp utilities/queue.lisp utilities/test.lisp utilities/utilities.lisp total
3.3.3 Assignment 3 \AI Search Algorithms"
Students in both groups were provided with an English language description to implement an algorithm presented in the subject9 . They were advised of online sources for code which they might nd useful10 and provided with suggestions for program structure.
3.4 Data Collection
Student results for both groups were collated and compared using a t test analysis (in SPSS). The results for the following students were not included in the nal analysis: students enrolled in a graduate diploma in Computer Science. These students were concurrently doing CSC2040 and therefore their results from this subject could not be used to divide students between the control and experimental groups; students who did not sit the nal exam (i.e. they had e ectively dropped out of the course although still enrolled, or under special consideration had a deferred exam); students who changed from one group to another during the course of the semester (University ethics required they be given this option); 9 The A* search algorithm, for a sliding tiles problem as described in [6]. 10 LISP code for tree structures and search provided with [2].
the one student who explicitly said he did not want to be included in the study; students who were identi ed as cheating on the nal assignment11.
In addition, adjustments to marks made for the submission of late assignments were ignored; that is, the raw assignment marks were used in the analysis. After removing the results of the students listed above, the control group included forty-one students and the experimental group included forty-three students (refer to Table 2). The two groups were compared on the scores of each of the three assignments, the nal exam, each question within the exam and the total nal grade for the subject. Results were also analysed to determine the impact of approach on gender and on prior, demonstrated, programming \ability". The latter was determined by grouping students into three achievement bands based on their results in the second year programming subject. Table 2: Student numbers in the control and experimental groups before and after exclusions. Before Exclusion Control (A) Experimental (B) After Exclusion Control (A) Experimental (B)
Total Male Female 61 52 8 63 51 12 41 43
35 34
6 9
3.5 Student Perceptions
Students were asked to complete a survey which used ve point Likert scales and open comments to report on their perceptions of the approach they used in Assignments 1 and 2. The survey was validated by asking both discipline and evaluation experts to evaluate the survey. Modi cations to the survey were made based on their comments. The reliability of the survey was checked by trialing the survey with CSC3091 students. Again modi cations were made based on student comments. In the second last week of lectures students were asked to complete the survey in the nal quarter of the lecture. Students who were involved in the trial of the survey were asked not to complete the survey. Sixty- ve students completed the survey. A t-test analysis was used to compare the control and experimental groups' responses on the ve point Likert scales. 11 \Cheating" in this context means submitting code that was obviously directly copied, or copied with only super cial syntactical changes, from another student. In some cases the students involved admitted the assignment had been a group e ort, in others, the assignment had been passed along. Cheating was detected by a combination of: the use of an automated LISP cheater-checker that looked at patterns in the allocation of internal registers; inspection by the human marker; keyword searches on certain LISP functions or structures.
3.6 Ethical Issues
Ethical issues for an experimental methodology included: 1. not disadvantaging either group. Because there was clear support for both approaches and no previous research indicated conclusively that either group would be disadvantaged, the study could be undertaken; 2. student choice. Students had to be informed that the experimental study was taking place and they had to be allowed to change groups if they so desired; 3. standardisation. If signi cant di erences in student marks between groups resulted, standardisation had to be used to correct for the perceived disadvantage.
4 Results 4.1 Student comments
While there was no signi cant di erence between the two groups in responses to the Likert scales in the survey, student comments were insightful. While not claiming to represent all student perceptions of the two di erent teaching methodologies, the following types of comments were echoed by students in the di erent groups. Some students in the control group commented that writing code from scratch was an important skill while other students likened the approach to being \thrown in the deep end" and wanted to be given code \to look at and understand". Students in the experimental group commented that the provision of the example code showed them how \the code should look (its structure, syntax etc)." They also commented that for Assignment 2 (the vacuuming cleaning agent simulation) they had diculties because there was \too much code" to look at and understand. Students in both groups responded that neither of the methods showed them how to document code. Some students in both groups thought that their approach had prepared them well for the exam while other students in both groups thought that their approach had not prepared them well for the exam.
4.2 Assignments and the exam
t-test analysis showed no signi cant di erence between the experimental and control groups on nal subject scores, total examination scores, speci c programming examination question scores, Assignment 1 or Assignment 3. There were no signi cant di erences between gender and achievement in either the control or experimental groups. There was also no signi cant di erence between achievement in the second year programming subject CSC2040 and achievement in control or experimental groups although overall success in CSC3091 was predicted by success in
CSC2040. There was a signi cant di erence between the two groups in success on Assignment 2; the experimental group obtained a signi cantly higher score for the assignment (refer to Tables 3 and 4). Table 3: Signi cant di erence in mean scores for Assignment 2 between the experimental and control groups. Group
Sample Mean Std Size Score Dev Control 41 17.7683 4.537 Exper'tal 43 20.5930 2.823 Mean Di erence = -2.2174 Levene's Test for Equality of Variances: P=0.000
SE of mean 0.709 0.430 F=14.475
Table 4: Signi cant di erence in mean scores for Assignment 2 between the experimental and control groups. Var. t-val. df 2-Tail SE of 95% Sig. di . CI equal -3.44 82.00 0.001 0.820 (-4.457, -1.193) uneq. -3.41 66.36 0.001 0.829 (-4.480, -1.169)
5 Discussion and future directions 5.1 Student Comments
Comments from students in the experimental group, referring to diculties with the amount of code they were given, may re ect the predominance of the \start small" approach in their previous programminglearning. Lack of comfort with exposure to signi cant amounts of code have wider implications for the degree curriculum as a whole. The comments from students indicating lack of guidance or examples for documentation requirements were unexpected, given the amount of example code provided (especially for the experimental group) which was documented, and given that all students should have completed a second year software engineering subject as a prerequisite, in addition to documentation being required for all previous programming subjects.
5.2 Di erences in Assignment 2
It is interesting to consider possible explanations for the di erence in achievement of the control and the experimental groups in Assignment 2. One explanation is that, despite the care taken while setting the assignments to ensure that the problems had similar complexity and time requirements, the coding requirements for the experimental group may have been
in some sense \easier". Another explanation is that students found it easier to complete the assignment when given a foundation from which to start, i.e. the supplied code provided a better starting point. While students in both groups rated the approaches equivalently in terms of helpfulness in learning to write LISP code, in open ended comments, some students from the control group said that they wished they'd had examples of code from which to work. Experimental group students commented that being provided with examples of code assisted them to learn the new language.
6 Conclusions
5.3 No di erence for Assignment 3 or Exam
Appendix 1: Example Questions from Assignment 1
Despite the signi cant di erences between the two groups in the results for Assignment 2, the skills of both groups at the end of the subject were the same, as tested by both the end of semester programming assignment and programming questions on the end of semester examination. Clearly the results of this research do not support the use of the \start small" teaching methodology over the \code provided" teaching methodology, or vice versa. However, it could be argued that students at the third year level of a computer science degree have already learnt the generic skills of constructing and documenting code. These generic skills might then be applied when learning any new programming language. If this is the case, then the teaching approaches trialed in this project may not impact di erently on third year students' programming skills. It is however, possible that the teaching approaches may impact di erently on the programming skills of rst year students who have little programming experience. Of course the lack of signi cant di erence between the two groups on the exam and the third assignment may result from the di erences in student learning preferences. The two di erent teaching methodologies may appeal di erentially to di erent student learning styles. If this was the case, a signi cant di erence in outcomes would not result because the students were not assigned to the teaching methodologies on the basis of preferred learning style. While students were supplied with some code examples for the third, common, programming assignment, that assignment was essentially intended for the students to complete their own LISP program starting from scratch. It is quite possible that a different assignment, requiring students to modify and add to existing code, may have produced di erent outcomes, for example with the experimental group performing better. However, this was not undertaken due to the restriction on the number of assignments that could be set for the subject.
Write a function intersperse that takes an object and a list, and returns a new list in which the object appears between each pair of elements in the original list.
Clearly the results of this research do not support the use in this context of one teaching methodology over the other. There is however, scope for further research into the impact of using the two methodologies on student programming skills. For instance further research could be conducted to determine the impact of the two teaching methodologies on the programming skills of rst year computer science students. Another possible project could repeat the research reported in this paper but vary the third assignment to re ect the \code provided" teaching methodology rather than the \start small" teaching methodology.
Control Group Problem 4 (4 marks)
1. Using recursion. 2. Using iteration. Example: > (intersperse '- '(a b c d)) (A - B - C - D)
Experimental Group Problem 4 (4 marks)
The permute function takes a list as argument and returns a list of permutations of the original list. For example: Example: > (permute '(a b c)) ((A B C)(A C B)(B C A)(B A C)(C A B)(C B A))
Unfortunately, the following version of permute does not work properly, nor was it written in a very readable style. Correct this version so that it works; in the process, try to also improve the style of the code and comment it. (Look at the code examples in the directory /cs/cc/usr/examples/csc3091 to see examples of good LISP programming style). Before you start, make sure that you understand how the more basic functions, such as length, append and mapcar work. Experiment with these basic functions by calling them with simple arguments.
(defun permute (var) (if (> (length var) 1) (do ((n (length var) (1- n)) (var2 var (append (rest var2) (list (car var2)))) (result ())) ((
