IMPROVING PASS RATES IN INTRODUCTORY PROGRAMMING Peter Chalk
Tom Boyle
Poppy Pickard
Department of Computing, Communications & Maths London Metropolitan University London N7 8DB
[email protected]
Learning Technology Research Institute London Metropolitan University London N7 8DB
[email protected]
Department of Computing and Electronics Technology Bolton Institute Bolton BL3 5AB
[email protected]
Claire Bradley
Ray Jones
Ken Fisher
Learning Technology Research Institute London Metropolitan University London N7 8DB
[email protected]
Department of Computing, Communications & Maths London Metropolitan University London N7 8DB
[email protected]
Department of Computing, Communications & Maths London Metropolitan University London N7 8DB
[email protected]
succeed.
ABSTRACT
Java has become a very popular candidate for the teaching of introductory programming at university level. Java meets the constructivist criterion of being an 'authentic' topic for study [1]. It is a powerful, real world language that can be used to create applets for the Web or full software systems. Tutors also like the language because it embodies the Object Oriented paradigm that is so influential in modern computing. There are thus good reasons for teaching Java. Even universities that can select from among the best students, however, report difficulties in teaching Java. Thus Jenkins and Davey [2] say:
There is a national crisis in the teaching and learning of Programming. This is well documented in the conferences organised by the LTSN National Subject Centre for the Information and Computer Sciences. A major project was initiated in February 2002 to tackle this problem in two higher education institutions. The project initiated substantial changes to the curriculum supported by a major new online teaching and learning environment. The project has involved over 700 students at the two higher education institutions. The results for the first cohort of students to benefit from these developments show improvements in pass rates of between 12 to 21 percentage points.
1. INTRODUCTION
"Anyone who has presented an introductory programming module will be all too familiar with students who appear to be totally unable to grasp the basic concepts. Others who come to supervise final year dissertations will have been faced with students who insist that they want to avoid programming at all costs."
Many universities in the UK are facing larger student numbers and a more varied intake, and this is particularly acute in computing departments. A national concern over the teaching and learning of introductory programming is well documented. A key issue is that students find programming a difficult subject to learn. Early failure to understand key concepts in the first few weeks seems to undermine students’ confidence and their ability to
2. DESCRIPTION OF THE PROJECT
Keywords Learning Programming, Learning Java, learning objects.
To tackle this problem London Metropolitan University in the Department of Computing, Communications Technology and Mathematics, in collaboration with Bolton Institute, instituted a major action research project to improve the overall learning experience for students, and to improve retention and success.
Introductory Java programming is taught to over 600 students at London Metropolitan University each year in three distinct modules: HND, BSc and a conversion MSc course. There are a further 120 undergraduate students at Bolton Institute involved in the project.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission.
An internal study at London Metropolitan University (then the University of North London) during 2000-1 highlighted a range of issues being faced in introductory programming. The report made
4th Annual LTSN-ICS Conference, NUI Galway © 2003 LTSN Centre for Information and Computer Science
6
Each week, the students were given a set of exercises, including one assessed exercise. This exercise had to be annotated in their log book and submitted using the VLE (WebCT) to their tutor. Student submissions to the VLE, and their log book, were marked weekly.
recommendations for change, which helped shape the subsequent development work. The author of this study subsequently moved to Bolton Institute, which then collaborated in the ongoing project. Development work began in March 2002, with the creation of a core development team, consisting of departmental module tutors, and a senior researcher from the university’s Learning Technology Research Institute. During the summer the core team was supplemented by the addition of a multimedia developer and a research fellow.
This approach was again shared across the different modules. A common framework was provided which individual tutors could tailor to suit their particular student cohorts. Online learning environment
The group carried out significant developments in three main areas: •
Curriculum development;
•
Organization of the teaching environment;
•
Online learning environment.
The third major activity was building a major new online learning environment. This included the development of substantial learning and course support materials delivered through WebCT, the university's 'virtual learning environment'. These attractive resources are available over the Web; the students can access them from the labs or from their homes.
Curriculum development A common curriculum was established that, as far as possible, could be used in the modules. The use of a common curriculum allowed the concentration of resources that were traditionally dissipated in parallel development of course material. The four module tutors developed and shared a common set of material. This supported the development of rich teaching resources that would not have been possible for individual tutors working in parallel.
A key innovation was the creation of learning objects, which incorporate multimedia elements and interactive activities. These were stand-alone learning resources that were independent of WebCT (they were held on a separate server) but the students could access them through WebCT. The decision to structure the resources as learning objects was based on a number of considerations. It provided a good mechanism for parallel development in team-based development. Different individuals and groups could work on different objects. The development of discrete learning objects, based on clear learning goals, provided a basis for close attention to pedagogical quality. Formative evaluation, usually based on feedback from members in the team, was improved. The objects could, in turn, be independently evaluated for their usefulness and attractiveness to students. The learning object repository also provided a base for the continued evolution in the quality of the courses. Course leaders in the future would be able design the structure of their course and populate it with appropriate existing learning objects. They would also be able to develop their own objects that could be added to the repository for future re-use.
A good example was the use of interactive visualization to introduce topics that the students otherwise found very abstract and difficult to grasp. All the students' early programs involve creating graphical output that directly displays the results of the code (instantiating graphical objects and changing the attributes of these objects). This was supported by a special graphics library developed by one of the tutors. This common, collaborative approach to curriculum development thus supported the tutors in developing a richer set of learning and teaching support resources than would have been possible for any tutor on their own. Teaching environment A second major activity was to plan, implement and evaluate improvements to the organizational environment. This included issues such as getting the students over the crucial first few weeks of the course, more effective use of tutorial assistants (TAs), and better co-ordination between lectures and TA activity in the lab.
An example of a learning object developed for the project is given in Figure 1. This provides a Flash based multimedia learning experience. This particular object introduces the 'While' loop, using both interactive animations and scaffolded support for the learner's construction of the target construct.
Learning a new language and a new set of problemsolving techniques, as in programming, is a complex activity. One common method is to set small scale problems within a regime of continuous assessment, supervision and constant feedback, starting with realistic problems and gradually unfolding the complexity of real modern software.
Figure 2 illustrates how this object appeared to the students through WebCT. Though each object is an independent cohesive learning 'event', the students accessed these events through a rich, integrated WebCT environment. Only a brief overview of the design of the learning objects has been given here. A fuller description of
7
environment. WebCT simply provided the overall delivery platform, with the learning objects being loaded dynamically at run-time.
the design of the learning objects, which elucidates the influences of software engineering and pedagogical principles is provided in [3] and [4].
3. EVALUATION A thorough evaluation of the project has been (and continues to be) conducted. The evaluation includes: •
Gathering information on the students' responses to the new developments – through observation, questionnaires, and interviews;
•
Extensive logging and monitoring of the use of the online resources;
•
Assessment of the impact on success/fail rates in the modules concerned.
An evaluation framework for the first semester was managed by a researcher who is part of the design team. Students responded to three questionnaires, start, middle, and end of the semester, together with individual interviews conducted several weeks into the course. WebCT provided page-tracking statistics to monitor the use of learning objects.
Fig 1: Visualization of program behaviour
The in-course assessment and examination provided further basis for comparison together with results from previous cohorts taking the same subject. Table 1 gives a reflection of student opinion of the main elements in the teaching and learning environment. The results are based on the questionnaires given to the students. It is surprising how closely the Bolton and London opinions match despite the difference in student bodies: London Metropolitan University with over 600 students studying for HND, BSc and MSc on a variety of courses and Bolton with 120 students studying for a BSc in a range of computing disciplines such as Computer Games Software Development. The age profile of the Bolton students is lower than that of the London students. The culturally diversity at Bolton is narrower than London, and only 5% of students are female compared to 25% in London.
Figure 2: Accessing learning objects from WebCT
From the outset there were plans for a larger initiative in learning resource development. Developing learning materials is a resourceintensive and time-consuming activity, and the lack of resource sharing amongst computing departments is a national concern. London Metropolitan University is a partner site in the UK LTSN National Subject Centre for the Information and Computer Sciences. The present project is acting as the preliminary step in exploring the potential of setting up a shareable repository of learning objects for introductory programming. The idea is that resources could be developed by and shared amongst institutions. The learning objects were therefore developed both to meet our immediate pedagogical needs and to serve this larger goal.
How useful are:
Very useful
Useful
Not very
Use less
useful Lectures?
37%
48%
13%
2%
31%
50%
16%
3%
Lab exercises?
42%
44%
13%
1%
48%
42%
9%
1%
Text book?
18%
38%
27%
18%
8%
39%
17%
7%
29%
64%
8%
0%
15%
80%
5%
05
Text learning aids?
To ensure reusability the eLearning objects were developed and maintained outside of the WebCT 8
Animation learning aids? Quizzes learning aids?
in
46%
44%
10%
1%
36%
50%
12%
2%
34%
56%
8%
2%
24%
59%
17%
0%
The questionnaires and interviews give an indication of the student's views of the main features of the new Java course. However, the acid test is the impact of success rates at the end of the modules. Table 2 provides the figures for the increases in pass rate achieved across the four modules. The module results show a marked improvement in both institutions, when compared with the previous year's results. In each case there are positive and large increases in the numbers of students passing the module. However, again, these figures should be treated with caution, and further results are being sought to confirm any improvements in understanding.
Results for London in normal type, and for Bolton in Italics. Number of students who returned questionnaire = 318 (LondonMet: 223, Bolton: 95). Table 1. Results from questionnaire – usefulness of module components
Students show a very positive response to the learning objects (referred to in the course as learning aids) and also the lab exercises which were based around using the graphic software library.
Course
Percentage Increase Pass Rate
Further feedback on learning aids was sought through structured interviews primarily undertaken at London Metropolitan University. The students’ comments give more insight into their views Ö
MSc London Metropolitan
+12
HND London Metropolitan
+15
BSc London Metropolitan
+19
Usefulness of text-based learning aids:
BSc Bolton
+23
in
Table2. Increase in pass rates for the four modules
“Good – better than reading a big book.
Column 2 in the table indicates the percentage points increase in pass rate. Thus an increase of 50 to 60 would be counted as a 10 point increase, not a 20% increase (i.e. 10/50 multiplied by 100).
Better on the eye.” “Very useful – no problem understanding it.” Usefulness of the animated learning aids: “Good, you can see the code, shows what’s
These increases may be attributed to the holistic impact of the changes made in the course. It is difficult, and in some cases may be impossible, to tease out the influences on individual factors. We are carrying out further detailed analysis to elucidate, as far as possible, the main factors related to failure in the modules.
going on when you press run. Interactive.” “Good – shows step by step the program. Animations help a lot.” Usefulness of the quizzes built into the learning objects:
7. FURTHER WORK
“OK – helps you to look closely at the syntax of the code to get the right order.”
The Java module is continuing to be taught during the second semester in both institutions, but with smaller cohorts. A second version of the graphic software library has now been written and this is being piloted at Bolton. The classes have been simplified to make the sources code more accessible to students.
“Good – reconfirms you know what you’re doing.” The best feature of the course: “Learning aids very helpful as I don’t have all the books.”
The learning objects will remain the same for this semester pending the completion of the evaluation. A future aim of the learning object work is to create a shareable repository that can be used across different universities and managed by the LTSN National Subject Centre for Information and Computer Sciences. This is in line with the elearning movement nationally which is working towards standardization and reuse [5].
“Everything is on the web. Can access from home.” Most of the comments made were positive, in keeping with the pattern of the comments above. The proportion of negative comments made about the learning aids was very low, although care must be taken with reading too much into student comments. Tracking statistics showed that most students were regularly using the learning aids. Usage tended to increase just before assessments, indicating revision usage.
8. REFERENCES [1]
9
Grabinger R. S. and Dunlop J. C. (1995) Rich environments for active learning: a definition. ALT-J, 3(2), 5-34.
[2]
[3]
Jenkins T. and Davy J. (2001) Diversity and motivation in introductory programming. Italics, 1(1). Available at http://www.ics.ltsn.ac.uk/pub/italics/issue1/tjenk ins/003.html Boyle, T. (2003). Design principles for authoring dynamic, reusable learning objects. Australian Journal of Educational Technology, 19(1), 46-58.
10
[4]
Bradley, C., Boyle, T., and Haynes, R., (2003), Design and evaluation of multimedia learning objects, to appear in the proceedings of the ED-MEDIA 2003 conference, June 23-28, Honolulu
[5]
CETIS – Centre for Educational Technology Interoperability Standards: http://www.cetis.ac.uk/
