Open Learning

This article was downloaded by: [Newcastle University] On: 06 November 2014, At: 06:38 Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Open Learning: The Journal of Open, Distance and e-Learning Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/copl20

Collaborative learning amongst distance learners of mathematics a

Kurien Philip & Sugata Mitra

a

a

School of Education, Communication and Language Sciences , Newcastle University , Newcastle upon Tyne , UK Published online: 14 Sep 2012.

To cite this article: Kurien Philip & Sugata Mitra (2012) Collaborative learning amongst distance learners of mathematics, Open Learning: The Journal of Open, Distance and e-Learning, 27:3, 227-247, DOI: 10.1080/02680513.2012.716655 To link to this article: http://dx.doi.org/10.1080/02680513.2012.716655

PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/termsand-conditions

Open Learning Vol. 27, No. 3, November 2012, 227–247

Collaborative learning amongst distance learners of mathematics Kurien Philip and Sugata Mitra*

Downloaded by [Newcastle University] at 06:38 06 November 2014

School of Education, Communication and Language Sciences, Newcastle University, Newcastle upon Tyne, UK This paper describes the effects of collaborative support on groups of distance learners of General Certificate of Secondary Education mathematics. Two experiments are described. In each case, learners were provided with teleconferencing with peers, experts and mentors in addition to structured study materials. No improvements were noticed in course completion rates and the learners were seen to disengage from the teleconference over time. When provided with a virtual learning environment, learners were, once again, found not to use the practice or support features of the system. The results from our experiments are intriguing in that they seem to indicate that scaffolding in the form of structured material, collaborative environments, and a virtual learning environment does not seem to affect performance in any way other than an improvement in revenue realisation. Keywords: virtual learning environment; student support; revenue realisation

Introduction Distance learning, since its inception over a hundred years ago, has promised flexible learning at the learners’ convenience. Over the years this promise has been diluted and made ineffective by an apparently simple problem (Carr, 2000; Galusha, 1997). Distance learning requires a higher amount of commitment from the learner to reach the promised learning outcome and, as a result, many distance learning organisations are challenged about the non-completion rates among learners. The sustainability of distance learning organisations depends on finding ways to retain students and provide more effective (therefore more financially sustainable) distance learning programmes. In this paper we report the results of experiments to study the impact on effectiveness of distance education courses using group learning, support through encouragement and structuring of content. The questions we address are as follows: (1) Will people complete their learning if they are able to connect with other learners? (2) Is there an increase in the learning effectiveness among these learners? (3) Will constant encouragement help the learners with a better learning experience and thereby improve course completion rates? (4) Will such intervention help with better financial sustainability?

*Corresponding author. Email: [email protected] ISSN 0268-0513 print/ISSN 1469-9958 online Ó 2012 The Open University http://dx.doi.org/10.1080/02680513.2012.716655 http://www.tandfonline.com

Downloaded by [Newcastle University] at 06:38 06 November 2014

228

K. Philip and S. Mitra

The experiments were carried out at International Correspondence Schools Limited (ICS), Glasgow, Scotland. ICS are among the world’s first distance education providers, having started admissions from 1890. Collaborative and group learning has been researched since the 1970s (see, e.g., Blumenfeld, Marz, Soloway, & Krajcik, 1996). Computer-supported collaborative learning is a relatively recent field of great interest. ‘The inclusion of collaboration, computer mediation and distance education has problematised the very notion of learning and called into question prevailing assumptions about how to study it’ (Stahl, Koschmann, & Suthers, 2006). Computer-supported collaborative learning amongst children became possible only when computers became available to children and when Internet bandwidths became useable for audio visual media and entertainment. Research on computer-supported collaborative learning and children is relatively recent (see, e.g., Crook, 1998). The ‘Hole in the wall’ experiments (Mitra et al., 2005; Mitra & Rana, 2001) consisted of computers embedded into walls of slums and villages and used by unsupervised groups of children. These experiments showed that children in groups can learn to use computers and the Internet on their own, with no prior knowledge and with no, or very little, literacy. Subsequent experiments showed that such groups of unsupervised children can attain educational objectives by themselves, in the absence of schools (Inamdar, 2006). Children have been shown to learn algebra by themselves (Nicaud, Bittar, Chaachoua, Inamdar, & Maffei, 2004). Several instances of educational progress have been measured (Dangwal, 2005) and the process by which such unsupervised and self-organised learning happens have been described (Dangwal & Kapur, 2008). This body of research seems to show that children in unsupervised groups can (Mitra et al., 2008): • learn to use computers and the Internet on their own; • search the Internet for information, read, understand and evaluate what they have found; • answer questions about curricular subjects they have not been taught; and • compensate for inadequacies in the quality of school teaching. We decided to design our interventions based on these results.

Experimental design The experimental design consisted of: (1) (2) (3) (4) (5)

selecting a course on which interventions would be made; selecting samples of students whose performance would be monitored; selecting measures for academic and financial performance; designing the interventions; and applying the interventions and observing their effect on performance.

Course selection ICS has many courses on offer. The range includes General Certificates of Secondary Education (GCSEs) and A-levels, degrees, professional, vocational, information

Open Learning

229

technology, creative and interest based courses. The registration numbers range from a few in number in some courses to hundreds each month for some others. While selecting a course, we had two options:

Downloaded by [Newcastle University] at 06:38 06 November 2014

(1) Select a course that is among the best selling but with low completion numbers, and analyse the impact of the experiment on the course completion rates. (2) Select a course that is in the mid-to-high selling range with low completion numbers, and analyse the impact of the experiment on the course completion rates. For the best-selling courses, marketing efforts are least likely to change during the duration of the experiments, since the courses are already doing well. Thus, taking the best-selling course would ensure that external/alternate factors (such as marketing promotions, etc.) have a minimal impact on measurements. Further analysis also revealed that selecting the best-selling course, in this case a GCSE mathematics course, would also provide us with the maximum variety in audience profiles – from school children, to home scholars, to working and retired professionals – all with their own different personal reasons for undertaking the course. We decided to select the GCSE Mathematics as our pilot course. Learner group selection With a rolling registration process, each month sees registration numbers in the hundreds for the GCSE Mathematics course at ICS. We had two options for selecting an experimental group: • We could take new registrations and group them together as the experimental group. • With many existing learners already at various stages of course completion, it might help to get people at different stages of completion and analyse the impact of the interventions on course completion. We decided to use existing learners in the experimental group. Although contacting these learners would be more difficult, using them in the experimental group would give us a wider range of students at various stages of study. This would enable us to establish any relationship between the self-effort put in by the learners and the support they required to complete a lesson. Also, existing learners were more likely to permit our intervention (which would be optional) than new learners who did not have any prior understanding of the organisation and would not understand why some were to be treated differently. Course description GCSE Mathematics is one of the highest selling courses. A well-structured course, created by experts, the course is designed for 120 hours of learning time. The course has six units and each unit has activities and progress checks at regular intervals within the unit. Each unit ends with an evaluative test paper. (Note: Unit 5 has two test papers and Unit 6 has none.)

Downloaded by [Newcastle University] at 06:38 06 November 2014

230

K. Philip and S. Mitra

Course completion data Each activity and progress checks have solutions provided at the end of the unit, while the test paper has to be completed by the learner in a self-determined controlled environment and submitted to their tutor for evaluation. The tutor evaluates the test paper and in addition to the score provides feedback on the solutions, the methods used and any tips to help with the test papers. The test papers are the only documents that are submitted to the organisation and therefore are used as the measurement criteria for determining course completion percentage. Based on data gathered over seven years (2002–2009) with a sample of 13,429 students, the test papers’ submissions decrease as the learners progress in the course. This is shown in Figure 1. We notice that while about 30% of learners submit the first test papers, only about 4% complete the course. Completion of the GCSE Mathematics course refers to the learner submitting all five test papers to the organisation. Beyond this, there is no method to track whether the learner has gone ahead and actually attempted the GCSE examination. Processes The processes involved getting together a group of learners and providing them with a way to connect with each other. Based on prior experience with learners, and considering the drop-out rate among distance learners, it was decided to form groups of seven to eight learners. Two options were explored for moderators: • Non-expert moderator – an individual who is not a subject-matter expert. Would be able to guide the discussions but unable to answer any questions raised during the discussions. • Subject expert moderator – an individual who is a subject matter expert. Would be able to guide the discussions as well as answer any questions raised. The ‘challenge’ with an expert as a moderator could be that the moderator would not give enough time for the learners to think about and solve the challenge by intervening with additional knowledge, information or the solution itself.

Figure 1. GCSE Mathematics test paper submission data from 2002 to 2009 (13,429 learners).

Open Learning

231

Downloaded by [Newcastle University] at 06:38 06 November 2014

It was decided to use both type of moderators in selected groups. This would enable us to compare the effectiveness of both kinds of experts. Technology With a plethora of technologies available for collaboration, many options such as live classroom, web-based chat rooms, web-based conference rooms, and so forth, were explored. Keeping in mind the audience profile, and also the constraints on available technology, the requirement was to have the simplest system possible. This would ensure that the focus was on the group discussion rather than ‘struggling with’ technology. With this focus, it was decided to use a telephone conference system (the Plain Old Telephone System). To ensure that this service did not lay any further financial burden on the learner, a free-phone number was provided. The learner dials a free-phone number and keys in a pre-provided passkey. When the moderator dials in and connects using the moderator passkey, all learners are connected together on a voice call.

Next steps Having selected the course and decided on the type and kind of groups and the conference mechanism, we laid out the next steps for the process as follows: (1) Create a list of possible time slots for the group calls and allocate tutors to the groups. (2) Define a format for the call/group interaction. (3) Create guidelines for call handling for moderators. (4) Create a list of learners, contact them to check for participation in the pilot programme and allocate groups based on time preference. (5) Run the pilot programme and track data. 1. Create a list of possible time slots for the group calls and allocate tutors to the groups. The first step was to create a list of possible time slots for the group calls. Evening time was preferred as this would be most suited to working professionals pursuing a distance learning course. It was also decided to have weekly, fortnightly and monthly calls to assess the relationship between attendance and frequency of group interactions. The availability of moderators for the group calls also influenced the time planning. It was decided to use in-house subject mentors as non-expert moderators for four (of the seven) learning groups. The existing tutors (who are external to the organisation and on contract), would function as the subject-matter expert moderators for three (of the seven) learning groups. As subject tutors are external to the organisation, they were agreeable to any time. The organisations’ employees, who work as student mentors, however, were constrained by their working hours. Because of this limitation, the time slot for the non-expert moderators was planned to be from 6:00 to 6:30. The other group time slots were as given in Table 1.

232

K. Philip and S. Mitra

Table 1. Schedule of group calls. Day of the week

Downloaded by [Newcastle University] at 06:38 06 November 2014

Monday Tuesday Wednesday Thursday Friday Saturday (fortnightly) Saturday (monthly)

Timeslot 6:00–6:30 p.m. 7:00–7:30 p.m. 6:00–6:30 p.m. 6:00–6:30 p.m. 7:00–7:30 p.m. 10:30–11:00 a.m. 10:30–11:00 a.m.

Moderator type Non-expert Expert Non-expert Non-expert Expert Expert Non-expert

2. Define a format for the call/group interaction It was decided to leave the format of the calls as open as possible. The decided format was as follows: (1) Welcome by the moderator (three to four sentences to get learners at ease). (2) House rules by the moderator: (a) Wait for a person to complete speaking before starting to speak. (b) Share only name and place of residence. DO NOT share phone numbers, addresses, age or any other personal details. (c) It is a mixed forum with different age groups, beliefs, etc. so do keep social/cultural sensitivities in mind. (d) Speak slowly and softly so that all people can understand. (3) Self-introductions by the participants (name, and place of residence only). (4) Free-flow conversation. Moderator to help enable learners to share their experiences and also share their own experiences. The objective of the moderator should be to steer all conversations towards useful information in learning. If the conversation is waning, the moderator can check out subjectspecific topics and facilitate discussion on these. At no stage should the moderator be carrying the conversation alone. The call is planned for half an hour. A shorter call is also fine. (5) Conclusion and concluding remarks by the moderator focusing on the things discussed – especially focusing on points that will encourage all people to continue with and complete their studies. 3. Create guidelines for call handling for moderators To ensure a consistent experience for all participants, it was important to create guidelines on how the call was to be handled. A document was prepared for this purpose. 4. Create a list of learners, contact them to check for participation in the pilot programme and allocate groups based on time preference Using the organisation’s student management system, student information consisting of profile (age, course start date), stage of learning (number of test papers submitted), and contact details were collected and a random set of 300 learners was identified.

Open Learning

233

The only consistent mode of contact available was through the system recorded telephone number. It was decided to allocate two people for two hours a day to contact the students over a 10-day period (40 hours total effort). The template of the message to the learners was:

Downloaded by [Newcastle University] at 06:38 06 November 2014

Greetings, hope you are doing well. Thank you for joining ICS for your course. We know that continuing with and completing a distance learning course is not easy – as you are always learning alone. To help you feel a part of the community and enable successful learning, we are planning to get learners on the same course together in a group. This will enable you to connect up with learners in the same subject, discuss problem areas, share challenges and in addition to learning yourself, also support others in their learning. Half an hour a week is a small investment to enable you to complete your course successfully.

The calling team was instructed to ask learners whether they were willing to participate. Those who opted to join were provided with options for available time slots and allocated a time slot. At the end of the 10 days: • 307 learners had been contacted, • 29 (about 10%) learners evinced interest in participation, and • learners were distributed across five time slots: (a) (b) (c) (d) (e)

Monday – six students. Tuesday – seven students. Thursday – five students. Saturday fortnightly – seven students. Saturday monthly – four students.

Except for one learner who was being home-schooled, the other learners were 22–65 years old.

The first experiment The process defined for the pilot programme is shown in Figure 2. The first part of the process was an introductory call where all learners were contacted to confirm their participation in the group calls. The second part of the process involved calling learners the day before to confirm participation. This was a very resource-intensive task as either learners would not accept their telephone calls or were not available – in which case the callers had to try repeatedly to contact the learners. This task allocation of contacting the learners by telephone was over and above the regular tasks assigned to the student services support team. The pilot programme started on 16 August. The plan was to keep the learners connected for four months (until just before Christmas), and to measure the impact of these group calls on their progress in learning. Some conversations from the calls are as follows: • Working woman learner with two primary school children from England:

Downloaded by [Newcastle University] at 06:38 06 November 2014

234

K. Philip and S. Mitra

Figure 2. First experiment programme process chart. Only time to learn is when the kids are learning. The kids are excited to see that mom is also studying. I am able to help them as well as spend some time learning on my own. Does not give much time, but helps to continue.

• Man from England working towards a career change: I am doing my computer science and animation degrees from a college near my house. My professor asked me to join a mathematics course as it helps with building my analytical skills. Whenever you are trying to learn math, always apply it to a real-life scenario. Think how you can use it in a real-life scenario and you will be able to remember the concept as well as use it more effectively. One of the biggest problems that I have faced is in booking my exam centre. My recommendation to all people on this call is – start early. For the June date, start in January at least.

• Man from England: Long time since I have been through formal education so this is challenging. My two daughters are preparing for their GCSEs so this helps me to keep in touch and also help them with their GCSEs.

• Unemployed woman from England: It is very difficult to find time to study. I try and apply my learning to my daily tasks such as when shopping for groceries etc and that helps me to remember the basic concepts.

• Unemployed woman from England: I have already completed my masters from another country. But to get a job here, I need qualifications from this country so now I am pursuing my GCSE. It is very difficult to study alone. I am able to spend 6 to 8 hours each day towards the three GCSEs that I am currently pursuing.

Open Learning

235

It is very helpful to talk to some other people who are studying. It tells you that there are others there who are also doing the same course.

• Working woman from England: After a tough day, it is very difficult to study, I try and spend as much time as I can, but you are so tired that you are unable to. I try and spend even 15 minutes if I can. Progress is slow, but I am getting there.

Downloaded by [Newcastle University] at 06:38 06 November 2014

When you complete a part, glance and keep in mind what you have to do next. That helps me to get back into learning at a very short notice to myself. Helps when you have just 15 minutes in between two tasks.

Group call data Although there were five groups planned for four months, they could not be sustained beyond seven sessions. The attendance data from the various group calls across the weeks are shown in Figure 3. Analysis of non-continuation of group calls The feedback during the calls was positive: Good experience to learn from others. It is good to know others who are learning along with you. I liked where we discussed challenges that we faced.

However, it is possible that the lack of defined learning objectives for each call and learners’ reluctance to openly share problems were the main reasons for non-participation and therefore non-continuation. Moderator skills and the frequency of the calls may also have played a part. Being a study forum, people would dial into the calls. After introductions and housekeeping instructions, the moderator would leave the floor open for discussions. It was noticed that if there were ‘talkative’ people in the call, they would initiate and lead the discussions. The discussions would start with what they did during the week and the challenges they faced in their learning. The interactions were very positive. Helpful information was exchanged between the learners that included study skills, personal study schedule, challenges faced, examination bookings and scheduling, assignment submission, and so forth. All people participated – either on their own or when prompted by the moderator. The first call in all groups was very positive but was invariably led by the most ‘talkative’ individual in the group or had to be excessively guided/prompted by the moderator. At the end of the first call, all participants found the group call a good idea and were willing to participate in the next call. In the second set of group calls, the participants had much less to share. The moderator guided the learners towards discussing the topics they were studying. As learners were at different stages of their learning, all participants could not benefit from the discussion. Many learners did not come prepared with their doubts/queries.

K. Philip and S. Mitra

Downloaded by [Newcastle University] at 06:38 06 November 2014

236

Figure 3. Group call participants over various time periods and with different moderators.

Downloaded by [Newcastle University] at 06:38 06 November 2014

Open Learning

237

This also led to inefficient use of the time during the call. The second call was felt to be less effective and participatory than the first call. In the subsequent group calls, although many learners confirmed their intention to participate during the reminder telephone calls, they did not do so. The dwindling number of participants also led to a decrease in the effectiveness of the calls. This further accelerated the drop-out rate. Interpersonal trust may also have played a role. ‘Trust’ (interpersonal trust; Renzl, Matzler, & Mader, 2000) is a key component for sharing information with others. The dwindling numbers of participants did not give the group any time to build trust amongst themselves. This also contributed to a decrease in the effectiveness of group learning and accelerated the drop-out rate. Moderator skills were also considered as a factor influencing the effectiveness of group learning. Mentors and tutors went through mock calls so that they were used to handling the conferencing equipment and made aware of the nuances of handling group calls. Moderator skills may have been a factor – but definitely not one of the leading factors in the non-continuation of the group calls. The high frequency of the calls may have also played a role in learners dropping out of the calls. During one of the reminder telephone calls, one of the learners quipped: ‘I would like to join in – but this week I have not been able to do much – so not sure what I will share on the call’. This may indicate that, where people look at distance learning as a flexible mode of study, there may be some weeks where they are unable to study or do not study due to other personal reasons. The second experiment Research into many learning theories and pedagogies has established the relationship between learner interactions and effectiveness of learning. This leaves us in no doubt that group interactions are good and required. The challenge is how to ensure that groups are created, sustained, effective and taken to completion – especially in a distance learning environment. Based on our learning from the group learning and interactions discussed earlier in this document, it was decided to start a new pilot with the following (Hamond, 2005; O’Neill, Scott, & Conboy, 2010): (1) Larger groups: considering the high drop-out rate in attendance to the group calls (> 80%), it would help to have larger groups (around 50 learners per cohort group). This would ensure that (a) there was more interaction and (b) in spite of the large drop-out rates in attendance there were at least seven to eight people attending each group call. Managing a large group would (c) also ensure less overhead on per-person support time and cost. (2) All cohorts at the same point in learning: to enable all learners to contribute to the discussions, it would help to have people in groups who are at the same point in learning. Managing these groups and maintaining all of them at the same pace of learning would be a challenge, but that challenge could be left to be addressed at a later date and time. (3) Fortnightly group calls: based on feedback from learners and tutors, it was felt that weekly group calls were very constraining and did not provide any space for learning flexibility. Fortnightly group calls would provide learners

238

K. Philip and S. Mitra

(4)

Downloaded by [Newcastle University] at 06:38 06 November 2014

(5)

(6)

(7)

with the required flexibility – to spend the recommended/required time for learning across two weeks. Defined objectives for discussion: as all learners would be starting from the same point and moving forward together, a defined set of objectives for discussion would help learners to be prepared for the group discussion. A recommended learning plan would be circulated to all learners at the start of their learning journey. This learning plan would have the group call dates and the set of objectives to be discussed in the group call. Exercises to enable discussions: to enable effective discussions, the learners would be sent a set of exercise questions and solutions. The exercises would be sent a week before the group call date. The solutions would be sent the day before the group call date. This would enable the learners to compare their solutions with the given solutions and clarify any doubts during the call. Regular email interactions to reduce on overheads of telephone calls: emails are a non-intrusive method of contacting learners (when compared with telephone calls). It was decided to draft a series of emails to be sent to learners at regular intervals. These emails, in addition to alerting the learners on forthcoming group activities, would also be used to send the additional questions and solutions to the group of learners. Use subject experts as moderators: as each group discussion focused on defined subject topics, it required the presence of subject experts as moderators.

With these factors in mind we restarted group calls from 17 September, 2010. The new group was started with 50 learners – all of whom had joined in the month of August (first week of August 2010) and therefore would be at a similar stage of learning. A learning plan was created for Unit 1 (total of five units in the course) and sent to the learners. The learning plan had topics, recommended completion dates and dates for group calls. Every fortnight, three emails were planned: • The first email (sent on day two of the fortnight – a Thursday) mentioned the objectives for the fortnight and also the day for the group calls. After the first start-up week, this email also included a summary of the group call and was sent every Thursday following the group call. • The second email (sent on day seven of the fortnight – a Wednesday) included a reminder about the next group call and a set of exercises to complete to enable discussion of their doubts/queries. • The third email (sent on day 13 of the fortnight – a Tuesday) included a reminder of the group call for the next day and also included the solution to the exercise sent a week earlier. Figure 4 shows the cycle of emails and occurrence of group call within the fortnightly cycle. The emails were sent to all learners in the group, irrespective of their attendance or non-attendance in the group call. This cycle was repeated each fortnight so that the learners came to expect emails in a pattern and could plan their learning in advance. The emails were in alignment with the learning plan that had been sent to them earlier.

Open Learning

239

Email with reminder and exercise set questions

Intro email with invite for Intro Group call

Email with reminder and exercise set solutions

Email with Summary of Group call and Invite for next group call

Downloaded by [Newcastle University] at 06:38 06 November 2014

Group call

Figure 4. Second experiment programme process chart.

Working groups Over the first three months (17 September 2010–19 December 2010), each session had about five to nine learners (Table 2 and Figure 5). The discussion that ensued was acceptable. The sharing of information was much less than expected. Most of the sharing was done by the tutor. The learners stated their queries, and the tutor answered them. The expectation was that the Table 2. Call attendance over a four-month period: number of learners on each group call for Unit 1: Group Call. Topic

Date

Introductory call

22 September 2010 6 October 2010 20 October 2010 3 November 2010 17 November 2010 05 December 2010

Addition, factors & multiplication Fractions and decimals Percentage & everyday arithmetic Speed, ratio and proportion, time, conversion, travel, graphs Test paper discussion

Number of learners

Figure 5. Number of learners on each group call for Unit 1: Group Call.

8 9 7 6 8 5

Downloaded by [Newcastle University] at 06:38 06 November 2014

240

K. Philip and S. Mitra

learners answer the questions raised by their peers and the tutor should just be a facilitator. The planned measure of success was test paper submission. GCSE Mathematics is a course that has six test papers that have to be submitted for successful completion. The test papers are distributed across the course for submission at the end of each of the units (Table 3). The data for the experimental group were compared with two other control groups – one group that started a month earlier and the other group that started one month after the experimental group (refer to Table 4). At the end of Unit 1, when compared with two of the control groups, the submission of Test Paper 1 for the experimental group was higher (Table 5). Based on these data, it seems that there is a positive change. However, these data need to be compared with overall ICS historical trends for GCSE Mathematics. The data also need to be mapped across all test papers to draw conclusions about the success of the intervention and engagement strategies. When compared (Figure 1), we realise that the submission of Test Paper 1 has traditionally been high. Over an extended period, the number of people submitting test papers decrease with only 4% of all learners on GCSE Mathematics completing all test papers. Learners were given a break for holidays during the Christmas season 2010. Before the holiday break, a survey email was sent to all learners asking them the examination date that they were working towards (June 2011 or November 2011). Although we would have wanted all learners to work towards the June 2011 examination date (they had registered for their course in September 2010 and the course is supported for one year – until September 2011), only eight students responded that they would like to sit for the June examination date. The other learners did not respond to the query. (One of the reasons attributed to the lack of response from the learners was that if they mentioned a date, it may be taken as a commitment from them which they might not be able to keep.) To cater to their requirements, it was planned to split the group of 50 into two: • Group 1 (with eight learners; also called G8 or Group of 8) working towards the June 2011 examination date. • Group 2 (with 42 learners; also called G42 or Group of 42) working towards the November examination date. The group call sessions for Unit 2 onwards were planned to restart on 5 January 2011. A decision was taken to keep the learning group together until mid-February Table 3. Course units and test papers. Unit Unit Unit Unit Unit Unit

Test Paper 1 2 3 4 5

Unit 6

Test Paper 1 Test Paper 2 Test Paper 3 Test Paper 4 Test Paper 5 Test Paper 6 No test paper

Open Learning

241

Table 4. Description of experimental and control groups. Group

Registration timeframe

Control Group 1 Experimental group Control group 2

Last week of August 2010 First week of September 2010 Second week of September 2010

Number of learners 50 50 50

Table 5. Submissions of Test Paper 1. Control Group 1

Downloaded by [Newcastle University] at 06:38 06 November 2014

12 (24%)

Experimental group

Control Group 2

15 (30%)

12 (24%)

and then split them into two groups. This decision was also communicated to the learners. A new learning plan for Units 2–6 was created and sent to them. The learning plan focused on completing all six units by June. Lack of automation The reason for running the learning group in common until mid-February was primarily influenced by the lack of automation of systems. The services team handling the sending of emails was understaffed and therefore could not send two parallel sets of emails, exercises and solutions regularly. This additional time would also enable the setting up of a virtual learning environment (VLE) and provide additional resources including scheduled emails for the learners. The learning groups started as scheduled on 5 January 2011. The learning calls started as scheduled – from 12 January 2011. Attendance in the two calls in January (12 and 26 January 2011) and the one call in February (9 February 2011), before the split, was normal (refer to Figures 6 and 7 for details). Learners for the November examination date were sent an email with a new learning plan focused on com-

Figure 6. Group attendance in Group 1 (eight learners) until the end of March (group calls discontinued after March).

242

K. Philip and S. Mitra

Downloaded by [Newcastle University] at 06:38 06 November 2014

Figure 7. Group attendance in Group 2 (42 learners), week of 1 May.

pletion of all activities by the last week of October 2011 – ready for the examination in the first week of November 2011.

Setting-up (loading and configuring resources) on the VLE To ensure that we could provide value-added support to learners and maintain uninterrupted communication with them, Moodle was selected as the VLE. The following learning resources were provided for the learners: • • • • • • •

Learning plan – topics structured for each fortnight. Introduction/rationale for learning for each set of topics for the fortnight. Topic-wise quiz questions for each week of study (two sets for the fortnight). Consolidated quiz/recap questions at end of month (for topics for the month). External resource links to videos on specific topics. Exercise set and solution files for group discussions. Forums for learning group discussions/communications.

A tutor was also assigned to the group to keep track of their progress. Login accounts were created for the learners and introductory emails with login details and help files were sent to the learners. G8 move to the VLE The G8 learners were the first to move to the VLE. The eight learners who had opted for the June date attended the group calls very infrequently during Unit 1. They never attended the group calls for Unit 2. With the split into a smaller group of eight, these learners did not attend any of the new group calls. For this reason, a decision was taken to provide only online VLE-based support to these learners. They would have access to all the additional resources put on the site. Although they were kept constantly updated regarding the availability of exercises and other resources on the VLE, none of these learners logged into the system to access the resources. G42 for the November examination date meanwhile continued with group calls. There was a steady drop in attendance. With attendance at an all time low of only one person attending the group call, it was decided to move the group of 42 also onto the VLE.

Open Learning

243

Downloaded by [Newcastle University] at 06:38 06 November 2014

Impact of group calls As mentioned earlier, the impact of the group calls on learning and engagement are measured by the number of test papers submitted. Based on the comparison data between the experimental group and the two control groups starting with Unit 2 learning, and as of End April 2011, the data are as given in Figures 8–10.

Figure 8. Number of test paper submissions by Control Group 1.

Figure 9. Number of test paper submissions by the experimental group.

Figure 10. Number of test paper submissions by Control Group 2.

244

K. Philip and S. Mitra

Downloaded by [Newcastle University] at 06:38 06 November 2014

Control Group 1 submitted more test papers than the experimental group. However, we need to note that individual performances of two learners from the Control Group 1 have affected these numbers. These learners have submitted all test papers and completed their course much in advance of the stipulated time. This behaviour is typical of learners who are undertaking this course with a focused objective and are self-driven. The experimental group and Control Group 2 did not have learners whose behaviour was comparable with the two exceptional learners in Control Group 1. If we remove these two learners from Group 1, the test paper submissions become equal to those of the experimental group. When compared with Control Group 2, the experimental group performed better by submitting more test papers. Scaffolding and the VLE In addition to a well-structured content, learning resources and support were provided to the learners. The VLE enabled course managers to be in constant contact with learners. A learning plan provided the learner with topics structured for each fortnight enabling them to plan their learning. This was designed so that it did not require more than five hours a week – or less than an hour a day. An introduction/rationale for learning for each set of topics put the learning in context and provided the correct starting point for learning. The external learning resources provided additional information to support learning. A set of recap/quiz questions enabled learners to check their progress and review their learning and then focus on topics that needed more work. The discussion forum enabled learners to stay in touch with their cohort of learners as well as get their queries answered quickly. Despite all resources being made available and the benefits of the resources and the methods expounded to the learners, the uptake of online resources was very low. Out of a total of 50 learners in the control groups who had access to the VLE, there were only seven users of the VLE – who also cannot be classified as ‘regular’ – who came online to use the quiz and practice questions. Course revenues At ICS, a gross enrolment value (GEV) is associated with each course. This is a cumulative total of the value of registrations/enrolments (i.e. expected income) over a time period (GEV = cost of coursenumber of student registered). Students have the option of paying the course fees in full or in instalments. In addition to the GEV, a revenue realisation figure is calculated from the total receipts (full payments or instalments) against the same period. These figures enable the company to control outstanding payments and also to compare the relative efficiency of payments for different courses. Figure 11 shows the GEV and revenue realisation for the experimental and two control groups described in this paper. We notice that there is a 6% increase in the revenue realisation from the experimental group over the others. When taken over the entire cohort of GCSE Mathematics admissions (not just the 50 learners per group, as we have done here) this would represent a significant reduction in payments outstanding. This is an important factor for shareholders of the company.

Downloaded by [Newcastle University] at 06:38 06 November 2014

Open Learning

245

Figure 11. General economic value (GEV) and revenues realisation.

Discussion We summarise our findings as follows: (1) Past data (Figure 1) show that while 30% of the distance learners of GCSE Mathematics at ICS submit their first test papers, only 4% go on to complete the course. (2) In experiment one, of 307 randomly selected learners, less than 10% showed interest in group calls. (3) Of those who did show interest, very few attended the calls regularly and their numbers tapered to zero in a few weeks. However, their comments about the usefulness of group calls were uniformly positive. (4) In experiment two, about 10% of the experimental group participated at all in group calls. (5) The larger group size did not result in any improvements in group call attendance. (6) There were no significant differences between the experimental and control groups in test paper submissions. (7) About 15% of the experimental group are active users of the VLE. (8) Revenue realisation is 6% higher with the experimental group. Conclusion The GCSE Mathematics course from ICS is one of the company’s most popular products. The results from our experiments are intriguing in that they seem to indicate that scaffolding in the form of structured material, collaborative environments, and a VLE does not seem to affect performance in any way other than an improvement in revenue realisation. We consider this negative result quite important.

246

K. Philip and S. Mitra

There are several studies of the characteristics of distance learners (Hara & Kling, 1999; Dabbagh, 2007). However, most of the studies do not mention certain characteristics that our experimental group of learners have mentioned. These are as follows:

Downloaded by [Newcastle University] at 06:38 06 November 2014

(1) They are distance learners because they find it difficult to find time to study. (2) They are distance learners because they find it hard to follow a timetable. (3) They have not taken a distance learning option to ‘prepare’ for an examination, but to ‘practice’, independently and as they wish. (4) They have registered for a distance learning course in order to help someone else prepare for an examination. Learners with the above characteristics would find timetabled scaffolding or other support to only increase their difficulties, while they realise and appreciate that the intention is for their benefit. ‘It’s a good idea, but I can’t join because of my situation’ seems to sum up what our learners are saying. We conclude that the GCSE Mathematics course at ICS would be a highly effective and profitable course if it was positioned as a course that provides plenty of self paced practice with little or no scaffolding, except for revenue realisation purposes. It is important to mention that the results of this study apply only to distance learners of GCSE Mathematics. We do not suggest or imply that the conclusions be generalised to distance learners as a whole. Acknowledgements The authors are grateful to Gerry Black, Joanne McManus and Sally Pulvertaft for providing advice and guidance during various stages of the work. Administrative support from Fiona McCusker and Christine Younger was vital for the completion of this work. Financial assistance from Knowledge Transfer Partnerships, Technology Strategy Board, UK, is gratefully acknowledged.

References Blumenfeld, P., Marx, R., Soloway, E., & Krajcik, J. (1996). Learning with peers: From small group cooperation to collaborative communities. Educational Researcher, 25(8), 37–40. Retrieved November 7, 2009, from http://www.jstor.org/pss/1176492. Carr, S. (2000). As distance education comes of age, the challenge is keeping the students. The Chronicle of Higher Education, 23, A39. Retrieved September 30, 2011, from http://chronicle.com/free/v46/i23/23a00101.htm. Crook, C. (1998). Children as computer users: the case of collaborative learning. Computers and Education, 30, 237–247. Dabbagh, N. (2007). The online learner: Characteristics and pedagogical implications. Contemporary Issues in Technology and Teacher Education, 7(3). Retrieved January 18, 2010, from http://www.citejournal.org/vol7/iss3/general/article1.cfm. Dangwal, R. (2005). Public computing, computer literacy and educational outcomes: Children and computers in rural India. In C-K. Looi, D. Jonassen, & M. Ikeda (Eds.), Towards sustainable and scalable educational innovations informed by the learning sciences (pp. 59–66). Amsterdam: IOS Press. Dangwal, R., & Kapur, P. (2008). Children’s learning processes using unsupervised ‘hole in the wall’ computers in shared public spaces. Australasian Journal of Educational Technology, 24(3), 339–354. Galusha, J. (1997), Barriers to learning in distance education. The Infrastruction Network. Retrieved October 14, 2010, from http://www.infrastruction.com/barriers.htm.

Downloaded by [Newcastle University] at 06:38 06 November 2014

Open Learning

247

Hamond, M. (2005). A review of recent papers on online discussion in teaching and learning in higher education. Journal of Asynchronous Learning Networks, 9(3), 9–23. Hara, N. & Kling, R. (1999). Students’ frustrations with a web-based distance education course. Retrieved January 25, 2011, from http://firstmonday.org/article/view/710/620 Inamdar, P. (2006). Computer skills development by children using hole in the wall computers in rural India. Australasian Journal of Educational Technology, 20(3), 337–350. Jonesa, A. & Issroff, K. (2005). Learning technologies: Affective and social issues in computer-supported collaborative learning. Computers & Education, 44(4), 395–408. Retrieved January 18, 2011, from http://www.sciencedirect.com/science/article/pii/S03601 31504000570. Mitra, S., Dangwal, R., Chatterjee, S., Jha, S., Bisht, R.S., & Kapur, P. (2005). Acquisition of computer literacy on shared public computers: Children and the ‘Hole in the wall’. Australasian Journal of Educational Technology, 21(3), 407–426. Mitra, S., Dangwal, R., & Thadani, L. (2008). Effects of remoteness on the quality of education: a case study from North Indian schools. Australasian Journal of Educational Technology, 24(2), 168–180. Mitra, S., & Rana, V. (2001). Children and the Internet: experiments with minimally invasive education in India. The British Journal of Educational Technology, 32(2), 221–232. Nicaud, J.F., Bittar, M., Chaachoua, H., Inamdar, P., & Maffei, L. (2004). Experiments of Aplusix in four countries. International Journal for Technology in Mathematics Education, 13(2), 79–88. O’Neill, S., Scott, M., & Conboy, K. (2010). A Delphi study on collaborative learning in distance education: The faculty perspective. British Journal of Educational Technology. Retrieved December 1, 2011, from http://dx.doi.org/10.1111/j.1467-8535.2010.01132.x. Renzl, B., Matzler, K., & Mader, C. (2005). Impact of trust in colleagues and management on knowledge sharing within and across work groups. Paper presented at the the 6th European Conference on Organizational Knowledge, Learning, and Capabilities. Bentley College, Boston. Ed. S. Newell, 4–10. Retrieved January 17, 2011, from http://www2. warwick.ac.uk/fac/soc/wbs/conf/olkc/archive/oklc6/papers/renzl_matzler__mader.pdf. Stahl, G., Koschmann, T., & Suthers, D. (2006). Computer-supported collaborative learning: An historical perspective. In R.K. Sawyer (Ed.), Cambridge handbook of the learning sciences (pp. 409–426). Cambridge. Retrieved April 12, 2011, from http://GerryStahl.net/ cscl/CSCL_English.pdf.