eLearning benefits - Aug09:Layout 1
7/8/09
12:53
Page 1
A model to measure the tangible benefits of eLearning Megan Quentin-Baxter (Higher Education Academy Subject Centre for Medicine, Dentistry and Veterinary Medicine), Jacquie Kelly (JISC infoNet)Stephen Probert (Higher Education Academy Subject Centre for Business, Management, Accountancy and Finance), Cary Macmahon (JISC TechDis) and G Ferrell (JISC infoNet)
BACKGROUND A model for describing and collecting evidence with which to evaluate technology-enhanced learning was developed as part of the Tangible Benefits of e-Learning project which took place in the UK in 2007. The Higher Education Academy Subject Centres, JISC services, the Association for Learning and Teaching and CETIS collaborated with higher education institutions at the invitation of the JISC and the Higher Education Funding Council for England.
SUMMARY OF WORK This small study sampled technology-enhanced pedagogic innovation (elearning) to support learning in business, health and the humanities in the UK, and documented the results as 36 case studies from which a model illustrating the ‘tangible benefits’ was developed. Case studies were ‘typed’ by whether they were automating existing processes, supported administration, lead learning and teaching or experimented with new pedagogies, and by whether information flowed upwards in organisations, or down. The model illustrated how elearning innovation relied primarily on qualitative evidence, while evidence of the benefits of process-automation can be quantified. The model may help institutions to choose the most appropriate type of evaluation strategy when elearning innovations are being tested.
SUMMARY OF RESULTS The 36 case studies were placed on the model according to their transformative effect and benefits profile (Figure 1), providing a map of the type of evidence (web logs, student feedback, etc.) which could be collected in different areas to demonstrate ‘tangible benefits’. Figure 1. Tangible benefits of elearning model illustrating the position of 36 case studies in terms of evidence of benefits
CONCLUSIONS The model illustrated how technology enhanced pedagogic innovation relied primarily on qualitative evidence (which takes at least 3 years to accumulate), while evidence of the benefits of process-automation could be quantified, often in terms of what the innovation cost, and how much it had saved. Limitations included a 6 month time-frame to complete the project, logistical challenges and the difficulties of preparing a single, short report which adequately reflected the complexities manifest in the case studies.
TAKE-HOME MESSAGE The model may help institutions to choose the most appropriate and efficient type of evaluation strategy when elearning innovations are being tested.
KEYWORDS Research methodologies
Computer-based assessment Theories of learning
Europe
Experiential Games learning Simulation
Portfolios Collaborative/peer-to-peer E-learning/computers
Virtual patients Professionalism Education Feedback Teamwork Communication
Learning skills
REFERENCES Clark, R.E. (1994). Media will never influence learning. Educational Technology Research and Development, 42, 21-29. Economic and Social Research Council (ESRC) (2006). Technology enhanced learning, the teaching and learning research programme. ESRC, London. www.tlrp.org/tel/index.html [viewed 18 July 2008]. Higher Education Funding Council (HEFCE) (2005). HEFCE strategy for eLearning. www.hefce.ac.uk/pubs/hefce/2005/05_12/ [viewed 18 July 2008].
The model illustrated how elearning relied primarily on qualitative evidence, while evidence of the benefits of process-automation could be quantified. Evidence of a return on investment (ROI) and value for money (VFM) can be more clearly demonstrated towards the left of Figure 1, with the creative research and development (R+D) zone towards the right. In the R+D zone evidence for the efficacy of an innovation relied on a teacher’s professional instincts and student satisfaction, and may appear to cost more to provide than can be justified in financial terms. Contributors estimated that the break-even time for innovative R+D activities was at least three years. There were no examples of an automated process appearing in the R+D zone, implying that the drivers for automation were rational rather than innovative. Learners’ understanding of a particular subject fell towards the centre of the graph where the tangible benefits were measured in terms of course or module pass rates, as well as qualitative measures of achievement. E-assessment was the only process demonstrating some consistency across subjects, but even that was broadly focussed.
Contact lead author at: MEDEV, School of Medical Sciences Education Development, Faculty of Medical Sciences, Newcastle University, Newcastle upon Tyne NE24HH T. +44 191 2225888 F. +44 191 2225016
[email protected]
JISC infoNet (2006). The CAMEL Project: Collaborative approaches to the management of e-Learning. Northumbria University. www.jiscinfonet.ac.uk/camel [viewed 18 July 2008]. JISC infoNet (2008). Exploring tangible benefits of e-Learning: Does investment yield interest? Northumbria University, 38p, ISBN: 978-1-86135349-8. www.jiscinfonet.ac.uk/case-studies/tangible/ [viewed 18 July 2008]. Laurillard, D. (2002). Rethinking university teaching: A conversational framework for the effective use of learning technologies. 2nd Edition, RoutledgeFalmer, New York, 268p. ISBN 0-415-25679-8. Schein, E.H. (1989). The role of the CEO in the management of change: The case of information technology. In T.A. Kochan & M. Useem (Eds) Transforming organizations (pp. 80-95). Oxford University Press. books.google.com/ [viewed 18 July 2008]. Zuboff, S. (1988). In the age of the smart machine: The future of work and power. New York: Basic books, 496p, ISBN 978-0-46503-211-2.
