DESIGNING LEARNING EXPERIENCES WITH MULTIMEDIA

DESIGNING LEARNING EXPERIENCES WITH MULTIMEDIA LEARNING OBJECTS: TOWARDS A HUMANCENTRED APPROACH Sandra Cairncross and Tom McEwan, HCI Research Group, School of Computing Napier University {s.cairncross,t.mcewan}@napier.ac.uk

ABSTRACT This paper explores the design of learning experiences that use multimedia learning objects (MLOs). A history of learning objects is presented and design issues examined. The need for a learner-centred approach to design is identified and a design and evaluation framework is presented. Issues associated with its use are explored. Keywords Multimedia; learning objects; human-centred design; ISO13407; evaluation

1. INTRODUCTION Patterns of teaching and learning are changing, particularly in the post-school arena. There is a growing need to offer greater flexibility in “product delivery” to students. This has led to many universities and colleges adopting a blended approach to supporting teaching and learning, combining taught classes with independent study resources available on-line, and providing support through emails and discussion fora. This has led to consideration of the requirements for the resources that learners can use (and re-use) in the construction of knowledge. These resources are required to support learning both within and outside the classroom. Reusable learning objects have been attracting attention as one means of meeting this demand, providing resources that can be shared and re-used by learners in a variety of contexts. For example, a set of PowerPoint™ slides on a key topic can be used by lecturers in different institutions or as a handout for further study. Animations could be used to illustrate processes. In both cases, the learning outcomes, and the means of assessing whether they have been achieved, appear to be intrinsic to the definition of a learning object. One measure of evaluating the learning object’s effectiveness would therefore be whether learners achieve the learning outcomes in the various contexts of use. This in turn is clearly related to the process of ensuring that all learning experiences result in the desired learning outcomes. In this paper we reflect on the specific case of multimedia learning objects (MLOs), which we will define here as self contained multimedia applications that can be used by learner to help achieve one or more specific learning outcomes. We explore how best to use human-centred evaluation to increase the chance of successful integration

of MLOs into the wider learning experience. We describe one such approach that places the learner and their needs at the centre of the design process. The centrality of evaluation throughout the lifecycle is highlighted and a design and evaluation framework is proposed to assure this. We conclude by exploring practical issues associated with using the framework

1.1 A quick history of learning objects The provenance and definitions of learning objects is explored in more detail elsewhere [16]. To summarise briefly: Wiley suggests that Hodgins, chair of the IEEE LTSC (Learning Technology Standards Committee) was first to use the term, in 1994 [19]. Others active in the field of educational technology use related ideas [2,6,13], such as combining learning content with learning process, but neither use Hodgins’ term at this stage. Yet by 1999 “multimedia learning objects” are cited as a typical example of a reusable interactive resource [10]. In 2002 the LTSC propose standards for learning objects [11], while metadata standards continue to evolve for all digital media. This has coincided with educators and learning technologists setting up domain-specific workshops (such as previous British HCI Group Educators’ Workshops) and special interest groups to share reusable materials and approaches. Researchers have started to explore some issues associated with the reuse of such resources [1]. However, care must be taken with both the design of MLOs themselves and their integration into the wider curriculum to ensure that they are relevant to the learners’ needs. A human-centred approach is thus recommended, which, in this context, implies consideration of learning needs and human-computer interaction principles.

2. HUMAN-CENTRED DESIGN WITH, AND FOR, MLOS Our research has found that context of use is an important consideration, as to whether learners can learn successfully using multimedia applications or indeed even use them at all [4,5]. It is not enough to focus solely on the design of the application: integration into the curriculum must carefully be considered. Failure to do so may result in it not being used regardless of how well it is designed. Therefore any lecturers wishing to use MLOs to support some of their teaching should consider carefully how it is going to be used and what support they will make available to learners.

In carrying out the above research a series of MLOs were developed. In so-doing, a User-Centred Design approach was adopted: Norman points out that for any design to be successful in terms of developing usable and understandable products then that design must be based on the needs and interests of the users and be informed by an understanding of their limitations and capabilities [17]. MLO design should therefore be informed by learning considerations as well as HCI. Issues here include media selection and presentation, access to information and support for learning activities. A process model based on an iterative waterfall model with pedagogical concerns being embedded at each stage was developed [4]. It was used to guide development in different domains and its usefulness reflected upon. The model was revised in light of experience resulting in the design and evaluation framework shown in Figure 1 [16].

Requirements Gathering and Specification

Analyse Learning Requirements Define Learning Outcomes Define context Define Content

includes not just what is to be learned (content) but the results of that learning (outcomes) and who will be using the application and where (context).

3. EVALUATION AND LEARNING DESIGN Evaluation should, and must, extend beyond the evaluation of actual MLOs, important though this is. Evaluation should take place at every stage of the lifecycle, including before, during and after the design of the MLOs themselves. As many viewpoints as possible should be considered, including teachers and lecturers as well as students. Other practitioners also urge evaluation throughout the process, in one example adopting a more blended-learning approach as a result of evaluation [8].

3.1 Evaluating learners The starting point should be the learners themselves: the Information Display Information Access User Activities Application Design

Focus: Who ? What? Why ? Where? Focus: Presentation, Access and Engagement HCI Theory

Evaluation Learning Theory

Focus:Usability and Learning Effectiveness Focus: Method of Use and Fitness for Purpose

Curriculum Integration

Development Release Adoption

Build (Prototype and Implement) Functionality Testing

Figure 1 Design and Evaluation Framework for Multimedia Learning Objects

The design and evaluation framework proposes an iterative approach to development and deployment. It recognises that there are different cycles of iteration, between stages and within stages. For example, problems encountered during the development phase may be resolved within that phase or require changes to be made to the original specification. The framework locates the application in the setting in which will be used, stating that learning is the focus of the initial requirements gathering and specification phase. This

lecturer considering the interactive multimedia to support the teaching and learning of their students is asked firstly to consider general characteristics of both the learners and the topic area. The lecturer can then consider areas where multimedia can add value in this particular context. The lecturer’s expertise here is valuable. They will have experience of the patterns of difficulties faced by different learners, and of ways to circumvent these. Within their communities of practice they will have joined debates about changing nature of learners’ previous experiences, for example, the gradual reduction of mathematical expertise in computing students. In some cases this may

result in defining explicit pre-requisites for a programme of study, but our experience is that many assumptions are made which are only slowly confounded with each year’s experience.

developed. In either case the MLO itself should be evaluated. There will be different aspects to this. A detailed evaluation of the MLO involves evaluating both its usability (how easy it is to use) and its learning effectiveness.

3.2 Evaluating the context of use In recent years there has been a growth in enthusiasm for approaches that stimulate “deep learning”. Intrinsic motivation, which is one characteristic of deep learning situations, can be the result of inspiring situations [7]. Conversely poor teaching skills, problems with facilities, cancelled lectures, lack of support for impairments can all result in demotivation, leading to shallow learning and a failure to internalise the learning outcomes. Compelling multimedia content is expensive to produce, and like Hollywood movies, there is no guaranteed formula for success. For every low-cost success there are expensive failures, but even the best movie will not engage you if you are unable to concentrate on it due to, say, distractions. Therefore understanding the context in which a MLO will be used becomes significant; particularly if it is to be used to best effect. Areas of difficulty for learners, where the use of MLOs may help, can be pinpointed through examining the curriculum and previous experiences of students. Interviews with previous students can be helpful in identifying problematic areas and misconceptions. An overview of how multimedia can be used to support key aspects of learning and teaching has been presented previously [3]. In particular it was noted that the key features of multiple media, user control over the delivery of information and interactivity might help learners come to a deeper understanding through supporting conceptualisation and contextualisation of the new material being presented; actively involving the learner in the learning process; and promoting internal reflection. Interactive multimedia learning applications can also be helpful to learners with specific needs. For example, some learners may not be able to attend classroom sessions and would benefit from the virtual equivalent. These could also be of use to those learning in a second language, as well as learners with physical impairments. The use of audio in conjunction with, or as an alternative to, text-based exposition could benefit students with dyslexia. Conversely, the needs of students with hearing difficulties should not be forgotten and screen-based alternatives to audio should be provided.

3.3 Evaluating the curriculum In identifying a topic to be delivered using interactive multimedia the external rate of change of the domain should be considered. Any areas likely to require frequent and extensive updating should have their suitability reviewed, as it may not prove feasible to maintain such an application. If a search of available MLOs reveals that no suitable objects exist then a new one should be designed and

3.4 Evaluating the educational effectiveness Educational software should encourage the learner to think, and to take an active role in the learning process [14]. For them, a usable educational system should build upon the learner’s existing knowledge and support the learner in carrying out tasks. However poorly designed interfaces can mean that valuable learning time is lost, through having to spend time learning how to use the application [15]. A balance has to be struck between designing applications that are easy to learn how to use, whilst ensuring that learners can learn from them.

3.5 Curriculum Integration Integration into the curriculum can then be explored. Consideration should be given to how the application will be used, who will be using it, when, where and for what reasons. In addition the level of support likely to be required should also be considered at this stage. The choices should be informed by the preferences and needs of learners. This is in keeping with a systems approach to educational design [9]. There are different options here. MLOs can be used for independent study, freeing up valuable, limited classroom time to discuss topics that the students find difficult. It can be used in the classroom: some MLOs may lend themselves to being used within a lecture to illustrate key points, others in a tutorial with learners working on their own or in small groups, with the lecturer on hand in the event of any questions. This can then be followed by selfstudy, both to explore the topic in further depth or as a revision aid. It should be recognised that for many students, learning with multimedia will be new and as such require new skills to make most effective use of it. It may be helpful to accompany the introduction of MLOs with workshops to develop time management skills. This may be difficult (and indeed counter-productive) to achieve on an individual module basis but should certainly be addressed at a programme level in order to take a more strategic approach to the introduction of new learning technologies. Evaluation should not end with the introduction of the interactive application: its effectiveness should be monitored as an ongoing process. If problems emerge during the module the lecturer may need to adapt the planned method of use, or to supplement the learning application with other means, for example classroom sessions or printed material. However it is not enough to make adaptations solely in light of the experience of those who have already used the

application. Social situations cannot be revisited or repeated; each group of learners and each learning situation is unique [18]. A ‘rethink’ is necessary for each new group of learners. The framework should thus be revisited each time it is proposed to use an application.

4. CONCLUSIONS This framework is based on a reflection on the experiences of developing and teaching with MLOs. The design and evaluation framework is learner-centred, encompassing both the technical aspects of interface design and the social aspects of integration into the curriculum. It is underpinned by consideration of HCI principles and pedagogical theory. The importance of contextual factors is emphasised through highlighting the need for on-going evaluation following release and integration. Feedback so obtained can be used to inform changes to the requirements and design when it is being revised or up-dated. Even if no further versions are to be developed the design team should still seek feedback, reflect on that feedback and the actual process of design, and apply the results to the development of other MLOs. Reflective practice should be integral to the work of any educational designer and insights gained can be used to inform future practice. Done collectively it can promote sharing of information. This is essential if design teams are to avoid repeating mistakes and to build on successes.

5. REFERENCES [1] Boyle, T., & Cook, J. (2003) Learning Objects, Pedagogy and Reuse in Seale, J.K., ed. Learning technology in transition: from individual enthusiasm to institutional implementation. Lisse, Switzerland: Swets & Zeitlinger pp31-44 [2] Boyle, T., 1997. Design for multimedia learning. London: Prentice Hall. [3] Cairncross, S., & Mannion, M., 2001. Interactive Multimedia and Learning: Realising the Benefits Innovations in Education and Teaching International, 38 (2), 56-164. [4] Cairncross, S., & Mannion, M., 1999. How Multimedia Functions in Engineering Education. IEE Engineering Science & Education Journal, 8 (3), 100-106. [5] Cairncross, S., & Mannion, M., 2000. Multimedia and Learning: Patterns of Interaction, In: McDonald, S., Waern, Y., and Cockton, G., eds. People and Computers XIV – Usability or Else! Proceedings of HCI 2000.London, UK: Springer-Verlag, 375-388 [6] Davies, P., & Brailsford, T., 1994. Guidelines for multimedia courseware developers in higher education

Volume 1: delivery, production and provision. University of Nottingham; UCoSDA. [7] Duff, A., 2004. The revised approaches to studying inventory (RASI) and its use in management. Active Learning, 5 (1,) pp 56-72. [8] Eklund, J., Kay, M., & Lynch, H.M., 2003. e-learning: emerging issues and key trends. Commissioned paper by The Australian Flexible Learning Framework Policy and Research Program to inform the Flexible Learning Advisory Group’s planning for 2004 and beyond. [online]. Available from: http://www.flexiblelearning.net.au/research/2003/elearning2 50903final.doc [30 Nov 2003] [9] Ellington, H., Percival, F., & Race, P., 1993. Handbook of educational technology 3rd edn. London: Kogan Page. [10] Guenther, R., 1999. List of resource types (Dublin Core working draft). Available online at http://es.dublincore.org/documents/1999/08/05/resourcetypelist/index.shtml [3 Jan 2004] [11] IEEE 1484.12.1, 2002. Standard for Learning Object Metadata. [online]. Available from: http://ltsc.ieee.org/wg12/par1484-12-1.html [30 Nov 2003] [12] ISO 13407, 1999. Human-centred design processes for interactive systems. International Organization for Standardization. [13] Laurillard, D., 1993. Rethinking university teaching: a framework for the effective use of educational technology. London: Routledge. [14] Mayes, J.T. and Fowler, C.J.H., 1999. Learning Technologies and Usability: a Framework for Understanding Courseware. Interacting with Computers, 11, 485-497. [15] Mayes, J.T., Kibby, M.R., & Anderson, A., 1990. Signposts for conceptual orientation: some requirements for learning from hypertext. In: R. McAleese & A. Green eds. Hypertext: state of the art. Oxford: Intellect, 121-129. [16] McEwan, T., & Cairncross, S., & (in press) Evaluation and Multimedia Learning Objects: Towards a Human-Centred Approach, International Journal of Interactive Technology and Smart Education [17] Norman, D. 1988. The psychology of everyday things. New York, NY: Basic Books. [18] Prosser, M., & Trigwell, K., 1999. Understanding learning & teaching: the experience in higher education. Buckingham: Open University Press. [19] Wiley, D. A. (2000). Connecting learning objects to instructional design theory: A definition, a metaphor, and a taxonomy. In D. A. Wiley (Ed.), The Instructional Use of Learning Objects: Online Version. Retrieved November 30th 2003 from the World Wide Web: http://reusability.org/read/chapters/wiley.doc