Int. J. Learning Technology, Vol. 6, No. 3, 2011
223
Using an affective multimedia learning framework for distance learning to motivate the learner effectively Makis Leontidis*, Constantin Halatsis and Maria Grigoriadou Department of Informatics and Telecommunications, University of Athens, Panepistimiopolis, GR-15784 Athens, Greece E-mail:
[email protected] E-mail:
[email protected] E-mail:
[email protected] *Corresponding author Abstract: The aim of this paper is to present the affective multimedia learning framework of the MENTOR as well as the web and multimedia technologies which were used for its implementation. The MENTOR is an affective web-based adaptive educational system for distance learning. The basic concern of MENTOR is to retain the student’s emotional state positive during the learning process. To achieve this, MENTOR incorporates an affective module which enhances the traditional learning practices with an affective multimedia dimension. The foremost and endmost goal of MENTOR is to provide the learner with a more personalised and friendly multimedia environment for learning, according to his personality, mood and emotions. Keywords: affective computing; web-based adaptive educational systems; WBAES; multimedia learning; distance learning; artificial intelligence. Reference to this paper should be made as follows: Leontidis, M., Halatsis, C. and Grigoriadou, M. (2011) ‘Using an affective multimedia learning framework for distance learning to motivate the learner effectively’, Int. J. Learning Technology, Vol. 6, No. 3, pp.223–250. Biographical notes: Makis Leontidis is currently a Research Scientist in the Laboratory of Educational and Language Technology in the Department of Informatics and Telecommunications, National and Kapodistrian University of Athens. He received his BSc and PhD in Computer Science from the Department of Informatics and Telecommunications, National and Kapodistrian University of Athens, and his MSc in Studies in Education from H.O.U. His research interests are in the area of affective computing, artificial intelligence in education, web adaptation and personalisation environments and emotional processes, implemented on web educational systems. He has published articles in numerous international journals and refereed conference proceedings. Constantin Halatsis received his BSc in Physics in 1964 and his MSc in Electronics in 1966, both from the University of Athens. In 1971, he received his PhD in Computer Science from the University of Manchester, UK. Since 1981 he has been a Full Professor at the Department of Informatics and Telecommunications, University of Athens. He has been visiting at CERN and the University of Cyprus. His teaching and research interests cover a wide area, including artificial intelligence, computer architecture, cryptography and e-government. He is the author of more than 120 technical papers published in refereed international scientific journals and conference proceedings. Copyright © 2011 Inderscience Enterprises Ltd.
224
M. Leontidis et al. Maria Grigoriadou received her BA in Physics from the University of Athens, Greece, in 1968, and her DEA and doctorate degrees from the University of Paris VII, Paris, France, in 1972 and 1975, respectively. She is now a Professor in Education and Language Technology, and the Head of the Education and Language Technology Group of the Department of Informatics and Telecommunications, University of Athens. She has many publications in international journals and proceedings of international conferences as well as more than 600 citations to her research work. Her current research interests concern the areas of adaptive learning environments, web-based education, ITS, educational software and computer science education.
1
Introduction
A new field that is located in the scientific area in the intersection of artificial intelligence (AI), cognitive psychology and physiology, has come to surface with the promise to cover this deficiency and offers a wide range of methods, techniques and applications which take into account affectivity. This field is called affective computing and owes its name to Rosalind Picard who studied and developed in her book affective computing (Picard, 1997) methods and techniques related to the computer’s capability to recognise, model, respond, and express emotions in order to interact effectively with users. These features which are basic components of human emotional intelligence (Goleman, 1995), remain today major concerns of the designers of affective machines (D’Mello et al., 2007). On the other hand, the internet is the ideal environment for the promotion of the personalised learning according to the student needs. Various educational systems such as the adaptive educational systems (AES) and lately the web-based educational systems (WBES) have been developed to this direction. These systems allow the identification of students’ learning needs, support the selection of the suitable learning strategies as well as the appropriate presentation of the instructive material. At the same time, the system monitors the progress of the student, supports properly his efforts when it is necessary and evaluates him. In addition to the latest advances of the internet technology, the rapid development of the multimedia technology enables instructional designers to create powerful web-based multimedia-integrated educational systems (Merchant et al., 2001). Integrating in their systems a wide range of media, such as images, graphics, sound, animation and video, they provide a multi-sensory learning environment that can support learners to accomplish their learning goals with a more intriguing and fascinating way. The use of multimedia elements enriches the educational material and maximises the learners’ ability to comprehend and retain more efficiently the learning information as well as promotes a more active and exploratory way of learning (Syed, 2001). A significant advantage of the multimedia technology is their intrinsic motivational ability that can be used in order to prompt and engage the learner into the learning process. According to Lowyck et al. (2004), challenge, control, fantasy and curiosity are the key factors which make a multimedia learning environment motivating. These features form a positive learner’s attitude towards the educational system and increase the levels of the perceived satisfaction. Moreover, various studies have implied that the motivational character of the multimedia educational systems is related directly to the
Using an affective multimedia learning framework for distance learning
225
creation of positive emotions. Mayer and Moreno (2003) observed that the aesthetic design of multimedia environments is able to induce emotions which affect learner’s cognitive processes and performance. Wolfson and Case (2000) found that the appropriate design and integration of multimedia components in the learning systems can foster an optimistic disposition in the learner and produce positive emotions such as joy and satisfaction. According to the above perspective, the modelling emotional tutoring (MENTOR) which is a web-based adaptive educational system (WBAES), uses an affective module in order to recognise the affective state of the student during his interaction with the educational environment and thereafter to provide him with a suitable learning strategy constructing in this way an affective learning path (Leontidis and Halatsis, 2007). Exploiting the main advantages of the multimedia learning approach the MENTOR’s framework adopts a multimedia perspective to motivate effectively the learner in order to achieve his learning goals. By enriching its educational material, which consists in introductory topics of AI, with multimedia components, the MENTOR enables learning through exploration, discovery and experience while supporting learners in order to understand the concepts of the knowledge domain. The remainder of this paper is organised as follows: the next section, presents the basic concepts of the affective computing. Section 3 analyses the significant role of the multimedia in learning. Section 4 presents the architecture of the MENTOR’s affective module and its operation as well. Section 5 introduces the basic concepts of the proposed affective multimedia learning framework. In the following section, the evaluation framework and the experimental results of our study are presented in detail. Finally, we cite the conclusions and further work.
2
Basic concepts of affective computing
The term affective computing involves the intention of AI researchers to model emotions in intelligent systems. According to Picard (1997) an affective system must be capable of recognising emotions, respond to them and react ‘emotionally’. Personality determines all those characteristics that distinguish one human being from another. It is related to his behaviour and mental processes and has a permanent character. The most known model of personality is the five factor model (FFM) (McCrae and John, 1992) and results from the study of Costa and McCrae (1992). It is a descriptive model with five dimensions: openness, conscientiousness, extraversion, agreeableness, and neuroticism. Due to these dimensions the model is also called OCEAN model. The descriptive character of FFM and the particular characteristics that accompany each type of personality (traits) allow us to model the student’s personality (Ghasem-Aghaee and Oren, 2007) and use this information in educational applications (Conati and Maclaren, 2009). The FFM provides us with a reliable way in order to connect a student’s personality with his mood and emotions that he possibly develops during the learning process. Based on this we are able to initiate student’s emotional state and select the suitable pedagogical strategy. Although many efforts have taken place there is not an explicit definition for the emotion. It is easy to feel, but it is hard to describe it. According to Scherer (2000),
226
M. Leontidis et al.
emotion is the synchronised response for all or most organic systems to the evaluation of an external or internal event. Nevertheless, various attempts have been made, but the cognitive theory of emotions, known as OCC model, which formulated by Ortony et al. (1988), keeps a distinctive position among them. The three authors constructed a cognitive theory of emotion that explains the origins of emotions, describing the cognitive processes that elicit them. The OCC model provides a classification scheme for 22 emotions based on a valence reaction to events, objects and agents. In our work we adopt the OCC model, because it elicits the origin of emotions under a cognitive aspect and it is possible to be computerised. So, based on this model we are able to classify and interpret student’s emotions in the learning process.
3
Multimedia e-learning approach
In recent years, with the consolidation of the internet, the multimedia technology is incorporated in many WBES influencing notably the learning process (Clark and Mayer, 2007). As a consequence, new learning approaches and teaching methods have been developed in order to take into account the multimedia features. Thus, the multimedia perspective became an important parameter of learning, contributing in this way to an efficient educational process. The advances of internet and multimedia technology had a great impact on the development of multimedia WBES which integrate various media in order to present their educational content (Evans and Gibbons, 2007). As a result, more and more e-learning environments of this kind became available and used widely in distance learning. The adoption of the multimedia approach by the educational systems for distance learning offers a satisfactory answer to the many restrictions on time, space and equipment, which characterise usually these systems. By employing various media types including text, images, graphics, diagrams, sound, animation and video the multimedia technology enhances the traditional presentation of a web-based course and preserves the learner active, creative and engaged in the learning process (Passerini and Granger, 2000). The use of multimedia components in distance learning courses supports efficiently the interaction of learner with the educational system while keeping his attention concentrated on the learning material (Bouhnik and Marcus, 2006). Additionally, the integration of diverse media types enables the educational material to convey the key concepts and ideas of the knowledge domain to the learner in an easy and interesting way. In this manner, the effective retention and better understanding of a subject, which has proven to be beneficial for the learner’s progress, are ensured. As a result, the employment of multimedia technology in a web-based learning framework provides the educational process with a great potential for improving learner’s performance (Liaw, 2004). Moreover, the emergence of the multimedia technology has a major impact on the existing learning and teaching patterns which have been modified and improved adequately in order to respond accordingly to the novel educational challenges. The incorporation of media elements in the web-based educational material increases the effectiveness of the learning process, as supported by many researchers. Kettanurak et al. (2001) found that the interactivity of the multimedia educational material motivates the student efficiently and enhances learner’s performance by influencing positively his disposition. Jacobson (2008) reports that learner’s who made use of multimedia material
Using an affective multimedia learning framework for distance learning
227
improved considerably their problem-solving skills. Schar and Krueger (2000) found that the use of interactive multimedia technologies can lead to valuable learning experiences and high learning performances. This observation was also supported by Frear and Hirschbuhl (1999), who found that the use of multimedia in e-learning systems provides the learner with the opportunity to develop advanced cognitive skills such as understanding and retention of complex concepts of a knowledge domain. Mayer (2001), in his multimedia learning theory supports the opinion that multimedia learning improves the cognitive potential of the learner. Thus, the learner is provided with the ability to use the acquired knowledge for reasoning and inference in order to deal successfully with real situations. Almaoui et al. (2007) summarise the advantages of the multimedia educational material in a learning perspective which offers a framework for the production of flexible and adaptable learning content as well as the creation of realistic educational simulations. Moreover, they support the view that the multimedia technology supplies learners with many options in order to select an appropriate learning path according to their needs, through the presented educational material. This view provides evidence to the allegation of Herrington and Oliver (1999), that multimedia enable learning through exploration, discovery, and experience, contributing decisively to the active participation of the learner in the learning process. Considering the above reasons, more and more educational designers have recently paid close attention to the multimedia technology in order to build their systems and produce valuable learning material. As a result, the educational content (lecture notes, animations, presentations, and simulations) of a traditional e-learning system is augmented with multimedia capabilities. In addition, the consideration of the internet technology in the development of the contemporary e-learning systems, supplies these systems with amazing web abilities which enhance the learning process. Thereby, a traditional e-learning system can be transformed into a web-based and highly-interactive multimedia educational system, which is ideal for distance learning. Exploiting the main advantages of the multimedia technology, the MENTOR’s learning framework adopts a multimedia approach to motivate effectively the learner in order to achieve his learning goals. The MENTOR consists of dynamic interactive multimedia e-learning material on topics of AI. Using interactive learning modules, the development of which is based on multimedia components such as images, graphics, digital sound, animation and video, the MENTOR can support learners efficiently in order to understand the concepts of the knowledge domain. Additionally, by integrating multimedia learning objects into its educational material aims at preserving high the learner’s interest as well as delivering the learning information in a multi-modal way. Furthermore, employing a wide range of media it aims to create and/or maintain a positive mood in the student, since this is a crucial factor for the learning process (Goleman, 1995).
4
The architecture and the basic operations of the MENTOR’s affective module
MENTOR is a WBAES which is augmented with an ‘affective’ module with the aim of recognising the emotions of the student during his interaction within the educational environment and thereafter to provide him with a suitable learning strategy (Leontidis
228
M. Leontidis et al.
and Halatsis, 2007). The operation of the affective module is based on the FFM (McCrae and John, 1992) and the OCC model (Ortony et al., 1988). The affective module is being attached to the MENTOR providing the system with the essential ‘emotional’ information in order to determine the strategy of learning in collaboration with the cognitive information. The architecture of MENTOR is presented in Figure 1. The MENTOR’s affective module has three main components: the emotional component (EC), the teacher component (TC) and the visualisation component (VC), which are respectively responsible for: a
the recognition of student’s personality, mood and emotions during the learning process
b
the selection of the suitable teaching and pedagogical strategy
c
the appropriate visualisation of the educational environment.
The combined function of these components ‘feeds’ the educational system with the affective dimension optimising the effectiveness of the learning process and enhancing the personalised teaching. The main purpose of MENTOR is to create the appropriate learning environment for the student, taking into account particular affective factors in combination with cognitive abilities of the student offering in this way personalised learning. Figure 1
The architecture of the MENTOR
4.1 The EC Concerning the MENTOR, responsible for the recognition of the student’s emotions is the EC. This component (Figure 1) is composed by three subcomponents, the personality recogniser (PR), the mood recogniser (MR) and the emotion recogniser (ER), which are responsible for the recognition of the personality, mood and emotions of the student. As it
Using an affective multimedia learning framework for distance learning
229
has been already mentioned, there are five personality types. When the student uses the system for the first time, the PR subcomponent selects a suitable dialogue specified by the FFM to assess the type of a student’s personality. The dialogue is articulated in accordance to Goldberg’s questionnaire (Goldberg, 1999). As a result, the student’s traits are being recognised and are being used by the TC for the suitable selection of pedagogical and teaching strategy. For example, a student that has been recognised as openness, according to FFM is imaginative, creative, explorative and aesthetic (Costa and McCrae, 1992). These characteristics are evaluated by the TC providing the system with an exploratory learning strategy, giving more autonomy of learning to the student and limiting the guidance of the teacher. The MR subcomponent provides the system with a dialogue that can elicit emotions depending upon the semantics and its context. This dialogue is used in every new session and defines the current student’s mood (Figure 2). Based on this dialogue the student’s mood is recognised either as positive or as negative. In our approach, positive mood consists of emotions like joy, satisfaction, pride, hope, gratification and negative mood consists of emotions like distress, disappointment, shame, fear, reproach. As a result, we have an initial evaluation of the current emotions of the student. Thus, if the student is unhappy for some reason, the MR recognises it and in collaboration with TC, it defines the suitable pedagogical actions that decrease this negative mood and try to change it into a positive one. Finally, the ER subcomponent is in every moment aware of the student’s emotions during the learning process, following the forthcoming method. Figure 2
The initialisation process of the student’s emotional state recognition (see online version for colours)
So as to deal efficiently with the emotions elicitation process, the affective module has the learner affective model (LAM) where the affective information is stored. The LAM stores the system’s knowledge about the learner, such as general information about him, his Affective Style, his current emotional state, his knowledge level and his learning
230
M. Leontidis et al.
goals (Leontidis and Halatsis, 2009). It reflects the learner’s individual characteristics and also tracks how well he performs on the material which is being taught while supports his interaction with the MENTOR. In this manner the LAM can deal with the cognitive abilities as well as the affective preferences of the learner. Consequently, the adaptation of MENTOR relies decisively on the LAM which stores the affective as well as the relevant cognitive information about the learner. The LAM was implemented by adopting an ontological approach (Figure 3), so that the representation of the learner’s affective information can be achieved in an efficient way. During the interaction with the system the affective module builds a learner’s affective model for each individual learner and continuously updates it in order to keep always the current affective state of the learner. The LAM is dynamically updated and refined during interaction by implicit relevance feedback provided by the learner. Each time a learner interacts with the educational material of MENTOR the system updates appropriately the LAM. More specifically, a request for a web page (passed to a PHP-presentation layer by the web-server for accepting the files and update the database) triggers adaptation rules that perform LAM updates. The adaptation rules define how the LAM is updated. When the learner accesses a page from the system’s educational material the corresponding rules associated with the access attribute are triggered. This mechanism allows maintaining the model in such a way that it always represents the cooler interests and non-interests of the user. In this way it is possible for the learner’s current affective state to be stored and therefore to deal with the HTTP protocol’s stateless nature which is one of the major problems in the development of a web-based application (Berners-Lee et al., 1996). Figure 3
An excerpt from the LAM (see online version for colours)
Using an affective multimedia learning framework for distance learning
231
Furthermore, we adopt a decision tree approach, an AI technique (C4.5 algorithm) (Quinlan, 1996) to extract information from the LAM and to make inferences about the emotions of the student. This process comprises three steps which respectively are the following: 1
the creation of the decision tree
2
the extraction of the rules from the decision tree
3
the triggering of the extracted rules to infer student’s emotions.
This approach, which is used for carrying out the representation and the inference of emotions, is based on the OCC model which combines the appraisal of an event with the intentions and desires of a subject. Thus, taking advantage of this model, MENTOR infers about the student’s emotions after the occurrence of an educational event which is related to his learning goal.
4.2 The TC The TC consists of two sub-components, the teaching generator (TG) and the pedagogical generator (PG), as illustrated in Figure 1. These sub-components are responsible respectively for the formation of the appropriate teaching and pedagogical strategy. The TG is a sub-component which is responsible for the selection and presentation of the suitable educational material, according to the LAM. The LAM provides information about the cognitive status of the learner such as his learning style, the knowledge that has already been acquired and his learning preferences and goals. Evaluating this information, the TG decides about the sequence of the educational material, if a theoretical or practical subject will be presented next to the student and what kind would this be, for example a more or less detailed theoretical topic or an easier or a trickier exercise. The PG is a sub-component which is responsible for the formation of the pedagogical actions which will be taken into account during the learning process. Once the recognition of the student’s emotions and his emotional state has been stored in the LAM, the PG has all the necessary information in order to support and motivate the student to the direction of the achievement of his learning goals. As a teacher does in the real class, the PG encourages the student, gives him positive feedback, congratulates him when he achieves a goal, and keeps him always in a positive mood, with the view of engaging him effectively in the learning process. Combining the interaction of its two sub-components, the TC forms the appropriate affective tactic for the learner. In this way, a traditional instructional tactic is enhanced with a motivational one and this would be proved beneficial to the student from two aspects (Mohan et al., 2003). The first concerns the planning of the teaching strategy and the educational content, which and what topic will be taught to the student next and which method will be used for it. The second is more related to the delivery planning, how this topic will be taught. At this point, it should be noted, that the outputs of the two sub-components might be contradictory. For example, the TG evaluates the current knowledge state of the learner
232
M. Leontidis et al.
and suggests a difficult exercise. On the other hand, relying on his current emotional state, the PG recommends an easier one, because it judges that the student’s confidence is low. So, resolving an easier exercise, it estimates that his confidence will be reinforced. In that case, the TC is designed so that, it would rather promote its PG recommendation. Let us examine the reverse case, where the TG suggests a trivial problem to a confident Openness student. This suggestion might be considered as motiveless by the PG, compared to the student’s current emotional state. To tackle with this conflict, a more difficult problem is presented by the TC, demanding the student’s harder effort and challenging his interest further. The role of the TC, however, is not restricted only to the reassurance of the appropriateness of the teaching method or the educational material. It is concentrated also on providing the learner with encouraging actions in order to preserve his positive emotional state. To achieve this, the MENTOR’s affective module has to be aware of the student’s emotions. The input that comes from the EC, which is in charge of the detection of the student’s motivational state, is evaluated appropriately and thereafter the MENTOR adapts his reaction adequately to motivate the student either by encouraging him or by praising him and in every case sustain his disposition flourishing. Once the affective module is aware of the student’s emotions, it can proceed into the selection of the proper affective tactic (Figure 4). Figure 4
A sample of the MENTOR’s affective tactics and their production rules
AT1: Congratulate the student AT2: Give help to the student AT3: Praise the student AT4: Express sympathy in case of fail AT5: Explain the need for help AT6: Express admiration for the student AT7: Play a game with student AT8: Encourage the student AT9: Present a part of a video clip AT10: Tell a joke
Let us examine, for example, the case of a student whose personality belongs to the Extraversion category, but his mood is recognised in the current session as negative. For this type of student the TG has already selected an exploratory teaching method without examining his emotional state. Before the TC applies this method, it interacts with the PG. By analysing furthermore why his mood is negative, it comes to light that the student is anxious for some reason. The system takes upon making the student feel relaxed firstly by opening a short dialogue with him. Thereafter, it presents to him either a joke or a funny video clip (AT9/AT10) (Figure 5), according to his preferences which are stored in the LAM. Finally, it motivates him either by encouraging him or by praising his abilities (AT3/AT8).
Using an affective multimedia learning framework for distance learning Figure 5
233
The proposed affective tactic (AT9) of a video clip’s presentation (see online version for colours)
Note: The video clip presents in a funny way the problem of cannibals and missionaries, one of the most known problems in the AI basic concepts course.
Another case is when a conscientiousness student fails to accomplish a given task. Then negative emotions such as sadness or disappointment can appear. He seems to be less confident in the current session and there is the danger of giving up the trial. He fears maybe that he has not got the ability to deal with a project that was assigned to him and he will not live up to his teacher’s expectations. According to Figure 4, there are pedagogical actions which can be applied in order to eliminate the student’s negative emotions. For instance, the system may praise him for his effort (AT4/AT3), give him help (AT2) and encourage him to try again (AT8). Then the system presents him an easier problem to reinforce his confidence and to foster positive emotions. In this way, the student has great chances to resolve the problem, so that his confidence would be regained and positive emotions such as happiness or satisfaction can preserve an upbeat to the student’s mood.
4.3 The VC The VC of the MENTOR’s affective module is responsible for the application of the suggested affective tactic and the presentation of the suitable educational material. It is composed by three subcomponents, the control unit (CU), the presentation coordinator (PC) and the MENTOR’s pedagogical agent. The structure of the VC is illustrated in Figure 6.
234 Figure 6
M. Leontidis et al. The architecture of the VC (see online version for colours)
The CU of the VC selects the proper courses of action in order to apply the suggested affective tactic. The major concern of the CU is to mediate between the PC and the MENTOR’s agent as well as control their operation, according to the provided affective tactic. In particular, when an affective tactic is received by the VC, the CU is in charge to evaluate it in order to decide which of the PC or the MENTOR’s agent will perform the affective tactic. In case that the affective tactic demands the adaptation of the learning environment or the presentation of a different type of learning task, the specific action is realised by the PC. Otherwise, if the affective tactic is related to a particular affective action, then this action is performed by the MENTOR’s agent. The PC is responsible for the appropriate formation and presentation of the educational material in relation to the provided affective tactic. Taking advantage of the modular structure of MENTOR’s educational material, which is analysed in detail in the next section, the PC offers an efficient mechanism for the adaptation of the educational content to the knowledge needs of the learner. In this way, the appropriate content of the html learning documents is formed accurately, according to the proposed affective tactic. The operation of the PC is supported by four sub-components, the content manager, the multimedia controller, the presentation rules provider and the HTML document generator. These sub-components are respectively in charge of the appropriate selection of the learning content, the proper integration of the multimedia objects, the supply of the presentation rules and the final construction of the html document. The third basic part of the VC, which is the MENTOR’s pedagogical agent, is visualised by an animated character with the aim of providing messages and gestures for assisting and motivating the learner in order to accomplish his learning goals. The operation of the MENTOR’s agent is based on two essential parts, the action database
Using an affective multimedia learning framework for distance learning
235
and the performance manager. The action database stores the verbal and physical behaviours of MENTOR’s agent in a schema which is called affective action. The main concern of the performance manager is to select a suitable verbal and physical action from the action database in order to form the affective action that is going to be performed by the agent.
5
The affective multimedia learning framework of MENTOR
MENTOR uses the multimedia technology in order to create an interesting and motivational educational environment. As it is already referred, MENTOR’s environment is anticipated to attract student’s attention as well as motivate appropriate him in order to achieve his learning goals. For this reason, it makes use of multimedia learning objects which enrich the training process with a more memorable, understandable and easy accessible educational content, contributing in this way to an authentic learning experience. As the integration of multimedia components in the learning systems can foster a positive disposition in the learner and produce positive emotions, the MENTOR adopts this perspective into its affective learning framework. Therefore, the use of multimedia technology constitutes the essential ingredient in order to implement the MENTOR’s affective dimension. Taking advantage of the multimedia technologies, the MENTOR provides learners with an opportunity to explore the educational material under an interesting and creative way. Nowadays, the web development logic, demands the use of a wide range of media that can be integrated suitably in an educational system (Roesler et al., 2009). Following this logic, the MENTOR’s VC makes use of a variety of multimedia components such as text, graphics, sounds, animations and videos in order to achieve an efficient delivery of the educational material. The appropriate combination of different media types as well as their suitable integration into the MENTOR’s learning objects supports learners’ efforts and contributes to a successful training process. Moreover, the MENTOR makes use of the multimedia advantages with the aim of keeping the interest of student alive, his creative spirit excited and his efforts high. In particular, the VC of MENTOR uses text in order to present the main concepts of a theoretical part of a topic in an appropriate brief but comprehensive way. Using a cohesive instructional style, the educational content is explained to the learner putting emphasis on the understanding and explanation of the key concepts. In order to demonstrate and clarify complicated concepts of the knowledge domain, the VC uses graphics. In this way, the learning process is enriched visually while the learner’s interest remains active. Another important ingredient of the MENTOR’s multimedia framework is sound. In this case the learning experience is enhanced, as the presented text or graphics of the educational material are accompanied by a sound narration. By making use of the proper sound intonation the educational system can emphasise on the main ideas of the educational material. The MENTOR uses the sound not only to underline the presented text but also to encourage or to congratulate the student in relation to his progress. In this manner, the learner’s affective state is preserved positive and his disposition is sustained flourishing. The persistent motivation of the student as well as the positive preservation of his mood, are crucial parameters of learning and the use of animation is adopted by the MENTOR towards this
236
M. Leontidis et al.
direction. The VC uses this kind of media in order to achieve visual stimulation and motivation of the learner, while introduces complex principles of the educational material. In this manner, learner’s attention is expected to stay focused on the training process. The primary structure of MENTOR’s educational material is organised hierarchically into five levels. These levels are respectively the course, lesson, concept, document and multimedia learning object level as illustrated in Figure 7. Thus, the knowledge domain of MENTOR can be represented in five hierarchical levels of knowledge abstraction. This modular and hierarchical structure is selected in order to manage and organise efficiently the MENTOR’s educational content in relation to its multimedia dimension. In particular, each course is categorised into several lesson units. Each lesson unit is dedicated to only one topic which consists of individual concepts relative to the topic’s subject. Each concept is subdivided into html documents which contain various multimedia learning objects. The multimedia learning objects are the essential learning element of MENTOR. They are integrated into the MENTOR’s learning material and can be combined suitably in order to form various types of presentation as well as different learning goals. In essence, the multimedia learning objects are knowledge resources enriched with various media. They are designed as elements of training which can be used or reused in various educational cases. They can be combined with other learning objects to form more complex items of learning. The MENTOR distinguishes five different types of multimedia learning objects which are the: 1
theory
2
exercise
3
activity
4
assistance
5
assessment.
Each one has specific characteristics and aims to be used in different educational situations. For instance, a multimedia learning object which belongs to the Assistance type can be defined either as text, sound or animation and used in educational cases which demand additional description, further explanations or concrete examples of a concept. Adopting the model of the multimedia learning objects, the knowledge domain of an educational system can be represented by a wide range of media. In the case of MENTOR, each of these objects can be incorporated in the educational material either as text, graphics, sound, animation, or video. In this way, different kinds of media can be integrated into an html document. According to the MENTOR’s learning material organisation, an html document contains content that defines and/or explains a concept. As each concept is formed by one or more html documents, the concept level can describe the certain idea of the presented topic in detail, using a variety of media. In this manner, the lesson units, which are in the next upper level, are defined from multimedia concepts. Lesson units can have any number of concepts, but a concept can only appear once within any lesson unit of a course.
Using an affective multimedia learning framework for distance learning Figure 7
237
The hierarchical structure of MENTOR’s educational material (see online version for colours)
In order for the hierarchical structure of the MENTOR’s educational material to be more flexible and efficient another secondary organisation is used additionally to the primary that is already described. Thus, in a parallel level each html document can be further categorised into three types of presentation namely, a
lecture
b
practice
c
evaluation, as illustrated in Figure 8.
While each one differs in the degree of interactivity, their main aim is to preserve efficient and flexible the teaching process. In particular, the lecture type is comprised of html documents which contain the theory, examples and explanations of a topic. The lecture type documents deal with specific subject matters or abstract principles of the examined topic. A document of this type, enriched with the multimedia dimension, is the virtual equivalent of a conventional lecture or an oral presentation. The main aim of the lecture documents is to help the learner in better understanding of a subject matter, providing him with definitions, representative cases, additional learning resources and references presented as hyperlinks. The documents of the practice type are html documents that consist of exercises, case-studies and small projects. They are used mainly to exercise specific knowledge and apply abilities that aimed to be acquired by the learner during the course. Each document of this type incorporates activities which require the learner to participate by doing something, such as matching statements or problem-solving tasks. Thus, the documents of the practice type offer an opportunity for practice over the presented topics of the lecture documents as well as a chance for the learner to interact with real world applications of the topic in question. The practice documents enriched with interactive
238
M. Leontidis et al.
multimedia objects enable the student to understand, assimilate and retain basic learning concepts of the subject matter. The documents of the evaluation type are html documents with the aim of appraising the acquired knowledge of the learner. The documents of this type include assessment elements which cover the basic concepts that the student should have learned throughout the particular lesson unit. An evaluation document contains tests, questionnaires or exams and makes use of the multimedia learning objects in order to improve the learning outcomes of the student. Thus, both the assessment tests and the examination tasks are integrated into the html document with an interactive way. In this manner, they aim at motivating the learner in order to respond successfully to the challenges of the examined subject matter as well as keeping his attention focused on the evaluation process. The modular structure of the MENTOR’s educational material into well-organised levels provides an efficient mechanism for the adaptation of the learning content to the knowledge needs of the learner. Additionally, it allows the flexible reuse or modification of the educational resources. More specifically, by using the LAM the MENTOR’s educational environment is customised in an adaptive manner to accommodate learners’ various personality types. The rationale behind this approach is that accommodating the individual affective, cognitive and personality differences of learners, leading to an increased learning performance and satisfaction. The LAM is updated dynamically according to the individual affective and cognitive needs and preferences of learners in order to provide an adaptive learning environment tailored to each learner. For instance, an extrovert learner has usually a strong preference for learning resources such as demonstrations, experimentations or simulations in image, animation or video format, while a conscientiousness learner has usually a strong preference for learning resources such as theory, explanations or examples in text, hypertext or audio format (Table 1). Figure 9 exemplified how the educational content can be adapted to the user interface layout in learning environments in order to support learner’s emotional state, interests and preferences. Table 1
Suitable multimedia type in relation to the MENTOR’s learning resources for an extrovert and a conscientiousness learner, respectively
Extrovert learner
Conscientiousness learner
Demonstrations (image, animation, video)
Theory-synthesis (text, hypertext)
Experimentation-problem solving activities (image, animation)
Analysis-explanations (text, audio, hypertext)
Simulation (video)
Examples (text, hypertext)
According to the MENTOR’s learning framework (Leontidis and Halatsis, 2009), an extrovert learner feels more comfortable with experimentation, active process of the learning information and visualised educational material. On the other hand, a conscientiousness learner processes the educational information thoughtfully in order to go into deeper information and understanding. As a result, he prefers texted educational material with many details and various examples. Consequently, the VC in order to preserve a positive emotional state adapts and presents the educational material with more images, demonstrations, simulations, or videos on the screen for a learner who belongs to the extraversion category, while with more text, hypertext, explanations, examples and audio for a learner who belongs to the conscientiousness category. Similar
Using an affective multimedia learning framework for distance learning
239
are the processes and the adaptation of the educational material for the rest types of learners. The suggested affective tactics play a pivotal role in providing the customisation of the MENTOR’s educational environment. According to these tactics and based on multimedia information guidelines (Carmagnola et al., 2008, Stock et al., 2007), the interface is subdivided into appropriate placeholders where the learning information is displayed. The process is fully automated, based on a set of built-in adaptation rules in relation to the provided affective tactics. In this manner, the course pages are dynamically generated by the VC for each learner according to his affective model. In particular, the appropriate educational html document is constructed dynamically by exploiting a well-defined structure which is stored in a XML file and filling the corresponding placeholders with the proper multimedia learning objects. The above described adaptation process is illustrated in Figure 9, with a multimedia html document which is generated in two different versions for an extroverted and a conscientiousness learner, respectively. Each of these documents is dynamically constructed by selecting and ordering the adequate multimedia objects in the corresponding placeholders according to the learning information and the affective model of learner. Relied on this structure, the VC forms accurately the appropriate content of the learning html documents. In this way, whenever the VC is requested by the teaching component for a specific concept to be presented to the learner, constructs the content pages according to the learner’s emotional and learning individual needs and the selected affective tactic, as well. In particular, taking advantage of the previously described organisation of the educational material, the MENTOR’s VC creates the main page which consists of all the available courses of the MENTOR’s knowledge domain. Currently, there is one available course in the MENTOR, an introductory course on AI. MENTOR’s courses are hierarchically structured into smaller lesson units. The lesson units are visualised with a menu structure, where each of them can be easily accessed, as shown in Figure 10. The lesson units of the available course are: •
introduction to AI
•
perception
•
problem-solving
•
knowledge and reasoning
•
planning
•
learning.
As depicted in Figure10 the specific menu structure supports the modular and hierarchical organisation of the educational material while promoting its multimedia dimension. Each lesson unit consists of learning concepts and each concept can be defined by a set of multimedia html documents. Each html document which is characterised either as lecture, practice or evaluation type according to its learning nature, contains the appropriate multimedia learning objects. In this manner, each type of the html documents is formed by the appropriate combination of the multimedia learning objects, as illustrated in Figure 8.
240
M. Leontidis et al.
Figure 8
The secondary organisation of MENTOR’s educational material according to the three types of presentation (see online version for colours)
Figure 9
Interface layout for a conscientiousness and an extrovert learner (see online version for colours)
Notes: The presentation of the educational material for the conscientiousness learner comprises mainly (hyper) text and audio-driven html documents. Conversely, the presentation of the educational material for the extrovert learner comprises mainly image and video-driven html documents.
Using an affective multimedia learning framework for distance learning
241
Figure 10 The modular organisation of the MENTOR’s learning material as it is visualised with a menu structure in the learner’s browser (see online version for colours)
In conclusion, the modular structure of the MENTOR’s educational material aims at delivering effectively the learning information as well as allowing the student to learn the material in segments, according to his particular learning goals. In addition, this structure supports the efficient presentation of the educational content in order to guide appropriately the learner through a well-organised and suitably formatted learning path in relation to his individual needs. To this direction, the learner’s affective model of MENTOR plays a significant role since provides the VC with the necessary information of the student’s profile in order to define the order and the structure of the educational material’s appearance. Thus, adopting the proposed multimedia learning approach the MENTOR’s affective dimension is promoted, supporting efficiently the learner in order to achieve his learning expectations.
6
Evaluation study
In this section the evaluation of the MENTOR is presented and the corresponding experiments are described in detail. The evaluation concerns the impact of the system in the learning process of the students with the MENTOR. For this purpose the learning performance of an ‘affective’ multimedia WBAES is compared with the corresponding of a non-affective WBAES. The latter was deprived of the incorporation of the affective module.
6.1 Affective versus non-affective MENTOR The affective multimedia version of the MENTOR integrates the affective multimedia learning framework by incorporating the affective module, while the non-affective
242
M. Leontidis et al.
version deprives of the affective module, that is operates only as a traditional WBAES. As a result, the non-affective version cannot be aware of the emotions and the learner’s affective state. Furthermore, it lacks of the operation and the services of the pedagogical agent which are consist in the appropriate messages and gestures for assisting and motivating effectively the learner in order to accomplish his learning goals. The realisation of the agent as an animated character aims to carry out affective behaviours that foster positive emotions in the learner. Consequently, the ablation of the affective module eliminates the motivational contribution of the agent. The key differentiation of the MENTOR’s affective version comparing with the non-affective, stems out mainly from the existence of the LAM which is responsible for the affective adaptation of the MENTOR. Taking into account the affective style of the learner (Leontidis and Halatsis, 2009), the LAM provides the TC with all the necessary information in order to form the appropriate affective tactic. Furthermore, by applying the suggested affective tactic the educational environment is adapted adequately presenting the most suitable content according to the affective needs of the learner (Figure 5). To illustrate the difference between the two versions of MENTOR, let us consider the case of a learner whose personality belongs to the Agreeableness category and fails to an evaluation test. Then negative emotions such as disappointment or fear can appear. In this case, the system’s affective version may express sympathy and praise him for his effort, give him help and encourage him to try again. Then the MENTOR’s VC in collaboration with the TC presents him an easier test in order to reinforce his confidence and foster positive emotions. In this way, the student has great chances to respond successfully to the test and positive emotions such as satisfaction or hope can appear. Conversely, the non-affective version lacks of the ability to provide the learner with the proper affective tactic and the consequent emotional support. To summarise, the affective module transforms a typical AES to an affective one by providing emotional awareness, affective tactics, pedagogical agent’s enhancement and affective-multimedia adaptation of the educational content.
6.2 The framework of the evaluation study 6.2.1 Participants of the experiments In order for the process of sample gathering to be the appropriate the following procedure has been followed. A sample of 120 students in the field of computer science has been selected. Their age was between eighteen and twenty-five years old (M = 20.9, SD = 2.27). The students were assigned with a questionnaire containing items relative to the field of AI. From the statistical evaluation of their answers a group of 108 out of the initial 120 was selected. The criterion was the lowest average which signifies the lowest starting knowledge on the domain and the lowest possible dispersion around it (M = 6.24, SD = 3.18). In this way, a homogeneous group was formed with the same average a-priori knowledge about the learning domain. In total, there were 65 male and 43 female students. They were randomly divided into two groups, the experimental group A and the control group B with 54 students in each group. The students of group A make use of the affective multimedia version of the MENTOR, while the students of group B with the normal version of the adaptive system.
Using an affective multimedia learning framework for distance learning
243
6.2.2 Questionnaires To evaluate the student’s acquired knowledge and learning gain after the interaction with the MENTOR the following questionnaires were built and proposed: •
Pre-test questionnaire: The students of the experimental and the control group attended an individual test. The test was formed by 20 items in order to check the starting knowledge of the sample. The main aim of the test was to measure the initial knowledge of each participant in the field of AI. It was designed as a set of multiple choice and true/false items. The answers of each student are analysed by statistics methods and the results compared with the post-test.
•
Post-test questionnaire: The students of group A interacted with the affective multimedia version of the WBAES while the students of group B did not. Therefore, the group B deprived the affective multimedia dimension of MENTOR. The whole time of the interaction process was 45 minutes. After that, all participants were assigned with a post-test questionnaire, with the aim of assessing their learning performance. The post-test questionnaire consisted of 30 items and aimed to measure the acquired knowledge after having interacted with the MENTOR. The structure of the post-test questionnaire was designed, similarly to the pre-test questionnaire, as a set of multiple choice and true/false items.
6.2.3 Experimental setup and statistical analysis The experiment started with the pre-test questionnaire. The students of the two groups were asked to complete the pre-test using pencil and paper. The achieved score of each student was recorded. The results were subjected to a statistical analysis which is shown in Table 2. The pre-test results demonstrate that both groups had similar knowledge level on the knowledge domain of AI. Table 2
t-Test results for the difference between experimental and control group means Group
N
Mean
Std. deviation
Std. error mean
EG
54
39.241
2.6913
.3662
CG
54
42.185
4.6177
.6284
EG
54
74.673
6.7047
1.0108
CG
54
71.393
6.9025
1.0406
Group statistics (pre-test) Grade
Group statistics (post-test) Grade
After completing the pre-test, students interacted with the system. The group A deal with the educational material of the affective version of MENTOR, while the group B with the educational material of the non-affective version. At this phase the students had the chance to practice and enhance their knowledge dealing with the educational material via the system’s interface. During this interaction, the students’ actions were recorded in the system’s log files, with the aim of providing information concerning the time they spent
244
M. Leontidis et al.
and their performance. Afterwards, the student’s were assigned with a post-test and were asked to complete it using again pencil and paper. The results of the post-test were subjected to a statistical analysis that is shown in Table 2. We made use of the t-test for the result’s computation of the experiment. The independent variable was defined as the difference between the score obtained by each student in the post-test and the one obtained in the pre-test. This independent variable allowed us to measure the potential progress in student’s learning, after his experience with the MENTOR.
6.2.4 The research question We consider the study effectiveness primary from the aspect of the test score, so the research question (RQ) is formulated as: RQ Do students interacting with the affective multimedia version of the MENTOR achieve better learning results than students interacting without the affective multimedia version of the MENTOR?
6.2.5 Group analysis In order to answer the RQ we performed the analysis of the statistical differences between groups by means of the two-sample independent t-test. Based on the RQ the null and the alternative hypotheses are formed as follows: Null Hypothesis H0
There is no difference, between the group A and the group B, after the interaction with the learning environment of the two versions of MENTOR.
Alternative Hypothesis Ha
The two groups A and B are different in terms of the learning performance after the interaction with the learning environment of the two versions of MENTOR.
Since a preliminary Levene’s test for equality of variances indicated that the variances of the two groups were not significantly different, a two-sample t-test was performed that does assume equal variances, defining the significance level at α = 0.05. The provided results are presented in Table 3. By analysing furthermore the results from this table we realise that the mean grades of the students who interact with the affective version of MENTOR (EG) (M = 74.67, SD = 6.70, N = 54) was significantly different from these using the non-affective version of MENTOR (CG) (M = 71.39, SD = 6.90, N = 54), t(106) = 2.26, p = 0.026, where p
