Interfaces and Human Computer Interaction 2008 - Semantic Scholar

ISBN: 978-972-8924-59-1 © 2008 IADIS

MULTI-MODAL AIDED PRESENTATION OF LEARNING INFORMATION: A USABILITY COMPARATIVE STUDY Dimitris Rigas and Marwan Alseid Department of Computing, School of Informatics, University of Bradford Bradford, BD7 1DP, England

ABSTRACT This paper presents a comparative two-group experimental study to explore if the addition of multimodal interaction metaphors would enhance the usability of e-learning interfaces. Two independent groups of users were involved in the experiment each of which tested one of the two interface versions provided by the experimental e-learning tool. The first interface was based on textual approach in delivering learning information about class diagram notation. In the second interface, recorded speech, earcons and avatar with simple facial expressions were combined to present the same information. Efficiency, effectiveness, and user satisfaction were considered to evaluate the usability of both interfaces. This paper reports and discusses results in regard to effectiveness only. Results of the experiment showed that users of the multimodal-based interface significantly performed more successful tasks than their counterparts who used the text-based interface. Therefore, adding audio visual interaction metaphors to the interface of e-learning applications can improve its usability. KEYWORDS Multimodal interaction, e-learning, usability, speech, earcons, avatar.

1. INTRODUCTION Focusing only on the visual channel of the user and disregarding other human senses (i.e. auditory channel) during the human-computer interaction process resulted in usability problems in the interface of many computer applications including e-learning. For example, users’ visual channel could be overloaded (Brewster 1997) and important communicated information could be missed (Oakley, McGee et al. 2000). The reviewed previous work demonstrated the remarkable positive effects of involving speech and non-speech sounds on the usability of computer interfaces. Other studies showed that avatar with facial expressions could be beneficial in e-learning interfaces. However, incorporating more multi-modal interaction metaphors in elearning interfaces is still under research. In this paper, we investigate the use of speech sounds (recorded speech) and non-speech sounds (earcons) alongside avatar with simple facial expressions for enhancing the effectiveness of e-learning interfaces. This investigation was conducted by comparing the effectiveness of two interface versions of the experimental e-learning tool each of which utilized different metaphors in delivering the same learning material about class diagram notation. More details are given in the following sections.

2. PREVIOUS WORK Using a variety of multi-modal metaphors in human-computer interaction has been evaluated by a series of studies. These studies proved that including these metaphors can enhance the usability of user interfaces. When these senses are involved in human computer interaction, users will feel that they interact with computer more naturally and they can use the most suitable type of interaction metaphor to their abilities (Dix, Abowd et al. 2004). As a result, the amount of information delivered by one specific interaction metaphor will be reduced (Brown, Newsome et al. 1989) and different channels could be used to communicate different types of information (Sarter 2006).

234

IADIS International Conference Interfaces and Human Computer Interaction 2008

Earcons are short auditory musical stimuli (Blattner, Sumikawa et al. 1989) that has been applied to involve the auditory channel in user interfaces. It has been utilised effectively to enhance user interaction with components frequently appeared in computer interfaces such as scrollbar (Brewster, Wright et al. 1994) and progress bar (Crease and Brewster 1998). Also, it was successfully used to provide users with auditory feedback related to events during the interaction (Rigas and Alty 1998). Other studies demonstrated that earcons can improve students' attitude and their understanding (Upson 2002). Moreover, including recorded speech and earcons in the interface of multimedia on-line learning assisted users to complete complex learning tasks more successfully (Rigas and Hopwood 2003). Another multimodal metaphor that incorporates both of visual and auditory interaction is the avatar. It is a computer-based character that has the ability to express feelings, emotions, and other linguistic information via various facial expressions and body gestures (Beskow 1997). Avatar could be employed in virtual learning environments as a virtual presenter (Sheth 2003). Also, the addition of avatar’s facial expressions especially the smiling could motivate students and increase their learning achievement (Theonas, Hobbs et al. 2008). The use of computer networks and machines in the learning process resulted in the appearance of what is called e-learning (Alexander 2001; Mikic and Anido 2006) and the continuous development of information and communication technology (ICT) caused an accelerated development in educational technology. As a result, e-learning becomes a vast area of research and technologies used in the development of e-learning applications have been increased (Hamilton, Richards et al. 2001) where mobile devices such as cell phones, PDAs and Tablet PCs were involved in mobile learning (Mikic and Anido 2006). E-learning has many advantages. For example, a consistent presentation of the same learning materials and skills could be offered without limitations of time and location (Mikic and Anido 2006). Also, e-learning could be applied to prompt students (Theonas, Hobbs et al. 2008) and to satisfy a collection of pedagogical methodologies (Spalter, Simpson et al. 2000). However, e-learning applications exposed to some difficulties and challenges. Some examples include the difficulty of accessing the required technology and the ability of students to use it (Brady 2001). Moreover, when learning styles (activist, reflector, theorist or pragmatist) (Honey and Mumford 1986) were analysed towards ICT, it was found that students felt uncomfortable using computers and experienced deficiency in face-to-face interaction (Shaw and Marlow 1999). Therefore, satisfaction of users and their ability to access ICT technology must be improved (Shaw and Marlow 1999) and pedagogical principles must be utilised (Govindasamy 2001). The reviewed literature showed the potential of multimodal interaction in enhancing human-computer interaction in a multiple of applications including e-learning. In this study, we investigated the use of combination of recorded speech and earcons alongside avatar with simple facial expressions to communicate learning material, and how the addition of these metaphors would affect the usability of e-learning interfaces.

3. EXPERIMENTAL TOOL In order to empirically investigate the effect of using multimodal metaphors in e-learning interfaces, two different versions of the experimental e-learning tool were built; textual-based interface version and multimodal-based interface version. Each version offered different modalities to present the same learning information about notations frequently used in class diagrams. In addition to class attributes and operations, information about the multiplicity of a given class and the associations among classes were also communicated. Three common examples of class diagrams with increasing level of complexity were included in both versions, each of which represented specific problem and displayed separately. Text-based interface simulates the interaction style applied in most of the existing e-learning interfaces. An example screenshot of this interface is shown in Figure 1A. It presents the information about class diagram notation in a textual approach and only visual channel of the user could be used to receive it. No other human senses were involved in the interaction. When the mouse cursor placed over a notation in the class diagram (refer to 1 in Figure 1A), an explanation of that notation is appeared textually in the notes text box (refer to 2 in Figure 1A).

235

ISBN: 978-972-8924-59-1 © 2008 IADIS

1

Neutral

Happy

Sad

Smile

2

(A) Text-based interface

(B) Multimodal-based interface

Figure 1. Examples of both interfaces used in the experimental tool

Figure 2. Used facial expressions

In the multimodal-based interface (see Figure 1B), a combination of visual and auditory interaction metaphors were provided to replace the notes text box used in the text-based interface. A human-like avatar with simple facial expressions (see Figure 2) was added to the interface to verbalize the explanations about classes. The usage of these expressions basically depended on the content of the presented information. Earcons and recorded speech were used to communicate the information related to multiplicity and associations respectively. The aim of including these audio visual metaphors was to reduce the load on users’ visual channel and to incorporate their auditory sense in obtaining the presented information. A total of six types of multiplicity were needed in the three class diagram examples included in the study. Therefore, six earcons were created using simple musical tones starting at middle C in the chromatic scale. The design of these earcons was based on proposed guidelines (Brewster, Wright et al. 1995; Rigas 1996). Each earcon was utilized to represent one type of multiplicity. Two earcons were composed of only one auditory message to represent the multiplicities: one (1) and many (*) which means zero or more. Each of the other four earcons had two auditory parts separated by a very short time of pause (0.6 sec) in between, and communicated one of the multiplicities: zero or 1 (0..1), one or more (1..*), two or more (2..*), and one or two (1..2). Therefore, a musical illustration of the values 0, 1, 2, and * was needed to create the appropriate earcons. For this purpose, only one note of seashore sound was used to distinguish zero and to represent it in the multiplicity 0..1. Also, different numbers of piano tones with rising pitch were used to represent the other values as follows: one tone to communicate 1, two tones to communicate 2, and four tones to communicate many (*).

4. EXPERIMENTAL DESIGN This experiment aimed at exploring which of the two interface versions of the experimental e-learning tool would be of higher level of effectiveness, which has been measured by the number of successfully completed tasks. So, it was hypothesized that users of the multimodal-based interface will perform more successful tasks than users of the text-based interface. Tow independent groups of users were involved in the experiment each of which consisted of 15 users. The first group (control) applied the text-based interface and the second group (experimental) applied the multimodal-based interface. Users’ participation in the experiment took place in an individual basis. Most of the participants in both groups had a scientific background, and normally they use computer for ten or more hours a week. Additionally, the majority of users in both groups had a limited or no experience in class diagrams. At the beginning of the experiment, each user attended two tutorials. The first one is a five-minute introduction about class diagram notation and the second tutorial is a two-minute demonstration of the interface version under assessment. After that, each user performed six tasks. These tasks were common for both groups and were gradual in terms of its difficulty. In each task, different types of information about classes, associations and multiplicities were presented by the applied interface (as explained in section 3). At the end of each task, two questions related to the communicated material were asked by the interface and should be answered by the user without any help. To answer the first question (recall), some of the communicated key information must be recalled from users’ memory and then inserted using the keyboard. In the second question (recognition), user should recognize and then chose the correct answer(s) among a set of 2 to 4 options given in the form of check boxes or radio buttons. When data were collected and analysed,

236

IADIS International Conference Interfaces and Human Computer Interaction 2008

Figure 3. Percentages of successfully competed recall and recognition tasks in both groups

each question was considered as a task. Therefore, each user performed 12 tasks evenly divided into 6 recalls and 6 recognitions.

5. RESULTS AND DISCUSSION The total number of performed tasks in each group was 180 (15 user * 12 tasks per user) half of which based on recall activities and the other half was of recognition nature. The answers to the required tasks were checked and the number of correctly completed tasks for each user in each group was calculated and then used for the statistical analysis. Results of the experiment showed that the number of successfully performed tasks in the experimental group was higher than the control group. The experimental group successfully completed 149 tasks (83%) while users in the control group correctly completed 104 tasks (58%). The difference was found to be significant (t=4.75, cv=1.70, p