Modeling Audio-Based Virtual Environments for Children with Visual Disabilities Sánchez, J. & Baloian, N. Department of Computer Science, University of Chile Chile
[email protected],
[email protected] Abstract: Diverse studies have pointed out the relevance of model-based educational software. This idea has not been implemented fully when developing software for children with visual disabilities. We introduce AudioVida, a virtual environment for children with visual disabilities and describe how this software meets a formal model previously described by the authors. As a result, we identify some issues in the design of AudioVida and made some improvements. Children also evaluated the usability of AudioVida by finding technical and functioning problems and thus helping us to redesign and improve the software. As a result, we designed audio-based educational software and children were very motivated and enjoyed the experience of navigating and interacting with AudioVida to develop spatial navigation skills.
1. Introduction The literature agrees that educational software should be more accessible to people with visual disabilities. Most software relies heavily on graphical user interfaces impeding navigation by those without visual sensory channels. Even though the literature presents diverse products using haptic and sound interface devices they have not been extensively used in educational software for people with disabilities. A growing line of research in educational software for children with visual disabilities is using audio as a critical sensory channel to assist the construction of meaning and knowledge (Lumbreras & Sánchez, 1998, 1999; McCrindle & Symons, 2000; Mereu, S. & R. Kazman 1996; Savidis et al., 1996; Sánchez, 2001; Sánchez & Lumbreras, 1999, Sánchez et al., 2003; Sánchez & Flores 2004; Zajicek et al., 1998). A number of these studies have shown that when using audio-based applications blind children can develop and rehearse cognition. Most of these studies focus on the development of 3D audio interfaces to map the entire surrounding space through audiobased interfaces to develop cognitive processes such as tempo-spatial structures, memory, haptic perception, mathematics learning, and problem solving In addition to the need for more quality software for children with visual disability diverse developers have claimed the need for explicit model knowledge and skills in educational software. There is a growing interest to design models to construct educational software for these children in order to map the learners’ needs and interests by using audio and tact. Educational software for learners with visual disabilities should consider a model representing the interacting world, a model for representing the knowledge to be learned, and the learner's mode (Sánchez et al., 2004). Diverse authors propose methodologies to develop educational software (Baloian et al., 2002; Dillenbourg & Self, 1992; Kass, 1989; Sánchez, Baloian & Flores, 2004) by assisting developers in considering critical components for educational software design. Most of these methodologies consider modeling the knowledge to be learned and the skills to be developed when interacting with the software. This study builds upon these studies by presenting AudioVida, audio-based educational software for blind children developed by considering a model proposed in a former study (Sánchez, Baloian & Flores, 2004) that proposes a model to develop software for these children. We also include a usability study with end-users and analyze how AudioVida meets the proposed model by highlighting the relevance of using models to develop and evaluate educational software. Finally, we extended the model by proposing detailed guidelines to develop educational software for children with visual disabilities.
2. The Model We have previously stated that our model is based on existing methodologies we adapted for this particular case (Sánchez, Baloian & Flores, 2004). The proposed model uses a combination of incremental and evolutionary software development. We identify the stages of analysis, design, development, and validation. We have also
identified activities that consider with more accuracy the particularities of developing educational software for the blind. First, a cycle through the mentioned stages is executed to produce a prototype. Then the same cycle is executed as many times as necessary to produce a version which satisfy the requirements. During these iterations the result of previous stages are considered input for the next ones. The “workflow” of the processes is depicted in figure 1.
Figure 1: Schema of the development process Analysis: Consists of two sub-stages, A and B. The first stage is to define cognitive goals to be achieved by the learner. This corresponds to the definition of software requirements. The second stage is to define procedures and functions to evaluate the achievement of cognitive goals. Design: In this stage a metaphor will be defined for a “world” or scenario where the learner constructs knowledge through the interaction with this world. Normally, this is game type of software and “playing rules” are defined. This leads to define the model of the world and the knowledge to be constructed. Development: Consists of three sub-processes. The first process is the computational implementation of models of the world and learner. Here we recommend exploring the possibilities of developing editors for implementing different scenarios of the same “world”. The second stage is the implementation of the evaluation process and the feedback to the student. The third stage is the projection of the models. Here we identify input and output variables of models as well as parameters and results (including the feedback to the learner) of the evaluation function. These values have to be “projected” properly over the haptic, audio, and visual (for people with residual vision) input/output devices available. Here we verify a wide variety of input/output devices to avoid limiting to traditional devices such as joysticks and keywords for input and sound for output. Haptic devices such as tablets, electronic boards, and Phantom (http://www.escrs.org/eurotimes/April2003/newdevice.asp) can give blind users sensations of being “touching” virtual objects. It is important to do these actions after setting the models in order to ovoid limiting from the begging of software design. Some guidelines to do the projection are given below based on the literature and our own experience in developing software for blind children. Validation: Consists of two sub-processes. First, we develop usability tests to get data about how well the system fit our objectives in order to attain the cognitive goals set at the beginning. We emphasize the analysis of some elements of human-computer interaction. Second, we analyze these results and study how the metaphor, models, and the projection of input/output variables can be improved. Normally an error in the integrity of the system for learning can imply to review the metaphor and models used. Usability issues can lead to review the projection.
3. Making the model operational By applying the described methodology we obtain software with the architecture shown in the figure 2.
Figure 2: Architecture of the resulting software
Guidelines We define the attributes, characteristics, and criteria to be considered in software design for people with visual disabilities. What key elements should be included in educational software for blind children? What is the relevance of these elements? Do they have the same importance? According to Sánchez (2001) if we integrate different learning models to software design three types of educational software can be found: Software for presenting, representing, and constructing information and knowledge. The concept and characterization of them depend on the underlying theories of learning in each case. The model we propose in this paper is oriented to software for constructing learning. In addition, we can define some generic attributes of audio-based educational software for people with visual disabilities such as construction, navigation, interactivity, content, and interface. Construction consists of allowing users to make actions and construct by using available tools. Educational software for learners with visual disabilities can meet this attribute if appropriate cues are provided to construct a mental scheme in users to help them to adequately perform throughout the virtual environment. This attribute should be oriented to develop a concept of educational software in children with visual disabilities. Software focused on construction allows users to interact directly and freely with objects and entities, and take their own decisions about the course of actions. Users should “appropriate” the software through constructive actions. Navigation consists of giving options to users to move throughout the virtual space as a way of mentally construct the navigated space, ending up with navigation and orientation skills for real spaces. The virtual navigation is produced by programmed cues in functions of the keyboard as well as audio cues clearly defined. The navigation can be made through keyboard or any other input device that support an autonomous access. When using the keyboard as input mean the keystrokes used should be standardized in such a way that when new software is designed the interaction does not imply a new learning. Interactivity consists of allowing immediate feedback with the user. This helps cognitive interaction for understanding and meaning change. To allow an adequate interaction with children with visual disabilities the software should provide: audio cues, well defined images (for children with residual vision), and the use of keystrokes and commands previously standardized. Interactivity should enhance the initiative and self-learning by discovering and applying the content learned with the software in different contexts. Content refers to whenever a software is designed for people with visual disabilities the instruction should be audible. For children with low vision the interface should include clearly defined word messages with directions. Verbal directions should be clear, brief and precise because the user should remember them when interacting with the software. Users should have the option to re-listen verbal directions when needed. Interface consists mainly of designing fonts, images, colors, icons, buttons, and audio. Fonts should be sans serif, avoiding the use of roman or serif typography, unless uppercase letters are used. Decorative fonts should be avoided. Size should be 18 points and up with medium line spacing and background with contrasting colors. The best contrasting colors for people with residual vision are yellow/blue and black/white. Images should have good
contrast with backgrounds. They should be simple, clear, and precise in such a way that users can easily recognize them with their residual vision. Iconic images should be preferred instead of photographic ones. Strong and contrasting colors within blue and yellow spectrum colors should be used. These colors are better perceived by children with residual vision because they are diametrically opposed to both chrome and luminescence. Icons should be simple and big with clear messages and good contrast with backgrounds. They should be easily interpreted by users when using voice and sound in order to orientate them and understand the meaning of icons. Buttons should be big and simple with associated voices and sounds for children with visual disabilities to orientate and understand their function. Audio is critical in software for children with visual disabilities. It is the most relevant interface element in this type of software. Good quality audio should be included and to identify clearly how and when should be used for educational software purposes. Most interaction events should include audio to describe icons, buttons, menus, and contexts. Voices should be in accordance with the target user. Software should use interesting, motivating, and pleasant voices for children.
4. Introducing AudioVida AudioVida is a game-based virtual environment for children with visual disabilities to play and learn. The game occurs in a virtual labyrinth where each corridor is identified with an individual sound. The child has to walk through corridors to look for food that make him/her to have a balanced diet. This process is represented in scores and energy to be obtained in a determined time. If users consume foods that don’t fit a balanced diet, they “loose” lives. The main objective of AudioVida is to create diverse routes in a virtual environment within a complex audio labyrinth where blind children by analyzing shifting routes can recognize points of reference (foods), interpret the location and spatialization, recognize different shifting routes, and solve problems. In addition, the user has to apply basic knowledge about the characteristics of foods and the importance of a healthy and balanced diet by working concept such as fibers, carbohydrates, proteins, sugars, energy, calories, and cholesterol. AudioVida was implemented in C++ with Opengl libraries and DirectSound. To integrate 3D objects we used Deep Exploration to transform 3D object into Opengl/C++ code. AudioVida begins with an introduction to explain the software and motivate the learner to play the game. Then the initial interface presents the options menu (see Figure 2): the game description, controls that explain users how to interact with AudioVida, play to begin the game, change vista to play in first person immersed in the corridor (see Figure 3) or play from top of the labyrinth (see Figure 4), and finally to exit the game. Different game vistas can be exchanged during the game according to the user’s needs (see Figures 3 and 4). The user has to move through the labyrinth by looking for foods that can be found freely in corridors (see Figure 5) or hidden in trunks placed in closed corridors. The game includes a bottom menu with the state of the system containing indicators of the energy gained, lives left, score, and time elapsed. If the users don’t know the characteristics of the selected food they can choose the help option (see Figure 7) to get information about the characteristics of foods to make a better selection.
5. Usability evaluation Methodology. We selected a sample of three blind users ages 10, 12 and 15. We applied a usability test for endusers and observed the interaction of each one with AudioVida and recorded the learner’s behavior. The usability tests is composed of two parts: Part one has 18 statements such as: I like the software, the software is interesting and challenging, the software makes me to be active, I would like to play the game again, I recommend the software to other children, I have learned with this software, the software has different levels of difficulty, I felt controlling the software, the software is interactive, the software is easy to use, the software is motivating, the software can adapts to my rhythm, the software allow me to understand new things, I like the sound of the software, the sounds of the software can be identified clearly, the sound of the software conveys information, and screen images, color, and brightness conveys information. Each user assigned scores from 1 to 10 to each of statement.
Figure 3: AudioVida interface with the options menu
Figure 4: Playing immersed in the corridor
Figure 5: Playing from on top Part two consisted of five open questions: what do you like about the software? What you don’t like about the software? What you would like add to the software? What do you think the software is useful for you? What other uses can you give to the software?
Figure 6: A glass of water at the end of the corridor
Figure 7: A closed corridor hiding a trunk with foods
Figure 8: A view of the state of AudioVida and help on the screen
The testing procedure was to present the software, to leave the user to listen the directions and initial motivation of AudioVida, to check the optimal uses of keystrokes, and to leave them to interact freely with the software as many times as they wish. This interaction was observed and recorded by three special education teachers each with one learner. Once the session ended each user answered the usability test.
Results. During observation of users interacting with the software we detected some implementation issues such as to add different levels of complexity and to differentiate corridors already navigated by learners, technical problems such as sound localization and sounds that did not work when the user approach them, and functioning issues such as doors that could not close once opened. All of these were solved when we redesigned the software. Even though these design issues users were very motivated to play, comprehended well the main purposes of the game, and interacted easily with AudioVida. As a result of applying the usability test we can conclude that all indicators were well evaluated by users. The results varied between 7 and 10 average scores in a 1-10 scale. The indicators better evaluated were 1. Do you recommend this software other children? The software is easy to use, I like the sounds of the software, I can easily identify the sound of the software, I like the software, I would like to play again the software, the software can adapt to my rhythm, and the sound of the software conveys information (see Figure 9). Do you recommend this software other children? 10 9
The software is easy to use
8
I like the sounds of the software
7 6
I can easily identify the sound of the software
5
I like the software
4
I would like to play again the software
3 2
The software can adapt to my rhythm
1
The sound of the software conveys information
Figure 9. Statements better evaluated in the usability test
The statements with lower scores in the usability test were: The software is challenging, the software makes me to be more active, the software has different levels of difficulty, and the software is motivating (see Figure 10).
10 9
The software is challenging
8 7 6
The software makes me to be more active
5 4 3
The software has different levels of difficulty
2 1
The software is motivating
Figure 10. Statements with lower scores in the usability test
From the answers to open questions we can interpret that there was a high level of users’ satisfaction. They comprehended the purpose of the game and suggested valid improvements in terms of the type of software such as to include different levels of difficulty. Finally, when we analyze the data from end-users in the usability testing of AudioVida we can interpret that the most relevant results are: the easy to use, the motivation for end-users, and the audio interface is meaningful to users and assist them efficiently during interaction.
6. AudioVida, mapping the model In parallel to the usability evaluation we evaluated AudioVida according to our model. This means that we checked the characteristics of AudioVida against the description of the software according to the model. This was made by teachers of blind children. The results are shown in Table 1. Interestingly, the evaluation according to the model shows a lack of an intelligent learner’s model that can make the game more interesting and effective for learning. This problem was also detected in the usability test where children mentioned that the game was not very challenging. By taking these results we developed a new version of the AudioVida, but now using the procedures and guidelines of the model. After that we identified the components that according to the model any software of this type should have: Model of the real world: The metaphor in AudioVida corresponds to a labyrinth in which learners should find and eat different type of foods. They have to combine the food in order to get a daily balanced diet that gives them the higher score. The student is “coached” by the system in order to find and select the best selection. Editors: In order to present different scenarios to learners there are two editors. One editor is for editing food in such a way that the teacher can define new types of food that can appear in the labyrinth by giving them the nutrition facts, appearance, and the characteristic sound. The other editor enables the teacher to create new labyrinths and places food anywhere, by picking them up from a palette of a previously defined set with the first editor. Computer Model: This is the computer model of the labyrinth and foods designed by the teacher and represented by data structures and rules coded as instructions. The state variables of this model are: The position of learners inside the labyrinth, the position of objects, foods, and their nutritional balance. Student’s Model: How much the student has learned during the game is estimated by the computer according to the strategies (or the absence of them) to move inside the labyrinth and find the appropriate food. This information is used to give feedback about how learners are doing with the diet and to orientate them inside the labyrinth. Projection: Because of simplicity when interacting the projection of input variables is done over the keyboard and the output variables are mapped totally as audio information: sometimes as speech (such as giving feedback about the state of the game) and sometimes as sounds (when advancing, passing through important objects, turning, etc). Interface: The main input/output interfaces of AudioVida are Audio System: That provides feedback to learners during the interaction, Keyboard: That allows users to enter to their interaction options with the software, Screen: Specifically designed for learners with residual vision to perceive through strong contrasts an orientation element with the graphic information presented.
Indicators
Score 1=Low, 5=High 3,9 4,2
The metaphor is adequate for the learning method (construction) The model represents well the metaphor It is possible to define different interaction environments 3,0 (editors) The software represents somehow what learners have to learn 4,0 The software evaluates coherently what the learner have to learn 3,7 The software provides adequate feedback to the learner 3,0 There are input/output devices for interaction purposes 3,7 Users can know where they are 2,8 Users can know what to do in any moment 2,9 The font typography used is adequate 4,1 The size of the font is adequate 4,0 Colors and contrasts are used adequately 3,4 The design of buttons and icons are adequate 3,6 The interface generates adequate audio feedback for learners with 3,4 visual disabilities Table 1: Heuristic evaluation results of AudioVida
Heuristics
METAPHOR
LEARNING
INTERACTION
INTERFACE
7. Conclusions This study introduces AudioVida as a proof of concept of our modeling methodology proposed elsewhere. We have extended the model by proposing implementation guidelines and usability tested AudioVida to analyze the level of user’s acceptance of the software. We believe that educational software based on models can assure a better quality of the software. The model allows evaluating and detecting possible issues that can be improved such as in the case of AudioVida. We believe that educational software for children with visual disability can serve both to develop and to evaluate software. Usability evaluation is critical in software design because it can be very useful to detect problems such as levels of complexity, and some technical and functioning issues that are more easily identified by blind learners. The use of sound facilitated the interaction with the software. In addition, the fact that AudioVida was game-based motivated the interaction of blind children with the software. Our research and other previous studies lead us to identify at least three fundamental aspects in designing educational software for children with visual disabilities: 1. To be based in a model, 2. To be game-oriented, and 3. To consider a usability testing with end-users doing concrete tasks. Finally, the next step will be to evaluate the cognitive impact of AudioVida to determine the effect on developing and using skills for spatial cognition, route identification, and setting points of reference. Acknowledgements This report was funded by the Chilean National Fund of Science and Technology, Fondecyt, Project 1030158.
References Baloian, N., Luther, W. & Sánchez, J. (2002). Modeling educational software for people with disabilities: Theory and Practice. Proceedings of the Fifth International ACM SIGCAPH Conference on Assistive Technologies, ASSETS 2002, July 8-12, Edinburgh, 111-117. Dillenbourg, P., Self, J. (1992). A Framework for Learner Modeling. Interactive Learning Environment Vol. 2, Issue (2), 111137. Kass, R. (1989). Student Modeling in Intelligent Tutoring Systems –Implications for User Modeling. In User Models in Dialog Systems (A. Kobsa & W. Wahlster, eds.), 386-410. Lumbreras, M., Sánchez, J. (1999). Interactive 3D Sound Hyperstories for Blind Children. Proceedings of ACM CHI ‘99, Pittsburg PA, USA, 318-325. Lumbreras, M., Sánchez, J. (1998). 3D aural interactive hyperstories for blind children. International Journal of Virtual Reality 4 (1), 20-28. McCrindle, R. & Symons, D. (2000), Audio space invaders. In Proceedings of the Third International Conference on Disability, Virtual Reality and Associated Technologies, ICDVRAT 2000, Sardinia Italia, 23-25 September, 2000. pp. 59-65. Mereu, S., Kazman, R. (1996). Audio enhanced ·D interfaces for visually impaired users. Proc. ACM CHI 96, 72-78 Sánchez, J. (2001). Interactive virtual acoustic environments for blind children. Proceedings of ACM CHI ´2001, April 2-5, Seattle, Washington, 23-25. Sánchez, J. (2001). Visible Learning, Invisible Technology. Santiago: Dolmen Editions. Sánchez, J., Lumbreras, M. (1999). Virtual Environment Interaction through 3D Audio by Blind Children. Journal of Cyberpsychology and Behavior, CP&B 2(2), 101-111. Sánchez, J., Baloian, N., Hassler, T. Hoppe, U. (2003). AudioBattleship: Blind Learners Collaboration through Sound. Proceedings of ACM CHI 2003, April 5-10, Ft. Lauderdale, Florida, USA, 798-799. Sánchez, J., Baloian, N., Flores H. (2004). A methodology for developing audio-based interactive environments for learners with visual disabilities. World Conference on Educational Multimedia, Hypermedia & Telecommunications ED-MEDIA 2004. June 21-26, 2004. Lugano, Switzerland, 124. Sánchez, J., Baloian, N., Hassler, T., Hoppe, U. (2003). AudioBattleship: Blind Learners Collaboration Through Sound. Proceedings of ACM CHI 2003. Also published by SIGCHI publications, ACM Press. April 5-10, Fort Lauderdale, Florida, 798-799. Sánchez J., Flores, H. (2004). AudioMath: Blind Children Learning Mathematics through Audio. Proceedings of The 5th International Conference on Disability, Virtual Reality and Associated Technologies, ICDVRAT 2004, September 20-22, Oxford, United Kingdom, 183-189 Savidis, A., Sthphanidis, C., Korte, A., Crispie, K., Fellbaum, K. (1996). A generic direct-manipulation 3D-auditory environment for hierarchical navigation in non-visual interaccion. Proc. ACM ASSETS ‘96, 117-123 Zajicek M., Powell C., Reeves C. (1998). A Web Navigation Tool for the Blind, ASSETS’98, 3rd ACM/SIFCAPH Conf. On Assistive Technologies, Los Angeles, USA, 204-206
