Integrating digital games based learning environments with eye gaze ...

2011 International Conference on Pattern Analysis and Intelligent Robotics 28-29 June 2011, Putrajaya, Malaysia

Integrating Digital Games Based Learning Environments with Eye Gaze-based Interaction Nurul Hidayah Binti Mat Zain1, Azizah Jaafar, Faculty

of Computer and Mathematical Sciences (FSKM) , Universiti Teknologi MARA (UiTM), 40450 Shah Alam, Selangor,MALAYSIA

Faculty

of Information Science and Technology (FTSM) , Universiti Kebangsaan Malaysia (UKM) 436000 Bangi, Selangor, MALAYSIA 1

[email protected] 2 aj@[email protected]

Abstract— Game designer needs accurate and reliable interaction design model to design and develop a Digital Gamebased Learning (DGBL) environment especially when the use of the user’s eyes as active game controls as well as input device. This is especially true when we discuss about implement a new technology in learning process to enable disabled person specifically in motor impairment using eye gaze interaction to play game based learning application. This concept paper review past work on eye gaze gaming and discusses the problems faced by disabled person when using eye gaze to control game. Finally, this paper proposed exploratory method of intergrate an eye gaze interaction into DGBL environments. Keywords— digital game-based learning, eye tracking, eye gaze, gaze based interface

I. INTRODUCTION AND MOTIVATION FOR RESEARCH Digital Game-based Learning (DGBL) or educational game is games that have been specifically designed to teach people about certain subject, expand concepts, reinforce development, understand an historical event or culture, or assist them in learning a skill as they play [1]. Nowadays, DGBL is one of the few ways to meet the need of the information age as well as those of today’s learners raised with digital media. DBGL has the potential to provide effective and powerful learning environments in which the learners can develop the essential skills for the information age, including critical thinking and problem solving skill as well as acquiring domain expertise [2, 3, and 4]. Furthermore, DGBL are becoming popular both within the able-bodied and disabled users. These games assume the use of keyboard and mouse as standard input device, which in some cased is not appropriate for user with disability. Since users with disabled often retain good control of their eye muscles when fine motor control of other muscle groups is lost, it is obvious that users should utilize very effectively eye gaze for interacting with computers. Although, most of the existing work on gaze based interaction for disabled users focuses on 2D desktop applications and text entry [5, 6, and 7].

In general, eye gaze is an interaction simply by looking at the screen which is eye movement replaces mouse movement. Unfortunately, game designer especially in Malaysia, does not have such comprehensive interaction design model to be refer in intergrating eye gaze interaction into Digital GameBased Learning (DGBL) Environments, although the lab facilities for eye tracking system is available in Malaysia’s higher institution such as in Universiti Kebangsaan Malaysia (UKM). An interaction design model that close to the real system as possible; at the same time, it should not be too difficult or complicated to use for analyzing the behaviour of the system is required. II. BACKGROUND STUDY A. The Eye Gaze Interaction There is not yet a benchmark for the best way to interact with eye gaze interfaces. Previous studies propose different techniques for using the gaze for interaction. The eye gaze interfaces can on basis of how interaction is made with them be divided in two main categories, gaze based and gaze added interfaces [8]. 1) Gaze based interface: Only the user’s eyes control the GUI and no extra input device is used. Interaction with an eye gaze interface can be made in a command or non-command based way. In the command based interface the gaze is used for object selection in the same way like the mouse. In the non-command based interface the user’s gaze is registered and analyzed to find out where the user’s attention is. The eye movements used to operate the interface can be “natural” or “unnatural”. 2) Gaze added interface: In a gaze added interface, the eye tracker is combined with others input devices like mouse used as a complement to manual input devices like mouse or keyboard. B. The Characteristics of Eye Gaze- Based Interaction There are many ways to design methods for using eye movements as input in a human-computer dialogue. Special interfaces and applications have been developed where interaction is made only or partly with the user’s gaze. Clearly there are both benefits and some problems with eye tracking,

978-1-61284-406-0/11/$26.00 ©2011 IEEE

MU-09



depending on the interface used. A summary of some general characteristics of eye based interaction is given in Table 1. TABLE I CHARACTERISTICS OF EYE GAZE-BASED INTERACTION (SOURCE: [8])

Characteristics Fast

Easy

Reveal attention

Midas touch

Always “on”

Contact free

Explanation The eyes move very fast. If the user wants to operate an object in the interface, he looks at it and then uses an input device. Hence, the eyes will provide input faster than any other input device. Previous studies suggest that interacting with the eyes can be intuitive, natural and easy to learn. By calculating the user’s point of gaze, the computer could reveal where his attention is. The eyes lack equivalence to the mouse buttons and hence there has to be another way to confirm actions. Dwell time, winks or a gaze-button can be used. The eyes are always “on”, the input is continuous unlike manual input devices that are active when the user chooses. Point of gaze can be detected without physical contact. The user can interact with a computer even if he can’t use his hands. It could be that they are occupied or due to disability. If both hands are used for input, the use of eyes can offer a “third hand”. If eyes are used as input instead of a manual input device, arm- and hand-movements will be diminished. This can reduce fatigue and prevents injuries like Repetitive Strain Injuries (RSI).

The major issue in this related study is eye tracking calibrations become poor resulting in users not being able to select objects easily [10]. The problem was undertaken by designing levels with careful attention to camera positions, size and the locations of selectable objects. Second issue is users would look towards a new target while attempting to apply a vocal command on a previously selected object. The strategy to overcome this problem by implementing a time lag in selecting items, which is provide the user with enough time to voice any command. Wilcox, et al. [10] describe the main advantage when combining eye tracking and voice recognition is that they have achieved a completely hands free input system. Voice introduces redundancy into the system where eye control might fail to provide the required accuracy. Finally, this type of input system should increase the appeal of games as well as increasing accessibility for disable person. They recommended when designing a point-and click style game using eye tracker, aesthetics becomes ergonomics as scenes and selectable elements must be laid out and scaled for optimal usage; buttons must be bigger and targets must be carefully positioned in order to produce an effective system.

B. Rabbit Run Rabbit Run is a first person maze game which was created to compare gaze and voice input with traditional techniques, such as mouse and keyboard [11]. Extra input O’Donovan, et. al [11] explores if gaze and voice controlled game interaction is a possible alternative to mouse Reduce fatigue and keyboard. The game needed to be relatively simple so users could understand it quickly and finish within a reasonable time frame. It also needed to include common gaming tasks, such as navigation and object selection. The main objective this game is to navigate through the warren maze and find the exit in the shortest time possible. To earn III. GAZE IN GAMING RELATED STUDY extra points coins distributed throughout the maze could be Recent innovations in the video game industry include collected. alternative input modalities for games to provide an enhanced The measurement has been made based on (1) and more immersive user experience. Gaze control has Performance (2) Accuracy and Control (3) Game Navigation recently been explored as an input modality in games [9]. and Difficulty, (4) Game Naturalness and Enjoyment and (5) From previous studies few article related with eye gaze based Game Map Usefulness. interaction for educational game were identified. There are as The major advantages in their study are the game could be follows: played completely hands-free and present opportunity for disabled users for whom traditional interaction techniques are A. The Revenge of the Killer Penguins not sufficient. The gaze/voice option was also selected by The Revenge of the Killer Penguins is a third person most participants as the more immersive form of interaction in adventure puzzle game using a combination of eye tracking the user evaluation. technology and voice recognition for novel game features [10]. The main problem encountered in the implementation was Wilcox, et al. [10] reported about gaze and voice are the collision response. The finding reported gaze/voice would exploited in game features. The first feature is SmartText, have surely done better when collision issues been resolved. which is dialogue and text are displayed on the screen until it Furthermore, researcher recommend look at the effects of has been detected that the player has read it. Other features background noise on voice recognition in games and include detecting voluntary blinks and winks to be used as investigate how the animation and the game characters could controls. There are two different modes of control in the main be adapted to react to the player’s gaze and voice. More game. (1) User selects objects by looking at them and (2) user socially realistic scenarios could be created if game characters commands by vocalizing the particular words. Otherwise, they responded in ways appropriate to the tone of the player’s can perform each command by blinking and winking at voice and/or the focus of their gaze. objects.



C. Gaze Controlled Games A report that reviews past work on eye tracker gaming and future development possibilities in different sub domains within was done by Isokoski. Table 2 summarized issues that identified in a few game genres [9]. Isokoski, et al. [9] agree that people with disabilities which interfere with manual control in computer gaming, eye control offers an opportunity to play many games. Turn-based games and puzzle games can usually be adapted for eye tracker use. Adapting games that require constant and fine control is more difficult, and even when possible the adaptations tend to require access to the source code of the game. TABLE 2 ISSUES IN A FEW GAME GENRES (SOURCE: [9]) Game Genres Computerized board games and puzzles

Issues All user interface elements must a big enough for easy selection with a pointer controlled with an eye tracker. Games may also be programmed using platforms which read pointer events directly from the device, making it difficult for eye tracking software to feed mouse events into the game such as games that are based on the DirectX library and programming interface by Microsoft.

Shoot-em-up (scrolling shooters)

Eye control in these games case is challenging, since shoot-em-up games require constant or at least frequent control of position. They often also require a dissociation of attention to the target, and position control of the shooting device. The player needs to look at a target, but simultaneously stay out of the line of fire if the target defends itself.

First person shooters

Acquiring and following targets with eyebased aiming is easy. Due to the inaccuracy in fast eye movements, tracker inaccuracies, and tracker noise, hitting small targets efficiently is difficult. It may be possible to add some aiming assistance which could be used when eye tracker FPS performance needs to be competitive with players using a mouse. In one player situations such systems may not be necessary.

3rd person action and adventure games

Controlling the game character remains complicated and it is difficult to map all the controlled degrees of freedom to eye movements. Using eye trackers to control these games is possible in principle, but most games do not work without modifications. The issues may include incompatibility with the eye trackers mouse emulation, too small targets, need for text input, numerous keyboard controls, and action sequences that require fast and precise pointing with a mouse.

D. Game Play Experience in a Gaze Interaction Game Nacke, et. al [12] provide an impressive exploration key points of game experience in a gaze interaction game. In their study, the results of the game experience showed a very positive game experience in gaze interaction game. Most particularly positive affect, immersion and flow scored high for the gaze input game. Challenge and competence scored high compared to prior studies [13]. They discover as expected, negative affect and tension dimensions scored low, indicating a very pleasant game experience. Nacke, et. al [12] reported that gaze interaction games provide a positive game experience, where the challenge of controlling the game by gaze and keyboard results in positive affection and feelings of flow and immersion. E. Use Of Eye Movements For Video Game Control Smith, J., & Graham [14] compared the use of an eye tracker to a desktop mouse for game control and chose three different games spread across three different genres, utilizing the input modalities in different ways: 1. Quake 2: A first person shooter where the user controls orientation with either the mouse or the eyes. 2. Neverwinter Nights: A roleplaying game in which an avatar is moved through an interactive environment through pointing. 3. Lunar Command: An action/arcade game in which moving objects are targeted through pointing.



They classified result into two types: • Performance measures: These measures describe how well users performed when playing the games. These measures are specific to each game. • Subjective measures: These measures are intrinsically comparative and are designed to capture the participants’ opinions of the input modalities. Subjects were asked to indicate a preference across 6 different criteria. These measures are not game specific, although they were collected separately for each game. They indicate that the eye tracker can increase the amount of immersion experienced when playing a video game. Additionally, they found that user discover the eye tracker more enjoyable to use when playing a game that required the user to direct an avatar around a screen by pointing. IV. ACCESSIBILITY ISSUES IN DIGITAL GAME-BASED LEARNING (DGBL) Computer games have become an essential part of social and cultural environment [15], and have been proposed as a potential learning tool by others educational researchers [16, 17] and game developers [18, 19]. The motivation of games could be combined with curricular contents and are called as a ‘Digital Game-Based Learning’ (DGBL) [20]. Games that include educational objectives and subject matter are believed to hold the potential to render learning of academic subjects more learner-centred, easier, more enjoyable, more interesting, and, thus, more effective [21, 22, 23). Specifically, games provide potentially powerful learning environments for a number of reasons [15]: • they can support multi-sensory, active, experiential, problem-based learning • they favour activation of prior knowledge given that players must use previously learned information in order to advance • they provide immediate feedback enabling players to test hypotheses and learn from their actions • they encompass opportunities for self-assessment through the mechanisms of scoring and reaching different levels, and • they increasingly become social environments involving communities of players. Apart from knowledge acquisition, game playing can also favour the development of various skills, such as critical thinking and problem-solving skills [24]. Since 1998, many organizations became familiar with the need to make their websites accessible to the disabled. Section 508 originated in the Rehabilitation Act of 1973, but was later strengthened by the US Congress in the Workforce Investment Act of 1998. For the first time, accessibility gets national attention in United Stated. Section 508 is associated with the requirement that Federal websites be accessible to the disabled. However Section 508 also requires access to electronic and information technology procured by Federal agencies [25]. In response to Section 508, the W3C (World Wide Web Consortium) developed standards for accessible

web sites and related technologies [26] such as games application. Disabled gamers are consumers, and access to gaming is a quality of life issue. Gamers play games for entertainment, not to experience a sense of frustration. Unfortunately, once players are not likely to be entertained, it’s more possible that they are angry or confused. Table 2 presents some common problems disabled gamers may encounter in current games: TABLE 2 COMMON PROBLEMS FOR DISABLED GAMERS (SOURCE: [27])

Problem Inability to follow a storyline

Reason • No subtext available, story is advanced by cut scenes. (Auditory) • Story is very complex and difficult to follow. (Cognitive)

Unable to complete a puzzle or task

• Vital clues given in cut scenes with no text available. (Auditory) • All clues are given as text. (Visual) • Requires precise timing with controller. (Mobility) • Requires the ability to position a cursor accurately (Mobility)

Unable to determine how game is played

• Lack of a tutorial mode • Poor documentation • Documentation written at too high a level for intended audience • Game only supports limited set of devices

Inability to use adaptive hardware Player’s character gets killed/injured repeatedly in game

• Not recognizing audio clues. (Auditory) • No indication of dangerous situation • Inability to respond quickly with controller (Mobility). • Unable to alter game speed. (Mobility)

Some issues in Table 2 demonstrate commonalities between able-bodied and disabled while other issues are specific to gamers with certain disabilities. Accessibility is a quality of life issue, since equal access is not being provided and in some cases, this could be a legal issue, particularly in places where full accessibility to services is required by law [9]. As games move from entertainment value to educational value, public schools need games to be accessible to all students. Despite of the existence of a disability, accessibility benefits all players through the reduction in frustration levels. The success of this exploratory research is promising and such success will contribute significantly in produce accurate, reliable interaction design model that will use to integrate eye gaze interaction into DGBL environments. Such model can also help in design and develop accessible DGBL application that beneficial to people with disabilities or users requiring both hands for their task.



V. PROPOSED METHODS OF INTEGRATING DGBL WITH EYE GAZE INTERACTION DESIGN FOR MOTOR IMPAIRMENT USERS A. Phase 1: Research Identification and Establishing Requirements for the User Experience This study start with identify needs and establish requirements of users. It begins with identify target user and understanding the problem of the research by making preliminary study to the target users. Once the problem has been identified, the planning to tackle the problem must be carried out. The planning will be performing by defining objectives, scope and significance of the study. The literature review undertaken to identify the technologies provided by the eye tracking system because the primary objective of this study is to integrate eye gaze interaction into DGBL environments. The significance of the research questions are concluded based on information gathering on the following study domains: • State of Art Technology • Eye Tracking • Eye Gaze Interaction • Digital Game Based Learning B. Phase 3: Design a DGBL - Eye Gaze-based Interaction Model Based on the information gathering and the theoretical background, we proposed a DGBL - Eye Gaze-based interaction designs model that meet user’s requirements. This phase can be categorized by two activities: conceptual design and physical design. Conceptual design involves producing the conceptual model for the product and describes what the product should do, what it should look like and how it should behave [28]. Physical design considers the details of the product.

which is an essential part of user centred design processes. It is necessary to evaluate prototypes. If designers made wrong assumptions or missed requirements, a usability test is likely to reveal them. E. Phase 6: Evaluate a DGBL prototype that use an eye gaze interaction on the target user At this phase, we will evaluate DGBL prototype that use an eye gaze interaction on the target user to determine the usability and acceptability of the product or design that is measured in terms of a variety of criteria. VI. CONCLUSION An eye gaze interaction still new technology in Malaysia. It also faces the added challenge of requiring fast real-time interaction if participation by disabled users in a game is to be on an equal balance as other able-bodied participants. This paper provided with some related study using gaze in gaming and proposed exploratory method of intergrate DGBL environments with eye gaze interaction design. In addition, those methods were identified that can be used in design and develop a good user interface for an eye gaze educational game. VII. FUTURE WORK A methods will be applied to eye gaze educational game study and will be validated using usability testing. A group of disable participant will be selected to implement eye as an input device using SensoMotoric Instrumesnts (SMI) RED remote eye tracking. The analysis data collected and a new interface design structure for educational game is proposed. These are left for the future work. REFERENCES [1]

[2]

C. Phase 4: Develop DGBL prototype based on DGBL-Eye Gaze Interaction Design Model At this phase, we will develop DGBL prototype with eye gaze as an input device for people able-bodied or disabled using conceptual design and physical design. D. Phase 5: Integrate an eye gaze interaction into Digital Game-based Learning (DGBL) prototype In this phase, we will integrate an eye gaze interaction into DGBL prototype. The apparatus for eye gaze interaction and software for analysis user’s behaviour are: • SensoMotoric Instrumesnts (SMI) RED remote eye tracking which is high-resolution sensor allows the subject free head movements across a wide range, while the powerful software automatically locates the eyes and compensates motion. Non-moving optics ensure optimal robustness during stationary and mobile use. • The Observer XT is the professional and userfriendly event logging software for the collection, analysis, and presentation of observational data. The Observer XT normally used in area usability testing

[3] [4] [5] [6]

[7]

[8]

[9] [10] [11] [12] [13]

Jaspaljeet, S. L. L. W., Mohana Shanmugam,Saraswathy Shamini Gunasekaran, Siva Kumar Dorairaj. (2008). Designing Computer Games to Introduce Programming to Children. Paper presented at the Information Technology and Multimedia at UNITEN (ICIMU’ 2008). M. Prensky, "Digital game-based learning," Computers in Entertainment (CIE), vol. 1, p. 21, 2003. D. W. Shaffer, et al., "Video games and the future of learning," Phi delta kappan, vol. 87, p. 104, 2005. K. Squire, Game-based learning: Present and future state of the field: Masie Center e-Learning Consortium, 2005. C. Lankford, "Effective eye-gaze input into Windows," In Proceeding of the ETRA Symposium (pp. 23-27). New York, NY, USA, 2000. P. Majaranta and K. J. Räihä, "Twenty years of eye typing: systems and design issues,” In Proceeding of the ETRA (pp. 15-22). New Orleans, LA, USA, 2002. A. J. Hornof and A. Cavender, "EyeDraw: enabling children with severe motor impairments to draw with their eyes,"In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 161-17). Portland Oregan, USA, 2005. E. Jönsson, "If looks could kill–an evaluation of eye tracking in computer games," Unpublished Master's Thesis, Royal Institute of Technology (KTH), Stockholm, Sweden, 2005. P. Isokoski, et al., "Gaze controlled games," Universal Access in the Information Society, vol. 8, pp. 323-337, 2009. T. Wilcox, et al., "Gaze and voice based game interaction: the revenge of the killer penguins," 2008, p. 1. J. O'Donovan, et al., "Rabbit Run: Gaze and Voice Based Game Interaction," 2009. L. Nacke, et al., "Gameplay experience in a gaze interaction game," 2009, pp. 49-54. M. Grimshaw, et al., "Sound and immersion in the first-person shooter: mixed measurement of the player's sonic experience," 2008.



[14]

[15] [16]

[17] [18] [19]

[20] [21]

[22]

[23] [24]

[25] [26]

[27] [28]

Smith, J.D., Vertegaal, R., Sohn, C. "ViewPointer: Lightweight CalibrationFree Eye Tracking for Ubiquitous HandsFree Deixis". In ACM User Interface Software and Technology 2005. Seattle, WA: ACM Press, 2005. D. G. Oblinger, "The next generation of educational engagement," Journal of Interactive Media in Education, vol. 2004, 2004. T. W. Malone, What makes things fun to learn? A study of intrinsically motivating computer games. Palo Alto, CA: Xerox Palolto Research Center, 1980. J. P. Gee, What video games have to teach us about learning an literacy. New York: Palgrave Macmillan, 2003. M. Prensky, Digital game-based learning. New York: McGraw-Hill., 2001. D. Gibson, C. Aldrich, and M. Prensky, Games and simulations in online learning: research and development frameworks: Information Science Publishing, 2007. M. Prensky, "Digital game-based learning," Computers in Entertainment (CIE), vol. 1, p. 21, 2003. Y. B. Kafai, "The educational potential of electronic games: From games-to-teach to games-to-learn," Cultural Policy Center, University of Chicago. http://culturalpolicy.uchicago.edu/events/conference-2001video-games.shtml [Last access: 12/03/2011]. Malone, T., “What makes things fun to learn? Heuristics for designing instructional computer games” In Proceedings of the 3rd ACM SIGSMALL Symposium and the 1st SIGPC Symposium (pp. 162–169). Palo Alto, USA, 1980 M. Prensky, "The digital game-based learning revolution," Digital Game-Based Learning, New York: McGraw-Hill.2001. A. McFarlane, et al., "Report on the educational use of games: An exploration by TEEM of the contribution which games can make to the education process," 2002. http://www.teem.org.uk/publications/teem_gamesined_full.pdf [Last access: 12/03/2011]. L. McLawhorn, "Leveling the accessibility playing field: Section 508 of the Rehabilitation Act," NCJL & Tech., vol. 3, p. 63, 2001. J. Brewer, "Web accessibility highlights and trends," In Proceedings of international cross-disciplinary workshop on Web accessibility (W4A). (pp. 51-55.). New York, NY, USA, 2004. K. Bierre, et al., "Game not over: Accessibility issues in video games," In Proceeding of the HCI (pp. 22–27). H. Sharp, et al., Interaction design: beyond human-computer interaction: Wiley, 2007.