Designing, Using and Evaluating a Serious Game for ...

Designing, Using and Evaluating a Serious Game for Balance Improvement on Elderly People Gabriel M. Rossito¹ Thiago L. Berlim¹ Sadi A. Peruzzo Junior¹ Marcelo da S. Hounsell¹ Antônio V. Soares² ¹Santa Catarina State University (UDESC), Computer Science Department (DCC), Brazil ² Associação Catarinense de Ensino (ACE), Núcleo de Pesquisas em Neuroreabilitação (NUPEN), Brazil

Figure 1: A player (left) using the game and a screenshot of the game interface (right).

Abstract

[email protected]

Balance difficulty is a condition that affects people as ageing happens and it raises the risk of falling. It is of major concern with elderly people because their overall health makes it difficult for them to recover from injuries that are caused by falling around. These injuries not so rarely leads to severe disabilities, long stay in hospitals and, even death. This paper presents a Serious Game (SG) that uses an interface based on a RGB-D camera to stimulate balance and reduce the risk of falling in elderly people. Existing gesture-based games targeted to this goal have not being developed to this specific population and do not provide adaptation for each player. The game called SIRTET-K3D has been inspired by a famous game but uses inverse visuals and game play. The game challenges the player to catch and/or dodge from objects. The way the challenges were composed gradually works the balance of the player. SIRTET-K3D has been assessed threefold: by game designers; by specialists and; by the target audience. The data gathered shows that the game fulfills game-designers´ and specialist’s expectations and, above all, that it does improve balance metrics. Game scoring was found to correlate to existing clinical assessments; it suggests that SIRTET-K3D can also be used as a replacement for them.

1. Introduction

Keywords: serious games, balance, elderly Authors’ contact: {gabrielmesquit,thiagoberlim,sadijrp}@gma il.com [email protected]

Over the years, the Brazilian population and its life expectation grew a lot. Only in the last decade the population over 65 years does not only accompanied this growth, but increased its composition in Brazil. In 2000, they represented 5.9% of the population, while in 2010 this value grew to 7.9% [IBGE 2010]. Frail elderly are among the over 65 population but are those of over 80 or more years of age. Being a natural process and accompanied by the evolution of medicine, the elderly population tend to grow in the future. In fact, health care and adaptations of the society are necessary, if not fundamental, since aging is also followed by a decline of several organs functions and functional and/or structural changes [Zanardini et al. 2007]. Even to daily tasks, health problems can occur and may affect one or several human functions such as walking itself and, a recurring factor in the aging process is the imbalance and adverse potentials that can lead to tipping and falling. Balance is a key factor and a complex process of the human body to maintain postural control. However, it should be considered that the balance given by the postural control is a skill upheld from the interaction of multiple sensory and motor processes [Horak 2006], these are: biomechanical constraints; drive strategies; sensory strategies; spatial orientation; dynamic control and; the cognitive process.

In recent years, games have raised interest from the scientific community due to its potential to be motivating and, at the same time, being able to deal with more serious issues than just entertainment. Serious Games (SG) then appeared and became a focus of research to a wide variety of applications ranging from education to health issues and behavior changes. The objective of this paper is to present a Serious Game called SIRTET-K3D and the results obtained with its use. This paper extends previous work [Rossito et al. 2014] by given extra details on design adjustment and clinical evaluation of the game. Furthermore, we intend to demonstrate and advocate the use of Serious Games in the scenario of motor rehabilitation and treatment of balance disorders, especially to the elderly. The paper is organized into sections, divided as follows: in Section 1 is given an introduction to the work; in Section 2 are presented and discussed related work; in Section 3 we present the SIRTET-K3D and the way it was designed; in Section 4 we present the tests throughout their development and use; in Section 5 are discussed some improvements and adjustments in the game; in Section 6 the conclusions of the work are presented.

2. Related Work 2.1 Development of an interactive game as a rehabilitation tool The focus of the work of Lange et al. [2010] is to use Kinect to create Serious Games for rehabilitation that is reliable and which may have the goal of improving aspects of health. Lange et al. [2011] used the Unity3D graphics engine to create a virtual character to treat the balance of people who have survived a stroke or brain injury. The prototype created was set in a mine, where the player is to be positioned within a mining cart and should collect gems along the way, as the cart automatically moves in the rails. Lange et al. [2011] showed that, with a sample of 20 participants, some problems were found like: difficulties in depth perception of the virtual environment; lack of instructions and; needed of a better theme. However, the game obtained a positive interest and enthusiasm of patients/players working their balance and playing simultaneously. There were no tests to assess the user and to confirm the balance improvement of patients, thus not confirming the effectiveness of the game. Although they created an interesting game that treats different patients using the Kinect, the system does not suggest physical measures or compensatory tracking on gametime and does not provide postural data and medical information specialists to analyze.

2.2 Balance improvement with commercial games With the use of games in interventions, Rendon et al. [2012] show that it is possible to increase strength, improve balance and reduce falls in elderly people. The study was based on the comparison of two groups: one using conventional therapies and another using WBB (Wii Balance Board) games in a population of 40 individuals chosen between 60 and 95 years, which were capable of performing physical activities for as long as 60 minutes. They were evaluated by TUGT (Time Up-and-Go Test; [Schoene et al. 2013]), ABC (Activities-specific Balance Confidence) and a test to identify depression in older people (Geriatric Depression Scale - DS). The results showed significant improvements in the group that used WBB compared with the control group. Among them as minor changes to the TUGT, a large variation in ABC and a small change in the GDS were perceived. However, all patients required at least some help to complete the entire series of exercises. 2.3 Individualization and independence in the use of Serious Games for balance Conducting a study with commercial games for the improvement in risk of falling, the group of Singh et al. [2012] obtained balance improvement on elderly using the WBB. Based on the study, 46 postmenopausal women over 56 years who could walk without any extra help were chosen. They were divided into two groups: normal exercises and; exercises using the WBB. Games used were: Balance Bubble, Ski Slalom, Table Tilt, Soccer Heading and Tight Rope Walk, chosen because they all use the anterior-posterior and medial-lateral center of gravity movements. The evaluation was conducted and analyzed by the PPA (Physiological Profile Approach) and the ABC-6 questionnaire; people participated in the trials for 6 weeks, twice a week. The results of the trials showed that both groups had significant improvements in the reliability of balance and decreased the risk of falling, confirming that games could be used in the process for the rehabilitation and balance improvement of postmenopausal women. It was noticed in this study the individual’s ability improves in due time, as each player could play at the time and how long thought best. The study opposes to Rendon et al. [2012], showing that the balance of the working player can improve reliability and, after a while, players were able to play independently. 2.4 Dual skill in balance using Kinect Committed to programs that help the improvement for fall prevention of elderly, Kayama et al. [2012] developed a system using the Kinect for motor and cognitive work sought to improve balance, decrease the risk of falling, track users' movements, avoiding cheating and evaluating physical measures.

The game consists of a mixture of Tai Chi Chuan and Sudoku. The player uses Tai Chi movements to solve Sudoku, selecting and placing the numbers correctly through predetermined poses. The time the player takes to solve the Sudoku is presented as evaluation at the end of the challenge. The game has been evaluated with the aid of 45 people over 60 years of age who had balance problems. All participants underwent cognitive, motor and dual-task trials. The authors claim that time is the differential variable, being the determiner of the evaluation of the test in question, assigning the note and the final evaluation. Based on the same game, Okamoto et al. [2012] evaluated user performance, comparing its use among younger and older people. With a group of 20 elderly patients between 65 and 80 years old and 16 young people between 20 and 23 years old. If was found that the game is able to effectively improve performance of dual tasks and the players showed improvement in FR and OLS tests, but there was no apparent improvement in TUGT and TS tests. Finally, it was found that patients were motivated to continue to use the game. 2.5 Comparing interventions in the treatment Another study that evaluated the use of video games for the rehabilitation of balance in the elderly was Pluchino et al. [2012]. They conducted a study based on 3 training programs and compared them. The programs were: basic exercises for balance; using Tai Chi and; a program using WBB games. 40 people, ranging in age from 65 to 80, were divided into groups for each workout, 14 for normal exercises, 14 for Tai Chi and 12 using the WBB. For the analysis, FES (Falls Efficacy Scale) questionnaire, TUGT, OLS (timed One-Leg-Standing), FR (Functional Reach) and POMA (Performance Oriented Mobility Assessment) tests were used. While the FES did not reveal any significant improvement, functional tests showed improvement for all workouts, i.e., this study showed that under the conditions applied, the three forms of treatment to improve balance are effective. 2.6 Discussion It can be seen that some work used COTS (Commercial Off the Shelf) games to treat balance disorders. The use of COTS games conflict with initiatives focused on games targeted to a specific population. For instance, Eldergames project [Gamberini et al. 2006] has established some guidelines for designing games for the elderly. These guidelines highlight the fact that COTS games are far from ideal for this crowd. Such games were not designed for this purpose but designed as a form of entertainment, which undoubtedly impacts the results of treatment. In addition, COTS game rely too much on

the capabilities of the specialist conducting the therapy using the game. The guidelines mentioned are targeted to cognitive games [Gamberini et al. 2006] but no to exergames which raises the need to develop more games to elderly in order to gain such information. Furthermore, despite the proximity of the work of Lange et al. [2011] and Kayama et al. [2012], all works presented have the same limitation: most games have the potential to generate useful information for health professionals but none of them do so. Analyzing the work presented, limitations were confirmed in studies using technologies like WBB which, when used, provides information only from the Center of Balance of the user, limiting its application. On the other hand, Kinect was more versatile, allowing individuals adjustments and gathering information derived from the user to improve the interaction with games. Such information obtained from this analysis determined the adoption of Kinect for SIRTET-K3D. It was also possible to identify that for the development of a Serious Game it is really necessary to have the partnership with subject matter specialists to identify the needs and requirements of the game targeting the needs of players and of themselves.

3. SIRTET-K3D SIRTET-K3D has been designed and developed following the guidelines of MOLDE, a MeasureOriented Level DEsign methodology [Farias et al. 2014]. Results from MOLDE helped design the game structure and is presented as follows. 3.1 Presentation SIRTET-K3D [Rossito et al. 2014] was inspired by Tetris. Its visual and gameplay were reversed (as its name). The game places the player in a tunnel and objects come from the bottom of the tunnel and goes toward the player. Some objects needs to be touched (the targets), others need to be avoided (the obstacles). But no particular game console is required (as it runs over a standard personal computer) and no device needs to be held (as a camera-driven gesture-based interface is used). This game requires a Kinect that captures the image of the person. Thus, K3D was added to the name to highlight the use of Kinect in a three dimensional workspace. For the development of SIRTET-K3D the following technologies were used: Unity3D, NITE and OpenNI. The OpenNI (Open Natural Interaction) is a framework for creating application of natural interaction. It enables communication between hardware and software, with focus on Kinect, so that it can also handle the information generated by the software and application. The OpenNI provides ready-

made classes and execution processes to use the Kinect in C++, giving quick access to many of its features. NITE is a middleware developed for OpenNI by PrimeSense. Accompanying OpenNI, it is a closed software, but available to developers free of charge. The algorithms offered by NITE, using the data acquired by the hardware, makes it possible to recognize the location and track the hands of the player; separate the player from the environment; identify the body and track it; recognize various body positions; among others. Unity3D is a graphics engine for creating games that work with 3D interactivity. It is famous for allowing the games created to be published on different platforms like iOS, Windows, Android and PS3 and has a free version for development. The Unity3D does not have direct interaction with the OpenNI and Kinect, therefore, when it is necessary to make them communicate, any module that serve as a bridge, for example, Zigfu, which has an open beta for developers, need to be used. Figure 2 depicts SIRTET-K3D interface and use where the player’s avatar can be seen touching a target object (presented in blue) where his arm becomes the same color as the objet at the time of the collision.

combination of them); the size of each object, and; its type (target or obstacle). Phases were set as: A=“Easy Targets”; B= “Easy Obstacles”; C = A + B; D = A + “Difficult Targets”; E = B + “Difficult Obstacles”; F = D + E. The programming variables that were used for Level controlling are presented in Table 1. Notice that only one difficulty variable changes from one level to the following. Levels

SPO

IBC

PST

1

SLOW

LONG

SHORT

2

SLOW

MEDIUM

SHORT

3

MEDIUM MEDIUM

SHORT

4

MEDIUM MEDIUM MEDIUM

5

FAST

MEDIUM MEDIUM

6

FAST

SHORT

MEDIUM

7

FAST

SHORT

LONG

Table 1: SIRTET-K3D: level transitions.

These variables include the speed of the objects (SPO - regardless if they are targets or obstacles), the interval between challenges (IBC) and the persistence time (PST - the bigger the persistence, the longer the player is required to sustain a specific position while touching the object). The former is divided into SLOW, MEDIUM and FAST and; the latter are divided into LONG, MEDIUM and, SHORT intervals. 3.4 Composing Challenges (Input Data)

Figure 2: SIRTET-K3D initial interface.

At the sides of the tunnel an accompanying shadow (the same color as the object) follows the approaching object to give users a better sense of depth. 3.2 Level Design Phases (or missions) were set manually as a loop (sequence) of “challenges” that are composed of targets (blue objects to be touched) and obstacles (red objects that one should dodge from). These cannot be created randomly because one should respect the capabilities of the audience in focus and, the sequence has to be incremental in terms of physical demand. To compose the challenges, a template is used that defines positioning for targets and obstacles. Physiotherapists can control each Phase: the number of challenges; the composition of each challenge (targets only, obstacles only or a

The composition of the challenges (obstacles and targets) is given through editable XML (eXtensible Markup Language) files. The first contains the parameters for each phase. The second XML file contains the targets and obstacles, its position in the tunnel and their size (width and height). The third XML file contains the parameters of the game settings, such as speed of objects and interval between the onset of each challenge, as well as the environment specific details. Some challenges were positioned up to 20% beyond player tip of the hand in both sides in order to create imbalance and postural adjustment. Figure 3 shows the template that indicates to the game how to generate an obstacle (at the top) at a specific position (at waist level close to the person) within the hand reach (see vertical dotted lines) and at lower altitude

(see horizontal dotted lines) that will require the player to deviate a bit.

This scale can be used to determine the imbalance state of the player: it may be a scale proportional to other clinical balance scales. For each phase there is a normalized score from 0 to 100, this is the sequence variation in the first phase, where the player will have a performance from 0 to 100, in the second it will have a variation from 100 to 200 and so on; thereby providing a direct comparison of the players who have achieved higher levels and thus obtain a higher score. The transition between phases is given from verifying the player's score. To advance from one stage to another, the player must achieve a minimum of 80% of the possible score for that phase. If, at the end of the phase, the player is below 25% of the total possible score, it will return to previous phase. 3.7 Reports (Output Data) At the end of each session, SIRTET-K3D generates a CSV (Comma Separated Values) file to monitor the evolution of the player containing relevant information for the analysis of an expert. Such information includes the session ID, date, duration; score the stage and the final level achieved by the player.

4. Evaluation SIRTET-K3D has been assessed three-fold: a) Test of Potential, just after implementation by a group of students well acquainted with computer games and interested in game design; Figure 3: Obstacle (top) and Target (bottom) challenges on the setup template.

At the bottom of Figure 3 there is a target object (see the green block at the right side) at the lower level but beyond the users´ reach (it is beyond the dotted line on the right). 3.5 Interface The collision with “target objects” occurs only with body extremities (hands, feet and head), which was done to avoid accidental collisions with the whole body and also to encourage correct movement. However, the collision between player and obstacles occurs over the whole body of the avatar. 3.6 Scoring The score was determined to be not only how many points the player acquired in the game, but also an absolute indicator of how he performed at that session. To do this, a scoring scale from 0 to 100, is assigned regardless of how many and what challenges were specified.

b) Formal test with the target population (elderly) to measure actual or potential benefit of the game towards the intended outcome (in this case, balance improvement); c) Informal test with the professionals that accompanied the use of SIRTET-K3D in a real setup of use regarding their views and the target population’s view of the game. 4.1 Test of Potential This test was performed to assess the potential of the game to fulfill the requirements to be a digital game. The test was inspired by the test used by Dowling et al. [2013] which analyzes, through a questionnaire, usability and ease of use of the game, which are important points considering that the target audience of the game are elderly. The questionnaire was applied to eight participants who heard a brief explanation about the game and their role in the evaluation, and watched a live demo of the game. These participants were computer science students with some culture in playing games and even, have had implemented a game themselves.

All questions have options that range from 1 to 5, e.g., from the lower level (nothing, too difficult, too weak) to a upper level (a lot, too easy, too strong) accordingly to the question. First, the characterization of the participants was taken, seeking their qualifications and experience in serious games. The first question was their education level and of eight participants, two were postgraduates, one was a graduated and five were undergraduates. The next questions were intended to measure the level of knowledge and experience about games and serious games itself. The sequence of questions can be seen below and graphics for better visualization of the results are following (Fig. 4): Q1: How much do you know about games? (playing, understanding advantages/disadvantages, studying, researching) Q2: What do you know about Serious Games? Q3: What is your knowledge level about game design? Q4: What is your knowledge level about serious game design?

Figure 5: Operational evaluation

There was another question in the operational evaluation, a discursive one, which was: Q8: Have you any suggestions on some kind of game that could help balance improvement using Kinect/Natural Interface? The participants described only two problems / suggestions. The first comment was a suggestion to maybe use a different mechanic, replacing the tetraminos by emerging silhouette from the end of the tunnel, with printed positions in them, such as holes, to facilitate understanding of the elderly. This suggestion was discarded because it resembles an existing COTS (Commercial Off-The-Shelf) game. The second comment was just an opinion that COTS games aimed for balance do not meet the requirements that SIRTETK3D may be capable of meeting. The following part of this Test of Potential was to seek the opinion of each participant about the game from the target population´s perspective. The sequence of questions can be seen bellow and the graphics for better visualization can be seen in Figure 6: Q9: In your opinion, how much fun the elderly will have using SIRTET-K3D?

Figure 4: Characterization of participants

Afterwards, an assessment on how the target population would take the game from the designer’s point of view was asked. The sequence of questions as well as a graphic (Fig. 5) for better visualization can be seen bellow:

Q10: In your opinion, how much attractive the graphics of the game will be to the elderly?

Q5: In your opinion, how much will be the difficulty of the elderly in using the Kinect? Q6: In your opinion, how much will be the difficulty of the elderly in understanding the instructions of the SIRTET-K3D? Q7: In your opinion, how much will be the difficulty of the elderly in making the movements/challenges (reach and avoid) presented by SIRTET-K3D?

Figure 6: Individual evaluation.

Q11: In your opinion, how difficult the elderly will feel about the SIRTET-K3D?

Q12: In your opinion, how much the sound given by SIRTET-K3D will assist the elderly?

on TUGT was achieved on functional aspects directly related to balance.

Q13: In your opinion, how much visual feedback will help using SIRTET-K3D?

TUGT pre (secs)

TUGT pos (secs)

FR pre (cm)

FR pos (cm)

Minimum

12,1

8

13

19,5

Maximum

23

16,4

23

31

Average Standard Deviation

15,2

10,5

17,7

26,3

4,05

3,2

3,7

4,4

Finally, an open question was presented: Q14: Have you any suggestions on some kind of game or something that could help balance using Kinect/Natural Interface? Six of eight participants suggested changes and/or reported problems in the game. The most recurrent problem was the difficulty in understanding the distances between the avatar towards the objects, the sense of depth and the environment itself. Another recurring problem was the lack of a "demo" phase to introduce concepts and help the player to get used to or learn more about the game and, finally, on the disposal of data on the interface, suggesting bigger letters. After this assessment, a lightning effect on the objects as they come closer to the player was included and the camera parameters were changed to exaggerate a bit the perspective effect. These changes were thought of in order to help improve the sense of depth. 4.2 Tests with the target population A formal pilot study has been conducted with six elderly individuals (n=6) of 81,2 year of age on average (+/- 7,9), 3 male and 3 female, that used SIRTET-K3D twice a week for 12 weeks in a session that lasted from 15 to 45 minutes standing in front of the camera, according to the elderly capabilities (Study approved by the Ethics in Research Board of the Centro Educacional Luterano Bom Jesus/IELUSC under the number 393.274). Well known clinical tests were selected such as TUGT – Timed Up and Go Test [Schoene et al. 2013] and FR – Functional Reach [Soares 2009]. TUGT assesses functional mobility (dynamic balance) by measuring the time elapsed for the individual to stand from a sit position, walk 3 meters turn around, walk back and sit. The time taken allows to grade the risk of falling as severe, moderate and low [Soares et al. 2003] and therefore is considered a predictor for risk of falling specially for frail elderly. FR is a static balance kind of test that assesses postural changes. The test requires an individual that is comfortably standing upright and is close to a wall to raise his arm pointing forward (parallel to the wall where a scale is set) to try to reach further as far as possible without moving his feet. The difference between the comfortable position to the farther one is measured [Figueiredo et al. 2007]. Table 2 shows the results of testing the population before and after using the game. As can be seen from Table 2, an improvement of 48,6% on FR and 30,9%

p-value * p-value