Creating an emotionally reactive computer game responding to affective cues in speech
Christian Martyn Jones School of Mathematical and Computer Sciences Heriot-Watt University, Edinburgh EH14 4AS, UK
[email protected]
James Sutherland School of Mathematical and Computer Sciences Heriot-Watt University, Edinburgh EH14 4AS, UK
[email protected]
Computer games are becoming visually realistic, and coupled with more violent storylines and characters there is concern among parents and parts of the media that violent games could encourage aggressive behaviour in those who play them. To stimulate other emotions in game players we have developed an emotionally responsive computer game. Based on a traditionally aggressive role-playing environment, we have removed the violent elements such as shooting and fighting and replaced them with a goal to navigate obstacles using both positive and negative emotions. Affective cues in the speech of the player are analysed automatically using an emotion recognition system and these then affect the physical and behavioural attributes of the gaming character. The character can be better able to overcome obstacles depending on the emotional state of the player. Emotional interface, Affective computing, Emotion recognition, Computer games.
1. INTRODUCTION Violence in computer games has been criticised by parents and lobby groups who believe that violent games can invoke violent behaviour. Rockstar North’s Grand Theft Auto series has been blamed in the US for youngsters shooting at passing cars. Activision’s Doom linked to the Columbine high school shootings. The UK prosecution of 17 year old Warren Leblanc for the murder of 14 year old Stefan Pakerrah in February 2004 reinvigorated the debate of possible association between violent behaviour and violence in computer games. The depiction of violence in games has become more extreme over recent years. Today all games developers can create highly realistic and believable visuals using lighting and sound effects seen in the film industry. It can appear that developers are turning to more violent scenarios and gameplay in order to differentiate their game from others. However there are more ways to engage with game players than simply more extreme violence, and there are many more emotions that games could stimulate rather than feelings of anger, rage and aggression [1]. This paper reports on a feasibility study to automatically recognising emotions (other than anger) using affective cues in the speech of game players. Characters in the game can then respond to the player’s emotional state. 2. EMOTIONALLY RESPONSIVE COMPUTER GAME The new emotionally responsive computer game is built on the traditional first person shooter game Half-Life developed by Sierra Entertainment for the PC. Half-life places the player in the role of Gordon Freeman a research associate in a decommissioned missile base. Here the player must fight off a range of inhuman monsters and government assassins in the attempt to escape the base. The games contains the common elements of a shooter game including a range of weapons, violent creatures which must be killed and levels which have to be navigated. Modifying the Half-life game using software development kits we have been able to eliminate the need for aggression (to fight and kill) and replace this with other human emotions such as happiness and sadness. We have developed a new world for Gordon Freeman where the player must navigate levels using not only mouse and keyboard controls but also emotional feelings. These emotional feelings are recognised from affective cues in the speech [2] of the player using our acoustic emotion recognition system [3] and which similarly alter the behavioural characteristics of Gordon Freeman in the game. For example, if the game player is upbeat (happy) then Dr. Freeman’s is larger and his behaviour in the game more energetic enabling him to jump over ravines, Figure 1. Whereas if the game player is downbeat (bored or sad) Dr. Freeman becomes small, slower and lethargic
HCI 2005 – PROCEEDINGS VOLUME 2
1
CREATING AN EMOTIONALLY REACTIVE COMPUTER GAME RESPONDING TO AFFECTIVE CUES IN SPEECH
facilitating walking along narrow pipes or navigating mine fields, Figure 2. The game player talks to Dr. Freeman as if guiding him through the obstacles. The player’s emotion, determined by the recognition system, modifies Dr. Freeman’s behaviour in the game which then affects the ease at which the game player can complete the level.
FIGURE 1: Broken bridge game obstacle
FIGURE 2: Minefield game obstacle
5. EVALUATION OF THE EMOTIVE COMPUTER GAME The emotive computer game was evaluated using ten computer game players aged 16 to 24 and experienced with first-person shooter games. Players were given a short demonstration of the association between their emotion and the physical and behavioural attributes of the game character. Each player then commenced the level in a jail cell and had to navigate the character to freedom. The participants were timed to reach the end goal, recording when and where the player invoked changes in the emotional behaviour, any occasions when the player felt the game had not correctly recognised their emotion, and counting the number of times the character perished. Death of the game character results in a 1 second pause and then reinstates the character at the end of the last obstacle. The results show that the system is capable of recognising emotion. On average the number of errors in the detection of player emotion per game was four with a correct decision accuracy of 80%. Occasions when the player had to repeat speech due to erroneous recognition appeared not to frustrate the player or hinder the gameplay, however users suggest that it would be unacceptable for frequent errors when playing a full length computer game. The mappings of emotion to character behaviour and physical attributes were learned quickly and were easily remembered by the gamers. However there was some concern regarding potential repetition in the gameplay if the player has only two emotional states and repeatedly has to switch from one to another to overcome obstacles. In response we wish to increase the number and range of emotions which are detected and their affect on the game character. The players suggest that the ability to scale the character physics and behaviour proportionally to the degree of the emotion expressed would provide more subtle gameplay. All participants reported that they would like to see anger used in the game, with half of the players wanting a fear state. A consideration of this research was to develop an emotional game without using more traditional aggressive and violent gameplay. However our findings suggest that game players would like to be able to interact with their games using their full range of emotional states including anger and aggression. 6. FUTURE RESEARCH We note the need to extend the range of emotional states recognised by the game and the physical and behavioural responses of the game characters so that the game does not become repetitive. We wish to extend the study to consider additional uses of emotion recognition for computer games. These include automatically controlling the expressions of game characters in multiple player roll-playing environments thereby allowing players to know the emotional state of each other; allowing players to interact with game characters emotionally so that game characters respond with different information depending on the way the player talks to them; and creating acting games where the player must use emotion to act out scripts with a director awarding points on how well they perform. Building emotional interaction into computer games may provide a new genre of gameplay which is not wholly reliant on aggression and violence but instead rewards positive feelings and emotional interaction. REFERENCES 1. Freeman, D. (2004) Creating Emotion in Games, New Riders Publishing 2. Cowie, R. et al (2001) Emotion recognition in Human-Computer Interaction. IEEE Signal Processing, Jan, 32-80 3. Jones, C. and Jonsson, I-M. (2005) Speech patterns for older adults, HCII2005, Las Vegas, 22-27 July, Erlbaum
HCI 2005 – PROCEEDINGS VOLUME 2
2
