Since the arrival of the iPhone and Android, LBSs soon established ..... Special issue on Ambient and Social Media Business and. Application (Part I)' ...
Designing Seamless Mobile Augmented Reality Location Based Game Interfaces Mark Lochrie, Klen Copic Pucihar School of Computing and Communications InfoLab21, Lancaster University Lancaster, LA1 4WA {m.lochrie, k.copicpucihar}@lancaster.ac.uk ABSTRACT The majority of mobile devices nowadays harness location and orientation-sensing capabilities, permitting for sensor based Mixed Reality (MR) gaming experiences such as Augmented Reality (AR) games. Nonetheless, only few entertainment implementations of sensor based AR systems emerged, predominantly due to low precision of sensory information causing crude and jerky augmentation, significantly impacting meaningful augmentation that inevitably affects gameplay experience. In this paper, we present a novel pervasive mobile AR Location Based Game (LBG) named Time-wARpXplorer (TARX) where a seamless design approach is used in order to address the quality of augmentation. TARX comprises of a mobile client and an online authoring tool, created to encourage visitors and locals to explore the city of Lancaster by travelling back in time and space to discover their immediate and distant surroundings. Within the game, time travel is linked to present day through the implementation of Foursquare’s checkin platform to further raise awareness of historic sites. This paper focuses on the design considerations implemented through an iterative design process and prototyping of the mobile client, of which a novel interface emerged.
Categories and Subject Descriptors H.5.m [Information Interfaces and Presentation]: Miscellaneous
General Terms Design
Keywords Pervasive, Mobile, Mixed Reality, Augmented Reality, Location Based Game, Player Generated Content, Seamful Design.
1. INTRODUCTION In 1916, Einstein expanded his special theory, to include the effect of gravitation on the shape of space and the flow of time, referring to this as the, general theory of relativity [7]. Nearly a century after Einstein expressed this ‘general theory of relativity’, time travel has yet to be realised. However, the emergence of the Internet and networked media has opened fissures within the neat chronological representations of time by allowing memories and stories to seep through theses fissures and come back to the Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. MoMM2013, 2-4 December, 2013, Vienna, Austria. Copyright 2012 ACM 978-1-4503-1307-0/12/12 …$15.00.
Adrian Gradinar, Paul Coulton Imagination Lancaster LICA Building, Lancaster University Lancaster, LA1 4YW {a.gradinar, p.coulton}@lancaster.ac.uk present. Location Based Games (LBGs), Mixed Reality Games (MRG) and Augmented Reality Games (ARG) are game genres that would appear particularly adept to reveal these cracks in time and space. The emergence of new sensing capabilities, nowadays on board of the majority of mobile devices, enabled for wide adoption of location based games, however, this is not the case in Mixed and Augmented Reality Games. In this paper we explore what are the main challenges restricting wide scale adoption of Mixed and Augmented Reality Games in outdoor environments, where phone orientation and location is generally limited to on-board sensors such as GPS, accelerometers, gyroscopes and magnetometers. Through iterative seamless design approach and playtesting of a mobile AR LBG named TARX (built upon previous research [10, 11]), low precision of augmentation is identified as the main factor affecting gameplay experience. By applying iterative design process and prototyping in a novel game interface improving gameplay experience has emerged.
2. BACKGROUND TARX merges principles of LBGs and Augmented Reality (AR) interface. Here basic principals and previous work related to LBG, Mixed Reality Games (MRG) and Augmented Reality Games (ARG) are presented. Finally, an investigation into the technology issues and limitations surrounding designing such games is discussed.
2.1 Location Based Games LBGs have in fact been around for decades, in our backyards and on our streets. However, it is predominantly the digital counterparts to such games that are now referred to as LBGs particularly over the last decade, as they have become practical realisations of Location Based Services (LBSs) aimed at providing users with contextualised information based on their surroundings such as what or who is around and/or turn-by-turn navigation. However these services offer little in terms of a playful experience for more casual social walking, exploration and discovery activities. Since the arrival of the iPhone and Android, LBSs soon established themselves as data map providers to be used within social networks and mobile LBGs. Digital LBGs have yet to capture a similar player base as seen in Geocaching [12, 17]. This can be attributed to the fact that digital LBGs, have often drawn inspiration from console games that involve movement [12], rather than the exploiting players current surroundings, exploring and discovering new locations, which are motivations addressed by Geocaching, of which appeal to a wider demographic of participants from groups such as families to singular person walking activities [17]. The traits observed in many LBGs are based on movements rather than unveiling the location and
requiring a constant player attention, which has lead to the conclusion of what context, is location used within LBGs. In contrast to the term ‘location’ applied to many LBGs, where such games the players’ location and surroundings are independent from the actual game [12, 13] as the players’ actual locations are not contextualised within these games. Designing LBGs to be more than simply moving through space to actually permit players to interact with their surroundings can unlock greater depth of gameplay. Location may be defined by direct or implied solutions. Direct solutions such as GPS and AGPS, gyroscope and magnetometer (digital compass) can determine the location in majority of outdoor settings with no need of predefining the environment. As apposed to an implied solution whereby users’ location is determined by interaction with predefined objects or systems that have a known location relative to the environment. Implied solutions vary from utilising the camera lens, Wi-Fi hotspots, Radio Frequency Identification (RFID) and Near Field Communication (NFC) [19]. Although, scalability of implied solutions are limited they can increase interaction between user and object to a more physical presence. Early LBGs such as PACLAN [19] used NFC to leverage precise positioning to improve LBGs when GPS could not be utilised and also as a method of determining a physical connection between players and objects.
2.2 Mixed and Augmented Reality Games Nowadays, beside location sensing capabilities, the majority of mobile devices also support orientation-sensing capabilities. The mobile phone orientation may be determined through on-board accelerometers, gyroscope and magnetometer (digital compass). Coupling location and orientation sensing capabilities that permit for a Mixed Reality Gaming (MRG) experiences, such as Augmented Reality Games (ARG). Augmented Reality (AR) is often used to describe the merging of physical and virtual environments; the augmentation of virtual/audible data over real world environments, used to produce new and exciting ways to reveal further information, based on user’s physical surroundings. Although usually described within research literature as the registration of computer generated graphical information on the users’ view of the real/physical world [14]. It was, Ronald Azuma that provided the first clear definition of AR, in which he stipulated the following characteristics; the combination of real and virtual, interaction in real time and expressed in three-dimensions (3D). AR can be broken down into vision-based and sensor-based solutions. Their has been recent success in commercial adoption of vision-based AR within games on popular handheld gaming platforms such as Nintendo’s 3DS1 and Sony’s PS Vita2. Visionbased AR has also seen many examples of playful experiences within the research community ranging from single, to multiplayer games. An example of single player AR game is a laser tag style game [3]. Additionally, games integrating multiplayer aspects have also emerged, such examples are: AR Tennis [8], Cows vs. Aliens [16] and MapLens [15]. Irrespective of successful pioneering outdoor AR like games, such as Mosquito Hunt built by TinyMotion [21] and Mirage Money [4], sensor-based AR solutions have yet to make its mark in the gaming industry. Nevertheless, in the research domain, examples
1
Nintendo - AR Games - http://goo.gl/n2QM3
2
Sony, PS Vita AR games - http://goo.gl/gLb7C
of a playful sensor based AR experiences have already been released into the wild. One such example is the augmented reality prototype and game ‘Free All Monsters’ [6, 12], a game coupling POI information with a game design that drew upon the successful characteristics of popular outdoor activities.
2.3 Challenges of sensor-based mobile ARG Mobile AR is commonly implemented through magic-lens paradigm [1] where the mobile phone acts as a transparent glass pane showing and enhanced view of the environment laying behind the pane. Here, the enhanced view consists of a live camera video stream overlaid by digital content that is rendered in correct spatial context of captured video stream. Mobile phone location and orientation sensing capabilities facilitate implementation of AR interfaces, however, crude accuracy and low frequency of sensory information may create an inaccurate and often jerky augmentation. For example, low precision of GPS (in ideal conditions limited to approximately 2 metres [20]) may have a significant impact on achieving meaningful augmentation, which may consequently affects gameplay experience especially in scenarios where objects are found in close proximity to the user [5]. Inaccuracy is one of the key problems of sensor based AR solutions, which has also been identified in an online user survey on current mobile augmented reality applications conducted by Olsson et al [18]. Therefore, it is vital to design experiences around the quality of phone sensing capabilities. One example of designing AR solutions around sensory limitations was presented by Burnett et al [2] where alternative visualization method is used to support navigation through space and time to solve the problem of visualizing depth within sensor based AR interfaces. Burnett et al introduce common art techniques for adding perspective to 2D visualisations. A static linear plane (grid) was implemented to the interface to create a sense of a vanishing point, and by modulating the size and scale of the Point of Interest (POI) icon represent depth. To further facilitate depth perception, atmospheric perspective by adjusting icon transparency depending on object distance is also used. Such visualization operates in reduced resolution of depth representation by which it overcomes the problem of sensory inaccuracy by reducing the resolution of depth represented.
3. Time-wARpXplorer Design Although, Einstein’s ‘general theory of relativity’ has yet to be realised, the advances in today’s mobile devices and social platforms have opened fissures within exploring places through space and time. The aim of TARX is to provide a platform for the local community and tourists, to explore the city in space and time and in a playful manner. The premise of the game is to explore and discover a modern day city, navigating through space and time, by augmenting photos of historical locations over the present day. Every player of the game has the ability to store the photos and data they collect as they navigate their way through the city in their own sticker album. TARX consists of two main interfaces: the mobile client known as Time-wARpMachine and the online authoring tool. The mobile counterpart utilises sensor based AR techniques such as GPS, accelerometers, magnetometers and gyroscopes integrate inside modern day mobile phones.
Figure 1 - Screenshots of the TimewARp-Xplorer mobile client 'warp' interface, moving left (a) to right (c). The use of combining these readings obtained from such sensors allow the position and orientation of the phone camera to be estimated [9] in relation to 3D space. It is this estimation that is the cause of the fundamental issues with sensor based AR. The inaccuracies of GPS can be highly variable due to spatial scattering, which will ultimately impact on the contextual sensitivity of the augmentation consequently affecting gameplay experience. Based on the initial testing and prototyping it was deemed necessary to further develop the interface (version 2) solving some of the issues based on the data collected and observed whilst trailing version 1 of the game [10]. In order to solve the inaccuracies problem, an iterative prototyping approach was adopted where playtesting was applied to evaluate the implementation of each iteration. The results of the process are the interfaces presented in Figure 1.
When the device is flat (horizontal) the map view is displayed to the user, however when in an upright (vertical) position the AR finder is re-established (Figure 1b). The decision to switch between two views was made to improve system usability, save on battery power (as constant AR operation would greatly reduce battery life), to encourage players to be more aware of their environment by not having to constantly look down or through the mobile device, to remove the need to hold the device in an upright orientation whilst walking, as it is not ideal for the player and also has greater social impact; if the device was in an upright position (similarly to taking a photo) those around may think people were being recorded.
One of the design challenges was the inaccuracy of using sensor based AR with GPS and the POI’s bouncing around when viewed at close distances. This is because the system cannot determine the true location of the players’ physical location and the varying coordinates of the GPS is constantly fluctuating and achieving at best accuracy of up to 2 metres [20]. To solve the issue, the game interfaces were designed to work in three ways. The first being when the player is out of range on the target, a traditional map interface is adopted (Figure 1a). The player is encouraged to move to predefined distance range in order to launch the AR finder (Figure 1b). The optimal distance range was defined experimentally while trilling the application to a range of 40-80 metres from the POI. The AR finder can be activated only when the player reaches the desired distance range.
Upon arrival at the warp destination, the player then has the ability to warp back to a random time. Warp view is activated by clicking on the marker (figure 1b: squared window icon) to reveal the photograph from a time gone by. The exposed photograph is then download to the player’s device (Figure 1c) and shown to the user at a fixed place on phone screen. Here yet again, the design had to follow the inaccuracy of sensor based solutions. Instead of providing automatic photo alignment, users are given opportunity to fine align the photo. This is achieved by rendering photo in a way to only occlude part of the screen, while camera video stream continues to be rendered in the background (see Figure 1c). Now the emphasis is placed on the player to fine align the (transparent) photograph with the camera captured video rendered behind the photograph. After alignment with real world environment, the user is able to capture a snapshot of ‘a time gone by inside a present day environment’ (Figure 1c: camera icon). This snapshot is then stored on the players’ device and can be shared through various sharing services.
Once within the range, the system incorporates the use of how the player physically holds the device (vertical or horizontal).
A successful warp is determined by the return of the photo of the location. To enable the player to revisit the warp location,
the game adds the information for the location and the photographs into the players’ personal sticker album. These achievements are awarded to players for the amount of interaction they have with the game and specific requirements that are met by playing the game. Every player has their own hall of fame (wARpWall) that displays the player’s history along with any achievements unlocked. Not only are the badges viewed on the players Time-wARpMachine but also online as a souvenir the player can take away, along with the tracked journey. Alongside utilising a badge-based system, TARX uses a ranking system to award explorers with a range of ranks for the type of explorer they are.
4. CONCLUSIONS This paper presents a follow on from previous research of building a multi-faceted LBG that combines the successful attributes seen in many outdoor activities with a digital twist, through the use of AR magic lenses. The concept of TARX was to provide users with a toolkit to create their own stories and experiences for where they live. Although the current design has been built upon from previous research [10], the interface has dramatically been improved based on initial prototyping and playtesting, an extensive evaluation of this improved interface is required by analysing its longitudinal performance during studies based on trials held with cooperation with the local museum. From long-term results, the authors expect to be able to establish player traits, to define what makes a successful mobile LBG, in terms of how players interact with AR interfaces in a gaming environment. Therefore, to understand if mobile LBGs can have similar success to those seen in other less technical outdoor activities, these games need to be designed with scalability in mind. However, this can only be achieved once greater player and location data is collected. The authors would argue that AR is a useful approach for extending user interactions within a mixed reality space, nevertheless it is important for designers of AR applications to consider practical uses within environments rather than novelty explorations.
5. ACKNOWLEDGMENTS The authors would like to thank Lancaster Museum for their support in sourcing initial photographs for the project.
6. REFERENCES [1] Bier, E. A., Stone, M. C., Pier, K., Buxton, W., and DeRose, T. D. “Toolglass and magic lenses: the see-through interface,” in SIGGRAPH ’93: Proceedings of the 20th annual conference on Computer graphics and interactive techniques, 1993, no. Mmm, pp. 73–80.
[2] Burnett, D., Coulton, P, Murphy, E., and Race, N., Designing Mobile Augmented Reality Interfaces for Navigation Through Space and Time,
[3] Chehimi, F, Coulton, P & Edwards, R 2008, 'Using a Camera Phone as a Mixed-Reality Laser Cannon.' International Journal of Computer Games Technology, vol 2008, pp. 1-6.
[4] Chehimi F., Coulton P, and Edwards R. 3D Motion Control of Connected Augmented Virtuality on Mobile Phones, proceedings of International Symposium on Ubiquitous Virtual Reality-VR. (ISUVR08) ,GIST, Gwangju, South Korea.10th to 13th July, 2008.
[5] Čopic Pucihar, K., Coulton, P., Exploring the Evolution of Mobile Augmented Reality for Future Entertainment Systems. Currently in review for the journal of Computers in Entertainment.
[6] Coulton, P., Lund, K., and Wilson, A., 2010. Harnessing Player Creativity to Broaden the Appeal of Location Based Games. In Proceedings HCI2010 – Play is a Serious Buisiness, Dundee, Scotland, 6th-10th September 2010.
[7] Einstein, A., Lawson, R. W., Geroch, R., & Cassidy, D. C.
(1920). Relativity: The special and the general theory. Pi Pr.
[8] Henrysson A., Billinghurst M., and Ollila M., Face to Face Collaborative AR on Mobile Phones. Proceedings International Symposium on Augmented and Mixed Reality (ISMAR’05), pp. 80–89, Austria, 2005
[9] Kähäri, Markus and Murphy, David J., MARA - Sensor Based Augmented Reality System for Mobile Imaging, In Proceedings of ISMAR 2006, Santa Barbara, USA, Oct. 2006.
[10] Lochrie, M, Copic Pucihar, K, Gradinar, A & Coulton, P 2013, 'Time-wARpXplorer: creating a playful experience in an urban time warp'. in Proceedings of Physical and Digital in Games and Play Seminar. Physical and Digital in Games and Play, Tampere, Finland, 29-30 May.
[11] Lochrie, M, Lund, K & Coulton, P 2010, 'Community generated location based gaming.’ in: BCS '10 Proceedings of the 24th BCS Interaction Specialist Group Conference. British Computer Society, Swindon, pp. 1-4, HCI 2010 Play is a serious business, Dundee, 8-12 September.
[12] Lund, K., Coulton, P., and Wilson, A. 2011. Participation inequality in mobile location games. In Proceedings of the 8th International Conference on Advances in Computer Entertainment Technology. ACM, New York, NY, USA, , Article 27 , 8 pages.
[13] Lund, K, Lochrie, M & Coulton, P 2012, 'Designing Scalable Location Based Games that Encourage Emergent Behaviour: Special issue on Ambient and Social Media Business and Application (Part I)' International Journal of Ambient Computing and Intelligence (IJACI), vol 4, no. 4, pp. 1-20.
[14] Milgram, P. and Kishino, F., Augmented reality: A class of displays on the reality-virtuality continuum. in SPIE 1994, (1994), 282-292.
[15] Morrison, A., Oulasvirta, A., Peltonen, P., Lemmela, S., Jacucci, G., Reitmayr, G., Näsänen, J., and Juustila, A. (2009). Like bees around the hive: a comparative study of a mobile augmented reality map. In Proceedings of the 27th international conference on Human factors in computing systems (CHI ‘09). ACM.
[16] Mulloni, A., Wagner, D., & Schmalstieg, D. (2008, September). Mobility and social interaction as core gameplay elements in multi-player augmented reality. In Proceedings of the 3rd international conference on Digital Interactive Media in Entertainment and Arts (pp. 472-478). ACM.
[17] O’Hara, K. 2008. Understanding Geocaching Practices and Motivations. In Proceedings of the 26th Annual SIGCHI Conference on Human factors in Computing Systems. 5 – 10 April 2008, Florence, Italy, pp.1177-1186.
[18] Olsson, T. (2012). User expectations and experiences of mobile augmented reality services. Tampereen teknillinen yliopisto. Julkaisu-Tampere University of Technology. Publication; 1085.
[19] Rashid, O, Bamford, W, Coulton, P, Edwards, R & Scheible, J 2006, 'PAC-LAN: Mixed reality gaming with RFID enabled mobile phones' Computers in Entertainment (CIE), vol 4, no. 4, pp. 1-17.
[20] Rashid, O., Mullins, I., Coulton, P., and Edwards, R. Extending cyberspace: location based games using cellular phones. Comput. Entertain. 4, 1, Article 4 (January 2006).
[21] Wang, J., Zhai, S., & Canny, J. (2006, October). Camera phone based motion sensing: interaction techniques, applications and performance study. In Proceedings of the 19th annual ACM symposium on User interface software and technology (pp. 101110). ACM.
