Implications for Location Systems in Indoor Wayfinding for ... - CiteSeerX

Implications for Location Systems in Indoor Wayfinding for Individuals with Cognitive Impairments Alan L. Liu, Harlan Hile, Gaetano Borriello, Henry Kautz, Brian Ferris Department of Computer Science and Engineering University of Washington Box 352350 Seattle, WA 98195-2350 USA {aliu,harlan,gaetano,kautz,bdferris}@cs.washington.edu

Abstract— We studied an indoor wayfinding application for individuals with cognitive impairments using Wizard-of-Oz techniques. This gave us insights into the technological considerations for the location system underlying such an application. We have abstracted these into generally desirable properties for location-aware systems for wayfinding. In addition to general requirements for accuracy and robustness, we discuss what other information about the environment should be provided that supports wayfinding. Our work showed that combining rich environment information with user location is key to making indoor wayfinding applications feasible.

I. I NTRODUCTION This paper presents our findings regarding the features and requirements of a location system that supports indoor wayfinding applications. Using a Wizard-of-Oz testing environment to simulate the location system, we evaluated an indoor wayfinding application prototype designed for individuals with cognitive impairments, such as traumatic brain injury, cerebral palsy, mental retardation, and Alzheimer’s disease. The results from our study informed us not only of what kinds of wayfinding directions were useful but also when and where they should be given [1]. This in turn suggests what properties are important in a location system, including general properties for accuracy and robustness and what information about the environment would be useful to the user. We used the Wizard-of-Oz method so that potential users could walk through a realistic experience of using a wayfinding application without all of the underlying system being fully implemented. This prototyping approach is particularly important in collaborative, iterative design with individuals with cognitive impairments, because simply “thinking through” designs on paper is rarely effective [2]. The ultimate goal of our work is to create “smart” community wayfinding devices for individuals with cognitive impairments [3] that will interact appropriately with the user automatically.

Pat A. Brown, Mark Harniss, Kurt Johnson Center for Technology and Disability Studies University of Washington Box 357920 Seattle, WA 98195-7920 USA {pabrown,mharniss,kjohnson}@u.washington.edu

II. BACKGROUND Our application’s design draws upon psychological models of spatial navigation [4], [5], usability studies of interfaces by people with cognitive impairments [6], and interviews conducted with individuals with disabilities and their caregivers regarding wayfinding [7]. Difficulty in wayfinding can be caused by a variety of cognitive impairments, including traumatic brain injury, cerebral palsy, mental retardation, and Alzheimer’s disease. The degree of impairment can vary widely; an individual who is mildly impaired might only have problems in following a complex route that involves changing modes of transportation, while someone with a more significant impairment may become lost in their own home. Almost any degree of impairment often has serious negative consequences on subjects’ quality of life [8]. Current methods for aiding people with wayfinding are labor-intensive. For example, a job coach from an employment agency, who works with individuals with cognitive impairments to support them in learning new jobs and their routines in order to help the individual maintain paid employment, may work for months helping a person learn how to travel to and from work, and even then, the effort may be unsuccessful. Although this paper focuses on an application for people with cognitive impairments, it is likely that a design that allows an application to be used with low cognitive overhead will also be attractive to many people without disabilities. The lessons learned from our study may be useful, for example, in tour guide applications [9] and other location-aware applications [10]. III. P ROTOTYPE D ESIGN AND S TUDY We designed an interface suitable for use on a handheld device, such as a PDA, to send directions to the user. Directions sent to users consist of a subset of images, audio, and text. The design was a result of several rounds of pilot testing involving members of our research group and job coaches from a community based rehabilitation program. Job coaches were

Fig. 1. Sample photograph with overlaid arrow, indicating direction onto the catwalk. Fig. 3. Location wizard GUI running on a tablet PC. Red mark shows participant location and orientation.

Fig. 2.

Sample photograph with highlighted area (note the room number).

able to consider the needs of their clients when evaluating the system. Based on the pilot tests, we refined the types of images presented and the wording of the audio and text messages. We use four types of images: photos, arrows and other generic symbols, photos with overlaid arrows, and photos with an outlined area. Photos are of landmarks and other interesting features. Arrows and other generic symbols are used to tell a user to turn or stop, which can be used at times when appropriate photos are not available or distinctive enough. Overlaid arrows on photos are intended to disambiguate where to go as well as provide additional indication of where a user should go next (see Figure 1). Some photos contain a highlighted area (e.g., a room number or elevator button, Figure 2). The need to include arrows and outlines became clear as a result of our pilot testing. In particular, with small images of indoor landmarks, it can be difficult for a user to know where to focus. A. Implementation Our prototype client application is implemented in Java and SWT and runs under Windows Pocket PC 2003 on a HP iPAQ handheld with a WiFi wireless adapter. The software supports display of images up to 240x320 resolution. We use images with 240x180 resolution in order to leave room for text to be

Fig. 4. Interaction diagram. Location wizard provides location information on the participant (1) sent over WiFi (2), while the navigation wizard uses that location information to decide which directions to send to the participant’s device (3). Directions are sent over WiFi and ack’d for robustness (4). Participant then follows directions displayed on device (5).

displayed as well. Arrows and highlighted regions are overlaid on the photos manually and in advance. The device acts as a client to a remote server controlled by the navigation wizard, a person who sends instructions to the client on what image and text to display and what audio to play, based on the participant’s location and heading. To gather location and orientation information, we use a location wizard, a person who follows study participants and transmits their location and orientation to the navigation wizard in realtime using a simple map-based GUI that runs on a Tablet PC. This is a Java program that sends location and orientation updates 4 times per second. Orientation is automatically set based on the direction of motion, but can also be adjusted manually by the location wizard. Figure 3 shows the location wizard interface, implemented on a Tablet PC, and Figure 4 shows the system diagram. B. Method We used a within-subjects, counterbalanced study design where the interface guided each of seven participants with cognitive impairments through three routes of differing complexity using three different subsets of modalities. The studies

were done in our Computer Science and Engineering building, which was unfamiliar to all participants. We chose three routes that traversed through different parts of the building in order to minimize any learned familiarity. Route 1 involved no floor changes, while Route 2 involved using an elevator, and Route 3 involved taking the stairs down one flight. C. Materials and tasks We had seven participants with a variety of cognitive impairments. Each participant was shown the device and examples of the interface. They were led to the starting location of each route by the location wizard, and their task was to follow the device’s directions until they arrived at the route’s destination. Two of our researchers from the Department of Rehabilitation Medicine followed in order to take notes, get feedback from the participant, and provide assistance in case the participant became confused or uncomfortable. All participants gave permission to audio record their session. At the end of each study, we asked participants a series of questions about what they thought of the interface and how they typically navigate when they are in unfamiliar locations. IV. S TUDY I NSIGHTS Our study made clear that any wayfinding application needs rich knowledge of the environment in addition to accurate location in order to effectively guide users. This information needs to be presented in a clear and familiar way on a device with a practical form factor. A. Location and orientation accuracy An underlying location system should have location accuracy of approximately 2-3 meters to adequately distinguish between rooms. In turn, a location system would need to produce at least 1 update/second while a user is traveling at walking speeds and there are turns or stops coming up. This is dependent on the individual user and the indoor environment. In environments where turns are spaced further apart, less accuracy would be acceptable. Knowing whether a user is facing the correct way is more important than fine-grained orientation, so 30-45 ◦ orientation accuracy can be satisfactory. However, direction of motion does not provide sufficient orientation information. Users may stop and look around when they are confused, and it is important for the wayfinding application to give directions appropriate to their current orientation. Additionally, orientation information can allow the wayfinding application to recognize signs of confusion–like stopping and looking around–and respond accordingly. Users need directions that communicate a sense of responsiveness by the application. It is important for the application to deliver directions at the right time, such as near intersections and destinations. Directions sent too early can cause confusion and be dismissed or forgotten. Directions sent too late can cause users to miss turns or destinations. Directions also need to convey a sense of how long they are applicable. For instance, two participants walked by the

Fig. 5. Left: Direction telling users to proceed to their destination and stop. This direction very closely followed the direction to turn right, and often came too late. In addition, one participant thought the skewed arrow was confusing. Right: Map showing spacing of turns and the path of a participant with difficulty locating the destination.The red dot indications the final destination reached.

destination in the case of one route where the destination was close to a previous turn, because they received the direction telling them to stop at the room that was their destination (see Figure 5) only after they had already passed by it at their normal walking pace. B. Maps of indoor environments In order to produce directions to guide users, the wayfinding application needs the location system to provide a map that contains the structure of the environment, such as the position of hallways, rooms, stairs, and elevators. Such maps would have to be created for each area where users want to travel. As a bonus, location systems that provide this information can leverage this encoded knowledge to produce more accurate location estimates [11]. C. Vocabulary to describe routes There needs to be a mechanism to translate abstract physical position to semantically meaningful place names, as textual and audio directions are more effective when they describe routes in terms of the latter. It is possible to use low-level directions such as “turn left” or “walk forward,” but users navigate more quickly when given a better sense for what is ahead. However, using unfamiliar vocabulary can cause confusion. This was evident with the use of the word “walkway” to describe the catwalk structure (see Figure 1) that was part of a route. Participants who were unfamiliar with the term or unsure that it applied to the catwalk asked for confirmation from researchers. D. Photos of the environment A database of geo-located images would allow a wayfinding application to include annotated photos of the environment to directions. These could be used to enhance directions by disambiguating text and audio, and also convey more

information about the route to users who are more visually inclined. Photos were reviewed positively because they also gave the participants more time to consider the direction and make visual associations. Several participants noted the arrows overlaid on the images were helpful because they “tell you where you’re going.” Directions with photos increased the cognitive load on participants, who needed to interpret features in the photos, which were small and of a relatively uniform building environment. Participants suggested that photos might be more useful if they showed more easily seen landmarks and objects, such as bright red fire extinguishers. E. Device should be lightweight, readable, socially accepted Ideally, a wayfinding application would run on a device that is easy for a user to integrate into his or her life. A lightweight device is more practical to carry around and can be personalized. A smaller device is more portable and less likely to draw negative attention, but would need a bright, readable screen. The location system would more likely run, at least in part, in the infrastructure of the building or on remote servers. For example, a location estimate may be computed by the client device (for privacy reasons) while photos may be provided by a server. Using vibration feedback in addition to audio could help avoid missed directions, and address one of the participants request for a ‘silent mode.’ One participant also expressed a preference for a wireless headset, saying he would be less selfconscious. Although none of the participants had complaints about the size of the iPAQ, one participant thought it was too heavy and was concerned that there would be social stigma attached to using one. V. D ISCUSSION Each of the specific insights from or study can be abstracted to a general principle for location systems and/or the wayfinding applications built upon them. A. Utilize multiple sensors for robustness We believe that a suitable location technology would have to combine and extend a variety of existing location technologies in order to provide the level of coverage, accuracy, and ubiquity needed for our wayfinding application to be of practical value to our user population. GPS is a good solution for localization in outdoor areas where there is coverage, but performs poorly indoors as well as near tall buildings and other obstructions. Using WiFi radio beacons to localize has the benefit of leveraging existing WiFi covered areas, such as indoor environments and dense urban areas, where GPS is typically inadequate. Scalable WiFi solutions are the focus of several indoor localization techniques [11][12]. However, WiFi techniques do not provide orientation. WiFi localization also does not work in dead-zones, where there are no WiFi signals. Elevators are one type of area that is highly likely to lack WiFi coverage, but they are also an area

where we found that location information was most needed. One study participant, clearly having difficulty determining when to exit the elevator when other riders were exiting on different floors, reported that, “It was confusing; I wasn’t sure what to do.” Sensors such as barometers and gyroscopes could be used for dead reckoning by smoothing over transients in WiFi signal strength due to dead-zones, other people walking by, the user turning, etc. A particle filter-based localization algorithm could leverage these extra sensors by using them to improve the accuracy of its motion model [13]. As a bonus, sensors such as a compass or gyroscope would be useful in detecting how much a user is turning. A navigation application might then use the tentativeness of a user’s motion to better grasp the user’s level of confusion. Computer vision could also enhance user location and orientation estimation [14][15]. Photographs taken by a user with a camera phone could be sent to a server for location estimation. Annotations, such as arrows, could then be added automatically to those photographs, and might be easier for the user to understand quickly. B. Automatically obtain maps of the environment Potential users of our wayfinding application may find themselves in places outside their routine paths. Therefore, the underlying location system must be scalable and easy to deploy in a wide area. Infrastructure systems could provide information about the environment (such as maps, photographs, and labels) [16]. However, it is desirable for a wayfinding application to work well even when this information is not available. Current WiFi localization systems share this undesirable dependency on existing mapping and infrastructure information. Specifically, most WiFi localization techniques require a training set of signal strength readings labeled against a ground truth location map, which is prohibitive to collect and maintain as maps grow large. Recent research has attempted to address this problem by solving the simultaneous localization and mapping (SLAM) problem for WiFi: given an unlabeled sequence of signal strength readings, reconstruct the underlying beacon locations and localize the scanning device within the resulting topological map. Techniques for accomplishing this generally use a form of dimensionality reduction to reduce the highly-dimensional WiFi readings into low-dimensional latent location. The resulting method allows an agent to easily build localization maps for new areas, allowing wayfinding at a truly large scale [17]. C. Provide directions in semantically meaningful terms Using understandable place names along routes is a crucial feature for any textual or audio directions that a wayfinding application might provide. One approach to determine the labels of common route features might be to use machine learning to train an algorithm to automatically label locations as hallways and rooms [18]; this could be used in conjunction with SLAM. A complementary approach using manual labeling could produce labels for nonstandard environmental

features, such as walkways or unique landmarks, or names of places with special meaning to specific users [19]. D. Intelligently mix use of graphics and photographs For photo usage in a wayfinding application to be practical, there must be a way to quickly collect and organize relevant photographs. It would also be beneficial to automatically identify landmarks in the environment. One approach might be to use a large database of photos taken by individuals or robots and apply computer vision techniques to dynamically create photos appropriate to a user’s viewpoint. In addition, recent work suggests that it is possible to reconstruct some of the 3-D structure of the environment where the photos were taken, making it possible for the application to overlay arrows in the correct 3-D space of the photograph without manual intervention [14]. Using localization, these algorithms could be “bootstrapped” with knowledge of the constraining area, making it possible to create the appropriate images in realtime by radically decreasing the size of the database of relevant photos [15]. E. Use a form factor that fits into everyday life A promising client platform for our application would be the mobile phone. Mobile phones are lightweight and have vibration feedback, and because of their widespread use, having one would be more socially accepted than a PDA. In addition, newer and more featureful phones support GPS and WiFi, which can be used for localization, and Bluetooth, which supports the use of wireless headsets and communication to additional sensors (e.g., accelerometers are already present in some modern phones). With the decline in both size and price of these small sensors, we hope that the mobile phone will be a rich platform for developing wayfinding applications such as ours. VI. C ONCLUSION We studied an indoor wayfinding application using Wizardof-Oz techniques. This gave us insights into the technological considerations for the location system underlying such an application. We have abstracted these into generally desirable properties for a location system; note that these properties are not simply asking for “more accuracy.” Our work showed that connecting location estimates to the actual environment is key to making indoor wayfinding applications feasible. There are several key problems that require attention: • Providing accurate real-time estimates of orientation as well as location; • Access to or automatic construction and labeling of building floorplans; • Access and automatic annotation of photographs of the indoor environment; Our future work will focus on these issues. There is no single location system today that meets all requirements. We believe that the development of such a system will be a fertile area of research. Although gleaned

from studies with individuals with impairments, these requirements are clearly desirable for any indoor wayfinding application. Utilizing such technologies for individuals without impairments would have the added benefit of reducing the cost for users with special needs and easing the transition for individuals with newly acquired impairments by providing a familiar interface. ACKNOWLEDGMENT This work is funded by the National Institute on Disability and Rehabilitation Research (NIDRR) Grant #H133A031739. R EFERENCES [1] A. L. Liu, H. Hile, H. Kautz, G. Borriello, P. A. Brown, M. Harniss, and K. Johnson, “Indoor wayfinding: Developing a functional interface for individuals with cognitive impairments,” in Assets ’06: Proc of the 8th Intl ACM SIGACCESS conf on Computers and accessibility, 2006, in press. [2] A. Lepisto and S. Ovaska, “Usability evaluation involving participants with cognitive disabilities,” in Proceedings of the third Nordic conference on Human-computer interaction, 2004, pp. 305–308. [3] D. J. Patterson, L. Liao, K. Gajos, M. Collier, N. Livic, K. Olson, S. Wang, D. Fox, and H. Kautz, “Opportunity knocks: a system to provide cognitive assistance with transportation services,” in Proceedings of the Sixth International Conference on Ubiquitous Computing, 2004. [4] R. Golledge, Wayfinding behavior: Cognitive mapping and other spatial processes. John Hopkins University Press, 1999. [5] K. Lynch, The Image of the City. MIT Press, 1960. [6] A. Sutcliffe, S. Fickas, M. Sohlberg, and L. Ehlhardt, “Investigating the usability of assistive user interfaces,” Interacting with Computers, vol. 15, no. 4, pp. 577–602, August 2003. [7] P. A. Brown, M. Harniss, and K. Johnson, “Cognitive support technologies: Implications for design and user interface,” in Proc of the Intl Conf on Technology and Persons with Disabilities, March 2006, http://www.csun.edu/cod/conf/2006/proceedings/2853.htm. [8] M. Sohlberg, B. Todis, S. Fickas, P. Hung, and R. Lemoncello, “A profile of community navigation in adults with chronic cognitive impairments,” Brain Injury, vol. 19, no. 14, pp. 1249–1259, December 2005. [9] K. Cheverst, N. Davies, K. Mitchell, and C. Efstratiou, “Developing a context-aware electronic tourist guide: Some issues and experiences,” in Proceedings of CHI-2000, 2000, pp. 17–24. [10] J. Hightower and G. Borriello, “Location systems for ubiquitous computing,” IEEE Computer, vol. 34, no. 8, pp. 57–66, 2001. [11] B. Ferris, D. Haehnel, and D. Fox, “Gaussian processes for signal strength-based location estimation,” in Proc. of Robotics: Science and Systems, 2006, 2006, in press. [12] (2006) Navizon - peer-to-peer wireless positioning. [Online]. Available: http://www.navizon.com [13] D. Fox, J. Hightower, L. Liao, D. Schultz, and G. Borriello, “Bayesian filtering for location estimation,” IEEE Pervasive Computing (special issue on Dealing with Uncertainty), vol. 2, no. 3, pp. 24–33, 2003. [14] N. Snavely, S. M. Seitz, and R. Szeliski, “Photo tourism: exploring photo collections in 3d,” ACM Trans. Graph., vol. 25, no. 3, pp. 835–846, 2006. [15] L. Paletta, G. Fritz, C. Seifert, P. Luley, and A. Almer, “Visual object recognition in mobile imagery for situated tourist information systems,” in Proc. Workshop on Wearable and Pervasive Computing, 2005. [16] J. H. Kang and G. Borriello, “Harvesting of location-specific information through wifi networks,” 2nd International Workshop on Location- and Context-Awareness (LoCA 2006) (at Pervasive 2006), pp. 86–102, 2006. [17] M. Bowling, D. Wilkinson, A. Ghodsi, and A. Milstein, “Subjective Localization with Action Respecting Embedding,” The International Symposium of Robotics Research, vol. 143, 2005. [18] B. Limketkai, L. Liao, and D. Fox, “Relational object maps for mobile robots,” in IJCAI ’05: Proc of the 19th Intl Joint Conf on Artificial Intelligence, 2005. [19] J. Hightower, S. Consolvo, A. LaMarca, I. Smith, and J. Hughes, “Learning and recognizing the places we go,” in Ubicomp ’05: Proc of the 7th Intl Conf on Ubiquitous Computing, ser. Lecture Notes in Computer Science. Springer-Verlag, September 2005, pp. 159–176.