Route Selection Algorithm for Blind Pedestrian - IRIT

International Conference on Control, Automation and Systems 2010 Oct. 27-30, 2010 in KINTEX, Gyeonggi-do, Korea

Route Selection Algorithm for Blind Pedestrian Slim Kammoun, Florian Dramas, Bernard Oriolaand Christophe Jouffrais IRIT, CNRS & University of Toulouse, Toulouse, France (Tel : +33(0)561556305; E-mail: {kammoun, dramas, oriola, jouffrais}@irit.fr) Abstract: The vast majority of existing route selection processes is designed for vehicle navigation. In this paper we describe an adapted routing algorithm for visually impaired pedestrians based on users needs. Our aim was to find the most adapted route that connects origin and destination points, and which can provide the Blind with a sparse but helpful mental representation of the itinerary and surroundings. Based on multiple brainstorming sessions and interviews with blind people and an orientation and mobility (O&M) instructor, different classes of objects were defined and tagged in the Geographical Information System. The optimal route was then selected using the Dijkstra algorithm. This method will be used in NAVIG (Navigation Assisted by Artificial VIsion and GNSS), an assistive device for the Blind, whose aim is to improve orientation, mobility and objects localization. Keywords: Route Selection, Blind, User Needs, Assistive technology, Geographical Information System.

1. INTRODUCTION

ceiver with a Geographical Information System (GIS) in order to determine the user location. Though, GPS is inaccurate in many environments such as areas with high buildings. Therefore, those systems are not usable to guide blind users downtown, which is the primary usage. To improve GPS positioning, many propositions have been made. [1,7,11] suggest the use of Differential GPS (DGPS) that reduces the nominal error range from 10-20 meters to less than 1 meter in ideal conditions. A major problem comes from the hight cost of special equipements and the weight of the receiver (at least 0.5 kg) that is not appropriate to pedestrian mobility. Furthermore, the error range is still insufficient for blind pedestrian mobility. The ODILIA project [3] proposes the use of a dead reckoning device that enables a user to determine his current position even though the satellite signals are lost by calculating subsequent positions out of the users speed [8]. The NAVIG project [4] introduces a novel approach based on the combination of GNSS positioning with the localization of visual landmarks through embedded computer vision system. According to blind people interviews, we suggest that blind pedestrians may greatly benefit from creating their own spatial representation, even a sparse one, of the environment. To do so, the sole improvement of user position is not enough; one also needs to render information related to the surroundings. As defined by [2], GIS can be assimilated as a tool for capturing, manipulating, displaying, querying, and analyzing spatial data. Due to dynamicity and complexity of the environment, presentation of geographic information to visually impaired people, including congenitally Blinds, is complicated. It is obvious that branded GIS are not accurate enough for assisting blind pedestrian navigation (absence of sidewalks, pedestrian crossing, etc.). GIS include a digitized database and software designed to select routes, track the travelers path, and inform the user about local features. While progress on GIS precision have been made, procedure of path selection has been done with a variety of network algorithms using criteria such as shortest path, linear programming

Route selection is the procedure of choosing an optimal pathway between an origin and a destination. Traditionally, path selection for pedestrians is assumed to be the result of minimizing procedures such as selecting the shortest or the quickest path. For the Blind, a longer route can be more convenient than a shorter route, in order to avoid various difficulties. Our aim is to introduce a new classification of the geographical data to perform an optimal route selection for blind pedestrians who rely on an electronic travel aid. This classification was based on multiple brainstorming and interviews with blind people and an orientation and mobility (O&M) instructor. According to this classification, we designed a specific route selection algorithm to be used by blind pedestrians assisted by the NAVIG system [4, 18] which is directed towards increasing the autonomy of visually impaired users.

2. ASSISTED NAVIGATION FOR THE BLIND According to [16] mobility can be defined as the ability to travel safely, comfortably, gracefully and independently through the environment while orientation consists on knowing where one is, in absolute terms of reference. Over the past four decades, several assistive devices for visually impaired persons have been developed to improve mobility and orientation of the Blind. They can be classified into two main categories: • The Electronic Travel Aids (ETAs) designed to help the Blind in detecting obstacles. They are based on ultrasonic or laser telemeters [see e.g. 6] or computer vision [see e.g. 9]. • The Electronic Orientation Aids (EOAs) that can provide the Blind with some degree of situational awareness and guidance in unknown environments, up to now mainly based on GPS systems. The approach used in EOAs relies on Location Based Services (LBS) especially GPS or Differential GPS systems and combines geo-spatial positioning through GNSS re-

978-89-93215-02-1 98560/10/$15 ©ICROS

2223

traveling salesman, network optimization, and locationallocation formats. In this paper we describe a route selection algorithm adapted to visually impaired pedestrians based on user needs.

3. ANNOTATION OF GEOGRAPHICAL DATA Most spatial databases used in GIS have been developed without considering the needs of the Blind [13]. Integrating the user during the design of an assistive device becomes necessary in order to obtain better results, especially when the system is intended for impaired persons. In the course of the NAVIG project, we have been working with the Institute for Young Blind (CESDV-IJA, Toulouse, France). Based on multiple brainstorming and interviews with blind people and an orientation and mobility (O&M) instructor (see [17] for details), we defined a classification of GIS database objects including four main classes that need to be specially tagged: 1. Points of Interest (POI): Places or objects that are potential destinations. They also are useful or interesting places to allow a better understanding of the environment while traveling (e.g. public buildings, shops, etc.). 2. Landmarks (LM): Locations that can be detected by the user in order to confirm its own position within the itinerary (e.g. changes in the ground texture, telephone poles, traffic lights, etc.) 3. Walking Areas (WA): All the possible pedestrian paths as defined in [10] (e.g. sidewalks, and pedestrian crossings). 4. Visual Points (VP): In the NAVIG system, the user positioning is based on a GPS receiver and embedded computer vision. The aim is to combine GPS positioning of the user with geolocalized visual targets detected by the embedded vision (see figure 1). Hence we defined a category of points called Visual Points (VP) that are geolocalized and tagged in the GIS database. For each object in the database, many tags are possible. For instance, a bus stop is tagged as LM because it may be detected by the user. It is also tagged as POI (it is a possible destination) and as VP (it can be detected by the embedded artificial vision module).

Fig. 1 NAVIG’s positioning module

Fig. 2 An illustration of 4 possible routes between a starting point and a destination. For each route a score is computed and the most appropriate route, including POI, LM and VP, is selected

score is calculated. Figure 3 shows a theoretical example of a selected route according to the presence of different types of proposed points. Details on the algorithm are presented in the subsequent sections.

4. ROUTE SELECTION ALGORITHM Finding ones way to an unknown destination is challenging for visually impaired persons. Figure 2 shows an example of route selection problematic. In order to reach the same destination, blind pedestrians may choose one of the four different paths represented by different colors. Some of them are shorter or have less turns but maybe less suitable for blind pedestrians (e.g. absence of pedestrian crossing). Our aim is the selection of the most appropriate route, based on user needs, and relying on the proposed classification of geographical data. We propose that an optimal route may be computed by solving the minimization problem using the Dijkstra algorithm [5], taking into consideration the defined classes of objects in a connected graph. For each section linking two nodes, a

978-89-93215-02-1 98560/10/$15 ©ICROS

4.1 Graph construction Maps can be represented as a connected graph. Each section links two nodes in both directions. To construct the graph we respect 2 rules: 1. Each origin and destination must be a node, 2. Intersection between two WAs is a node. Node numbers are randomly affected. Figure 4 illustrates a crossing (left) and his graph transformation (right). 4.2 Profit Function For each section linking two nodes, we distinguish different types of profits: • User profit (Up) depends on the presence of POI and LM in a section of the graph,

2224

as distance, width of sidewalk, number of turns, existence of pedestrian crossing, etc. From a cognitive point of view, the process of path selection in human navigation has been studied by Golledge [12]. Table 1 shows the results obtained from experiments undertaken with 32 sighted persons. Table 1 Ranking of criteria most often used by sighted person in route selection proposed by Golledge [12]. Criteria Shortest Distance Least Time Fewest Turns Most Scenic/Aesthetic First Noticed Longest Leg First Many Curves Many Turns Different from Previous Shortest Leg First

Fig. 3 Illustration of a selected route due to the presence of POIs, LMs and VP.s

Based on these criteria and brainstorming sessions with blind people and an O&M instructor, we identified the three most important factors to consider in path selection for the Blind: sidewalk (presence and width), presence of pedestrian crossing, and length of a section. Indeed, sidewalks represent the most important WA for visually impaired pedestrians. And sidewalk width is essential to allow fluidity of movement while using a white cane or a guide dog. We considered that maximal width is equal to 5 meters, and we attributed costs to sidewalk width that range from 1 to 5 (see Table 2). Crossing the road is really challenging for a Blind, and is dangerous in absence of pedestrian crossing. Hence, we penalized road crossing in general, and we added extra penalties in absence of pedestrian crossing and accessible traffic light. Sections lengths are very important, it is the first criteria used in human navigation (see [12]), so a normalization is needed. We adopted a strategy proposed by [15], which ensures that proportions between sections are maintained. The longest possible section is being attributed a cost of 5. Then the cost assigned to each possible section is proportional to its length. Finally we computed a function called cost (C) for each section of the graph (see Eq. (4)). We allocated the same weight for each section of the graph.

Fig. 4 Crossing (left) and corresponding graph transformation (right)

System profit (Sp) depends on the presence of VP in a section of the graph. To compute the overall profit of a section, we allocated relative weights for both user and system profits. Due to specificity of pedestrian navigation and especially for the Blind, greater importance is allocated to the user. Hence, user profit was considered twice as compared to the system profit (X=2 and Y=1). POIs are major cues for providing blind users with a spatial representation of the environment. LMs are important cues for helping the user to confirm that his travel is correct, but are not as useful as POIs for spatial orientation. VPs are used by the system only in order to refine the user positioning. Therefore, we put a weight of 2 on POI, 1 on LM and a reduced weight of 0.5 on VP. Profit for each section of the path is calculated with Eqs. (1) ∼ (3) with i and j representing two linked nodes.

•

P =

j X [(X ∗ U p), (Y ∗ Sp)]

Rank 1 2 3 4 5 6 7 8 9 10

(1)

i j X Up = [(2 ∗ P OI), LM ]

C(i, j) = C(SW ) + C(P C) + C(SL) (2)

Table 2 Weights attributed to sidewalks widths.

i j X Sp = (V P ∗ 0.5)

Sidewalks Width Cost

(3)

i

4.3 Cost Function Route selection should not be limited to the computation of the profit function. Several studies [13, 19] indicated other criteria that influence itinerary choice, such

978-89-93215-02-1 98560/10/$15 ©ICROS

(4)

< 5m 1

< 4m 2

< 3m 3

< 2m 4

4.4 Route selection Finally, we computed a score for each graph section with Eq (5). The optimal route is selected by solving the

2225

< 1m 5

minimization problem using Dijkstra algorithm. S(i, j) = P (i, j) − N C(i, j)

(5)

5. SIMULATION To test the performance of the proposed algorithm, a simulation was performed in an urban environment (Toulouse, France). Geolocalized data was extracted from a relational database provided by the Grand Toulouse urban community that is a partner of the NAVIG project. Map data was acquired in digital format, and additional manual preprocessing was made to create a navigable network that included WAs, LMs, POIs and VPs. Our aim was to find the most helpful route that connects origin and destination points (star and diamond symbols respectively in Fig. 5). We first constructed a connected graph that fulfills the rules defined above, with nodes 1 and 4 corresponding to origin and destination points (Fig. 6). As pedestrians can use WAs in both directions, the graph is not oriented. To reach the destination, a blind pedestrian can choose one of several possible paths. For instance, the path made of nodes [1, 2, 6, 5, and 4] is the shortest one. Alternatively, the path [1, 2, 3, and 4] may be considered as the easiest one because it contains the fewest number of turns. Of course, several other paths exist. The selected route should provide the Blind with a maximum of safety and efficiency, and should, in addition, help the Blind to create a mental representation of the surroundings.

Fig. 6 Graph representation of the area selected in Fig.5

score of each section as relative weight. Table 3 shows that the path constructed with nodes [1, 2, 6, 5, 4] presents a global score of (-12) and has the lowest global cost. Table 3 Computed score for each section of the graph (Up: User profit, Sp: System profit; P: Profit; C: cost). Section (1,2) (2,3) (3,4) (4,5) (5,6) (6,7) (6,2) (7,8) (8,9) (9,10) (10,11) (11,12) (12,13) (13,1)

Sp 0 0.5 1 0 0.5 0 0.5 0 0 0 0.5 1 0 0

P 0 10.5 17 14 4.5 0 20.5 0 0 6 0.5 1 0 10

C 5.5 8 10.75 6.75 2 5.5 13 4.75 4.25 4.25 5.75 5.75 5.25 6.5

Score 5.5 -2,5 -6.25 -7.25 -2.5 5.5 -7.5 4.75 4.25 4.25 5.25 4.75 5.25 -3.5

5.1 First prototype We designed the first prototype of the NAVIG system (see Fig. 7). It runs on a notebook with an Intel i7 820QM processor (1.73 GHz) and 4GB of memory. In this first prototype, we used satellite data from the GPS element (based on the Angeo system1 developed by the project partner NAVOCAP) with monochrome stereoscopic cam-

Fig. 5 Graph representation with objects annotation (WA, LM, POI and VP). We computed the score for each section of the graph (see Table 3). The optimal route is selected by solving the minimization problem using Dijkstra algorithm with

978-89-93215-02-1 98560/10/$15 ©ICROS

Up 0 5 8 7 2 0 10 0 0 3 0 0 0 5

2226

era system (Bumblebee, distributed by Point Grey Research) to localize VPs and improve user positioning. The complete GIS module is under development and will consist of: 1/ a digitized map of Toulouse, France, annotated with the four classes of geographical features presented here; 2/ a route selection software - including the current algorithm and vocal interaction -; and 3/ an algorithm designed to track the travelers path.

based on user needs (adapted walking areas and landmarks), the method includes visual points in route selection. VPs are geolocalized objects of the environment that may be detected by a rapid object localization software (see [21] and [22]). In the NAVIG prototype, detection of VPs by embedded cameras improves user positioning (fusion with GPS positioning), but also allows the user to reach final targets such as doors of buildings, mailboxes, ATM machines, etc. (objects of interest are presented via virtual sounds in a head-related reference frame). Interestingly, this assistive device may also help the Blind to categorize similar objects (e.g. real time currency recognition in [23]). [20] reports that an efficient spatial representation leads to a successful navigation. We suggest that points of interest localized and pointed out by the system may help blind users in creating a useful spatial representation of the surroundings. Therefore, we included points of interests in route selection algorithm. In a scenario where a blind user wants to reach a new place from a subway station, the system would mention the location of, e.g., the surrounding streets, the church, the bank, the mail office, etc. In this scenario, the user is being guided to the final destination, but he is also creating, during the current travel, a sparse representation of the environment that will be useful for future travels. This method will be implemented and evaluated in an assistive device directed towards increasing the autonomy of visually impaired users in exterior and interior environments, at familiar and unknown locations, at both small and large scale (see [18] for details). In the absence of any cognitive study in the literature, we conducted brainstorming sessions with visually impaired subjects and an O&M instructor in order to allocate costs and profits. For instance, we computed the overall profit with user profit equal to twice the system profit. This arbitrary weighting would be more significant if it was deduced from an empirical study including blind users. Finally, we are thinking about using automatic learning during navigation tasks, and weight computation based on user habit and preference. For instance, user and system profits could be dynamically modified by automatic learning of user habit and preference. Ideally, respective weights allocated to POI, LM and VP in profit function may also be dynamically modified from one user to another (for instance in a user profile). Another important point in blind pedestrian navigation is the need for multimodal route selection in the sense of including several transportation modes. Thus, a usable route selection method must integrate the bus and tramway stops, as well as the metro stations like nodes in the graph.

Fig. 7 NAVIG prototype including a GPS receiver, a stereoscopic camera and a head motion tracking device mounted on a helmet. Microphone and headphones were added for speech and audio interaction. The Notebook computer is located in the backpack.

Fig. 8 Blind user equipped with the First prototype

6. DISCUSSION In this paper, we showed that annotation of geographical data may be very helpful in route selection algorithm specifically designed for the visually impaired people. Route selection was based on solving minimization problem using Dijkstra algorithm in a connected graph. For each section, between two nodes, profits and costs were chosen based on the needs and the observed behavior of the Blind in navigation tasks. Then a score was computed for each section. Our method finally selects routes that consider peculiarities related to Blind mobility and orientation. The first type of peculiarity corresponds to adapted walking areas (e.g. large sidewalk that allows traveling with white cane or dog, presence of pedestrian crossing, etc.). The second type of peculiarity consists in finding non-visual landmarks that help Blind to confirm their own position. In a typical scenario, the system would mention that the paving of the sidewalk is going to change. The user can feel it and is confident about the path he is following. In addition to selection of the most appropriate path

978-89-93215-02-1 98560/10/$15 ©ICROS

7. ACKNOWLEDGMENTS This work was supported by the French National Research Agency (ANR) through TecSan program (project NAVIG ANR-08TECS-011) and the Midi-Pyrnes region through APRRTT program.

2227

REFERENCES [1]

[2]

[3]

[4]

[5] [6]

[7]

[8]

[9]

[10]

[11]

[12]

A. Helal, S. E. Moore, and B. Ramachandran, “Drishti: an integrated navigation system for visually impaired anddisabled,” Proceedings of the Fifth International Symposium on Wearable Computers ISWC, IEEE Computer Society, Washington, pp. 149-156, 2001. P. A. Burrough, “Principles of Geographical Information Systems for Land Resources, Assessment,” Oxford, Clarendon Press, Monographs on soil and resource surveys, No. 12, 1986. B. Mayerhofer, B. Pressl, and M. Wieser, “ODILIAA Mobility Concept for the Visually Impaired,” Proceedings of the Fifth International Symposium on Wearable Computers, linz, Austria, pp. 11091116, 2008. B.F.G. Katz, P. Truillet, S.Thorpe, and C. Jouffrais, “NAVIG: Navigation Assisted by Artificial Vision and GNSS,” Multimodal Location Based Techniques for Extreme Navigation, Pervasive, helsinki, Finland, in press. E.W. Dijkstra, “A note on two problems in connexion with graphs,” Numerische Mathematik, 1959. R. Farcy, R. Leroux, A. Jucha, R. Damaschini, C. Gregoire, A. Zogaghi, “Electronic travel aids and electronic orientation aids for blind people: technical, rehabilitation and everyday life points of view,” Conference & Workshop on Assistive Technologies for People with Vision Hearing Impairments Technology for Inclusion CVHI 2006, M.A. Hersh (ed.), Kufstein, Austria, 2006. H. Petrie, V. Johnson, T. Strothotte, A. Raab, S. Fritz and R. Michel, “MOBIC: Designing a Travel Aid for Blind and Elderly People,” Journal of Navigation, Royal Institute of Navigation, London, pp. 45-52, 1995. B. Hofmann-Wellenhof, M. Wieser and K. Legat, “Navigation: Principles of Positioning and Guidance,” Springer Verlag, Wien, 2003 Ju. Jin Sun and Ko. Eunjeong and Kim. Eun Yi, “EYECane: navigating with camera embedded white cane for visually impaired person,” Assets ’09: Proceedings of the 11th international ACM SIGACCESS conference on Computers and accessibility, Pittsburgh, Pennsylvania, USA: ACM, pp. 237-238, 2009. J. Zheng, A. Winstanley, Z. Pan and S. Coveney, “Spatial Characteristics of Walking Areas for Pedestrian Navigation,” In Third International Conference on Multimedia and Ubiquitous Engineering, IEEE, China, pp. 452-458, 2009. J. M. Loomis, R. G. Golledge, R. L. Klatzky and J. M. Speigle and J. Tietz, “Personal guidance system for the visually impaired ,” Assets ’94: Proceedings of the first annual ACM conference on Assistive technologies, New York, NY, USA, pp. 85-90, 1994. R.Golledge, “Path selection and route preference in

978-89-93215-02-1 98560/10/$15 ©ICROS

[13]

[14]

[15]

[16]

[17]

[18] [19]

[20]

[21]

[22]

[23]

2228

human navigation: A progress report” in Spatial Information Theory A Theoretical Basis for GIS, pp. 207-222, 1995. R. Golledge, R.L. Klatzky, J.M .Loomis, J. Speigle, and , J. Tietz, “A geographical information system for a GPS-based personal guidance system” International Journal of Geographical Information Science, 1998. T. V¨olkel and G. Weber, “A New Approach for Pedestrian Navigation for Mobility Impaired Users Based on Multimodal Annotation of Geographical Data,” UAHCI’07: Proceedings of the 4th international conference on Universal access in humancomputer interaction, Beijing, China, pp. 575-584, 2007. T. V¨olkel and G. Weber, “RouteCheckr: personalized multicriteria routing for mobility impaired pedestrians,” Assets ’08: Proceedings of the 10th international ACM SIGACCESS conference on Computers and accessibility, Halifax, Nova Scotia, Canada, pp. 185-192, 2008. Working Group on Mobility Aids for the Visually Impaired and Blind, Committee on Vision, “Electronic Travel Aids: New Directions for Research,” The National Academies Press, 1986. A. Brock, J.L.Vinot, B. Oriola, S. Kammoun, P. Truillet and C. Jouffrais, “ Methodes et outils de conception participative avec des utilisateurs nonvoyants,” Proceedings of the 22th French-speaking conference on HCI, Luxembourg, in press. NAVIG, http://navig.irit.fr Y. Akasaka and T. Onisawa, “Personalized Pedestrian Navigation System with Subjective Preference Based Route Selection,” Intelligent Decision and Policy Making Support Systems, pp. 73-91, 2008 J.F. Fletcher, “Spatial representation in blind children. 1: Development compared to sighted children,” Journal of Visual Impairment and Blindness, pp. 318-385, 1980. F. Dramas, B. Oriola, B.F.G. Katz, S. Thorpe, C. Jouffrais, “Designing an assistive device for the blind based on object localization and augmented auditory reality,” Assets ’08: Proceedings of the 10th international ACM SIGACCESS conference on Computers and accessibility, Halifax, Nova Scotia, Canada, pp. 263-264, 2008. F. Dramas, S. Thorpe and C. Jouffrais., “ Artificial Vision For The Blind: A Bio-Inspired Algorithm For Objects And Obstacles Detection,” International Journal of Image and Graphics, World Scientific, in press. R. Parlouar, F. Dramas, M. M-J Mace, C. Jouffrais, “Assistive device for the blind based on object recognition: an application to identify currency bills,” Assets ’09: Proceedings of the 11th international ACM SIGACCESS conference on Computers and accessibility, Pittsburgh, Pennsylvania, USA, pp. 227-228, 2009