Development of Dialogue Systems for a Mobility ... - ACM Digital Library

Development of Dialogue Systems for a MOlbility Aid for Blind People: Initial Design and Usability Testing Thomas Strothotte, Stefi Fritz, Raker Michel and Andreas Raab Institut ftir Simulation und Graphik Universitat Magdeburg Universitatsplatz 2 D-39106 Magdeburg Germany {tstrlsteffilmichellraab} @isg.cs.tu-magdeburg.de

Helen Petrie and Valerie Johnson Sensory Disabilities Research Unit Psychology Division University of Hertfordshire Hatfield AL10 9AB United Kingdom phone: +44 707 284629 {H.L.PetrielV.Johnson} @herts.ac.uk

L.ars Reichert and Axe1 &halt Department of Mathematics and Computer Science Free University of Berlin Takustr. 9 D-14195 Berlin Germany phone: +49 30 838 75 100 {reichertlschalt)@inf.fu-berlin.de are obstacles on the pavement in front of them or branches overhanging their path. Neither can they look into the distance and see by the church spire the direction they need to follow. The long cane and the guide dog are mobility aids which give assistance for the first group of problems, which constitute part of the task of navigating through the immediate environment or micro-navigation (Petrie, 1995). However, these traditional primary aids give no assistance for the second group of problems, those related to navigating through the more distant environment or macro-navigation. Many electronic travel aids (El’As) have been developed since the 1960s (see surveys in Brabyn, 1985, 1995; Farmer, 1980), but these have all addressed the micro-navigational problem. With the advent of computer-based geographical information systems (GIS) and the Global Positioning System (GPS), the technology is becoming available to develop a travel aid for blind people which will address the problems of macro-navigation. The MoBIC Project, funded by the European Union, is developing such an aid. Several other projects are also exploring the use of GPS and GIS to assist blind travellers (e.g. Fruchterman, 1985; Loomis, 1994) although the approaches and emphases differ from that adopted by the MoBIC system. The Arkenstone system (Fruchterman, 1995) concentrates on using standard map data which is widely available, but not augmented with information specifically for blind people and unlike MoBIC does not provide complex route planning mechanisms (see the next section). The system under development by Loomis provides a virtual acoustic environment for travellers, guiding them to buildings and other objects in the environment by sound signals which appear to come from the location of the building or object. The components of navigation which are explored by Loomis’ system are in many ways complementary to those explored by the MoBIC system. The MoBIC travel aid is intended to be complementary to the primary mobility aids of the long cane and guide dog. Its

ABSTRACT This paper presents a new travel aid to increase the independent mobility of blind and elderly travellers. This aid builds on the technologies of geographical information systems (GIS) and the Global Positioning System (GPS). The MoBIC Travel Aid (MoTA) consists of two interrelated components: the MoBIC Pre-journey System (MOPS) to assist users in planning journeys and the MoBIC Outdoor System (MOODS) to execute these plans by providing users with orientation and navigation assistance during journeys. The MoBIC travel aid is complementary to primary mobility aids such as the long cane or guide dog. Results of a study of user requirements, the user interface designs, and the first field trial, currently being conducted in Berlin, are presented. KEYWORDS: visually disabled users; mobility and navigation; GPS; GIS; user trials. INTRODUCTION Travel for blind people poses many more problems than it does for sighted people. Blind pedestrians cannot see if there Permission to make digital/hard copies of all or part of this material for personal or classroom use is granted without fee provided that the copies are not made or distributed for profit or commercial advantage, the copyright notice, the title of the publication and its date appear, and notite is given that copyright is by permission of the ACM, Inc..To copy otherwise,

to republish, to post on servers or to redistribute to lists, requires specific permission and/or fee. ASSETS ‘96, Vancouver, British Columbia Canada Q 1996 ACM 0-89791-776-6/96/04..$3.50

139

main purpose is to provide information about the more distant environment. Sighted travellers use all kinds of information from the distant environment to navigate. Examples include being able to see across intersections, read street signs and see distant landmarks such as tall buildings. It is this range in which the MoBIC system will show its importance and provide the blind traveller with information until now unavailable without vision. Indeed, the MoBIC system has the potential to give a traveller more accurate macro-navigation than is possible by an unassisted sighted traveller, because the system can calculate routes beyond the capacity of sighted travellers and have details of timetables and other information which sighted travellers often would not have in their head. The MoBIC system will also provide some information relevant to The MoBIC Travel Aid (MoTA) can be seen as a set of cooperating tools designed to cover all aspects of travelling and journey preparation for blind people. It consists of two basic parts: the MoBIC Pre-Journey System (MOPS) and the MoBIC Outdoor System (MOODS). The MoBIC Pre-Journey System (MOPS) allows travellers to plan a journey by accessing information about the environment in which the journey will take place (e.g. maps, public transport information, opening hours of facilities). This is done on a normal personal computer (PC) with, if necessary, input/output devices adapted to the special needs of users. In addition, the PC may have telecommunication facilities to give remote access to special databases (e.g. centrally stored timetable information). The MOPS system is able to provide blind travellers with access to information sources that are usually restricted to One major source is map information. sighted people. Traditionally maps have been a particular problem for blind travellers as they must be produced in tactile form and may be difficult to use. The MoBIC system uses digital maps to make information available to blind travellers. In investigating different sources for digital maps, the MoBIC project has found that no single source is able to fulfill the requirements for blind and elderly travellers. To assist blind or elderly people in travelling, maps need to be augmented with information of particular relevance to the requirements of these groups (e.g. type of surface underfoot, access routes without many steps, entrances usable by wheelchairs). It is therefore necessary to exploit several sources of map information: data for vehicle navigation, maps from the ordnance survey etc. The features from each source relevant for the current purposes are extracted and stored in a MOPS database. In addition to maps, there are other types of information normally available in print only which are needed for optimal trip planning, for example information about public transport facilities and timetables. It is envisaged that MOPS will allow travellers to gain remote access to databases offering these types of information. The MOPS system assists travellers in exploring information about the localities to which they wish to travel. A Dialogue Management System deals with the special needs different users may have by employing a user model and a knowledge base. The user model consists of facts about a user with respect to relevant criteria (e.g. degree and kind of disability of the person, use of other mobility aids). The knowledge base provides rules about how this information is used to ensure the best assistance for travellers. With the assistance of the MOPS system, users can plan their trip to a specific destination. Based on the user model and criteria which the user may specify (e.g. the shortest route, the safest

micro-navigation, such as the type of surface underfoot and the likelihood of certain obstacles in the vicinity (e.g. lampposts, parking meters, trees). The paper is organized as follows. The next section will present an overview of the MoBIC Travel Aid and its two main components. Then the user requirements study and some of the results which were obtained will be presented. Next the implications of the user requirements for the design of the interfaces for the MoBIC system will be discussed. Finally, the first results of a field trial in Berlin will be presented. THE

MOBIC

TRAVEL

AID

route or a route without steps) the system will derive suitable routes and give recommendations to the traveller. The design of MOPS has placed special emphasis on the way information normally accessible only by sighted people is prepared and presented to the different user groups using the system. With appropriate assistance users can conveniently plan their route and store it. This journey plan can then be transferred to the MOODS system for use while travelling. The MoBIC Outdoor System (MOODS) is the traveller’s assistant during the journey. It takes the journey plan and assists travellers in reaching their goal by matching the information in its GIS against the traveller’s position provided by a GPS receiver, augmented by a differential signal, which are integrated into the MOODS system. A Trip Management System advises the traveller by giving directions, warnings and other relevant information. If travellers deviate from their intended route, the system can give a warning. If this deviation happens accidentally the MOODS system will assist travellers in finding their way back to the planned route. However, if it happens by choice the system will be able to find the planned route again when required. For this purpose, the MOODS system has the GIS on hard disk or accessible via remote database. The advantage of this approach is that a single traveller who is equipped with a travel aid is not dependent on any infrastructure in the environment (e.g. talking signs or beacons; see Brabyn, 1995; Stephens, 1995). A critical factor in the acceptability of the MoODS system is the size and weight of the unit. Blind or elderly travellers will need to be able to carry the unit in addition to their usual travel accessories (e.g. briefcase, shopping bag) and any other mobility aids (e.g. long cane or walking stick). Bearing these factors in mind, the MoODS unit was designed to be easily carried in a shoulder bag or in the pockets of a special jacket which leaves both hands free. It consists of a hand-held computer integrating a GPS receiver, compass, and rnobile telecommunication facilities along with appropriate input/output devices (e.g. speech output via earpiece, small keypad for input). The equipment is designed as a series of modules to enable updating of individual modules or substitution of other modules for different user groups. To facilitate this procedure, the interfaces between separate components and between the hardware components and the software are fixed. The MoBIC system combines massproduced components with adaptations to the particular needs of users if required, to ensure the most cost-effective solution. At present, the outdoor system weighs about 6 kilograms, carried in the pockets of a specially designed jacket. Most of the weight is due to the batteries, which we expect to be reduced with the advent of new hardware in the next year.

140

The telecommunication facility is used for two purposes. One purpose is to enable access to remote databases during travel. This may be useful if there is a need for more information when travelling which was not included in the original journey plan. The other purpose is to provide the ability to summon immediate assistance in the case of emergency. A call could be placed to the police, medical services or a special service centre for blind or elderly travellers. USER

a user. These preferences may be determined by a certain disability such as having to use a wheelchair or simply from a desire not to use buses. The opportunity to add personal comments and reminders to the route description (e.g. shopping to collect as well as the name or the room number of persons to visit) was also highly favoured by all interviewees. Information to be provided by the MoTA should contain all possible route details but such information should be accessible at different levels of detail. A summary of the information to be provided, in terms of essential, desirable and ideal is presented in Table 1, below. For the MoODS system all potential users interviewed expressed their desire to be able to control the flow of information and to be able to request more or less, as they felt they needed at the time. Some users will require detailed guidance whereas others simply need reassurance that they are where they expect to be and are headed towards their intended destination. Similarly, the same user will require different levels of information in different situations. If unsure about current location more detail may be required, whereas if a user is quite confident about the location and the route, a constant flow of instructions may be distracting. People know their limits in terms of attention and know what needs to be accentuated and when. The point was made, by one potential user, that sighted people choose what they attend to and that blind and partially sighted people should also have this choice. It was therefore recommended that the MoODS system should incorporate a number of different commands to control the level of detail provided and the following levels were suggested. l Level 1. - Basic information (to check you are still on course) - Direction of travel, grade of road and nearest crossing, any known obstacles. l Level 2 - Detailed information - To include the type of detail selected for inclusion in the pre journey plan e.g. shops, public buildings etc. l Level 3 - Transport information - Nearest bus stop, rail or tube station taxi rank and telephone.

REQUIREMENTS

The first step in designing the MoTA was an extensive study of user requirements in which potential users of the system were interviewed about their current levels of mobility, problems which they currently encounter in travelling independently (some of which might be addressed by the MoTA system and others which might not) and their opinions and ideas for the MoTA. In addition, interviews were conducted with mobility officers and others involved in the training and rehabilitation of visually disabled people. The experience and expertise of those taking part were drawn upon to ascertain the nature of current training practices in the United Kingdom and to seek opinions regarding the difficulties encountered by visually disabled travellers and the ways in which the MoTA may be helpful. Participants in both groups (potential users and mobility trainers) were enthusiastic about the possibility of being able to plan a route at home before embarking on a journey and being able to explore a new area before actually visiting it. The ability to place certain constraints on the planning of a route was also highly favoured. That is, users may request the shortest route from the station to the bank or the safest route from their home to a friend’s house. MOPS therefore has to decide which route may be considered as being safe (e.g. routes with no clockwise crossings or routes avoiding busy roads). Further constraints may result from the personal preferences of

Table 1: Information to be presented in the MOPS component Essential Route Planning

InteractIon Levels

Desirable

Basic maps, routes including Street names Distance information Kno.wn hazards Road gradings

Option to add more detail on several different levels, e.g. Shops Public buildings Road crossings Public transport Then more detail concerning these e.g. Specialist shops Nearest Church Ticket Ofice User should dictate the User should be able to level of detail incorpo- request area maps or rated in any particular specific routes Shortest route journey plan Safest route 141

Ideal Up to date information concerning transient obstacles, e.g. Roadworks Scaffolding Diversions

User should be able to edit the journey plan with own comments and memos

on-journey route planning, address inquiries or error reaction. A number of the secondary commands therefore require complex (parameterised) instructions. To assist users with these commands on-line help is available. Examples of primary and secondary commands are given in Table 2, below. Output from the MOODS obviously blocks environmental sounds to some extent which can be dangerous for blind travellers. Therefore the MoODS interface allows the user to control the quantity of output. The output range extends from a level in which the system provides continuous navigational instructions through to a level in which everything but warnings are suppressed. In addition, output can be suppressed at any time by pressing a mute button. The interface to the MOPS system consists of the standard PC cursor keys or a touch tablet with a tactile grid or map overlay for exploration of the digital map and synthetic speech or Braille for output. In MOPS users may position themselves on any the area of the map which they would like to explore. To build up a mental image of the environment they can move through this virtual area giving directions concerning where they would like to proceed. Figure 1 shows an example of the visual presentation of the digital map information. The system reacts by verbally describing a user’s position and selected features of the environment. These features depend upon the needs of the particular user. Blind people, for example, may wish to learn about potential orientation cues and obstacles on their route.

In discussing output options for the MOODS, participants were strongly against the idea of having to wear A large number of participants expressed headphones. worries about headphones blocking out environmental sounds which are vitally important for them in negotiating the environment. A number of participants suggested that a device something like a mobile phone, which one could bring to the ear when one needed information would be preferable to headphones.

IMPLICATIONS FOR THE INTERFACE THE MOBIC TRAVEL AID

DESIGN OF

The design of the interfaces for the MoTA is based on the results of the user requirements study. These requirements imposed a number of constraints on the development of the hardware and software. The interface to MOODS consists of a small keypad for input and synthetic speech output. As the keypad has only a limited number of keys the input commands have been grouped into two classes. One class consists of commands that are essential for the use of MOODS. These commands are called “primary” commands and are executable by a simple action, e.g. one button press. This easy input enables elderly and less advanced computer users easy access to MOODS. Examples for primary commands are: “Where am I?“ or “How do I have to proceed to reach my destination?“. Secondary commands on the other hand provide additional features to increase the flexibility of MOODS, e.g.:

Table 2: Examples of MOODS commands

me! Where is the next “object”?

secondary

object

What is the estimated travel time to the

secondary

destination? Help on commands

object/ position

secondary

position of object, route leading

to it estimated travel time help text

142

Figure 1: Example of the visual presentation of the digital map information

The way these verbal descriptions are given is similar to the first adventure games for home computers that used text output, e.g. “You have entered the hall of the castle. You are looking south. In front of you are stairs leading upwards. To your right is a torch. To your left is a door.” In this game the player could now use the up arrow cursor key to climb the stairs to the next floor or turn left with the left arrow cursor key to leave the hall. One particular problem a game player had to solve was to build up a mental map of the castle on the basis of this exploration with the cursor keys. The use of a touch tablet for exploration of digital maps may help blind people gain a better understanding of the spatial layout of an area. A touch tablet is an input device with a flat surface that responds to touch. Users can press a point on the surface and receive information. As in the case of cursor key exploration the selected position is described verbally. But in this case, the system does not know the orientation of a user. Thus, the description may not make use of relative directions, like “to your left” or “behind you”. Instead, the system has to use compass directions or similar absolute coordinate systems to present the orientation information. To enhance the usability of a touch tablet a tactile grid or overlay may be added. The overlay will present information about the geometry and the topology of the area. Thus users will obtain both tactile and verbal information about the area. Careful evaluation of such a component is necessary to determine the extent to which this helps in creating a mental map of the area.

FIRST

FIELD

TRIALS

The first field trial of the MOPS and MOODS components is currently being conducted in Berlin. Results from the trial are not yet available for thorough statistical analysis, but initial qualitative results from the trial participants are promising. This section will describe the disign and first impressions from the trial. To introduce participants to the MoBIC system, a series of 11 tutorials has been devised, organized into three levels of difficulty. As the Berlin field trial is using only the first MOPS and MOODS prototypes (both in terms of hardware configurations and software) and its main purpose is to test the feasability of the prototypes, only the first five tutorials are being used by participants. The trial participants are provided with comprehensive information in Braille. This includes a basic introduction to the MoBIC system and reference copies of the menus for the MOPS software and the tutorials themselves. In the tutorials currently being used the test persons are taught: For the MOPS system: - how to explore an area, - how to plan a journey, - how to explore the planned journey, for the MOODS system:

143

- how the system works, - how to make a jouney using the system.

ACKNOWLEDGEMENTS

The six test participants in the Berlin trial are aged between 25 and 61 years, four are totally blind, the other two are partially sighted with the ability to recognize outlines. Five of them need a primary mobility aid (long cane) for travel purposes, one of the partially sighted persons uses no primary mobility aid at all. Two of the participants have no experience with computers, the others use computers both in their work and privately. Three use Braille-based computer systems, the other a synthetic speech system. The test area for the field trial is the campus of the Free University of Berlin. It is a relatively open area with mostly two storey buildings, a lot of trees and an irregular street layout. The MOPS system hardware for the trial consists of an Intel Pentium computer with a standard keyboard and an Infovox speech synthesizer. The MOODSsystem hardware is basedon a PC104 credit card sized computer and includes a GPS receiver and a cellular phone to receive a differential GPS signal. It also includes a synthetic speech output card, a compassand a special keypad for input. Each participant works closely with a trainer who assists them in understanding the instructional material and gives help on the trials whenever it is needed. Trial sessions last between 1 l/2 and 2 hours and are recorded on video; in addition, computer log files of all interactions with the system are generated. Trainers also keep a written protocol of each session. So far the participants have worked through the first three tutorials which cover the MOPS system and are about to start to use the MOODS system. Their first comments suggest that the participants are enjoying working with the system. Like the participants in the user requirements interviews the trial participants think the MoBIC system has great potential and even the MOPS system as a stand-alone system should be useful to blind people. One participant commented that by exploring a route two or three times using MOPS, she was sure she would find her route easily without the MOODS system. Participants appreciated the fact that they could control the flow of information from the system. At any point on the route they can request additional information, can request that the information is repeated or suppress information output Finally, important feedback has been instantaneously. obtained from participants for improvements to the MoBIC system. For example, providing exact distance information (e.g. “After 132 meters on your right, there is the Institute of Computer Science.“) may not be necessary or particularly useful as some participants have little experience of judging distance. CONCLUSIONS

A travel aid to increase the independent mobility of blind and elderly travellers has been developed on the basis of extensive studies of user requirements. A field trial of the first prototypes of the MOPS and MoODS systems is being conducted in Berlin at present and will be completed in November, 1995. This will allow for further development of the systems before a more extensive field trial is conducted in Birmingham starting in February 1996. 144

The work described in this paper is being carried out within the framework of the MoBIC Project (TP 1148) supported by the Technological Initiative for Disabled and Elderly (TIDE) Programmeof the Commission of the European Union (DGXIII). We would like to thank our project partners: Uppsala University, Sweden;University of Birmingham, UK.; Royal National Institute for the Blind,UK.; British Telecom, UK; and F.H. PapenmeierGmbH, Germany. We would also like to thank the many people who participated in the MoBIC user requirements study and the Berlin field trial for their time and assistance. REFERENCES

Brabyn, J. (1985). A review of mobility aids and means of assessment.In D.H. Warren and E.R. Strelow (Eds.),&.&st& al sensmg for the blind. Boston: Martinus Nijhoff. Brabyn, J. (1995). Orientation and navigation systemsfor the blind: an overview of different approaches. In J.M. Gill and H.Petrie (Eds.), Proceedings of the Conference on Orientation and Navigation Svstems for BltnQ Persons, Hatfield, UK. 1 - 2 February. London: Royal National Institute for the Blind. Farmer, L.W. (1980). Mobility devices. In R.L. Welsh and B.B. Blasch (Eds.), &un&tions of orientation and mobility, New York: American Foundation for the Blind. Fruchterman, J.(1995). Arkenstone’s orientation tools: Atlas Speaks and Strider. In J.M. Gill. and . H.Petrie (IEds.), Proceedingsof NaVig&iQJl Svstems for Blind Persons, Hatfield, UK. 1 - 2 February. London: Royal National Institute for the Blind. Loomis, J.M., Golledge, R.G., Klatzky, R.L., Spiegle, J.M. and Tietz, J. (1994). Personal guidance system for the visually impaired. Proceedings of ASSETS ‘94: First Annual ACM Conference on Assistive Technolopies, Los Angeles. October 31 - November 1. New York: ACM Press. Petrie, H. (1995). User requirementsfor a GPS-basedtravel aid for blind people. In J.M. Gill and H.Petrie (Eds.), -dings of the Conference on Orientation and Navigation Svstems Persons, Hatfield, UK. 1 - 2 February. London: Royal National Institute for the Blind. Stephens, R. (1995). The OPEN Project. In J.M. Gill and H.Petrie (Eds.), ProceedinPsof the Conference on Q&ntti and Navigation Svstemsfor Blind Persons, Hatfield, UK. 1 - 2 February. London: Royal National Institute for the Blind.