universal navigation of web pages and efficient map exploration of preferable ... in combination with the haptic information channel help users to have a better ...
3D HapticWebBrowser: Towards Universal Web Navigation for the Visually Impaired Nikolaos Kaklanis1,2, Konstantinos Votis2 Konstantinos Moustakas2 and Dimitrios Tzovaras2 1
Department of Computing, University of Surrey Guildford, United Kingdom 2 Informatics and Telematics Institute, Centre for Research and Technology Hellas 6th Km Charilaou-Thermi Road, 57001 (PO Box 60361) Thermi-Thessaloniki, Greece {nkak, kvotis, moustak, Dimitrios.Tzovaras}@iti.gr
ABSTRACT When viewed at a macroscopic scale, an interesting emergent property of the Web is that it constantly evolves towards being the most important entry point for information production and consumption by everyone. This purpose is defeated right if a single user with a disability cannot access a given piece of information on the Web. However, information on the Web is graphically-orientated and in most cases visually impaired users have very restricted access and find it difficult to recognize this kind of visual representation.For visually impaired people and especially for blind users alternative information presentation ways must be found, which would replace visual information. 3D HapticWebBrowser, is a free open source web browser to address the problems of visually impaired in accessing the Web. Issues of multimodal interaction and haptic technologies enable universal navigation of web pages and efficient map exploration of preferable 2D maps.
Categories and Subject Descriptors K4.2 [Social Issues]: Assistive technologies for persons with disabilities; H.5.2 [Information Interfaces and Presentation]: User Interfaces
General Terms Design, Human Factors
Keywords Haptic navigation, multimodal maps, haptic interaction, blind users, visually impaired users, virtual environment
1.
INTRODUCTION
In the past few years the scientific community turned its interest to non-visual forms of representation, using the haptic and the auditory information channels. Thus, haptic interfaces have been around for more than a decade. However, only recently a substantial decrease in price has allowed a strong diffusion of these technologies into the accessibility domain. The adoption of haptic devices into people with disabilities and especially to visually impaired users, who meet huge obstacles in accessing the Web, can efficiently enhance the accessibility of the web content and especially the support of symbolic information, like Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. W4A2010 - Challenge,/ April 26-27, 2010, Raleigh, USA. Co-Located with the 19th International World Wide Web Conference.
Copyright 2010 ACM 978-1-4503-0045-2 ...$5.00
maps which are totally inaccessible by existing audio rendering engines or other assistive technologies. So, even the best audio rendering engines still suffer from some intrinsic limitations, such as sequential navigation, long processing time and difficult navigation within a long page or across web pages. Additionally, audio rendering only works when HTML is well-formed and is in compliance with appropriate accessibility guidelines (e.g. WCAG 2.01, section 5082, etc.). Concerning maps navigation, the introduction of Scalable Vector Graphics (SVG3) has been also proposed for partially support of the visually impaired. The goal of the implemented 3D HapticWebBrowser is primarily to present a framework that allows haptic navigation through the Internet in addition to the haptic exploration of conventional 2D maps found on the web. The supported haptic interaction allows the system users to perceive the structure of the virtual environment, which corresponds to the structure of the visited web site as it can be presented in a typical web browser. This is very important because it gives to the visually impaired the opportunity to navigate and interact through the internet in a way that makes navigation really accessible. Haptic navigation, on contrary to audio rendering that imposes sequential navigation, enables free movement within the 3D scene. Consequently, the time required to switch from one screen object to another significantly may be reduced. Some studies showed that visually impaired and especially blind users may be able to recognize simple objects [1] using a haptic device such as the Phantom4. However, when the environment becomes more complex, the need for additional information becomes imperative. Moreover, auditory channels (e.g. earcons) in combination with the haptic information channel help users to have a better perception of the size of a 3D virtual object as well as navigating efficiently in 2D maps.
2. ENABLING 3D HAPTIC WEB NAVIGATION The 3D HapticWebBrowser enables users to quickly access web content. Thus, users can quickly navigate between HTML elements that are being transformed into “hapgets” (hapticallyenhanced widgets) [2], which are 3D widgets with haptic feedback. Each hapget contains a 3D representation with haptic feedback, a description, an earcon and a relevant haptic icon. 1
http://www.w3.org/TR/WCAG20/
2
http://www.section508.gov/
3
http://www.w3.org/Graphics/SVG/
4
http://www.sensable.com/haptic-phantom-omni.htm
The haptic icon in conjunction with the earcon and the unique 3D shape of the hapget help the visually impaired user to identify the type (e.x. button, selection list, hyperlink, image, etc.) of the object that is being “touched” using the haptic device. Each hapget has also concrete haptic characteristics such as: static friction, dynamic friction, stiffness, and some haptic effects (buzz effect, constraint effect, and inertia effect) that make the identification more easy and efficient. A speech synthesis and a speech recognition engine have also been integrated to allow human-computer interaction via natural language. Figure 1 illustrates a screenshot of the application and depicts the correspondence between the hapget representing a map (the red object at the left) and the original map as it can be shown in a common web browser.
(Optical Character Recognition) as well as a TTS7 (Text-toSpeech) module is being used. Since the map environment is static, a force-field haptic rendering method is supported by the proposed framework that generates a 3D force field from visual data. While the user is navigating in the virtual environment, the force value that corresponds to the user’s current position is rendered through the haptic device. The typical spring-damper model [4] is being used to calculate the force feedback’s magnitude. When the user is in contact with a hapget whose alternative text or URL show that it is a map, the user can choose to step into map exploration mode (figure 3).
Figure 3. Multimodal map exploration.
Figure 1. The 3D haptic Web browser environment. Moreover, supporting the universal design of UIs, the 3D HapticWebBrowser generates automatically a UsiXML5 document describing the 3D scene (including the position, the description and all the other special characteristics of the hapgets) for each visited web page.
3. HAPTIC EXPLORATION OF 2D MAPS The 3D HapticWebBrowser has also the ability to provide 2D map exploration. The envisaged map image analysis that leads to the generation of a multimodal (audio, haptic) map focuses on the extraction of the street names and road structure. Figure 2 presents all the steps of the map image analysis, which contain: (a) road names identification, (b) road network structure identification and (c) 3D map model construction.
An initial evaluation has been performed with the participation of four visually impaired users. During system’s evaluation, the participants were asked to perform web navigation in addition to 2D maps exploration for route planning purposes and to explore unknown environments. All of them were able to successfully complete these tasks and all of them were enthusiastic with the haptic interaction of the system.
4.
FUTURE WORK
Ongoing work is currently being done in several components in the system, including the extension of the system to cope directly with Google 2D maps as requested by our users. A number of interesting questions are still to be addressed in the problem of location map accuracy and in the optimization of the adopted OCR technologies. Moreover, additional guidance features such as distances, auditory signals and warnings (e.g walk close in cross roads, etc.) will be included to an updated version.
5.
ACKNOWLEDGMENTS
This work is partially funded by the EU FP7 project AEGIS, contract no. 224348
6.
Figure 2. Steps of the multimodal map generation. To support the extraction of the road names, an erosion filter is being applied to the primary image and then the produced image is subjected to threshold dithering to two colours. A color inversion and then a region growing algorithm are being applied in order to get the image segments containing the road names. A chromatic identity is being given to each segment to enable the identification of each distinct road (Fig. 2b). By knowing the position of each image segment containing a road name in conjunction with the angle from the X axis and the chromatic identity of each road, the graph of the road network structure is generated as illustrated in Fig. 2c. The 3D representation of the map is generated as a grooved line map [3] (Fig. 2e). Finally, for the transformation of the road names into speech, an OCR6 5
6
REFERENCES
[1] Magnusson, C., Rassmus-Gröhn, K., Sjöström, C., Danielsson, H.: Navigation and recognition in complex haptic virtual environments – reports from an extensive study with blind users. In: Eurohaptics 2002, Edinburgh, UK, July 8-10 (2002) [2] Kaklanis, N., Gonzalez Calleros, J.M., Vanderdonckt, J., Tzovaras, D.: Hapgets, Towards Haptically-enhanced Widgets based on a User Interface Description Language, Proc. of Multimodal interaction through haptic feedback (MITH) Workshop (Naples, May 28-30, 2008) [3] R. Ramloll, W. Yu, S. Brewster, B. Riedel, M. Burton, and G. Dimigen, “Constructing sonified haptic line graphs for the blind student: First steps.”, in ACM conference on Assistive technologies, (Arlington, USA), 2000 [4] G.C. Burdea, Force and Touch Feedback for Virtual Reality, Wiley-Interscience, 1996
http://www.usixml.org/ The OCR that has been used (http://code.google.com/p/tesseract-ocr)
is
tesseract
v.2.04
7
The TTS that has been (http://espeak.sourceforge.net/)
used
is
espeak
v.1.42.04
