UNCONSTRAINED WALKING PLANE TO VIRTUAL ENVIRONMENT FOR SPATIAL LEARNING BY VISUALLY IMPAIRED Kanubhai K. Patel1, Dr. Sanjay Kumar Vij2 School of ICT, Ahmedabad University, Ahmedabad, India,
[email protected] 2 Dept. of CE-IT-MCA, SVIT, Vasad, India,
[email protected]
1
ABSTRACT Treadmill-style locomotion interfaces for locomotion in virtual environment typically have two problems that impact their usability: bulky or complex drive mechanism and stability problem. The bulky or complex drive mechanism requirement restricts the practical use of this locomotion interface and stability problem results in the induction of fear psychosis to the user. This paper describes a novel simple treadmill-style locomotion interface that uses manual treadmill with handles to provide needbased support, thus allowing walking with assured stability. Its simplicity of design coupled with supervised multi-modal training facility makes it an effective device for spatial learning and thereby enhancing the mobility skills of visually impaired people. It facilitates visually impaired person in developing cognitive maps of new and unfamiliar places through virtual environment exploration, so that they can navigate through such places with ease and confidence in real. In this paper, we describe the structure and control mechanism of the device along with system architecture and experimental results on general usability of the system. Keywords: assistive technology, blindness, cognitive maps, locomotion interface, Virtual learning environment.
1
INTRODUCTION
Unlike in case of sighted people, spatial information is not fully available to visually impaired and blind people causing difficulties in their mobility in new or unfamiliar locations. This constraint can be overcome by providing mental mapping of spaces, and of the possible paths for navigating through these spaces which are essential for the development of efficient orientation and mobility skills. Orientation refers to the ability to situate oneself relative to a frame of reference, and mobility is defined as “the ability to travel safely, comfortably, gracefully, and independently” [7, 18]. Most of the information required for mental mapping is gathered through the visual channel [15]. As visually impaired people are handicapped to gather this crucial information, they face great difficulties in generating efficient mental maps of spaces and, therefore, in navigating efficiently within new or unfamiliar spaces. Consequently, many visually impaired people become passive, depending on others for assistance. More than 30% of the blind do not ambulate independently outdoors [2, 16]. Such assistance might not be required after a reasonable number of repeated visits to the new space as these visits enable formation of mental map of the new space subconsciously. Thus, a good number of researchers focused on using technology to simulate
visits to a new space for building cognitive maps. Although isolated solutions have been attempted, no integrated solution of spatial learning to visually impaired people is available to the best of our knowledge. Also most of the simulated environments are far away from reality and the challenge in this approach is to create a near real-life experience. Use of advanced computer technology offers new possibilities for supporting visually impaired people's acquisition of orientation and mobility skills, by compensating the deficiencies of the impaired channel. The newer technologies including speech processing, computer haptics and virtual reality (VR) provide us various options in design and implementation of a wide variety of multimodal applications. Even for sighted people, such technologies can be used (a) to enhance the visual information available to a person in such a way that important features of a scene are presented visibly, or (b) to train them through virtual environment leading to create cognitive maps of unfamiliar areas or (c) to get a feel of an object (using haptics) [16]. Virtual Reality provides for creation of simulated objects and events with which people can interact. The definitions of Virtual Reality (VR), although wide and varied, include a common statement that VR creates the illusion of participation in a synthetic environment rather than
going through external observation of such an environment [5]. Essentially, virtual reality allows users to interact with a simulated environment. Users can interact with a virtual environment either through the use of standard input devices such as a keyboard and mouse, or through multimodal devices such as a wired glove, the Polhemus boom arm, or else omni-directional treadmill. Even though in the use of virtual reality with the visually impaired person, the visual channel is missing, the other sensory channels can still lead to benefits for visually impaired people as they engage in a range of activities in a simulator relatively free from the limitations imposed by their disability. In our proposed design, they can do so in safe manner. We describe the design of a locomotion interface to the virtual environment to acquire spatial knowledge and thereby to structure spatial cognitive maps of an area. Virtual environment is used to provide spatial information to the visually impaired people and prepare them for independent travel. The locomotion interface is used to simulate walking from one location to another location. The device is needed to be of a limited size, allow a user to walk on it and provide a sensation as if he is walking on an unconstrained plane. The advantages of our proposed device are as follows: • It solves instability problem during walking by providing supporting rods. The limited width of treadmill along with side supports gives a feeling of safety and eliminates the possibility of any fear of falling out of the device. • No special training is required to walk on it. • The device’s acceptability is expected to be high due to the feeling of safety while walking on the device. This results in the formation of mental maps without any hindrance. • It is simple to operate and maintain and it has low weight. The remaining paper is structured as follows: Section 2 presents the related work. Section 3 describes the structure of locomotion interface used for virtual navigation of computer-simulated environments for acquisition of spatial knowledge and formation of cognitive maps; Section 4 describe control principle of locomotion device; Section 5 illustrates the system architecture; while Section 6 describe the experiment for usability evaluation, finally Section 7 concludes the paper and illustrates future work. 2
The string walker [12]. The basic idea used in these approaches is that a locomotion interface should cancel the user’s self motion in a place to allow the user to move in a large virtual space. For example, a treadmill can cancel the user’s motion by moving its belt in the opposite direction. Its main advantage is that it does not require a user to wear any kind of devices as required in some other locomotion devices. However, it is difficult to control the belt speed in order to keep the user from falling off. Some treadmills can adjust the belt speed based on the user’s motion. There are mainly two challenges in using the treadmills. The first one is the user’s stability problem while the second is to sense and change the direction of walking. The belt in a passive treadmill is driven by the backward push generated while walking. This process effectively balances the user and keeps him from falling off. The problem of changing the walking direction is addressed by [1, 6], who employed a handle to change the walking direction. Iwata & Yoshida [13] developed a 2D infinite plate that can be driven in any direction and Darken [3] proposed an Omni directional treadmill using mechanical belt. Noma & Miyasato [17] used the treadmill which could turn on a platform to change the walking direction. Iwata & Fujji [9] used a different approach by developing a series of sliding interfaces. The user was required to wear special shoes and a low friction film was put in the middle of shoes. Since the user was supported by a harness or rounded handrail, the foot motion was canceled passively when the user walked. The method using active footpad could simulate various terrains without requiring the user to wear any kind of devices. •
3
STRUCTURE INTERFACE
OF
LOCOMOTION
RELATED WORK
We have categorized the most common virtual reality (VR) locomotion approaches as follow: • Omni-directional treadmills (ODT) [3, 8, 14, 4], • The motion foot pad [10], • Walking-in-place devices [19], • actuated shoes [11], and
Figure 1: Mechanical structure of locomotion interface. There are three major parts in the figure: (a) A motor-less treadmill, (b) mechanical rotating base, and (c) block containing Servo motor and gearbox to rotate the mechanical base.
his balance. 4
Figure 2: Locomotion interface. As shown in Figure 1 and 2, our device consists of a motor-less treadmill resting on a mechanical rotating base. In terms of its physical characteristics, our device’s upper platform (treadmill) is 54” in length and 30” wide with an active surface 48” X 24”. The belt of treadmill contains mat on which 24 stripes along the direction of motion, at a distance of 1” between two stripes. Below each stripe, there are force sensors that sense the position of feet. A typical manual treadmill passively rotates as the user moves on its surface, causing belt to rotate backward as the user moves forward. Advantages of this passive (i.e. non-motorized) movement are: (a) to achieve an almost silent device with negligible-noise during straight movement, and (b) the backward movement of treadmill is synchronized with forward movement of user leading thereby jerk-free motion. (c) Also in case of the trainee stopping to walk as detected by non-movement of belt, our system assists and guides the user for further movement. The side handle support provides the feeling of safety and stability to the person which results in efficient and effective formation of cognitive maps. Human beings subconsciously place their feet at angular direction whenever they intend to take a turn. Therefore the angular positions of the feet on the treadmill are monitored to determine not only user’s intention to take a turn, but also the direction and desired angle at granularity of 15o. Rotation control system finds out angle through which the platform should be turned, and turns the whole treadmill with user standing on it, on mechanical rotating base, so that the user can place next footstep on the treadmill’s belt. The rotation of platform is carried out using a servo motor. Servo motor and gearbox are placed in lower block which is lying under the mechanical rotating base. Our device also provides for safety mechanism through a kill switch, which can be triggered to halt the device immediately in case the user loses control or loses
CONTROL PRINCIPLE LOCOMOTION DEVICE
OF
Belt of treadmill of device rotates in backward or forward direction as user moves in forward or backward direction, respectively, on the treadmill. This is a passive, non-motorized, movement of treadmill. The backward movement of belt of treadmill is synchronized with forward movement of user leading thereby non-jerking motion. This solves the problem of stability. For maneuvering, which involves turning or side-stepping, our Rotation control system rotates the whole treadmill in particular direction on mechanical rotating base. In case of turning as shown in Figure 3, when foot is on more than three strips then user wants to turn and we should rotate the treadmill. If middle strip of new footstep is on left side of middle strip of previous footstep then rotation is on left side and if middle strip of new footstep is on right side of middle strip of previous footstep then rotation is on right side.
Figure 3: Rotation of treadmill for veer left turn (i.e. 45O) (a) Position of treadmill before turning (b) after turning
Figure 4: Rotation of treadmill for side-stepping (i.e. 15 O) (a) Before side-stepping and (b) after sidestepping
In case of side-stepping as shown in Figure 4, When both feet are on three strips then compare
distance between current and the previous foot positions to determine whether side-stepping has taken placed or not. If it is more than a threshold value, the side-stepping has taken placed otherwise there is no side-stepping. If it is equal or less than maximum gap distance then that is forward step, so no rotation is performed. After determining the direction and angle of rotation, our software sends appropriate signals to the servo motor to rotate in the desired direction by given angle and, accordingly, the platform rotates. This process ensures that the user places the next footstep on the treadmill itself, and do not go off the belt. The algorithm to find direction and angle of turning is based on (a) number of strips pressed by left foot (nl), (b) number of strips pressed by right foot (nr), (c) distance between middle strips of two feet (dist) and (d) threshold for the distance between middle strips of two feet. The outputs are direction (Left Turn - lt, Right Turn - rt, Left Side stepping - ls, or Right Side stepping – rs) and angle to turn. Different possible cases of turning and sidestepping are shown in Figure 5. ALGORITHM 1: if (nl>3) && (dist>d) then //Case-1 2: find θ 3: left_turn = true //i.e. return lt 4: elseif (nl==3) && (dist>d) then //Case–2 5: θ = 15o 6: left_side_stepping = true //i.e. return ls 7: elseif (nl>3) && (dist3) && (dist>d) then //Case–4 11: find θ 12: right_turn = true //i.e. return rt 13: elseif (nr==3) && (dist>d) then //Case–5 14: θ = 15o 15: right_side_stepping = true //i.e. return rs 16: elseif (nr>3) && (dist
