342
Behaviour, integrative and clinical neuroscience
Navigation with a sensory substitution device in congenitally blind individuals Daniel-Robert Chebata, Fabien C. Schneiderb, Ron Kupersc and Maurice Ptitoa,d Vision allows for obstacle detection and avoidance. The compensatory mechanisms involved in maintaining these functions in blind people using their remaining intact senses are poorly understood. We investigated the ability of congenitally blind participants to detect and avoid obstacles using the tongue display unit, a sensory substitution device that uses the tongue as a portal to the brain. We found that congenitally blind were better than sighted control participants in detecting and avoiding obstacles using the tongue display unit. Obstacles size and avoidance strategy had a significant effect on performance: large obstacles were better detected than small ones and step-around obstacles were better avoided than step-over ones. These data extend our earlier findings that when using a sensory substitution device, blind participants outperform sighted controls not only in a virtual navigation task
but also during effective navigation within a human-sized c 2011 Wolters obstacle course. NeuroReport 22:342–347 Kluwer Health | Lippincott Williams & Wilkins.
Introduction
learning to navigate without vision drives hippocampal plasticity and volumetric changes in congenitally blind individuals.
Vision is without a doubt a great facilitator of navigational tasks. Sighted humans can easily select a path to navigate through a hallway scattered with obstacles. Visual cues regulate foot placements by providing constant spatial updates of the distance to the obstacle in order to adapt locomotor behaviour while approaching an obstacle [1]. For instance, visual influences make it much easier to step over an obstacle [2]. Avoiding obstacles and creating a mental map of the environment is obviously more difficult in the absence of vision [3], and one of the greatest navigational challenges faced by blind individuals is obstacle avoidance. Despite visual deprivation from birth, congenitally blind individuals are still capable of avoiding obstacles through cluttered and complex environments in everyday life, and of completing and integrating paths [4]. Indeed, congenitally blind individuals are able to generate cognitive representations of space stemming from the remaining intact senses [5,6], and they preserve the ability to recognize a travelled route and represent spatial information mentally [7]. Moreover, congenitally blind individuals can outperform the sighted ones in certain spatial tasks [4], and their superior navigational skills correlate with a larger anterior hippocampus [6]. The enlargement of the anterior part of the hippocampus is accompanied by a volume reduction in the posterior portion of this structure [8], suggesting that the taxing demands of c 2011 Wolters Kluwer Health | Lippincott Williams & Wilkins 0959-4965
NeuroReport 2011, 22:342–347 Keywords: congenital blindness, navigation, obstacle detection, sensory substitution Chaire de recherche Harland Sanders en Sciences de la vision, E´cole d’Optome´trie, Universite´ de Montre´al, Canada, bDepartment of Radiology, University Hospital of Saint-Etienne and Centre de Neurosciences Cognitives UMR 5229, Bron, France, cInstitute of Neuroscience and Pharmacology, Panum Institute, University of Copenhagen, Copenhagen and dDanish Research Center on Magnetic Resonance, Hvidovre Hospital, Hvidovre, Denmark a
Correspondence to Ron Kupers, PhD, Institute of Neuroscience and Pharmacology, Panum Institute, Copenhagen University, Blegdamsvej 3B, Copenhagen DK-2200, Denmark Tel: + 45 35456890; fax: + 45 35453989; e-mail:
[email protected] Received 23 February 2011 accepted 25 February 2011
In an earlier study, we found that congenitally blind individuals outperform their sighted counterparts in a virtual navigation task when using the tongue display unit (TDU), a sensory substitution device that translates visual images into electrotactile pulses applied to the tongue [9]. However, no studies have been published, which compared the ability of congenitally blind and sighted individuals to navigate in a more natural setting using the TDU. The purpose of this study was, therefore, to compare navigational capacities of congenitally blind and blindfolded sighted participants in a life-size obstacle course. We hypothesized that congenitally blind individuals will perform better than age-matched and sexmatched blindfolded sighted controls in detecting and avoiding obstacles.
Materials and methods Participants
Sixteen right-handed congenitally blind participants with no history of light perception (10 male; 38 ± 12 years) and 11 sighted controls (six male; 34 ± 15 years) participated in the study. Causes of blindness were retinopathy of prematurity in five cases; congenital cataract, Leber’s amaurosis, retinitis pigmentosa, detached retina, retinoblastoma, glaucoma and microphthalmia each in DOI: 10.1097/WNR.0b013e3283462def
Obstacle detection and avoidance in the blind Chebat et al. 343
one case; a (medical) accident in another two cases. The research protocol was approved by the Local Ethics Committees of the ‘Montreal Association for the Blind and the Center for Interdisciplinary Research in Rehabilitation’, and of the Institut Universitaire de Ge´riatrie de Montre´al.
detect and avoid obstacles with the TDU. The dimensions of the obstacle course and the location of the different obstacles are illustrated in Fig. 2a. The entire obstacle course was composed of two different hallways, each measuring 1.2 m wide by 15 m long. The hallways were painted in white with a nonreflective paint to
Materials Visual-to-tactile sensory substitution system
Fig. 2
We used a lingual visual-to-tactile sensory substitution system, the TDU, that has been described in detail in earlier publications [10]. The TDU is composed of a tongue array consisting of 100 small circular electrodes arranged in a 10 10 matrix, a laptop computer and a webcam. In brief, the participant wore a webcam mounted on a pair of light-tight safety goggles, and the electrode array was placed on the tongue. The TDU device, illustrated in Fig. 1, is capable of transmitting images to the tongue in the form of electrotactile pulses [11]. The visual field of the camera is represented on the tongue array. Every time an object enters within the visual field, the visual image is translated into electrotactile pulses that are transmitted to the tongue through the electrode array. The obstacles are thus ‘drawn’ with electrical current on the tongue in real time from the images provided by the webcam (see video at www.tonguevision.blogspot.com).
(a)
250 m
(b)
15 m 120 m 14 Box 13 Triangle
12
11
10
Blocked road
9
Experimental setup
A life-size obstacle course was used to assess the abilities of congenitally blind and sighted control participants to Fig. 1
Low tube
8
7
High tube
6
Ground bar
5 (c) 4
3
2
1
∗
0 (a) A blindfolded participant wearing the tongue display unit. The camera mounted on a pair of safety goggles and fastened with an elastic band around the head. The webcam communicates through bluetooth to a laptop that translates the images into electrotactile stimulation, which is relayed to the tongue using a 10 10 electrode array [boxed area in (b) and shown magnified in (c)]. The electrodes become active when a dark object enters within the visual field of the camera.
Out
In
∗
Schematic drawing of the obstacle course. (a) The obstacle course with the two hallways. Hatched lines indicate 1 m distances. The exact location of the obstacles is represented with a different symbol for each obstacle type. (b) The different obstacle types. (c) An actual picture of the obstacle course. The asterisk indicates the place from where the picture was taken in the hallway.
344 NeuroReport 2011, Vol 22 No 7
Stimuli
Different kinds of objects were used to assess the obstacle detection and negotiation abilities of the participants. The obstacles were made of cardboard and styrofoam to avoid injury on impact and had different sizes and shapes. Six types of stimuli were used: (i) boxes (60 60 60 cm), (ii) triangles (60 45 2 cm), (iii) tubes (10 45 10 cm), (iv) branches (10 45 10 cm), (v) road blocks (150 10 10 cm) and (vi) ground bars (15 145 20 cm). They were made to resemble obstacles that can be realistically encountered on a sidewalk of any large city. The obstacles were positioned in the hallways according to Fig. 2a. A first person view of the hallways is provided in Fig. 2c. Each hallway run was repeated twice. Procedure Familiarization
We first familiarized participants with the TDU, fitting them with the device and making sure that they could ‘see’ with it. Participants were told that the stimulation on the tongue represented the field of view of the camera. They were encouraged to interact with the camera by waving their hands in front of the lens and feeling the sensation on the tongue at the same time. All participants could accurately distinguish a vertical line from a horizontal one and were given a ‘visual’ tactile acuity test [10]. This familiarization period was followed by a short demonstration on how to point to obstacles while equipped with the TDU. Participants were standing and facing a 3.65-m long corridor that was different from the actual obstacle course. They stood at the entrance of the hallway and were familiarized with each obstacle individually. A single obstacle was placed 2.75 m in front of them. The participants were asked to point to the obstacle with the camera cyclopean-eye as they tried to estimate its shape and distance. Participants were asked ‘‘How big is the obstacle?’’, ‘‘How far away do you think it is?’’ or ‘‘What does it look like?’’. Obstacle course
In the second part of the experiment, the participants were placed at the entrance of the obstacle course. Their task was to point to and avoid the obstacles. For the detection task, accurately pointing to an obstacle was scored as a correct response. For the avoidance task, negotiating a path without hitting or touching each obstacle was scored as a correct response after each obstacle was successfully passed. A w2 statistical analysis was carried out on the number of correct responses to ascertain that performance was above chance level.
Grouping of obstacles
It is more difficult for blind individuals to step-over (SO) obstacles than go around them [12] and to detect small obstacles compared with large ones [13]. During analysis we therefore regrouped the stimuli according to the strategy used to avoid them into step-around (SA) and SO obstacles. The ground bar was the only SO obstacle and was compared with the other obstacles. All other obstacles except for the ground bar were SA obstacles. Obstacles were also grouped into the categories large and small based on their total surface area. The box was the largest obstacle with a surface of 0.36 m2 and was compared with other obstacles that were all less than 0.17 m2. Paired samples t-tests were carried out for comparison of the participant’s own scores on the groupings of obstacles. For group comparisons of the performance for each grouping, a one-way analysis of variance was performed.
Results All participants were able to describe the distance (near, far), size (large, small) and type (SA, SO) of the obstacles. They could also accurately point to and avoid obstacles. Congenitally blind participants performed significantly better than sighted control participants both for the detection and avoidance task. Moreover, obstacle size and avoidance strategy were important factors in determining performance for both groups. As shown in Fig. 3, both congenitally blind and sighted control
Fig. 3
∗
100
∗∗
80 Correct response (%)
contrast with the obstacles (Fig. 2b) painted in a black nonreflective paint (Fig. 2c). Lighting in the obstacle course was provided by a set of neon lights, placed strategically to maximize uniform lighting and avoid shadows.
60
40
20
0 CB
SC
Detection
CB
SC
Avoidance
Obstacle detection and avoidance, all obstacle types confounded, expressed in percentage of correct responses. Asterisks indicate the level of significance of the group difference for the avoidance and detection tasks (*P r 0.05; **P r 0.005). CB, congenitally blind; SC, sighted controls.
Obstacle detection and avoidance in the blind Chebat et al. 345
participants could detect {congenitally blind [w2(12,1) = 64.8; P r 0.001]; sighted controls [w2(9,1) = 47.27; P r 0.001]} and avoid {congenitally blind [w2(12,1) = 119; P r 0.001]; sighted controls [w2(9,1) = 68; P r 0.001]} obstacles above chance level. Moreover, obstacle size and avoidance strategy had a significant influence on the performance in both paradigms. Large obstacles were significantly easier to detect than small ones for both congenitally blind [t(12) = 4.9; P r 0.001] and sighted control participants [t(9) = 5.2; P r 0.001]. SA obstacles were easier to detect than SO ones, for both congenitally blind [t(12) = 4.5; P r 0.001] and sighted control [t(9) = 3.5; P r 0.05] participants. Large obstacles were also easier to avoid than small ones for both groups [congenitally blind t(13) = 3,7; P r 0.05; sighted controls t(9) = 3.3; P r 0.05]. As shown in Fig. 4, SA obstacles were easier to avoid than SO ones for both groups [congenitally blind t(12) = 4.9 P r 0.001 and sighted controls t(9) = 3.6; P r 0.05]. An intergroup comparison
Detection
∗∗
∗∗
∗∗ ∗
100 Correct response (%)
Discussion In this study we show for the first time that congenitally blind participants are significantly better than blindfolded sighted controls in detecting and avoiding obstacles when using a sensory substitution device that uses the tongue as a portal to the brain. This extends our earlier findings of superior performance of congenitally blind participants in a virtual navigation task using the TDU [9]. Process of distal attribution
Fig. 4
80 60 40 20 0 L
S
L
S
SA
SC
CB
SO
SA
CB
SO SC
Avoidance 100 Correct response (%)
indicated that congenitally blind participants performed significantly better than sighted control participants in detecting [w2(14,9) = 17.53; P r 0.005] and avoiding [w2(14,9) = 51; P r 0.005] obstacles. Although both groups were equally good at detecting large, SA and SO obstacles, congenitally blind participants were significantly better in detecting [F(9) = 2.4; P r 0.05] and avoiding [F(9) = 2.4; P r0.05] the small ones.
∗∗
∗∗
∗
∗
80 60 40 20 0
L
S CB
L
S SC
SA CB
SO
SA
SO
SC
Obstacle detection and avoidance expressed in percentage of correct responses and grouped according to obstacle category. The graphs show differences between obstacles of opposing categories. Large obstacles are compared with small ones, and step-around obstacles are compared with step-over ones. Significant differences are indicated by asterisks (*P r 0.05; **P r 0.001). CB, congenitally blind; L, large; S, small; SA, step-around; SC, sighted controls; SO, step-over.
Our results are in accordance with previous reports showing that well-trained blind participants are able to point accurately to targets, to slalom walking and bat a ball using a sensory substitution device [14]. Participants can use touch to guide movements by attributing the sensation on the tongue to the perception of the distal obstacle in space. This process, called distal attribution [15], is known to occur in sensory substitution [16]. Distal attribution typically involves three steps (see [17] for review), starting with the realization that there exists a relationship between movements of the camera and the stimulation on the tongue. The next step is the discovery that the stimulation reflects the presence of an outside object. The third component is the apprehension that the stimulation discloses the object shape and its position in perceptual space [17]. Participants went through this process quickly, after only a few hours of training and thereby followed the three general steps of distal attribution in the same order. In keeping with other studies using touch or sound [10,11,14,17], participants relied on exploratory head movements to extract and organize their lingual sensations into meaningful concepts. By updating sensations derived from different fields of view, participants learned to infer the relationship between their lingual sensations and the relative shape and position of the obstacle in space. Performance and types of obstacles
The finding that for both groups, SA obstacles were easier to detect than SO ones is concordant with results obtained in low-vision participants for whom vertical objects are easier to detect than horizontal ones [13]. Under normal conditions it is harder for visually impaired individuals to detect and SO ground objects and to avoid small objects than to go around them [12]. In addition, blindfolded sighted participants who had viewed a
346 NeuroReport 2011, Vol 22 No 7
hallway before locomotion can walk with relative ease through it and avoid SA obstacles [2]. In line with finding by Kuyk et al. [12], we observed that our participants often misjudged the height of the SO obstacles, thus hitting the obstacles as they attempted to avoid them. This is probably because of the fact that in addition to a precise positioning of the obstacle in space (like is needed for SA obstacles), participants needed to judge the height of the object accurately to step over it. However, congenitally blind participants improved their performance during the second time they traversed the two hallways, suggesting the potential of the TDU as a navigational aid. However, some researchers have expressed scepticism as to the potential of sensory substitution devices such as the TDU to be meaningful for navigation in real world situations [18–20]. Admittedly, we tested our participants under optimal experimental conditions such as uniform lightning without shadows, maximum object contrast, etc. Therefore, our findings cannot be generalized automatically to less idealized daily life conditions, characterized by the presence of lower contrast objects and varied light conditions. In contrast, we would like to emphasize that these results were obtained after just a few hours of training with the TDU.
Spatial navigation in the absence of vision
Our results extend earlier findings showing supranormal abilities of blind participants in several spatial tasks. These studies reported that congenitally blind participants outperform the sighted in sound localization [21] and proprioceptive [14] tasks. Ittyerah et al. [22] showed that congenitally blind children can form mental representations of space, and are more accurate when pointing to a target than blindfolded sighted control children. Far from being deficient, congenitally blind individuals have a different way of understanding space, stemming from senses other than vision and hence develop different strategies to configure space [16]. Loomis [18] has argued that blind participants are strongly impaired in several spatial tasks. However, when the perceptual advantages of vision for perceiving space in sighted participants are nullified by blindfolding them, their performance in spatial tasks is comparable [4,5,7] or worse than that of congenitally blind individuals [6]. The spatial tasks for which blind participants have been shown to be impaired, including mental rotation, path integration and homing, are irrelevant for correctly performing our navigation task. Indeed, participants were using tactile guidance to navigate through the corridor and besides obstacle detection and avoidance, no other spatial abilities were required to solve the task. Other studies have also found a perceptual advantage for the blind in their remaining senses, which is believed to rely on cross modal plasticity (see [23] for review). The
occipital lobe of the congenitally blind individual is recruited in a vast number of tasks to fit environmental demands [23,24]. For example, the recruitment of the occipital lobe for tactile and auditory tasks is well documented in the blind individual [23]. This recruitment of the occipital lobe for the processing of perceptual information other than vision is believed to confer to the blind individual’s supranormal sound localizing abilities [25], and finer tactile acuity [10]. It has been suggested that the occipital lobe is acting as a multimodal brain structure [23].
Conclusion We show for the first time that when equipped with a sensory substitution device, congenitally blind participants outperform sighted controls in spatial tasks involving pointing and avoiding obstacles. Our data urge for an investigation of the potential of the TDU for spatial navigation in more natural settings, using low-contrast obstacles under varied illumination conditions and with the presence of shadows and light reflections.
Acknowledgements This study was supported by grants from The Harland Sanders Foundation (M.P.), the Danish Medical Research Council (M.P., R.K.), the Lundbeck foundation (R.K.) and the Canadian Institutes for Health Research (D.R.C.).
References 1
2
3 4 5 6
7 8
9
10 11 12 13
14
McFadyen BJ, Bouyer L, Bent LR, Inglis JT. Visual-vestibular influences on locomotor adjustments for stepping over an obstacle. Exp Brain Res 2007; 179:235–243. MacLellan MJ, Patla AE. Stepping over an obstacle on a compliant travel surface reveals adaptive and maladaptive changes in locomotion patterns. Exp Brain Res 2006; 173:531–538. McVea DA, Pearson KG. Object avoidance during locomotion. Adv in Exp Med Biol 2006; 629:293–315. Loomis JM, Klatzky RL, Golledge RG. Navigating without vision: basic and applied research. Optom Vis Sci 2001; 78:282–289. Thinus-Blanc C, Gaunet F. Representation of space in blind persons: vision as a spatial sense? Psychol Bull 1997; 121:20–42. Fortin M, Voss P, Lord C, Lassonde M, Pruessner J, Saint-Amour D, et al. Wayfinding in the blind: larger hippocampal volume and supranormal spatial navigation. Brain 2008; 131:2995–3005. Passini R, Proulx G, Rainville C. The spatio-cognitive abilities of the visually impaired population. Environ Behav 1990; 22:91–118. Chebat DR, Chen JK, Schneider F, Ptito A, Kupers R, Ptito M. Alterations in right posterior hippocampus in early blind individuals. Neuroreport 2007; 5:329–333. Kupers R, Chebat DR, Madsen KH, Paulson OB, Ptito M. Neural correlates of virtual route recognition in congenital blindness. Proc Natl Acad Sci U S A 2010; 107:12716–12721. Chebat DR, Rainville C, Kupers R, Ptito M. Tactile-‘visual’ acuity of the tongue in early blind individuals. Neuroreport 2007; 3:1901–1904. Bach-y-Rita P. Tactile sensory substitution studies. Ann N Y Acad Sci 2004; 1013:83–91. Kuyk T, Elliott JL, Biehl J, Fuhr PS. Environmental variables and mobility performance in adults with low vision. J Am Optom Ass 1996; 67:403–409. Genensky SM, Petersen HE, Clewett RW, Yoshimura RI. A secondgeneration interactive classroom television system for the partially sighted. Am J Optom Physiol Opt 1978; 55:615–626. Jansson G. Tactile guidance of movement. Int J Neurosci 1983; 19:37–46.
Obstacle detection and avoidance in the blind Chebat et al. 347
15
Epstein W, Hughes B, Schneider S, Bach-y-Rita P. Is there anything out there?: a study of distal attribution in response to vibrotactile stimulation. Perception 1986; 15:275–284. 16 Loomis JM. Distal attribution and presence. Presence 1982; 1:113–119. 17 Auvray M, Hanneton S, Lenay C, O’Regan K. There is something out there: distal attribution in sensory substitution, twenty years later. J Integr Neurosci 2005; 4:505–521. 18 Loomis JM. Sensory substitution for orientation and mobility: what progress are we making? Sidebar 1.1 (pp. 7–10) to chapter 1, perceiving to move and moving to perceive: control of locomotion by students with vision loss by David A. Guth, John J. Rieser, and Daniel H. Ashmead (pp. 3–44). In: Wiener WR, Welsh RL, Blasch BB, editors. Foundations of orientation and mobility. 3rd edition. Vol. 1 (History and Theory). New York: AFB Press; 2010. 19 Dodds AG, Howarth CI, Carter DC. The mental maps of the blind: the role of previous visual experience. J Vis Impair Blindness 1982; 76:5–12.
20 21
22 23
24
25
Rieser JJ, Guth DA, Hill EW. Sensitivity to perspective structure while walking without vision. Perception 1986; 15:173–188. Lessard N, Pare´ M, Lepore F, Lassonde M. Early-blind human subjects localize sound sources better than sighted subjects. Nature 1998; 17:278–280. Ittyerah M, Gaunet F, Rossetti Y. Pointing with the left and right hands in congenitally blind children. Brain Cogn 2007; 64:170–183. Cattaneo Z, Vecchi T, Cornoldi C, Mammarella I, Bonino D, Ricciardi E, Pietrini P. Imagery and spatial processes in blindness and visual impairment. Neurosci Biobehav Rev 2008; 32:1346–1360. Ptito M, Moesgaard SM, Gjedde A, Kupers R. Cross-modal plasticity revealed by electrotactile stimulation of the tongue in the congenitally blind. Brain 2005; 128:606–614. Gougoux F, Zatorre RJ, Lassonde M, Voss P, Lepore F. A functional neuroimaging study of sound localization: visual cortex activity predicts performance in early-blind individuals. PLoS Biol 2005; 3:20–27.
