Obstacle Avoidance in a Terrestrial Mobile Robot Provided with a

Obstacle avoidance in a terrestrial mobile robot provided with a scanning retina Fabrizio MURA* , Nicolas FRANCESCHINI C.N.R.S.,.Laboratoire de Neurobiologie, LNB3 3 1, Chemin Joseph Aiguier 13402 Marseille Cedex 09, FRANCE Tel: +33 91 16 41 29; Fax: +33 91 22 08 75 E-mail: enfranceschini @ irlnb.cnrs-mrs.fr Abstract. This study describes the role of a novel visual sensor termed the “Scanning Local Motion Detector” (or SLMD) for the visual navigation of a terrestrial mobile agent. Following a brief theoretical description of the principle of retinal scanning coupled with motion parallax, we show that a single retina of 24 pixels - when oriented in the mobile agent’s forward line of move - allows the latter to avoid the visually detected obstacles in a reflex manner by correcting its trajectory in real time. The results of the preliminary computer simulations yield credit to a zig-zag algorithm which allows the agent to avoid the oncoming obstacles while immediatly “keeping an eye” on its lateral regions. In parallel with the simulation, we designed and built a hardware prototype of a scanning retina with coarse optical resolution (average angular sampling of 3”) mounted on top of a circular mobile platform. Following the test and calibration of the visual sensor’s scanning mechanism, the robot was assigned to avoid the contrasted walls of a rectangular arena. The preliminary results of the robot’s successful performances confirm the validity of the zig-zag algorithm. This prototype shows that navigation by visual perception does not necessarily require a high resolution visual imaging system with digital processing. The scanning motion confered upon the robot’s retina was inspired by the recent findings of the laboratory about scanning motion of the fly retina.

1. A bionic approach to the perception of visual motion. An increasing number of studies focus on the problem of sensory-motor control with various insights provided by experimental data on the structure and function of biological systems ([21], [6], [22], [12], [7], [17], [20]). Moreover, recent trends in “biomorphic devices” ([15]) suggest that artificial systems can achieve a high degree of robustness and autonomy by embedding their design through an evolutionary process reminiscent of the path taken by animal evolution.

The motivations underlying the bionic approach derive from a common set of observations about invertebrate and vertebrate sensory systems: the sensors work at relatively low bandwidths, sensory devices are robust with regard to fluctuations and they often work in parallel with asynchronous input flows. Consequently, artificial systems are able to exhibit intelligent behaviours as a result of their sensorymotor interactions with the environment ([6], [4]).In particular, the performances of visuomotor systems are likely to depend on the sensor’s abilities to extract relevant significant informations about the structure of the surroundings to meet the needs and assignements of a given task ([loa]). An ongoing central problem in visually guided navigation is related to the estimation of the optic flow by visual perception of motion ([21], [91, [231, [21). For instance, motion of the image can be induced by the robot’s movements ([71, [22], [12], [24], [25]) orby the sensor’s own relative motion ([3], [lo], [SI). We argue that robots do not inevitably require high resolution sensors with large pixel arrays (such as CCD cameras) in order to avoid obstacles: a distributed obstacle avoidance algorithm was successfully implemented - onboard a terrestrial mobile platform - with inputs from a compound eye having less than 120 pixels and with spatial sampling angles of 1”-5”([22], [ 121). In this particular case, the design of the sensor’s optical and electronic architecture and functional organization was based on experimental data related to the processing of visual motion in the fly’s compound eye ([ 141, [ 111). The hardware design of this prototype of artificial compound eye ([22]) has readily underscored the importance of the principle of motion parallax (261) as a means of perceiving the optic flow, referring to the induced retinal velocities caused by the robot’s own motion. Also termed “monocular stereovision”, this principle was previously described in psychophysics ([26], [13], [27], 211) and studied in computer vision ( [ 5 ] ,[I], [16]). Motion parallax can be applied in a straightforward manner to a pair of adjacent visual axes centered on 0, oriented at cp with

* Present address: the University of Tok o de artment of Mechano-Informatics, Miura-Shimoyama Laboratory, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113, fAbA8.

0-7803-3652-6/96/$5.00 0 IEEE

regard to the horizontal line of move and separated by A q (see figure la). This basic configuration forms a Local Motion Detector (or LMD). When travelling from 0 to 0’ at constant velocity V, the two axes detect the contrast point M whose local angular velocity 51, is thus defined by:

R t = V/D,,,.sin(q).Aq/sin(Aq)

(1)

The angular velocity obtained under pure translation is similar to the general expression of the translatory optic flow ([27]) when A q converges to zero i n (1). Another study on the visual control of a mobile robot revealed the fundamental property that the range of vision of the LMD becomes linearly proportional to

the robot’s translatory speed ([17]). A detailed analysis shows that the motion detector’s performances depend on the combination of two essential constraints ([20]). First of all the geometric constraint states that the range of vision decreases when the azimutal orientation q of the visual axes approaches the robot’s line of travel. Secondly, the kinematic constraint states that the range of vision of an LMD located at any given azimuth increases with the robot’s translatory speed. A means of solving the two aforementioned constraints - without having to increase the number of pixels in the eye - was suggested by ([19]): the principle is to generate a controlled retinal scanning movement of the visual axes during each of the LMD’s translatory moves.

of move

move

9 Figures la (left) and l b (right). Fig. la: 2D sketch of the principle of motion arullax applied to apairof optical axes centered The angular velocity a, conesponding to the on 0 moving i n tmnslation towards 0 , in fmnt of a light-contrasted point retinal displacement of M is inversely propodional to the distance D separating M fmm 0 . Note that the triangular region OO’M hatched in ra is termed here the area of vision of the LMD: any contrast point belonging to this region will cross the two visual axes an8 tius excite the LMD. Fig. Ib: by applying an anterograde scanning movement (i. e. from back to front) to the visual axes during the translatory motion of the eye, the resulting angular velocity appears to be the linear sum of the angular velocities induced by each movement (provided that both axes rotate at the same amplitude). In this case, the contrast point M is detected by the second visual axis at 0’.

d.

Under these conditions, the resulting angular velocity R, appears to be the linear sum of the angular velocities induced by the translation of the optical centre R,and the scanning movement ofthe visual axes Qr:

n,=n,+n,

(2)

Because the scanning movement induces a purely rotatory angular velocity, the component R does not depend on the distance to any detected obstacle. Therefore, the translatory angular velocity can be expressed as a function of the angular amplitude ofscan A{ (i. e. the scanning ofthe visual axes between the points 0 and 0’,(see figure lb). In this case, we assume that both axes of the motion detector scan a t the same amplitude), by simple geometric considerations:

R t = V/Do,,,.sin(q).(Aq-A{)/sin(Aq-A{)

(3)

Note that when no scanning is generated (i. e. when A{=O), then R t is identical to the expression in equation ( I ) . In the general case, retinal scanning must be performed in the anterograde sense (i. e. the visual axes move in the posterior to anterior direction) in order to ensure an increase in the probability of detecting a contrast point belonging to the visual environment. The increase in the probability of detection is explicitly achieved by controlling the amplitude of scan through the relative scanning factor a defined as:

a=A5/Aq

(4)

where A q is the angular separation between the two visual axes and A< the amplitude of scan. W e show that the kinematic and geometric constraints of visual motion can be reduced ([ 191) by a proper control of the scanning factor. Thus, by generating an anterograde scan of the visual axes during the robot’s motion, the underlying circuit forming the Scanning Local Motion

Detector (SLMD) is able to retrieve the translatory angular velocity of contrast points. Also if the amplitude of scan is known as well as the robot’s translatory velocity, then the distance to the obstacle can be computed in a straightforward manner using equation (3).

2. Visually-guided obstacle avoidance with a frontal scanning sensor. In order to highlight the improvements brought about by retinal scanning, a digital simulation of a narrow retina comprising a minimal amount of pixels was run (24 visual axes with an average angular separation over the eye, AT, of roughly 3”). The pixels of this eye are organized as a linear array whose field of view is oriented around the robot’s frontal line of move so that there is an equal number of visual axes on each side (see figure 2a). According to the rule of anterograde scanning, the visual axes on each side of the line of move will scan the environment in opposite senses thereby providing a kind of “vergence scanning”. Since we have chosen to work with a monocular eye endowed with a single scanning actuator - the desired scanning pattern for both sides can be appropriately obtained by running a time sequence of two scan cycles in succession: each hemifield of the eye (comprising 12 pixels) acquires a scanned image during one translation step of the robot. Consequently we can display the sensor’s surface of

vision in the plane during two successive 1eWright acquisition cycles: a s it is shown in the figures 2b (a=O),2c (a=1)and 2d (a=2),the surface of vision increases with increasing a.In theory, the kinematic and geometric constraints of motion perception are solved whenever 0122: however, the angular precision of the measurements also decreases as a increases. Thus, the choice of a = 2 is a necessary and sufficient condition which satisfies the two aforementioned constraints while minimizing the uncertainty of the measured angular velocities.

3. The reflex “zig-zag” obstacle avoidance strategy. In search of a viable visuomotor guidance strategy for the robot, we ran some preliminary computer simulations using a 2D model of the previously designed scanning retina, coupled to a kinematic model of the wheeled mobile platform. For the purpose, the visuomotor system was surrounded by the 2D geometrical model of an environment built with light-contrasted obstacles of various shapes. In the context of the study, we chose to inhibit the visual inputs during every rotation of the platform by decoupling the translatory motion and the corrective steering: whenever an obstacle generates a corrective steering, the robot comes to a halt, performs a pure rotation according to the desired correction (under no visual control whatsoever) and then resumes the translatory motion.

2a

Figures 2a (to left), 2b (top right), 2c (bottom left) and 2d (bottom right). Fig 2a: angular configuration of the frontal scanning retina mountel on top of a terrestrial mobile platform. This retina has 24 visual directions (1 2 are overlooking on each side of the platform) and the average angular separation between pairs of visual axes is 3”. The distances D, and D, ?e the boundary values of the “zig-za6” algorithm (see correspondinf text). Fig. 2b: dunng a ure translation of the eye (no retinal scanning), the area of the retina s surface of vision (drawn in b ack) is represented by tfe superposition of every LMD’surface of vision around the eye. This field of vision is built from a time sequence of two translato% steps, where each step roduces a surface of vision on one side of the robot. A major proportion of this surface is represented y the surface of vision o!the LMDs having the largest azimuth. Figs 2c and 2d: by having the visual axes scan the plane in the anterograde sense, the surface of vision is increased: theoretically, the range of vision alon each visual axis becomes infinite as soon as a21 (for convenience, the drawings of 2c and 2d have been cut vertically). Tfus, when 1x22, the surface of vision occupies the entire an);ular extent of the eye. The singularity of the translatory optic flow along the robot’s line of move corresponds to the pole of no motion” (1. e. Q,=O).

Summarizing the results of the simulation, it turns out that a zig-zag obstacle avoidance strategy allows the robot to minimize the collisions in a large number of environmental situations by taking into account the absence of lateral vision. In essence, the zig-zag strategy consists of having the robot’s tuming angle related to the visual inputs through a two-stage reflex algorithm. First of all, the visual inputs are selected and secondly the selected data are fused together. The selection phase is described in the following way: upon detection of an obstacle measured at D, under ql, the algorithm computes a tuming angle 8, which, i n the environment’s frame of reference, is:

the “repelling function” p(D.,D,,D,) produces a corrective angle with a iinearly decreasing amplitude as Di increases: p(D,,D,,D,) is a continuous function ofDi within [D, ; D,]. In a first step, we choose to work with constant values of D, and D,, where D, is the largest distance which triggers a zig-zag manoeuver and D, is the smallest distance which triggers a maximum corrective angle. Consequently, a detected obstacle satisfying D$D, will not induce a corrective manoeuver, because it is not close enough to affect the robot’s trajectory. Note that the surface of vision bounded by [D, ; DJ is a portion of a ring (see figure 2a): also, the region in front of the robot for which DID, is identified as a

I - -

“safety area” in which the presence of any obstacle is highly undesirable. Following the selection phase, a couple of maximum values (el, 8 ) are sorted out among 8,on each side of the eye. h e s e two values are then fused according to a set of three basic rules: -rule 1: Approach. First of all, if the distances corresponding to the obstacles detected on the left AND the right sides are both within ID, ; DJ, then the amplitude of the corrective steering angle is equal to Iv,-cpJand the robot tums to the opposite side of the closest obstacle. -rule 2: Avoid. Secondly, if the distance to the obstacle on one side is smaller than D, AND if the distance to the obstacle on the other side is within ID, ; DJ, then the amplitude of the corrective steering angle is equal to 18,+8,1 and the robot tums to the opposite side of the closest obstacle. -rule 3 : Half tum. At last, if the distances to the two obstacles are smaller than D,, then the amplitude of the corrective steering angle is 18Oo-l(p1-(pJand the robot turns to the opposite side of the nearest obstacle. A closed loop simulation of the “zig-zag” algorithm is performed in the situation where the robot is assigned to wander in a square room of l m x l m with light-contrasted points randomly separated by 5+ lcm. The resulting plot of the robot’s trajectory displays a random wandering path with no collisions (see figure 3a): this simulation was performed with a maximum corrective angle of 95” (this corrective value is applied upon detection of an individual obstacle verifying DID,).

--- - - - --

I

I

I I

-

1DO

I I

I

I I

. I

Figures 3a (le&)and 3b (right). Fig 3a: plot of the tlajectoly of the “zig-zag” obstacle avoidance behaviour i n a simulated terrestrial mobile robot endowed with the previously described frontal scanning retina (see figure 2a). The robot is assigned to wander in a simulated light-contrasted arena of lmxlm whose walls are textured with blackhhite s&ipes(random wdth of Sflcm): the robot is launched fmm the centre of the room during lo? time steps ( 1 step lasts 0.2s). In ths case, the safety &stance D=2.Rplatrm and Dz=4.RpIk,, (Rplar !s the platform’s physical radius, 0.25m). Also, the maximum steerin angle generated by the tetection of a sm$ obstaae i s 95“ and the scanning factor ~1 (equ. 4) is 1.5. Fig 3b: at the enfof the. simulation, the points conesponding to the detected obstacles can be displayed ,in 2D: the fluctuations i n the estimated distances are due to the accumulation of various factors such as the angular imprecision due to retinal scannm , the errors in measurements and the fiiite digital sampling steps. Under the same initial conditions, a simulation using d D s instead of SLMDs yielded 52 collisions with the arena. The robot has the opportunity to approach (as angle: since the robot lacks perception about the well a s to escape) the comers of the room without environment in the lateral fields, it compensates by colliding, thanks to the half turn manoeuvers. The generating large turns, thus allowing the frontal eye to success of this zig-zag avoidance strategy is mainly explore the environment corresponding to a previously due to the relatively large amplitude of the turning unrevealed lateral field.

In order to confirm the results of the simulation, a small platform (diameter-0.25m, see fig. 4) was built and a narrow scanning retina was mounted on top of it. Although the platform is not completely autonomous (the command circuits, the energy and the algorithm implementation are offboard) the visual processing is performed onboard by analog SLMD circuits of the

same kind as the LMD boards used in the previous robot (see[22]). A remote 486 SOMhz PC computer, connected to the robot via an A/D interface is able to run the obstacle avoidance algorithm in real-time by commanding the robot’s lateral wheels. The figure 5 shows a plot of the trajectory of the platform’s geometric centre during a session of 1 minute: the

Energy supply and I/Os Black cover Lens

Bumper Figure 4: photography of the constructed mobile robot with the frontal scanning retina placed on topof the platform’s circular basis. Three types of cables allow the robot to operate i n real-time: the power suppl the remote scannercontrol and the steering motor control and the link of the LMD s remote PC which suppolts the sokvare implementation of the zig-zag algorithm. robot is assigned to move within a rectangular arena (1.4mx1.8m) whose walls are textured with black and white vertical stripes with random widths in the range of 0.04m to 0.12m. The robot is initially launched in the centre of the arena and moves towards the west side (indicated by the black arrow) at the constant velocity of O.lm/s. In this run, the zig-zag avoidance strategy is implemented with a discretized repelling function p(R ,SZ,,SZ,) related to the measured translatory optic dow SZ,, and the set boundary values SZ, and R,, since the analog SLMD circuits’ outputs are proportional to angular velocities.

Figure 5: the robot is assi ned to wander within a rectangular arena (1.4mxlSmY at the constant speed of O.lm/s: the contrast ints on the walls of the arena are random1 spaced i n t% range of [0.04m;0.12mI. The plot was perzrmed by positionnng the robot in the centre of the arena and launching i t towards the west wall. In this plot, the scanning factor of the retina c1 is set to 1.5.

It turns out that the robot successfully avoids the obstacles when the maximum steering angle is set to 90’. Reduction of this steering angle has shown to induce a progressive increase in the number of collisions. Also, the trajectory displays several typical patterns of “approach and avoid” m a n o e w e s which were observed in the simulations. However, in this session, no half tums were performed: this could be due to the presence of a slight asymmetry of the visual system on each side of the robot. Such an asymmetry could also explain the large proportion of turns to the right side.

Conclusion. We have reported some preliminary results about the role of a biologically-inspired scanning retina endowed with a minimal number of pixels and a coarse optical resolution (24 pixels; 3” of average angular spacing) for the visual guidance of a terrestrial mobile platform. In theory, we show that the visual perception of motion is enhanced by an active anterograde scanning movement of the visual axes. Furthermore, the amplitude of scan can be properly controlled by the scanning factor a so as to solve the kinematic and geometric constraints intrinsic to the perception of visual motion: the sensor’s field of vision depends 1) on the robot’s translatory displacement and 2) on the visual axes’ orientation with regard to the robot’s line of move. Our theory was applied to the visual guidance of a terrestrial platform: the validity of the reflex zig-zag algorithm was tested in digital simulations and confirmed by the preliminary results of the hardware implementation, in the restricted case where 1) the visual inputs are inhibited during the robot’s rotations and 2) the

corrective manoeuven are decoupled from the translatory steps. This workis, to our knowledge, the first mechano-opto-electronic implementation of a scanning retina with a direct application to visual guidance of a mobile robot. The choice of a minimalistic sensor configuration is deliberate, as it logically follows the streamline of ideas which have been developped in previous works ([6], [22], [12], [17], [18]). It is ourbelief that the robotic systems of the future may have something to gain from the biological systems, since the latter have acquired substantial experience in the field of sensory-motor control though the pressure of evolution.

Acknowledgements: the authors wish to thank R. Chagneux and N. Martin for their help and critical comments. Many thanks also to M. Boyron and J. Roumieu for their invaluable technical contributions to the construction of the robot. The authors are indebted to Prof I. Shimoyama for fruitfull comments and suggestions on the content. This research was supported by the Centre National de la Recherche Scientifique (C.N.R.S. life sciences and PIR Cognisciences), the Region Provence Alpes Cote D’Azur Corse (P.A.C.A.C.) and the Commission of the European Communities (ESPRIT BRA SSS 6961 and HCM). F. Mura was supported by a graduate fellowship from C.N.R.S. (MREN). References. [I] ALBUS JS, HONGTH(1990): Motiondepth and image flow. IEEEintl Conf Robotics and automation, Cincinnati, Ohio, 1161-1170. [2] ALOIMONOS Y (1993): Active vision revisited. In: Active Perception, ed Y Aloimonos, Lawrence Erlbaum, pp 1-18. [3] BALLARD D, BROWN CM (1993): Principles in animate vision. In: Active Perception, ed Y Aloimonos, Lawrence Erlbaum, pp 245-282. [4] BEER R (1990): Intelligence as adaptive behaviour: an experiment in computational neuroethology. Academic press vol. 6. [5] BRIOT M, de SAINT-VINCENT AR (1986): Three-dimensional perceptory systems for autonomous mobile robots. In: proc 6th intl conf on Robot Vision & Sensory Systems, Paris, France, pp 127-137. [6] BROOKS RA (1986): A robust layered control system for a mobile robot. IEEE J of Robotics and automation, RA-2, 14-23. [7] COOMBS D, ROBERTS K (1992): “Bee-bot”: using peripheral optical flow to avoid obstacles. In Intelligent Robots and Computer Vision XI, SPIE Bellingham (USA) vol 1825, 714-721. [SI DIAS J, BA’ITISTA J, SIMPLICIO C, ARAUJO H, de ALMEIDA AT (1993): Implementation of an active vision system. In preprints of the Intl Workshop on Mechatronical Computer Systems for Perception and Action, MCPA’93; Halmstad, Sweden, 45-52. [9] DICKMANNS ED, GRAEFE V (1988): Dynamic monocular machine vision. Machine Vision and Applications, 1, 223-240.

[lO]EKLUNDHJO, PAHLAVANK, UHLINT(1993): Computer vision and seeing systems. In preprints of the Intl Workshop on Mechatronical Computer Systems for Perception and Action, MCPA’93, 241267. [loa] FLEET D (1992) Measurement of Image Velocity. KAP, Dordrecht NL. [l 11 FRANCESCHINI N, REHLE A, LE NESTOUR A (1989): Directionally selective motion detection by insect neurons. In: Facets of Vision, eds DG Stavenga & RC Hardie, ch 17, 360-390. [12] FRANCESCHINI N, PICHON JM, BLANES C (1992): From insect vision to robot vision. Phil Trans R SOCLond B: 337, 283-294. [ 131 GIBSON JJ, OLUM P, ROSENBLATT F (1955): Parallax and perspective during aircraft landings. Amer J Psychology, 68, 372-385. [14] GOETZ K (1965): Behavioural analysis of the visual system of the h i t f l y Drosophila. In: NYFi information processing in Sight Sensory Systems, Caltech, Pasadena 85-100. [l5] HASSLACHER B, TILDEN MW (1995) Living machines. Robotics and Autonomous Systems: the biology and technology of intelligent autonomous agents, Elsevier, Spring 1995. [16] JARVIS R (1993): Range sensing for Computer Vision. In: three-Dimensional Object Recognition Systems, eds Jain & Flynn. [17] MARTIN N, FRANCESCHINI N (1994): Obstacle Avoidance and Speed Control in a Mobile Vehicle Equipped with a compound eye. In: Proc of Inteligent Vehicles’94, Ed I Masaki, Paris pp 381-386. [18] MURA F, FRANCESCHINI N (1994): Visual control of altitude and speed in a flying agent. In: proc of the 3rd Intl Conf on Simulation of adaptive Behaviour, SAB’94, Brighton UK, Eds D Cliff, P Husbands, JA Meyer S Wilson pp 91-100. [19] MURA F, FRANCESCHINI N (1996): Visual perception by motion parallax is enhanced with retinal scanning. Submitted to Biol Cybern. [20] MURA F, MARTIN N, FRANCESCHINI N (1996): Biologically inspired eye movements for visually guided navigation of mobile robots. E S A ” 96, Bruges Belgium, ed M Verleysen, pp 135-147. [21] NAKAYAMA K (1985): Biological motion processing: a review. Vision research, 25, pp 625-660. [22] PICHON JM, BLANES C, FRANCESCHINI N (1989): Visual guidance of a mobile robot equipped with a network of self-motion sensors. In: Mobile Robots IV, SPIE 1195 Bellingham (USA), pp 44-53. [23] R A V N D, ALBUS JS (1992): A closed form massively-parallel range-from-image-flow algorithm. IEEE Transactions SMC, 22, pp 322-327. [24] SANDINI G, GANDOLFO F, GROSS0 E, TISTARELLI M (1993): Vision during action. In Active Perception, ed Y Aloimonos, Lawrence Erlbaum, pp 151-190. [25] SOBEY P (1994): Active navigation with a monocular robot. Biol cybern, 71, 433-440. [26] VON HELMHOLTZ H ( 1867): physiological Optics Vol I1 (in french). Reprint 1962 edition, Dover, London. [27] WHTESIDE TCD, SAMUEL GD (1970): Blur zone. Nature, 225, 94-95.