The Autonomous Mobile Robot AURORA for Greenhouse Operation A. Mandow, J.M. Gómez-de-Gabriel, J.L. Martínez, V.F. Muñoz, A. Ollero, A. García-Cerezo Department of Systems Engineering and Automation University of Málaga, 29071 Málaga, Spain e-mail:
[email protected] Abstract - AURORA has been conceived in order to substitute hard and unhealthy human work inside greenhouses by means of an autonomous mobile robot outfitted with appropriate sensors and operation devices. Emphasis has been put in the development of a new robotic platform specifically designed for greenhouse tasks, governed by a control architecture that supports both autonomous navigation and shared human control. Keywords: AURORA; autonomous mobile robot; autonomous navigation; greenhouse operation; shared human control; agriculture; mobile robots; telerobotics; reactive navigation; navigation behaviors. _____________________________________________________________________________________ This document is a self-archiving copy of a copyrighted publication. The published article is available in: http://dx.doi.org/10.1109/100.556479
How to Cite: Mandow, A., Gómez-de-Gabriel, J.M., Martínez, J.L., Muñoz, V.F., Ollero, A., García-Cerezo, A. (1996), The Autonomous Mobile Robot Aurora for Greenhouse Operation. IEEE Robotics and Automation Magazine, 3(4):18-28. @ARTICLE{Mandow1996RAM, author={Mandow, A. and G\'{o}mez‐de‐Gabriel, J. M. and Mart\'{i}nez, J. L. and Mu\~{n}oz, V. F. and Ollero, A. and Garc\'{i}a‐Cerezo, A.}, journal={IEEE Robotics Automation Magazine}, title={The autonomous mobile robot AURORA for greenhouse operation}, year={1996}, volume={3}, number={4}, pages={18 ‐28}, doi={10.1109/100.556479}, ISSN={1070‐9932}, }
_____________________________________________________________________________________ © 1996 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.
The Autonomaus Mabile Robot AURORA far Greenhause Operatian AURORA has been conceived in order to substitute hard and unhealthy human work inside greenhouses by means of an autonomous mobile robot outfitted with appropriate sensors and operation devices. Emphasis has been put in the development of a new robotic platform specifically designed for greenhouse tasks, governed by a control architecture that supports both autonomous navigation and shared human control. Keywords: Mobile robots, autonomous navigation, agronomic operations, control architecture, telerobotics, ultrasonic transducers ...................•............................•.................................... obotics in agriculture has experienced significant development in the last decade. In particular, research has been devoted to plant inspection [1], transportation [2], grafting [3] and, especially, harvesting of fniits and vegetables in horticulture [4]. Research has al so been carried out in the field of mobile robots that are equipped with devices for agricultural operation. The introdLiction of autonomous machinery in agriculture must conform to specific constraints, different from those of typical industrial applications [5]. These robots must cope efficiently with all the harsh navigation conditions presented by agronomic environments, where few assumptions can be made about the nature of the rugged terrain or about the obstacles-mostly plantsthat offer peculiar sensing conditions, are hard to model, and evolve in a daily basis. Greenhouses are translucent buildings for rearing Or hastening the growth of plants. This agricultural technique is massively used for intensive production of horticultural and ornamental products in regions with otherwise adverse natural climatic conditions beéause it allows an improved use of water and daylight. These constructions are commonly rectangular, and although their dimensions are very variable, they
R
18 • IEEE Robotics & Automation Magazine
are on the order of 100 meters in length and 20 meters in width. The distribution of plants inside greenhouses varies with different facilities and cultures, bui plants are typically arranged in double rows, with narrow corridors for human operation. The favorable conditions created inside greenhouses for plant growth cause pests and undesirable organisms to thrive as well. The most common approach for fighting them is the use of pesticides, fungicides and other chemical products that must be sprayed directly over the plants. Recent studies have reported evidence that spraying operations pose a hazard to the health of knapsack sprayer human operators [7], who are specially exposed when working inside greenhouses, in conditions of high temperature and poor ventilationó Recent studies in Almeria, a Spanish province with more than 20,000 hectares of greenhouse cultures, have identified skin diseases, mucous alterations, conjunctivitis, and deviant results in blood analyses from a sample of greenhouse workers, not to mention the long-term cumulative effects, which may have a noteworthy effect on chronic diseases and even mortality. Therefore, the automation of spraying, as well as other greenhouse operations like measurement and control bf environmental conditions,
1070-9932/96/$5.00 © 1996 IEEE
December 1996
harvest support, plant inspection, and artificial pollination, has a significant social and economic impacto Current methods for greenhouse automation of spraying are based on fixed gantry structures that spread the treatment from above. These solutions imply installing the whole system exclusively for one greenhouse. Their main drawbacks stem from their lack of flexibility: they are not suitable for all types of greenhouses, they require an investment for each different greenhouse, and their operation is limited to Figure 1. TheAURORAMobile Robot. gross spraying from a fixed upward position, which does not allow for efficient localized or systematic treatments. The AURORA mobile robot (see Fig.1) allows unmanned agronomic operations in greenhouses. The mobile platform, which navigates autonomously along greenhouse corridors by means of a low cost sensor system, carries an integrated spraying device. A remote console supports human supervision and shared control. This system aims to efficiently eliminate the need of human walk-through operation for such hazardous tasks as knapsack chemical spraying. The following requirements have been considered for the development of the prototype: 1. Reliability of the components to environmental factors such as temperature and humidity, that may, for instance, affect sensor performance. 2. Navigation in unaltered greenhouses. It is undesirable (and probably impractical) to modify their layout or to introduce beacons or any other artificiallandmarks for robotic use, since agricultural machinery is made profitable by using it on wide extensions or on numerous different plantations. 3. Low cost, so that an eventual commercial prototype can be more competitive than fixed equipment for greenhouse automation. 4. Flexibility, so that it can navigate autonomously in different kinds of greenhouses and plantations. 5. Multifunctionality is a desirable trait for competitive agricultural robots, so it must be possible to swap between different kinds of tasks easily. 6. Supervisable autonomous operation. The user needs immediate and comprehensive information about the status of the task while the robot is working autonomously inside the greenhouse. The user should also be able to remotely share control, or even take complete control, in exceptional ci rcumstances. 7. Friendly user interface, so that tasks can be easily specified by end-users with minimal training. 8. Robust control system for safe operation. Although a simple spraying device has been incorporated in the prototype, the platform can be used to perform a variety of greenhouse operations when equipped with the adeDecember 1996
quate actuators and sensors. AURORA can navigate autonomously in greenhouses while performing a purposeful agronomic task according to a mission specified through its high level user interface. A teleautonomous system has also been implemented within the AURORA control architecture so that remote supervision of autonomous tasks and shared human control are possible. An overview of the main components of the AURORA mobile robot is shown inFig. 2.
GREENHOUSE NAVIGATION AND OPERATION From the navigation point of view, one of the most remarkable characteristics of greenhouses is that objects populating them (i.e., rows of plants) leave very narrow free spaces for navigation and operation. This trait can also be found in some industrial environments, such as storage facilities [6]. Productivity demands narrow corridors so that the number of plants grown inside the greenhouse is maximized. Automation of tasks by means of a mobile robot must accommodate to this principie, which means navigation in unaltered environments in which objects can be very close. Not only robot dimensions are affected by this trait, but also the navigation philosophy and the sensor system. Typical approaches for autonomous navigation in unaltered environments are either planned or reactive. Planned approaches rely on a map modelling the environment and a position estimation system for following a geometric trajectory, whereas reactive approaches are based on the perception of objects around the robot in order to avoid collision and find safe areas. The first of these two approaches poses three practical drawbacks for greenhouse autonomous navigation: 1) An explicit, up-to-date quantitative map description of the environment is necessary. In fact, trying to obtain an accurate map of a greenhouse and keeping it conveniently updated is not practical, since its extensive dimensions make it difficult to obtain measures with precision, and the shape of plants is very hard to model, not to mention their evolution in a day-to-day basis. 2) Even if that map exists, the robot needs an accurate position estimation system. An uneven terrain prone to wheel slippage makes readily available information from internal sensors (dead reckoning and inertial sensing) insufficient to determine the actual posture of the vehicle with the reliability imposed by such environments, where a position error of a few centimeters means a collision. 3) In the absence of accurate position estimation, the robot would not dare traversing intricate areas on account of uncertainty. Purely planned navigation algorithms usually deal with a representation of the world in which either the robot or the IEEE Robotics &Automation Magazine • 19
it could be inferred from a sequence of instructions such as "folIow the row of plants on the right; at the end of it turn left and folIow the corridor," and so on. Each possible instruction in the high level plan can be associated to the execution of a basic behavior. The set of behaviors to be implemented has to be designed by considering the nature of the environment and the kind of tasks to be performed, together with available sources of sensor information. The desígn of each navigation behavior involves identification of the natural landmarks to be tracked by the vehicle, as weIl as those for the detectíon of end-of-behavíor conditions. A complete description of the greenhouse navigation behaviors designed for AURORA is presented beIow.
THE AURORA MOB/LE ROBOT
Figure 2. An overview of the AURORA MaMe Robot.
surrounding objects are expanded according to the degree of uncertainty in robot position [10]. As greenhouse tasks demand considerably long paths, high uncertainty would make narrow corridors disappear from the map. The increase of uncertainty is particularly high in agricultural environments on account of the ruggedness of the terrain. On the other hand, purely reactive approaches continuously link actions to sensed information about the environment. They result in robots that find their way to a targeted position, moving among obstacles without colIiding with them, but they are not very adequate to accomplish the objectives of the missions intended for AURORA, which require traveling exhaustively along all greenhouse corridors. Recent trends in mobile robotics research have focused on the integration of reactive low level behaviors with high level planning in order to obtain speed and reliability in task-oriented navigation [11, 12]. Thus, instead of showing a purely reactive behavior, the robot can perform purposeful activities according to a strategic plan [9]. This approach is suitable for greenhouse missions, where a plan with a high level of abstraction is necessary for task-directed navigation whereas the basic navigation behaviors must demonstrate a high degree of reactivity to cope with a dense obstacle density. Moreover, the topological layout of agronomic environments (usualIy characterized by paralIel corridors) and their natural landmarks are particularIy fit to a qualitative description through high level task specification. Without an explicit representation of the environment, an implicit description of 20 • IEEE Robotics & Automation Magazine
Mechatronics The mechatronic system consists of an octagonal mobile platformo Its dimensions, constrained by narrow greenhouse corridors as weIl as the need to carry an on-board agronomic operation device, are 80 cm in width, 140 cm in length and only 1 m high, to achieve more stability. The operating device consists of a commercial knapsack sprayer, conveniently adapted to be included within the robot system, but maintaining its original smalI fueled motor. The power system is based on a 2600 Watt fueled on-board AC generator, which enables continued autonomous operation in agricultural areas. Furthermore, this solution facilitates the recharge of energy by refueling the tank with common gasoline, which is more easily available in most agricultural environments than battery recharging facilities. Moreover, the use of an AC power source has also permitted direct connection to the electric supply through a wire. This alternative has turned out to be very useful when AURORA works in closed smalI indoor environments, specialIy in the development laboratory. Every component of the robot has been carefulIy chosen, considering it must operate in a difficult environment with high temperature, humidity and luminosity levels. In addition, the vibrations produced by the spraying device and the AC generator must be taken into account. An industrial computer chassis houses aH the low-power components of the robot, providing room, isolation, ventilation, protection and power supply. The locomotion system is a modification of the RAM-l dual configuration [8]. BasicalIy, it consists of four wheels located in the vertices (J. of a rhomb with a diagonal in the longitudinal axis, which l/y renders high maneuverability (zero turning radius), essential for constrained motion spaces. The Figure 3. Steering angle. December 1996
rear and front wheels are for steering, while the two central parallel wheels are for propulsion. For motion control, it is necessary to apply different speeds to the driving wheels and the corresponding steering angle to the directional wheels. This non-holonomic configuration is shown in Fig. 3, where it can be observed that the steering angle a is only a function of the desired curvature y. The main drawback of this locomotion system is the actuator redundance that arises from the difficulty in perfectly synchronizing the steering and propulsion systems. Wheels are an important issue to consider when dealing with traction of agricultural machinery. In order to maintain soil humidity in greenhouses, the floor is usually covered with either gravel or an isolating fabrico The noteworthy difference in terrain softness of these alternative solutions would make it advisable to adopt different wheels, depending on the characteristics of the target greenhouses. In particular, the bigdiameter rubber pneumatic wheels (40 cm) adopted for AURORA are efficient for the second type of covering. The clearance of terrain irregularities requires independent suspension for every wheel. However, the pitch and roll angles are not measured for navigation; they are considered motion perturbations that must be compensated indirectly by the navigation system. The front and rear directional wheels in the longitudinal axis are steered at the same time by a triphase AC motor with a rigid link, while the two parallel drive wheels in the transverse axis are driven independently by triphase AC motors. The torque applied by the driving motors to the wheels is increased by a mechanical reduction gear, resulting in a top speed of 0.82 mlsec. Each of the motors has a digital frequency governor that controls its velocity of rotation depending on the input voltage. No brakes have been installed on the driving motors, as long as the driving speed is not high and the working surface is more or less plain. Additionally, a small CC motor with reduction is in charge of the activationldeactivation of the spraying device. The state of the motors' digital frequency governors and controllers as well as the coherence of sensor readings are continually tested in order to stop the system immediately in case of failure. Additional proprioceptive sensors have been included in order to obtain the vehicle's power status, the spraying device status, andthe detection of malfunction conditions. An incremental encoder placed at the end of each driving motor shaft is used to control the motor speed and to estimate the distance traveled by the vehicle. Likewise, the information provided by another relative encoder is used to control and measure the steering angle. Sensor System The reactive capability of autonomous navigation relies heavily on external sensor information. Therefore, the performance of the navigation system will depend on the type and configuration of the robot's sensing system, especially if a low cost solution is to be adopted. Moreover, the design of a cost-effective sensor system for a real world robot should be related to the operational requirements of both the environment and the tasks to be performed. In particular, greenhouses pose specific considerations for December 1996
MRA
[
MRA
I MRD,MRA
I MRA,MRD
D
1
I
D
Figure 4. The sensor ranges. Table 1: Technical specifications oí ultrasonic sensors. Sensing range Switching frequency Ultrasonic frequency Repeat accuracy
SRD
MRD
MRA
6to30cm
20 to 100 cm
20 to 130 cm
8Hz
4Hz
4Hz
400 kHz
200 kHz
200 kHz
±0.45mm
±1.5mm
±2mm
5°
5°
Resolution Angle
5mm 5°
the choice of sensor types and configuration. Thus, long range sensors may become blinded when close objects cast shadows over more distant obstacJes or landmarks, while utilizing only short range sensors makes the robot myopic-and helplessly lost-when it reaches wider areas.The nature of agronomic tasks imposes further requirements for the sensor system, which must meet both operation and navigation aims. Sorne optical sensors, like infrared or low-cost laser range finders, may not be reliable for outdoor or high luminosity environments, where they can be dazzled by direct or reflected sunlight. Most existing applications use image processing and sonars. Image processing is an interesting solution, but it requires considerable computational cost, appropriate illumination conditions and does not provide a complete scope of the surroundings of the robot, so it must usually be complemented with additional sensors. Ultrasonic sensors offer a good and relatively low-cost solution, for they can be adjusted to work at different range distances and can thus be combined to sense both the immediate surroundings (a few centimeters) and mid-range distances (a few meters) from the robot. Moreover, ultrasonic sensed information, which can be digital (Le., merely indicating the presence of an obstacJe within the range) or analog (providing a value function of the distance to the nearest object), is frequently updated and requires little processing. Considering these criteria, a configuration composed of different kinds of ultrasonic sensors has been adopted for AURORA. Nevertheless, some limitations of sonars must be weighed and coped with. They produce numerous erroneous readings, IEEE Robotícs & Automatíon Magazine • 21
Industrial PC-80486 MS-DOS+ '--t---r-' Real-Time Kernel
~
ge O
'-' e O
Navigation and operatíon behavíors
""§
~~000 ~-'" 00°
'-'
.~~ oo"§ ~~ E =5~ c1l 3=~ ~o F
ü¿
el:
ü
g ~.g E ~ ~~ a5 « 0-' 8
·c ::J E E O
ü
I I
/V
:~ r .
I~ 1200 baud halfduplex Radio Modem
IV
U)
r:2
"O
e
ro e O .¡¡;
.2'
ID
Q.
::J U)
~
E
ID
eL
Figure 5. AURORA s sonar arrangement.
Figure 6. The AURORA implementation of the control architecture.
depending on the quality of the reflecting surfaces, and provide poor or no information about distant objects. The main source of errors in sonar readings is related to the reflective properties of objects to be detected. When folIowing a row of plants, especialIY those with thick foliage, like tomato plants, the rate of erroneous readings is not high because part of the beam is always reflected by some of the leaves. However, as long as the surface of plants is not homogeneous and varies from one species to another, redundancy is needed in order to obtain a reliable behavior. Another problem of ultrasonic sensors arises from the fact that ambient temperature affects the speed of sound and therefore sonar measures. For this reason, operation inside greenhouses requires sensors that are equipped with temperature compensating devices. Moreover, the use of industrial transducers is necessary in order to avoid corros ion produced by the on-board spraying systems. Using a graphical computer simulation, a worst case scenarío (numerous erroneous readings, uneven terrain, heterogeneous surfaces, etc.) was modeled to try out several feasible sensor configurations. As a result of this work, it was decided that AURORA would not carry a classical homogeneous sonar ring, but a specific configuration, so that a good relation between cost and efficiency is achieved. A combination of short range and long range digital and analog sonars has been implemented, covering the ranges shown in Fig. 4. The number and types of sonars that have been used are the following: four Short-Range Digital (SRD) sensors, two Mid-Range Digital (MRD) sensors, and four Mid-Range Analog (MRA) sensors with adjustable orientation (see Table 1 for technical details). These sensors conform to degree of protection IP65, so they 22 • IEEE Robotics & Automation Magazine
are not affected by the corrosive effects due to environmental humidity and smalI droplets drifted from the spray nozzles. Fig. 5 shows a CAD drawing of the sensor configuration. At each side of the robot there are two SRD sensors placed at different heights, keeping the 15 cm distance between them required to avoid interference. They produce redundant information about the immediate objects Uust a few centimeters away) surrounding the robot. This information is complemented by that of the side MRA and MRD sensors, which provide the distance to both immediate and distant objects. The front side of the robot is covered by two more MRA sensors. The sensor configuration of this prototype only covers the front half of the robot, so autonomous navigation has been restricted to forward motion, assuming there is enough room for turning at the end of each greenhouse conidor. The spray nozzles are oriented backwards, so sonar readings cannot be affected by the spray mist. The use of ultrasonic sensors for autonomous navigation in greenhouse spraying tasks offers the additional benefit of allowing night-time operation, sin ce no light is necessary. This way, toxic products have already settled when human operators perform other operations in the daytime. The external sensor system also incorporates a video camera, which has been added for remote human supervision purposes, and is not currently used by the autonomous navigation system. An on-board video transmitter sends analog TV images to the teleoperator station.
THE AURORA CONTROL ARCHITECTURE AURORA's backbone is provided by its control architecture, December 1996
Figure 7. The on-board user consofe.
~ARTIV
S""
Reseted
~~ E=,
r
~~
1
(exception)
(user key)
S top (u ser)
Star! (user key) Error (exception)
G)
Figure 8. States for vehicle control.
which blends the autonomous and te!eoperation systems in a coherent and comprehensive way, allows a friendly interface for task specification, remote operation and supervision, and supports the low level control of the vehicle. The actual implementation (depicted in Fig. 6) makes use of low cost computing resources and a standard operating system with a real-time kernel), which adds up to low cost and high availability. The control architecture has been designed by using traded and shared control concepts, allowing interactions with the environment, the user and the operator. The user locally interacts with the robot for start-up and shutdown, task programming through its on-board console, and joystick motion control for transportation or maintenance. On the other hand, the operator, who is located remotely, can communicate with different levels of the control system, which enables remote emulation of the user operations, performing collaborative guidance or taking control for resolving special situations, when required by the autonomous system or when desired by the operator. The architecture follows a hierarchical decomposition on five levels: User, Supervisor, Reference generation, Executive and Servo. Upper levels are characterized by sort real time constraints whereas the lower levels are hard real-time systems. Hardware mapping of the processes is depicted by doUed-line boxes in Fig. 6. A single PC80486 computer performs the soft and hard real time functions, according to the timing constraints of the horizontal levels of this hierarchy. Interface devices (such as motor control) are carried out by specific control processors that deal with the hardest real time requirements. The user level handles local user communication for startDecember 1996
up, shutdown, task programming, transporting and maintenance. Three emergence push-buUons are accessible for the local user to stop and lock the motion of the robot at any moment. The user interface consists of an embedded 2x40 character LCD display to show the current robot and task status, and a key pad with special editing and control iconic keys, as can be seen in Fig. 7. This user interface is also emulated on the teleoperation station console thanks to the remote user connection with the communication controller. The core of the supervisor level is the Supervisor Sequential Controller (SSC). Its sequential event-driven task description coordinates the behaviors of the reference generation subsystems. These subsystems issue "events" to the SSC in order to report the current status and to notify the normal termination of an activity or an unexpected condition. They must also give timely response to su eh incoming "events" as an activity start, activity stop, or synchronization point acknowledgment. The SSC coordinates the overall behavior by sending the activity start events and waiting for the incoming events and elock events. For this reason, the navigation subsystems are built as "behavior based" processes. The set of active (executing processes) activities, and consequently their emergent behavior is controlled by the SSC and driven by the incoming events. The rererence generation level is composed of a set of basic behaviors, each of which generates a specific robot motion schema. The activation of the right behavior is controlled by the SSC, and its output supplies motion commands to the lower level, based on the current sensorial information. The activation of these processes admits associated parameters for improving its performance. The normal or abnormal process termination is notified to the SSC, which selects the next step in the task sequence. The set of behaviors needed for the targeted tasks inelude "Wall following," "Corridor Following," "Turning," "Nozzle," "Safety" and "Advance," but new behaviors can be easily added if different kinds of tasks were to be performed by the robot. Sorne of these behaviors can be combined in order to obtain the desired performance during task execution. AURORA's executive level control is the interface with the internal sensors and actuators, encapsulating hardware details. This leve! ineludes a module responsible for sensor management and interfacing, and another for direct user guidance that enables driving the vehiele to its operation starting place by ordering changes of speed and curvature through a joystick. A third module, dedicated to vehiele control, achieves the vehiele dependent functions and brings robot independent services. Internal functions of this module inelude the inverse kinematic mode! so that the upper levels can be independent from the actual vehicle, considering actuation limits such as the maximum driving speed. This module also supervises the activation of the low level control loops, establishing different vehiele states and their related transitions (which can be seen in Fig. 8). The "starting" state represents the control initialization, "Normal" represents regular operation state, "Fail" is for control malfunctioning, and "Off' for detected external motor deactivation (when an emergence button is pressed). Another important task implemented in the vehicle control module is the coordination of the redundant locomoIEEE Robotics & Automation Magazine • 23
tion system, needed because of the difference between the acceleration of the driving and steering motors. Lack of coordination causes wheel slippage that increases position uncertainty and the difficulty to reliably control robot motion. Periodically, this coordinatíon task establíshes the speed of the dríving motors (characterized by a fast response) as a functíon of the actual position of the steering motor, which renders a slower response. Dead-reckoning is also achieved in this level thanks to a periodic process that maintains the vehiele status: X and Y position, orientation, distance traveled, as well as the actual speed and curvature. FinaIly, the servo level, the lowest in the control hierarchy, provides a set of hardware independent services for controlling the vehicle's mechatronic. eommon services of the hardware control module are wheel speed and steering reference settings. It is composed of the service high level interface libraries and the low level feedback control hardware and software. A dedicated processor board is devoted to feedback control, which iheludes the driving, steering and spraying motors, emergency buttons, the user joystick, management of data from sonars, and electrical connection for sensors and actuators.
add robustness to navigation. Thus, task specification aIlows introducing approximate odornetry by .using a numerical input along with each havigation instruction, as in "folIow the wall at least for 2 meters," or "turn left up to 90°." A gross measure of distance or angle suffices to simply and effectively avoid errors such as turning left prematurely because sorne plants are missing or have not been detected at the left while still in a corridor. Regarding operation, the nature of the spraying task imposes the condition of non-stop advance of navigatioh behaviors concurrently with it. AURORA's on-boardspraying device must be activated and deactivated depending on the current stage of the navigation task (see Fig. 9). This is specified by associating the spraying behavior to the navigation instructions that must be executed simultaneously (e.g., "Follow the corridor while operating"). Similarly, any other kind
TASK SPEClFICATION AND EXECUTlON A task is specified as a sequen ce of qualitative activities to be performed by the robot. By taking advantage of the inherent structure of greenhouses, this specification can be relatively simple, since it can be expressed as a sequence of ihstructions chosen from a reduced set of basic behaviors identified for that particular kind of environment, like foIlowing a corridor, foIlowing a wall, and turning. The user interface allows the edition of a sequence of behaviors for both navigation and operation, either from the on-board keypad or from the teleoperation station. For navigation, a different key has been assigned to each possible behavior, and programming is just as easy as pressing the iconic buttons in the desired sequence. At run time, each instruction in the task is instantiated by a behavior at the reference generation level of the control architecture. A sequential control approach, besides its simplicity, offers a natural and effective way for navigation control. Nevertheless, operation and safety behaviors are concurrently executed and managed by the supervisor during navigation. Behaviors rely on the perception of the structure of the plantation, and natural landmarks are sensed to detect the end-of-behavior conditions, which depend on the next behavior in the sequence. For instance, while following a corridor, external sensors are used to detect the presence of plants and their distance to both sides of the robot, and the end-ofbehavior condition will be triggered when the sensed data reveal that the end of the conidor has been reached, such as when no plants are detected. Data from external sensors plays a fundamental role in this scheme, but the combination of reactivity and odometry results in safer navigation behaviors all the same. While dead reckoning is unreliable for following a complete mission, it is also true that a rough odometric estimation relative to the short stretch in which a behavior is active can certainly 24 • IEEE Robotics & Automation Magazine
Figure 9. Autonomous spraying wifh AURORA.
+'-11-1~~.~
__________
V
_ ____
Far
J
,,,I,7¿ AURORA
~~~_1~
Close
Very: Close
~...
Figure 10.. Interprefation of three side sensor distances. C urvatu re (rñ ' ) Control ........ .
Compensatory
0.0 Compensatory Control
... -------_.,._-------
,, ,
• • •1
,
:
------:---------r-------_·--
Figure 11. The basics of corridor following. December 1996
of operations could also be linked to navigation steps or included in the sequence. The way these operations are specified would depend on their complexity. When a navigation behavior is active, it performs a particular interpretation of both internal and external sensor data, and produces reference values for the low level control of the vehicle: the desired velocity and curvature. It must be emphasized
Radio Modem
+ Real-Time Kernel joystick
Figure 12. Overview of the Tele-Operator Station (TOS).
Figure 13. The TOS is used for remotely supervising AURORA s operation from an adjacent facility by means of a radio modem, a TV receiver, a personal computer, and a video monitor.
f.!§.¡>,::¡",,;-
!fu% SUPEFtUI$!m !mv,€:':f) SPílc¡;:rnAU. >.kIt:¡. L"Urv.
0.74 -1.04
'Fthn
-LOO 2.00
U-~¡:j:«;
Figure 14. Hardcopy of the actual computer screen from the TOS. lt renders textual information about robot status (upper left comer) and a realtime graphic representafion ofsensor readings. A replication of the onboard display appears at the top of the image. The two fines starting upwards from the front wheel represent the commanded (and undelayed) curvature and the actual (delayed) curvature. (Colors are not shown in the figure.) December 1996
that this is not done by directly mapping sensor readings to control actions in a pure reactive fashion. The reason why specific navigation rules have had to be devised in order to implement behaviors is that rows of plants used as a reference are not perfectly even walls at all. A curvature command directly obtained from the current measured distance to them would render an uneven trajectory with abrupt curvature changes, even if no sensor errors occurred. What is more, given the proximity of the plants, such a navigation philosophy invariably results in the collision of the robot. Therefore, efficient navigation behaviors have been designed to cope with the possibility of erroneous sonar readings, uneven references and very close objects. They make use of two kinds of redundancy in order to reliably accomplish their aims: repeating readings from each sensor before taking an action, and integrating the results from different sensors covering the same area. The main features of AURORA's navigation behaviors are discussed below. Corridor Following. This behavior uses information from sensors at both si des of the robot. lts aim is to maintain symmetry between them. It has been implemented as a set of rules that considers three sensing distances at each side of the robot: very close, close, and far, as can be seen in Fig. 10. A very close distance is assumed if any of the short range sensors is active or if the analog sensor provides a reading lower than a given value. If that value is exceeded or the mid-range digital sensor is active, the distance is considered to be close. Far distances are inferred if no side sensor is active. There are three possible states for the corridor following behavior: centered, biased to the left and biased to the right. The centered state is achieved when both sides provide the same interpretation, either both close (a relatively wide corridor) or both very close (a constrained corridor). In such cases a zero valued curvature reference is produced. A biased state is reached if a different interpretation is repeatedly detected at each side. Presuming that greenhouse corridors are relatively straight and that while following a constrained corridor, the robot is always more or less aligned with it, smooth control actions are necessary for correcting a bias. As a result of this, two steps are repeatedly taken until the center state is regained (as illustrated in Fig. 11): first, a predefined curvature reference, maintained for a short period of time, produces a small change in orientation, making the robot to turn away from the "very close" side, and second, the curvature is set to zero, so that the robot proceeds with its new orientation for sorne time. Once the centered state has been reached, a slight compensatory curvature reference is given with the opposite sign in order to avoid the robot to oscillate excessiveIy from one si de to the other. Wall Following. The wall following behavior is similar to that for corridor following. It utilizes only the lateral sensors corresponding to the si de of the wall. A biased status is detected if the wall comes into the short range of the sensors (too close) or when a particular distance is detected through the MRA analog sensor (too far). The intermediate positions produce zero curvature. Turning. A combination of local odometry and reactive sensing has been implemented for this behavior. Odometry is IEEE Robotics & Automation Magazine • 25
monitor, as can' be seen in Fig. 13. An overview of the TOS scheme is shown in Fig. 12. This is a PC based workstation, with a radio modem, . a genlock module (for computer on video overlay), a TV receiver, and a video monitor. This station enables the operator to perform such activities as monitoring, task programming, task intervention, and shared or traded robot control. Communication between TOS and AURORA features multi -Ievel Figure 15. Executing an autonomous greenhouse spraying task. a) Corridor following, b) Reaching the end of the access, which means communication at the user, supervicorridor, c) Tuming. sion, reference generation used as a guide to limit the angle to be turned. However, if a and executive levels of control. This makes possible the close object is detected by the side sensors before that angle is achievement of many different operations from the operator completed, the turn will also be finished. The turning station, including complete control. curvature is specified beforehand; in greenhouse operation, Several manlmachine interfaces have been experimented the change to a new conidor presents a narrow turning space, during development: SpaceBall"" (6 dof force/torque sensor), so the default vaIue is that which mini miz es the area covered mouse, digital (incremental) joystick, analog (absolute) joyby the robot during the turn, that is a turning radius stick and keyboard in varÍous operating modes. These experisituated at the inner edge of the robot (in the case of ments have put forward the fact that a device with undelayed AURORA, 0.4 meters). . visual or tactile feedback is necessary for robust teleoperation. Safety. Executing concurrently with the navigation behavThe computer screen layout gives graphical representation iors presented aboye, the safety behavior contains a set of rules of the remote vehicle's status and sensor's readings (see Fig. to avoid collisiQn. The front analog sensors are used to detect 14). This information can be switched from wireframe to any object on the way of the robot during task execution. In its numerical representation and can be overlapped onto real current implementation, AURORA just stops and waits for the video images in order to have up to délte information of the object to be removed, which most of the times is the only possicommanded speed, needed for incremental interfaces, as well ble solution; obstacle avoidance has strong limitations while as about proximityto objects that are not within the camera operating inside a greenhouse, considering there is usually no fieId of view whiIe visually driving the vehicle. space left to avoid the obstacles on the way. Data communications are implemented Viél an RS-232 serial link, and a 1200 baud, half-dupIex radio modem for wireless REMOTE SUPERVISION AND CONTROL data transmission, as well as a communication channel for Besides the fully autonomous nélvigation system presented anaIog video and audio transmission for visual feedback only. aboye, AURORA has also been provided with a teleauThe communication protocol establishes the rules that tonomous system for operation and navigation. The need for make possible the exchange of information between the two such a system stems from several practical facts about robotic subsystems. Special requirements arise when managing operation inside greenhouses. control communications: reliabiJity, real-time and efficiency. First of all, the user of a robot which is operating This protocol follows the clientlserver approach, where the autonomously inside his or her greenhouse will TeIeoperator station is the client, and the mobile robot is the undoubtedIy want to know what is happening inside. A way server. In this schema, the TOS is responsible for initiating must be provided to remotely supervise autonomous the message transfer, and AURORA for returning the approoperation in order to detect unexpected events that may priate response, i.e., the TOS sends commands and AURORA halt operation, such as obstacles on the way. returns status messages. Every package consists of a Apart from supervision, the possibility of taking control single command or status message. The command and status is also desirable, either for helping the robot out message sets that embody the communication language are of trouble, cooperating with it for special operations that static, and previously established. may be hard to program, or even driving it to its starting position, out of the greenhouse after task execution, or from RESULTS AND FUTURE WORK one greenhouse to another. AURORA has been tested in different greenhouses, where. it The operator, who is located away from the working envihas been used for autonomous navigation and spraying tasks. ronment, communicates with the robot by means of the TeIeA successfuI demonstration for local greenhouse owners and Operator Station (TOS), which is a real-time computer the media took place in March 1995. Fig. 15 shows a sequence system, with man/machine interfaces attached, and a video of images from one of these experiments, in which AURORA 26 • IEEE Robotics & Automation Magazine
December 1996
navigates autonomously through a greenhouse corridor using only the information provided by the ultrasonic sensors. The program introduced through AURORA's keypad consisted of a loop of instructions so that the robot sprayed a greenhouse while navigating through a1\ its corridors. In these tests, AURORA repeatedly detected the same environmental landmarks specified in the task (corridor, turning point and wa1\), accomplishing the mission independently of the robot's precise starting position or the exact path fo1\owed. The images of Fig. 15 demonstrate how constrained navigation space can be in greenhouses. Under these circumstances, sharp control actions could easily traverse the robot across the conidor, making it stuck in it. While following a corridor (Fig. 15 a) only slight curvature changes are made for trajectory control. The uneven surface provided by the rows of plants is coped with by the ad hoc navigation algorithms presented above, which make use of redundant measurements. The second image in Fig. 15 represents the moment when the robot approaches the end of a corridor, which is detected as a free area by the side sonars. At this point, the vehicle prepares for turning by getting out of the corridor with enough space to proceed with the maneuver, which is about 80 cm, computed by dead reckoning. However, not all free spaces detected by the side sensors are really the end of a corridor (maybe a plant is missing); hence, if any object is detected at that si de along these 80 centimeters, the turning operation will be aborted, and the robot will resume corridor following. The turning behavior (Fig. 15c) sets a curvature of 2.5 m- 1 which minimizes the area swept by the robot. At the same time, the spraying activityassociated to the previous behavior is halted. The turn will come to an end either when the specified odometric angle is completed (in this case, 90°) or when close objects are detected by the inner si de sensors. The ultrasonic sensor configuration has proved effective for autonomous navigation, but it is also true that in sorne of the experiments AURORA was "afraid" of proceeding along a corridor when a loose branch hanging on from a plant was detected by the front sensors, activating the safety behavior. This is due to the impossibility of inferring the dimensions and shape of objects from ultrasonic readings. In these cases, the autonomous mission was resumed by a command from the remote supervisor. An interesting way to overcome this problem would be to take advantage of the on-board camera for simple image processing, which would add reliability for autonomous navigation at a low cost. Besides detecting natural landmarks, inexpensive color stickers could also be placed on the wa1\s at the end of the corridors for increased robustness. On the other hand, the issue of sensor fusion would certainly have to be coped with. Future work will also involve the development of special operation devices for greenhouse operations with mobile robots. The simple knapsack sprayer adapted for AURORA has been useful as a first approach to the spraying problem, only for demonstration purposes, but the design of a more efficient actuator is required for real applications. Furthermore, now that a reliable autonomous platform is available, other kinds of tasks should also be considered: harvesting, transportation of fruits, plant inspection, etc. December 1996
A CKNOWLEDGEMENTS This work has been partially funded by the P.N.!. C. AlmeriaLevante (Junta de Andalucía) and CICYT TAP93-0581. We are grateful to A. Simón and F. Carcía, from the Department of Mechanical Engineering (University of Málaga), for the construction of the prototype, and to our colIeagues M. A. Martínez and A. J. Reina for their valuable cooperation. We appreciate the support of the agricultural research centers of La Mayora (in Málaga), that let us use their greenhouses for the development tests, and FIAPA (in Almeria), site of the demonstration.
REFERENCES [1]
[2]
[3]
[4]
[5]
[6]
[7]
[8]
[9]
[10]
[11] [12]
G. Sandini, F. Ferrari, G. Garibotto, E. Grosso, M. Magrassi, M. Massa, and R. Toscan, "Vision-based Operations in Greenhouses." Proe. of the 2nd Intemational Workshop on Roboties in Agrieulture & the Food Industry. Genova (Italy), 1991. G. Gilbert and L. Ivan, "Handling Robots for Horticultural Greenhouses." Proe. of the 2nd Intemational Workshop on Roboties in Agriculture & the Food Industry lARP, pp. 9-17 Genova (Italy), June 1991. A. Bourely, E. Masson, E. Molto, M. Dominguez, F. Juste, F. Ferrer and S. Gudin, "ROSAL, a Grafting Robot for Woody Plants." Proe. of lARP 4th Workshop on Roboties in Agrieulture and the Food lndustry. Toulouse, Franee, October 1995. A. Bourely, G. Rabatel, A. Grand d'Esnon, and F. Sevila, "Fruit Harvest Robotization: 10 Years of CEMAGREF Experienee on Apple, Grape and Orange." Proe. of the AG-ENG 90 Conference (Germany), 1990. F. Sevila, "Mobile Roboties in Structured Environment, The Case of Hortieulture," Fruit, Nut and Vegetable Production Engineering. Published by INlA, Spain, 1994. E. Byler, W. Chun, W. Hoff and D. Layne, "Autonomous Hazardous Waste Drum inspeetion Vehic1e." IEEE Robotics and Automation Magazine, Mareh 1995. N. Bjugstad and T. Torgrimsen, "Operator Safety in Orehid Spraying." Fruit, Nut and Vegetable Production Engineering. Seleeted papers trom the 4th intemational Symposium. Published by INlA, Spain, 1994. A. Ollero, A. Simón, F. Garda and V. Torres, "Meehanical Configuration and Kinematie Design of a New Autonomous Mobile Robot." lntelligent Components and lnstruments for Control Applications, pp. 461-466. Pergamon Press, 1993. A. Ollero, A. Mandow, J. Gómez-de-Gabriel and V. F. Muñoz, "Autonomous Mobile Robot Operation and Navigation in Industrial Environments," Robotics & Computer lntegrated Manufacturing, Pergamon, Vol. 11, No 4, pp. 259-269, 1994. Lozano-Pérez T. and Wesley M. A. (1979). "An Algorithm for Planning Collision-Free Paths Among Polyhedral Obstac1es. Mobile Robot Trajectory Planning With Dynamie and Kinematie Constraints." ACM Communications of the ACM. Vol. 22, No 10. Gat, E., (1993) "On The Role of Stored Internal State in the Control of Autonomous Mobile Robots," Al Magazine, pp. 64-73, Spring 1993. Arkin, R. C., (1989) "Motor Sehema Based Mobile Robot Navigation." IntemationalJoumal ofRobotics Research, Vol 8, pp. 92-112.
Anthony Mandow graduated in Computer Science Engineering in 1992 from the University of Málaga, Spain, where he was awarded a post-graduate fellowship and is currently a research assistant and Ph.D. candidate in the Department of Systems Engineering and Automation. Mr. Mandow has taken part in several projects in the field of robotics and is the author of a dozen IEEE Robotics & Automation Magazine • 27
papers published in conferences, magazines and books. His major research interests inelude intelligent navigation systems, fuzzy modeling and sensor integration for real-world robot applications. Jesús M. Cómez de Cabriel was born in Córdoba, Spain in 1966. He received the M.Sc. degree in Computer Science in 1990 from the University of Málaga, Spain. He joined the Engineering Systems and Automation Research Croup as an assistant professor. Now he is a full professor of computer control systems at the Engineering School of the University of Málaga. His research is concentrated on telerobotics and control architectures. He participated in the construction of two mobile robots and a teleoperation system. Jorge Luis Martínez Rodríguez was born in June 1966 in Málaga, Spain. He received his B.Sc. in computer science in April1991 from the University of Málaga and his Ph.D. in October 1994 in computer science for his work in tracking explicit paths for wheeled mobile robots. Since November 1994, DI. Martínez has been assistant professor of control systems and robotics in the engineering school of the University of Málaga. He has collaborated in the development of the RAM-1 and Aurora autonomous vehicles and is currently working on RAM-2. Professor Martínez has had around a dozen artieles published at international congresses. Victor F. Muñoz was born in Málaga in 1966. He received the M.Sc. degree in Computer Science in 1990, from the University of Málaga. After holding a postgraduate fellowshíp, he joined the Engineering Systems and Automation Research Croup at the aboye university, where he began his research on mobile robot trajectory planning. In 1995 he received his Ph.D. and is now an assistant professor of robotics at the Engineering School at the University of Málaga, and an IEEE associate member. His research concentrates on smooth trajectory planning and generation for mobile robots. He has participated in the construction of two mobile robots and he is currently interested in both mobile robot navigation and medical robot applications. Aníbal Ollero received his Electrical Engineering degree (1976) and the Doctor of Engineering degree (1980) with honors from the University of Seville where he was assistant professor. Later he was full professor at the Universities of Santiago in Vigo and Málaga, where as also head of the Department and Director of the Engineering School. He was also a "stagiere" at the Laboratoire d'Automatique et d'Analyse des Systemes (LAAS-CNRS), Toulouse, France (1979) and a visiting 28 • IEEE Robotics & Aufomation Magazine
scientist (1990-1991) at the Robotics Institute, Carnegie Mellon University, Pittsburgh, USA. Since December 1992 he has been full professor at the Escuela Superior de Ingenieros, University of Seville. Professor Ollero is the author of a book on computer control ("Premio Mundo Electronico" Spanish award) and more than 150 scholarly publications. He participated and led more than 20 research and development projects on Robotics, lntelligent Control, and Artificial Intelligence, funded by SpaÍlish agencies, regional agencies, the European Community (ESPRIT and Telematics Application Programme) and several industries. He also developed industrial applications and consulting activitíes for companies and Spanish agencies. Professor Ollero is currently the Chairman of the "Components and Instruments Committee" of the International Federation of Automatic Control (IFAC), a member of the IEEE, and a member of several scientific societies on Control, Robotics and Artificial Intelligence. He is also a member of the European Laboratory of Intelligent Techniques Engineering (ELITE) and of the ERUDIT ESPRIT network of excellence of the European Community. He leads a research group in the "Asociación para la Investigación y Cooperación Industrial de Andalucia (AIClA)" and participates, through this Association, in several research projects and consulting activities for industries. Currently his main research interests are Robotics, Intelligent Control and Perception Systems. Alfonso Carda-Cerezo was born in 1959 in Sigüenza (Spain). He received the Industrial Electrical Engineer and the Doctoral Engineer Degree from the Escuela Técnica Superior de Ingenieros Industriales of Vigo, University of Santiago, in 1983 and 1987, respectively. From 1983 to 1988 he was Associate Professor in the Department of Electrical Engineering, Computers and Systems at the University of Santiago de Compostela. From 1988 to 1991 he was Assistant Professor at the same university. Since 1992 he has been a Professor and Head of the Department of System Engineering and Automation at the University of Málaga, Spain. Since 1993 he has been Director of the Escuela Tecnica Superior de Ingenieros Industriales of Málaga. In 1984 he was awarded the "Diploma de Honor" in the "Cran Premio Tribuna del Inventor Innovador," Mundo Electrónico, in Barcelona, Spain. Dr. CarCÍa-Cerezo is a member of the Automation and Robotics Research Institute of Andalucia. He is also a member of IEEE, the Spanish Association for Fuzzy Logic, and the IFAC Committee on Components and Instruments. He has authored or co-authbred about 60 journal artieles, conference papers, book chapters and technical reports. His current research included the application of fuzzy logic and neural networks to real time control and supervision of industrial processes, mobile robots and autonomous vehicles.
December 1996
