Mobile Robots for Tree Care

Information and Technology for Sustainable Fruit and Vegetable Production

FRUTIC 05, 12 – 16 September 2005, Montpellier France

Mobile Robots for Tree Care Simon Blackmore UniBots Ltd, England. [email protected] Henrik Have The Royal Agricultural and Veterinary University, Denmark. Rashid Shariff University Putra Malaysia, Malaysia. Noboru Noguchi Hokkaido University, Japan Abstract Mobile robots for tree care are inevitable. Many of the base technologies have been developed and recent breakthroughs in behavioural robotics now give us the framework to build truly autonomous robots that can work for long periods of time by themselves servicing and caring for trees. Examples are given of simple line following robots for tree spraying, through to ones that can mechanically remove weeds under a canopy and robots that build their own maps based on sensory data. Keywords: Agricultural robot, Autonomous tractor, mechanical weeding. INTRODUCTION Agriculture, horticulture and forestry have benefited from many advances in technology over the years, with developments in mechanisation, plant breeding, and chemical inputs. We are on the brink of a new wave of developments now, in the form of smarter machines that can carry out a number of repetitive tasks by themselves. These machines are currently being developed at universities and research organisations, so it is only a matter of time before they become a commercial reality. Tree crops offer a special set of opportunities and challenges for mobile robots as the trees are highly structured (planted in rows), semi permanent (in place for many years) and have a closed canopy, when mature, that will inhibit GPS reception. As the trees grow and develop the operations that are needed change as does the way in which we interact with the trees. Autonomous vehicles are already widely used in industrial production areas, where environments are regular and stable but robots for use in open environments are much more difficult to develop. No such machines have yet been marketed for professional use, but many experimental vehicles have been developed. The control of the newer ones have been based on the high precision Real Time Kinematic (RTK) GPS, computer vision and other advanced sensor systems (Wilson 2000) Pilarski et al. 2002) and involve relatively simple system architectures. During recent years many researchers have started to develop more rational and adaptable vehicles based on a behavioural approach (Blackmore et al. 2004; Bak and Jakobsen 2003; Have et al. 2002). Truly autonomous mobile robots will use significantly more complex control systems and

737



architectures to deal with the complexity of the interaction between robots and the seminatural environment in a safe and efficient manner. There are three main categories of operation for mobile robots for tree care; Ground care around the trees, interaction with the canopy and manipulation inside the canopy. Robotic ground care The primary task in this category is controlling grass and weeds to reduce the competition for moisture and nutrients. Many farmers use herbicides to control weeds but there is a growing concern that they are damaging the environment. The alternative is to use physical methods that can be combined with autonomous vehicles. One such example is the autonomous Christmas tree weeder (ACW) being developed in Denmark (Have et al. 2005). Christmas trees are seen as an agricultural crop, as they are grown for 7-10 years on agricultural land before harvesting. When they are first transplanted, weeds are controlled by using spring tine harrows as the young saplings are very supple. After about 2 years they become more brittle and need extra care as wear on shoots by weeds can affect the growth and shape and thus the quality of the mature tree. When the tree is over about 1.5 meters, the canopy is usually large enough to compete on its own. This intermediate stage is where an autonomous weeder can be used. A standard ride-on lawn mower from Stiga, was adapted for computer control. The man-machine interfaces were removed and substituted with linear actuators. The main cutter was taken off and an active side cutter mounted on the right hand side. The vehicle functions (steering, clutch/brake, gearshift and throttle) were controlled by a CPU running Simulink from Matlab. Simulink was chosen as a programming environment to help speed up the development cycle and reduce the possibility of programming errors. When the trees are small, the primary navigation system can be GPS as there is a clear view of the sky. The ACW used a RTK GPS that had a dynamic accuracy of 2-3 cm. The position of each tree was surveyed before tests started.

Figure 1. (Left) Young Christmas trees with patchy weeds. (Right) Autonomous Christmas tree weeder with side cutter control box and RTK GPS aerial.

738



The tree positions were used to define a suitable vehicle route plan, which in turn was used to control the machine movement in the rows. The tree positions were also used to dynamically control the swing-arm rotor cutter relative to the trees. The distance was calculated and when the cutter became too close, the arm was withdrawn until the tree (position) had passed. One problem with this method is to know exactly where each tree is located before the ACW can begin. In existing trials, each tree was geo referenced manually with an RTK GPS. Alternatively, a system for automatic registration has been developed by Khot (Khot et al. 2005) that used an RTK GPS, Fibre Optic Gyroscope (FOG) and a scanning laser to register (simulated) trees (Figure 2 left). When the tree trunks are clear and visible automated systems like these can be used but when the trunks are obscured by low foliage, the task can only be done manually. As the ACW navigation system kept the vehicle on the predetermined track between the tree rows, the controller calculated the distance between the cutter and the tree. When in active mode and the cutter became too close, the arm was withdrawn until the tree has passed. In passive mode the cutter surround was allowed to touch the tree and push the arm backwards until it could spring out again. This method was the most accurate as it cut the grass and weeds right up to the tree itself and did no significant damage to the tree, although repeated touching may cause problems. Tests (Figure 2 centre) have shown this to be an effective weed control for young Christmas trees. When in active mode, the cutter could repeatedly get to within 3-4cm of the tree trunk without touching it, even when the canopy totally obscured the trunk from sight. Robotic tree care When a tree canopy becomes large enough to obscure the required GPS line-ofsight, other methods of navigation are needed to keep a mobile robot between the rows. One a priori factor is that we know the structure of the trees (they are grown in rows) and we know the row widths. Using the laser scanner this tree structure can be recognised and used to define the desired tractor route. One method to extract the tree rows from the laser data is to use a Hough transform (Barawid et al. 2005). In tests, there was no significant difference between the routes under GPS control and laser control (Figure 2 right).

739



Figure 2. (Left) Robot mapping simulated trees, (Centre) Autonomous Christmas tree weeder cutting grass under trees, (Right) Robot tractor scanning mature trees. Robotic tree sprayers Two commercial autonomous tree sprayers have been developed in Japan. Although they use a very simple method of steering they still function happily by themselves with smart sending and control of the spray nozzles. For the vertical boom type robot (Fig 3 left), the primary form of navigation is achieved by using the rigid properties of plastic irrigation pipes that are laid out on the ground between the trees. The steering linkage is joined to small jockey wheels that attach to the irrigation pipe and the robot follows wherever the pipe leads. It has the capability to sense and adjust the spray pattern according to the tree height. The air assisted type (Fig 3 right), follows a wire in the ground and senses the EMF to be able to adjust the steering. It can sense when it comes out from the trees during turning and switch off and on automatically. Both systems were developed by Institute of Agricultural Machinery, BRAIN, Japan.

740



Figure 3 (Left) Autonomous vertical boom sprayer, (Right) Autonomous air assisted sprayer. Robotic instrumentation platform There are a number of measurement tasks under tree canopies that could be automated. Many sensing systems require lengthy periods of time to get a representative sample which makes this data expensive and unpractical if a person is required to attend. If these sensors can be mounted on an autonomous mobile instrumentation platform then large areas could be surveyed over long periods of time without having to pay a person to be there. One such environment is under oil palm trees in Malaysia. With the advancement of mobile computing technology such as handheld computing that integrates with wireless technology and the GIS software itself, autonomous mobile robots can be assigned to work in large oil palm plantations. These systems can be combined together with sensor systems such as infrared data association (iRDA). The robot position can be determined by RTK GPS where there is no canopy and by laser scanning when the canopy is closed. Topography under the trees can be estimated between known and accurate positions from the RTK GPS by using a FOG as the runs are only comparatively small. Supplemental context data can be made available by adding other sensors such as inclinometers for aspect and posture correction. Additional sensors can add value to the process by measuring the Leaf Area Index (LAI) which is an important part of modelling oil palm yields. Alternatively, an electronic nose would make it possible to determine if the appropriate fertilizer and pesticides have been applied at the correct locations, according to a pre-planned spraying schedule. This will be an efficient way to monitor the activities of the labourers and sub-contractors assigned to this task. Soil moisture can be mapped and areas that are water logged can also be monitored and appropriate drainage work carried out in a timely manner so that tree health is maintained and productivity of the tree can be fully realized. Using a vision system, oil palm fruit bunches that are at the correct ripening stage can be identified and harvesting of this bunches carried out. This will avoid the harvesting of under ripe bunches and minimize the harvesting of over ripe bunches, both of which give a lower grade of oil (Shariff et al. 2005). Robotic manipulation Many research projects have built robotic fruit harvesting systems using machine vision and robotic arms. These tend to be static laboratory robots, using colour

741



segmentation to identify the ripe fruits and laser or stereo vision to identify the range to the fruit. The arm can then move to the predetermined coordinates to grab and pick the fruit. One main problem is occlusion of the fruit. Some researchers have overcome this by growing the trees and vines in a Y configuration so that the fruit hangs down, ready to be picked. These systems, although technically sound have not offered many advantages over hand picking as they tend to be very slow. This may well change as more CPU power becomes available. Recent research has lead to significant improvements in the handling and throughput of mechanical harvesting from trees that could make the system more viable (Peterson 2005). A much larger forest machine that is not really a robot but very advanced nonetheless, is the semi-automatic walking forest machine built in Finland by Plustech, now owned by John Deere (See Figure 4 left). It is simple to operate as the legs are all computer controlled and the driver controls the speed, direction and attitude. Legs are used as it can walk over fallen trees and congested areas, whereas a wheeled vehicle would be stopped by the first fallen log! The original development of the walking robot called the Mecant, was done at the Helsinki University of Technology (Halme 1999).

Figure 4. (Left) Two Forest Walking Machines and (Right) the prototype Mecant Another walking forestry robot has been designed at SLU in Sweden. Figure 4 shows the concept vehicle in the forest but it is only made of wood! (Figure 5 left)

742



Figure 5. (Left) Mock-up of an autonomous tree harvester, (Right) Robotic tree climber. A robotic tree climber was developed at Cranfield University in England that could move along the ground, wrap itself around a date palm and climb up to inspect the fruit. (Figure 5 right) Although the prototype worked, it was never commercialised (Shamsi et al. 1998) CONCLUSIONS Various aspects of tree care can be automated to the point of making them fully autonomous. Technically many of the sub tasks have already been developed but require a smarter mobile platform that can work for long periods of time unattended. Navigation amongst young trees is relatively easy as the clear view of the sky allows the use of RTK GPS but when the canopy has closed or the trees are mature other proximity sensing techniques are needed. These mobile robots are likely to be first commercialised in the areas where the economic benefits are clearest and the tasks are the simplest. Literature Cited Bak, T. and Jakobsen, H. (2003) Agricultural Robotic Platform with Four Wheel Steering for Weed Detection. Biosystems Engineering 87:2125-136. Barawid, O. C., Tsubota, R., Ishii, K., and Noguchi, N. (2005). Agricultural autonomous vehicle that used 2-dimensional laser scanner as the navigation sensor. First Asian conference on Precision Agriculture. ed. S. Shibusawa. Fuchu, Tokyo 183-8509, Japan, Tokyo University of Agriculture and Technology. pp.89-94. Blackmore, B. S., Fountas, S., Tang, L., and Have, H. (2004) Design specifications for a small autonomous tractor with behavioural control. The CIGR Journal of AE Scientific Research and Development http://cigr-ejournal.tamu.edu/Volume6.html:Manuscript PM 04 001 Halme, A. (1999). MECANT-ten years history of a walking machine test-bed designed for developing outdoor applications. 2nd International Workshop on climbing & walking robots. Portsmouth, University of Portsmouth. 81. Have, H., Blackmore, B. S., Keller, B., Fountas, S., Nielsen, H., and Theilby, F. (2002). Autonomous weeder for Christmas tree plantations - a feasibility study. AgEng02. Hungary, Scientific Society of Mechanical Engineering. -8.

743



Have, H., Nielsen, J., Blackmore, B. S., and Theilby, F. (2005). Development and test of an autonomous Christmas tree weeder. 5th European conference on Precision Agriculture. ed. J. Stafford, V. The Netherlands, Wageningen Academic Publishers. pp.629-635. Khot, L. R., Tang, L., Blackmore, B. S., and Nørremark, M. (2005). Navigational Context Recognition using Onboard Sensory for Autonomous Weeding Robot in Tree Plantations. Transactions of the ASABE (In Press). Peterson, D. L. (2005) Harvest Mechanization Progress And Prospective For Fresh Market Quality Deciduous Tree Fruits. Horttechnology 15:172-75. Shamsi, M., Kilgour, J., Godwin, R. J., and Blackmore, B. S. (1998). Design Development and Testing a Date Harvesting Machine. 13th International Conference on Agricultural Engineering. CIGR. -12. Shariff, A. R. B. M., Chin, K. S., Abdullah, A. F., Mansor, M., Halim, R. M., and Mispan, R. M. (2005) Estimation of Free Fatty Acid (FFA) in the Oil Palm Fruit Using Imaging Technique. Journal of Institute of Engineers Malaysia(submitted). Wilson, J. N. (2000) Guidance of agricultural vehicles -- a historical perspective. Computers and Electronics in Agriculture 25:1-23-9.

Des robots mobiles pour l’entretien des arbres Mots clés : robot agricole, traction autonomes, désherbage mécanique Résumé Les robots mobiles pour l’entretien des arbres sont inéluctables. La plupart des technologies nécessaires ont été développées et de récentes avancées nous fournissent maintenant un cadre pour concevoir des robots réellement autonomes, capables d’accomplir leur tâches d’entretien des arbres pendant de longues périodes temporelles. Notre propos est illustré par des exemples de simple suivi d’alignements pour la pulvérisation, mais aussi par des robots qui peuvent effectuer un désherbage mécanique sous une canopée ou encore construire leurs propres cartes environnementales à partir de capteurs.

744