Orchard Tree Modeling for Advanced Sprayer Control and Automatic Tree Inventory Carl Wellington1 , Joan Campoy1 , Lav Khot2 , and Reza Ehsani2 Abstract— This paper describes two related precision agriculture applications within an orchard environment that are enabled by building models of the trees using a vehicle-mounted ladar sensor. First, we describe an advanced spray control application. Ladar data is fused with GPS to build a local 3D map of the trees. Then, using a Markov random field probabilistic ground model to separate trees from the ground, and tree density estimates computed from the ratio of ladar hits and pass-throughs, we control the flow rate of individual nozzles and the amount of air-assist on a modified sprayer. This improves the effectiveness of the spray, reduces the amount of chemical used, and reduces chemical drift. Second, we describe an automated tree inventory method that can count individual trees both when the boundaries between trees are visible and when presented with a wall of trees where the canopies all blend together. We use a hidden semi-Markov model to probabilistically combine ladar data of the trees with a prior on the expected tree spacing to generate a map of tree locations and size estimates. Both of these applications leverage probabilistic tree modeling strategies to incorporate contextual information to better interpret sensor data of the trees, and these approaches have been experimentally validated during field testing in a Florida citrus orchard.
I. INTRODUCTION Within the broader domain of outdoor robotics, agricultural applications generally have strong environmental structure that can be leveraged to improve performance. For tree orchards, this structure normally is in the form of rows of trees, as shown in Figure 1. We describe two related applications that use vehicle-mounted sensors to build a model of the trees in an orchard to reduce spray chemical usage while maintaining good leaf deposition, and automatically generate a tree count inventory for yield prediction and other uses. Trees in an orchard can vary significantly in height, width, and canopy density. This is especially true in orchards where trees are removed due to disease, so that small young trees are interspersed with larger mature trees (see Figure 1). We use a laser range finder (ladar) mounted on a tractor or other vehicle driving through the orchard and fuse this data with GPS to build a model of the trees that can then be used for various applications. By accumulating sensor data into a 3D map, we can include contextual information using probabilistic techniques, improving performance over solutions that use the sensor data directly without contextual information. This work was supported by the United States Department of Agriculture Specialty Crop Research Initiative (USDA-SCRI) 1 National Robotics Engineering Center (NREC), Robotics Institute, Carnegie Mellon University, Pittsburgh, Pennsylvania, USA. Corresponding author email:
[email protected] 2 Citrus Research and Education Center (CREC), University of Florida, Lake Alfred, Florida, USA
Fig. 1. Overhead imagery (left) and corresponding ladar height map (right) produced by a ladar mounted on a vehicle with GPS of a block of trees in a citrus orchard where there have been significant tree losses from disease
For spraying applications as shown in Figure 2, the optimal spray location and quantity is dependent on the size and density of the tree canopy. Existing advanced commercial sprayers such as the Durand Wayland SmartSprayTM and the Roper Tree-SeeTM use ultrasonic sensors to enable or disable banks of nozzles based on the presence of trees, as shown in Figure 3a. These sprayers significantly reduce chemical usage and unwanted chemical drift beyond the tree canopy but are limited because their ultrasonic sensors have low spatial resolution, the nozzles are turned on and off in groups, and they are not able to discriminate between tree canopy that should be sprayed and ground or weeds that should not be sprayed. Several researchers have investigated the use of ladar sensors to measure tree characteristics for advanced sprayer control [1][2] and the use of ladar to measure canopy density [3]. We expand on this work by accumulating ladar data in a 3D voxel density map [4], applying a probabilistic ground model to separate the trees from the ground, and then using the height and density measurements from the trees to control air flow and individual nozzle chemical flow rates of a modified commercial axial-fan airblast sprayer [5]. This allows us to reduce spray chemical usage by only applying it where necessary and with flow rates optimized to improve chemical deposition. For yield prediction and farm management, it is important to have an accurate and up-to-date inventory of tree locations and sizes. Currently, this is generally performed manually by driving up and down the rows of the orchard with a
(a) Ultrasonic sensors Fig. 2.
Autonomous tractor spraying a citrus orchard
hand counter to collect tree counts of different sizes per row or block, but this is a tedious process and does not give the locations of individual trees. Several researchers have investigated the use of vehicle-mounted ladar to automatically extract tree height and volume measurements [6][7][8], including comparisons with ultrasonic [9] and stereo camera measurements [10]. Although these more direct estimates of tree volume have been shown to correlate with fruit yield [11], in our conversations with growers we have found that there is a desire to automatically generate a tree inventory that is compatible with their current manual process but also gives GPS coordinates for each tree and integrates within their existing Geographic Information System (GIS) orchard management process. By counting trees instead of estimating overall tree volume, we allow the growers to use their existing yield prediction methods, combine the automated system results with manually collected tree counts from current and previous years, and most importantly compare this new approach to their existing approach directly which lets them build trust in the system. To produce an accurate tree count both when tree boundaries are visible and when the tree canopies blend together, we use a probabilistic model to combine ladar data of the trees with the expected tree spacing to generate a map of tree locations. This approach has been compared against manually collected ground truth data in a large citrus orchard. These applications both leverage sensor-based tree models to provide benefit to the farmer, and these approaches have been experimentally validated using real data from Florida citrus orchards, although other crops could similarly benefit. II. S PRAYER C ONTROL An axial-fan airblast sprayer is shown operating in Figure 2, and Figure 3a shows a baseline commercial sprayer that uses ultrasonic sensors to turn banks of nozzles on and off. To achieve greater spray control, we have added pulse width modulation controlled solenoid valves shown in Figure 4 that allow the sprayer to adjust individual nozzle spray output rates. As shown in Figure 5, we also added air-diverting louvers mounted on both sides of the sprayer to change the amount of air-assist the spray mix receives. The variable liquid flow rate and variable air-assist allow precision chemical application control and can be commanded over CAN bus. More details on the sprayer retrofit are available in [5].
(b) Ladar rear
(c) Ladar front
Fig. 3. Comparison between commercial ultrasonic sensors (a) and two alternative mounting locations for our 270 degree ladar that scans perpendicular to tractor motion: high behind the cab facing down (b), low in front of the hood facing up (c)
9
10
10
9 8
8
7
7
6
6 5 4
5 4
Nozzle Angle 3
2
1
Left Nozzles
1
2
3
Right Nozzles
Fig. 4. Rear of retrofitted axial airblast sprayer, showing fan intake for air-assist and radial computer controlled variable flow rate nozzles
(a) Louver open (max air-assist)
(b) Louver closed (min air-assist)
Fig. 5. Side of retrofitted sprayer showing air-diverting louvers that were added to provide variable air-assist to the spray nozzles
To overcome the limitations of ultrasonic sensors described in the introduction, we use a 270 degree ladar sensor mounted as shown in Figures 3b and 3c to scan perpendicular to the tractor motion. We accumulate this ladar data using a standard commercial GPS-INS that provides the position and orientation of the vehicle to create a 3D map of the trees as shown in Figure 6. Accumulating data into a 3D map in this way allows us to build a consistent map over different vehicle speeds, during turns, and as the vehicle pitches and rolls when traveling through an orchard, although for orchards where these motions are relatively small, accumulating data using vehicle speed alone is often sufficient [4]. The goal of this work is to use this 3D tree map to control the sprayer as shown in Figure 7. To do this effectively, the system needs to avoid spraying the ground and adapt the nozzle and air flow rates based on tree height and density.
Fig. 6. Example 3D ladar data of the trees collected while driving up and down the rows
Fig. 8. Producing a ground height estimate from the 3D ladar data allows the system to discriminate between trees that should be sprayed and ground surface that should not be sprayed Fig. 7. Visualization of the spray control system during a live test showing estimated ground surface in gray, the 3D voxelized ladar data colorized by height, and the active spray nozzle zones filled in with green
Many citrus orchards have driving lanes that alternate between flat rows called “beds” or “tops” (see Figure 2) and lower swale rows (see Figure 8). In swale rows, the bottom several nozzles of the sprayer may spray into the ground instead of on the trees as desired, as shown in Figure 8. The sprayer can be manually configured for swales but this is a time consuming process and the depth and shape of swales varies throughout the orchard. We use a Markov Random Field (MRF) to estimate the ground surface from the accumulated 3D ladar data [12]. This probabilistic model combines the lowest 3D ladar data in each location with a prior distribution on ground smoothness that enforces consistency between neighboring locations. Intuitively, estimating a ground surface with a MRF is like fitting a rubber sheet to the lowest ladar data where the neighborhood prior controls the stiffness of the rubber sheet. Figure 8 shows the MRF estimate of the ground surface in gray with the tree ladar data colorized by height above the ground estimate. As shown in Figure 8, swales often have weeds that prevent the ladar from measuring the actual ground surface. Using the ladar data directly in these cases can lead to undesired spraying of weeds or the ground. The MRF estimate uses neighborhood context and the full 3D ladar point cloud to smooth through the weeds and beneath the trees to provide an accurate ground surface, which allows the spray control algorithm to target only the trees without spraying the ground regardless of the ground profile of the row it is operating in, as shown by the active nozzle indications in Figure 8. This use of spatial contextual information is one advantage over systems that operate on
Tree height
3D voxel density grid
2D density slice
Fig. 9. Example showing (left) 3D ladar data in a voxel density representation colorized by height along with the probabilistic ground height estimates, and (right) a 2D vertical slice of the voxels, colorized by density
individual ladar scans independently. With an accurate ground model, we analyze a region on each side of the sprayer in the 3D ladar map to find the angles corresponding to the bottom and top of the tree. By adjusting the location and length of this region (in the direction of tractor motion), we can control exactly when the sprayer turns on and off at tree boundaries, which affects the spray deposition for trees next to gaps or at the end of the row. As shown in Figure 9, we accumulate the ladar data in small volumes of space called voxels (volumetric pixels). By discretizing the volume of space around the sprayer, we can perform ray-tracing for each ladar return and keep track of the hits and pass-throughs in each voxel, producing a density measure defined as the ratio of hits to the total hits and pass-throughs in that voxel. This approach was originally used to discriminate between penetrable vegetation and solid obstacles [13]. We use it to generate a tree density measure, as shown by the tree slices in Figure 9. Healthy, mature trees generally have a dense canopy that allows little ladar penetration, so the voxels on the canopy surface have many
Nozzle Number
10
6 4 2 0
10 9 8 7 6 5 4 3 2 1 0
Nozzle Number
8
20
40
60
80
Distance Along Row (m)
100
120
50
100
Nozzle Activation Percent
(a) Bed row where sprayer is at the same level as the trees (see Figure 2) 10
Nozzle Number
Fig. 10. Nodding ladar on autonomous tractor used in spraying experiments
6 4
hits and very few ladar pass-throughs. Conversely, young trees or other trees with less dense canopy allow ladar rays to pass through the outer canopy and produce returns on the interior of the tree or behind the tree. These rays increase the number of pass-throughs in the voxels of the outer tree canopy and result in lower density scores in our 3D voxel map. Additional details of this approach are available in [4]. Using the above information, we compute the tree height and density. These values are mapped to a set of spray application protocols to control the nozzle rates and air-diverting louver positions. These protocols have been found through spray pattern evaluations and tracer studies that measured sprayer application efficiency for different combinations of nozzle flow rate and air-assist settings [5]. For mature trees with tall dense canopy, the sprayer should generally be operated with all nozzles open and full air-assist so there is little opportunity for chemical savings. However, controlled experiments with the sprayer in a test orchard containing small trees showed that using fewer nozzles could provide similar deposition with less chemical usage, and partially closing the louvers to reduce the air-assist to 70% reduced the amount of spray that was propelled beyond the canopy and increased spray deposition efficiency [5]. To test real-world performance, we have integrated this sprayer with an autonomous tractor [14] that has a nodding ladar on its roof that scans the area in front of the tractor for safeguarding (see Figure 10). We pass this sensor data to the tree modeling algorithm described above, and it sends commands to the sprayer over CAN bus to control the nozzles in real time. Unlike the vertically mounted ladar shown in Figure 3, the horizontal nodding ladar shown in Figure 10 does not scan up beyond horizontal from the roof of the tractor, so it does not sense the tops of tall trees. Therefore, for this experiment, we operate the top four nozzles as a single bank that is triggered for trees that are as tall as the roof of the tractor, instead of maintaining individual nozzle control across the entire sprayer as we normally do. This system autonomously sprayed approximately 80 acres (32 hectares) of a citrus orchard with varied tree size, including the area shown in Figure 1. Figure 11 shows the active nozzles for the left side of the sprayer over example segments of both row types during autonomous sprayer testing. These plots show the importance of controlling individual nozzles in areas with small or varied trees, where many nozzles are off for a large percentage of time. Comparing the row segments shown with a simple sprayer that has all nozzles on all the time (which would
2 0
10 9 8 7 6 5 4 3 2 1 0
Nozzle Number
8
20
40
60
80
Distance Along Row (m)
100
120
50
100
Nozzle Activation Percent
(b) Swale row where the sprayer operates below the trees (see Figure 8) Fig. 11. Nozzle activation patterns and histogram for the left side of the sprayer while spraying portions of the orchard shown in Figure 1
show up as solid yellow with no red in the nozzle activation patterns with a histogram showing all nozzles at 100%), the proposed approach had 40% the amount of nozzles active on the bed row in Figure 11a and 26% the amount of nozzles active on the swale row in Figure 11b. The proportion of upper nozzles active in both example rows depends directly on the proportion of tall mature trees in that row, so areas with more mature trees and fewer gaps in the canopy would have higher nozzle activation percentages. Comparing the lower nozzle activations between the two row types in Figure 11 shows the sprayer adapting to different conditions using the ground height estimation and turning off the lower nozzles based on the location of the base of the trees in that row. In a bed row (e.g. Figure 2), the tractor driving lane is at the same height as the base of the trees, and nozzle 1 is generally not used because it would be spraying the ground. The sprayer sits lower in swale rows (e.g. Figure 8), and the bottom two nozzles are generally off. Although the sprayer sits lower in the swales than the bed rows, the trees are also farther to the side so the upper spray angle is often similar between beds and swales. The combination of ground height estimation and individual nozzle control allows the sprayer to automatically adapt to different row types and varying sprayer roll angles to minimize the amount of chemical that is wasted by being sprayed into the ground, without time consuming manual adjustments of sprayer nozzle angles. III. T REE I NVENTORY As described in the introduction, it is important for growers to have an accurate tree inventory of the trees in their orchard. This is especially true when trees are removed due to disease or other problems. Figure 1 shows a block with many trees removed and a variety of different size trees due to re-planting at various times. As with the smart sprayer, we use a vertically mounted 270 degree ladar synchronized with a GPS receiver to scan the trees and build up an accurate 3D point cloud. Figure 12 gives an example of the 3D ladar data and detected trees, showing that trees have been removed since the overhead imagery was taken.
Fig. 12. Tree counting interface showing 3D ladar data and detected trees compared to overhead imagery, showing that overhead imagery is out of date (trees on the west side of the row to vehicle’s right have been removed)
Fig. 14.
Mature tree canopy that appears as a wall of trees to the sensors
Observing the sensor data in vertical perpendicular slices of a row of trees, there are essentially three different cases to consider: the slice contains a tree, the slice does not contain a tree (a gap), or the slice is at the boundary of a tree. We can define each of these cases as a state and model the transitions between them as shown in Figure 15, showing the constraint that the T ree state needs to move through the Boundary state to get to a Gap state. Fig. 13. Data collection truck with ladar and GPS mounted on a pole connected to the hitch and ground truth touch screen interface mounted inside the cab
We have experimented with using the ladar data collected by the tractor during spray control and autonomous operation for tree inventory, but the grower wanted a stand-alone system to be able to collect large amounts of data over the entire grove independently of a tractor, so we mounted the ladar and GPS on a pole attached to a pickup truck as shown in Figure 13. We also provided a touchscreen interface that allowed the driver to provide ground truth GPS tree locations and size classifications by clicking buttons as they passed the center of each tree. This ground truth data is used to evaluate the performance of the automatic tree inventory approach. For trees that are separated like in Figure 12, we can look at the height profile directly to find gaps between the trees and produce an accurate count. However, orchards are generally planted so that when the trees are mature, they fill the space and create a wall of tree canopy, as shown in Figure 14. Manual hedging activities can accentuate this further. Figure 14 also shows that the tree trunks are generally hidden behind the canopy so they are not easily used for counting trees. To handle both separated trees and continuous walls of trees, we use a probabilistic formulation to combine features extracted from the 3D ladar data with a prior distribution on expected tree spacing. We use the GPS location of the ladar data to look up in a GIS spatial map of the orchard what the expected tree spacing is in that location. We also use this map to automatically split the dataset into individual rows, and we run our algorithm on all the data from one row together. This allows the tree modeling algorithm to optimize from both ends of a wall of trees to find the best tree count that matches both the ladar data and the expected tree spacing.
Tree Fig. 15.
Boundary
Gap
State transitions for an HMM model of the trees
We divide the length of the row of trees into discrete locations 20cm long. The goal of the algorithm is then to assign one of the above three states to each discrete location. At this point we could make the Markov assumption that the state of each location is independent of the others in the row given its neighbors and we would have a Hidden Markov Model (HMM) [15]. However, the width of a tree in this model is not well modeled by repeated self-transitions within a T ree state because this would induce an exponential distribution that makes trees less likely the wider they are, as shown in Figure 16. Instead, we would like to incorporate the expected tree spacing in a probabilistically valid way. To account for an explicit prior on tree spacing, we use a Hidden Semi-Markov Model (HSMM) [15], as shown in Figure 17. Unlike an HMM that transitions (possibly to the same state) at every step, an HSMM remains in a single state value for some random duration generating observations from that state at each step. As shown in Figure 16, we can explicitly model the distribution over the duration of any particular state using a Gaussian as shown or any other distribution, including non-parametric distributions. After that duration, the state transitions to a new value according to the transition matrix and the process repeats. The transitions between states remain Markov, but the individual steps are not Markov since the probability of transitioning depends on how long the system has been in that state. In our application, the duration represents the number of 20cm discrete locations that make up a tree, tree gap, or the boundary between trees or tree gaps.
1 0.8 0.6
0.4 0
2
4
6
8
10
12
14
16
18
20
0.6
0.4
0.2
0.05
0.4
0.2
0
1
2
Height (m)
State Duration
0
3
0.2
0
1
1.0
Boundary 0.5
Duration
0.5
(b) Gap
0.5
1
Height Diff (m)
1.5
(c) Boundary
0.05
Gap Duration
0
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
2
2.5
3
3.5
4
4.5
5
Tree Width (m)
0.4
Fig. 17. State transitions for an HSMM model of the trees, showing the explicit distribution over the duration of each state before it transitions
0.2 0 0
0.5
1
1.5
Boundary Width (m)
p(data | state)
Fig. 19. HSMM duration prior distributions for T ree and Boundary states (Gap state prior distribution matches T ree prior distribution) p(data | Tree) p(data | Boundary) Detected tree or gap Labeled tree
0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0
110
120
130
140
150
160
Distance Along Row (m)
170
180
(a) Individual trees visible p(data | Tree) p(data | Boundary) Detected tree or gap Labeled tree
p(data | state)
0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0
180
190
200
210
220
Distance Along Row (m)
230
240
250
(b) Continuous wall of trees
p(data | state)
To fully define an HSMM, we need the state transition model shown in Figure 17, the observation models that define distributions over the ladar features for each state shown in Figure 18, and the prior distributions on the duration for each state shown in Figure 19. Figure 18 shows the probability of different observations based on the ladar data for each state. The T ree state is likely to have ladar data above the ground, the Gap state is likely to have ladar data near the ground (although there are sometimes weeds in the gaps so this distribution is more uniform than the T ree case), and the Boundary state is likely to have ladar data with a large difference in height. These distributions were set manually by analyzing the ladar features from several rows of orchard data. Figure 19 shows the prior distributions for different state durations. The T ree distribution is a broad Gaussian centered on the expected tree spacing from the GIS for this example of 11 feet (3.3m) with a hard limit of 15 feet (4.6m). The Gap distribution matches the T ree state, and the Boundary state duration is assumed to be small. These priors were set through a visual analysis of the ground truth data and ladar data for several typical rows, and they were kept fairly broad to account for variable tree spacing within a row. Figure 20 shows examples of the HSMM process at work for cases when the trees are separated, the tree canopy merges together, and when there are tree gaps that contain replanted trees. The plots show the probability of the ladar observations computed from the functions in Figure 18 for the T ree and Boundary states (the Gap state is not shown to reduce clutter since it is close to the inverse of the T ree state). These observation probabilities are combined with the prior distributions shown in Figure 19 by the HSMM through a set of backwards-forwards dynamic programming equations that find the optimal state sequence given the assumptions in the model [15]. Because we have required the T ree and Gap states to transition through the Boundary state, we can easily extract estimated tree centers and gap centers from the midpoints between the boundary transitions, as well as the estimated tree width from the duration of each
0 0
0.1
p(Tree Width)
Duration
1.0
3
HSMM data observation probability functions for each state
p(Boundary Width)
Tree
Fig. 18.
2
Height (m)
(a) T ree Fig. 16. The prior distribution over the duration of a single state in an HMM is exponentially decreasing, whereas the prior distribution for an HSMM can be modeled explicitly, as shown by the Gaussian in this example
1
0.8
p(data | Gap)
0.6
0.1
0
1 0.8
p(data | Boundary)
Exponential state duration distribution of HMM Explicit state duration of HSMM (ie Gaussian) 0.15
p(data | Tree)
Probability
0.2
0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0
p(data | Tree) p(data | Boundary) Detected tree or gap Labeled tree
20
30
40
50
60
Distance Along Row (m)
70
80
90
(c) Gaps with replanted trees Fig. 20. Input sensor data probabilities and HSMM detected trees and tree gaps compared with ground truth labeled trees
state. These detected tree and gap locations are shown along with the ground truth tree locations marked by the operator using the interface in Figure 13. Given that the ground truth measurements are based on button presses by an operator as they pass by trees while driving down the row, there is some uncertainty in their exact locations, but they serve as a useful
(a) Isolated Fig. 22.
(a) Individual trees visible
(b) In weeds Examples of newly replanted trees
(b) Continuous wall of trees
Fig. 21. Ground truth tree labels (+’s) compared to system tree detections (yellow circles with +’s) shown on overhead ladar height map. Ground truth tree categories: orange (large), cyan (medium), blue (small), magenta (replant), green (stump) Fig. 23. Interface to capture replanted trees manually during data collection
comparison for the automated algorithm. As shown in Figure 20a, when the shape of the individual trees is visible, the probability of the ladar sensor data for the T ree state clearly shows the tree boundaries, and the sensor data at the boundary often matches the Boundary state as well. With clear observations like this, the expected tree spacing from the prior distribution has little effect and the trees are marked reliably. However, note that even in this case neighboring tree canopies often touch which can be seen in Figure 20a at tree boundaries where the probability of the data for the T ree class goes down only slightly. Figure 20b shows part of a row that looks like the wall of trees in Figure 14. The ladar sensor data matches the T ree class over most of the row with occasional gaps where a tree was removed. In these cases, the sensor data provides very little information, so the prior over tree width shown in Figure 19 becomes the dominant component in the HSMM probabilistic optimization. The advantage of using a probabilistic model is that we can clearly state our assumptions about how the orchard is set up and then let the model perform the optimization using the sensor data collected for each row. We do not need to pick a single threshold on height or some other feature that will only work some of the time to detect trees. Similarly, this approach incorporates the expected tree spacing explicitly to fit trees to continuous masses of tree canopy without needing to write any special rules to handle this case. Figure 20c shows an example with tree gaps where trees have been removed because of disease. The ground truth tree labels show that two very small trees have been replanted in the space of each tree that was removed, and these small trees are not detected, as discussed in more detail below. The HSMM output tree locations and tree sizes and the corresponding manual ground truth labels are drawn on
an overhead map of the ladar height data in Figure 21. As described above, the tree model is able to extract tree locations that match the ground truth labels well for a variety of tree sizes when individual trees are visible in the ladar data and when the tree canopy blends together. However, as in Figure 20c, Figure 21 shows that newly replanted trees (labeled as magenta +’s) are not detected by the algorithm and often barely register in the ladar height map. Figure 22 shows a couple examples of newly replanted trees. These trees are often approximately the same height as weeds that occur among the trees. Given the challenge of reliably detecting these replanted trees (without counting similarly sized weeds), we use a hybrid approach where the person driving the data collection truck (Figure 13) through the orchard uses the interface shown in Figure 23 to manually mark the locations of replanted trees. We have found that the driver can generally perform this task without slowing down much, unless they are going through an area that is full of replanted trees, so this is a viable option for a human driven vehicle but would not work for an autonomous system or if the data is being collected passively from a vehicle performing other tasks. An example of the final system output that is inserted into the orchard management system as a GIS shape file is shown by the circles in Figure 24 that are plotted on the ladar height map. Having GPS locations for each tree integrated into the rest of the orchard management tools is very valuable for the grower and can give them much higher resolution than hand counts per row or block. To quantify these results we compared the counts per block produced by this system to the corresponding hand counts over a pump zone, which is a 1/2 mile square section of the orchard (160 acres or approximately 65 hectares). We did not
approaches and others to provide additional value to the farmer. As described above, we utilized the ladar sensor used on our autonomous tractor for safeguarding and navigation to additionally control the sprayer. GPS is already available on many large agricultural vehicles, and we believe many additional applications become possible once it is more common for agricultural vehicles to have additional sensors onboard, and showing added value to the farmer is a powerful way to help this technology become adopted. ACKNOWLEDGMENT We would like to thank Southern Garden Citrus for supporting our testing in their orchard, and we would like to thank Tim Gast and Gabriel Gast for their tree inventory data collection and ground truth labeling. Fig. 24.
Example tree inventory results overlaid on the ladar height map
include the small replanted trees in this analysis because they are handled separately as described above. Table I shows that the overall tree counts of the automated system are within 1% of the hand counts, although some of the blocks with few trees have larger errors. The over-counting in blocks 8-10 is mostly from larger replants that are intermittently detected by the system. We are currently working on better integrating the manual labels for replanted trees with the automated system to eliminate these over-counting errors. The under-counting is generally from small trees that are pressed into the side of a larger tree. TABLE I T REE C OUNTING R ESULTS OVER A 1/2 M ILE S QUARE P UMP Z ONE (N OT I NCLUDING N EW R EPLANTS ) Block 1 2 3 4 5 6 7 8 9 10 11 12 Total
Hand Counts 1569 1474 1206 1238 1542 1551 406 172 150 151 698 450 10607
Auto Counts 1552 1466 1197 1260 1529 1526 408 183 168 174 706 462 10631
% Diff -1% -1% -1% 2% -1% -2% 0% 6% 12% 15% 1% 3% 1%
IV. C ONCLUSION We have described two related applications that use probabilistic approaches to interpret ladar sensor data and generate tree models in an orchard environment. We leverage contextual information through a Markov random field ground model and a hidden semi-Markov model that incorporates expected tree spacing. These applications show the benefit of having a range sensor with GPS mounted on an agricultural vehicle, enabling more precise spray application and an automated tree inventory. In addition to further improving the performance of the two algorithms, we are looking at ways to combine these
R EFERENCES [1] P. Walklate, J. Cross, G. Richards, R. Murray, and D. Baker, “Comparison of different spray volume deposition models using lidar measurements of apple orchards,” Biosystems Engineering, pp. 253– 267, 2002. [2] S. Planas, K. R. Rosell, E. Gil, L. Monterola, and A. Escola, “Optimizing pesticide spray application in tree crops,” in ASABE Paper No. 061128, 2006. [3] J. Wei and M. Salyani, “Development of a laser scanner for measuring tree canopy characteristics, phase 2: Foliage density measurement,” Transactions of the ASABE, vol. 48, 2005. [4] J. Campoy, J. Gonzalez-Mora, and C. Dima, “Advanced sensing for tree canopy modeling and precision spraying,” in ASABE Paper No. 1009470, 2010. [5] L. R. Khot, R. Ehsani, G. Albrigo, P. A. Larbi, A. Landers, J. Campoy, and C. Wellington, “Air-assisted sprayer adapted for precision horticulture: Spray patterns and deposition assessments in small-sized citrus canopies,” Biosystems Engineering, 2012. [Online]. Available: http://dx.doi.org/10.1016/j.biosystemseng.2012.06.008 [6] J. Wei and M. Salyani, “Development of a laser scanner for measuring tree canopy characteristics, phase 1: Prototype development,” Transactions of the ASABE, vol. 47, 2004. [7] K. Lee and R. Ehsani, “A laser scanner based measurement system for quantification of citrus tree geometric characteristics,” Journal of Applied Engineering in Agriculture, vol. 25, no. 5, pp. 777–788, 2009. [8] J. R. R. Polo, R. Sanz, J. Llorens, J. Arn, A. Escol, M. RibesDasi, J. Masip, F. Camp, F. Grcia, F. Solanelles, T. Pallej, L. Val, S. Planas, E. Gil, and J. Palacn, “A tractor-mounted scanning lidar for the non-destructive measurement of vegetative volume and surface area of tree-row plantations: A comparison with conventional destructive measurements,” Biosystems Engineering, vol. 102, no. 2, pp. 128 – 134, 2009. [9] M. S. Tumbo S., J. Whitney, T. Wheaton, and W. Miller, “Investigation of laser and ultrasonic ranging sensors for measurements of citrus canopy volume,” Applied Engineering In Agricultire, pp. 367–372, 2002. [10] M. Swanson, C. Dima, and A. Stentz, “A multi-modal system for yield prediction in citrus trees,” in ASABE Paper No. 1009474, 2010. [11] Q. U. Zaman, A. W. Schumann, and H. K. Hostler, “Estimation of citrus fruit yield using ultrasonically-sensed tree size,” Applied Engineering in Agriculture, vol. 22, no. 1, pp. 39–43, 2006. [12] C. Wellington, A. Courville, and A. Stentz, “A generative model of terrain for autonomous navigation in vegetation,” International Journal of Robotics Research, vol. 25, no. 12, pp. 1287–1304, 2006. [13] A. Lacaze, K. Murphy, and M. DelGiorno, “Autonomous mobility for the Demo III experimental unmanned vehicles,” in Assoc. for Unmanned Vehicle Systems Int. Conf. on Unmanned Vehicles (AUVSI), July 2002. [14] S. J. Moorehead, C. K. Wellington, B. J. Gilmore, and C. Vallespi, “Automating orchards: A system of autonomous tractors for orchard maintenance,” in IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS) Workshop on Agricultural Robotics, 2012. [15] L. R. Rabiner, “A tutorial on hidden markov models and selected applications in speech recognition,” Proceedings of the IEEE, vol. 77, no. 2, pp. 257–286, 1989.
