Precision Agric (2008) 9:391–405 DOI 10.1007/s11119-008-9083-z
Evaluation of an algorithm for automatic detection of broad-leaved weeds in spring cereals T. W. Berge Æ A. H. Aastveit Æ H. Fykse
Published online: 27 September 2008 Ó Springer Science+Business Media, LLC 2008
Abstract Lack of automatic weed detection tools has hampered the adoption of sitespecific weed control in cereals. An initial object-oriented algorithm for the automatic detection of broad-leaved weeds in cereals developed by SINTEF ICT (Oslo, Norway) was evaluated. The algorithm (‘‘WeedFinder’’) estimates total density and cover of broad-leaved weed seedlings in cereal fields from near-ground red–green–blue images. The ability of ‘‘WeedFinder’’ to predict ‘spray’/‘no spray’ decisions according to a previously suggested spray decision model for spring cereals was tested with images from two wheat fields sown with the normal row spacing of the region, 0.125 m. Applying the decision model as a simple look-up table, ‘‘WeedFinder’’ gave correct spray decisions in 65–85% of the test images. With discriminant analysis, corresponding mean rates were 84–90%. Future versions of ‘‘WeedFinder’’ must be more accurate and accommodate weed species recognition. Keywords
Image analysis Machine vision Patch spraying Site-specific weed control
Introduction Weeds compete with crop plants for sunlight, moisture and nutrients and can have a detrimental impact on crop yields and quality if uncontrolled. But the conventional practice of applying herbicides uniformly across fields seems undesirable because weeds tend to be distributed in patches (e.g., Dieleman and Mortensen 1999; Heijting et al. 2007). T. W. Berge (&) Plant Health and Plant Protection Division, BIOFORSK – Norwegian Institute ˚ s, Norway for Agricultural and Environmental Research, Høgskoleveien 7, 1432 A e-mail:
[email protected] T. W. Berge H. Fykse Department of Plant and Environmental Sciences, Norwegian University of Life Sciences, ˚ s, Norway P.O. Box 5003, 1432 A A. H. Aastveit Department of Chemistry, Biotechnology and Food Science, Norwegian University of Life Sciences, ˚ s, Norway P.O. Box 5003, 1432 A
123
392
Precision Agric (2008) 9:391–405
Substantial reduction in herbicide use has been possible when only the patches are treated (e.g., Nordmeyer 2006) while acceptable levels of weed control seem to be maintained (e.g. Dicke et al. 2007). A pre-requisite for patch spraying or site-specific weed control (SSWC) is information on the field-specific spatial weed distribution. Since manual field observations are very time-consuming and air- and space-borne platforms are not capable of detecting infestations at the resolution needed, and their viewing windows are very restricted (Lamb and Brown 2001), the use of ground-based platforms seems most appropriate (Brown and Noble 2005). The information on weed distribution obtained through ground-based platforms is needed for both map-based (weed detection and weed kill in two operations) and real-time (weed detection and weed kill in one operation) implementations of SSWC. Two types of ground-based detection sensors have been investigated. The first makes use of non-imaging opto-electronic sensors to measure the reflected light in two channels and subsequently calculates an index of vegetative presence over an area. Today such sensors are commercially available for real-time spraying on fallows, in urban areas and along rail-tracks (Biller 1998). They can be adapted for wide-row crops, e.g., cotton (Sui et al. 2008), vineyards and orchards (www.ntechindustries.com, accessed October 2007). Possibility to use such technology in cereals has just recently also been investigated (Dammer and Wartenberg 2007). In these situations all vegetation is assumed to be weeds, and the task is reduced to a vegetation-soil differentiation based on the significant difference in reflectance characteristics between vegetation and soil (Zwiggelaar 1998). The second ground-based automatic weed detection technology receiving much attention is object-oriented image analysis, which has the potential to separate plant species based on geometric, textural or spectral differences, or combinations of these characteristics. Several algorithms without weed species differentiation (e.g. Andreasen et al. 1997; Pe´rez et al. 2000) and with weed species differentiation (e.g., Gerhards et al. 1993; Søgaard 2005; Gerhards and Oebel 2006) have been developed, but a commercial solution is still awaited. The aim of this study was to evaluate ‘‘WeedFinder’’, a prototype of an object-oriented image classification algorithm for automatic detection of broad-leaved weed seedlings in cereals, developed by SINTEF ICT (Oslo, Norway). ‘‘WeedFinder’’ is a relatively simple algorithm with no weed species differentiation, and must be viewed as an initial step in the development of a more complex operational algorithm. Cereals grown with narrow row spacing (less than 0.15 m) in Northern Europe are relatively competitive against weeds. Therefore, weed control can be based on weed damage thresholds or spray decision models. The tested images were collected from winter wheat fields in autumn, but were considered representative of the appearance of crop and weed at the time of post-emergence spraying of spring cereals. ‘‘WeedFinder’’ was evaluated for its ability to predict ‘spray’ and ‘no spray’ decisions according to a modified version of a spray decision model developed for Norwegian spring cereals (Fykse 1991).
Materials and methods Field characteristics and image acquisition ˚ s, SE Norway (59°380 N, 10°420 E) were Two farmer fields (Myhrer and Holstad) in A surveyed. The climate of this district is temperate with a mean temperature of -4.8°C in January and 16.1°C in July, and annual precipitation of 785 mm (average for 1961–1990).
123
Precision Agric (2008) 9:391–405
393
Table 1 Field, crop and image sampling characteristics in 2004
Field Myhrer
Holstad
Approximate area surveyed (ha)
0.24
0.77
Date of sowing
29 August
4 September
Sowing density (kg ha-1)
180
220
Winter wheat cultivar
Mjølner
Magnifik
Crop stage, BBCH
14–21
13–14
Date of image acquisition
30.09–2.10
11.10–13.10
Soil textures were medium and stiff clays (Myhrer and part of Holstad) and silty sand (part of Holstad). At Holstad the test area was gently undulating and at Myhrer it was flat. The fields were sown with two cultivars of winter wheat, Triticum aestivum L., at the normal row spacing of 0.125 m (Table 1). Images were acquired vertically with a commercial digital camera (Nikon Coolpix 5400) mounted on a tripod about 0.45–0.50 m above the soil surface. To eliminate shadows and ensure an even illumination the photographed area was shielded by a parasol if the sun was shining. The internal flash of the camera was always used. Image quality was set to 2,592 by 1,944 pixels, jpg compression ‘fine’ (i.e., roughly one-fourth of original), aperture f: 7.9, automatic shutter speed (1/30 s in majority of images) and macro mode for well focused images. Each image covered approximately 0.27 m2 (0.6 m 9 0.45 m) of the ground, and about five crop rows. The ground resolution was about 4 pixels mm-1. Images were acquired (50 per transect) in continuous transects (eight at Myhrer, 14 at Holstad) about 30 m long across the sowing direction (Fig. 1). Transects were separated by 5, 50, 65, or 90 m (Fig. 1). At Holstad, the images were taken at the normal time for postemergence herbicide application, crop stage BBCH 13–14 (Anonymous 2001). At Myhrer, the crop growth stage was BBCH 14–21 (Table 1). The dominating weed species at Holstad were Lamium purpureum L. and Stellaria media (L.) Vill. Galium aparine L. was also relatively common. Other species present are listed in Table 2. A relatively common species at Myhrer, absent at Holstad, was volunteer spring turnip rape (Brassica rapa ssp. oleifera (DC.) Metzg.). Young individuals of Poa annua L. were seen in some places in both fields, while Elytrigia repens (L.) Desv. Ex Nevski was only seen a few times at Myhrer. The weed flora was more diverse at Myhrer with no species clearly dominating (Table 2). About ‘‘WeedFinder’’ Because the resolution of images, the red/green balance and the size and shapes of the weed leaves may vary between series of images, ‘‘WeedFinder’’ was designed with six user-adjustable parameters (Table 3). The differentiation of vegetation (crop plus weeds) from soil background is based on a ratio between the green and the red channel (Fig. 2b) from the original colour image (Fig. 2a). The histogram of the ratio values shows a weakly bimodal distribution (Fig. 2c), where the left peak represents pixels of background (soil, etc.) and the right peak represents pixels with green vegetation. The appropriate threshold value between background and vegetation is calculated dynamically through Otsu’s method (Otsu 1979) for each of the images from the lower and upper user-adjustable input parameter values (parameter Nos. 1 and 2, respectively, Table 3). The contours of the plant
123
394
Precision Agric (2008) 9:391–405
(a)
30 m
50 m 15 m
(b) 30 m 65 m
90 m Sowing direction
15 m 50 m
30 m Fig. 1 Diagrams to show the image sampling design: (a) Myhrer and (b) Holstad. Small squares indicate the images
Table 2 Weed species at the two fields with growth stages (BBCH scale) indicated
123
Field Myhrer
Holstad
Brassica rapa ssp. oleifera (DC.) Metzg. ´ . Lo¨ve Fallopia convolvulus (L.) A
12–15
–
12
11
Fumaria officinalis L.
10–15
12
Galium aparine L.
22/34
10–11/22
Lamium purpureum L.
14
10–14
Lapsana communis L.
12–14
13
Myosotis arvensis (L.) Hill
13
–
Polygonum aviculare L.
14
–
Spergula arvensis L.
–
15
Stellaria media (L.) Vill.
24/34
12–24/33
Tripleurospermum inodorum (L.) Sch. Bip.
13–14
–
Viola arvensis Murray
12–14
10–13
Elytrigia repens (L.) Desv. ex Nevski
12
–
Poa annua L.
10–12
10–11
Precision Agric (2008) 9:391–405
395
Table 3 Overview of the user-adjustable parameters in ‘‘WeedFinder’’, the possible values (minimum, default and maximum values) and the two values used in the initial sensitivity analysis Parameter
Possible values
Sensitivity analysis
Min
Default
Max
Low
High 0.7
1
Lower threshold for vegetation/soil separation
0
0.6
1
0.5
2
Upper threshold for vegetation/soil separation
1
1.2
2
1.1
1.9
3
Minimum weed size, mm2
0
9
?
5
100
4
Maximum weed size, mm2
Minimum size
2,500
?
300
3,000
5
Lower threshold for roundnessa of weed leaves
0
0.35
1
0.1
0.7
6
Distance between weed objects to be merged, mm
0
25
100
5
35
a
Roundness: 1 = circle, 0 = line
Fig. 2 Stages in differentiation of vegetation from background by ‘‘WeedFinder’’: (a) input red-green-blue image, (b) green/red ratio image shown as a grey scale image, (c) histogram of the ratio values (x-axis label = value of the green/red ratio (range: zero to infinity), y-axis label = number of pixels) and (d) segmented binary image
pixels in the binary image (Fig. 2d) are then extracted and size (parameter Nos. 3 and 4, Table 3) is the first feature to separate weeds from cereals. For plant objects larger than the defined maximum area of a weed object, roundness (parameter No. 5) of the objects is used to find weeds that are partially occluded by cereal leaves. Furthermore, if several weed
123
396
Precision Agric (2008) 9:391–405
objects are within a pre-defined proximity (parameter No. 6), they are defined as a single individual. Outputs from ‘‘WeedFinder’’ are estimates of the following variables: ‘‘Vegetation cover (%)’’, ‘‘Broad-leaved weed cover (%)’’, ‘‘Cereal cover (%)’’, ‘‘Free broadleaved weeds (individuals m-2)’’, ‘‘Partially occluded broad-leaved weeds (individuals m-2)’’ and ‘‘Total number of broad-leaved weeds (individuals m-2)’’. ‘Cover’ is the proportion of pixels in the image containing the actual category. Optimizing ‘‘WeedFinder’’ To find the optimal values of the user-adjustable parameters (Table 3), sensitivity analysis based on twenty images (ten per field) was conducted. Software, ‘‘Ground Truth’’, also developed by SINTEF ICT, was used to make a ground truth dataset of the actual weed density (individuals m-2) and weed cover (%) in the images. All visible individual broadleaved weeds were outlined manually in digital displays of the images (by mouse) using ‘‘Ground Truth’’. First, the six user-adjustable parameters in ‘‘WeedFinder’’ were varied systematically by low and high values (Table 3). The resulting absolute differences in weed density and weed cover between ‘‘WeedFinder’’ and ‘‘Ground Truth’’ were analysed as a 26 factorial design (six factors, each with two levels corresponding to high and low levels of the parameters) by analysis of variance (ANOVA). After the important parameters were identified, values were tuned to their optimized values. Evaluation of ‘‘WeedFinder’’ To evaluate the ability of ‘‘WeedFinder’’ to predict total broad-leaved weed density, 284 images divided between 110 (40 at Myhrer, 70 at Holstad) sampled systematically from all transects (five images per transect 9 (eight transects at Myhrer ? 14 at Holstad)) and 174 images from two complete transects per field not already used in the systematic sample (87 images at each field) were included. To evaluate the automatic predictions of total broadleaved weed cover, only the 110 images (40 at Myhrer, 70 at Holstad) sampled systematically were used (due to time restrictions). Spray decision model To evaluate the use of ‘‘WeedFinder’’ to predict spray decisions, test images were grouped into ‘spray’ and ‘no spray’ decisions according to an adapted version of a spray decision model previously suggested for broad-leaved weed seedlings in spring cereals (Fykse 1991). In the original Fykse model, the decision was ‘spray’ if the density of five specific species, either alone or in combination, exceeded defined threshold values, or if the total broad-leaved density was 175 individuals m-2. Since ‘‘WeedFinder’’ was not designed to recognize these five species, only the total density criterion could be assessed, and it is termed criterion 1 in the following. In the original Fykse model, the decision was also ‘spray’ if cereal cover was below 40%, or if the total broad-leaved weed cover was above 8%. In the present study it was not possible to apply these cover criteria directly because the total broad-leaved weed cover never reached this relatively high value (8%), presumably due to the relatively small field of view and/or the field-specific flora—and growth conditions. Instead, the relationship: ‘‘Total broad-leaved weed cover (%)’’ divided by ‘‘Cereal cover (%)’’, i.e., criterion 2, was used. This relationship, as assessed by image analysis, has been found suitable for the quantification of weed/winter cereal competition
123
Precision Agric (2008) 9:391–405
397
previously (Gerhards and Ku¨hbauch 1993). In the present paper, a range of values of 0.15–0.30 for this criterion was tested. Since the best correspondence between ‘‘Ground Truth’’ and ‘‘WeedFinder’’ was achieved when criterion 2 was 0.25, this value was chosen. This value is also very close to 0.2, which would be the corresponding value derived from the Fykse model (8%: 40% = 0.2). When both criteria were applied, the decision was ‘spray’ if either criterion 1 or criterion 2 was fulfilled. However, when applied to predict true spray decisions, the two criteria were strongly correlated, and criterion 2 produced the opposite spray decision to criterion 1 in less than 2% of the images in the evaluation set (n = 110). Therefore, it was decided to test criterion 1 as the only criterion in the spray decision model, in addition to testing both criteria. The original level of criterion 1 in the Fykse model, i.e., total broad-leaved weed density 175 m-2, represented the approximate weed density at which herbicide treatment gave crop yield increase compared to an untreated control (Fykse 1991). Since this value represents a mean of many field trials (cereal species, years, etc.), and weed damage thresholds never can be fixed, but are affected by, for example, weed species, cereal species (Christensen 1994), grain and herbicide prices (Young et al. 2003), the original threshold level of criterion 1 (175 individuals m-2) was also modified by 25%, to 220 individuals m-2, i.e., the increased threshold level, and to 130 individuals m-2, i.e. the reduced threshold level. By modifying by ±25% we were able to evaluate the ability of ‘‘WeedFinder’’ to predict correct spray decisions despite use of an altered threshold level. Criterion 2 was not modified in the same manner. ‘‘Cereal cover’’, which was needed for computation of criterion 2, was not determined in ‘‘Ground Truth’’ due to time restrictions, but was estimated as: ‘‘Vegetation cover (%)’’ given by ‘‘WeedFinder’’ (with the optimized parameter values) minus true ‘‘Total broadleaved weed cover (%)’’ (determined in ‘‘Ground Truth’’). This was considered appropriate since the estimates of ‘‘Vegetation cover (%)’’ obtained by ‘‘WeedFinder’’ were very accurate as assessed by visual comparison between the original colour image and the binary image (cf. Fig. 2). Estimation of vegetation cover against soil background based on the green and the red channels has previously been successful (e.g., Blasco et al. 1999). Spray decisions based on classification method 1 In this method, the decision model described above was used as a simple look-up table to assign spray decisions according to the ‘‘WeedFinder’’ outputs of ‘‘Total broad-leaved weed density (individuals m-2)’’, ‘‘Total broad-leaved weed cover (%)’’ and ‘‘Cereal cover (%)’’. Spray decisions were determined by both criteria (n = 110) and by criterion 1 only (n = 284). The spray decisions determined by ‘‘WeedFinder’’ were compared with spray decisions determined by ‘‘Ground Truth’’ image by image, and success rates (i.e. spray decision predicted by ‘‘WeedFinder’’ equals true spray decision), spraying error (i.e., true decision ‘no spray’ determined as ‘spray’ by ‘‘WeedFinder’’) and mapping error (i.e., true decision ‘spray’ determined as ‘no spray’) were calculated. Spray decisions based on classification method 2 In the second classification method, the discriminant analysis procedure DISCRIM in SAS software (SAS Institute 2007) was used. The complete ground truth dataset (n = 304 = 110 from systematic sample ? 20 from sensitivity analysis ? 174 from complete transects) with true ‘spray’/‘no spray’ decisions based on criterion 1 only (due to time constraints) of the spray decision model, was split into a calibration set and an
123
398
Precision Agric (2008) 9:391–405
evaluation set. The calibration set (n = 130 = 110 from systematic sample ? 20 from sensitivity analysis) containing the quantitative ‘‘WeedFinder’’ outputs: ‘‘Total broadleaved weed cover (%)’’, ‘‘Cereal cover (%)’’ and ‘‘Total broad-leaved weed density (individuals m-2)’’ and the true ‘spray’/‘no spray’ categories, was used to calibrate the classification criterion according to linear discrimination (Johnson and Wichern 2002). Separate discrimination rules were calibrated for each level of the spray decision model (reduced, original and increased level). These three functions classified the evaluation set (n = 174) into one of the two spray decision categories based on the three quantitative ‘‘WeedFinder’’ outputs, and success rate, mapping and spraying errors were calculated.
Results Optimized parameter values in ‘‘WeedFinder’’ The sensitivity analysis showed that the two user-adjustable parameters of ‘‘WeedFinder’’ reflecting size of the vegetation object (Nos. 3 and 4, Table 3) were of most influence.
Fig. 3 True values versus ‘‘WeedFinder’’ estimated values of: (a) Total broad-leaved weed density (n = 284 = 40 ? 87 (Myhrer) and 70 ? 87 (Holstad)) and (b) Total broad-leaved weed cover (n = 110 = 40 (Myhrer) ? 70 (Holstad)), with 1:1 lines (–––) included for reference
123
Precision Agric (2008) 9:391–405
399
Parameters Nos. 1 and 2 were also important, since very high or very low values of these caused the vegetation-background segmentation to fail. As these two parameters are actually calculated dynamically from the initial values, and the default values resulted in correct segmentation in 99% of cases, the default values were chosen as the optimized values. The sensitivity analysis also showed that roundness of the vegetation object and the distance parameter (Nos. 5 and 6, Table 3) were not important. Therefore, they were set to the intermediate value and the default value, respectively. The optimized values for parameters Nos. 1–6 were 0.6, 1.2, 30 mm2, 400 mm2, 0.5 and 25 mm, respectively. Evaluation of ‘‘WeedFinder’’ The estimates by ‘‘WeedFinder’’ and the corresponding true values of total broad-leaved weed density and weed cover were positively correlated, but there were serious dispersion and discrepancy from the 1:1 relationships (Fig. 3). True weed densities above about 300 broad-leaved weed individuals m-2 were under-estimated by ‘‘WeedFinder’’, and densities smaller than 175 weeds m-2 were generally over-estimated (Fig. 3a). ‘‘Total broad-leaved weed cover (%)’’ revealed a similar picture (Fig. 3b). The mean total broad-leaved weed density was much higher at Holstad (311 individuals m-2) than at Myhrer (81 individuals m-2) as estimated with the program ‘‘Ground Truth’’ (Fig. 3a). Classification method 1 (look-up table) gave relatively moderate average (across both fields) success rates for the three spray decision levels: 65–79% using both criteria (Table 4) and 67–85% using criterion 1 only (Table 5). At the reduced and the original levels in particular, the dominating proportion of average incorrect spray decisions constituted spraying error (Tables 4 and 5). At Holstad, success rate was relatively stable across the three levels and, for the reduced and the original levels, it was considerably higher than at Myhrer (Tables 4 and 5). However, while the mapping error was 2% or less at Myhrer, this error type was as high as 7% (original level) and 13% (increased level) at Holstad by method 1 (Tables 4 and 5). Classification method 2 (discriminant analysis) yielded higher and more stable average success rates than method 1, viz. 85% (reduced level), 84% (original level) and 90% Table 4 Percentages of correctly classified spray decisions (in bold) (whose sum is defined as the ‘success rate’) and misclassified spray decisions divided between ‘mapping errors’, i.e., true decision ‘spray’ determined as ‘no spray’ by ‘‘WeedFinder’’ (in italics) and ‘spraying error’, i.e., true decision ‘no spray’ determined as ‘spray’ by ‘‘WeedFinder’’ (underlined) by use of classification method 1 and both criteria of the spray decision model Field
Threshold model -25%
Average
‘‘WeedFinder’’: ‘‘Ground Truth’’
‘No spray’
‘No spray’ ‘Spray’
Myhrer
‘No spray’ ‘Spray’
Holstad
‘No spray’ ‘Spray’
Original ‘Spray’
‘No spray’
14
35
0
51
19
?25% ‘Spray’
‘No spray’
‘Spray’
35
19
53
12
5
40
9
26
67
52
38
74
17
0
14
2
7
2
7
10
16
25
7
40
9
0
74
7
60
13
38
Number of observations in the evaluation set: 40 (Myhrer) ? 70 (Holstad) = 110 images
123
400
Precision Agric (2008) 9:391–405
Table 5 Percentages of correctly classified spray decisions and misclassified spray decisions, using classification method 1 and criterion 1 (weed density) of the spray decision model (for details of classification categories, see Table 4) Field
Threshold model -25%
Average
‘‘WeedFinder’’: ‘‘Ground Truth’’
‘No spray’
‘No spray’ ‘Spray’
Myhrer Holstad
Original ‘Spray’
‘No spray’
21
32
1
46
?25% ‘Spray’
‘No spray’
‘Spray’
45
18
62
8
4
33
7
23
35
55
72
25
88
9
‘Spray’
1
9
0
3
0
2
‘No spray’
9
13
22
13
40
8
‘Spray’
1
77
7
58
13
40
‘No spray’
Number of observations in the evaluation set: 40 ? 87 (Myhrer) and 70 ? 87 (Holstad) = 284 images
Table 6 Percentage correctly classified spray decisions and misclassified spray decisions using classification method 2. Only criterion 1 (weed density) of the spray decision model was used to make true spray decisions (for details of classification categories, see Table 4) Field
Threshold model -25%
Average
‘‘Weed Finder’’: ‘‘Ground Truth’’
‘No spray’
‘No spray’
‘No spray’
52
11
59
7
66
4
3
33
9
25
7
24
88
5
92
7
95
5
3
3
1
0
0
0
‘No spray’
16
2
26
11
35
10
‘Spray’
18
64
13
50
9
45
‘No spray’ ‘Spray’
Holstad
?25%
‘Spray’
‘Spray’ Myhrer
Original ‘Spray’
‘No spray’
‘Spray’
Number of observations in the evaluation set: 87 (Myhrer) ? 87 (Holstad) = 174 images
(increased level) (Table 6). However, method 2 resulted in either equal (the increased level) or even higher (the two other levels) average mapping errors than method 1 when only criterion 1 was applied. Contrary to method 1, method 2 gave considerably higher success rates at Myhrer (91–95%) than at Holstad (76–80%) (Table 6). The mapping errors were clearly much higher at Holstad (9–18%) than at Myhrer (0–3%) by this method (Table 6). There was clear spatial dependency in the spray decisions along the transects for both the true spray decisions and those predicted by ‘‘WeedFinder’’ (Figs. 4 and 5). At these two Myhrer transects, mainly ‘no spray’ decisions occurred in ground truth (Fig. 4, middle column). Classification method 2 appeared better than method 1 at Myhrer (Fig. 4), with the proportions of incorrect classification being only 4–8% compared to 6–50% for method 1. At Holstad, where the weed infestation was high at these transects (Fig. 5), the two methods yielded similar proportions of incorrect classification: 13–26% (method 1) and 17–23% (method 2).
123
Precision Agric (2008) 9:391–405 WeedFinder, method 1
401 WeedFinder, method 2
Ground Truth
Reduced threshold Original threshold Increased threshold
(8%)
(50%)
(6%) (19%)
(4%)
(6%)
0
15 m
30 0
15 m
30 0
30
15 m
Fig. 4 Spatial distribution of spray decisions in two transects at Myhrer, for ‘‘Ground Truth’’ (middle column), classification method 1 of ‘‘WeedFinder’’ (left column) and classification method 2 of ‘‘WeedFinder’’ (right column), at three levels of the threshold model: Reduced (upper row), original (middle row) and increased threshold level (bottom row). Spray decisions in ‘‘Ground Truth’’ and method 1 of ‘‘WeedFinder’’ were based on criterion 1 (total broad-leaved weed density). Numbers in brackets are the respective proportions of images with incorrect spray decision compared to the ‘‘Ground Truth’’. Each square represents one image (0.27 m2). Legend: open squares = ‘no spray’, filled squares = ‘spray’. Distance between transects: 5 m WeedFinder, method 1
WeedFinder, method 2
(13%)
(23%)
(26%)
(23%)
(22%)
(17%)
15 m
30 0
15 m
30 0
Reduced threshold Original threshold Increased threshold
0
Ground Truth
15 m
30
Fig. 5 Spatial distribution of spray decisions in two transects at Holstad, for ‘‘Ground Truth’’ (middle column), classification method 1 of ‘‘WeedFinder’’ (left column) and classification method 2 of ‘‘WeedFinder’’ (right column), at three levels of the threshold model: Reduced (upper row), original (middle row) and increased threshold level (bottom row). Spray decisions in ‘‘Ground Truth’’ and method 1 of ‘‘WeedFinder’’ were based on criterion 1 (total broad-leaved weed density). Numbers in brackets are the respective proportions of images with incorrect spray decision compared to the ‘‘Ground Truth’’. Each square represents one image (0.27 m2). Legend: open squares = ‘no spray’, filled squares = ‘spray’. Distance between transects: 5 m
Discussion It is suggested that SSWC will be profitable when automatic detection becomes a reality, due to reduced mapping costs and the possibility of real-time herbicide treatment (e.g.
123
402
Precision Agric (2008) 9:391–405
Lutman and Miller 2007). Although real-time application was the ultimate goal for ‘‘WeedFinder’’, a prototype of it was tested off-line in the present study, to evaluate its reliability in an envisaged SSWC context with ‘spray’/‘no spray’ decisions. Generally, ‘‘WeedFinder’’ over-estimated images with low true weed infestations, and under-estimated images with high true infestations, which was not unlike previous findings (e.g. Andreasen et al. 1997; Pe´rez et al. 2000; Hague et al. 2006). The main reasons for the over- and under-estimations are small cereal leaf parts (leaf tips) incorrectly identified as weed objects and overlapping weed leaves, respectively. The problems of touching and partial occlusion of leaves remain the most challenging tasks to be solved for automatic weed detection within field crops (Tellaeche et al. 2008). The work of Pedersen et al. (2002) and even an earlier version of ‘‘WeedFinder’’ (Clausen et al. 2000) had less dispersion and a much better 1:1 relationship between the manual and automatic weed assessments than achieved presently. The main reason for the discrepancy between present and earlier results obtained by ‘‘WeedFinder’’ was probably that the previous version also included the ratio between the principal axis of leaves to distinguish between broad-leaved weeds and cereals. If the shape parameter ‘roundness’ (parameter No. 5, Table 3) had been tested on all the objects defined as weeds due to size, and not only as a parameter to search for partly occluded weeds, the predictions would probably improve. A selection of other shape characteristics (e.g. as suggested by Gerhards et al. 1993) and/or textural information (e.g. Meyer et al. 1998) would also probably improve the species discrimination. The images used originally (Clausen et al. 2000) had a too small field-of-view to allow identification of the cereal rows as has been included in detection techniques by many others (e.g., Blasco et al. 1999; Pe´rez et al. 2000; Ribeiro et al. 2005; Hague et al. 2006; Ge´e et al. 2008). The reason why the vegetation-background segmentation was not successful in a few images, resulting in vegetation being erroneously estimated to cover almost the entire image (in about 1% of the images), was possibly a combination of very low vegetation cover and a high proportion of relatively bright stones and clods. Since such conditions are not rare, this suggests that the vegetation-background segmentation step requires some modifications. Furthermore, the need for standard illumination conditions during image acquisition must be solved by feasible means, such as by use of a powerful flash. The possibility of increasing the row spacing to enhance the cereal-weed discrimination based on image analysis has been suggested (Lutman and Miller 2007), and has been tested for e.g., organically grown cereals (Ribeiro et al. 2005) and under conditions where excessive tillering of winter wheat can be expected (Hague et al. 2006). For conventional spring cereals in temperate areas we doubt that this is a sustainable option, because the weed-suppressing effect of the crop is likely to be reduced if the crop row spacing is increased (Ha˚kansson 2003). Pedersen et al. (2002) reported that their image-based procedure for weed density estimation was optimized when spring barley row spacing was 0.36 m, and suggested that the tramlines could be used for automatic weed detection. Recently, Dammer and Wartenberg (2007) demonstrated how a sprayer could be controlled in real-time by an opto-electronic sensor, which assessed the weed infestation level within the non-drilled tramlines. Instead of using the tramlines, where the weeds might have substantially different germination and growth conditions than elsewhere, omitting one crop row beneath the camera’s field of view would probably improve species differentiation and weed density estimation. However, the opportunity to include the component of crop cover as a predictor in the decision model (e.g. Ngouajio et al. 1999) will then be blocked, unless a multi-camera system is implemented. A multi-camera system and individual spray boom section control (i.e., higher detection and application
123
Precision Agric (2008) 9:391–405
403
resolution) would also improve the quality of the site-specific herbicide treatment (cf. Figs. 4 and 5). As the exact total weed densities are often irrelevant to the value of site-specific herbicide application maps (cf. Wiles 2005), ‘‘WeedFinder’’ was also evaluated in terms of its ability to predict ‘spray’/‘no spray’ decisions. Average success rates (i.e., spray decision predicted by ‘‘WeedFinder’’ equals true spray decision) across fields for method 1 were 73% and 77% (depending on spray decision model criteria, cf. Tables 4 and 5). For method 2, the average success rate was 86% (cf. Table 6). These were in the same range as reported for images grouped into four weed infestation levels (mostly Avena sterilis L. and Papaver rhoeas L.) according to weed, barley and soil covers predicted by image analysis (Ribeiro et al. 2005). Recently, Tellaeche et al. (2008) reported a high success rate (92%) for ‘spray’/‘no spray’ decisions based on image analysis of A. sterilis within barley. The question that remains to be answered is ‘are these success rates sufficiently good for automatic prediction of SSWC decisions to be viable?’ For any target success rate, we emphasize that the mapping error should be as low as possible, since it is more risky (from a farmer’s viewpoint) that areas above damage threshold are not receiving herbicide (i.e., mapping error) than areas below threshold are receiving herbicide (i.e., spraying error). The accuracy in success rate could possibly have been enhanced by using other classification methods such as neural network classifiers (e.g. Burks et al. 2005). However, simplicity for the end-users should be kept in mind (Lutman and Miller 2007), meaning that as little training and calibration as possible should be required. Hence, for operational use, a simple look-up table seems more appropriate because an optimal calibrated classifier will probably depend on individual situations. A simple look-up table would also simplify the possibility of including flexibility in the final software, i.e., the possibility for the enduser to adjust the threshold level according to, for example, weed density, and herbicide and grain prices. Since spray decision models in practice should be dynamic (cf. Christensen 1994; Young et al. 2003), the target success and error rates resulting from automatic weed detection and SSWC must be maintained even with changed levels of the decision model. Results at Myhrer showed acceptable rates at the increased threshold level applying method 1 and criterion 1 (cf. Table 5) and for all threshold levels applying method 2 (cf. Table 6). Elsewhere, results were relatively variable for the different threshold levels with success rate quite moderate and mapping error relatively large (cf. Tables 4–6). We must therefore conclude that the present version of ‘‘WeedFinder’’ is not robust enough for field applications. Future versions of it must be more accurate in discriminating between cereals and weeds, and recognition of troublesome weed species must be included. Acknowledgments This work was funded by the Norwegian Institute of Agricultural and Environmental Research (BIOFORSK). We thank M. Helgheim, BIOFORSK, for help with the true values of weed density in part of the evaluation dataset. The authors also thank S. Clausen, T. Kirkhus and K. Kaspersen, SINTEF ICT (Oslo, Norway), for support with the programs ‘‘WeedFinder’’ and ‘‘Ground Truth’’.
References Andreasen, C., Rudemo, M., & Sevestre, S. (1997). Assessment of weed density at an early stage by use of image processing. Weed Research, 37, 5–18. Anonymous. (2001). Growth stages of mono- and dicotyledonous plants. BBCH Monograph. Edited by U. Meier, Federal Biological Research Centre for Agriculture and Forestry. http://www.bba.de/veroeff/ bbch/bbcheng.pdf. Accessed 5 May 2007.
123
404
Precision Agric (2008) 9:391–405
Biller, R. H. (1998). Reduced input of herbicide by use of optoelectronic sensors. Journal of Agricultural Engineering Research, 71, 357–362. Blasco, J., Benlloch, J.V., Agusti, M., & Molto´, E. (1999). Machine vision for precise control of weeds. In G. E. Meyer & J. A. DeShazer (Eds.), Proceedings of the SPIE Conference on Precision Agriculture and Biological Quality Vol. 3543 (pp. 336–343). Boston, USA: SPIE. Brown, R. B., & Noble, S. D. (2005). Site-specific weed management: Sensing requirements-what do we need to see? Weed Science, 53, 252–258. Burks, T. F., Shearer, S. A., Heath, J. R., & Donohue, K. D. (2005). Evaluation of neural-network classifiers for weed species discrimination. Biosystems Engineering, 91, 293–304. Christensen, S. (1994). Crop weed competition and herbicide performance in cereal species and varieties. Weed Research, 34, 29–36. Clausen, S., Nicolas, S., Kirkhus, T., & Fykse, H. (2000). Automatic weed mapping in cereal fields using image processing techniques. In I. Solheim (Ed.), Proceedings: en samling artikler og sammendrag fra ˚ s, Norway: NOBIM 2000-sekretariatet (ISBN 82-7747-108-4). NOBIM konferansen 2000 (pp. 1–5). A Dammer, K.-H., & Wartenberg, G. (2007). Sensor-based weed detection and application of variable herbicide rates in real-time. Crop Protection, 26, 270–277. Dicke, D., Gerhards, R., Bu¨chse, A., & Hurle, K. (2007). Modelling spatial and temporal dynamics of Chenopodium album L. under the influence of site-specific weed control. Crop Protection, 26, 206–211. Dieleman, J. A., & Mortensen, D. A. (1999). Characterizing the spatial pattern of Abutilon theophrasti seedling patches. Weed Research, 39, 455–467. Fykse, H. (1991). Skadetersklar for ugras i va˚rkorn (Damage thresholds for weeds in spring cereal). In ˚ s, Norway: Statens fagtjeneste for Faginfo 2/Statens fagtjeneste for landbruket (pp. 165–173). A landbruket (ISSN 0803-2173). Ge´e, C., Bossu, J., Jones, G., & Truchetet, F. (2008). Crop/weed discrimination in perspective agronomic images. Computers and Electronics in Agriculture, 60, 49–59. Gerhards, R., & Ku¨hbauch, W. (1993). Dynamic decision model for weed control methods in cereals by means of digital image analysis. Journal of Agronomy and Crop Science, 171, 329–335. Gerhards, R., Nabout, A., So¨kefeld, M., Ku¨hbauch, W., & Nour Eldin, H. A. (1993). Automatic identification of 10 weed species in digital images using Fourier descriptors and shape parameters. Journal of Agronomy and Crop Science, 171, 321–328. Gerhards, R., & Oebel, H. (2006). Practical experiences with a system for site-specific weed control in arable crops using real-time image analysis and GPS-controlled patch spraying. Weed Research, 46, 185–193. Hague, T., Tillett, N. D., & Wheeler, H. (2006). Automated crop and weed monitoring in widely spaced cereals. Precision Agriculture, 7, 21–32. Heijting, S., Van der Werf, W., Kruijer, W., & Stein, A. (2007). Testing the spatial significance of weed patterns in arable land using Mead’s test. Weed Research, 47, 396–405. Ha˚kansson, S. (2003). Weeds and weed management on arable land: An ecological approach. Wallingford, Oxon, UK: CABI Publishing. Johnson, R. A., & Wichern, D. W. (2002). Applied multivariate statistical analysis. Upper Saddle River, New Jersey, USA: Prentice Hall. Lamb, D. W., & Brown, R. B. (2001). Remote-sensing and mapping of weeds in crops. Journal of Agricultural Engineering Research, 78, 117–125. Lutman, P. J. W., & Miller, P. C. H. (2007). Spatially variable herbicide application technology; opportunities for herbicide minimisation and protection of beneficial weeds. HGCA Research Review No. 62. UK: The Home-Grown Cereals Authority. Meyer, G. E., Mehta, T., Kocher, M. F., Mortensen, D. A., & Samal, A. (1998). Textural imaging and discriminant analysis for distinguishing weeds for spot spraying. Transactions of the ASAE, 41, 1189– 1197. Ngouajio, M., Leroux, G. D., & Lemieux, C. (1999). A flexible sigmoidal model relating crop yield to weed relative leaf cover and its comparison with nested models. Weed Research, 39, 329–343. Nordmeyer, H. (2006). Patchy weed distribution and site-specific weed control in winter cereals. Precision Agriculture, 7, 219–231. Otsu, N. (1979). A threshold selection method from gray-level histograms. IEEE Transactions on Systems Man and Cybernetics, 9, 62–66. Pedersen, B. B., Hansen, P. K., & Walter, A. M. (2002). Weed density estimation from digital images. In H. H. van Laar (Ed.), Proceedings of the 12th European Weed Research Society Symposium (pp. 416– 417). Doorwerth, The Netherlands: EWRS Bookstore. Pe´rez, A. J., Lo´pez, F., Benlloch, J. V., & Christensen, S. (2000). Colour and shape analysis techniques for weed detection in cereal fields. Computers and Electronics in Agriculture, 25, 197–212.
123
Precision Agric (2008) 9:391–405
405
Ribeiro, A., Fernandez-Quintanilla, C., Barroso, J., & Garcia-Alegre, M. C. (2005). Development of an image analysis system for estimation of weed pressure. In J. V. Stafford (Ed.), Precision Agriculture ’05 (pp. 169–174). Wageningen, The Netherlands: Wageningen Academic Publishers. SAS Institute. (2007). SAS/STAT version 9.1, Cary, North Carolina, USA. Sui, R., Thomasson, J. A., Hanks, J., & Wooten, J. (2008). Ground-based sensing system for weed mapping in cotton. Computers and Electronics in Agriculture, 60, 31–38. Søgaard, H. T. (2005). Weed classification by active shape models. Biosystems Engineering, 91, 271–281. Tellaeche, A., BurgosArtizzu, X. P., Pajares, G., Ribeiro, A., & Ferna´ndez-Quintanilla, C. (2008). A new vision-based approach to differential spraying in precision agriculture. Computers and Electronics in Agriculture, 60, 144–155. Wiles, L. (2005). Sampling to make maps for site-specific weed management. Weed Science, 53, 228–235. Young, D. L., Kwon, T. J., Smith, E. G., & Young, F. L. (2003). Site-specific herbicide decision model to maximize profit in winter wheat. Precision Agriculture, 4, 227–238. Zwiggelaar, R. (1998). A review of spectral properties of plants and their potential use for crop/weed discrimination in row-crops. Crop Protection, 17, 189–206.
123
