CSIRO PUBLISHING
www.publish.csiro.au/journals/cp
Crop & Pasture Science, 2009, 60, 60–70
Improved wheat yield and production forecasting with a moisture stress index, AVHRR and MODIS data A. G. T. Schut A,B,E, D. J. Stephens C, R. G. H. Stovold D, M. Adams D, and R. L. Craig D A
Department of Spatial Sciences, Curtin University of Technology, GPO Box U1987, Perth, WA 6001, Australia. B Cooperative Research Centre for Spatial Information, 723 Swanston Street, Parkville, Vic. 3052, Australia. C Department of Agriculture and Food Western Australia (DAFWA), Locked Bag 4, Bentley Delivery Centre, WA 6983, Australia. D Landgate, PO Box 2222, Midland, WA 6056, Australia. E Corresponding author. Email:
[email protected]
Abstract. The objective of this study was to improve the current wheat yield and production forecasting system for Western Australia on a LGA basis. PLS regression models including temporal NDVI data from AVHRR and/or MODIS, CR, and/or SI, calculated with the STIN, were developed. Census and survey wheat yield data from the Australian Bureau of Statistics were combined with questionnaire data to construct a full time-series for the years 1991–2005. The accuracy of fortnightly inseason forecasts was evaluated with a leave-year-out procedure from Week 32 onwards. The best model had a mean relative prediction error per LGA (RE) of 10% for yield and 15% for production, compared with RE of 13% for yield and 18% for production for the model based on SI only. For yield there was a decrease in RMSE from below 0.5 t/ha to below 0.3 t/ha in all years. The best multivariate model also had the added feature of being more robust than the model based on SI only, especially in drought years. In-season forecasts were accurate (RE of 10–12% and 15–18% for yield and production, respectively) from Week 34 onwards. Models including AVHRR and MODIS NDVI had comparable errors, providing means for predictions based on MODIS. It is concluded that the multivariate model is a major improvement over the current DAFWA wheat yield forecasting system, providing for accurate in-season wheat yield and production forecasts from the end of August onwards.
Introduction Knowledge about the actual wheat yield (t/ha) and production (total tonnage) before harvest is vital for a wide range of users such as farmers, agribusiness, and government departments. Wheat Currently, strategic planning for wheat in Western Australia, depends on feedback from farmers (questionnaires) and expert inventories. The accuracy of estimates relies on the response rate and the farmer’s ability and willingness to estimate accurately. In order to have the information well before harvest, questionnaires are distributed around mid-season in July. Alternatively, yield and production can be estimated using remotely sensed information (Labus et al. 2002; Doraiswamy et al. 2003; Mo et al. 2005; Patel et al. 2006; Moriondo et al. 2007). A wealth of studies looked at the relationships between the NDVI and wheat yield, using temporal variation (Benedetti and Rossini 1993; Quarmby et al. 1993; Labus et al. 2002; Manjunath
et al. 2002; Kalubarme et al. 2003; Moriondo et al. 2007) or spatial variation (Smith et al. 1995; Serrano et al. 2000; Rodriguez et al. 2005). Although promising in most cases, relationships between remotely sensed indices and crop status are affected by management practices, soil type, water content, and fertility. Therefore, these relationships are empirical by nature and thus geographically specific and sensitive to scale. Another more generic approach is to use the NDVI derived from remote sensing as an input into agro-meteorological models (Dorigo et al. 2007). A commonly used approach is to integrate the fraction of intercepted radiation derived from remotely sensed data in crop growth models (Bouman 1995; Jongschaap 2006; Patel et al. 2006). Alternatively, outputs of agro-meteorological models and remotely sensed crop indices can be combined into a single statistical function relating to yield (Boken and Shaykewich 2002).
Abbreviations: ABS, Australian Bureau of Statistics; AVHRR, advanced very high resolution radiometer; CBH, Cooperative Bulk Handlers; CR, cumulative rainfall; DAFWA, Department of Agriculture and Food, Western Australia; LGA, Local Government Area; MODIS, moderate resolution imaging spectrometer; NDVI, normalised difference vegetation index; PLS, partial least squares; RE, relative prediction error; RMSE, root mean squared error; SI, STIN stress index; STIN, stress index model. CSIRO 2009
10.1071/CP08182
1836-0947/09/010060
Assessment of wheat yield and production
The Western Australian (WA) wheatbelt has a typical Mediterranean climate with semi-arid growing conditions. Accordingly, the amount and distribution of rainfall is the most important factor determining the length of the growing season and in-season crop growth (Stephens and Lyons 1998; Hill and Donald 2003). The typical sandy soil types have a very limited water storage capacity, enhancing dependence on regular rainfall. Based on soil characteristics, rainfall, and temperature data, the STIN calculates a weighted cumulative SI to account for the effect of water availability and drought on wheat growth (Stephens et al. 1989; Stephens and Lyons 1998). STIN is currently used in the system of yield forecasting in Western Australia. It is also used to assess the effects of drought on crop production and forms the basis for assessing exceptional circumstances and drought relief assistance (Stephens et al. 1994; Stephens 1998). There are several factors affecting wheat yields that are unaccounted for in the current system, such as the effects of crop management, frosts, and the effects of pests and diseases. In years with very late or insufficient opening rains, only a small fraction of seeds sown may emerge, severely reducing the attainable yield. The first objective of this study is to improve the current inseason yield forecasting system run within the DAFWA. The second objective is to develop a production forecasting system at the LGA level. To this end, the SI and the NDVI calculated from NOAA-AVHRR data were combined following the approach of Boken and Shaykewich (2002). The third objective is to base yield and production forecasts on MODIS data. MODIS has a better spatial resolution and a better radiometric calibration than AVHRR, allowing more accurate future forecasts at a sub-LGA or even farm level. To this end, a NDVI time-series was constructed combining MODIS and AVHRR data. This provided means to create a long MODIS NDVI time-series, essential for model calibration and testing. Materials and methods Ground-truth data The ABS collects yield and production data using census and sample surveys. In a census, all farmers are asked to provide estimates of the area sown, including failed crops, and the total production for each crop type. Data from surveys are based on questionnaires from a sample of the population of farmers. Census and survey questionnaires are sent out in summer and collect post-harvest information. For the years 1991–96, 2000, and 2005, census data are available, and for 1997–99 and 2001, only survey data are available at the LGA level. At a higher aggregation level of the statistical divisions, survey data are also available for the years 2002–04 providing a complete series for 1991–2005. Missing data at the LGA level for the years 2002–04 were estimated using data obtained from CBH. CBH is the dominant grain handler in WA and processes over 85% of all wheat produced. Annually, CBH sends a survey out to all farmers, requesting information about the crop area sown and the expected fraction of that area that will be delivered to its storage bins at harvest. For production, accurate data from the total tonnage received at the CBH storage bins are available. Yield was estimated from the tonnage delivered and the estimated area of
Crop & Pasture Science
61
wheat delivered. As the acreage delivered is a pre-harvest estimate, it is expected that this estimate is biased. The area actually delivered will probably be smaller or larger than estimated in below- and above-average years, resulting in underestimated or overestimated yields, respectively. Comparison of mean yields at the state and statistical division level revealed that the area of wheat sown as reported by the ABS and CBH had diverged after 2001, whereas the production data had more minor differences. The ABS yield data were used as reference in this study because they are expected to be of better quality than the CBH because they are based on data from all farmers and are collected post-harvest. To correct for higher CBH LGA yields after 2001, correction factors were determined per statistical division per year. Recent CBH LGA yields were multiplied with a proportional correction factor for each year. The missing yield data at the LGA level for the years 2002–04 in the ABS series were estimated from the temporally corrected CBH yields. For this, ABS yields and corrected CBH yields for the census years 1991–96, 2000, and 2005 were compared after removal of erroneous LGAs (see below). CBH production data indicate the total tonnage of wheat delivered to bins. It is expected that the fraction of the wheat produced that remains on the farm increases in years with a low production. This is because a proportion of the grain produced is retained on farms as seed for the following year, as sheep feed, or is not harvested when yields are very low (crop failure). This means that the estimated ABS yields are biased. To correct for these factors, yield estimates were compared with yields from control varieties that were used for at least 10 years in DAFWA cultivar selection trials. The cvv. Cadoux, Carnamah, Cascades, Machete, Spear, and Westonia were selected for this purpose. These cultivars were used in trials across the WA wheatbelt and covered the full period. There were 457–2181 (non-irrigated) trials available; the number of trials peaked in the late 1990s and, in 2005, about 450 trials were available. Trial sites are chosen so they accurately represent the WA wheatbelt. The large number of trials available at the statistical division and state level, averages out effects of soil type and trial management. Yield anomalies (relative deviations from the trend-line, based on a fitted first-order regression) were determined for each trial–cultivar combination and mean yield anomalies across trials at a statistical division and at a state level were calculated. These yield anomalies were compared with yield anomalies of CBH and ABS data at the statistical division and state level. Calculation of stress index In this study, STIN was used to calculate the SI on a weekly basis. The SI is calculated using rainfall data from weather observation stations of the Australian Bureau of Meteorology and standard soil types. For each LGA, data from one weather station were used. Based on an optimised maximum extractable soil water for each LGA, STIN calculates a cumulative moisture stress index to account for surplus or deficit water availability to plant growth (Stephens et al. 1989; Stephens and Lyons 1998). The negative effects of late sowing are incorporated through a sowing date routine.
62
Crop & Pasture Science
A. G. T. Schut et al.
AVHRR data and processing The remotely sensed data used in this study were derived from the AVHRR sensor on board the NOAA satellite and the combined MODIS sensors on board the Terra and Aqua satellites, and were provided by Landgate (Midland, WA). The time-series included combinations of data measured by the AVHRR/2 sensor on board the NOAA-11 (1991–94), NOAA-9 (1994–95), and NOAA-14 (1995–2001) satellites and the AVHRR/3 sensor on board the NOAA-16 (2001–03) and NOAA-17 (2004–05) satellites. Standard cloud-masks were used and fortnightly composites based on maximum NDVI values were determined (for further details see Hill and Donald 2003). All areas with non-agricultural land (remnant vegetation, salt pans, etc) were masked out. The remnant vegetation mask was based on the land cover grid of the Australian Greenhouse Office (Furby 2002), and MODIS pixels including 30% remnant vegetation or more were masked out (G. Donald, pers. comm.). The NDVI was calculated using wavebands in the red and infrared region (Baret and Guyot 1991). Afterwards, the mean NDVI per LGA was calculated, including all pixels from crop and pasture areas. Fortnightly composites were re-sampled to weekly values by linear interpolation. The mean temporal weekly NDVI values per LGA were smoothed to remove discontinuities in the time-series, using a digital filter running in forward and backward mode to minimise effects of start and end points [the filtfilt function from the signal processing toolbox (Matlab 2000)]. The filter weights per week were based on a normal distribution function with a standard deviation of 2 weeks. The spectral response functions of the red and infrared band (used for NDVI) differ slightly for AVHRR/2 and AVHRR/3, respectively (Trishchenko et al. 2002). As a result, the NDVI values showed a dramatic shift when changing from AVHRR/2 (NOAA 9–14) to AVHRR/3 (NOAA 16–17) in May 2001. Based on work by S. Cridland (pers. comm.), cross-sensor NDVI calibration coefficients were established for Australia, and AVHRR/3 NDVI was converted to AVHRR/2 NDVI equivalents with a linear regression: AVHRR/2 = –0.12 + 1.12 AVHRR/3. This equation was validated with Landgate NOAA 14 and NOAA 16 imagery from a region near Northam (WA), collected between October 2000 and April 2001. The relationship accurately converted AVHRR/3 to AVHRR/2 until March 2001.
repeat cycle, spatial resolution, wavelengths used for NDVI calculation, and sensor characteristics. Also, the masking of MODIS is far more accurate than for AVHRR due to its better spatial resolution. Although temporal patterns detected with both sensors are in general similar, conversion from MODIS to AVHRR or viceversa is not straightforward. MODIS NDVI values were converted to match AVHRR NDVI values for the overlapping period (2003–06). Relationships between NOAA and MODIS NDVI values were different for the first and second half of the season and were slightly non-linear. For each year, Richardsfunctions were fitted to establish a relationship between MODIS and AVHRR NDVI values for the periods between January–September and October–December. The relationship of 2004 was most consistent and was used to convert NDVI values from MODIS to AVHRR/2 equivalents from October 2003 onwards. Using the two different functions in a year does result in a discontinuous time-series. In this study, however, the normalisation procedure used in the regression corrects for temporal discontinuities as data were independently normalised (see pre-processing below) and consistency in values from the same week across years is more important than consistency between weeks within the same year.
Conversion of MODIS NDVI to AVHRR NDVI MODIS NDVI data were provided by Landgate (Midland, WA). As with the AVHRR imagery, remnant vegetation and clouds were masked out and composite images were created on a fortnightly basis. Afterwards, LGA means of NDVI were calculated, interpolated to weekly values, and smoothed accordingly. The approach used in this study requires a long time-series to calibrate the models. Long time-series for MODIS NDVI were not available. Alternatively, when MODIS NDVI can be transformed to AVHRR NDVI or vice-versa, data can be combined, resulting in suitable time-series of NDVI data (Gallo et al. 2005). There is a considerable difference between MODIS and AVHRR NDVI values (Trishchenko et al. 2002). Several factors affect the measured values, such as viewing angle,
Table 1. Maximum number of x-variables and latent vectors (LV) included in the partial least-squares models
Multivariate regression model A PLS regression model was used to relate yield or production (y variates) to a combination of x variates. With PLS analysis, the x variables are recombined into latent vectors in such a way that it best describes the variation in both the x and y blocks (Geladi and Kowalski 1986). The number of latent vectors included in the model was maximised at 10. Several models were compared with different x-variables (Table 1). The SI in Model 1 stands for the STIN stress index value. This model represents the current forecasting system and is therefore the control. For the models including NDVI, 4 weekly re-sampled NDVI values were included from Week 6 onwards (including weekly values did not improve accuracy). For CR, weekly values from Week 31 were included. In contrast to rainfall in other periods of the year, rainfall after Week 31 (1 August) is critical for yield (grain filling), but is not reflected in the NDVI signal because plants start to senesce in spring and NDVI values start to decrease. Including CR values of other
Model
x variates
1 2 3 4 5 6 7 8 9
Year + SI Year + CR Year + SI + CR + SI CR Year + NDVIr Year + SI + NDVIr Year + SI + CR + NDVIr Year + sID + NDVI Year + sID +NDVI + CR Year + SI + sID + NDVI + SI NDVI Year + SI + sID + NDVI + CR + SI CR + SI NDVI
10
x variables
LV
2 15 30 2 3 15 17 26 23
2 10 10 2 3 10 10 10 10
91
10
Assessment of wheat yield and production
periods in the year did not improve accuracy. For the models including NDVI, a scalar (1 for AVHRR/2 and 2 for AVHRR/3) for sensor type (sID) was also included. The NDVIr indicates a ratio between NDVI values in 2 selected weeks: NDVIr ¼
NDVInw ; NDVIdw
where 6 nw 38 and
nw þ 4 dw last week where the week numbers for the denominator (dw) and numerator (nw) were iteratively determined by minimising the model errors in Model 4. The nw is selected from weeks before the peak NDVI (Week 38) and the dw must be equal to or smaller than the last week of the season. In-season forecasts can only be based on data available at the moment of forecasting. The smoothed NDVI values were re-calculated accordingly with original NDVI data until the week of forecasting. Models including data from Weeks 6–44 (end of season), decreasing by a fortnight to a period of Week 6–32 (mid August), were compared to evaluate in-season forecasting ability. Selection of LGAs included From the 68 LGAs in the wheatbelt of WA, a total of 57 LGAs producing at least 15 000 t of wheat were used in this study. LGAs with very noisy NDVI patterns (all years for Esperance, Ravensthorpe, and Tambellup; Jerramungup in 2000; Kent and Lake Grace in 1992; Gnowangerup in 1993) were excluded from the analysis. For yield, large RMSE values were found for the relationship between ABS and CBH yield data for the LGAs of Toodyay, Irwin, and Dandaragan, and these LGAs were not included. For production, all years of York and Dalwallinu, Mullewa in 1994 and 1997, and Yilgarn from 2000 to 2005 were excluded. For production, original data from CBH were used as ground-truth, without trend or other corrections. This resulted in a dataset including 50 and 51 LGAs with 747 and 745 observations for yield and production, respectively. Pre-processing of the data All x and y variates were normalised to a mean of 0 and a standard deviation of 1 (by subtracting the mean and dividing by the standard deviation) per LGA. Then, data were combined into one large matrix, consisting of 747 records for yield and 745 records for production. The models were calibrated (trained) on the normalised data. For calibration, fitted values were rescaled using the means and standard deviations of each LGA in the calibration set. For the leave-year-out validation, predicted values were multiplied with the standard deviation and mean of the data per LGA in the calibration set. For the analysis, Matlab 7.1 including the Signal Processing and PLS toolboxes was used (Matlab 2000; Wise et al. 2005). ArcMap 9.2 was used to map the results. Cross-validation on independent data In the calibration step, model accuracy was evaluated by a leaveone-out procedure. However, this procedure overestimates the
Crop & Pasture Science
63
accuracy of the model as values from the same year are more likely to be similar than values from different years. Such a structural redundancy in the dataset can invalidate the leave-one-out approach to evaluate model predictability (Golbraikh and Tropsha 2002). Therefore, a leave-year-out approach was used to test the predictability of an unknown year. This means that models were calibrated on all years with data from one year left out. Then, yield and production for each LGA of the year left out were predicted using the calibrated model. This was repeated for all years. To obtain insight into the prediction capability of the model, we used the percentage of variation accounted for by the cross-validated model (using the leave-year-out predictions) with regard to the total variation in the dataset: P ðy ^yi Þ2 Q2 ¼ 1 Pi i yÞ2 i ðyi where y, y, and ^y are the measured, mean, and estimated value of yield or production, respectively (see also Golbraikh and Tropsha 2002). The Q2 has strong resemblance to R2 but can become negative if the prediction of the model is inadequate (e.g. in case of over-fitting). The Q2 is (as R2) strongly sensitive to the variation within the dataset, and so should be considered with caution. Results Corrections of ground-truth data The linear relationship fitted (ABS yield = 0.187 + 0.901 CBH yield) between ABS yields and CBH yields (corrected for temporal inconsistency) for the census years 1991–96, 2000, and 2005 yielded an R2 value of 0.91 and a RMSE of 0.116 t/ha. However, the residuals were slightly larger for 2000 and 2005 (RMSE of 0.10–0.13 t/ha) compared with 1991–96 (RMSE of 0.05–0.08 t/ha). In general, mean RMSE values across years were larger in the west and south-west (typically above 0.1 t/ha with a relative error of 5–10%) of the WA wheatbelt than in central or eastern parts (typically below 0.1 t/ha with a relative error
