International Journal of Remote Sensing Vol. 32, No. 13, 10 July 2011, 3689–3709
Evaluating the effectiveness of smoothing algorithms in the absence of ground reference measurements
Downloaded by [UB Bodenkultur Wien] at 01:17 29 June 2011
CLEMENT ATZBERGER*† and PAUL H. C. EILERS‡ †Joint Research Centre of the European Commission (JRC), Institute for the Protection and Security of the Citizen, Agricultural unit (MARS), 21027, Ispra, Italy ‡Erasmus Medical Centre, Department of Biostatistics, 3015 GE Rotterdam, The Netherlands (Received 27 October 2009; in final form 8 March 2010) Time series of vegetation indices like NDVI are used in numerous applications ranging from ecology to climatology and agriculture. Often, these time series have to be filtered before application. The smoothing removes noise introduced by undetected clouds and poor atmospheric conditions. Ground reference measurements are usually difficult to obtain due to the medium/coarse resolution of the imagery. Hence, new filter algorithms are typically only (visually) assessed against the existing smoother. The present work aims to propose a range of quality indicators that could be useful to qualify filter performance in the absence of ground-based reference measurements. The indicators comprise (i) plausibility checks, (ii) distance metrics and (iii) geostatistical measures derived from variogram analysis. The quality measures can be readily derived from any imagery. For illustration, a large SPOT VGT dataset (1999–2008) covering South America at 1 km spatial resolution was filtered using the Whittaker smoother.
1.
Introduction
Coarse resolution NDVI time series have been used successfully within various environmental studies. Prominent applications are listed in table 1. For all applications a high reliability of the analysed time series is required, meaning that the recorded signal should be closely related to the status of the observed land surface. This is often not the case as the time series are more or less strongly affected by (undetected) clouds, dust and aerosols, which add (negatively biased) noise to the signal (Goward et al. 1991). Part of the noise is removed through the standard maximum value compositing (MVC) (Holben 1986). With MVC only the highest NDVI value in a predefined compositing period is retained. This results in fewer but more reliable NDVI values representing the time series. However, it is also known, that additional filtering is necessary for removing the remaining noise. To denoise the MVC images, a number of filtering techniques have been proposed (table 2) and several research groups continue working on this issue. There is currently no agreement on which filter performs best for smoothing remotely sensed time series. On the contrary, serious drawbacks have been noted for all techniques (see recent discussions in Chen et al. 2004, Beck et al. 2006, Hird and *Corresponding author. Email:
[email protected] International Journal of Remote Sensing ISSN 0143-1161 print/ISSN 1366-5901 online # 2011 Taylor & Francis http://www.tandf.co.uk/journals DOI: 10.1080/01431161003762405
3690
C. Atzberger and P. H. C. Eilers
Table 1. Prominent examples using medium to coarse resolution NDVI time series for environmental studies. Application
Prominent examples
Measurements of phenological variability
Downloaded by [UB Bodenkultur Wien] at 01:17 29 June 2011
Establishment of continental phenological models Unravelling of teleconnections between climatic anomaly indices and vegetation response Modelling of spatio-temporal patterns of vegetation activity Assessments of global and climate change
Reed et al. 1994, Sto¨ckli and Vidale 2004, Bradley et al. 2007 Justice et al. 1985, White et al. 1997, Moulin et al. 1997 Anyamba and Eastman 1996, Liu and Negro´n Jua´rez 2001, Kogan 2000 Martı´nez and Gilabert 2009, Yu et al. 2003
Tucker et al. 2001, Shabanov et al. 2002, Zhou et al. 2001, Li and Kafatos 2000 Derivation of plant functional types, Paruelo et al. 2001, Xiao et al. 2002, Azzali phenoregions and land-use categories and Menenti 2000, Geerken 2009 Ecological research and wildlife distribution Pettorelli et al. 2005, Kerr and Ostrovsky modelling 2003 Support to biodiversity mapping Hurlbert and Haskell 2003, Coops et al. 2009a,b Agricultural monitoring and yield predictions Zhang et al. 2005, Fuller 1998 Mapping of (subpixel) area fractions of summer/ Atzberger and Rembold 2009 winter crops Drought mapping and assessment McVicar and Jupp 1998, Ji and Peters 2003
Table 2. Overview of smoothing techniques for remotely sensed time series of vegetation indices. Type of filter
Abbreviation
Prominent applications
Harmonic Series and Higher Order Splines
HS
Double logistic Asymmetric Gaussian
DL AG
Savitzky–Golay Wavelets Weighted least squares windowed regression Running medians Best Index Slope Extraction Mean Value Iteration Whittaker smoother
SG WL SWETS
Roerink et al. 2000, Jakubauskas et al. 2001, Moody and Johnson 2001, Hermance 2007, Bradley et al. 2007, McCloy and Lucht 2004 Beck et al. 2006, Zhang et al. 2003 Jo¨nsson and Eklundh 2002, Jo¨nsson and Eklundh 2004 Chen et al. 2004 Sakamoto et al. 2005, Li and Kafatos 2000 Swets et al. 1999
4253H BISE
Velleman 1980 Viovy et al. 1992, Lovell and Graetz 2001
MVI WT
Ma and Veroustraete 2006 Atzberger and Eilers 2010, Atzberger and Rembold 2009
McDermid 2009). The problem of developing robust, accurate and fast filters is amplified by the difficulty to obtain (ground-based) reference measurements that could serve for validation purposes. The difficulties mainly arise because of the coarse scale of the sensors and the required representativeness of the reference measurements in space and time (Justice et al. 2000). As a consequence of these difficulties, new filters
Downloaded by [UB Bodenkultur Wien] at 01:17 29 June 2011
Effectiveness of smoothing algorithms
3691
are usually compared (visually) against the existing filter (e.g. Jo¨nsson and Eklundh 2002). Other studies judged the satellite-derived vegetation phasing against phenological (point) observations (Beck et al. 2007, Maignan et al. 2008, Soudani et al. 2008). The geostatistically derived signal-to-noise ratio (SNR) as defined by Atkinson et al. (2007) and Asmat et al. (2007) was used only occasionally for assessment of filter performance (Chappell et al. 2001, Atzberger and Eilers 2010). Alternatively, it was proposed to use synthetic datasets for filter comparison (e.g. Hird and McDermid 2009). Using the latter approach, however, one relies inevitably on the representativeness of the assumed noise characteristics. The objective of the study is to present candidate indicators that could be useful for evaluating the effectiveness of existing and new filters, when ground reference measurements are unavailable. Three types of indicators will be presented and discussed: (i) plausibility checks, (ii) distance metrics and (iii) geostatistical measures derived from variogram analysis. To illustrate the proposed quality indicators, the Whittaker smoother (Atzberger and Eilers 2010) was applied to a large (10-day MVC) SPOT VGT dataset covering South America (1998–2008) at 1 km spatial resolution. 2. 2.1
Material and methods Study area
The region of interest covers a large area of approximately 18 000 000 km2 (image size: 4400 4000) including Brazil, Argentina, Uruguay, Paraguay and Bolivia (figure 1). The region is situated between 0 S, 70 W (top left) and 40 S, 35 W (bottom right). Climatic conditions within the study area are highly variable. Vegetation types range from tropical evergreen moist forests to xeric and montane shrublands. The key ecoregions of the area are shown in figure 1(a) according to Olson et al. (2001). The distribution of the main land cover types is depicted in figure 1(b) based on the GLC2000 land cover map (Bartholome and Belward 2005). The average annual precipitation amount is shown in figure 1(c) and the average annual air temperature in figure 1(d). Both climatic variables were extracted from the Joint Research Centre of the European Commission (JRC) global meteo (ECMWF) database. Example profiles (presented in figure 5) were extracted for five land cover classes (decimal geographic latitude/longitude in parentheses): evergreen broadleaved forest (-44.2902, -12.9152), deciduous broadleaved forest (-43.2188, -13.5223), shrub cover (-42.4063, -16.0045), herbaceous cover (-44.6027, -15.2634) and crop land (-43.0402, -14.1027). 2.2
SPOT VGT time series
For the study, 10-day MVC-NDVI images from SPOT VGT (1 km spatial resolution) covering the time period from April 1998 to December 2008 were used. The images were pre-processed by the Flemish Institute for Technological Research (VITO) within a framework contract with the JRC using a consistent processing algorithm including geometric, radiometric, and atmospheric corrections. After smoothing, the 10 annual cycles between July 1998 and June 2008 were retained for analysis (i.e. 360 image layers).
3692
C. Atzberger and P. H. C. Eilers
70 W 0 S
60
50
40
70 W 0 S
(a)
10
10
20
20
60
50
(b)
Major Biomes
GLC 2000 Evergreen forest Deciduous forest Shrubland Cropland Herbaceous Bare
a Trop. subtr. moist forest
30
b Trop. subtr. dry forest c Trop. subtr. shrublands
30
Downloaded by [UB Bodenkultur Wien] at 01:17 29 June 2011
d Temperature grasslands e Flooded grasslands f Montane shrublands g Xeric shrublands
40
40 70 W 0 S
60
50
40
70 W 0 S
(c)
10
10
20
20
30
Annual 40
40
Precipitation < 1000 1000–1500 1500–2000 2000–2500 2500–6000
30
40
60
50
40
(d)
Temperature 25
Figure 1. Characteristics of the study region (South America) covered by the SPOT VGT time series: (a) major eco-regions (Olson et al. 2001), (b) broad land cover classes according to GLC2000 (Bartholome and Belward 2005), (c) average ECMWF annual precipitation amount, (d) average annual ECMWF air temperature.
2.3
Filtering of the SPOT VGT time series
To remove the negatively biased noise which is present in the NDVI time series, the Whittaker smoother was used (Atzberger and Eilers 2010). This filter has been made popular by Eilers (2003). The smoother fits a discrete series to discrete data and puts – similar to the work of Hermance (2007) and Hermance et al. (2007) – a penalty on the roughness of the smooth curve. The algorithm is extremely fast, gives continuous control over smoothness with only one parameter and interpolates automatically. It adapts well to boundaries (e.g. start and end of the time series) and does not assume periodicity such as Fourier filtering or wavelets. In addition, its fast cross-validation can be used for self-acting choice of the smoothing parameter. To keep the article
Effectiveness of smoothing algorithms
3693
concise, we report here only the basic formula. A more detailed description of the algorithm can be found in the original work Eilers (2003) and in Atzberger and Eilers (2010). A small toolbox of Matlab functions for Whittaker smoothing is available free of charge via the Internet at http://pubs.acs.org. Suppose a (noisy) series y, sampled at equal distances of length m is given. The aim is to fit a smooth series z to y. Then, two conflicting goals have to be balanced: (i) fidelity to the data and (ii) roughness of z. The smoother z is, the more it will deviate from y. A balanced combination of the two goals is the sum Q ¼ S þ kR with
X
(1)
ð yi z i Þ 2
(2)
ððzi zi1 Þ ðzi1 zi2 ÞÞ2 :
(3)
S¼
Downloaded by [UB Bodenkultur Wien] at 01:17 29 June 2011
i
R¼
X i
The lack of fit to the data, S is measured as the usual sum of squares of differences. The roughness of the smoothed curve, R is expressed here as second order differences. The number k is called smoothing parameter. The idea of penalized least squares is to find the series z that minimizes Q. The larger k is, the stronger the influence of R on the goal Q and the smoother z will be (at the cost of the fit of the data getting worse). In matrix notation we get a linear system of equations ðI þ kD’DÞz ¼ y
(4)
where I is the identity matrix and D is a matrix with m – 2 rows and m columns where each row contains the pattern 1–2 1, shifted such that di,i ¼ 1, di,iþ1 ¼ –2 and di,iþ2 ¼ 1 (all other elements of D are 0). To handle missing data (e.g. cloud flags), the missing elements of y are simply set to an arbitrary value, and a vector w of weights is introduced, with wi ¼ 0 for missing observations and wi ¼ 1 otherwise. We get the final system of equations that permits calculating the smoothed curve (z) ðW þ kD0 DÞz ¼ Wy
(5)
where W ¼ diag(w), a diagonal matrix with w on its diagonal. At the positions where y is missing, z is automatically and smoothly interpolated. This feature can also be used as an easy device for smoothing and detailed interpolation (e.g. to a daily time step), as well as for extrapolations/forecasts. One way of choosing a value for the smoothing parameter k is tuning it until a visually pleasing result is obtained. Using cross-validation a more objective choice can be made, described in Eilers (2003). This automatic cross-validation was used in the present study. To deal with the negatively biased noise of NDVI time series the original approach was slightly modified. The filtering ensures that the smoothed data approach the upper envelope of the original NDVI values. To achieve an upward biased fit, an iterative approach similar to the work of Chen et al. (2004) and Beck et al. (2006) was used. Matlab (Mathworks 2007) was used for smoothing the time series. To avoid overloading the memory, the data cube was divided in blocks of 200 200 pixels (each with 386 layers). The pixels were filtered sequentially. Using Matlab 7.5 this took 36 h
3694
C. Atzberger and P. H. C. Eilers
on an Intel Xeon CPU with two 2.67 GHz processors and 16 GB of RAM. To filter one (land) pixel thus took approximately 0.5 s.
Downloaded by [UB Bodenkultur Wien] at 01:17 29 June 2011
2.4
Variogram fitting
The original and filtered time series were submitted a geostatistical analysis to serve two main purposes: (1) to assess the SNR of the data before and after filter application and (2) to evaluate if the smoother preferentially recognizes (and subsequently corrects) low quality observations. Variogram fitting was done for non-overlapping clusters of 66 66 pixels. These clusters then become our ensemble of discrete variogram ‘cells’. (Note that the term ‘cell’, as used in this discussion, is a sub-cluster of 66 66 non-overlapping pixels.) Since more than 1 000 000 variograms had to be fitted, pre-computed look-up-tables (LUT) were used for estimating the variogram parameters. Each 10-day time bin and cell was processed independently. The detailed procedure is described in Atzberger and Eilers (2010). From the fitted variograms, the nugget variance, c0, was extracted. The SNR of the image cell was calculated from the nugget variance and the average NDVI of the cell, m (Asmat et al. 2007, Atkinson et al. 2007) m SNR ¼ pffiffiffiffiffi (6) c0 Note that the SNR calculated in this way has a spatial resolution of 66 km as we had to restrict the variogram analysis to non-overlapping cells. 3.
Candidate quality indicators
In the following, we will present several quality indicators, thought to be useful for characterizing the effectiveness of the smoothing process. The indicators are summarized in table 3 and will be briefly described and illustrated in the following sub-sections (sections 3.1–3.6). The examples are not to be understood as a closed list but should foster the development of additional quality indicators.
Table 3. Proposed quality indicators that could be helpful in characterizing the effectiveness of smoothing algorithms in the absence of ground reference measurements. Quality indicator
Verification based on
Increased temporal persistence of Z-score profiles after smoothing Positive correlation between noise estimate and cloudiness Increased class separability after filter application
plausibility check plausibility check distance metric
Reduced intra-class variability of pseudo-invariant targets after filtering Negative relation between noise estimate and SNR of unfiltered data Increased SNR after application of smoother
distance metric
Required ancillary data none
SNR
climatology of cloudiness land cover information location of invariant targets none
SNR
none
Effectiveness of smoothing algorithms NDVI profiles of temporally invariant targets
A simple way of evaluating the effectiveness of a smoothing algorithm is to analyse annual data of pseudo-invariant targets (figure 2). For the illustration, a pixel falling within the evergreen Atlantic forests of Brazil was selected (Rio Grande do Sul). The climate in this area shows only relatively small year-to-year variations and the intrayear LAI dynamic of these forests is known to be low. Due to undetected clouds and poor atmospheric conditions, the annual NDVI profiles derived from the original data (figure 2(a)) reveal strong scatter. Several very large drops in NDVI (0.1) are observed, which are not compatible with the gradual process of vegetation change in an evergreen forest. These oscillations must therefore be regarded as noise and removed (Chen et al. 2004). The application of the Whittaker smoother led to smooth NDVI profiles (figure 2(b)). As expected for such a target, the filtered curves fall now within a small NDVI range. By quantifying this intra-class homogeneity (e.g. coefficient of variation per dekad) one obtains a simple measure of the effectiveness of the smoothing compared to unfiltered data and/or alternative algorithms. Of course, such an approach relies on the presence of pseudo-invariant targets in the study area and to what extent the target-inherent annual NDVI profiles vary from year-to-year. 3.2
Temporal persistency of Z-score values
Time-series of NDVI follow an annual cycle of growth and decline as the index is related to vegetation density (Chen et al. 2004). This implies that the change of vegetation density (and hence NDVI) is a gradual process. This offers the possibility to use the temporal persistence of (filtered) time series as a quality indicator. The (a)
(b) 0.9 0.8 0.7 0.6
NDVI
Downloaded by [UB Bodenkultur Wien] at 01:17 29 June 2011
3.1
3695
Rio Grande do Sul Atlantic forests 29° 37′ S 50° 17′ W
0.5 0.4 0.3 0.2 Before filtering J
M
M
J Time
S
N
After filtering J
M
M
J Time
S
N
Figure 2. Illustration of the smoother performance on annual data of a pseudo-invariant target. For the illustration, a pixel within the (evergreen) Atlantic Forests (Rio Grande do Sul, Brazil) was selected. The displayed curves are annual data for the period 1999–2008 before filtering (a) and after application of the Whittaker smoother (b).
3696
C. Atzberger and P. H. C. Eilers 2003
3
2005
2004
Original Filtered 2
z-score
1
0
–1
Downloaded by [UB Bodenkultur Wien] at 01:17 29 June 2011
–2
–3 J
O
J
A
J
O
J
A J Time
O
J
A
J
O
J
A
Figure 3. Example temporal Z-score profile for a grassland pixel (latitude/longitude: -44.6027, -15.2634) within Minas Gerais (Brazil) for the period 2002 (July) to 2006 (June) before (dashed line) and after application of the Whittaker smoother (bold line).
temporal persistence can be described by the autocorrelation function. In the present study, we based our analysis on Z-score values ((NDVI – mNDVI)/sNDVI) which are widely used for monitoring vegetation anomalies (Ji and Peters 2003, Zhang et al. 2005). The Z-score profile of a randomly chosen grassland pixel is shown in figure 3. The Z-score values derived from the filtered data (bold line) show a clear pattern. The Z-score profile derived from the original data (dashed line) broadly follows the same pattern, but with many short-term fluctuations. Occasionally, a strong positive vegetation anomaly (Z-score . 1), followed by a negative anomaly (Z-score , -1), can be observed within only one time step (e.g. 10 days). Such an implausible behaviour is, for example, seen around April 2003 and most probably does not correspond to a real drop in vegetation activity. In the case of the filtered data, the noise is efficiently reduced resulting in curves with a much higher temporal persistency. To quantify the temporal persistency of the time series, we propose calculating the average (median) length of positive (Z-score . 1) and negative vegetation anomalies (Z-score , -1), as well as the typical duration of ‘normal’ vegetation conditions (-1 Z-score 1). The derived durations can then be assessed against existing knowledge about the meteorological forcing in the study region. To further illustrate this idea, consider the frequency distributions depicted in figure 4. The histograms summarize the results for the full scene and all image layers from 1998 to 2007. As expected from the high scatter of the original data (figure 3), the overwhelming majority (.95%) of all vegetation anomalies (positive as well as negative) last only one dekad (10 days) (figure 4(a)); a period certainly too short for being attributable to a mesoscale meteorological forcing. Similarly, the typical duration of ‘normal’ vegetation conditions (-1 Z-score 1) is also very short (2–4 dekads). This confirms former findings that large parts of the observed NDVI
Effectiveness of smoothing algorithms 1
z-score < –1 –1 200
Figure 9. Frequency distribution of the signal-to-noise ratio (SNR) of the original (white bars) and the filtered data (grey bars) derived from 10 years of SPOT VGT imagery (360 dekads) over South America at a resolution of 66 km. The median SNR is indicated in the graph.
3.6
SNR before and after filter application
A final proposition for the assessment of filter performance consists in comparing the SNR of the images before and after filtering. Under the assumption that the (geostatistically-derived) SNR is a valid indicator of image quality, the SNR should show a significant increase from unfiltered to filtered data, and possibly should also allow ranking different smoothers.
Effectiveness of smoothing algorithms 70 W 0 S
60
40 (a)
70 W 0 S
10
10
20
20
30
Downloaded by [UB Bodenkultur Wien] at 01:17 29 June 2011
50
SNR ratio < 50 50–100 100–150 150–200 Original data >200
40
60
3703 50
(b)
30
40
40
Filtered data
SNR ratio < 50 50–100 100–150 150–200 >200
Figure 10. Maps of the median of the signal-to-noise ratio (SNR) calculated over 10 years (360 dekads) for the original data (a) and after filter application (b). In the white areas the NDVI quality was insufficient for variogram fitting. The SNR was calculated for each cell (66 66 SPOT VGT pixel) and each time step separately.
With our database, application of the Whittaker smoother led to a significant increase of the estimated SNR (figure 9). Analysis showed that on average (all years and all image cells), the increase in the SNR amounts to 33 units (median: 23 units). This confirms the positive effect of the smoother which significantly reduced the nugget variance, while increasing the average signal. The increase of SNR was not distributed equally across the image. Instead, the maps displayed in figure 10 demonstrate that the increase in SNR was location dependent. The (cloudy) areas in the Amazon basin benefited most from the filtering: (1) because the nugget variance was strongly reduced and (2) the already high average NDVI (figure 6(a)) was further increased. 4.
Discussion
In the absence of reference ground-based observations, new filters are most often evaluated against one or two alternative techniques. The selection of the alternative techniques, however, may be biased by the wish to highlight the strength of the new filter. To avoid this bias, Hird and McDermid (2009) proposed testing filter on synthetic datasets with known noise characteristics. With the present article we propose some simple quality indicators that may be useful for evaluating the filter effectiveness on real data. To illustrate the proposed indicators, a large SPOT VGT time series covering South America was smoothed using the Whittaker filter. Variogram fitting was found very useful for deriving the SNR of the imagery before and after filter application. A statistical analysis of the resulting frequency distributions (e.g. median and other quantiles) permits a quantification of the filtering performance. By analysing the spatio-temporal patterns of the SNR (before and after filtering) one gains additional information on filter behaviour in different environmental settings.
Downloaded by [UB Bodenkultur Wien] at 01:17 29 June 2011
3704
C. Atzberger and P. H. C. Eilers
The geostatistically derived SNR can also be correlated with the estimated image noise. We propose verifying the existence of a negative relation between the SNR of the unfiltered image (i.e. the ‘image quality’) and the amount of noise (filter) estimated by the smoother (e.g. NDVIfiltered – NDVIoriginal). Using our test dataset such a negative relation was found. The observed negative relation by SNR and filter demonstrated that the filter identifies unfavourable observation conditions and takes larger actions on this low quality data. The observed negative relation between SNR and filter could also be valued in real-time filtering attempts. In such an approach one would use the SNR of the original data to ‘inform’ the smoother about the probable amount of noise in the image. This should be particularly useful for smoother suffering from strong boundary effects (e.g. the widely used Savitzky–Golay filter), but has not yet been investigated. A simple way of checking the plausibility of the estimated noise pattern (filter) is to compare the estimated noise against (globally) existing cloud climatologies (e.g. Mitchell and Jones 2005). For the analysed test dataset, consistent spatiotemporally patterns were observed, indicating that the smoother had to take (on average) larger actions in areas with higher cloudiness. Another plausibility check was proposed based on the average duration of vegetation anomalies. The lengths of these anomalies should be consistent with the (assumed known) mesoscale meteorological forcing of the area. Before filter application, the typical lengths of vegetation anomalies were far too short for being attributable to a meteorological forcing. Using metrics such as the JM distance (Schmidt and Skidmore 2003) it is possible to quantify the gain in class separability which can be attributed to a given smoother. This has been verified on temporal NDVI profiles and on NDVI metrics describing the seasonality of the data. The required land cover information for this approach is globally available (e.g. GLC2000, GlobCover, MODIS collection 5 global land cover). However, one has to keep in mind that the calculated distance metrics (and hence the final output of the filter evaluation) heavily depend on the accuracy of the available land cover information. Still, one of the simplest approaches for verifying noise reduction techniques is to analyse pseudo-invariant targets. If such a target exists in the region covered by the time series, the intra-class homogeneity can be used to assess the filter performance. Note that the results will be biased if the target-inherent NDVI profiles (e.g. measured under perfect measurement conditions) do not match the requirements for a pseudoinvariant target. 5.
Conclusions
The Whittaker smoother was used to illustrate some candidate indicators that may be useful for evaluating the effectiveness of smoothing algorithms in the absence of reference (ground-based) measurements. All indicators can be directly derived from the analysed time series. The required ancillary data (in particular land cover information and cloud climatology) are readily available for the entire globe. However, to use the two approaches based on the geostatistically-derived SNR (figures 8–10), one has to develop a computational solution that allows fitting large amounts of variograms. In the present study where more than 1 000 000 variograms had to be fitted (66 60 cells 386 times), we used for this purpose a look-up-table based approach described in detail in Atzberger and Eilers (2010).
Downloaded by [UB Bodenkultur Wien] at 01:17 29 June 2011
Effectiveness of smoothing algorithms
3705
For illustrating the various quality indicators, the present study relied on the Whittaker smoother (Eilers 2003). This smoother was preferred over alternative techniques listed in table 2 because it permits a rapid processing of large amounts of data (including incomplete time series with missing values), with the (sole) smoothing parameter being automatically optimized through a build-in cross-validation procedure. The smoother can be implemented very quickly in high level computing languages such as Matlab and is available free of charge via the Internet at http:// pubs.acs.org. Any other smoother could have been used as well. The main intention of the article was not to promote a specific smoother but to stimulate the use of the proposed candidate indicators with other noise reduction techniques. Ultimately, several smoothing algorithms should be compared on the same dataset and using the same set of quality indicators, but this was beyond the scope of the present study. Beside the proposed quality indicators, classical validation protocols are necessary to come to a decisive conclusion on what filter to use (at least for a given environmental setting). For this purpose, filtered time series should be included in ongoing ground validation exercises such as those described by Beck et al. (2007), Garrigues et al. (2008) and Fensholt et al. (2005). We also highly recommend continuing to evaluate smoother performances on synthetic datasets with known noise characteristics (Hird and McDermid 2009). Acknowledgements The authors would like to thank J. Hird (University of Calgary) and K. Richter (University of Naples) for their valuable comments. We also thank the two anonymous reviewers for their valuable comments/suggestions on how to improve the manuscript. References ANYAMBA, A. and EASTMAN, J.R., 1996, Interannual variability of NDVI over Africa and its relation to El Nin˜o/Southern Oscillation. International Journal of Remote Sensing, 17, pp. 2533–2548. ASMAT, A., ATKINSON, P.M. and FOODY, G.M., 2007, Image-based method for noise estimation in remotely sensed data. In Proceedings of SPIE – The International Society for Optical Engineering, 6748, art. no. 67480L. ATKINSON, P.M., SARGENT, I.M., FOODY, G.M. and WILLIAMS, J., 2007, Exploring the geostatistical method for estimating the signal-to-noise ratio of images. Photogrammetric Engineering and Remote Sensing, 73, pp. 841–850. ATZBERGER, C. and EILERS, P.H.C., 2010, A smoothed 1-km resolution NDVI time series (1998–2008) for vegetation studies in South America. International Journal of Digital Earth. DOI: 10.1080/17538947.2010.505664. ATZBERGER, C. and REMBOLD, F., 2009, Estimation of inter-annual winter crop area variation and spatial distribution with low resolution NDVI data by using neural nets trained on high resolution images. In Proceedings of SPIE, SPIE Europe Remote Sensing, Berlin, Germany. DOI: 10.1117/12.830007. AZZALI, S. and MENENTI, M., 2000, Mapping vegetation–soil–climate complexes in southern Africa using temporal Fourier analysis of NOAA-AVHRR NDVI data. International Journal of Remote Sensing, 21, pp. 973–996. BARTHOLOME´, E. and BELWARD, A.S., 2005, GLC2000: a new approach to global land cover mapping from earth observation data. International Journal of Remote Sensing, 26, pp. 1959–1977.
Downloaded by [UB Bodenkultur Wien] at 01:17 29 June 2011
3706
C. Atzberger and P. H. C. Eilers
BECK, P., ATZBERGER, C., HØGDA, K., JOHANSEN, B. and SKIDMORE, A., 2006, Improved monitoring of vegetation dynamics at very high latitudes: a new method using MODIS NDVI. Remote Sensing of Environment, 100, pp. 321–334. BECK, P.S.A., JO¨NSSON, P., HØGDA, K.A., KARLSEN, S.R., EKLUNDH, L. and SKIDMORE, A.K., 2007, A ground-validated NDVI dataset for monitoring vegetation dynamics and mapping phenology in Fennoscandia and the Kola peninsula. International Journal of Remote Sensing, 28, pp. 4311–4330. BERENS, P. and VELASCO, M.J., 2009, Circular Matlab Toolbox. Copyright (c) 2009, Philipp Berens. All rights reserved. BRADLEY, B., JACOB, R., HERMANCE, J. and MUSTARD, J., 2007, A curve fitting procedure to derive inter-annual phenologies from time series of noisy satellite NDVI data. Remote Sensing of Environment, 106, pp. 137–145. CHAPPELL, A., SEAQUIST, J.W. and EKLUNDH, L., 2001, Improving the estimation of noise from NOAA AVHRR NDVI for Africa using geostatistics. International Journal of Remote Sensing, 22, pp. 1067–1080. CHEN, J., JO¨NSSON, P., TAMURA, M., GU, Z., MATSUSHITA, B. and EKLUNDH, L., 2004, A simple method for reconstructing a high-quality NDVI time-series data set based on the Savitzky–Golay filter. Remote Sensing of Environment, 91, pp. 332–334. COOPS, N.C., WARING, R.H., WULDER, M.A., PIDGEON, A.M. and RADELOFF, V.C., 2009a, Bird diversity: a predictable function of satellite-derived estimates of seasonal variation in canopy light absorbance across the United States. Journal of Biogeography, 36, pp. 905–918. COOPS, N.C., WULDER, M.A. and IWANICKA, D., 2009b, Demonstration of a satellite-based index to monitor habitat at continental-scales. Ecological Indicators, 9, pp. 948–958. EILERS, P.H.C., 2003, A perfect smoother. Analytical Chemistry, 75, pp. 3299–3304. FENSHOLT, R., SANDHOLT, I. and STISEN, S., 2005, Evaluating MODIS, MERIS, and VEGETATION – vegetation indices using in situ measurements in a semiarid environment. IEEE Transactions on Geoscience and Remote Sensing, 44, pp. 1774–1786. FULLER, D.O., 1998, Trends in NDVI time series and their relation to rangeland and crop production in Senegal, 1987–1993. International Journal of Remote Sensing, 19, pp. 2013–2018. GARRIGUES, S., LACAZE, R., BARET, F., MORISETTE, J.T., WEISS, M., NICKESON, J.E., FERNANDES, R. and YANG, W., 2008, Validation and intercomparison of global Leaf Area Index products derived from remote sensing data. Journal of Geophysical Research G: Biogeosciences, 113, art. no. G02028. GEERKEN, R.A., 2009, An algorithm to classify and monitor seasonal variations in vegetation phenologies and their inter-annual change. ISPRS Journal of Photogrammetry and Remote Sensing, 64, pp. 422–431. GOWARD, S.N., MARKHAM, B., DYE, D.G., DULANEY, W. and YANG, J., 1991, Normalized difference vegetation index measurements from the advanced very high resolution radiometer. Remote Sensing of Environment, 35, pp. 257–277. HERMANCE, J.F., 2007, Stabilizing high-order, non-classical harmonic analysis of NDVI data for average annual models by damping model roughness. International Journal of Remote Sensing, 28, pp. 2801–2819. HERMANCE, J.F., JACOB, R.W., BRADLEY, B.B. and MUSTARD, J.F., 2007, Extracting phenological signals from multi-year AVHRR NDVI time series: framework for applying highorder annual splines with roughness damping. IEEE Transactions of Geoscience and Remote Sensing, 45, pp. 3264–3276. HIRD, J.H. and MCDERMID, G.J., 2009, Noise reduction of NDVI time series: an empirical comparison of selected techniques. Remote Sensing of Environment, 113, pp. 248–258. HOLBEN, B.N., 1986, Characteristics of maximum-value composite images from temporal AVHRR data. International Journal of Remote Sensing, 7, pp. 1417–1434. HURLBERT, A.H. and HASKELL, J.P., 2003, The effect of energy and seasonality on avian species richness and community composition. American Naturalist, 161, pp. 83–97.
Downloaded by [UB Bodenkultur Wien] at 01:17 29 June 2011
Effectiveness of smoothing algorithms
3707
JAKUBAUSKAS, M.E., LEGATES, D.R. and KASTENS, J.H., 2001, Harmonic analysis of time series AVHRR NDVI data. Photogrammetric Engineering and Remote Sensing, 67, pp. 461–470. JI, L. and PETERS, A.J., 2003, Assessing vegetation response to drought in the northern Great Plains using vegetation and drought indices. Remote Sensing of Environment, 87, pp. 85–98. JO¨NSSON, P. and EKLUNDH, L., 2002, Seasonality extraction by function-fitting to time series of satellite sensor data. IEEE Transactions on Geoscience and Remote Sensing, 40, pp. 1824–1832. JO¨NSSON, P. and EKLUNDH, L., 2004, TIMESAT – a program for analyzing time-series of satellite sensor data. Computers and Geoscience, 30, pp. 833–845. JUSTICE, C., BELWARD, A., MORISETTE, J., LEWIS, P., PRIVETTE, J. and BARET, F., 2000, Developments in the ‘validation’ of satellite sensor products for the study of the land surface. International Journal of Remote Sensing, 21, pp. 3383–3390. JUSTICE, C.O., TOWNSHEND, J.R.G., HOLBEN, B.N. and TUCKER, C.J., 1985, Analysis of the phenology of global vegetation using meteorological satellite data. International Journal of Remote Sensing, 6, pp. 1272–1318. KERR, J.T. and OSTROVSKY, M., 2003, From space to species: ecological applications for remote sensing. Trends in Ecology & Evolution, 18, pp. 299–305. KOGAN, F.N., 2000, Satellite-observed sensitivity of world land ecosystems to El Nin˜o/La Nin˜a. Remote Sensing of Environment, 74, pp. 445–462. LI, Z. and KAFATOS, M., 2000, Interannual variability of vegetation in the United States and its relation to El Nino/Southern Oscillation. Remote Sensing of Environment, 71, pp. 239–247. LIU, W.T. and NEGRO´N JUA´REZ, R.I., 2001, ENSO drought onset prediction in northeast Brazil using NDVI. International Journal of Remote Sensing, 22, pp. 3483–3501. LOVELL, J.L. and GRAETZ, R.D., 2001, Filtering Pathfinder AVHRR Land NDVI data for Australia. International Journal of Remote Sensing, 22, pp. 2649–2654. MA, M., and VEROUSTRAETE, F., 2006, Reconstructing pathfinder AVHRR land NDVI timeseries data for the Northwest of China. Advances in Space Research, 37, pp. 835–840. MAIGNAN, F., BRE´ON, F.-M., BACOUR, C., DEMARTY, J. and POIRSON, A., 2008, Interannual vegetation phenology estimates from global AVHRR measurements. Comparison with in situ data and applications. Remote Sensing of Environment, 112, pp. 496–505. MARTI´NEZ, B. and GILABERT, M.A., 2009, Vegetation dynamics from NDVI time series analysis using the wavelet transform. Remote Sensing of Environment, 113, pp. 1823–1842. MATHWORKS, 2007, Matlab, the language of technical computing. Natick, MA: Mathworks, Inc. MCCLOY, K.R. and LUCHT, W., 2004, Comparative evaluation of seasonal patterns in long time series of satellite image data and simulations of global vegetation model. IEEE Transactions on Geoscience and Remote Sensing, 42, pp. 140–153. MCVICAR, T.R. and JUPP, D.L.B., 1998, The current and potential operational uses of remote sensing to aid decisions on drought exceptional circumstances in Australia: a review. Agricultural Systems, 57, pp. 399–468. MITCHELL, T.D. and JONES, P.D., 2005, An improved method of constructing a database of monthly climate observations and associated high-resolution grids. International Journal of Climatology, 25, pp. 693–712. MOODY, A. and JOHNSON, D., 2001, Land-surface phenologies from AVHRR using the discrete Fourier transform. Remote Sensing of Environment, 75, pp. 305–323. MOULIN, S., KERGOAT, L., VIOVY, N. and DEDIEU, G., 1997, Global-scale assessment of vegetation phenology using NOAA/AVHRR satellite measurements. Journal of Climate, 10, pp. 1154–1170. OLSON, D.M., DINERSTEIN, E., WIKRAMANAYAKE, E.D., BURGESS, N.D., POWELL, G.V.N., UNDERWOOD, E.C., D’AMICO, J.A. and KASSEM, K.R., 2001, Terrestrial ecoregions of the world: a new map of life on Earth. BioScience, 51, pp. 933–938.
Downloaded by [UB Bodenkultur Wien] at 01:17 29 June 2011
3708
C. Atzberger and P. H. C. Eilers
PARUELO, J.M., JOBBA´GY, E.G. and SALA, O.E., 2001, Current distribution of ecosystem functional types in temperate South America. Ecosystems, 4, pp. 683–698. PETTORELLI, N., VIK, J.O., MYSTERUD, A., GAILLARD, J.-M., TUCKER, C.J. and STENSETH, N.CHR., 2005, Using the satellite-derived NDVI to assess ecological responses to environmental change. Trends in Ecology & Evolution, 20, pp. 503–510. REED, B.C., BROWN, J.F., VANDERZEE, D., LOVELAND, T.R., MERCHANT, J.W. and OHLEN, D.O., 1994, Measuring phenological variability from satellite imagery. Journal of Vegetation Science, 5, pp. 703–714. ROERINK, G., MENENTI, M. and VERHOEF, W., 2000, Reconstructing cloudfree NDVI composites using Fourier analysis of time series. International Journal of Remote Sensing, 21, pp. 1911–1917. SAKAMOTO, T., YOKOZAWA, M., TORITANI, H., SHIBAYAMA, M., ISHITSUKA, N. and OHNO, H., 2005, A crop phenology detection method using time-series MODIS data. Remote Sensing of Environment, 96, pp. 366–374. SCHMIDT, K.S. and SKIDMORE, A.K., 2003, Spectral discrimination of vegetation types in a coastal wetland. Remote Sensing of Environment, 85, pp. 92–108. SCHNABEL, S.K. and EILERS, P.H.C., 2009, Optimal expectile smoothing. Computational Statistics and Data Analysis, 53, pp. 4168–4177. SHABANOV, N.V., ZHOU, L., KNYAZIKHIN, Y., MYNENI, R.B. and TUCKER, C.J., 2002, Analysis of interannual changes in northern vegetation activity observed in AVHRR data from 1981 to 1994. IEEE Transactions on Geoscience and Remote Sensing, 40, pp. 115–130. SOUDANI, K., LE MAIRE, G., DUFREˆNE, E., FRANC¸OIS, C., DELPIERRE, N., ULRICH, E. and CECCHINI, S., 2008, Evaluation of the onset of green-up in temperate deciduous broadleaf forests derived from Moderate Resolution Imaging Spectroradiometer (MODIS) data. Remote Sensing of Environment, 112, pp. 2643–2655. STO¨CKLI, R. and VIDALE, P.L., 2004, European plant phenology and climate as seen in a 20-year AVHRR land-surface parameter dataset. International Journal of Remote Sensing, 25, pp. 3303–3330. SWETS, D., REED, B., ROWLAND, J. and MARKO, S., 1999, A weighted least-squares approach to temporal smoothing of NDVI. In 1999 ASPRS Annual Conference, From Image to Information, Portland, Oregon, 17–21 May 1999 (Bethesda, MD: American Society for Photogrammetry and Remote Sensing), pp. 526–536. TUCKER, C.J., SLAYBACK, D.A., PINZON, J.E., LOS, S.O., MYNENI, R.B. and TAYLOR, M.G., 2001, Higher northern latitude normalized difference vegetation index and growing season trends from 1982 to 1999. International Journal of Biometeorology, 45, pp. 184–190. UDELHOVEN, T., VAN DER LINDEN, S., WASKE, B., STELLMES, M. and HOFFMANN, L., 2009, Hypertemporal classification of large areas using decision fusion. IEEE Geoscience and Remote Sensing Letters, 6, art. no. 5075593, pp. 592–596. VELLEMAN, P., 1980, Definition and comparison of robust nonlinear data smoothing algorithms. Journal of the American Statistical Association, 75, pp. 609–615. VIOVY, N., ARINO, O. and BELWARD, A. S., 1992, The Best Index Slope Extraction (BISE): A method for reducing noise in NDVI time-series. International Journal of Remote Sensing, 13, pp. 1585–1590. WHITE, M.A., THORNTON, P.E. and RUNNING, S.W., 1997, A continental phenology model for monitoring vegetation responses to interannual climatic variability. Global Biogeochemical Cycles, 11, pp. 217–234. WOLTER, K. and TIMLIN, M.S., 1998, Measuring the strength of ENSO events – how does 1997/ 98 rank? Weather, 53, pp. 315–324. XIAO, X., BOLES, S., LIU, J., ZHUANG, D. and LIU, M., 2002, Characterization of forest types in Northeastern China, using multi-temporal SPOT-4 VEGETATION sensor data. Remote Sensing of Environment, 82, pp. 335–348. YU, F., PRICE, K.P., ELLIS, J. and SHI, P., 2003, Response of seasonal vegetation development to climatic variations in eastern central Asia. Remote Sensing of Environment, 87, pp. 42–54.
Effectiveness of smoothing algorithms
3709
Downloaded by [UB Bodenkultur Wien] at 01:17 29 June 2011
ZHANG, P., ANDERSON, B., TAN, B., HUANG, D. and MYNENI, R., 2005, Potential monitoring of crop production using a satellite-based Climate-Variability Impact Index. Agricultural and Forest Meteorology, 132, pp. 344–358. ZHANG, X., FRIEDL, M.A., SCHAAF, C.B., STRAHLER, A.H., HODGES, J.C.F., GAO, F., REED, B.C. and HUETE, A., 2003, Monitoring vegetation phenology using MODIS. Remote Sensing of Environment, 84, pp. 471–475. ZHOU, L., TUCKER, C.J., KAUFMANN, R.K., SLAYBACK, D., SHABANOV, N.V. and MYNENI, R.B., 2001, Variations in northern vegetation activity inferred from satellite data of vegetation index during 1981 to 1999. Journal of Geophysical Research D: Atmospheres, 106, pp. 20069–20083.
