an evaluation of ecotope classification using

AN EVALUATION OF ECOTOPE CLASSIFICATION USING SUPERRESOLUTION IMAGES DERIVED FROM CHRIS/PROBA DATA Jonathan Cheung-Wai Chan1, Jianglin Ma1, Pieter Kempeneers2, Frank Canters1, Jeroen Vanden Borre3 and Desiré Paelinckx3 1

Cartography & GIS Research Group, Department of Geography, Vrije Universiteit Brussel, Pleinlaan 2, 1050 Brussels, Belgium 2 Centre of Expertise in Remote Sensing and Atmospheric Processes (TAP), Flemish Institute for Technological Research (VITO), Boeretang 200, 2400 Mol, Belgium 3 Research Institute for Nature and Forest (INBO), Kliniekstraat 25, 1070 Brussels, Belgium ABSTRACT This paper discusses the application of superresolution (SR) image reconstruction on multi-angle Chris/Proba images. The goal is to increase the spatial resolution of Chris/Proba images, with 18 bands from 0.4-1.0 ȝm in the hope to obtain a better ecotope classification. The SR approach chosen for this study is Total Variation [1], an iterative method which models the relationship between the desired high resolution image and the low resolution images, with the following components: a subsampling factor, a point spread function, an estimated rotation and shift, and a regularization term. This regularization approach is fast in implementation and flexible in handling noise. Efficient gradient descent methods can be used to find the desired high resolution image. The spatial resolution of the original image is improved from 25m to 12m using Total Variation. Subjective assessment through visual interpretation shows substantial improvement in detail. A tree-based ensemble classifier Random Forest [2] is used for the classification of 18 ecotopes. Overall accuracy shows a 10% increase with the SR derived Chris/Proba images, compared with a classification based on the original imagery. Our results demonstrate that SR methods can improve spatial detail of multi-angle images, and subsequently classification accuracy. Index Terms— Superresolution, Chris/Proba, ecotope classification, Random Forest 1. INTRODUCTION An efficient reportage of habitat status requires repeatable, stable and sustainable methods for long-term monitoring. In order to obtain adequate information on the diverse ecological and biological conditions within habitats, detailed land-cover classification at the ecotope level is needed. Ecotopes are landscape features with distinct ecological properties. The challenge of mapping ecotopes is that they

978-1-4244-2808-3/08/$25.00 ©2008 IEEE

have many classes and variants, making it very difficult, if not impossible, to classify them using conventional broadband multi-spectral data. Previous reports show that hyperspectral data with rich spectral information content offer interesting opportunities for detailed classification of ecotopes [3][4]. For an operational monitoring of environmental status using remotely-sensed data, not only stable and timely acquisitions are important, large areal coverage is also needed to provide mosaic snapshots of the environment. While space-borne images cover large areas, the spatial resolution of the imagery is often too coarse for regional-scale to local-scale applications. Airborne acquisitions can obtain data of higher spatial resolution, but they are not cost-effective. Today, it is still not realistic to suggest airborne hyperspectral images for a nation-wide survey, mainly because of prohibitive costs. Recently, remote sensing images are acquired from different angles for spectral calibration pertaining to view angle effects on surface reflectance [5]. Chris/Proba represents one of the latest developments in multi-angle remote sensing, providing spectral information of the same scene from five different angles. The additional information provides a means to quantify the uncertainty in variable retrieval from remote sensing data. Another possible application of multiangle acquisition, perhaps not anticipated as an initial objective of multi-angle observation, is to enhance the spatial quality of the images by using reconstruction methods widely known as superresolution algorithms [6]. Superresolution (SR) is a technique developed in the machine vision and image processing communities to enhance image resolution using images acquired at a subpixel shift. As such, SR techniques can be applied to multiangle images. The potential of SR methods for Chris/Proba images has not been investigated before. It nevertheless is an important subject because it could improve the applicability of space-borne multi-angular data. In this study, we investigate the benefit of SR image reconstruction for detailed classification of ecotopes using Chris/Proba data.

III - 322

IGARSS 2008

Fig 1. A block diagram to show the model relating the LR images to HR images

2. BACKGROUND OF SUPERRESOLUTION Superresolution refers to the reconstruction methods that can be applied to obtain an image with higher spatial resolution through the use of several lower-resolution (LR) images. The main objective is to achieve the best image quality possible from several LR images. However, not all LR images are useful for SR. The application of SR algorithms is possible only if the LR images satisfy the following conditions: the LR images are sub-sampled, that is to say aliases exist, and the images have sub-pixel shifts.

reconstruction approach, the projection onto convex sets approach, hybrid approaches and others [6]. It is important to note that SR can also be applied on a single-frame LR image [7]. However, in such case, it is more often referred to as image scaling, interpolation, zooming and enlargement. Superresolution reconstruction is an active subject and has already been applied to remote sensing images such as Landsat [8][9], SPOT [10], Quickbird [9], and multi-looking thermal data [11]. In this paper, we focus on its application to multi-angle Chris/Proba images. 3. DATA AND STUDY AREA

SR techniques are closely related to the problems of image restoration and image interpolation. The purpose of image restoration is to recover a degraded image without changing the dimension of the image. Based on similar theories as image restoration, SR can be considered as a second generation of image restoration techniques which also change image dimension. Image interpolation techniques can be used to increase the size of an image. However, the quality of a single LR image is limited and interpolation based on an undersampled image does not allow recovering the lost high-frequency information. Hence multiple observations of the same scene are needed. An important step in SR is to provide a model of the relationship between the original high resolution scene and a set of LR images of that scene. If we have N LR images and Yk is the kth LR image and the HR image is X (X and Yk are their matrix form), then this relationship can be formulated as follows:

Y k = Dk Bk M k X + V k

k = 1,…, N

(1)

where Mk is the warp matrix that represents the shift and rotation of the LR images. Bk is the blur matrix that represents the blurring effects that occur during the acquisition and Dk is the subsampling or decimation factor. Vk is the ordered noise vector. This model can be illustrated as a block diagram in Fig 1. The main procedure of SR consists of three steps: registration, interpolation and restoration. Based on the model given above, different approaches can be formulated. Conventional methods include the non-uniform interpolation approach, the frequency domain approach, the regularization

A Chris/Proba image was acquired on 18th Sept, 2007 using Mode 3 Land Channels, which are designed for land-cover classification in general. Data was registered in eighteen bands between 0.4-1.0 ȝm at 5 different angles [-55º, -36 º, 0 º, +36º, +55º]. The five observations were used as LR inputs for the SR reconstruction. Figure 2 shows the original Chris image acquired at nadir. The data were preprocessed using the de-stripping program implemented under BEAM, a pre-processing package for Chris data (http://www.brockmann-consult.de/beam/downloads.html). Then the multi-angular images were hand-registered and a subset that overlapped with our study site was masked out for our experiments. Our case study is ecotope mapping based on a classification scheme currently applied in Belgium for producing the biological valuation map. Within our study area at the Kalmthout site, 18 ecotopes were identified (Table 1). The number of samples used for the training and validation is listed in Table 1. 4. METHODOLOGY We applied different off-the-shelf SR methods on the Chris/Proba images. The method we have found to provide the best results is Total Variation [1], which can be described as a regularization approach. Regularization is not only a means to pick up a stable solution, but also helps to improve the rate of convergence. Our implementation basically follows the methodology described in [1] where an L1 norm of total variation is implemented with a fast regularization component that substantially reduces computational loadings.

III - 323

Ecotope Classes 1 2 3 4 5 6 7 8

Arable field Bare sand Deciduous forest – Beech Deciduous forest – Mixed oak-birch forest Coniferous forest – Scots pine Coniferous forest – exotic conifers Grassland - temporary Grassland – permanent agriculturally improved, species poor Grassland – permanent agriculturally improved, species rich Grassland – permanent, semi-natural Heathland – wet, open Heathland – dry, open Heathland – dry, with trees Heathland – grass encroached, open, dry Heathland – grass encroached, open, wet Heathland – grass encroached, open Urban Water - oligotrophic

9 10 11 12 13 14 15 16 17 18

Samples in Chris/Proba 735 84 17 279 1922 186 445 885

Class accuracy 88.9 89.7 33.3 55.5 84.8 58.8 69.3 75.3

Samples in the SR reconstructed image 2712 267 49 977 7478 647 1685 3296

Class accuracy 93.7 96.5 81.8 76.9 88.5 94.1 83.6 85.5

115

44.9

338

84.8

21 388 242 408 451 495 748 164 337

0.0 61.9 45.9 53.6 55.8 55.2 60.8 94.6 99.4

65 1422 839 1439 1680 1821 2827 536 1288

79.2 72.8 76.4 78.6 73.4 74.8 73.0 98.7 99.8

Table 1. Description of the ecotopes. The number of samples used for training and validation before and after the application of SR reconstruction. Class accuracies before and after SR reconstruction is also shown. Using the notation in [1], the formulation of the problem is as follows:

 ªN X = ArgMin «¦ Dk Bk M k X − Y k X ¬ k =1 P

P

+ λ ¦¦ α l =0 m=0

m+l

1

º X − S S X 1» ¼ l x

m y

5. RESULTS AND DISCUSSION (2)

where X is the desired HR image and Yk are the LR images. The three components used to describe the relationship between the HR image and the LR images are D, B and M which represent the sampling factor, the blurring effects, and l

the warping matrix respectively. S x corresponds to an edge image that shows the shift of X by l pixels in the horizontal direction, and

under a 7 x 7 window. The downsampling factor D was set at 2. The rest of the parameters are empirically defined and for this experiment were set as follows: Į = 0.6, Ȝ = 0.02, ȕ = 0.004.

S ym corresponds to one that shows the shift by

m pixels in the vertical direction. The solution for the problem is obtained by the steepest descent method:

  ­N X n+1 = X n − β ®¦ M kT BkT DkT sign( Dk Bk M k Xˆ n − Y k ) ¯ k =1 P P   ½ + λ ¦ ¦ α m+l [ I − S y−m S x−l ]sign( X n − S xl S ym X n ) ¾ l = − P m =0 ¿

(3)

This method first uses Fourier transformed SR images to estimate the rotation and shift which are the parameters for Mk. Then Bk was approximated by a Point Spread Function defined by a Gaussian distribution with a variance of 1.6

Our experiments showed that common SR algorithms cannot be applied successfully without a manual registration process. This probably is due to the large geometric distortion and variation in irradiance caused by the different acquisition angles. Sometimes this also implies huge differences in atmospheric conditions caused by moisture content and haze. Multi-angle remote sensing adds considerable complexity to the SR problem. In additions to geometric distortion due to different viewing angles, pixel resolution is not constant over the image and pixels are subjected to different blurring and noise effects. These problems are partially solved by registration. Initial experiments showed that while each of the five observations provides different information, the use of all images does not generate good SR results. In [11], it is stated that acquisition angles over 40º may pose problems for SR methods. Our results confirm this observation and our final HR image was obtained by using images taken at three angles only: the nadir, -36º and +36º. The derived SR images show apparent improvement in detail. Figure 2 shows the SR reconstructed image of band 18. The zoom illustrates clear improvements in image details for an increase in spatial resolution from 25 m to 12m. For the ecotope classification we used the powerful tree-

III - 324

Original image (25m) SR image (12m) zooms Fig 2. SR derived Chris/Proba image, band 18. The top zoom is from the SR image, the bottom zoom from the original image. based ensemble classifier, Random Forest. Due to the small number of training samples for certain classes, we validated our results using out-of-bag estimates within Random Forest [2]. The number of samples in the HR image increased after the SR reconstruction (Table 1). Compared to the original Chris/Proba data which is 25m in spatial resolution, the classification accuracy obtained with the SR image is about 10 percent higher (84%) for an increase of the pixel resolution to 12m. Accuracy improvements for certain classes are quite substantial. Though the accuracy would be a bit lower with a blind test set, which was not available in this experiment, improvement through the use of SR images may still be expected to be substantial, which is very encouraging. Visual assessment of the SR derived image clearly shows enhanced quality. Moreover, our first classification results show that the application of SR algorithms to space-borne multi-angle images might be able to improve the scale of the mapping as well as the classification accuracy. There are, however, many aspects of SR techniques that need to be investigated in more detail, e.g. the influence of atmospheric correction has not been thoroughly dealt with in this study due to the lack of ground measurements, parameter setting is still complicated and it is time-consuming to find the best combination of parameters, most algorithms are computationally heavy, etc. Proper evaluation of SR methods is also a problem since a high resolution data set with corresponding wavebands, and ideally acquired during the overpass of Chris/Proba, is mostly unavailable. More research is therefore needed to enhance our understanding of the subject as it may have important implications for the successful utilization of space-borne multi-angle hyperspectral data for reporting on the status of our environment.

6. REFERENCES [1] S. Farsiu, D. Robinson, and M. Elad, “Fast and robust multiframe super-resolution”, IEEE Trans. Image Process., vol. 13, no. 10, pp. 1327-1344. October 2004. [2] L. Breiman, “Random Forests”, Machine Learning, 45, pp. 532. 2001. [3] J.C.-W. Chan and F. Canters, “Ensemble classifiers for hyperspectral classification”, on CD-ROM, Proceedings of the 5th EARSeL SIGIS, Innovation in Environment Research, Bruges, 2325 April, 2007. [4] J.C.-W. Chan and D. Paelinckx, “Evaluation of Random Forest and Adaboost tree-based ensemble classification and spectral band selection for ecotope mapping using airborne hyperspectral imagery”, Remote Sensing of Environment, 112, pp. 2999-3011. 2008. [5] J. Verrelst, M.E. Schaepman, B. Koetz, and M. Kneubühler, “Angular sensitivity analysis of vegetation indices derived from CHRIS/PROBA data”, Remote Sensing of Environment, 112, pp. 2341-2353. 2008. [6] S.C. Park, M.K. Park, and M.G. Kang, “Super-resolution image reconstruction: A technical overview”, IEEE Signal Process. Mag., vol. 20, no. 3, pp. 21-36. May, 2003. [7] J.D. van Ouwerkerk, “Image super-resolution survey”, Image and Vision Computing, v. 24, pp. 1039-1052. 2006. [8] R.Y. Tsai, and T.S. Huang, “Multipleframe image restoration and registration”, in Advances in Computer Vision and Image Processing. Greenwish, CT: JAI Press Inc., pp. 317-339. 1984. [9] M.T. Merino and J. Núñez, “Super-resolution of remotely sensed images with variable-pixel linear reconstruction”, IEEE Trans. Geosci. and Remote Sensing, v. 45, pp. 1446-1457. 2007. [10] C. Latry, and B. Rouge. “Super resolution: quincunx sampling and fusion processing”, Proc. Int. Geoscience and Remote Sensing Symposium (IGARSS), Toulouse, France, pp.315–317. 2003. [11] A. Galbraith, J. Theiler, K. Thome, and R. Ziolkowski, “Resolution enhancement of multi-look imagery for the multispectral thermal imager”, IEEE Trans. Geosci. and Remote Sensing, v. 43, pp. 1964-1977. 2005.

III - 325