SVM for hyperspectral images classification based on 3D spectral signature Karim Saheb Ettabaa1, 2, Med Ali Hamdi3, Rafika ben Salem3 1
Laboratoire de Recherche en Informatique Arabisée et Documentique Intégrée (R.I.AD.I) Ecole Nationale des Sciences de l’Informatique. Campus Universitaire de Manouba, 2010 Manouba, Tunis, Tunisie Email:
[email protected] 2 Télécom Bretagne, laboratoire ITI Technopôle Brest Iroise CS 83818, 29238 Brest Cedex France 3
Laboratoire des Matériaux, Molécules et Applications IPEST, université de carthage, Carthage University, Tunisia
Email:
[email protected] Email:
[email protected]
Abstract— Hyperspectral imaging sensors acquire images in hundreds of continuous narrow spectral bands spanning the visible to infrared spectrum which led to obtain hyperspectral image with high spectral resolution. Thus each object presented in the image can be identified from their spectral response. The classification of multi-temporal hyperspectral image is a challenge task due to the problem of spectral variation over the time. In fact, many factors can affect the spectral signature of object like weather and climatic effects, so two images taken on the same area but at different times and under different conditions can lead to different spectral signatures for the same objects. This observation has fostered the idea of adopting 3D representation of spectral signature to classify multi-temporal hyperspectral image. The main objective of this representation is to have for each object a compact model which illustrate their spectral variation over the time, it represent the variation of reflectance as a function of time and spectral waveband. In this paper, we propose a new approach for multi-temporal hyperspectral image classification based on 3D spectral signature to solve the problem of spectral variation. This approach consist, foremost, to represent each pixel of classified image by a 3D spectral signature after the application of the powerful 3D modeling method "Non-uniform Rational Basis Spline" (NURBS), after, to apply local shape descriptor "spherical harmonic decomposition" to extract spectral features from each 3D spectral signature and, finally, to classify the image by means of supervised classifier "Support Vector Machines (SVMs)" with Radial Basis Function (RBF) kernel. To evaluate this approach, we used a series of multi-temporal hyperspectral "hyperion" images. Keywords— Multi-temporal hyperspectral images; NURBS; shape descriptor; SVMs.
I.
INTRODUCTION
Multi-temporal images processing is becoming more and more important in monitoring earth surface. The large collection of remote sensing imagery makes possible analyzing and interpretation of spatio-temporal pattern of environmental elements and impact of human activities. However, the problems of multi-temporal image classification are highly relevant in most remote sensing study. Most existing multitemporal classification methods use the spectral information
alone, ignoring the spatial and temporal correlation between images acquired from different dates. The variation of the spectral data over the time affects the performance of hyperspectral image classification. In facts, the spectral response is very sensitive to many factors and it is affected by several phenomena such as weather and climatic effects. So, two images taken on the same area but at different times and under different conditions can lead to different spectral signatures for the same objects witch lead to false classification. Most classification methods process each pixel alone without taking into account the spatial and temporal correlation between images acquired at different dates. To address these problems, a range of image processing techniques has been greatly expanded. Maximum likelihood was commonly used for classification and analysis of remote sensing data. Several other studies have demonstrated the usefulness of neural networks [1] and support vector machines [2]. In [3], authors proposed a new classification system of multi-temporal hyperspectral image to solve the problem of spectral variation, in fact, they presented a novel representation of spectral signature "3D-spectral signature" and they used different shape descriptors to classify the image. In our work, we tried to exploit the new representation of spectral signature proposed in [3] to develop a new classification system of multi-temporal hyperspectral images. This system is based on the use of the local shape descriptor "spherical harmonic decomposition" to characterize the 3Dspectral signatures and the supervised classifier "SVMs". The remainder of this paper is organized as fellows. Section II presents the 3D modeling of spectral signature. Section III briefly outlines the SVMs classifiers used to classify the image. Section IV presents the proposed approach. Section V reports classification results handled by the proposed classification system. Section VI draws the conclusion of this paper. II.
3D MODELING OF SPECTRAL SIGNATURE
Usually, the spectral signature represent the variation of reflectance over wavelength λ (1), 3D-spectral signature represent the variation of the reflectance as a function of the
wavelength λ and the time t (2). To have a compact 3D shape of spectral signature without losing information, we need to apply different methods of surface modeling (fig .1). Reflectance = f (λ)
(1)
Reflectance = f (λ , t)
(2)
The 3D representation of spectral signature is rarely proposed in the literature, in fact, the number of articles published in this context is limited. In [3], the authors proposed to use the Delaunay triangulation method to represent the multi-temporal spectral signature. In [4], a 2D interpolation methods was applied to generate the spectro-temporal reflectance surface like spline and minimum curvature.
||w|| is the Euclidean norm of w ( fig.2). The decision function is defined by: f ( x ) = sign ((w ⋅ x ) + b) (3) For the linearly separable case, the support vector algorithm simply looks for the separating hyperplane with largest margin. This can be formulated as follows: suppose that all the training data satisfy the following constraints:
( w ⋅ x i ) + b ≥ 1 fo r y i = + 1
(4 )
( w ⋅ x i ) + b ≤ 1 fo r y i = − 1
(5 )
These can be combined into one set of inequalities:
yi (w ⋅ xi + b) ≥ +1
(6)
The optimal hyperplane is the hyperplane that maximizes the margin between the samples and the separating hyper-plane which is equal to 2/||w||.
Fig. 2. A linear SVM in a two-dimensional space.
Fig. 1. Representation of 3D modeling of spectral signature of olive, (a) 5 multi-temporal spectral signature of olive, (b)3D spectral signature of olive.
This problem can be solved by the use of Lagrange multipliers; in this case, decision function is defined by: n
III.
f ( x ) = sign ( ∑ α i yi ( x ⋅ xi ) + b )
SVM
Support vector machine (SVM) is a supervised classifier, it has been proposed by Vapnik [5]. This classifier has been introduced to solve two-class pattern recognition problems using the Structural Risk Minimization principle. Given a training set in a vector space, this method finds the best decision hyper-plane that separates a set of positive examples from a set of negative examples with maximum margin. A. Optimal hyper-plane in the linear separable case Considering the training data (xi, yi)
1≤ i ≤ n,
yi ∈ {−1, 1} ,
xi ∈ R d , The points x which belongs the hyperplane satisfy (w ⋅ x) + b = 0 , where w is normal to the hyperplane, where w is normal to the hyper-plane, b/||w|| is the perpendicular distance from the hyperplane to the origin, and
(7)
i =1
b satisfy their conditions :
α i [ yi (w ⋅ xi + b) − 1] = 0 ∀ i = 1...n , αi ≠ 0 are the support
vectors.
B. Optimal hyper-plane in the nonlinear separable case To handle nonlinearly separable classes, a nonlinear transformation ϕ is used to map the original data points into a higher dimensional space, in which data points are linearly separable, called redescription space (Fig. 3).
regression and a probabilistic pixel-wise support vector machine.
Fig. 3. Linear discrimination in redescription space.
In this case, the equation of the separator hyperplane is h(x)= w φ(x) + b and the decision function is defined by: n
f ( x ) = sign ( ∑ α i yi ϕ ( x ) ϕ ( xi ) + b )
(8)
i =1
The determination of the scalar product < ϕ (x) ϕ (x i ) > is very expensive in computational complexity and memory and sometimes impossible to calculate. To solve this problem, several kernel functions have been used which we can cite: - Linear kernel: K(x, xi)= xt * xi
(9)
-Polynomial kernel: for degree d polynomials, the polynomial kernel is: K(x, xi)= (xt * xi+ b) d , b>0
(10)
-Radial Basis Function kernel (RBF): K(x, xi)= exp(|x- xi |2 /2σ2), σ ≠ 0
(11)
SVMs have shown a good performance for classifying high dimensional data when a limited number of training samples are available [6].Thus, they have been used for the classification of hyperspectral images. Indeed, many classification approaches have been proposed. In [7], authors developed spectral-spatial classification techniques capable to consider spatial dependences between pixels and they compared the presented approach with other recently proposed advanced spectral-spatial classification techniques. This method is based on the use of SVMs and Markov random field (MRF). In the first step, a probabilistic SVM pixel-wise classification of the hyperspectral image has been applied. In the second step, spatial contextual information has been used for refining the classification results obtained in the first step. This is achieved by means of the MRF regularization. Experimental results prove that the proposed techniques yield a motivate classification accuracies.
Recent searches try to use different techniques to extract features for SVM classification to characterize hyperspectral images such as Discrete Wavelet Transform (DWT) and Dual Tree Complex Wavelet Transform (DTCWT) [10], in this paper, authors developed a classification scheme that uses statistical features alone while classifying an image and they registered a good accuracies for half number of classes. In fact, they used tree different features vector, the first combining statistical features of all the sub-bands of DWT, the second combining statistical features of all the sub-bands of DTCWT and the third combining statistical features of all the sub-bands of DWT and DTCWT. In [11], authors proposed a filter approach for feature selection based on the use of genetic algorithms (GA) to select the most significant features. The goal of this proposed approach is to achieve a minimal number of features without loss of discriminative power. Most classification methods process each pixel independently without considering the correlations between spectral responses of objects and imaging times. In this work, we try to put into consideration the spectral variation over the time in the process of classification. The description of proposed approach is mentioned in the next section. IV.
PROPOSED APPROACH
On the attempt of multi-temporal hyperspectral images classification, we outline in this research an approach based on the use of SVMs and 3D-spectral signature (Fig. 4).
In [8], authors developed a new spatio-spectral classification method. They proposed a new graph kernel function for SVM which takes into account higher order relations in the neighborhood of the pixels and computes the spectral and spatial similarities. Experiment results show that the proposed kernel give the best classification accuracies. In [9], a new approach for semi-supervised learning is proposed to solve the problem of the limit number of training sample. The proposed approach adapts available active learning methods to a self-learning framework in which the machine learning algorithm itself selects the most useful and informative unlabeled samples for classification purposes. It is illustrated with two different classifiers: multinomial logistic
Fig. 4. Proposed approach
Steps of proposed approach:
- 3D modeling of spectral signature: this step consists to have a compact shape which illustrate the variation of spectral signature over the time. In this paper, we used Non-uniform Rational Basis Spline (NURBS) to obtain the 3D representation of spectral signature.
Steps for computing a spherical harmonics shape descriptor for a set of polygons:
NURBS is a mathematical model commonly used in computer graphics for generating and representing curves and surfaces.
2- Treat the voxel grid as a binary function defined on the set of points with length less than or equal to R and express the function in spherical coordinates:
1- Convert the polygonal surfaces into a 2R*2R*2R voxel grid.
f(r, θ, φ) = Voxel(r sin(θ) cos(φ) + R; r cos(θ) + R; r sin(θ) sin(φ) + R)
A NURBS curve is defined by : n ∑ wi Fi ,d (t ) Pi C (t ) = i =0n ∑ w F (t ) i = 0 i i ,d
Where r ∈ [0 R], θ ∈ [0 π], and φ∈ [0 2π]. (12)
3- Express each function f(r, θ, φ) as a sum of its different frequencies: f(r, θ, φ)= ∑ f rm (θ , ϕ )
(14)
m
Where d is
the
degree, Fi,d are
the B-spline basis
functions, Pi are control points, and wi the weight of Pi .
f rm (θ , ϕ ) =
• The degree: is a positive integer. The degree of lines is 1; the degree of circle is 2. The curve there will be smooth if the degree is high. The order of NURBS curve is equal to degree+1. • Control points: a NURBS curve requires at least N = d +1 control points with d the degree of the curve. Each control point owns a weight w > 0. If all the control points have the same weight (usually 1), the curve is called non-rational, if not the curve is called rational. Most NURBS curves are not rational. • Nodes: it's a set of N +degree - 1 parameter values, N the number of control points, that determines where and how the control points affect the NURBS curve. NURBS surfaces are obtained by extending the definition of NURBS curves. Given two vectors u and v nodes and (m+1)*(n+1) control points, the NURBS surface is defined by (13):
S ( u, v ) =
m
∑∑ F ( u ) F ( v ) w i, p
i =0 j =0 n m
j ,q
P
i, j i, j
∑∑ Fi, p ( u ) Fj , q ( v ) wi, j
m
∑a
mn
n =− m
A NURBS curve is characterized by:
n
Where
(13)
i = 0 j =0
- Characterization: to extract features of each 3D-spectral signature, we need to call for shape descriptors witch lead to have a digital signature. There are two types of shape descriptors: global descriptors such as shape histogram descriptor and local descriptors such as 3D shape spectrum descriptor and spherical harmonic decomposition. To characterize our 3D-spectral signatures we used spherical harmonic decomposition. This descriptor is based on the use of spherical harmonics to decompose a 3D model into a collection of functions defined on concentric spheres [12].
(2m + 1)(m − n)! Pmn cos θ einϕ 4π (m + n)!
(15)
4- Combining the different signatures over the different radius, we obtain a two-dimensional spherical harmonics descriptor for the 3D model. - Classification: in this work, we are in front of a multiclass classification problem where we have n classes, each class present material. Each material is submitted by m 3Dspectral signatures. SVMs were originally designed for binary classification, after; it has been extended for multiclass classification. In fact, there are two types of parallel approaches for multiclass SVM: • "One against all (OAA)": This method is most used for SVM multiclass classification. It constructs k SVM models where k is the number of classes. The ith SVM is trained with all of the examples in the ith class with positive labels, and all other examples with negative labels. Each SVM has a decision function fi. x is predicted in ith class which has the largest value fi(x). • "One against one (OAO)": This method constructs k(k1)/2 classifiers where each one is trained on data from two classes ( i, j). Each SVM has a decision function fij. There are different methods for doing the future testing after all classifiers are constructed, the most used is the “Max Wins” strategy, tand we predict x is in the class with the largest vote. Many researches [13], [14] tried to compare these two strategies of SVM multiclass classification, it proved the difficulty to decide the supremacy of one strategy to the other and it affirmed that the choice of the most appropriate method depends on the application. In our classification system, we tried to implement these two multiclass SVM methodologies to select the more suitable. To implement one against all methodology we used n SVMs, the ith SVM is trained with m features vector of m spectral signatures in the ith class with positive labels, and m*(n-1) feature vectors of spectral signatures with negative labels (Fig.
5). To implement one against one methodology we used n*(n1)/2 classifiers, the ith SVM is trained with m features vector of m spectral signatures in the ith class with positive labels, and m features vector of m spectral signatures in the jth class with negative labels (Fig. 6).
presented by a compact and smooth 3D-spectral signature obtained after the use of 3D modeling method "NURBS". A local shape descriptor "spherical harmonic decomposition" has been applied to characterize each 3D signature. To classify this image we used SVMs, thus, we used 10 samples for each class in the training step. Fig. 7 represents an extract from 3D-spectral signatures presented in the simulated image.
Fig. 5. Architecture of the multiclass SVM classifier "one against all". Fig. 7. Some 3D-spectral signatures
A. Results of experiment 1: classification with different kernels SVM is a kernel based method which tries to determine the discriminate hyperplane in the transformed kernel space, the choice of the best kernel function is relatively related by the classified image. Table 1 shows the best overall and class by class accuracies and computational time achieved by the different classifiers. TABLE I. OVERALL AND CLASS BY CLASS ACCURACIES AND COMPUTATIONAL TIME ACHIEVED ON THE TEST SET BY THE DIFFERENT CLASSIFIERS.
Method
Fig. 6. Architecture of the multiclass SVM classifier "one against one".
V.
VALIDATION AND DISCUSSION
To evaluate the proposed approach, we tried to classify a simulated multi-temporal image which contains fives classes: Water, vegetation, clay, olive and rock. Each pixel was
Classification accuracy (%)
Time (s)
C1
C2
C3
C4
C5
OA
SVMLinear
97,5
75
85
80
77,5
83
61,84
SVMPolynomial
97,5
80
85
80
77,5
84
61,65
SVM-RBF
97,5
85
90
80
77,5
86
61,55
In this experiment, we tried to vary the kernel function to select the most adequate for our classification system. In this case, we adopted the OAA strategy. To evaluate the performance of the classification we used the accuracies in each class witch calculate the percentage of pixels correctly
classified in each class and the overall accuracy (the mean of accuracies in each class). RBF kernel is mostly adopted with SVMs in many fields, it can discriminate classes with more accuracy. In our case, experiment results support the performance of this kernel, in fact, it provides an important overall accuracy (fig.8) and less computational time compared to other kernel function. It's the most suitable choice to classify our image.
shape descriptor to characterize the shape of obtained 3D signatures and the performance of SVMs. Experiments on simulated multi-temporal hyperspectral image reveal the performance of RBF kernel, in fact, it allows to have a higher overall accuracy than other type of kernel and the precision and the complexity of OAO strategy in the multiclass classification. REFERENCES [1]
[2]
[3]
Fig. 8. Class by class accuracies achieved by the different classifiers
B. Results of experiment 2: classification and multiclass strategies These experiments addressed the application of SVMs to the multiclass problem by the application of two different parallel approaches: OAO and OAA. The OAA and OAO multiclass strategies were designed and trained using nonlinear SVMs based on radial basis kernel functions. The comparison of these two strategies is based on classification accuracy, computational time and the number of SVMs. TABLE II. OVERALL ACCURACIES, NUMBER OF SVMS AND COMPUTATIONAL TIME ACHIEVED ON THE TEST SET BY THE DIFFERENT MULTICLASS STRATEGIES.
Multiclass strategies OAA OAO
Number of SVMs 5 10
OA (%)
Time (s)
86 87
61,55 62,8
For our classification system, the application of OAO strategy provides an overall accuracy equal to 86%, it use 5 SVMs and it spend 61,55 s. The application of OAO strategy use 10 SVMs, the overall accuracy and the computational time, registered in this case, are equal to 87 % and 62,8 s respectively. OAO approach ensures an overall accuracy slightly higher than OAA approach but it needs a longer run time and important number of SVMs. It's more accurate and more complex.
[4]
[5]
[6]
[7]
[8]
[9]
[10]
[11]
[12]
[13]
[14]
VI. CONCLUSION In this paper, we propose a multi-temporal hyperspectral image classification approach witch exploit a 3D modeling method NURBS to make a compact 3D-spectral signature to solve the problem of spectral variation over the time, a local
F. Ratle, G. Camps-Valls, J. Weston "Semisupervised Neural Networks for Efficient Hyperspectral Image Classification", IEEE transaction on geoscience and remote sensing , Vol. 48 , NO. 5, pp. 2271 - 2282, May 2010. S. Demir, B. Erturk "Improving SVM classification accuracy using a hierarchical approach for hyperspectral images", 16th IEEE International Conference on Image Processing, pp. 2849 - 2852, Nov. 2009. I. R. Farah, S. Hmissi, K. Saheb Ettabaa, and B. Souleiman " MultiSpectro-Temporal Analysis of Hyperspectral Imagery Based on 3-D Spectral Modeling and Multilinear Algebra”, IEEE transaction on geoscience and remote sensing , Vol. 51 , NO. 1, pp. 199 - 216, Jan. 2013. G. villa and al. " Spectro-temporal reflectance surfaces: a new conceptual framework for the integration of remote-sensing data from multiple different sensors", International Journal of Remote Sensing, vol. 34, No. 9-10, pp. 3699-3715, Oct 2012. B. Boser, I. Guyon et V. Vapnik « A Training algorithm for optimal margin classifiers», 5th Annual Workshop on Computational Learning Theory, pp. 144-152, 1992. G. Camps-Valls and L. Bruzzone, “Kernel-based methods for hyperspectral image classification,” IEEE Transaction on Geoscience and Remote Sensing, vol. 43, no. 6, pp. 1351–1362, Jun. 2005. Y. Tarabalka, M. Fauvel, J. Chanussot, and J. A. Benediktsson" SVMand MRF-Based Method for Accurate Classification of Hyperspectral Images", IEEE transaction on geoscience and remote sensing letters, vol. 7, no. 4, pp. 736 - 740, Oct. 2010. G.Camps-Valls, N. Shervashidze, and K. M. Borgwardt "Spatio-Spectral Remote Sensing Image Classification With Graph Kernels", IEEE geoscience and remote sensing letters, vol. 7, no. 4, pp. 741 - 745 Oct. 2010. I. Dópido, J. Li, P. R. Marpu, A. Plaza, J. M. Bioucas Dias and J. A. Benediktsson, "Semisupervised Self-Learning for Hyperspectral Image Classification", IEEE transactions on geoscience and remote sensing, VOL. 51, NO. 7, pp. 4032 - 4044, JULY 2013. S. Arivazhagan, K. Kavitha, R. Dhivya Priya "Performance Comparison of DWT and DTCWT in Hyperspectral Image Classification", International Journal of Research in Image, Video and Signal Processing - IJRIVSP Vol 01, NO.2, pp.55-60, April 2012 A. B. Santos, C. S. F. de S. Celes, A. de A. Araújo, and D. Menotti "Feature Selection for Classification of Remote Sensed Hyperspectral Images: A Filter approach using Genetic Algorithm and Cluster Validity", International Conference on Image Processing, Computer Vision, and Pattern Recognition, Vol. 2, pp. 675-681, 2012. T. Funkhouser, P. Min, M. Kazhdan, J. Chen, A. Halderman, D. Dobkin, and D. Jacobs. ”A search engine for 3d models” ACM Transactions on Graphics, Vol. 22, NO.1, pp. 83–105, 2003. A. Gidudu, G. Hulley et T. Marwala, «Image classification using SVMs: one-against-one Vs one-against-all», Proccedings of the 28th Asian Conference on Remote Sensing, Nov. 2007. J. Milgram, M. Cheriet et R. Sabourin, «One against one or one against all: which one is better for handwriting recognition with SVMs?», International Workshop on Frontiers in Handwriting Recognition, pp. 181-186, 2006
