Classification of Hyperspectral Image Using multiscale spatial texture ...

CLASSIFICATION OF HYPERSPECTRAL IMAGE USING MULTISCALE SPATIAL TEXTURE FEATURES Paheding Sidike1, Chen Chen2, Vijayan Asari1, Yan Xu3, and Wei Li4† 1

Department Electrical & Computer Engineering, University of Dayton, Dayton, OH, USA Department Electrical Engineering, University of Texas at Dallas, Richardson, TX, USA 3 Department Electrical & Engineering, Mississippi State University, MS, USA 4 College of Information Science and Technology, Beijing University of Chemical Technology, China 2

ABSTRACT Spatial information has shown significant contribution for hyperspectral image classification. Local Binary Pattern (LBP) can be used for extracting spatial texture features, however it is incapable of capturing textural and structural features of images at various resolution. Hence, we present a multiscale scheme on Complete LBP (CLBP) as well as on LBP to obtain better spatial features from hyperspectral imagery (HSI). Experiments conducted on two standard HSI datasets proved that the proposed multiscale scheme can improve the classification accuracy of both LBP and CLBP, and Multiscale CLBP provides better accuracy compared to the state-of-the-art spatial feature extraction methods for HSI classification. Index Terms— Hyperspectral imagery (HSI), local binary pattern (LBP), completed local binary pattern (CLBP), extreme learning machine (ELM), image classification 1. INTRODUCTION Advances in spatial resolution enhancement of hyperspectral imagery (HSI) provide new capability for further characterizing pixel signatures in a wide range of remote sensing applications, while attracting researchers’ interest in exploiting spatial information. Unlike the classical HSI classification methods which consider spectral signature of every pixel, the spatial-feature based approaches represent each pixel by extracting spatial context information of that pixel in every spectral band. Over the last decades, a great deal of HSI classification schemes that use spatial features have been proposed in the literature. For instance, a family of composite kernels which integrates spatial and spectral information is introduced in [1]. Morphological profile (MP), utilizes morphological operation to generate spatial structural features, has been investigated for HSI classification [2]. Due to its successful performance, the improved versions of MP, such as extended †

Corresponding author.

MP (EMP) [3] and extended multi-attribute MP (EAMP) [4], were developed. For noise-robust HSI classification, Chen et al. [5] employed a multi-hypothesis prediction approach to incorporate spatial features to reconstruct HSI. Li et al. [6] proposed to include spatial information in HSI classification using a multilevel logistic Markov-Gibbs random field prior. Kang et al. [7] effectively utilized edge-preserving filtering as a probability optimization process to improve the classification output. Another edge computation based approach was presented in [8], where spatial and rotational auto-correlations of local image gradients are obtained by gradient local auto-correlations (GLAC). Texture information is another useful factor that can aid in HSI classification. Markov random fields (MRFs) can be used to extract texture features since they measure spatial relationship between the central pixels and its neighboring pixels, which have been successful applied in HSI classification [9, 10]. Gabor feature as another texture descriptor have been used for HSI classification frameworks [11, 12]. In [11], two-dimensional Gabor features were generated in a principal component analysis projected subspace, while in [12] the three-dimensional Gabor filter bank was applied to hyperspectral images to capture specific orientation, scale, and wavelength-dependent properties of the data. Recently, local binary pattern (LBP) [13] has shown very promising performance in HSI classification [14]. In this technique, the LBP code image is generated for each band in the input HSI. To describe the spatial characteristics of the pixel, the LBP histogram for each pixel of interest is computed with its corresponding neighborhood region. However, this method did not consider the texture features from the magnitude component of the image local differences, as well as the local features from multiresolution of the image. Therefore, we propose a new spatial-feature based HSI classification framework which computes the complete LBP (CLBP) [15] with multiple scales. Since the CLBP contains local structural components: the difference

signs (i.e., original LBP) and the difference magnitudes, we can combine these two components to obtain rich textural information. Furthermore, multiscale analysis in LBP and CLBP could be used to further improve the classification accuracy. To best our knowledge, this is the first time to exploit multiscale LBP (MS-LBP) and Multiscale CLBP (MS-CLBP) for HSI classification. The rest of the paper is outlined as follows. Section 2 provides theoretical analysis of MS-LBP and MS-CLBP, along with the proposed HSI classification framework. Section 3 presents the experimental results and discussion about classification performance. Lastly, Section 4 makes several concluding remarks.

beneficial for image classification since the dominant feature may present at any spatial resolution and using single scale in LBP or CLBP may not guarantee to capture the most discriminate feature. Therefore, it would be great of interest to utilize multiscale LBP and CLBP for a spatial-feature based hyperspectral image classification. For simplicity, we fix the number of neighbors 𝑚 and experiment different scale values 𝑟 to find better results. However, 𝑚 could be varying and combining with different values of 𝑟. In order to generate a compact feature vector based on multiscale analysis, histograms of MS-CLBP and MS-LBP are first computed and then concatenated to form the final feature vector for each pixel in HSI. For example, the feature vector for each scale CLBP can be modelled as

2. METHODOLOGY 𝑓(𝑝𝑐 ) = [𝐶𝐿𝐵𝑃𝑆 𝑚,

2.1. LBP and CLBP LBP is a simple yet very effective rotation invariant texture feature. LBP is formed by comparing the center pixel, 𝑝𝑐 , (pixel of interest) with its 𝑚 neighbors to generated a m-bit binary code. The neighbors are usually evenly distributed around the center 𝑝𝑐 with a radius of 𝑟. LBP computation can be expressed as 𝑚−1

𝐿𝐵𝑃𝑚,𝑟 (𝑝𝑐 ) = ∑ 𝑇(𝑑𝑖 )2𝑖 ,

𝑇(𝑑𝑖 ) = {

𝑖=0

1, 𝑑𝑖 ≥ 0 0, 𝑑𝑖 < 0.

(1)

where 𝑑𝑖 = (𝑝𝑖 − 𝑝𝑐 ) represents the difference of intensity values between each sampling pixel and the center pixel. 𝑝𝑖 (𝑖 = 0,1, … , 𝑚 − 1) is the gray-level of each sampling pixel on the circle. LBP only considers the sign of the local difference to represent the textural pattern, whereas CLBP describes a local region by its center pixel (CLBP_C) and a local difference of the sign (CLBP_S) and magnitude (CLBP_M). CLBP_S is the same as the classical LBP, while CLBP_M is computed by 𝑚−1

1, 𝑥 ≥ 𝑡 (2) 𝐶𝐿𝐵𝑃_𝑀𝑚,𝑟 (𝑝𝑐 ) = ∑ 𝑇(|𝑑𝑖 |, 𝑡)2 , 𝑇(𝑥) = { 0, 𝑥 < 𝑡. 𝑖

𝑖=0

where |𝑑𝑖 | represents the magnitude of the local difference 𝑑𝑖 , and 𝑡 is a threshold which can be set to the mean value of |𝑑𝑖 | from the entire image. In [15], it showed that CLBP has much powerful texture discrimination capability than LBP. In this work, we only consider the CLBP_S and CLBP_M for computational efficiency. 2.2. Multiscale analysis Multiscale computation can be realized by combining multiple operators at varying parameters ( 𝑚, 𝑟 ). This is

𝑟𝑗

𝐶𝐿𝐵𝑃𝑀 𝑚,

𝑟𝑗

] , 𝑗 = 1,2, … , 𝑛

(3)

where 𝑛 is the total number of scale in the CLBP, and 𝑓(𝑝𝑐 ) represents the feature vector for a local region with 𝑝𝑐 as the center pixel. Note that we fix the number of neighbors 𝑚 in this representation. 2.3. Proposed HSI classification scheme Due to a large volumes of data generated by a hyperspectral source, data dimensionality reduction is suggested prior to further processing. In our method, PCA is used to obtain the first 𝐾 ( 𝐾 = 4 in our experiments) principal components (PCs) from HSI [16]. Then MS-CLBP or MS-LBP features are produced from these PCs. Finally all generated features are concatenated to create a compact feature vector for each pixel of interest in the data. For classification, we employ kernel extreme learning machine (KELM) classifier because of its fast computation speed and good classification performance [17, 18]. 3. EXPERIMENTS AND ANALYSIS We evaluate both MS-LBP and MS-CLBP operators on two standard HSI datasets as well as compare their results with three spatial feature extraction based approaches, including gradient local auto-correlation (GLAC) [8], LBP [14] and CLBP. 3.1. Dataset Two popular publicly available HSI datasets (Pavia University dataset and Salinas-A dataset) that used in the experiments can be found in [19]. The specifications of these two datasets are summarized as follows: Pavia University: This test site was collected by the Reflective Optics System Imaging Spectrometer (ROSIS) over Pavia, northern Italy. The dataset consists of 9 different

land cover classes, and it has 115 bands with each having size of 610 × 340 pixels. By removing the noisy bands, the remaining 103 spectral channels were used in the experiment. Salinas-A: This dataset was gathered by the Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) over Salinas Valley, California. After discarding twenty water absorption bands, there are 204 spectral bands left and with each band has a size of 83×86 pixels at a spatial resolution of 3.7-meter.

In GLAC, three parameters have impact on the classification accuracy. They are the patch size ( 𝑤 ), the number of orientation bins (𝐷), and the displacement interval (∆𝑟) in both horizontal and vertical directions. As suggested in [8], we choose the same parameter sets for the Pavia University dataset. As for the Salinas-A dataset, we tune the parameters using a subset of training samples to achieve the optimal performance. And the optimal parameters are found to be 𝑤 = 15 × 15, 𝐷 = 2 and ∆𝑟 = 8.

3.2. Parameter setup

92.00

88.00

86.00 84.00 82.00 [1]

[1:2]

MS-LBP

[1:3]

[1:4]

[1:5]

MS-CLBP

Fig. 2. Classification accuracy (%) of MS-LBP and MSCLBP on the Pavia University dataset. As for the amount of training and testing samples for the Pavia University dataset, we follow the settings as in [8], whereas 5 samples from each class are chosen in the SalinasA dataset for training and the rest of samples are used for testing.

100 95

OA (%)

90.00

OA (%)

In the proposed classification framework, selecting optimal number of feature scale for CLBP or LBP is important. In order to investigate the effects of the scaling factor 𝑟 for classification performance, we first randomly choose training samples and take the rest of the data as testing samples. Then we fix the value 𝑚 = 8 and vary the scale 𝑟 from 1 to 5 to conduct experiments. Figure 1 shows the overall accuracy (OA) versus different scales for CLBP. It can be seem that the best performance of CLBP is found at the scale of 2 for both Pavia University and Salinas-A datasets. Thus we choose 𝑟 = 2 for CLBP in the following experiments. As for the scale of LBP, we set it to 2 according to [8, 14].

90 85

98.00 80 r=2

r=3 Scale

Pavia University

r=4

96.00

r=5

Salinas-A

OA (%)

r=1

94.00 92.00 90.00

Fig. 1. Classification accuracy (%) of CLBP with different scales on the Pavia University and the Salinas-A datasets. Next, the number of scales for MS-LBP and MS-CLBP is studied. Since the scale is set from 1 to 5, there are 5 choices for both MS-LBP and MS-CLBP: {[1], [1:2], [1:3], [1:4], [1:5]}, where [1: 𝑠] indicates the feature combination of the scales from 1 to 𝑠 for 𝑠 = 1, 2, . . . , 5. Figures 2 and 3 present the classification results using different number of scales for the two datasets, separately. From these results, it is observed that the scale set [1:4] works best for MS-CLBP for all dataset, and MS-LBP performs better when we choose 3 scales for the Pavia University dataset, and 5 scales for the Salinas-A dataset.

88.00 [1]

[1:2]

MS-LBP

[1:3]

[1:4]

[1:5]

MS-CLBP

Fig. 3. Classification accuracy (%) of MS-LBP and MSCLBP on the Salinas-A dataset. 3.3 Results and discussion The performance of the proposed classification method along with the competing algorithms for the two experimental datasets is shown in Table 1. Note that we use KELM as the classifier for all these feature extraction algorithms and the results of Spec., GLAC, LBP for the Pavia University dataset are taken from [8]. To avoid any bias, all methods are

evaluated using 10 repeated trials with random selection of training set and testing set in each trial. The OA is then averaged over these 10 trials. From the results, it is evident that the classification with spatial features yields better performance than the spectral signature only (Spec.) based classification. For example, MS-CLBP provides over 13% and 6% higher accuracy than Spec. for the Pavia University dataset and the Salinas-A dataset, respectively. Table 1. Overall accuracy on the two HSI datasets. Methods Pavia University Salinas-A Spec. 78.09 % 89.64 % GLAC 87.35 % 93.54 % LBP 84.39 % 93.15 % MS-LBP 88.21 % 94.32 % CLBP 89.96 % 94.83 % MS-CLBP 91.51 % 96.09 % Comparing LBP with CLBP, the performance of CLBP is always better than LBP in both datasets, this maybe mainly due to the magnitude component (CLBP_M) in CLBP provides additional information around edges and corners in the image so that enriches the spatial pattern for better classification. Moreover, it is worth to mention that MS-LBP and MS-CLBP outperforms the classical LBP and CLBP, respectively. And fusion of CLBP at various scales, i.e., MSCLBP, produces the best results compared to all the other methods. This is because the combination of the magnitude component (CLBP_M) and the sign component (CLBP_S) with multiple scales is able to capture texture and structure features of local regions at various resolution. 4. CONCLUSION In this paper, we introduce a new framework for HSI classification using multiscale spatial texture features, namely MS-LBP and MS-CLBP. CLBP utilizes both the magnitude component and the sign component in image local difference to capture rich texture information, whereas multiscale version of CLBP combines such information at various image resolution. Experimental results reveal that the magnitude component and multiscale scheme can improve HSI classification. 5. REFERENCES [1] G.C. Valls, L.G. Chova, J.M. Marí, J.V. Francés, and J.C. Maravilla, “Composite kernels for hyperspectral image classification,” IEEE Geosci. Remote Sens. Lett., vol. 3, no. 1, pp. 93–97, Jan. 2006. [2] M. Pesaresi, and J. Benediktsson, “A new approach for the morphological segmentation of high resolution satellite imagery,” IEEE Transactions on Geoscience and Remote Sensing, vol. 39, no.2, pp. 309-320, Feb. 2001.

[3] J.A. Benediktsson, J.A. Palmason, and J. Sveinsson, “Classification of hyperspectral data from urban areas based on extended morphological profiles,” IEEE Transactions on Geoscience and Remote Sensing, vol. 43, no. 3, pp. 480-491, 2005. [4] M.D. Mura, J.A. Benediktsson, B. Waske, and L. Bruzzone, “Extended profiles with morphological attribute filters for the analysis of hyperspectral data,” Int. J. Remote Sens., vol. 31, no. 22, pp. 5975-5991, Jul. 2010. [5] C. Chen, W. Li, E. W. Tramel, and J. E. Fowler, “Reconstruction of hyperspectral imagery from random projections using multihypothesis prediction,” IEEE Trans. Geosci. Remote Sens., vol. 52, no. 1, pp. 365–374, Jan. 2014. [6] J. Li, J. Bioucas-Dias, and A. Plaza, “Spectral–spatial hyperspectral image segmentation using subspace multinomial logistic regression and Markov random fields,” IEEE Trans. Geosci. Remote Sens., vol. 50, no. 3, pp. 809– 823, Mar. 2012. [7] X. Kang, S. Li, and J. Bendiktsson, “Spectral–spatial hyperspectral image classification with edge-preserving filtering,” IEEE Trans. Geosci. Remote Sens., vol. 52, no. 5, pp. 2666–2677, May 2014. [8] C. Chen, J. Jiang, B. Zhang, W. Yang, and J. Guo, “Hyperspectral image classification using gradient local autocorrelation,” Asian Conference on Pattern Recognition, Nov. 2015. In press. [9] O. Eches, J. A. Benediktsson, N. Dobigeon and J. Y. Tourneret, “Adaptive Markov Random Fields for Joint Unmixing and Segmentation of Hyperspectral Images,” in IEEE Transactions on Image Processing, vol. 22, no. 1, pp. 5-16, Jan. 2013. [10] Y. Tarabalka, M. Fauvel, J. Chanussot, and J. A. Benediktsson, “SVM- and MRF-based method of accurate classification of hyperspectral images,” IEEE Geosci. Remote Sens. Lett., vol. 7, no. 4, pp. 736–740, Oct. 2010. [11] L.-Z Huo, P. Tang, “Spectral and Spatial Classification of Hyperspectral Data Using SVMs and Gabor Textures,” In Proceedings of IEEE International Geoscience and Remote Sensing Symposium, pp. 1708–1711, July 2011; [12] T.C Bau, S. Sarkar, and G. Healey, “Hyperspectral region classification using a three-dimensional Gabor filterbank,” IEEE Trans. Geosci. Remote Sens. vol.48, pp.3457–3464, 2010. [13] T. Ojala, M. Pietikainen, T.T. Maenpaa, “Multiresolution grayscale and rotation invariant texture classification with local binary Patterns,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, pp. 971–987, 2002.

[14] W. Li, C. Chen, H.J. Su, and Q. Du, “Local binary patterns and extreme learning machine for hyperspectral image classification,” IEEE Trans. Geosci. Remote Sens., vol. 30, pp. 3681–3693, 2015. [15] Z. Guo, L. Zhang, D. Zhang, “A completed modeling of local binary pattern operator for texture classification,” IEEE Trans. Image Process, vol.19, no.6, pp.1657–1663, 2010. [16] C. Rodarmel and J. Shan, “Principal Component Analysis for Hyperspectral image classification,” Surveying and Land Information Systems, vol. 62, No. 2, pp.115-122, 2002. [17] G. B. Huang, H. Zhou, X. Ding, R. Zhang, “Extreme learning machine for regression and multiclass classification,” IEEE Trans. Syst. Man Cybern. Part B Cybern., vol. 42. no. 2, pp. 513–529, 2012. [18] C. Chen, W. Li, H. Su, and K. Liu, “Spectral-spatial classification of hyperspectral image based on kernel extreme learning machine,” Remote Sensing, vol. 6, no. 6, pp. 5795-5814, June 2014. [19] http://www.ehu.eus/ccwintco/index.php?title=Hyperspectral_ Remote_Sensing_Scenes