ESA Living Planet Symposium 2016, 9-13 May 2016, Prague, Czech Republic
Patch-Based Image classification for Sentinel-1 and Sentinel-2 Earth Observation Image Data Products Florin-Andrei GEORGESCU
(1,2)
Radu TĂNASE
(2,1)
Mihai DATCU
(2,3)
Dan RĂDUCANU
(1)
(1)
(3)
Military Technical Academy, Bucharest, 050141, Romania (2) CEOSPACETECH, University POLITEHNICA Bucharest, 060042, Romania Deutsches Zentrum für Luft- und Raumfahrt (DLR),Oberpfaffenhofen, D-82234 Wessling, Germany
[email protected]
ABSTRACT In an era where the satellite image collections are in a continuous growth, Earth Observation (EO) image annotation and classification is becoming an important component of data exploitation. In this paper we present how feature extraction methods such as Gabor (G) and Weber Local Descriptor (WLD) are performing in a patch-based approach in the frame of Sentinel-1 and Sentinel-2 image data analysis. Having the goal to develop an application capable to join feature extraction and classification algorithms, in our assessment, we performed supervised support vector machines (SVM) and k-Nearest Neighbors (k-NN) classifications to extract a few generic classes from synthetic aperture radar (SAR), multi-spectral (MSI) and data fusion (DFI) images. The result of this study is intended to establish the optimum number of classes that can be found in the Sentinel-1 and Sentinel-2 images when using patch based image classification techniques. Also another important objective of this paper is to determine the best patch sizes suitable for this classification type in order to return best results for Sentinel-1 and Sentinel-2 EO images.
EARTH OBSERVATION IMAGE ANALYSIS
INTRODUCTION Due the continuous growth of Earth Observation (EO) image data collections acquired from a great variety of sensors, we can observe an increasing need for methods and techniques of querying remote sensing images, not only by their annotations but also by their semantic content. Various methods of content based image retrieval (CBIR) have been proposed in the remote sensing domain, but no general approaches are available. Regardless of the used method, most of the CBIR systems have the same function - to identify the most similar images with the query image. Regarding the idea of finding a common ground between synthetic aperture radar (SAR), optical data and even data fusion products, our goal is to develop an application capable to join feature extraction algorithms and classification algorithms. Therefore, this paper is presenting a framework of feature extraction methods for SAR, multispectral and data fusion image products that can be used in automatic or semi-automatic classification of urban areas. Our results demonstrate the usability of patch based image classification techniques that can be applied on Sentinel-1 and Sentinel-2 public data sets.
Considering a patch-based analysis scenario, we perform feature extraction and classification on Sentinel-1 and Sentinel-2 data products, having as working methodology the scheme presented in Fig. 1.
Figure 2. Color legend and sample patches Figure 1. Workflow for Feature Extraction and Earth Observation image classification
In order to complete our task, the feature extraction is performed on ground range detected SAR images, multi-spectral images and data fusion images in a patch-based approach and the analyzed scenes can be observed in Fig. 3 and Fig. 4. Also, in Fig. 4 can be seen the manual annotation used for quantitative and qualitative evaluation.
Figure 4. Data fusion (Sentinel-1 + Sentinel-2) (Left) and manual annotation of the analyzed scene (Right)
Figure 3. Sentinel-1 (GRD) vs Sentinel-2 (MSI) image products
RESULTS In this assessment we used a Sentinel-1 ground range detected (SAR) and a Sentinel-2 multispectral (MSI) image, covering 50 x 50km2, over Bucharest, Romania and both scenes have 10m spatial resolution. The SAR image is acquired on April 05, 2016 while the MSI data is acquired on December 23, 2015. Furthermore, the analyzed images are used to obtain the data fusion product (DFI).
As proven in [8] and [9], we extracted image features from patches of 25 x 25 pixels, covering a surface of 250 x 250m2 each. This patch size ensures the computation of image features in optimum condition and the extraction of the relevant information from the analyzed images.
Table 1. SVM Classification of Sentinel image data. Precision (P) - Recall (R) rates Using the manual annotation presented in Fig. 4, we realized the qualitative and quantitative evaluation of the feature classification results for SAR, MSI and DFI images. In Tables 1 and 2 are presented the classification accuracies for both feature extraction methods, Gabor and WLD, for each of the analyzed image product.
Table 2. kNN Classification of Sentinel image data. Precision(P) - Recall (R) rates We can observe from Tables 1 and 2 that the best classification results are obtained using Gabor features. Also, from these tables we can conclude that kNN classification provides better accuracies than SVM. In Fig. 5 are presented in a comparative way the results of SVM classification of Gabor and WLD features. As it can be seen in Table 1 and Fig. 5, if we represent the classification results in a thematic map, we will obtain a very similar output with the manual annotation.
CONCLUSIONS
Figure 5. SVM classification results - Gabor vs. WLD features for a) SAR, b) MSI and c) DFI
In this paper we propose an application capable to join feature extraction and classification algorithms for Sentinel-1 and Sentinel-2 data products in order to find a common ground between SAR, optical data and even data fusion products. Our results demonstrate the usability of patch based image classification techniques that can be applied on Sentinel-1 and Sentinel-2 public data sets.
REFERENCES [1] Shyu C., et al., 2007, GeoIRIS: Geospatial Information Retrieval and Indexing System - Content Mining, Semantics Modeling, and Complex Queries, IEEE TGRS, Vol. 45, No. 4 [2] Datcu M. and K. Seidel, 2005 Human-Centered Concepts for Exploration and Understanding of Earth Observation Images, IEEE TGRS, Vol. 43, No. 3 [3] Sheikholeslami G., W. Chang and A. Zhang, 2002, SemQuery: Semantic Clustering and Querying on Heterogeneous Features for Visual Data, IEEE TKDE, Vol. 14, No. 5 [4] Tuncel E., P. Koulgi and K. Rose, 2004, Rate-Distortion Approach to Databases: Storage and Content-Based Retrieval, IEEE Transactions on Information Theory, Vol. 50, No. 6 [5] Manjunath B. S. , et al., 2001, Color and Texture Descriptors, IEEE Trans. Circuits Syst. Video Technol., vol. 11, no. 6, pp. 703-715 [6] Chen J., et al., 2010, WLD: A Robust Local Image Descriptor, IEEE TBC, Vol. 32 , No. 9 [7] Georgescu F.-A., et al., 2014, A framework for benchmarking of feature extraction methods in Earth Observation image analysis, Image Information Mining 9th Edition, Bucharest [8] Georgescu F.-A., et al., 2016, Patch-based multispectral image classification assessment for Sentinel-2 data analysis, Big Data from Space 2016, Santa Cruz de Tenerife (Spain) [9] Georgescu F.-A., et al., 2016, Feature Extraction for Patch-Based Classification of Multispectral Earth Observation Images, IEEE, GRS Letter [10] Cui S., Dumitru C. O. and Datcu M., 2012, Very High Resolution SAR Image Indexing Based on Ratio Operator, IEEE, GRS Letter.
