Collaborative-Based Multi-scale Clustering in Very High Resolution Satellite Images J´er´emie Sublime1,2,3(B) , Antoine Cornu´ejols1 , and Youn`es Bennani2 1
2
UMR MIA-Paris, AgroParisTech, INRA, Universit´e Paris-Saclay, 75005 Paris, France {jeremie.sublime,antoine.cornuejols}@agroparistech.fr Universit´e Paris 13 - Sorbonne Paris Cit´e, Laboratoire d’Informatique de Paris-Nord - CNRS (UMR 7030), 99 Avenue Jean Baptiste Cl´ement, 93430 Villetaneuse, France
[email protected] 3 LISITE Laboratory - RDI Team, ISEP, Paris, France
[email protected]
Abstract. In this article, we show an application of collaborative clustering applied to real data from very high resolution images. Our proposed method makes it possible to have several algorithms working at different scales of details while exchanging their information on the clusters. Our method that aims at strengthening the hierarchical links between the clusters extracted at different level of detail has shown good results in terms of clustering quality based on common unsupervised learning indexes, but also when using external indexes: We compared our results with other algorithms and analyzed them based on an expert ground truth. Keywords: Multi-scale segmentation
1
clustering
·
Cluster
analysis
·
Image
Introduction
With the booming number of available satellite images and data, the automatic interpretation of remotely sensed images has become an increasingly active domain. With sensors now capable of getting images with a very high resolution (VHR) on a large spectral resolution, it is more and more difficult to design algorithms and methods capable of efficiently processing such data in a reasonable amount of time. Such process usually contains two steps: (1) A segmentation step that consists in grouping together connected groups of pixels with the goal of finding homogeneous segments the borders of which will be a good estimation of the objects present in the image [3,5]. The segments created using this process are supposed to be relevant and match the real objects that can be found in the picture. (2) A clustering step the aim of which is to analyze the segmented objects and create groups of elements that are similar. c Springer International Publishing AG 2016 A. Hirose et al. (Eds.): ICONIP 2016, Part III, LNCS 9949, pp. 148–155, 2016. DOI: 10.1007/978-3-319-46675-0 17
Collaborative-Based Multi-scale Clustering in VHR Satellite Images
149
In this article, we will focus on the clustering of already pre-processed images. The clustering of this type of image data set is challenging for several reasons: – The quality of the initial segmentation can have a huge impact on the quality of the data set, and therefore of the final clusters. While an over-segmentation (splitting elements of interest into several segments) can be fixed during the clustering process, in case of under-segmentation (several elements of interest into a single segments) several objects of interest and their clusters may be lost for good. – This type of data has spacial dependencies between the data that don’t exist in regular data sets. Another important aspect is that when the segments and regions are created, this adds a large number of new attributes that may be taken into considerations: surface of the region, perimeter and elongation; extrema, variance and average values of the attributes in a given region, contrast with the neighboring regions, etc. Quite often, these attributes are redundant. However, this type of data sets also have several advantages: The results can easily be evaluated simply by projecting them. Another common issue which consists in finding the right number of clusters is less of a problem when it comes to clustering of satellite images. With the recent progress in satellite imaging, there are several possible level of interest available on a very high resolution satellite image: At the first level, we can usually distinguish three main types of objects, namely water bodies, vegetation areas and urban areas. At a second level we can separate different types of urban blocs, different types of vegetation areas, and start to distinguish elements such as roads. When zooming even more, very high resolutions images enable detecting small urban elements such as individual houses, cars, trees, or swimming pools. As one can see, there is an obvious hierarchical relationship between the different objects of interests that can be detected when searching for different numbers of clusters. However, the huge size of these data sets usually makes them ineligible for hierarchical clustering algorithms because of their high computational complexity. Our idea in this article is to propose a framework in which different clustering algorithms suited for image data sets and looking for different numbers of clusters can exchange their information in order to cross-validate the structures they respectively find. Our proposed method is inspired from previous works done in collaborative clustering [4,6,7] that allow different clustering algorithms to exchange with a goal of mutual improvement. The difference between our proposed method and existing methods in the literature can be summed up in two points: – Our proposed method is generic and can work with any clustering algorithm. In particular, it is suited for clustering algorithms used in image data sets. – Unlike previously proposed collaborative frameworks, we don’t have the limitation that all collaborators should be looking for the same number of clusters. In our case, our framework is specifically designed for multi-scale analysis and therefore each collaborator will be looking for a different number of clusters.
150
J. Sublime et al.
The remainder of this article is organized as follows: In Sect. 2, we introduce our proposed collaborative method for multi-scale analysis in satellite images. In Sect. 3, we introduce our data set and the experimental results of our method when compared with other algorithms. Finally, this article ends with a conclusion and perspectives on future works.
2
Multi-scale Communication Between Different Algorithms
Let us consider X = {x1 , ..., xN }, xn ∈ Rd a data set containing N segments of a pre-processed image described by d numerical feature attributes. We consider that we have J clustering algorithms {A1 , ..., AJ } working on this data set. Each algorithm Ai has access to the same features but is looking for Ki clusters, Ki being a different number for each algorithm. Likewise, each algorithm Ai will i ]. output its own clustering solution S i = {s1 , ..., sN }, sn ∈ [C1i ..CK i Since we want these algorithms to communicate in order to exchange information on there clusters with a goal of mutual improvement, following an idea from earlier works we define Ω i,j the Probabilistic Confusion Matrix (PCM) mapping the clusters of any algorithms Ai and Aj as described in Eq. (1) where i,j represents the percentage of data from the cluster a of algorithm Ai that ωa,b are in the cluster b of algorithm Aj . ⎞ ⎛ i,j i,j ω1,1 · · · ω1,K j j i ⎜ . . .. ⎟ ⎟ where ω i,j = |Ca ∩ Cb | . . Ω i,j = ⎜ (1) . a,b . ⎠ ⎝ . |Cai | i,j i,j ωKi ,1 · · · ωKi ,Kj The PCM Ω i,j makes it possible to know whether or not the objects of two results have similar clusters, or if the two clustering results are dissimilar. Since our goal in this article is to have the different algorithms working on a same satellite image while exchanging their information, and since we know that the clusters to be found at different scales have a hierarchical structure, our aim will be to influence each local algorithm into tweaking its clusters in order to enforce this hierarchical structure. To do so, our proposed method consists in an iterative framework where all algorithms are optimized in parallel following a two-step process similar to the process in the Expectation-Maximization algorithm [2]. In our case, we use the Iterated Conditional Modes [1] with a local Gaussian Mixture model and a neighborhood function adapted to very high resolution satellite images [9]. Under these conditions, each local clustering algorithm Ai has a set of local parameters i }, θci = {πci , μic , Σci } where μic is the mean vector of cluster c for Θi = {θ1i , ..., θK i i the algorithm A , Σci its covariance matrix and πci the mixing proportion for this cluster.
Collaborative-Based Multi-scale Clustering in VHR Satellite Images
151
Algorithm 1. Collaborative multi-scale analysis: General algorithm Local step: forall the clustering algorithms Ai do Initialize each algorithm Ai on the data X. → Learn the local parameters Θ0i , initialize S0i end Collaborative step: while the system’s global entropy H decreases do Meta E-Step: forall the clustering algorithms Ai do For each data xn , assess sin |t+1 using Eq. (2) end Meta M-Step: forall the clustering algorithms Ai do Update parameters Θi |t+1 using Eq. (4) end t++ end Return S
Our proposed algorithm then runs the following two steps in an iterative way: – Expectation step: Equation (2) is optimized in parallel for each algorithm and each data. In this equation sin,c (Θi |t ) is the local algorithm responsibility function given in Eq. (3), and the remainder of this equation is a collaborative term the aim of which is to strengthen the hierarchy between the different clustering Ki J j,i scales. Z(i, n) = c=1 j=i ωq,c is a normalization constant. – Maximization step: For each algorithms, the parameters Θi are updated depending on the new solution vectors S i . This is done using a simple Maximum a posteriori approximation as shown in Eq. (4). ⎛
sin |t+1
⎞ J 1 j,i ⎠ , = Argmax ⎝sin,c (Θi |t ) + ωq,c Z(i, n) c
q = sjn |t
(2)
j=i
sin,c (Θi ) =
πci N (μic , Σci , xn ) · expf (Vx ,c) Z
(3)
In Eq. (3), Vx is the neighborhood of the data segment xn , and f (Vx , c) is an energy function used to weight the likelihood of the cluster c being chosen given the neighbor segments [9]. In Fig. 1(b), an example of such neighborhood is shown with a segment and its neighborhood being highlighted in yellow and red respectively.
152
J. Sublime et al.
Θi |t+1
⎧ N Nc = n=1 sin (c) ⎪ ⎪ ⎪ ⎨μi = 1 N si (c) · x n c n Nc = Argmax p(X|Θ) : ∀c n=1 N 1 i i i i T ⎪ Σ = s (c) · (x Θ n − μk )(xn − μk ) ⎪ c n=1 n Nc ⎪ ⎩ i πc = NNc
(4) In Algorithm 1, we show the general framework of our algorithm. The collaborative step is iterated until the global entropy H described in Eq. (5) stabilizes. Using this stopping criterion, we ensure that the hierarchy between the clusters of the different algorithms will be maximized when the algorithm stops. H=
J J i=1 j=i
3 3.1
Kj Ki −1 i,j i,j ωl,m ln(ωl,m ) Ki × ln(Kj ) m=1
(5)
l=1
Experimental Results Description of the Data
The VHR Strasbourg data set is a set made from a very high resolution image of the French city of Strasbourg, 1 px = (50 cm)2 , an extract of which is shown in Fig. 1(a).
(a) Extract of the original image
(b) Example of a segment neighborhood
Fig. 1. Original image and segmentation example in Central Strasbourg (Color figure online)
This image has been pre-processed into a data set made of 187058 segments, each of them described by 27 attributes either geometric or radiometric [8]. These attributes include the geographic position of the segment, the surface of the area covered by the segment, the mean RGB values, the contrast compared to neighbor pixels and segments, the brightness, and the standard deviations, among others. In addition to this information, this data set provides the neighborhood dependencies between the segments: number of neighbors, id number of each neighbor segment, and relative percentage of shared border.
Collaborative-Based Multi-scale Clustering in VHR Satellite Images
153
The segments in this data set have highly irregular shapes (cf. Figure 1(b)), and consequently the neighborhoods themselves are also irregular. Unlike in the pixel model where 1st order neighborhoods usually include 4 or 8 neighbors, in this data set each segment can have 1 to 15 neighbors depending on its shape and position. In order to validate our results, we had to find a ground truth. It was however too tedious a task to manually label the 187058 segments. Therefore we decided to rely on maps of the area made by expert geographers. The ground-truth that we use to assess our results was produced using a hybrid methodology, mixing data from topographic databases for roads and buildings, a supervised classification for different types of water and vegetation, as well as further manual refinement in order to reduce classification errors. The expert geographers provided 15 different classes, this number was reduced to 9 classes by regrouping those that are very similar or technically impossible to distinguish for an unsupervised algorithm (such as different types of crops, or similar vegetation classes with different heights). 3.2
Results
In our experiments, we used three algorithms: the EM algorithm for the Gaussian Mixture Model [2], the regular SR-ICM algorithm [9] and our proposed collaborative Co C-ICM algorithm. The algorithms were run searching for 3, 6 and 9 clusters, separately in the case of the EM and the SR-ICM algorithm, and collaboratively for our method. The results were assessed using the Davies-Bouldin index as a an internal criterion. This index assesses the compactness of the clusters and how well they are separated. The Davies-Bouldin index is not normalized and a lower value indicates better clustering results. It is worth mentioning that the Davies-Bouldin index usually gives better results with less clusters. As for the external index, we used the Rand Index to compare our results with the expert ground truth. We did not used the Adjusted version of the Rand Index because it was giving really poor results for the clusterings with 3 and 6 clusters which is much lower than in the ground truth. The Rand Index expresses in percentage how much the result matches with the expert ground-truth. The results of this experiments over a dozen simulations for each algorithm are shown in Table 1, where the best result for each number of cluster is highlighted in bold. As one can see, our proposed method performs better on average in 5 cases out of 6. We can also mention that the improvement on the Davies-Bouldin Index are more remarkable than these on the Rand Index. These results were to be expected since our proposed method doesn’t change the unsupervised nature of the base clustering algorithms that we use. It is therefore logical that we have better results with unsupervised indexes. We believe that the same explanation applies to justify that the SR-ICM algorithm achieves somewhat identical Rand Index results with these of our proposed method when dealing with only 3 clusters.
154
J. Sublime et al. Table 1. Experimental results Algorithm
Davies-Bouldin index Rand index
EM 3 2.36928 2.32855 SR-ICM 3 Co C-ICM 3 2.32674
0.67454 0.67606 0.67435
EM 6 2.88014 2.67816 SR-ICM 6 Co C-ICM 6 2.49726
0.75867 0.76935 0.77068
EM 9 2.62786 2.94065 SR-ICM 9 Co C-ICM 9 2.58836
0.78225 0.79063 0.792187
In Figs. 2(a) and (b), we show extracts of the results for the clusterings with 6 and 9 clusters. As can be seen on a color large scale version of the images, we achieve a decent segmentation at both scales with most elements being recognizable. On interesting remark is that while we did get a hierarchical structure between the clusters at different scales, the 3 initial clusters were not those that we would have had, had we used a supervised method. The 3 clusters found at the first scale by all algorithms (collaborative or not) were: vegetation areas, industrial buildings, and a cluster containing both urban elements and water bodies. Actually, water bodies only became a distinct cluster at the scale with 9 clusters. Nevertheless, the unsupervised methods that we used found clusters that overall made sense. In the case of our proposed method, the results were not only better when using internal and external criteria, we also had a better hierarchical structure between the clusters from different scales.
(a) Collaborative C-ICM result for (b) Collaborative C-ICM result for 6 clusters 9 clusters
Fig. 2. Clustering extracts (Color figure online)
Collaborative-Based Multi-scale Clustering in VHR Satellite Images
4
155
Conclusion
In this article we have propose a simple yet powerful method that makes it possible for several clustering algorithm to work together with the goal of achieving a multi-scale clustering of the same data set. Our proposed method based on collaborative techniques lifts up previous limitations where collaboration was only possible when the different algorithms were looking for the same number of clusters. By doing so, our proposed framework can find the hierarchical structures between the clusters found at different scales. We have tested our method on a very high resolution image data set for multi-scale clustering purposes, and we have compared our results with other specialized and more generalist methods available in the literature. While this article was focused on an application for clustering algorithms used with very high resolution satellite images -data sets for which our method seemed well suited-, it is our strong belief that our proposed method is generic and can easily be extended for any other types of clustering algorithms. Our future research will therefore most likely focus on trying this method with other clustering algorithms and application contexts. Acknowledgements. This work has been supported by the ANR Project COCLICO, ANR-12-MONU-0001.
References 1. Besag, J.: On the statistical analysis of dirty pictures. J. R. Stat. Soc. Ser. B 48(3), 259–302 (1986) 2. Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. Ser. B 39(1), 1–38 (1977) 3. Gonzalez, R.C., Woods, R.E.: Digital Image Processing, 3rd edn. Prentice-Hall Inc., Upper Saddle River (2006) 4. Grozavu, N., Bennani, Y.: Topological collaborative clustering. Aust. J. Intell. Inf. Proces. Syst. 12(3) (2010). https://cs.anu.edu.au/ojs/index.php/ajiips/issue/view/ 156 5. Pal, N.R., Pal, S.K.: A review on image segmentation techniques. Pattern Recognit. 26(9), 1277–1294 (1993) 6. Pedrycz, W.: Collaborative fuzzy clustering. Pattern Recognit. Lett. 23(14), 1675– 1686 (2002) 7. Pedrycz, W.: Interpretation of clusters in the framework of shadowed sets. Pattern Recogn. Lett. 26(15), 2439–2449 (2005) 8. Rougier, S., Puissant, A.: Improvements of urban vegetation segmentation and classification using multi-temporal pleiades images. In: 5th International Conference on Geographic Object-Based Image Analysis, p. 6 (2014) 9. Sublime, J., Troya-Galvis, A., Bennani, Y., Gancarski, P., Cornu´ejols, A.: Semantic rich ICM algorithm for VHR satellite image segmentation. In: IAPR International Conference on Machine Vision Applications, Tokyo (2015)
