dynamic self-organizing algorithm for unsupervised ...

DYNAMIC SELF-ORGANIZING ALGORITHM FOR UNSUPERVISED SEGMENTATION OF SIDESCAN SONAR IMAGES

1stAhmed NAIT CHABANEa 2ndBenoit ZERRa 3rdGilles LE CHENADECa a

Lab-STICC, UMR CNRS 6285, ENSTA Bretagne/Ocean Sensing and Mapping 2 rue François Verny 29806 CEDEX 9 Brest, France. [email protected] ; [email protected]; [email protected]. Tel: +332.98.34.87.12 Fax: +332.98.34.87.50

Abstract: This paper deals with the dynamic neuronal approach for segmentation of textured seafloors from sidescan sonar imagery. For classical approaches of sonar images segmentation, the result of the classification is a set of sediment clusters representing the different kinds of seabed. However, those classical approaches give satisfying results only when a comprehensive training set is available. If the training set lacks a particular kind of seabed, it will be unknown for the classifier and the classification will be reduced to the closest known sediment cluster. As it is not always feasible to know the entire seabed types before the training phase, a dynamic algorithm solution capable of incremental learning has been developed. The Dynamic Selforganizing maps (DSOM) algorithm used in this work is an extension version of classical SOFM (Self-Organizing Feature Map) algorithm developed by Kohonen combined with Adaptive resonance Theory (ART). It is based on growing neuronal map size during the learning processes. Therefore, the size of the map is small in the beginning but increase dynamically using control vigilance threshold. To assess the consistency of the proposed approach, the DSOFM algorithm is tested on simulated data clusters and on real sonar data. Keywords: Unsupervised clustering, SOFM (Self-organizing feature maps) algorithm, ART (Adaptive resonance Theory, DSOFM (dynamic self-Organizing feature maps), texture analysis, sidescan sonar images.

1. INTRODUCTION

Image segmentation is an important step in image analysis chain with different applications to image processing, pattern recognition and objects detection, etc. Segmentation algorithm consists on the process of image division into homogenous groups of pixels according to statistical measure similarity. Two families of segmentation algorithms can be distinguished: the supervised and the unsupervised approaches. In the supervised algorithms a priori knowledge is needed to get a successfully results. The most important supervised algorithms use Maximum Posteriori (MAP) or Maximum Likelihood (ML) techniques [1]. Like with other types of images, supervised algorithms of sonar images segmentation use ground truth, acquired at discrete locations by video, dredge or core data sampling, to assign labels to the seabed types. The supervised approach gives satisfying results only when a comprehensive training set is available. If the training set lacks a particular kind of seabed, it will be unknown for the classifier and the classification will be reduced to the closest known sediment class. As it is not always feasible to have seabed ground truth classes and to know the entire seabed types before the training phase, an unsupervised algorithm capable to detect clusters according to statistical similarity and independently to the expert interpretation is suitable for sonar images. This is what, automated sonar systems classification are becoming widely used. Recent progress in underwater robotics has led to the development of autonomous underwater vehicles (AUVs) capable of automatic data collection and interpretation. The onboard processing capability of these AUV allow for real-time implementation of algorithms for unsupervised seabed classification. The unsupervised approaches exploit the resemblance between statistical features estimated from images, with any a priori knowledge about data labeling or number of group. In this case, clustering algorithms are used to gather pixels or regions on similar groups. Approaches to unsupervised learning include: clustering algorithms (e.g., isodata, kmeans, mixture models and hierarchical clustering) [2][3], blind signal separation generally used for dimensionality reduction and features extraction (e.g., Principle component analysis (PCA), Independent component analysis (ICA)) [3] and neural network models that using unsupervised learning. Among these models, Self-Organizing Feature Maps (SOFM) developed by Kohonen [4] and Adaptive Resonance Theory (ART) developed by Carpenter and Grossberg [5]. Several works have applied successfully different approaches of artificial neural network (ANN) to the problem of seafloor classification [6][7][8]. In this work, a new approach for unsupervised segmentation of sidescan sonar images is proposed. Our approach is based on the mixture of two neural network algorithms: the SOFM and ART algorithms. The SOFM algorithm is powerful tool for clustering and Data Mining. It has been used for mapping high-dimensional data into generally one, two or three dimensional feature map [4]. The most important characteristic of SOFM algorithm consists on topology preservation of input space using neighbored function. It means that data of input space which are close in term of features distance will be close after projection by SOFM algorithm. This topology preservation of data allows best visualization and identification of data clusters. The SOFM algorithm is normally presented as two-dimensional (2D) grid

of neural nodes. A group of close nodes of the grid represent a given class cluster of the data. However, classical SOFM algorithm has some limitations. One of these problems is that the size of the grid and the number of nodes have to be predetermined. Therefore, more simulations tests must be conducted to define the appropriate size of the map. In the case of unknown structure of the data, an incremental or dynamic structure of the grid is suitable. Several dynamic neural network models have been developed, which attempt to overcome the limitation of the fixed size grid of the classical SOFM algorithm. The Neural Gas Algorithm (NGA) developed by Martinez and Shulten [9] is an unsupervised neural network. The main idea of this algorithm is to successively add a units (or nodes) to an initial small network by evaluating local statistical measures gathered during previous adaptation steps. Another Algorithm called Growing Cell Structures (GCS) developed by Fritzke [10] based on the same approach of NGA. However, the GCS has a fixed topology dimensionality (2-D or 3-D). Alhakoon et al [11] proposed a dynamic Self-organizing Maps with controlled growth (GSOM) for knowledge discovery. The advantage of GSOM is the control of the size of the grid using spread factor. The spread factor in this case is independent of data dimensionality and can be used as threshold to create different maps with different dimensionality. A dynamic Self-organizing feature map algorithm (DSOFM) is proposed in this paper. The unsupervised algorithm proposed is based on the combination of SOFM and the ART Algorithms. A detailed description of this algorithm is presented in the section 2. Section 3 presents some experiments and discussion of the DSOFM algorithm tested on synthetic data and on real sonar data. The section 4 gives a conclusions and recommendation of this work.

2. DYNAMIC SELF-ORGANIZING MAPS (DSOM)

The proposed algorithm in this work is based on the combination of two neural networks: SOFM and ART algorithms. The principle idea is to initialize a grid map with small number of nodes then a threshold based on the vigilance parameter of ART algorithm controls the growing size. Each input presented to the network is compared to each node of the network. If resonance is occurred, it means the presented input is matched to one of the nodes, then no growing of the grid. However, if the input is so far in term of Euclidean distance to the all nodes, new nodes are created and the size of the grid is extended. During the node growth, the weights values of the nodes are updated according to the learning process of classical SOFM. The proposed DSOFM is base on three major phases: initialization, growing phase and stopping step. The process is as follows: A. Input data processing The input preprocessing option concerns normalization. It is shown below that input normalization prevents a problem of category proliferation (more clusters) that could otherwise occur [12]. A normalization procedure called complement coding [5] is used. Each input vector x is a p-dimensional vector x( x1 , x2 ,........, x p ) .

0. Normalization: Each component xi is in the interval [0,1] 1. Compute the complement code xc of xi: xci  1  xi

(0)

The new input is 2p-dimentional c c c X  ( x, x )  ( x1 , x2 ,...., x p , x1 , x2 ,....., x p c )

vector

given

by:

B. Initialization phase The network is initialized with nine nodes (a grid of 3x3) with random values from the input vector space. The choice of this number of nodes to initialize the network is justified to implement a 2-D lattice structure and each node has at least two neighbors. C. Growing phase 0. A vector xi is chosen randomly presented to the input of the network. 1. Calculate the activated neuron of the presented input to neurons using:

T ( j) 

X(i)  w(i) ( w(i)   )

(1)

 : is the fuzzy AND operator [13] defined by: (x  y) = minimum(x, y); and where the norm . of a given vector is defined 2M

by: X =  X i i 1

X(i) : is an input from vector space. w(i) : is weight vector. T(j): presents the activated neurons.  : the bias defined in ART algorithm [5], this value must be within the range [0, 1] (although values very near to zero are best).

2. Selection of the winner neuron j* from the activated neurons:

w( j* )  Max(T ( j )) 3. Growing process with vigilance threshold:

If

X(i)  w( j*) w(i)



(2)

then w(i ) new  w(i )old   *V (i )*[X(i )  w(i)]

(3)

else node  node  3 w(i ) new_node1  [ * X (i)]  (1   ) w(i) winner1 w(i ) new_node2  [ * X (i)]  (1   ) w(i) winner2 w(i ) new_node3  [ * X (i)]  (1   ) w(i) winner3 w(i ) new  w(i )old   *V (i )*[X(i )  w(i)]

(4)

 : is the vigilance parameter, this value must be within the range [0, 1]. Low vigilance value minimizes a number of clusters and inversely a high value allows clusters proliferation.  : is the learning rate (   [0,1]). V (i) : neighbourhood function of SOFM algorithm (Gaussian function is very often used). node: is the number of nodes of the network, initialized by a grid of 3x3 size.

4. Return to step 0 as all samples are presented to the network. 3. DISCUSSIONS OF EXPERIMENTS

The subject of clustering algorithms is to discover a grouping of structures inherent in data. In this section experimental tests are used to show the capability of DSOFM algorithm for incremental clusters discovery. Two types of experiments are conducted. The first one is the application of the DSOFM algorithm for clustering on simulated synthetic data. The second experiment is devoted to test DSOMF algorithm on real sonar dataset. A. Experiment 1 The first data used for experiment is simulated of two datasets shown in Figure.1. The dataset in the right of the Figure.1 contains 7 clusters, 788 vectors, in 2-dimensions and the second contains 399 vectors, in 2-dimensions with 6 clusters. In the Figure.2 four random Gaussian data generated in interval [0,1] are used to test DSOFM algorithm.

Fig.1: Clustering of two simulated data using DSOFM algorithm.

A)

B)

C)

Fig.2: Example of application of DSOFM algorithm for simulated Gaussian data with different means and standar deviatios. A) 2 Gaussians, B) 3 Gaussians and B) 4 Gaussians. In the Figure.1, the result of the application of DSOFM algorithm to the two datasets demonstrates the ability for cluster discovering. The same observations is shown in the Figure.2, in all cases (2, 3 or 4 Gaussians) we show the deployment of the DSOFM neural network to dicover the different clusters.

A. Experiment 2 The sonar data used for our study were obtained during the BP’02 (Battlespace Preparation) experiments carried out by the SACLANT Undersea Research Centre in La Spezia, Italy. The system used is the Klein 5000 sidescan sonar operating at 455 kHz. For experiment and to assess the consistency of DSOFM algorithm, a data base of 400 images of four types of sediment (Posidonia oceanica, rock, ripples and Sand) is created from the sonar data images used. In the Figure.3, for a good comprehensive representation in 3-dimensional space of sonar datasets, only three features from Haralick [14] attributes are used: correlation, maximum of distribution and elongation factor.

A)

C)

B)

D)

Fig.3: Application of DSOFM algorithm for clustering of sonar data set. A) Two clusters: sand and rock, B) Grouping of neurones of the DSOFM network. C) Three clusters: sand, rock and vertical ripples. D) Four clusters: sand, rock, vertical ripples and posidonia. In the Figure.3, four type of seabed (Sand, Rock, Vertical Ripples and Posidonia) are introduced the DSOFM network gradually. Dynamically the new seabed type added is detected by the DSOFM network by increasing his grid size (number of nodes) and adapted his structure. In the first case of only sandy and rocky seabeds are presented to the DSFOM network, the size of the grid is 4X3 neurones. Then the size of the network becomes 5X3 neurones for the given seabed (sand, rock and vertical ripples). Finally, the size of the network for the all 4 seabeds is 7x3 neurones. 4. CONLUSIONS AND ONGOING WORK

This paper has presented a new dynamic approach for sidescan sonar images segmentation and classification. It is based on the combination of self-Organizing feature Maps and Fuzzy ART algorithms. The proposed approach gives good results in the specific case of a given data set. However, the objective is to assess the robustness of the algorithm by processing more comprehensive datasets. This will be achieved by processing existing datasets and by

implement a real time version of the classification algorithm for the future autonomous missions of the Daurade AUV. ACKNOWLEDGEMENTS The authors would like to thank the SACLANT Undersea Research Centre (NURC) and the GESMA (DGA/TN) for allowing the inclusion of data from the BP’02 experiment.

REFERENCES [1]

Duda, Richard O.; Hart, Peter E.; and Stork, David G., Unsupervised Learning and Clustering, Chapter 10 in Pattern classification (2nd edition), New York, NY: Wiley, p. 571, 2001. [2] Hastie Trevor; Robert Tibshirani; Friedman Jerome, The Elements of Statistical Learning: Data mining, Inference and Prediction. New York: Springer. pp. 485–586, 2009. [3] Acharyya Ranjan, A New Approach for Blind Source Separation of Convolutive Sources, 2008. [4] Kohonen T, The self-organizing map. Proc IEEE; 78(9):1464–80, 1990. [5] Carpenter, G.A. and Grossberg, S, The ART of adaptive pattern recognition by a self-organizing neural network. Computer 21: 77–88, 1988. [6] R. D. Muller, N. C. Overkov, J.-Y. Royer, A. Dutkiewics, and J. B. Keene, Seabed classification of the south tasminrise from simrad em12 backscatter data using artificial neural networks, Austral. J. Earth Sci., pp. 689–700, 1997. [7] W. K. Stewart, M. Marra, and M. Jiang, A hierarchical approach to seafloor classification using neural networks, in Proc. IEEE Int. OCEANS Conf., Oct. 1992, pp. 109–113. [8] B. Bourgeois and C. Walker, Sidescan sonar image interpretation with neural networks, in Proc. IEEE Int. OCEANS Conf., Honolulu, HI, 1991, pp. 1687–1694. [9] Thomas Martinetz and Klaus Schulten. A neural gas" network learns topologies". Artificial Neural Networks. Elsevier. 1991. pp. 397–402. [10] Bernd Fritzke. Growing Cell Structures - A Self-organizing Network for Unsupervised and Supervised Learning. Neural Networks. pp. 1441–140, Vol.7 n.9, 1993. [11] Alahakoon, D., Halgamuge, S. K. and Sirinivasan, B. Dynamic Self Organizing Maps With Controlled Growth for Knowledge Discovery, IEEE Transactions on Neural Networks, Special Issue on Knowledge Discovery and Data Mining, 11, pp 601-614,2000. [12] Moore, B. ART 1 and pattern clustering. In D. Touretzky, G. Hinton, and T. Sejnowski (Eds.), Proceedings of the 1988 Connectionist Models Sumer School. San Mateo) CA: Morgan Kaufmann Publishers, 1989. [13] L.A. Zadeh, Fuzzy sets, Information and Control, Volume 8, Issue 3, Pages 338-353, June 196. [14] Haralick, R.M. Textural Features for Image Classification. IEEE Trans, Systems, Man and Cybernetics, SMC-3, vol.6, p. 610-621, 1973.