Fuzzy Self-Organizing Map based on Regularized Fuzzy c-means Clustering Sándor Migály, János Abonyi, and Ferenc Szeifert University of Veszprém, Department of Process Engineering, H-8201, Veszprém, P.O.Box 158, Hungary, www.fmt.vein.hu/softcomp,
[email protected]
Abstract: This paper presents a new fuzzy clustering algorithm for the visualization of high-dimensional data. The cluster centers are arranged on a grid defined on a small dimensional space that can be easily visualized. The smoothness of this mapping is achieved by adding a regularization term to the fuzzy c-means (FCM) functional. Comparison to the existing modifications of the fuzzy c-means algorithm is given with application examples that show the good performance of the algorithm.
1. Introduction The recently developed clustering based computational intelligence methods are becoming increasingly popular in the pattern recognition community. They are able to learn the mapping of functions and systems, and can perform classification from labeled training data as well as explore structures and classes in unlabeled data. The visualization of high-dimensional data is also important pattern recognition task. Advanced visualization tools should be able to convert complex, nonlinear statistical relationships between high-dimensional data items into simple geometric relationships on a low-dimensional display and compress information while preserving the most important topological and metric relationships of the primary data items. Among the wide range of possible tools, the self-organizing map (SOM) is one of the most effective [1]. It implements an orderly mapping of a high-dimensional distribution onto a regular low-dimensional grid. The SOM algorithm can be considered as a generalized version of the hard c-means clustering algorithm. Hard clustering methods are based on classical set theory, and it requires an object that either does or does not belong to a cluster. Fuzzy clustering methods allow objects to belong several clusters simultaneously with different degrees of membership [2]. The data set is thus partitioned into c fuzzy subsets. In many real situations, fuzzy clustering is more natural than hard clustering, as objects on the boundaries between several classes are not forced to fully belong to one of the classes.
Recently, several approaches have been worked out to increase the performance of SOM by the incorporation of fuzzy logic. In a study written by Vuorimaa [3], the SOM algorithm was modiffied by replacing the neurons with fuzzy rules, allowing an efficient modeling of continuous valued functions. Chen-Kuo Tsao et al. [4] integrate some aspects of the fuzzy c-means model into the classical SOM framework. Finally, in [5], a fuzzy self-organizing map is presented based on the modifications of the fuzzy c-means functional. In this approach, the code vectors are distributed on a regular low-dimensional grid, and similarly to SOM, a penalization term is added in order to guarantee a smooth distribution for the values of the code vectors on the grid. The idea of the ordering of the clusters in a smaller dimensional space can also be found in [6], where the fuzzy c-means functional has been modified to detect smooth lines. The aim of this paper is to generalize the idea of smoothly distributed fuzzy clustering and show how the above mentioned approaches are related to the proposed tools.
2. Regularized Fuzzy c-means Clustering In this section after the review the basic FCM algorithm [2], the idea of the regularized FCM is presented. The last two subsections analyze the relationship of the solution to other modifications of FCM. 2.1 Fuzzy c-means Clustering In this paper the clustering of quantitative data is considered. The data are typically observations of some physical phenomenon. Each observation consists of n measured variables, grouped into an n-dimensional column vector z k = [z1k ,..., z nk ]T , z k ∈ ℜ n .
A set of N observations is denoted by Z = {z k k = 1,2,..., N } and represented as a n × N matrix: z11 z12 K z1N z z 22 K z 2 N Z = 21 (1) M M M M z n1 z n 2 K z nN In the pattern recognition terminology, the columns of Z called patterns or objects, the rows are called the features or attributes, and Z is called the pattern matrix. The objective of clustering is to divide the data set Z into c clusters. A c × N matrix U = [µ ik ] represents the fuzzy partitions if its elements satisfy the following conditions:
µ ik ∈ [0,1], 1 ≤ i ≤ c, 1 ≤ k ≤ N
(2)
where µ ik denotes the degree of the membership of the z k = [z1k ,..., z nk ]T -th observation belongs to the 1 ≤ i ≤ c -th cluster.
The objective of the FCM model [2] is to minimize the sum of the weighted squared distances between the data points, zk and the cluster centers, vi,, where the distances D i2,k are weighted with the membership values µ i,k . Therefore, the objective function is c
J (Z, U, V ) =
N
∑∑ (µ
ik
)m Di2,k
i =1 k =1
(3)
where U = [µ ik ] is a fuzzy partition matrix of Z, V = [v1 , v 2 ,..., v c ] is a matrix
consists of the cluster prototype vectors (centers), and m ∈ 1, ∞ ) is a weighting
exponent that determines the fuzziness of the resulting clusters and it is often chosen as m=2. Di2,k can be determined by any appropriate norm, e.g., an A-norm: Dik2 = z k − v i
A
=
(z k − v i )T A (z k − v i )
(4)
Usually spherical clusters are applied when A is an identity matrix, A=I. The minimization of the c-means functional (Eq. 3) with respect to the following constraints N
0
0 . Repeat for l = 1,2,... Step 1.: Compute the cluster centers:
∑ (µ ) N
vi(l ) =
k =1 N
(l −1) m zk ik
∑(
, 1≤ i ≤ c
)
m µ ik(l −1)
k =1
Step 2.: Compute the distances: Dik2 = z k − v i
2
=
(z k − v i )T A(z k − v i ), 1 ≤ i ≤ c, 1 ≤ k ≤ N
Step 3.: Update the partition matrix: If Dik > 0 for 1 ≤ i ≤ c, 1 ≤ k ≤ N , 1
µ ik(l ) =
2 m−1
Dik jk j =1 c
∑ D
until U (l ) − U (l −1) < ε
otherwise µ ik(l ) = 0
