tions, but retained the Euclidean metric. Here, a generalization to a metric which appears morenatur- a1 is made, through the use of a fuzzy covariance matrix,.
TP7
- 2:45
Donald E. Gustafson and William C. Kessel Scientific Systems, Inc. 186 Alewife BrookParlcway Cambridge, Massachusetts02138
Abstract
studied its use in differentiating hiatal hernia and gallstones. It appears that medical diagnosis A class of fuzzy ISODATA clustering algorithms may be an especially fruitful area of application has been developed previously which includes fuzzyfor fuzzy clustering, since biological systems are extremely complexand the boundaries between means. This class of algorithms is generalized to "disinclude fuzzy covariances. The resulting algorithm tinct" medical diagnostic classes .are not sharply closely resembles maximum likelihood estimation of defined. This has been suggested for cardiovascumixture densities. It is argued that use of fuzzy lar investigations[ 103. In a "hard" clustering algorithm, each pattern covariances is a natural approachto fuzzy clustering. Experimental results are presented which invector must be assigned to a single cluster. This "all or none'' membership restriction is nota realmay be obtaindicate that more accurate clustering istic one, sincemany pattern vectors may have the ed by using fuzzy covariances. characteristics of several classes. It is more natural to assign to each pattern vector a set of 1. Introduction memberships, one for each class. The implication of this is that the class boundaries are not "hard" but rather are "fuzzy". Another problem is that The notion of fuzzy sets, first put forth by a "hard" Zadeh [l], is an attempt to modify the basic con- the set of all partitions resulting from clustering algorithm is extremely large, making an ception of a space--that is, the seton which the given problem is defined. By introducing the conexhaustive search extremely complicated and expencept of a fuzzy--i.e., an unsharply defined set, a sive. Fuzzy clustering will generally lead to more different perspective is provided for certain probcomputational tractability [ 113. Another advantage lems in systems analysis, including pattern recog- of fuzzy clustering is that troublesome or outlying members of the dataset are more easily recognized nition. One of the significant difficulties in devel- than with hard clustering, since the degree of membership is continuous rather than "all-or-none." opment of a systematic approachto pattern recognition is that the phenomena of interest are modeledBezdek and D u m [12] have noted the relationship of by equations which contain functions and operators fuzzy clustering to estimating mixture distribuwhich may appear simple and natura1,but which yieldtions, but retained the Euclidean metric. Here, a generalization toa metric which appears morenatursome solutions which could be regarded as pathologa fuzzy covariance ical. The difficulty stems from our desire to dif- a1 is made, through the use of matrix, ferentiate between classes ainmanner whichis simple and easy to visualize.In doing so, we restrict the solutions in an unknown way. The use of fuzzy 2. Problem Formulation sets is an attempt to ameliorate this problem. Pattern classification problems have provided The definition of a fuzzy partition used here impetus for the development of fuzzy set theory. agrees with that of Ruspini [4], D u m 161 and BezRecently, fuzzy sets have provided a theoretical dek [13] and is a natural extension of the convenbasis for cluster analysis with the introduction of fuzzy clustering. The use of fuzzy sets in clustional partitioning definition. An ordinary, or tering was first proposed in [2] and several clas- "hard" partition isa k-tuple of Boolean functions w(-) = I w I , ~ , . .,wk) on the feature spacer C Rn sification schemes were developed [3]. The first which satisfy fuzzy clustering algorithm was developed in 1969 by ' l 15 j 5 k (1) wj(x) = 0 or 1, V x E , Ruspini [ 41, and used by several workers[5]. Folloving this,DUM [ 61 developed the first fuzzy extension of the least-squares approach to clustering [ 71 to an inf iand this was generalized by Bezdek nite family of algorithms. Several problems in medical diagnosis have If r representsthe j-thclass,with = ' k been attacked using fuzzy clustering algorithms. 0 V i # j and Uj=l rj = r, then wm(x) = 1 means Adey [8] achieved promising results in inter reting that x E rm and (2) insures that x is a member of EEG patterns in cerebral systems. Bezdek [9! has
.
necessary conditions: precisely one class. It is possible to pass from f irst-order this definition to a corresponding fuzzy partition by retaining (2) but repla- (1) with 'the relaxed condition O ~ Di v i
j=1
IM. I=p.+yj
REFERENCES 1. L.A. Zadeh,"FuzzySets",Informationand t r o l , Vol. 8 , pp.338-353,1965.
normalization constraints
Clusteringand Table 1 ComparisonofFuzzy Maximum Likelihood Solutions
Con-
765
A l l aesigrPente correctly M e .
All w
.m1 .5153 .so02 .4907 .5495 .9574 17 .9905 .95U 18 .W72 .9H)9 ,9606
.49% . 5 l l 9 .7047 .!XI85 .3676 .4934 .6599 .8664 .8757 -0104 .6992 .9353 .7921 .9899 .2424 .%80 -6850 .9%8.6268 .9911 .9794 .a836 ..9985 .74Q3 .9949 .9715
.9808
.M22 .5341 .9373 .9905 .9965 .9975
il
0
>0.98 except: ~
vs = [0.7004,0.2996]
o
V U * [0.225,0.725]
14 passes Figure 4:
Cluster AssignmentsU s i n g Fuzzy Covariance Seededat Class bans
xx
Sample
a = 2
ClassHeans
I
In1 = In21 = 1
9th-13th pass Figure 5(a): Figure 1:
Tvo-Class Configuration
9
x+-Xc
1
Cluster 2
Cluster Assignments Using FUZZY Covariance WithSeeds SI = (0.002,0), s2 = ( 0 , O )
Y
All assignments were correct. All wij>O.97 except: ~5 = [0.6836,0.3164]
vn = [0.2597,0.7403]
a = 2 In1
l=l%l=1
Cluster 2
20th pass Figure 2:
Figure 5(b): Cluster Assignments Using Bard ISODATA Means Seeded With Class Sample
v1
Cluster Assignments UsingFUZZY Covariances With Seeds S1~(0.001,0), S2=(O,O) After Convergence
[0.4134,0;5866]
~4 = [0.8393,0.1607] VU = [0.3620.0/6380]
Cluster 2 \ &x a = 2 A s 1
a = 2 8 passes matrix for rj Figure 6:
Figure 3:
Cluster Assignments Using Fuzzy ISODATA With Seeds SI = (O.OOl,O), S p = ( 0 , O )
Cluster Assignments Using FuzzyISODATA Seeded at S,-(O.OOl,O), S2=(0,0) and Sample Covariance Matrices
