Chen-Sen Ouyang and Shie-Jue Lee. Department of Electrical Engineering. National Sun Yat-Sen University. Kaohsiung 804, Taiwan, ROC. E-mail: { ouyang I ...
A Hybrid Algorithm for Structure Identification of Neuro-Fuzzy Modeling * Chen-Sen Ouyang and Shie-Jue Lee Department of Electrical Engineering National Sun Yat-Sen University Kaohsiung 804, Taiwan, ROC E-mail: { ouyang Ileesj } @water.ee.nsysii.edu. t w
Abstract The problems with which we are often confronted in neuro-fuzzy modeling are how to adequately decide the number of fuzzy rules extracted from a set of input-output data and how to precisely define the membership functions of each fuzzy rule. In this paper, we propose a hybrid algorithm that can automatically extract fuzzy rules from a set of numerical data points. Our algorithm is mainly composed of two phases, i.e., data partition and rule extraction. In the first phase, the data set is partitioned into several clusters according to the similarities between the data points. In other words, the nearby data points are grouped into the same cluster. This is completed by a sequence of combinations of movable fuzzy prototypes. In the second phase, a fuzzy IF-THEN rule is extracted from each cluster and the membership functions of the corresponding rule are determined by statistical techniques. Experimental results show that the proposed algorithm converges quickly and can generate fewer rules with a lower mean-square error.
1
Introduction
Knowledge acquisition is a very important issue in many fields such as control, expert systems, pattern recognition, communications, etc. The purpose of knowledge acquisition is to extract some information about the data distribution from the observed data. So far, many approaches, e.g., neural networks [l]and fuzzy systems [2, 31, have been proposed to solve this problem. The most popular method in recent years could be neuro-fuzzy modeling [ 4 , 5 , 6 , 7 ] . Neuro-fuzzy modeling is one of the efficient methods for construct'Partially supported by National Science Council under Q a t NSC-89-2213-E110-010.
0-7803-6583-6/00/$10.00 0 2000 IEEE
361 1
ing an approximate system from a set of input-output data. In general, it includes two major steps, i.e., structure identification and parameter identification. The goal of the first step is to identify the rough structure of a fuzzy system by acquiring some rough fuzzy rules from the observed data. Therefore, the number of fuzzy rules, the membership functions in each rule, and the mechanism of fuzzy reasoning must be decided in this step. For the purpose of higher precision, adaptable neural networks are adopted in the second step to refine the parameters of rough fuzzy rules. The advantages of neuro-fuzzy modeling are adaptability, quick convergence and high accuracy. In this paper, we focus on the structure identification of neuro-fuzzy modeling. In general, the process of structure identification is first to partition the given data set into several clusters so that similar data points are grouped into the same cluster. Then, a rough fuzzy rule is extracted from each cluster and the mechanism of fuzzy reasoning is also decided. Wong and Chen [8] proposed a hybrid clustering method for this step. In their method, each datum is represented as a movable vector. Then, the nearby vectors on the feature space will be moved to form a cluster, and a rough fuzzy rule is acquired from each cluster. However, there are some deficiencies in their method such as large number of movable vectors and slow convergence. Therefore, we propose a hybrid algorithm that generates fewer number of movable vectors and therefore speeds up the rate of convergence. In our method, each original input-output datum is defined as a movable fuzzy prototype. Besides, a fuzzy Gaussian membership function is used to describe the neighborhood of a prototype, with all the data in the neighborhood being similar to the fuzzy prototype. Then, a process of combination is performed. We calculate the similarity between fuzzy prototypes in each
,
pair through the use of membership functions. Prototypes with high degrees of similarity will be combined together to form a new movable fuzzy prototype. The process of combination is repeated until no combination is needed. After that, the similarity degrees that are too small are reset to zero, and fuzzy prototypes are moved according to the degrees of similarity. Then, we go back to the process of combination and the whole process is repeated. The algorithm will stop when no movement occurs. Consequently, the data are grouped into several clusters and we can extract a rough fuzzy rule from each cluster. The rest of this paper is organized as follows. In Section 2, the whole process of our hybrid algorithm is introduced in detail and one example is given for illustrating some properties of our approach. In Section 3, two experiments are demonstrated. Besides, we also compare the experimental results with those obtained by Wong and Chen's method [8].Finally, we conclude our paper in Section 4.
2
Our Hybrid Algorithm
Assume that we have a set of M input-output data points { X I , x 2 , . . . , X M } from an . . unknown sys. . .,xh,~h+~). Note that tem, where xi = (xf,xi, ( x i , x i , . . . , &) is the input vector of xi and xh+l is the corresponding output. Therefore, the relation among these input-output data of a system can be described by a function f : RN -+ R. The purpose of neuro-fuzzy modeling is to model the unknown system function f using an approximate function. As a result, we can treat the problem of neuro-fuzzy modeling as a problem of function approximation. To efficiently achieve the goal of structure identification in neuro-fuzzy modeling, we propose a clustering approach to automatically partition the data set into several clusters. After that, a rough fuzzy rule will be extracted from each cluster to describe the local behavior of data points belonging to the corresponding cluster. By combining these rough fuzzy rules with a mechanism of fuzzy reasoning, the behavior of the unknown system can be approximate properly. The detail of our method is discussed as follows.
2.1
Data Partition
In this subsection, a data partition process that can automatically group the data points into several clusters is described. In the beginning, our approach represents each datum as a fuzzy prototype. Also, each
3612
datum is labeled with the corresponding fuzzy prototype. The data points with the same labels form a cluster. To reduce the: number of fuzzy prototypes to prevent the over-fitting of data, two schemes, combination and movement: are repeatedly performed to combine the similar fuzzy prototypes into a new one. Besides, the data points corresponding to those similar fuzzy prototypes are re-labeled with the new fuzzy prototype. Finally, the nearby data on the data space are grouped to form a cluster. In the beginning, a movable fuzzy prototype pi = ( p i , p i , . . . , p h , p b + l ) irr defined for each original input-output datum x i . The initial value of pi is set to xi. Besides, a fuzzy Glaussian membership function G; is used to describe the fuzzy neighborhood of each element p; of pi. We represent Gk by the following equation: GL(xn)
=exp
[-
(+)2] Xn
- Pn
where p; and g i are the mean and standard deviation of G;, respectively. The initial value of each ok is set to a user-defined constant 5. We also label each datum xi with the corresponding pi. Assume that di denotes the number of input-output data points belonging to pi. Then, a scheme of combination is processed repeatedly until no prototypes need to be combined into another one. The scheme of combination is based on the similarity tests between fuzzy prototypes. We calculate the similarity grade rij between the reference fuzzy prototype pi and the comparative fuzzy prototype Pj by
To represent the similarity relation between pi and pj, we modify the values of rij and rji as follows: T" a3
-T.. - la
+
-- rij rji -2
According to each similarity grade rij, pi and pj will be combined into the same class if rij is greater than or equal to a user-defined threshold 0.After that, the set of fuzzy prototypes {PI,. . . , PN} is partitioned into several clusters. And the fuzzy prototypes belonging to the same class are combined to form a new fuzzy prototype. At the same time, the data points
labeled with original fuzzy prototypes are re-labeled as the new fuzzy prototype. For example, suppose we have a set of six data points {xi’li = 1 , 2 , .. . ,6} in R2 space and 0 = 0.1. After the calculations of {rijli = 1 , 2 , . . ., 6, j = 1 , 2 , . . .,6} by Eq. 1 and Eq. 2, we get a similarity matrix R as follows:
R=
1 0.03 0.13 0.09 0.01 0.06
0.03 1 0.06 0.04 0.12 0.06
0.13 0.06 1 0.15 0.05 0.11
0.09 0.04 0.15 1 0.08 0.14
Table 1: Original data points.
j 7
0.01 0.12 0.05 0.08 1
0.06 0.06 0.11 0.14 0.07 0.07 1
Therefore, the set of six data points can be divided into two clusters, i.e., {XI‘,xg’, xq’, xe’} and {x2‘,x5‘}.
Let’s consider again the new fuzzy prototype produced by combining several fuzzy prototypes. Supi mj} are combined into the same pose {pill class j, where mj denotes the number of fuzzy prototypes belonging to class j . The new fuzzy prototype pj’ can be calculated by the following equation:
<
