Simultaneous Optimization of ANFIS-Based Fuzzy Model Driven to Data Granulation and Parallel Genetic Algorithms Jeoung-Nae Choi1, Sung-Kwun Oh1, and Ki-Sung Seo2 1
Department of Electrical Engineering, The University of Suwon, San 2-2 Wau-ri, Bongdam-eup, Hwaseong-si, Gyeonggi-do, 445-743, South Korea
[email protected] 2 Department of Electronics Engineering, Seokyeong University, South Korea
Abstract. The paper concerns the simultaneous optimization for structure and parameters of fuzzy inference systems that is based on Hierarchical Fair Competition-based Parallel Genetic Algorithms (HFCGA) and information data granulation. HFCGA is used to optimize structure and parameters of ANFISbased fuzzy model simultaneously. The granulation is realized with the aid of the C-means clustering. Through the simultaneous optimization mechanism to be explored, we can find the overall optimal values related to structure as well as parameter identification of ANFIS-based fuzzy model via HFCGA, C-Means clustering and standard least square method. A comparative analysis demonstrates that the proposed algorithm is superior to the conventional methods.
1 Introduction Recently, a lot of attention has been directed to advanced techniques of system modeling. Specially, many researchers are concerned about ANFIS-based fuzzy modeling and there has been a diversity of approaches to ANFIS-based fuzzy modeling. Some enhancements to the model have been proposed by Oh and Pedrycz [5,11-13]. As one of the enhanced ANFIS-based fuzzy model, fuzzy relation model based on information granulation and genetic algorithms was introduced [5]. Here, binary coded genetic algorithm (GAs) was used to optimize structure and premise parameters of fuzzy model. And sequential optimization method by means of GAs was studied. It includes two optimization procedures such as structural and parametric identification. First the structural optimization is carried out and then using the results of structural optimization, the parametric optimization is executed. Yet the problem of finding “good” initial parameters of the fuzzy sets in the rules remains open. This study concentrates on optimization of information granulation-oriented ANFIS-based fuzzy model. We propose to use hierarchical fair competition based parallel genetic algorithm (HFCGA) for optimization of ANFIS-based fuzzy model. Based on the IG and the HFCGA, the simultaneous optimization method is introduced. In the ANFIS-based fuzzy model, there are two classes of variables to be optimized. Structural optimization involves the number of input variables, a collection of specific subset of input variables, the number of membership functions per input D. Liu et al. (Eds.): ISNN 2007, Part III, LNCS 4493, pp. 225–230, 2007. © Springer-Verlag Berlin Heidelberg 2007
226
J.-N. Choi, S.-K. Oh, and K.-S. Seo
variable, and the polynomial type of the consequence part of fuzzy rules, the parametric optimization concerns apexes of membership functions used in the premise part of the fuzzy. All of two classes of variables are optimize through HFCGA, C-Means clustering and LSM at once.
2 Design of ANFIS-Based Fuzzy Model Based on Information Data Granulation In the premise part of the rules, we confine ourselves to a triangular type of membership functions whose parameters are subject to some optimization. The CMeans clustering [7] helps us organize the data into cluster so in this way we capture the characteristics of the experimental data. In the regions where some clusters of data have been identified, we end up with some fuzzy sets that help reflect the specificity of the data set The identification of the premise part is completed in the following manner. Given is a set of data U={x1, x2, …, xl ; y}, where xk =[x1k, …, xmk]T, y =[y1, …, ym]T, l is the number of variables and , m is the number of data. [Step 1] Arrange a set of data U into data set Xk composed of respective input data and output data. Xk=[xk ; y]
(1) T
Xk is data set of k-th input data and output data, where, xk =[x1k, …, xmk] , y =[y1, …, ym]T, and k=1, 2, …, l. [Step 2] Complete the C-Means clustering to determine the centers (prototypes) vkg with data set Xk. [Step 3] Partition the corresponding isolated input space using the prototypes of the clusters vkg. Associate each clusters with some meaning, say Small, Big, etc. [Step 4] Set the initial apexes of the membership functions using the prototypes vkg. After premise part of identification, we identify the structure considering the initial values of the polynomial functions based on the information granules realized for the consequence and antecedents parts. [Step 1] Find a set of data included in the fuzzy space of the j-th rule. [Step 2] Compute the prototypes Vj of the data set by taking the mean of each rule.
V j = {V1 j , V2 j , … , Vkj ; M j }
(2)
[Step 3] Set the initial values of polynomial functions with the center vectors Vj.
The identification of the conclusion parts of the rules deals with a selection of their structure (type 1, type 2, type 3 and type 4) that is followed by the determination of the respective parameters of the local functions occurring there. The conclusion part of the rule that is extended form of a typical fuzzy rule in the TSK (Takagi-Sugeno-Kang) ANFIS-based fuzzy model has the form R j : If x1 is A1c and
and xk is Akc then y j − M j = f j ( x1 ,
In case of Type 3 (Quadratic Inference):
, xk )
(3)
Simultaneous Optimization of ANFIS-Based Fuzzy Model Driven
f j = a j 0 + a j1 ( x1 − V1 j ) +
+ a jk ( xk − Vkj ) + a j ( k +1) ( x1 − V1 j ) 2 +
+ a j (2 k +1) ( x1 − V1 j )( x2 − V2 j ) +
+ a j (2 k ) ( xk − Vkj ) 2
+ a j (( k + 2)( k +1) / 2) ( xk −1 − V( k −1) j )( xk − Vkj )
227
(4)
The calculations of the numeric output of the model, based on the activation (matching) levels of the rules there, rely on the following expression n
∑w y = *
n
j =1 n
∑
∑w
ji yi
=
ji ( f j ( x1,
∑
w ji
w ji
j =1
w ji
∑ wˆ
ji ( f j ( x1,
, xk ) + M j )
(5)
j =1
j =1
, wˆ ji =
n
∑w
n
=
n
j =1
where, wˆ ji =
, xk ) + M j )
j =1
ji
A j1 ( x1i ) ×
× A jk ( xki )
n
∑A
j1 ( x1i ) ×
× A jk ( xki )
j =1
The consequence parameters ajk can be determined by the standard least-squares method that leads to the expression
3 Hierarchical Fair Competition–Based Parallel Genetic Algorithm and Simultaneous Optimization of ANFIS-Based Fuzzy Model One of the central problems in evolutionary computation is to combat premature convergence and to achieve balanced exploration. Parallel Genetic Algorithm (PGA) is devised to solve this problem, and there are various PGA models such as global, fine-grained, and coarse-grained model [8]. The most popular model is coarse-grained model and Hierarchical Fair Competition model (HFC) is one type of PGA. It has multiple-deme (subpopulation), individuals evolve within each deme independently, and specified individuals migrate to other deme in regular generation interval [9]. Evolutionary process is similar to traditional GAs, but it include migration algorithm. In HFCGA, migration is executed in regular generation interval. Fig. 1 show the migration topology of HFCGA
Fig. 1. The migration topology of HFCGA (In case when the number of demes is 4)
228
J.-N. Choi, S.-K. Oh, and K.-S. Seo
The optimization of ANFIS-based Fuzzy model is carried out through proposed the simultaneous optimization method. In the sequential optimization method, first the structure identification is carried out and then the parameter identification is implemented based on results obtained in the structure identification. For structure identification, we search the structurally optimized model using the fixed membership parameters, and then find the parametrically optimized model via the parameter identification using the results obtained by the structurally optimized model. So, the boundary range of search space for parameter identification to find the optimized ANFIS-based fuzzy model is restricted within a certain area for structure identification. In order to alleviate the problem, the simultaneous optimization method is proposed. Genes for the structure and parameter identification of the ANFIS-based fuzzy model are arranged only within a chromosome. In the sequel, we can expand the boundary range of search space to find an optimized model. Hence by using the structurally as well as parametric optimization procedure simultaneously based on HFCGA, we can obtain the optimized ANFIS-based fuzzy model. Fig. 2 depicts the arrangement of chromosomes for the simultaneous optimization. Num ber of input variables to be used in fuzzy m odel
Num ber of variables
Inpu t variable
G enes for in put variables to b e selected
Input variable
...
Genes fo r num ber of m em bership functions according to selected inp ut variab les
input variable
Num ber o f MFs
Num ber of M Fs
Genes for structure identification
Param eters for 1st input variable
...
Order o f polynom ial
N um ber o f Order of MFs Polynom ial
G enes for param eter identification
Param eters for 2nd in put variable
Param eters for m th in put variable
...
Apexes of m em bership functions for second input variable First apex
Second apex
Num ber o f m em bership fu nctio ns is 2
First apex
Second apex
T hird ap ex
Num ber o f m em b ership functions is 3
...
First apex
...
n th apex
Num b er of m em bership functions is n
Fig. 2. Arrangement of chromosomes for simultaneous optimization
4 Experimental Studies In this section, we provide numerical examples to evaluate the advantages and the effectiveness of the proposed approach. We deal with the NOx emission process data of gas turbine power plant. Till now, almost NOx emission processes are based on ‘standard’ mathematical model in order to obtain regulation data from control process. However, such models do not develop the relationships between variables of the NOx emission process and parameters of its model in an effective manner. A NOx emission process of a GE gas turbine power plant located in Virginia, USA, is chosen in this experiment.[11-13] Using NOx emission process data, the regression equation reads in the form y = −163.77341− 0.06709x1 + 0.00322x2 + 0.00235x3 + 0.26365x4 + 0.20893x5
(6)
In the sequential and simultaneous optimization process, the number of input variables to be selected is confined to the range of two to five (2-4), the number of membership bound is two and three (2-3), and the polynomial order of the consequent
Simultaneous Optimization of ANFIS-Based Fuzzy Model Driven
229
part of fuzzy rules is chosen from four types, that is Types 1-4. Fig. 3 depicts traces of performance indexes when running sequential and simultaneous optimization base on HFCGA. In Fig. 4, the upper parts of the figure depict cluster groups and central values generated through C-Means clustering for each selected input variable, where central values are used to design the IG based fuzzy model. They are also used as initial apexes of the membership functions in the case of structure optimization in sequential method. The lower parts represent the tuned apexes of membership function carried out by simultaneous optimization base on HFCGA. Table 1 summarizes the results of comparative analysis of the proposed model with respect to other constructs.
Fig. 3. Performance index for sequential method and simultaneous method
Initial m e m b ersh ip fu nctio ns b y H C M clu ste rin g T u n ed m em be rship fun ction by H F C G A
Initial m em b ers h ip fu nc tio ns b y H C M c lu s terin g T u ne d m e m be rs hip fun c tion by H F C G A
(C) CDP vs. NOx
(D) TET vs. NOx
Fig. 4. Results of the C-Means clustering and the tuned apexes of the membership functions by means of simultaneous optimization method base on the HFCGA Table 1. Summary of performance of various intelligent models
Model Regression model Hybrid FR-FNNs [11] Multi-FNN [12] SOFPNN [13] Sequential Our model Simultaneous
2 (Linear) 2 (Linear) Second layer 3 (Quadratic) 2 (Linear)
Performance index PI E_PI 17.68 19.23 0.080 0.190 0.720 2.025 0.012 0.094 0.0117 0.0670 0.00043 0.01221
230
J.-N. Choi, S.-K. Oh, and K.-S. Seo
5 Conclusions In this study, we have developed a simultaneous optimization method for structure and parameters of fuzzy inference system that is based on Hierarchical Fair Competition-based Genetic Algorithms (HFCGA) and information data granulation. The HFCGA is used as optimization algorithm for ANFIS-based fuzzy model. Information granulation based on the C-Means clustering helps determine several parameters of ANFIS-based fuzzy model such as prototypes to be used in the consequence part of the fuzzy rules and the range of search space for parameters of premise part being used in HFCGA. Also, the structure and parameter of ANFISbased fuzzy model are optimized through simultaneous optimization methodology. The experimental studies showed that performance is better than some other previous models. The proposed model is effective for nonlinear complex systems, so we can construct a well-organized model. Acknowledgement. This work was supported by the Korea Research Foundation Grant funded by the Korean Government (MOEHRD)(KRF-2006-311-D00194).
References 1. Tong RM.: Synthesis of Fuzzy Models for Industrial Processes. Int. J Gen Syst. 4 (1978) 143-162 2. Pedrycz, W.: An Identification Algorithm in Fuzzy Relational System. Fuzzy Sets Syst. 13 (1984) 153-167 3. Takagi, T., Sugeno, M.: Fuzzy Identification of Systems and Its Applications to Modeling and Control. IEEE Trans. Syst, Cybern. SMC-15(1) (1985) 116-132 4. Oh, S.K., Pedrycz, W.: Identification of Fuzzy Systems by Means of an Auto-Tuning Algorithm and Its Application to Nonlinear Systems. Fuzzy Sets and Syst. 115(2) (2000) 205-230 5. Pderycz, W., Vukovich, G.: Granular Neural Networks. Neurocomputing. 36 (2001) 205224 6. Krishnaiah, P.R., Kanal, L.N., editors.: Classification, Pattern Recognition, and Reduction of Dimensionality, volume 2 of Handbook of Statistics. North-Holland, Amsterdam. (1982) 7. Lin, S.C., Goodman, E., Punch, W.: Coarse-Grain Parallel Genetic Algorithms: Categorization and New Approach. IEEE Conf. on Parallel and Distrib. Processing. Nov. (1994) 8. Hu, J.J., Goodman, E.: The Hierarchical Fair Competition (HFC) Model for Parallel Evolutionary Algorithms. Proceedings of the 2002 Congress on Evolutionary Computation: CEC2002. IEEE. Honolulu. Hawaii. (2002) 9. Lyu, M.R.: Handbook of Software Reliability Engineering. McGraw-Hill, New York. (1995) 510-514 10. Park, H.S., Oh, S.K.: Fuzzy Relation-Based Fuzzy Neural-Networks using a Hybrid Identification Algorithm. International Journal of Control, Automation, and Syst. 1(3) (2003) 289-300 11. Park, H.S., Oh, S.K.: Multi-FNN Identification Based on HCM Clustering and Evolutionary Fuzzy Granulation. International Journal of Control, Automation, and Syst. 1(2) (2003) 194-3202 12. Oh, S.K., Pedrycz, W.: A New Approach to Self-Organizing Multi-Layer Fuzzy Polynomial Neural Networks Based on Genetic Optimization. Advanced Engineering Informatics. 18 (2004) 29-39
