Neural Comput & Applic (2009) 18:633–641 DOI 10.1007/s00521-009-0289-4
ORIGINAL ARTICLE
An stable online clustering fuzzy neural network for nonlinear system identification Jose´ de Jesu´s Rubio Æ Jaime Pacheco
Received: 16 April 2007 / Accepted: 18 June 2009 / Published online: 8 July 2009 Ó Springer-Verlag London Limited 2009
Abstract In this paper, we propose a online clustering fuzzy neural network. The proposed neural fuzzy network uses the online clustering to train the structure, the gradient to train the parameters of the hidden layer, and the Kalman filter algorithm to train the parameters of the output layer. In our algorithm, learning structure and parameter learning are updated at the same time, we do not make difference in structure learning and parameter learning. The center of each rule is updated to obtain the center is near to the incoming data in each iteration. In this way, it does not need to generate a new rule in each iteration, i.e., it neither generates many rules nor need to prune the rules. We prove the stability of the algorithm. Keywords Fuzzy neural networks Clustering Nonlinear systems Identification Stability
1 Introduction Both neural networks and fuzzy logic are universal estimators, they can approximate any nonlinear function to any prescribed accuracy, provided that sufficient hidden neurons or fuzzy rules are available. Recent results show that the fusion procedure of these two different technologies seems to be very effective for nonlinear system identification [2]. In the last few years, the application of fuzzy neural networks to nonlinear system identification is very
J. J. Rubio (&) J. Pacheco Instituto Polite´cnico Nacional, ESIME Azcapotzalco, Seccio´n de Estudios de Posgrado e Investigacio´n, Av. de las Granjas no.682, Col. Santa Catarina, Delegacio´n Azcapotzalco, Me´xico, D.F., Mexico e-mail:
[email protected]
active area [10, 11]. Fuzzy modeling involves identification of structure and parameters. The second one is usually (and easily) addressed by some gradient descent variant, e.g., the least square algorithm and backpropagation. Structure identification is to select fuzzy rules; it often lies on a substantial amount of heuristic observation to express proper strategy knowledge. It often tackled by offline, trial and error approaches, like the unbiasedness criterion [12]. In offline identification, the update of the parameters and the structure take place only after the whole training data set has been presented, i.e., only after each epoch. Several approaches generate fuzzy rules from numerical data. One of the most common methods for structure initialization is uniform partitioning of each input variable into fuzzy sets, resulting to a fuzzy grid. This approach is followed in ANFIS [6]. In [1], the TKS was used for designing several neuro-fuzzy identifiers. This approach consists of two learning phases, structure learning which involves to find the main input variables of all the possible, specifying the membership functions, the partition of the input space and determining the number of fuzzy rules. Parameter learning involves the determination of unknown parameters and the optimization of the ready existing ones in the model, using some optimization method based on the linguistic information from the human expert and form the numeric data obtained from the actual system to be modeled. These two learning phases are interrelated, and none of them can be independent from the other one. Traditionally, these phases are done sequentially, the parameter updating is employed after the structure is decided. It is suitable only for offline operation. Most of structure identification methods are based on data clustering, such as fuzzy C-means clustering [5, 16], mountain clustering [11], subtractive clustering [3]. This approaches require that all input output data are ready
123
634
Neural Comput & Applic (2009) 18:633–641
before we start to identify the plant. So, the structure identification approaches are offline. In online identification, model structure and parameter identification are updated immediately after each input– output pair has been presented, i.e, after each iteration. Online identification also includes: (1) model structure and (2) parameter identification. There are a few online methods in the literature. In [7], the input space is partitioned according to a aligned clustering-based algorithm. After the number of rules is decided, the parameters are tuned by recursive least square algorithm, it is called SONFIN. In [8], it is the recurrent case of the above case, it is called RSONFIN. In [15], the input space is automatically partitioned into fuzzy subsets by adaptive resonance theory mechanism. Fuzzy rules that tend to give high output error are split into two, by a specific fuzzy rule splitting procedure. In [9], the author proposes that the radius to make clustering updates. In [19], the authors consider each group as a one rule, and each rule is trained by its group data, they give a time varying learning rate for backpropagation algorithm to prove the parameter learning error is stable. In this paper, we propose a new online clustering fuzzy neural network. Structure and parameter learning are updated at the same time in our algorithm, we do not make difference in structure learning and parameter learning. It generates groups with a given radius. The center is updated in order to obtain the center near to the incoming data in each iteration. In this way, it does not generates a new rule in each iteration, i.e., it neither generates many rules and nor need to prune the rules. We give a time varying learning rate for backpropagation training in the case of the centers and the widths in the IF part. We use extended Kalman filter to train the center of sets in the THEN part. We prove the stability in both cases.
2 Fuzzy neural networks for nonlinear identification Consider following unknown discrete-time nonlinear system yðk 1Þ ¼ f ½ Xðk 1Þ
ð1Þ
where Xðk 1Þ ¼ ½x1 ðk 1Þ; . . .; xN ðk 1Þ ¼ ½ yðk 2Þ; . . .; yðk n 1Þ; uðk 2Þ; . . .; uðk m 1Þ 2
