A Study on Nonlinear Model Identification Using Pseudo-Bacterial

A Study on Nonlinear Model Identi cation Using Pseudo-Bacterial Genetic Algorithm Norberto Eiji Nawa and Takeshi Furuhashi Laboratory of Bio-Electronics, Department of Information Electronics, School of Engineering Nagoya University Furo-cho, Chikusa-ku, Nagoya 464-01, JAPAN E-mail:

[email protected]

Abstract

This paper presents a comparative study of the PseudoBacterial Genetic Algorithm (PBGA). The PBGA proposed by the authors is ecient in improving local portions of chromosomes. The PBGA has been considered to be ef cient in the case where each locus in the chromosome has weak relationships with other loci. A rule base of a fuzzy inference system can be considered to be one of these cases. The weak relationships among fuzzy rules can be seen in the sense that the output is calculated using multiple rules, but each one of the rules still represents a unit with its own meaning and that can be locally improved. Numerical experiments are done to show the eectiveness of the PBGA and also the conjunct eect of the adaptive operator. The adaptive operator de nes the partitions for the bacterial operation and the crossover points according to the moving average of the truth values of the fuzzy rules. The results show the bene ts obtained with the combination of the PBGA with the adaptive operator.

1 Introduction Genetic Algorithms (GA) represent a class of optimization techniques inspired by the evolutionary process of biological organisms in Nature. Since the seminal work of Holland [1975/1992], GAs have been used in an uncountable number of dierent elds, including several attempts of synergetically combining the methodologies of so called Soft Computing (fuzzy logic, evolutionary computation, neural computation, etc.) in hybrid systems. Among the hybrid systems, the combination of fuzzy systems with evolutionary computation techniques has received increasing attention in recent times [Alander, 1996]. Fuzzy systems can represent non-linear systems using linguistic variables in a straightforward form when enough knowledge about the object system is available. However, the necessity of using a hybrid system arises when the necessary knowledge is incomplete or can not be easily described by an expert. GAs have been used in dierent ways and applications to perform the task of de n-

ing or optimizing the fuzzy rules and the membership functions of the variables [Furuhashi et al., 1995; Hashiyama et al., 1995; Homaifar and McCormick, 1995; Karr et al., 1989; Karr, 1991; Yuan and Zhuang, 1996]. In the previous works, the Pseudo-Bacterial Genetic Algorithm (PBGA) was applied for the discovery of fuzzy rules [Furuhashi et al., 1995], extraction of personal features for signature veri cation [Yang et al., 1995] and design of fuzzy controllers [Hashiyama et al., 1995]. The PBGA implements a local improvement mechanism based on the genetic recombination of bacterial genetics. It is ecient in improving local portions of chromosomes. The chromosomes are divided into several parts and each one of them is improved by the bacterial mutation of the PBGA. The authors have proposed an adaptive operator [Nawa et al., 1997a, Nawa et al., 1997b] to determine the parts in the chromosomes for the bacterial operation and also the cutting points for the crossover operator. This paper presents a comparative study between the PBGA, the PBGA combined with the adaptive operator and a GA with a radical local improvement mechanism (RLIM-GA) and a GA with a hill-climbing method (HCGA) for the task of building a fuzzy model of a non-linear equation. The adaptive operator decides the parts/cutting points according to the degrees of truth values of the fuzzy rules. It is expected to work eectively for the task of discovering fuzzy rule sets with less numbers of irrelevant rules, i.e. unused parts of chromosomes.

2 PBGA 2.1

Bacterial Genetics

The process of bacterial recombination that inspired the PBGA is the following: Bacteria can transfer DNA to recipient cells through mating. Male cells transfer strands of genes to female cells.

After that, those female cells acquire characteristics of male cells and transform themselves into male cells. By these means, the characteristics of one bacteria can be spread among the entire bacteria population. Another analogy is possible. Bacteriophages can carry a copy of a gene from a host cell and insert it into the chromosome of an infected cell. This process is called transduction. By transduction, it is also possible to spread the characteristics of a single bacterium to the rest of the population. These genetic recombination mechanisms have con gured a process of microbial evolution. Mutated genes can be transferred from a single bacterium to others and lead to a rapid evolution of the entire population. 2.2

Algorithm Description

A similar process to the bacterial genetics is implemented in the PBGA. Its algorithm is brie y described as follows: 1.

Generation of the initial population: n

2.

Genetic Operations:

mosomes are created and evaluated; (a)

Bacterial operation:

chro-

This genetic operation is applied to each chromosome one by one. Suppose there are p parts in a chromosome. The rst chromosome is chosen and it is reproduced in m clones. Portions within the i-th part (randomly chosen) of m-1 clones are mutated. The elite individual among the m chromosomes is selected and the i-th part of the selected chromosome is transferred to the m-1 chromosomes. On this stage, the i-th part of all the clones is replaced by the i-th part of the selected chromosome. This process, mutationevaluation-selection-replacement, is repeated. The mutation is applied to another randomly chosen part, dierent from the already chosen ones. When all the p parts have been tested, the best chromosome from the m individuals is selected to remain in the population and the other m-1 individuals are deleted. This genetic operation is applied to all the n chromosomes in the population. (b) Conventional genetic operations: The chromosomes with lower tness values are deleted and some randomly chosen chromosomes from the remaining group are reproduced. Chromosomes are mated and osprings are generated by crossover.

3.

2.3

Stop condition:

If the stop condition is satis ed, stop, otherwise, go back to 2. Adaptive Operator

In the previously described process, the division points that determine the parts of the chromosomes were arbitrarily xed during the whole search process. Also, the crossover points in the chromosome were randomly chosen. In order to improve the performance of identi cation of fuzzy models by the PBGA, the adaptive operator was introduced. The adaptive operator is used to determine the division points when the local improvement operation of the PBGA is to be applied. The probability of a certain locus to be selected is inversely proportional to the degree of truth value of the fuzzy rule it encodes. For each chromosome, the moving average of truth values of the fuzzy rules is calculated. The accumulated truth value of a rule is the sum of the truth values for each one of the entries in the training data. The moving average of the rule j , Mj can be de ned as: Mj

=

Tj 02

+ Tj 01 + Tj + Tj +1 + Tj +2 5

(1)

where Tj is the accumulated truth value of the fuzzy rule j . The accumulated truth value of a fuzzy rule is a measure of its quality. If a rule possesses a high value of accumulated truth value, it means that the rule was intensively and frequently red during the PBGA process. Consequently, this is an evidence of the eectiveness and utility of that rule. On the other hand, if a rule possesses a low value of accumulated truth value, this is an indication that the rule does not play an important role in the system. The motivation for using the moving average of the accumulated truth value is to identify portions of good quality rules in the chromosome. Those strings having high moving averages are considered to have a high concentration of good quality rules. These strings are desirable to be preserved. On the other hand, blocks with low moving averages have higher probabilities of being mutated. The crossover points are also adaptively determined according to the moving averages of the accumulated truth values of the fuzzy rules. The lower the moving average, the higher the probability of being selected as the crossover point. This selection of crossover points works well to build blocks with eective fuzzy rules. Eective fuzzy rules are highly desired in order to build more compact models.

3 Encoding Method Each chromosome in the population encodes the rules of the rule base of the fuzzy model as well as the membership functions of the variables. The lengths of the chromosomes are variable. Each rule contains information about the referred variables in the antecedent and consequent parts. The data about each of the membership functions are also encoded in each locus. In this paper, the membership functions are triangular, so their parameters are the pairs (center, width).

4 Experiments and simulation results Comparative experiments were performed in order to test the suitability and eectiveness of the PBGA with adaptive operator to the task of designing a fuzzy model of the following non-linear equation: y

= x1 + x02:5 + x3 3 x4 + 2 3 e23(x5 0x6 )

(2)

The train and test set had entries sampled from the following intervals: x1 [1 . . . 5], x2 [1 . . . 5], x3 [0 . . . 4], x4 [0:0 . . . 0:6], x5 [0:0 . . . 1:0] and x6 [0:0 . . . 1:2]. The function y (x1 ; x2 ; x3 ; x4 ; x5 ; x6 ) was modeled on 7 fuzzy variables, 6 variables on the function plus one dummy variable. 80 6-tuples were used as training data and another 80 entries were used as test data. The population of chromosomes was xed in 30 individuals and the number of clones produced in the bacterial operation was arbitrarily xed to 4. The number of parts was set to 3. Single point crossover generated half of the individuals of the new population; the cutting points were chosen at random, except in the case of the adaptive operator. Tournament selection was used to complete the other half of the population. In the case of the PBGA, the number of rules and the rules within one part that will suer mutation were randomly decided. A rule can have any of its data modi ed. When using the adaptive operator, the probability that one rule will suer mutation is inversely proportional to its degree of moving average of truth value. The performance index (P I ) used in the simulation session is de ned as follows:. PI

=

1 nentries

X

nentries

i=1

jy 0 y3 j + N umRules 3 ! i

i

yi

N umM AX

r

(3) where nentries is the number of entries in the train/test sets, yi is the desired output value, yi3 is the inferred value by the model, N umRules is the

number of fuzzy rules of the model, N umM AX is the maximum number of rules allowed to one chromosome and !r is an assigned weight. N umM AX was arbitrarily set to 50 and !r was set to 0.1 after some parameter adjusting experiments. The second term in P I increases the pay-o of the PBGA to models with fewer rules. Comparative experiments were performed in order to test the suitability and eectiveness of the combination of the PBGA with the adaptive operator. The GA with a radical local improvement mechanism (RLIM-GA) has a similar dynamics to the PBGA, however, the chromosomes are not divided in parts and instead of the bacterial operation, each fuzzy rule is independently improved through mutation, before the other genetic operations are performed. The GA with a hill-climbing method (HCGA) is identical to the PBGA, but it diers on the point that the entire chromosome is considered to be a single part. The weak relationships among fuzzy rules are characterized by the fact that the output value of a fuzzy system is calculated from contributions of multiple rules. However, the fuzzy rules themselves still represent unitary entities that can be clearly identi ed from a chromosome. TABLE I. Error of the built models(average of 50 runs) HCGA RLIM-GA PBGA PBGA+Adap.Op

train set 23.96% 14.52% 26.92% 10.81%

test set 24.17% 19.08% 27.60% 14.43%

Table I presents the errors of the models for the train and test sets. Two points can be understood from Table I. First, the combination of the PBGA with the adaptive operator performs better than the other methods, concerning both the train and the test sets. This is an indication of the eectiveness of the adaptive operator, especially when considering the considerable dierence in performance between the PBGA and PBGA with adaptive operator. The adaptive operator bases its actions on the moving average of the truth values of the fuzzy rules and combined with the PBGA it was able to bring the best results concerning the model error. The second point that can be taken from Table I is that the weak relationships argued to be present among fuzzy rules do really make a dierence: the RLIM-GA improves each rule independently but it does not perform better than the PBGA with adaptive operator, which improves parts of chromosomes, making use of those weak interactions.

TABLE II. Fuzzy rules of the built models (average of 50 runs) HCGA RLIM-GA PBGA PBGA+Adap.Op

(1) 17.82 23.82 18.42 16.32

(2) 43.78% 37.88% 41.37% 28.39%

Column (1) in Table II shows the average of the number of rules of the built models. Column (2) shows the percentage of non-used rules of the fuzzy models when the test data is applied to the models. From Column (1) it is possible to see that the combination of PBGA with the adaptive operator builds the most compact models among the four methods. Considering that it also builds the best models in terms of error, this is an indication that the fuzzy rules obtained are relevant rules, i.e., with a few number of rules a relative good performance is achieved. Column (2) shows that, when unknown test data is applied to the the models obtained by the PBGA with the adaptive operator, the percentage of non-used rules is comparatively smaller than the ones obtained with other methods, indicating that the obtained rules generalize better. TABLE III. Standard Deviations of the models (average of 50 runs)

HCGA RLIM-GA PBGA PBGA+Adap.Op

Error train test 0.070 0.060 0.058 0.056 0.107 0.841 0.015 0.025

Number of rules 11.942 9.209 13.134 6.300

Table III shows the standard deviations of the obtained results for the error of the built models when using the train data and test data, respectively, and the standard deviation for the number of rules of the built models. It can be seen that the variance of the results is less when using the adaptive operator combined with the PBGA.

5 Conclusions This paper presented a comparative study between the PBGA, PBGA with adaptive operator, GA with a radical local improvement mechanism (RLIM-GA) and the GA with a hill-climbing method (HCGA). The obtained results for the task of building a fuzzy model from numerical

data indicates that relevant fuzzy rules can be obtained when using the adaptive operator combined with the PBGA. The adaptive operator had an eective action for increasing the percentage of active rules in the built models. This led to more compact fuzzy models with better quality rules. GAs are basically tness function-oriented search methods, consequently additional mechanisms are necessary in order to construct not only good fuzzy models in terms of performance but also compact models and with optimized rules. Although the rules represent units by themselves, weak forces link them together, disabling attempts of optimizing them independently.

References [Alander, 1996] J.T.Alander,\Indexed Bibliography of Genetic Algorithms and Fuzzy Systems", University of Vaasa, Department of Information Technology and Production Economics, ftp://ftp.uwasa.fi/cs/report94-1/gaFUZZYbib.ps.Z, 1996. [Furuhashi et al., 1995] T.Furuhashi, Y.Miyata, K.Nakaoka, Y.Uchikawa, \A New Approach to Genetic Based Machine Learning for Ecient Finding of Fuzzy Rules", Lecture Notes in Arti cial Intelligence, Vol. 1011, pp.173-189, 1995. [Hashiyama et al., 1995] T.Hashiyama, T.Furuhashi, Y.Uchkawa, \A Study on Finding Fuzzy Rules for SemiActive Suspension Controllers with Genetic Algorithm", Proc. of the IEEE Int. Conf. on Evolutionary Computation 95, Perth, Australia. [1975/1992] J.H.Holland,\Adaptation in Natural and Arti cial Systems", Cambridge, MA: MIT Press. (First edition 1975, Ann Arbor: University of Michigan Press.) [Homaifar and McCormick, 1995] A.Homaifar and E.McCormick, \Simultaneous Design of Membership Functions and Rule Sets for Fuzzy Controllers Using Genetic Algorithms", IEEE Transactions on Fuzzy Systems, Vol.3, No.2, pp.129-139, 1995. [Karr, 1991] C.L.Karr, \Design of an Adaptive Fuzzy Logic Controller Using a Genetic Algorithm", Proc. of the 4th Intl. Conference on Genetic Algorithms, pp.450457. [Nawa et al., 1997a] N.E.Nawa, T.Hashiyama, T.Furuhashi, Y.Uchikawa, \A Study on Fuzzy Rules Discovery Using Pseudo-Bacterial Genetic Algorithm with Adaptive Operator", Proc. of IEEE Int. Conf. on Evolutionary Computation 1997, Indianapolis, USA. [Nawa et al., 1997b] N.E.Nawa, T.Hashiyama, T.Furuhashi, Y.Uchikawa, \Fuzzy Logic Controllers Generated by Pseudo-Bacterial Genetic Algorithm", Proc. of IEEE International Conference on Neural Networks 1997, Houston, USA. [Yang et al., 1995] X.Yang, T.Furuhashi, K.Obata, Y.Uchikawa, \A Study on Signature Veri cation Using a New Approach to Genetic Based Machine Learning", Proc. of the Int. Conf. IEEE SMC'95, pp.4383-4386. [Yuan and Zhuang, 1996] Y.Yuan and H.Zhuang, \A genetic algorithm for generating fuzzy classi cation rules", Fuzzy Sets and Systems, 84, pp.1-19, 1996.