Fuzzy Rule-Based Controllers that Learn by Evolving their ... - CiteSeerX

Fuzzy Rule-Based Controllers that Learn by Evolving their Knowledge Base.?

Luis Magdalena ([email protected]) Juan R. Velasco ([email protected]) ETSI Telecomunicacion, Universidad Politecnica de Madrid, Ciudad Universitaria s/n., E-28040 Madrid, Spain.

Abstract. Fuzzy Logic Controllers may be considered as knowledge-based systems, incorporating human knowledge into their Knowledge Base through Fuzzy Rules and Fuzzy Membership Functions. The de nition of these Fuzzy Rules and Fuzzy Membership Functions is actually aected by subjective decisions, having a great in uence over the performance of the Fuzzy Controller. In recent years, eorts have been made to obtain an improvement on system performance by incorporating learning mechanisms to modify prede ned rules and/or membership functions. Genetic Algorithms are probabilistic search and optimization procedures based on natural genetics. This chapter describes two dierent approaches to apply Genetic Algorithms to Fuzzy Logic Controllers whose Rule Base is de ned through a set of Fuzzy Rules. The use of a set of Fuzzy Rules (and not a Fuzzy Relational Matrix or a Fuzzy Decision table) is adapted to complex control applications containing a large number of variables, since it reduces the dimensionality of the Knowledge Base for these systems. The rst approach uses the Knowledge Base (containing a set of Fuzzy Rules and a set of Membership Functions) as the individual of the genetic system, working with a population of Fuzzy Controllers. The second one uses the Knowledge Base of the controller as the population of the genetic system (a single rule containing the description of the corresponding Fuzzy Sets is an individual of the population). Both systems have been applied to complex control problems (gait synthesis for biped robots and fossil power plants). Application examples cover o-line and on-line optimization, and knowledge diversi cation problems. Keywords. Fuzzy logic controllers, genetic algorithms, machine learning, rulebased systems ?

This work has been partially supported by the European Community and the Spanish Ministerio de Industria y Energa (Project PASO-095 - CORAGE), and CICYT (Spanish Comision Interministerial de Ciencia y Tecnologa) (Project TAP93-0971E)

1 Introduction Control engineers have found in Fuzzy Logic a powerful tool for coping with a wide range of complex systems. As a result, Fuzzy Logic Controllers [12] (FLCs) are being widely and successfully applied to dierent areas. Generally, the control algorithm of a conventional controller is analytically described by a set of equations. In the case of FLCs, this set of equations is replaced by a knowledge based algorithm. Fuzzy Logic Controllers can be considered as knowledge-based systems, incorporating human knowledge into their Knowledge Base through Fuzzy Rules and Fuzzy Membership Functions (among other information elements). The de nition of these Fuzzy Rules and Fuzzy Membership Functions is actually aected by subjective decisions, having a great in uence over the performance of the Fuzzy Controller. When designing an FLC an expert capable of providing the knowledge to be included in the controller is needed. If this is not possible, or the available knowledge is not good enough, the de nition or the re nement of the knowledge will be performed by means of a learning or adaptation process. Dierent interpretations have been applied when describing the characteristics and performances of FLCs. From the point of view of knowledge-based systems, FLCs can be interpreted as a particular type of real time expert systems. A second interpretation more suited to the analysis of the control properties of the FLC, considers FLCs as non-linear, time-invariant control laws. In addition, recent works have demonstrated the ability of Fuzzy Controllers to approximate continuous functions on a compact set, with an arbitrary degree of precision; dierent kinds of FLCs are universal approximators ([2, 3, 11]). Combining ideas related to these dierent interpretations, eorts have been made to obtain an improvement in system performance (a better approximation to an optimal controller, with a certain performance criterion) by incorporating learning mechanisms to modify prede ned rules and/or membership functions, represented as parameterized expressions. The main goal is to combine the ability of the system to incorporate experts' knowledge from a knowledge-based system point of view (knowledge engineering), with the possibility of ne tuning this knowledge by applying learning (machine learning) or adaptation (adaptive control) techniques through the analytical representation of the FLC. Ideas arising out of two main areas have been applied with this aim: ideas coming from Arti cial Neural Networks (ANNs) and from Genetic Algorithms (GAs). When applying ideas coming from ANNs ([17]), the learning techniques basically use the topological properties of IRn (as the properties of gradient), where IRn represents the space of parameters of the controller (an annotated bibliography on this topic is contained in [16]). GAs ([5, 7]) are probabilistic search and optimization procedures based on natural genetics, working with nite strings of bits that represent the set of parameters of the problem, and with a tness function for evaluating each one of these strings. The nite strings used by GAs may be considered as a representation of elements of IRn , but usually, the learning mechanisms make no use of this space of parameters' topological properties. This chapter describes the application of evolutionary strategies, with a learn-

ing purpose, to the Knowledge Base of Fuzzy Controllers. An introduction to the main characteristics of Fuzzy Logic Controllers is presented in Section 2, with special attention paid to the Rule-Based representation of FLCs. Since the evolutionary strategies are applied to the knowledge base of the FLC, the FLC is described in terms of knowledge-based expert systems. Section 3 focuses on the question of learning in FLCs, with special attention paid to Genetic learning. Sections 4, 5 and 6 introduce the main concepts of the Knowledge Base Evolution Approach, with some applications described in Section 7.

2 Fuzzy Controllers The FLC is described in terms of knowledge-based expert systems. From this point of view, three main questions are raised in the de nition or implementation of an FLC: the knowledge representation, the reasoning strategy, and the knowledge acquisition. Knowledge representation and reasoning strategy are brie y discussed in this section. Knowledge acquisition is closely related to the speci c application of the controller. When applying a Fuzzy Logic Controller (or any other kind of controller) to a system (to be controlled), a certain desired behavior of the system is known (implicitly or explicitly) and applied throughout the designing process of the controller. In the case of an FLC, the knowledge contained in the knowledge base of the system, is obtained through a knowledge acquisition stage or after a learning or adaptation process, that is usually guided by the realization of this goal (to obtain the desired behavior). From the point of view of this work, knowledge acquisition plays a secondary role2, since the primary role corresponds to learning (knowledge evolution). A fuzzy rule based system is characterized by a set of rules (knowledge representation) and by the de nitions of some concepts related to information processing (reasoning strategy): the aggregation operators, the fuzzy connectives and the inference method. Rules are de ned by their antecedents and consequents. Antecedents, and frequently consequents, are associated with fuzzy concepts:

Rn : if x1 is A1n and : : : and xm is Amn then y is Bn :

(1)

Occasionally consequents are analytical functions of the input variables, in most cases linear functions:

Rn : if x1 is A1n and : : : and xm is Amn then y = fn (x1 ; : : : ; xm ): (2) In both cases xi are input variables, Aij are fuzzy sets related to input variables, y is the output variable, Bk are fuzzy sets related to the output variable, and fn 2

Obviously, this secondary role is only a way of saying that this work is particularly devoted to learning. Nevertheless, in these cases where any knowledge acquisition technique is applied to avoid starting the evolution from a random knowledge, the goodness of these initial information could lead the evolution process to better or faster results.

are functions of the input variables like the linear expression below

fn (x1 ; : : : ; xm ) = a0n + a1n x1 + : : : + amn xm :

(3)

The fuzzy connectives and, between fuzzy concepts, are usually implemented through the product or the minimum operators. Systems using the rst type of rule (expression 1) are usually called Mamdani type controllers [20], while those that use the second type of rule (expression 2) are usually named TSK (Takagi, Sugeno and Kang) fuzzy systems [26]. In some cases, the knowledge represented by the set of rules is characterized by means of a fuzzy relationship. A fuzzy relationship over the collections of fuzzy sets A1 , . . . , Am , B , is a fuzzy subset over their cartesian product, A1  : : :  Am  B . In a fuzzy relationship, each element of the cartesian product (A1i ; : : : ; Amj ; Bk ), has a grade of membership to the relation, represented by R (A1i ; : : : ; Amj ; Bk ) and taking values within the interval [0; 1]. Table 1 shows the tabular representation of a fuzzy relation for an FLC with one input (X ) and one output (Y ) variable, with three fuzzy sets (A11 ; A12 ; A13 ) related to the input variable and four fuzzy sets (B1 ; B2 ; B3 ; B4 ) related to the output variable.

Table 1.

A fuzzy relation R.

R B 1 B2 B3 B4 A11 0:5 0:8 0:2 0:0 A12 0:0 0:3 1:0 0:1 A13 0:0 0:0 0:3 1:0

The complete behavior of a fuzzy system may be characterized by a fuzzy relation that is the combination of the fuzzy relations de ned by each element of the rule set. This combination may be represented through the connective also,

R = also (R1 ; : : : ; Rn );

(4)

where also is usually implemented by the maximum operator (any t-conorm or s-norm may be used as the also connective), generating a fuzzy output. This fuzzy output is a fuzzy subset of Y , from which a non-fuzzy output is usually obtained (deuzi cation). When using consequents that are functions of the input variables, the also connective is implemented as a weighted sum of rules output, and no deuzi cation is needed. In addition to the representation of the behavior of the fuzzy system through a fuzzy relation or with a set of fuzzy rules, a third possibility is the representation using a fuzzy decision table. A fuzzy decision table represents a special case of a crisp relation (the ordinary type of relations we are familiar with) de ned over the collections of fuzzy sets corresponding to the input and output variables. Table 2 shows a fuzzy decision table for an FLC with two input (X1 ; X2 )

and one output (Y ) variable, with three fuzzy sets (A11 ; A12 ; A13 ; A21 ; A22 ; A23 ) related to each input variable and four fuzzy sets (B1 ; B2 ; B3 ; B4 ) related to the output variable.

Table 2.

A fuzzy decision table.

A21 A22 A23 A11 B1 B2 A12 B1 B2 B3 A13 B1 B3 B4

The modular structure of an FLC is: 1. A Fuzzi cation interface that transfers the values of input variables into fuzzy information, assigning grades of membership to each fuzzy set de ned for that variable. 2. A Knowledge Base that comprises a Data Base and a fuzzy control Rule Base. The Data Base is used in the fuzzi cation and defuzzi cation processes. 3. An Inference Engine that infers fuzzy control actions employing fuzzy implications and the rules of inference of fuzzy logic. 4. A Defuzzi cation interface that yields a non-fuzzy control action from an inferred fuzzy control action.

Fig. 1.

Structure of an FLC.

Figure 1 shows the de ned structure. The fuzzy systems described in the following have this structure.

3 Fuzzy Logic Controllers with Learning Fuzzy Logic Controllers have two main applications: regulation systems and supervisory systems. Regulation systems are applied to maintain a variable or variables of a controlled system at a certain value (a set point) or following a prede ned trajectory. Supervisory systems are those systems that de ne the set points or the trajectories for the variables, representing the inputs of the regulation systems. Supervisory systems are usually related to a higher level of knowledge and abstraction, and are closer to a knowledge based system approach. Learning techniques are particularly applicable to supervisory systems, optimization in regulation systems has been broadly studied from the view point of Control Theory. As in any other kind of knowledge-based control system, one of the most important questions when de ning or implementing a Fuzzy Logic Controller to solve a certain control problem (in the following only supervisory systems are considered), is the derivation of the knowledge base. The performance of the system is strongly related to the quality of the knowledge base, and consequently with the process of knowledge acquisition. In some cases, this process does not take place because there are no experts available in the eld of a certain problem. It is possible too that the acquired knowledge only represents an incomplete or a partially incorrect description of the control algorithm. When the obtained knowledge is not considered good enough to be used as the control algorithm of the system, some kind of learning or adaptation is needed. In this case a wide range of possibilities is open, from a complete learning process that starts with an empty or a randomly obtained knowledge base, to the ne tuning of a near optimal one. From this point, the concept of learning and the methodology for incorporating it into an FLC are discussed, focusing learning from the point of view of Genetic Algorithms. As a rst step to attain an adequate coupling between Fuzzy Logic Controllers and Genetic Algorithms, a certain parallelism of some of its concepts and components are established.

3.1 Coupling Fuzzy and Genetic systems As has been shown, rule-based control systems are processes with two signi cant features: 1. They have a known goal: to produce a certain desired behavior3 of the controlled system. 2. They use a set of rules as elements of the control algorithm applied to perform the control operation. Under these premises, a strong parallelism can be made with Genetic Algorithms ([6, 8]): conventional GAs (used as function optimizers) try to nd the 3

This behavior may be implicitly or explicitly described, or even expressed in a qualitative way.

optimum value (minimum or maximum) of a given function by searching through a population of possible parameters of the function. When learning is applied to rule-based control systems the objective is to nd the optimum for a certain performance criterion by searching through a population of possible knowledge bases of rules or associated knowledge. In order to specify this parallelism, let us go over the four main concepts of every Genetic Algorithm (GA) system: 1. Population:

{ Function optimizer. Population is made up of sets of the function's

parameters. Every set of parameters is encoded as a string of elements from a nite alphabet (usually, a binary alphabet). Every individual in the population has the same length and is encoded in the same way. Completeness and consistency are two main characteristics required for the encoding algorithm: every possible set of parameters must be able to be encoded and every encoded individual must be able to be decoded into a real set of parameters. { Rule-based control system. In a rst approach, every rule-based control system has a signi cant part that can be seen as a population which can evolve: the set of rules of the knowledge base. As is shown in later sections, this approach is not unique: a knowledge base can be seen as a non separable system, so several authors de ne a population of KBs. Moreover, in fuzzy rule-based systems, the de nitions of fuzzy sets can be seen as a population itself. The only compulsory aspect is that individuals must be encoded soundly to assure consistency and completeness. 2. Goal: { Function optimizer. The goal for this kind of system is to nd the best adapted individual of the population (the one that produces the optimum value for the function). Usually, the algorithm is nished when the population converges on a big group of identical individuals (around 90% of the population). { Rule-based control system. As previously described, two dierent kinds of controller can be found: regulators, whose goal is to set the value of a given variable to a xed point, and supervisors, that de ne or suggest these set points, while trying to x, maximize or minimize (goal) another variable (for instance, any performance criterion). Both genetic-fuzzy systems presented in this chapter address this second kind of controller. 3. Fitness function: { Function optimizer. The utility of this function is to assign a strength value to every individual in the population. Regarding the parallelism that GAs have with real life, this strength value represents how adapted the individuals are to their environment. Usually, this tness function is the function for optimizing itself, or a function of it. { Rule-based control system. With the idea that every genetic system has to bind a strength value to each individual in mind, rule-based controllers tness function measures how good or bad the individual is: in

other words, how well the control algorithm described by the rules directs the controlled system to reach its goal. Unfortunately, the problem is not as easy as in function optimizers. There are several aspects that must be decided while de ning the tness function: (a) The designed controller can have three dierent goals, always de ned in relation to an objective variable or function: to maintain it at a given value (usually, regulators) or to minimize=maximize it (for supervisor systems). These three objectives may be seen as a cost minimization problem, where the cost function is de ned as:

C (t) = V (t) for minimizing, C (t) = ?V (t) for maximizing, C (t) = jV (t) ? valuej for maintaining a given value. After this cost de nition, every goal is reduced to a minimization problem. (b) On the other hand, not every control system is connected in a closed loop with the controlled process. Sometimes (mainly for supervisory systems), control systems become suggestion systems for process operators, and even in the situation of a full connected system, contradictory (or hard to reach) points can be set by the supervisor and not realized by the regulators. In these cases, when suggested values are not really obtained, it is very hard to know how good or bad the controller is. Thus, some estimations must be made. (c) A third aspect is to select when the system does have to evaluate this tness function. It may be carried out every control cycle with a structure of: . . . {read data{evaluate previous actions{perform new actions{. . . , or it may be done at the end of a longer process. (d) The last aspect is related to the number of rules involved in the inference process. Every rule has its own in uence on the nal control action. If the controller works properly, it is the result of the coupled actions of every rule being used. So two dierent bindings can be made: rst, a single tness value for every rule, and second, a sort of decoupling process by using proportional tness, depending on the in uence of the rules. 4. Reproduction algorithms: { Function optimizer. Classic GAs have three reproduction operators: (a) Selection This operator picks the best individuals from the population in order to use them as parents for the new ospring. (b) Crossover Mixes genetic code (in classic GAs, this code is a binary string) to create new individuals. The most used method is to select a random cut point from a pair of individuals (the same for both) to divide them, and then recombine the left part from the rst individual with the right part from the second one, and vice-versa. (c) Mutation From time to time, one bit of a randomly selected individual is changed

After these three operations, the new population obtained replaces the previous one, and it is ready to be evaluated, assigned a tness value, etc. { Rule-based control system. Due to the dierences among possible rule-based genetic systems, reproduction algorithms are solution dependent. However, it is possible to nd several common aspects: (a) Replacing the whole population is not usual. Elitism is often used to avoid the best/better individuals being killed and replaced: GA selects a number of parents lower than the number of individuals, and after crossover and mutation, children replace bad individuals from the original population. This way of evolution is nearer to real life, where new population does not replace its parents. (b) Crossover and mutation are adapted to particular genetic codes. Nevertheless, it is very common to include a new operator (inversion) to reorder the elements from the rule or the rules from the rule base. If this reordering is not done, it will be more probable for some elements than for others, to join together in any individual of the ospring. In fact, the rst and the last element of an individual, will not be joined together in the children after traditional crossover.

3.2 Genetic Algorithms and Fuzzy Logic Controllers The systems enumerated in this Subsection follow the general ideas previously described. They incorporate knowledge, represented through Fuzzy Rules and Fuzzy Membership Functions, and they apply dierent genetic or evolutionary techniques to create new (better) knowledge. Each one uses its own coding scheme, some of them replacing the strings of bits with more complex data structures. The tness function, the rst generation (initial population) and the goal are related to the task for which each FLC was designed. Genetic operators are applied to Fuzzy Rules, Fuzzy Membership Functions or both of them simultaneously. In a rst paragraph, the various techniques employed to evolve Membership Functions are described. Three dierent representations for the contents of the Rule Base of an FLC have been described in Section 2: a set of rules, a decision table and a relational matrix. The last three paragraphs describe some applications of GAs to FLCs working with each of these three representations. For those works that apply GAs to modify simultaneously the Membership Functions and the Rules, the description of both mechanisms is split between the two corresponding paragraphs.

Evolving Fuzzy Membership Functions. To translate membership func-

tions to a representation useful as genetic material, these functions are parameterized with one to four coecients and each of these coecients constitutes a gene of the chromosome for the GA. The most broadly used parameterized

membership functions are: triangular, trapezoidal, Gaussian, bell and sigmoidal. These parameterized functions may be classi ed into two main groups: the piecewise linear functions (trapezoidal and triangular, Figure 2) and the dierentiable functions (Gaussian, bell and sigmoidal, Figure 3). Triangular functions are used in [4, 9, 22] (symmetric, Figure 2 center) and [13] (Figure 2 right); trapezoidal functions in [10] and dierentiable functions in [25] (radial functions) and [14] (Gaussian functions). Each coecient constitutes a gene of the chromosome that may be a binary code (representing the coecient) [4, 9, 10, 13, 25] or a real number (the coecient itself) [14, 22]. 1

0

Fig. 2.

a

b

c

d

b

a

a

b

c

Examples of parameterized piece-wise linear functions.

1

1

1

1/2

1/2

1/e

0

Fig. 3.

b a

0

ba

0

a

Examples of parameterized dierentiable functions.

Evolving sets of rules. The set of rules describing a certain control algorithm for a Fuzzy Controller may contain a xed number of rules, or a variable number of rules. A rule base de ned as a set with a xed number of rules is used by Liska and Melsheimer [14]. The membership functions (Gaussian, as described in the previous paragraph) are labeled, encoding each rule with an integer number (label) that de nes the membership function related to a certain input or output variable that is applied by the rule (membership functions for every variable are ordered). The fuzzy clauses composing the fuzzy rule have the structure Variable is Fuzzy Set, where the Fuzzy Set is de ned by two dierent methods, the direct

and the indirect de nition. When a direct de nition is applied, the expression of the membership function, or the values of the parameters de ning it, are directly included in rule's code. In the previously described case the Fuzzy Sets are indirectly de ned, the set of membership functions constitutes an independent knowledge, that is included in the rules by means of references, using labels. On the other hand, the following works include the code of the membership functions as a part of the code of each rule. All of them use a variable number of rules. Applying this structure, i.e., a variable number of rules and a direct de nition of the fuzzy membership functions, GAs are applied to TSK fuzzy controllers in [13] and [25]. The rules of this kind of controller contain fuzzy expressions on its antecedent and linear equations on its consequent. GAs are applied to obtain the parameters of the membership functions, and the coecients of the linear equations. One gene is used to code each coecient of the consequent: rules for a system with two input variables need three coecients to obtain the output value (as shown in expression 2). In [25], the consequents are constants needing a single coecient for each consequent. Rules with fuzzy inputs and fuzzy outputs (like those in expression 1) are used in [4].

Evolving decision tables. In [27], GAs are used to modify the decision table of an FLC, which is applied to control a system with one output and two input variables. A chromosome is obtained from the decision table by going row-wise and coding each output fuzzy set as an integer in 0, 1, . . . , n, where n is the number of membership functions de ned for the output variable of the FLC. Value \0" indicates that there is no output, and value \k" indicates that the output fuzzy set has the k-th membership function. Applying this code to the fuzzy decision table represented in table 2, the obtained string is (0,1,2,1,2,3,1,3,4). The mutation operator changes a fuzzy code either up a level or down a level, or to zero (if it is already zero, then it chooses a non-zero code at random). Evolving relational matrices. Occasionally GAs are used to modify the fuzzy relational matrix (R) of an FLC with one input and one output. In this case, a chromosome is obtained by concatenating the m  n elements of R, where m and n are the number of fuzzy sets associated with the input and output variable

respectively. The elements of R that make up the genes may be represented by binary codes [23] or real numbers [22]. Applying this method to the relation in Table 1, the resulting code is (0.5,0.8,0.2,0.0,0.0,0.3,1.0,0.1,0.0,0.0,0.3,1.0).

4 The Fuzzy Rule-Based Approach When applying GAs to FLCs, there are two basic decisions to be made: how to code the possible solutions to the problem as a nite bit string, and how to evaluate the merit of each string. When a non-classical code is employed, the decisions to be made before applying the genetic or evolutionary system to the fuzzy controller, are slightly dierent. The question of \how to code as a binary

string", splits into two questions: the structure of the chromosomes (the code), and the translation of the knowledge into that code (Paragraph 3.1-Population). An additional problem is the adaptation of the genetic operators to the new code, since in some cases, the classical operators are not directly applicable to the new structure of the chromosomes (Paragraph 3.1-Reproduction Algorithms). The following Sections propose some answers to those questions, but before that, some characteristics of the employed FLCs have to be analyzed. The way we code a possible solution as a bit string or as any other type of genetic material, constitutes the way we convert the problem de ned in the space of parameters of the controller (represented by IRn ) to a new problem de ned in a certain space where GAs operate. The characteristics of the FLC aect the dimension and the properties of the space of parameters where the adaptation or learning process occurs. As a result, this learning process becomes faster or slower and even treatable or untreatable. The obvious conclusion is that an adequate selection of the characteristics of the FLC is crucial for the whole learning process.

4.1 General structure of the FLC The general structure of the FLC is composed of a normalizer, a fuzzi er, an inference engine, a defuzzi er and a denormalizer, i.e., the scheme of Figure 1 with the addition of normalization and denormalization elements. The input variables are linearly normalized from their real interval to [-1,1] interval. The fuzzy sets have trapezoidal membership functions de ned through four parameters (Figure 2, left). The rules applied by the inference engine have a fuzzy antecedent and a fuzzy consequent (as in expression 1), and are described as a set of fuzzy rules. The output variables are linearly denormalized from [-1,1] interval, to their real interval.

Normalized membership functions. The use of normalized variables in fuzzi cation and defuzzi cation processes, and consequently the use of normalized membership functions, may be interpreted with two dierent meanings: 1. Fixed normalization. If the normalization limits are xed a priori, the only eect is conceptual, producing an FLC that works with variables de ned in a normalized universe of discourse. From the point of view of learning or adaptation of the membership functions, the use of normalized functions de nes the range of variation for its parameters (if parameterized), parameters that constitute the genetic material of the learning process. 2. Variable normalization. If the normalization limits are considered as adaptable parameters throughout the learning process, the obtained eect may be interpreted as that of applying a controller with a parameterized gain with respect to each input and output variable. The parameterized gain (and a certain oset) for each variable is directly related to the corresponding normalization limits, and may be tuned throughout the learning process.

Trapezoidal fuzzy sets. The use of trapezoidal membership functions is one of

the possible selections of parameterized functions. Dierent types of parameterized Fuzzy Membership Functions have been described previously, and classi ed into two main groups: the piece-wise linear functions (triangular and trapezoidal) and the dierentiable functions (Gaussian, bell and sigmoidal). The selection of a particular type of function is related to the main characteristics they have. While the rst group produces a reduced computational complexity for the fuzzi cation and defuzzi cation processes, the second one is particularly interesting when derivatives are involved in the learning process (as in most of the fuzzy-neural systems). In the applications presented in the following, the learning process does not involve any kind of derivative, and the computational complexity leads us to select piece-wise linear functions. From piece-wise linear functions, trapezoidal membership functions are selected since triangular membership functions may be represented as trapezoidal functions with punctual core.

Rule base as a set of rules. When applying GAs to FLCs, rules are generally

represented as a fuzzy decision table or in some cases (systems with one input and one output) as a fuzzy relational matrix. The use of a decision table has some advantages from the point of view of an easier analysis of completeness and consistency of the knowledge base. The idea of working with sets of rules has been applied recently as a way to solve dimensionality problems ([4, 14, 15, 25]) as is commented on in the following sections. Our systems applies this idea, working directly with the set of fuzzy rules. Once selected a set of rules representation, the fuzzy sets de ning the rule could be described through a direct (including the expression of the fuzzy sets into the code of the rules) or indirect (with an independent representation referenced by labels) representation. The indirect representation reduces the computational complexity of the controller, while the direct representation increases its exibility. Both possibilities are explored.

4.2 The individuals of the population Working with an FLC with this set of characteristics, two dierent learning structures are possible, those produced by considering a system closer to the Pittsburgh approach (maintaining a population of complete solutions to the problem, Figure 4), or closer to the Michigan approach (using the rules as the individuals of the population, and representing a complete solution with the whole population, Figure 5). All the previously referenced works use a population of Knowledge Bases, but both approaches have their own advantages, and are explored by describing a couple of Fuzzy Genetic systems, the rst one closer to the Pittsburgh approach and the second one closer to the Michigan approach.

Fig. 4.

The genetic system works with a population of Knowledge Bases.

Fig. 5.

The genetic system works with a population of Rules.

5 Knowledge Bases as individuals of the population Using knowledge bases as the individuals of the population is one of the possibilities when incorporating learning into the fuzzy controller. The general structure of this process is that of Figure 4, a population of knowledge bases is evolved by a genetic system, and each of these KBs is evaluated by applying it as the control algorithm of the FLC throughout a certain period or periods. This approach is described by applying the four concepts de ned in Subsection 3.1 (Population, goal, tness function and reproduction algorithms) to an speci c application.

5.1 The Population: encoding Knowledge Bases The knowledge base has two components, the data base and the rule base. The main contents of the data base are: a set of parameters of the controller, a set of normalization limits de ned for each input and output variable, and a set of membership functions related to those variables. The rule base contains the control rules of the FLC.

Data Base. Three elements must be encoded: the parameters of the FLC, the

normalization limits and the membership functions. The set of parameters de nes the system dimension, that is, the number of input variables (N ) and output variables (M ), and (assembled on the vectors n and m) the number of linguistic terms (or the number of fuzzy sets) associated with each member of the set of input variables and output variables. The i-th component of vector n (n = fn1; : : : ; nN g) represents the number of linguistic terms associated with the i-th input variable. The j -th component of vector m (m = fm1 ; : : : ; mM g) is the number of linguistic terms associated with the j -th output variable. An array of (N + M )  2 real numbers represents the set of normalization limits. Each row in this array contains the limits of one input or output variable of the system (fvmin ; vmax g). The set of membership functions contains the membership functions of L fuzzy sets, where L could be obtained from n and m as shown in the following equations:

La =

N X i=1

ni ; L c =

M X j =1

mj ; L = L a + L c :

(5)

This set is represented by an array of L  4 real numbers ranged in [-1,1] (as the variables are normalized, the fuzzy sets must be de ned within the same range). Each row in the array contains the four parameters that describe a trapezoidal fuzzy set. We can obtain a string of reals by concatenating the rows of this array as has been done in some of the papers presented in Section 3.

Rule Base. The structure of the fuzzy control rules contained in our FLC with parameters fN; M; n; mg is: If xi is Cio and . . . and xk is Ckp (6) then yj is Djq and . . . and yl is Dlr where xi is an input variable, Cio is a fuzzy set associated with this variable (o  ni ), yj is an output variable and Djq is a fuzzy set associated with this variable (q  mj ). All fuzzy inputs are 'connected' by the fuzzy connective 'and'. Several fuzzy sets related to the same variable could be connected with the aggregation operator 'or', appearing in a single rule such as:

If xi is (Cio or Cip ) and : : : then yj is (Djq or Djr ) : : :

(7)

Each rule of the system is encoded into two strings of bits: one string of length

La for the antecedent (a bit for each linguistic term related to each input variable) and one string of length Lc for the consequent. To encode the antecedent we start with a string of La bits all of them with an initial value 0. If the antecedent of the rule contains a fuzzy input like xi is Cij , a 1 will substitute the 0 at a certain position (p) of the string: i?1 X p = j + nk : (8) k=1

This process is repeated for all the fuzzy inputs of the rule. The process to encode the consequent is quite similar to that described above, replacing n with m.

5.2 Goal and Fitness Function The FLC is designed as a supervisory system and its goal described by means of a certain performance criterion de ned when designing a speci c application. This performance criterion is based on overall behavior, covering not a single control cycle, but a longer period. The tness function, that also maintains an overall sense, is directly obtained from this performance criterion. No independent assignment of tness to individual rules is made, the obtained evaluation is directly assigned to the whole knowledge base (the individual of the population). Two dierent examples are presented in Subsections 7.1 and 7.2.

5.3 Reproduction algorithms: evolution operators The code that contains the information of the knowledge base is: 1. A string of 2 + N + M integers containing the dimensions of the FLC. 2. A string of (N + M )  2 real numbers containing the normalization limits of the variables. 3. A string of (La + Lc )  4 real numbers containing the de nition of the trapezoidal membership functions. 4. A string of rules (no xed length), where each rule is a string of La + Lc bits. It is possible to apply the evolution learning to any part of this code, but this approach works only with normalization limits (2) and with rule bases (4). The dimensions of the system and the membership functions are not modi ed.

Selection. Two dierent selection operators have been applied, including in both of them an elite process. The individuals with a larger tness value receive a larger number of copies, or have a higher probability of receiving a single copy (each individual receives a copy or not).

Crossover. The crossover operator produces two new chromosomes by mixing the information provided by its parents genes. A set of strings contains this information, two strings in our case (Normalization limits and Rules). The information contained in each string is not independent. Then, if possible, the operator must work simultaneously (and not independently) on both strings. Each chromosome is composed of a pair of subchromosomes encoding the rule base (r) and the data base (d). The crossover of mi = (ri ; di ) and mj = (rj ; dj ) produces two new chromosomes (mu and mv ). Rule base subchromosomes have no xed length, and their genes are rules: ri = fri1 ; : : : ; rik g rj = frj1 ; : : : ; rjl g:

(9)

To cross ri and rj a cutting point must be selected for each string. Since the lengths of the strings may not be equal, cutting points ( and ) are obtained independently for ri and rj

ri = fri1 ; : : : : : : ; ri jri +1 ; : : : ; rik g rj = frj1 ; : : : ; rj jrj +1 ; : : : : : : ; rjl g;

(10)

producing the new rules

ru = fri1 ; : : : : : : ; ri jrj +1 ; : : : : : : ; rjl g rv = frj1 ; : : : ; rj jri +1 ; : : : ; rik g:

(11)

After rule bases are crossed, the process of crossing data bases considers which rules from ri and rj go to ru or rv . The rules we use may contain fuzzy inputs and fuzzy outputs for only a subset of the input and output variables, then, normalization limits for the remaining variables have no in uence on the meaning of the rule. A larger in uence of a certain variable on rules that go to ru proceeding from ri , produces a higher probability for this variable to reproduce in du its corresponding range from di . The in uence is evaluated by simply counting the number of rules containing the variable, that are reproduced from ri to ru . The process of selection is independent for each variable and for each descendent (mu and mv ), consequently it is possible for both descendents to reproduce a certain range from the same antecedent.

Reordering. When a fuzzy system is characterized by a set of fuzzy rules, their

ordering is immaterial, the sentence connective also has properties of commutativity and associativity. When concatenating decision tables or relation matrices, the information given by a certain gene depends on its content and its position. In our string of rules, the meaning of a gene becomes independent of the position. Therefore, rule position is arbitrarily de ned for the members of the rst population, it is immaterial for the output, but it biases crossover. So an operator to reorder rules is added to the system.

This operator is applied to each set of rules produced by crossover operator, with a probability de ned as a parameter of the evolution system. When reordering a rule base (ri ), a cutting point ( ) is randomly selected with uniform distribution, to create a new rule base (rj ) ri = fri1 ; : : : ; ri jri +1 ; : : : ; rik g (12) rj = fri +1 ; : : : ; rik jri1 ; : : : ; ri g: The operator has no eect on the data base.

Mutation. The mutation is composed of two dierent processes: rule mutation

and range mutation. The rule mutation process works at the level of the La + Lc bits that compose the rule, and is similar to the classical genetic mutation applied to a string of bits. If a certain variable has the range [l ; u ], the range mutation process could be described by the following equation l (t + 1) = l (t) + KP1 S1 (u (t) ? l (t))=2 (13) u (t + 1) = u (t) + KP2 S2 (u (t) ? l (t))=2; where K 2 [0; 1] is a parameter of the learning system that de nes the maximum variation (shift, expansion or shrinkage), P1 and P2 are random values uniformly distributed on [0; 1], and S1 and S2 take values -1 or 1 with a 50% chance of either. For those variables that have a symmetrical range before mutation, the conditions P2 = P1 and S2 = ?S1 are added, to maintain symmetrical ranges after mutation.

6 Rules as individuals of the population 6.1 Population

This solution suggests a traditional fuzzy expert system architecture, with algorithms to assign a tness value to rules and to evolve the population of rules. For this system, rules have an IF THEN structure, where is a conjunction of terms < Vx 2 [a,b,c,d]>, and has the expression < Vy 2 [e,f,g,h]>. Each rule has a set of parameters associated to it: rule strength (credibility of the rule or tness value), rule strength average, life span, etc, used in the learning algorithms and in the fuzzy inference engine. Each term must be read as \Vx belongs to the set de ned by [a,b,c,d]", where [a,b,c,d] de nes a trapezoidal fuzzy set (Figure 2, left). Conditions may have an indeterminate number of terms. On the other hand, actions are unique. This is because rule strength is related to how good or bad the action proposed for the given conditions is: dierent actions should have dierent strengths. The following expression shows an example of rule: If V1 2 [0:1; 0:2; 0:4; 0:5] and V3 2 [0:3; 0:3; 0:4; 0:6] (14) then V7 2 [0:2; 0:3; 0:3; 0:4]

This rule de nition gives exibility to the system: facts are not limited to using prede ned fuzzy sets. Rule generation algorithms allows the system to nd the best adapted rules for the control system. Defuzzi cation is based on the COA method after weighting the consequents with rule strength and truth value.

6.2 Goal The objective of the system is to guide a speci ed variable X (the cost function) to its minimum value (Paragraph 3.1-Goal). It is possible to try to minimize the absolute cost value, but if the system does that, it is almost sure that at a given moment this absolute minimum is unreachable. It is possible too that the action carried out could be the best action at that moment, with only a little decrement in the cost function. The proposed system minimizes the variable in an N ?window, taking into account only the last N cases: the system is going to look for the local minimum at this moment. If we look for the minimum value in the last n cases, Xmin and Xmaxchange with time ([19]). System performance is: 8 1 if Xt  Xmin > > > < XX ?X?tX?1 if Xt > Xmin ^ Xt  Xt?1 SP = > min (15) Xt?1 ?Xt?1 if X < X t max ^ Xt > Xt?1 > X ? X max t ? 1 > : ?1 if Xt > Xmax

6.3 Fitness function This architecture has been designed with a suggestion system in mind. In such a system, the controller has to estimate the in uence of each applied rule to the process state, represented by the new data vector. To evaluate this rule by rule in uence we use the fuzzy set (A) suggested for the variable Va in the action part of the rule, and the real values for this variable (X ) at times t and t ? 1. Rule in uence is: 8  (X ) if  (X ) > 0 A t < A t RI = : 0X ?X if A (Xt ) = 0 ^ jA ? Xt j  jA ? Xt?1 j (16) t t?1 Xt?1 ?Xl if A (Xt ) = 0 ^ jA ? Xt j > jA ? Xt?1 j

where jA ? X j is the distance from X to the fuzzy set A (i.e., to the nearest point of the support of A), and Xl is the upper (if Xt > Xt?1 ) or lower (if Xt < Xt?1 ) limit of the Universe of Discourse of variable X . The nal rule evaluation is obtained by multiplying the rule in uence estimation by system performance. If this performance is positive (the process is working well), rule evaluation depends on the rule in uence: if the rule has been applied, nal evaluation will be positive; if not, it will be negative. Dual considerations may be made for negative performance. And, of course, once we have

the rule evaluation, it must be weighted by its overall truth value, to obtain the nal value4 . After this evaluation value is obtained, rules adapt their tness value. Rules increase their strength if their evaluation is positive, and decrease it if otherwise. The expression to obtain this strength variation is:



+ K
TR
E
(1 ? SR;t?1 ) if E  0 SR;t = SSR;t?1 + if E < 0 R;t?1 K
TR
E
SR;t?1

(17)

where K is a constant that allows xing of the memory of the system (K near 0 produces slow variations and vice-versa) TR is the truth value of the rule, and E is the nal evaluation of rule R (see [28]).

6.4 Reproduction Algorithms Unlike traditional systems, in this architecture GAs are not red in every cycle of the system. In fact, the rule generation process only works from time to time, when a particular KB has been used several times. While GAs do not work, the rule evaluation algorithm adjusts rule's tness value. The application of the GA has two steps: generation and insertion.

Generation: The new ospring A set of code adapted genetic operators has been designed:

Selection: It is made randomly according to rule strength: the more strength, the more the probability of being selected.

Uniform Crossover: Mixes genetic material for creating new individuals. As

shown in g 6, it uses as many cut points as the number of connectives that the rule has. This operator has been selected over standard crossover because it allows the system to create new rules with whatever possible antecedents. Traditional crossover needs some kind of antecedents reordering (inversion operator). Soft and Hard Mutation: Changes the fuzzy set (soft) or the variable (hard) of the fact. Soft mutation moves the (a; b; c; d) values of the trapezium to create a new fuzzy set. New values are in ([0; a + b?2 a ], [a + b?2 a ; b + c?2 b ], [b + c?2 b ; c + d?2 c ], [c + d?2 c ; 1]) ranges. This operator is introduced to create variety in fuzzy sets. On the other hand, hard mutation changes the variable of the fact (antecedent or consequent). It ensures that every variable is going to be taken into account. 4

The most signi cant aspect of this evaluation algorithm is that we can obtain values for every rule with a positive truth degree, regardless of their having been used at inference time or not.

V2

A

V3

B

V3

C

V6

D

V8

E

V8

F

V9

G

V4

H

V8

I

V8

I

V4

H

Initial Rules

V2

A

V3

B

V3

C

V6

D

V8

F

V8

E

V9

G

Uniform Crossover

Fig. 6.

Uniform crossover

Insertion of good new rules: The Limbo New rules, generated by GAs are

placed in the Limbo, a special place where rules are tested and evaluated without aecting the output of the controller. Only when a rule has proved a good one, is it accepted and inserted into the KB. As has been shown, the evaluation algorithm allows the assignation of strength to every rule with positive truth value, regardless of whether they have been used by the fuzzy inference engine or not. In fact, what the algorithm uses is the system performance and the proposed actions of true rules. The real action made by the controller is not taken into account. This means that rule strength may be updated for those rules that are in the KB and for those that are in the limbo and would have been able to be red. The rules that live in the limbo have been created by GAs but they have to prove their suitability before being used by the inference engine. In order to select or reject a rule from the limbo several parameters are used:

{ Rule parameters:  Rule Age (R:A:): Number of inferences made since rule joined the limbo.  Rule Use (R:U:): Sum of the successive truth values of the rule throughout the R.A.

 Rule Activations (R:Ac:): Number of inferences in which the rule should have taken part (the antecedent had a truth value greater than zero).

 Equivalent Rule Evaluation (E:R:E:): Constant evaluation that the rule

should have obtained to reach its present strength value after the same number of inferences. { Limbo parameters  Limit Age (L:A:): Rule Age at which a rule has to leave the limbo.  Minimum Rule Activations (M:R:Ac:): Value of R:Ac: that once reached allows a rule to be promoted from the limbo to the KB, before its Limit Age arrives.  Minimum Equivalent Evaluation (M:E:E:): Minimum value of the E:R:E: that allows a rule to be promoted to the KB.

With these parameters in mind, when new rules go into the limbo, old rules are analyzed looking for good or bad rules (Table 3):

Table 3.

Going out the limbo

condition

action

R:A:  L:A: and R:Ac: = 0 R:A:  L:A: and E:R:E: < M:E:E: Kill rule R:A:  L:A: and E:R:E:  M:E:E: R:Ac:  M:R:Ac: and E:R:E:  M:E:E: Move rule to knowledge base Otherwise Leave in the rule in the limbo

In order to prevent the convergence of the KB (which in this case means to have a lot of identical rules in it), there is an additional lter to avoid too many copies of any rule: if a new rule is going to pass from the limbo to the KB, it should not have more than a given number of copies of it in the KB. In other situations, the rule is not inserted into the rule base.

7 Application examples 7.1 A diversi cation problem The approach described in Section 5 has been applied to the controller (supervisory control) of a simulated anthropomorphic (1.75m, 70kg) biped walking machine ([15, 18]). The model is a six-link, 2-D structure, with two legs, a hip and a trunk (Figure 7). Each leg has a punctual foot, a knee with a degree of

Trunk: 0.65m, 35kg. α α

α α

Hip:

0.10m,

7kg.

Thigh:

0.50m,

9kg.

Shank: 0.50m,

5kg.

α α

Fig. 7.

Variables and dimensions of the model.

freedom and a hip with another degree of freedom. The contact of the stance-foot with the ground de nes an unpowered degree of freedom (1 ) whose behavior is

not directly controlled5, but controlled through the appropriate movements of the joints (2 ? 6 ). The control cycle of the FLC is 0.01 sc. The goal of the FLC is to de ne joint trajectories in such a way that the biped system describes a regular walk without falling or stopping. In this case, the GAs are applied not with an optimization aim, but with a diversi cation aim. The idea is to use biomechanical studies ([21, 24]) to obtain the KB of an FLC controlling a regular walk with a certain speed and stride length, and then apply GAs to create other knowledge bases capable of controlling regular walks with dierent speeds and stride lengths. When the information contained in a KB6 is applied by the FLC, a sequence of movements is produced on the biped model. This sequence of movements will be evaluated ([15]), based on the stability and regularity of the walk, over a tensecond simulation (a thousand control cycles). If the system falls before ending the simulation, the evaluation function measures the stability as a function of the time and the number of steps before falling, producing a value from 0 to 0:8. If the system has not fallen or stopped at the end of the simulation, a xed value is assigned to evaluate the stability (0:8), and the regularity is evaluated as a function of the standard deviation of the stride duration from the mean period throughout the ten-second simulation. This evaluation of the regularity produces a value from 0 to 0:2 (that is added to the 0:8 value previously obtained), where the value 0:2 means that the standard deviation is 0, decreasing linearly to value 0 when the standard deviation is equal to or greater than 20% of the mean period of the walk sequence. Some chromosomes will contain valid gait patterns, that is, gait description that produces a walk simulation without falling, or stopping the biped, and consequently obtaining evaluations on interval [0:8; 1]. Some tests have demonstrated the ability of the learning system to generate valid gaits [15]. Figure 8 shows four sequences of walk produced by genetically generated KBs. All of them have been obtained as the result of a single learning process with a reproduction rate of 0.8, a reordering rate of 0.5, a rule mutation rate of 0.01, a range mutation rate of 0.05 and a mutation constant (the parameter K on expression 13) equal to 0.5. The maximum population was 500 individuals the number of generations was forty and the initial population contained ve KBs producing valid gaits7, and a set of other individuals producing the fall of the biped system. These ve KBs were extracted from biomechanical studies and the generated gaits showed speeds in the [1.05,1.15]m=sc interval and stride lengths in the interval [0.67,0.68]m. The sequences (Figure 8) present simulation results obtained with the biped model, containing a twenty images per second representation, except sequence 4 that is a ten images per second representation. Each sequence represents a particular case: the best, the shortest, This problem is closely related to the simpler problem of controlling an inverted pendulum ([1]). 6 This KB is encoded by a chromosome that constitutes an element of the genetic population. 7 Four of these gaits had an evaluation 0:8 (standard deviation greater than 20% of the mean period), and the fth had an evaluation of 0:9491, that represents a standard deviation of 5% of the mean period 5

the longest and fastest, and the slowest gait.

Fig. 8.

1.

2.

3.

4.

Dierent genetic generated gaits.

The main characteristics of each sequence are described below: the mean speed of the biped (S ), the stride length (L) and the time covered by the sequence (T ). 1. 2. 3. 4.

The most regular (best): S = 1:21m=sc, L = 0:68m and T = 2sc. The shortest: S = 0:76m=sc, L = 0:64m and T = 3sc. The longest and fastest: S = 1:29m=sc, L = 0:78m and T = 2sc. The slowest8: S = 0:54m=sc, L = 0:67m and T = 4:5sc.

The genetic process has produced the evolution of the covered ranges for stride length and speed from the initial intervals [0.67,0.68]m and [1.05,1.15]m=sc, to [0.64,0.78]m and [0.54,1.29]m=sc, creating more than a hundred gaits covering this area. Fifty of these gaits have obtained an evaluation better than the best for an individual of the initial population (0:9491). The best evaluation obtained by an individual of the forty generation on the previously described experiment was 0:9817, that represents a standard deviation lower than 2%. 8

This sequence contains only ten images per second.

7.2 An o-line optimization problem The cart-pole balancing system is a classical control problem that literature has established as a benchmark in learning control system evaluation. The objective of such a system is the control of the translational forces applied to a cart, to position it at the center of a nite track, while balancing a pole hinged on the top of the cart. In our experiments we will use the model applied in [4, 1] maintaining the system's parameters and dimensions (Table 4). Table 4.

Model Dimensions and Parameters Symbol Description Value x Cart position [-1.0,1.0] m  Pole angle from vertical [-0.26,0.26] rad F Force applied to cart [-10,10] N g Force of gravity 9.8 m/s2 l Half-length of pole 0.5 m M Mass of the cart 1.0 kg m Mass of the pole 0.1 kg c Friction of cart on track 0.0005 N p Friction of pole's hinge 0.000002 kg m2

The FLC applied to this problem is similar to that previously de ned for the gait synthesis problem. The FLC only modi es its dimensions. The number of input variables (N ) is four (x(t); x_ (t); (t); _(t)), and the number of output variables (M ) is one (F (t)). The number of fuzzy sets per variable is ve. The control period is 10?2 sc. The initial knowledge for these experiments is random. The number of initial rules per KB is uniformly distributed on the interval [1,30]. The normalization limits for all the variables are symmetric ([-a,a]), their upper limit (a) is randomly obtained and is uniformly distributed within a prede ned range for each variable. The prede ned ranges for those variables related to the physical dimensions of the simulation model, are adjusted to the corresponding dimension (Table 4). These variables are: the cart position (x), the pole angle from vertical () and the force applied to the cart (F ). For other variables with no direct relation to the model dimensions (derivatives), dierent prede ned ranges have been tested. The set of prede ned ranges is:

x! [0,1] m x_ ! [0,1] and [0,5] m/s  ! [0,0.3] rad _ ! [0,1] and [0,5] rad/s F! [0,10] Nw

The evaluation process is similar to that described in [4]. A KB is tested for sixty seconds (6000 control cycles) starting from each of twenty initial conditions. A performance index based on cart position is then applied

P = 20 16000

X

X

20init: 6000cicles

exp(?jxj)

(18)

A partial simulation (60 seconds) is terminated if the system variables exceed the physical ranges (x = [?1; 1];  = [?0:26; 0:26]). The twenty dierent starting conditions are arranged in ve groups of four symmetrical positions (x; ). A set of experiments with dierent initial population and learning parameters has been de ned. The dimension of the population is two hundred members with an elite of fty individuals, and the processes have been stopped after two hundred generations. Dierent values of reordering rate, rule mutation rate, range mutation rate and maximum variation in range mutation (K on equation 13) have been applied. Only aiming to present a sample, a set of typical learning parameters for the process could be: a rule reordering rate of 0.5, a rule mutation rate of 0.1, a range mutation rate of 0.1 or 0.2 and a maximum variation of ranges (K ) of 0.2 or 0.4. A set of experiments applying these values has been carried out, producing FLCs capable of balancing the system throughout the whole simulation (sixty seconds starting from each of twenty dierent initial conditions). Some of these controllers have been obtained after less than fty generations of the genetic process. Figure 9 shows the phase plane of , for three dierent controllers (all of them obtaining an evaluation over 0.9) generated by three dierent learning processes. The initial values for x and x_ are zero. The initial values for  and _ are eight initial conditions (dierent from those used during learning) placed by small circles in g 9.

Fig. 9.

Pole's phase plane for three dierent controllers, and eight initial conditions.

7.3 An on-line optimization problem

Power plants are very complex processes where, despite a lot of small control loops to stabilize every variable of the plant, the strategic control to make the process work in the best situation, is carried out manually by human operators. In such a process, a supervisor control system may help these persons to obtain better performance values. This example shows how a genetic-fuzzy suggestion system has been implemented to reduce the heat rate (the consumption) in a particular power plant. It has been installed at Velilla (Palencia, Spain). An acquisition module gives 23 variables to the suggestion system. These variables where selected by operators as the main set for de ning the plant state. The supervisor will suggest 11 set points for a like number of operation variables. These set points are presented to the operator through an interface module. Operators will decide if suggestions should be made or not. It has been decided to use a continuous learning system (on-line optimization) because this environment is very complex and time variant (broken or dirty pipes, slow pumps, air temperature, humidity, . . . , are parameters that are out of our control and can aect the control system). New data come in every 10 minutes, and suggestions are made for every new data. Control rules are like IF Steam Pressure is in [0.20, 0.24, 0.30, 0.32] and Reheated Temperature is in [0.80, 0.82, 0.84, 0.90] THEN Air Flow should be in [0.10, 0.12, 0.15, 0.24] (in this case, values are normalized in [0,1]). After laboratory tests, the following parameters have been set: 1.- Max. Number of Rules in the KB: 1000. 2.- Learning Constant K (for strength update): 0.25. 3.- Age Limit for the limbo: 300. 4.- Minimum Rule Activations: 10. 5.- Minimum Equivalent Evaluation: 0.1. In order to assure that the system learns, the following experiment has been designed: The control system will start with a 100-rule random Knowledge Base, and it will be trained using three dierent les. Data from these les emulate data received from the acquisition module. In fact this situation emulates a blind system, with the interface module turned o. New received data (from the le) are never aected by suggestions made by the inference system9 . The goal of this test is to prove that even in this case, the learning system is capable of discriminating good and bad rules from the knowledge base, in short, capable of learning. Three les with 3514 cases (around 24 days of continuous operation) have been used for the experiment: 1. A real historic data le. This data le comes from real records of the power plant. 2. A random data le, to compare learning results with the rst case. 3. The real le of the rst case, with random cost function, to assure that learning is achieved thanks to the algorithm and has nothing to do with the data le used. 9

In fact, this situation is similar to what happens with the Limbo

1

100

2

100

80

80

60

60

60

40

40

40

20

20

20

00

0.2

0.4

0.6

0.8

1

00

0.2

0.4

0.6

0.8

1

00

100

100

100

80

80

80

60

60

60

40

40

40

20

20

00

Fig. 10.

0.2

0.4

0.6

0.8

1

00

3

100

80

0.2

0.4

0.6

0.8

1

0.2

0.4

0.6

0.8

1

20

0.2

0.4

0.6

0.8

1

00

Final (upper) and average (lower) strength distribution.

The six graphics (Figure 10) show the strength and average strength distribution after the 3514 cases. As it is shown, in both random cases (center and right) the strength distribution is around 0.5 (the initial strength), while in the real case (left) it is distributed over the upper part of the axis. Only less than a 10% of learned rules have a strength lower than 0.5. In conclusion, the proposed learning system based on Genetic Algorithms applied to fuzzy rules is able to learn good fuzzy control rules and can be used in a Fuzzy Control System for complex time variant processes.

8 Conclusions Fuzzy Logic Controllers have demonstrated their ability to cope with a wide range of complex control problems. However, the question of knowledge acquisition is aected by subjective design decisions having a great in uence over the performance of this kind of control systems. To overcome this problem, learning capabilities are added to the FLC. A brief analysis on how to add evolutionary learning capabilities has been presented, including some applications to dierent kinds of problems: knowledge diversi cation, o-line optimization and on-line optimization.

References 1. A.G. Barto, R.S. Sutton, and C.W. Anderson. Neuronlike adaptive elements that can solve dicult learning control problems. IEEE Transactions on Systems, Man and Cybernetics, 13(5):834{846, Sep/Oct 1983. 2. J.J. Buckley and Y. Hayashi. Fuzzy input-output controllers are universal approximators. Fuzzy Sets and Systems, 58:273{278, 1993.

3. J.L. Castro. Fuzzy logic controllers are universal approximators. IEEE Transactions on Systems, Man and Cybernetics, 25(4):629{635, April 1995. 4. M.G. Cooper and J.J. Vidal. Genetic design of fuzzy controllers. In Proceedings 2nd International Conference on Fuzzy Theory and Technology, October 1993. 5. K. DeJong. Learning with genetic algorithms: An overview. Machine Learning, 3(3):121{138, October 1988. 6. D.E. Goldberg. Genetic Algorithms in search, optimization & machine learning. Adisson Wesley, Reading, MA, 1989. 7. J.H. Holland. Adaptation in Natural and Arti cial Systems. University of Michigan Press, Ann Arbor, 1975. 8. J.H. Holland. Adaptation in Natural and Arti cial Systems. MIT Press, 1992. 9. C.L. Karr. Design of an adaptive fuzzy logic controller using a genetic algorithm. In Proceedings 4th. International Conference on Genetic Algorithms, pages 450{ 457. Morgan Kaufmann, 1991. 10. C.L. Karr and E.J. Gentry. Fuzzy control of pH using genetic algorithms. IEEE Transactions on Fuzzy Systems, 1(1):46{53, February 1993. 11. B. Kosko. Fuzzy systems as universal approximators. In Proc. 1992 IEEE International Conference on Fuzzy Systems, pages 1153{1162, San Diego, USA, March 1992. 12. C.C. Lee. Fuzzy logic in control systems: Fuzzy logic controller - part I and II. IEEE Transactions on Systems, Man and Cybernetics, 20(2):404{435, Mar/Apr 1990. 13. M.A. Lee and H. Takagi. Integrating design stages of fuzzy systems using genetic algorithms. In Proceedings 2nd IEEE International Conference on Fuzzy Systems, FUZZ-IEEE'93, volume 1, pages 612{617, March 1993. 14. J. Liska and S. Melsheimer. Complete design of fuzzy logic systems using genetic algorithms. In Proceedings 3rd IEEE International Conference on Fuzzy Systems, FUZZ-IEEE'94, volume II, pages 1377{1382, June 1994. 15. L. Magdalena. Estudio de la coordinacion inteligente en robots bpedos: aplicacion de logica borrosa y algoritmos geneticos. Doctoral dissertation, Universidad Politecnica de Madrid (Spain), 1994. 16. L. Magdalena. Analysis of hybrid models: fuzzy logic/neural nets. Technical Report MIX/WP2/UPM/1.2, Dep. Ingeniera de Sistemas Telematicos, E.T.S.I. Tele comunicacion, Universidad Politecnica de Madrid, March 1995. 17. L. Magdalena. A rst approach to a taxonomy of fuzzy-neural systems. In IJCAI'95 Workshop on Connectionist-Symbolic Integration: From Uni ed to Hybrid Approaches, 1995. 18. L. Magdalena and F. Monasterio. Evolutionary-based learning applied to fuzzy controllers. In Proceedings 4th IEEE International Conference on Fuzzy Systems and the Second International Fuzzy Engineering Symposium, FUZZIEEE/IFES'95, volume III, pages 1111{1118, March 1995. 19. L. Magdalena, J.R. Velasco, G. Fernandez, and F. Monasterio. A control architecture for optimal operation with inductive learning. In Preprints IFAC Symposium on Intelligent Components and Instrument for Control Applications, SICICA'92, pages 307{312, May 1992. 20. E.H. Mamdani. Application of fuzzy algorithms for simple dynamic plant. Proceedings IEE, Part D, 121:1585{1588, 1974. 21. T. McMahon. Mechanics of locomotion. International Journal of Robotics Research, 3(2):4{28, 1984.

22. D. Park, A. Kandel, and G. Langholz. Genetic-based new fuzzy reasoning models with application to fuzzy control. IEEE Transactions on Systems, Man and Cybernetics, 24(1):39{47, January 1994. 23. D.T. Pham and D. Karaboga. Optimun design of fuzzy logic controllers using genetic algorithms. Journal of Systems Engineering, pages 114{118, 1991. 24. F. Plas, E. Viel, and Y. Blanc. La marche humaine. MASSON, S.A., Paris, 1984. 25. K. Shimojima, T. Fukuda, and Y. Hasegawa. RBF- fuzzy system with GA based unsupervised/supervised learning method. In Proceedings 4th IEEE International Conference on Fuzzy Systems and the Second International Fuzzy Engineering Symposium, FUZZ-IEEE/IFES'95, volume I, pages 253{258, March 1995. 26. T. Takagi and M. Sugeno. Fuzzy identi cation of systems and its applications to modeling and control. IEEE Transactions on Systems, Man, and Cybernetics, 15(1):116{132, January/February 1985. 27. P. Thrift. Fuzzy logic synthesis with genetic algorithms. In Proceedings 4th. International Conference on Genetic Algorithms, pages 509{513. Morgan Kaufmann, 1991. 28. J.R. Velasco. Arquitectura para sistemas de control inteligentes. PhD thesis, E.T.S.I. Telecomunicacion. (Universidad Politecnica de Madrid), November 1991.