Application of Fuzzy, GA and Hybrid Methods to CNN Template Learning M.-D. Doan1 , S. Halgamuge2 , M. Glesner1 , S. Braunsforth1 Darmstadt University of Technology1 { Institute for Microelectronic Systems Karlstr. 15 { 64283 Darmstadt, Germany { Fax: (+49)6151/164936 phone: (+49)6151/165236, email:
[email protected] University of South Australia2 Institute for Telecommunications Research SPRI Building, Warrendi Road, The Levels, SA 5095, Australia email:
[email protected]
ABSTRACT: In this paper, a novel methodology is presented for template learning using GA based Fuzzy Systems. At the beginning of the learning procedure, several fuzzy systems composed of fuzzy rule sets are randomly initialized. In opposite to other works in template learning with classical GA-approach, the genetic algoriths are used to optimize the fuzzy rule sets, which nally produce an optimal template for the desired task. The nal rule base of characteristics of the input, and master images are then applied to the operations of a proper GA approach to alter the templates coecients in order to minimize the GA run time eort. Results of several applications are shown.
1 Introduction The use of genetic algorithms for generation of compact fuzzy rule based classier systems by partitioning the input space is presented in 6]. There is a large number of characteristics, which lead fuzzy systems to better performance. In general, fuzzy systems are suited for for problems, in which a mathematical model is hard to be found. For instance, input variables and parameters of a system are inaccurate or incomplete5]. In the same way, formulas and rules for the derivation of solutions might be inaccurate and incomplete. Fuzzy logic permits decisions with estimated values and incomplete informations. One has to take into account that decisions made by those systems might be wrong. They have to be corrected later when more informations are available. Without any informations as system input, a decision is impossible in any form of logic. For more complex problems, conventional non-fuzzy systems are used to be more expensive. They depends strongly on mathematical approximation (e.g. linearization of non-linear problems), which might lead to worse performance. Under these conditions, fuzzy systems dominate generally other conventional systems (linear, integral, and dierential systems). Fuzzy systems allow the use of qualitative terms, such as slower or a little bit faster. Those terms are implemented as symbolic instructions. Such terms are more \natural" evaluation rules than mathematical equations, so that human knowledge might be easily transfered. Learning CNN templates is a problem for which no mathematical model exists. In some applications using CNN templates there exist certain relations between source and output images. In those cases, where no accurate knowledge or rules exist, a hybrid system is one of the best approaches to nd the solution. Chapter 2 begins with the backgrounds for fuzzy systems and describes the proposed fusion technique of fuzzy logic and genetic algorithms. Chapter 3 investigates the two learning methods: the classical GA approach and the combination of GA and fuzzy rule based systems. Chapter 4 summerizes the paper and gives a short outlook to future works.
2 Genetic algorithms in fuzzy rule based systems 2.1 Fuzzy Systems
A fuzzy set is denoted by an ordered set of pairs, the rst element of which denotes the element x and the second A (x) the degree of membership: A = f(x A (x)jx 2 X g where A (x) is the membership function and X is the universe of discourse. The most important operators are intersection, union, complement and implication. There are many possibilities to realize the intersection and union and they are dened by pairs of operators as f*,*'g, with the properties such as commutativity, associativity, property of unit element and De Morgan's theorems. The widely known operators in fuzzy logic can be summarized as: MIN operator: (AandB) (x) = minfA(x) B (x)g, MAX operator: (AandB) (x) = maxfA (x) B (x)g, product operator: (AandB)(x) = A (x) B (x), probablistic sum: (AandB)(x) = A (x) + B (x) ; A (x) B (x), complement: A (x) = 1 ; A (x), fuzzy implication: A!B (x) = not(a (x)) OR B (x). Extensive theoretical work 11] may give the reader more insight into the foundation of fuzzy theory. Fuzzy control is probaly the most known application of fuzzy theory. Considering two fuzzy rules: R1: IF x1 is middling AND x2 is middling THEN y is middling R2: IF x1 is low OR x2 is high THEN y is high The inference consists of applying the Min or product operators in case of AND and MAX or probablistic sum operators in case of OR to the antecedent values, obtained by the fuzzication part, and determining a level of truth for the consequence. With the corresponding consequent membership function an area is generated. Those areas activated by fuzzy rules are aggregated (composition) to represent the fuzzy output. The next optional step is the defuzzication, if a crisp output is needed. Fig. 1 illustrates the dierent fuzzy steps. Fuzzification x1=middling x2=middling µ
if
µ x1
x1=low µ
if
µ x2
x2=high
y
or
µ x1
x1=high µ
if
Inference Composition and then y := middling
µ x2
x2=low µ
x1
x 1*=7
then y := high
or
y
then y:= low µ
x2
x 2*=3
µ y
Defuzzification y
y *=3
Figure 1: Fuzzy operators in a fuzzy systems.
2.2 Genetic Algorithm Based Design of a Fuzzy System
The most interesting part of designing fuzzy systems using genetic algorithms is the representation of the fuzzy rule base. Since the size of the search space increases exponentially with the size of the strings, the encoded bit string lengths should be kept as small as possible. The number of rules could evolve, but only the necessary rules should be kept. A method is proposed in 2] to generate conjunctive rules. An individual is dened as a set of rules whose number could evolve.
IF x1 IS A1 AND x2 IS A2 AND x3 IS A3 AND x4 IS A4 THEN F is B The membership functions Ai , where i 2 f1 2 g are approximated by symmetrical triangles. Only two parameters are needed to dene such triangles: the centers of the base of the triangles and their half lengths. The children generated by crossover between two strings should perform in a manner similar to their parents. Therefore, crossover should occur only between similar rules. In the algorithm proposed, the most closely matching rules, dened as the sum of the dierences between the input variable centers are combined rst, followed by the next most closely matching rules. Any rules from a longer string that are not matched are added at the end.
2.3 Data structures for GA based fuzzy system generation
A fuzzy system (a set of fuzzy rules), which calculates one output from several input values, can be considered as an individual, that inherits its characteristics to its children. Positions and edges of the membership functions are variable. Each individual is assigned during the evaluation phase to a tness value, which will be used in the next step for the search of the best individual. Population system 1
system 2
system 3
system 4
Indiviual rule 1
variable 1
rule 2
variable 2
rule 3
variable 3
rule 4
variable 4
output variables
Figure 2: Data structure for population, individual of the GA based fuzzy system. Each fuzzy rule consists of a number of input and output variables. For every variable, a membership function is introduced. A system, composed by several fuzzy rules, comprises an individual for the genetic algorithm, whose properties will be changed by mutation. By this way, a new individual can be created. The membership functions are approximated by symmetrical triangles. Only two parameters are needed to dene such triangles: the centers of the base of the triangles and their half lengths. If the center position and the half lengths are stored using one byte each, a fuzzy rule will need (2 number of variables) bytes. To describe a problem with four input variables adequately, 120 bytes are needed (on average 15 rules are required for each fuzzy system). With increasing number of input variables, the complexity of the solution space increases exponentially. Fig. 2 illustrates the data structure of the GA based fuzzy system. In the case of CNN templates the number of output variables should be 19. Another approach is to concentrate on the important rules, the weaker rules are eliminated by the genetic algorithm. One limitation of this method can be seen if the number of inputs is high.
3 GA and fuzzy systems for CNN template learning The main part of the software tool box consists of a GA core which proved to be a very e cient tool and a digital CNN simulator (DCNNS). Beside the two basic operations Mutation and Crossover of the GA core, three functions Deletion, Insertion, and Theft are implemented in order to delete, insert or exchange rules between individuals, respectively. Length of encoded string of each individual is therefore variable. Number of points for the Crossover function can be varied. In the case of template learning with GA, the number should be 19. The tness function is a programmable piecewise linear function. The CNN simulator operates with several types of image formats. For each simulation, a maximum number of iterations has to be specied. The DCNNS even stops if the output already converge before the maximum number of iterations is reached. Output changes after each iteration are remarked to detect non-convergence and network instability. Templates, which cause network instability and nonconvergence, will receive a minimum tness value.
MSF weighting of input value
Min
weighting of min. value with output MSF
with
Inputs
input MSF weighting of min. value with output MSF
Defuzzification
D
MSF weighting of input value
Min
weighting of min. value with output MSF
with input MSF weighting of min. value with output MSF
Defuzzification
D
MSF weighting of input value
Min
weighting of min. value with output MSF
with input MSF weighting of min. value with output MSF
Figure 3: Principal of implemented fuzzy systems with 4 inputs, 2 output variables, and 3 rules.
3.1 GA based method for CNN template learning
Fig. 4 illustrates the GA approach based on 3, 7]. Encoded string for each individual represents all 19 parameters of a template set. Practical experiences show that best result with respect to run time eort is achieved with a 19-point crossover function. Generation 0 of 0
