Classification of Fuzzy-Based Information using ...

International Journal of Computational Intelligence Research. ISSN 0973-1873 Vol.3, No.3 (2007), pp. 267-275 © Research India Publications http://www.ijcir.info

Classification of Fuzzy-Based Information using Improved Backpropagation Algorithm of Artificial Neural Networks 1

3

2

Mukul Jain , P. K. Butey and Manu Pratap Singh 1

Department of Computer Science, ICIS, Dr. B. R. Ambedkar University, Khandari Campus, Agra, Uttar Pradesh, India [email protected]

2

Department of Computer Science, ICIS, Dr. B. R. Ambedkar University, Khandari Campus, Agra, Uttar Pradesh, India [email protected] 3

Department of Computer Science, Kamla Nehru Mahavidhyalaya, Nagpur, Maharastra, India

Abstract: Artificial neural networks show inadequacy while classifying fuzzy based information. In this paper, a methodology has presented for adequate classification of fuzzy information. In this system, the fuzzy rules are used as inputoutput stimuli’s. The system consists of layered architecture of fuzzy-neural network. First hidden layer generates the degree of membership for all the input-output patterns pairs. This vector of degree of membership is now being trained into the neural network for generating the final classification of rules using the Backpropagation algorithm of neural network. Thus, neural network as a whole performs two tasks. First it generates the degree of membership and later it classifies the rules in different classes on the basis of membership function. A simulation program in C has been deliberated and developed for analyzing the consequences. The overall process has been illustrated by applying to two real-world classification problems i.e. IRIS and Post-operative Patient Data. Results show the adequacy of the classification and improvement in the epochs for learning the fuzzy rules. Keywords: Pattern Classification, Fuzzy System, Artificial Neural Networks, Fuzzy Logic.

I.

Introduction

The humans effortlessly recognize and distinguish objects right from the natural scenario. The human brain intercepts imprecise and incomplete sensory information provided by perceptive organs. They can perceive the grossly distorted, ambiguous, noisy or fuzzy patterns. They are also capable to distinguish overlapped patterns. Human performs the entire decision-processing task with biological neural networks [1], which consists of the interconnection of neurons in an immensely intricate fashion. Artificial neural network [2],

simulated structure for this interconnection, is capable for pattern recognition [3-5] task after proper learning of stimuli’s. But they are inadequate while classifying fuzzy information consisting linguistic rules [6]. Neural networks so as fuzzy logic [7] are dealing with important aspects of knowledge representation, reasoning and learning, but in different approaches with their advantages and weaknesses. Neural networks can learn and classify information from the examples, but it is nearly impossible to describe knowledge acquired in that way. On the other hand, fuzzy logic which enables approximate reasoning [8] has a structural knowledge representation in the form of fuzzy if-then rules but lacks the adaptability to deal with changing external environments. Thus, we incorporate neural network learning concepts in fuzzy inference systems, resulting in fuzzy-neural networks (FNN) [9-11] with their greatest application in implementation of classification of fuzzy information. Fuzzy logic, based on the Zadeh’s fuzzy sets [7], has mathematical potential for describing indeterminacy related with human cognitive processes, such as thinking, learning and reasoning. Fuzzy set theory provides a systematic calculus to deal with such information linguistically, and it performs numerical computation by using linguistic labels stipulated by membership functions. Moreover, a selection of fuzzy if-then rules forms the key component of a fuzzy system that can effectively model human expertise in a specific application. Fuzzy logic enables reasoning based on incomplete and imprecise information, known as approximate reasoning. On the other hand, artificial neural networks, with their diverse architectures built on the concept of an artificial neuron, are developed to ape biological neural systems in performing functions such as

268 learning or recalling for the various pattern recognition tasks. While fuzzy logic enables mechanism for reasoning based on incomplete and imprecise information, artificial neural networks provide some remarkable abilities such as learning, adaptation and generalization. Neural networks and Fuzzy logic have some common features such as distributed representation of knowledge, model-free estimation, ability to handle data with uncertainty and imprecision etc. The interest in Neuro-fuzzy systems [12-16] has grown in popularity tremendously over the recent years due to great impact in machine learning [17]. A neuro-fuzzy system is an improved neural network wherein learning performance will be much better. Fuzzy logic has tolerance for imprecision of data, while neural networks have tolerance for noisy data. Number of researchers have probed diverse edifice of neurofuzzy system as hybrid neuro-fuzzy models [18-26]. In this architecture a neural network and a fuzzy system is combined into homogeneous architecture. The system may be either interpreted as a special neural network with fuzzy parameters, or as a fuzzy system implemented in a parallel distributed form. Some intensions for approaching hybrid neuro-fuzzy models can be shown as follows. ƒ Automatically create or improve a fuzzy system by means of neural network methods. ƒ Generate fuzzy rules for classification of fuzzy information. ƒ Optimize the fuzzy rule base for classification of fuzzy information. A methodology for adequate classification for fuzzy based information using fuzzy-neural network system is presented in this paper. The fuzzy-neural network system employs a set of linguistic rules to describe the human behavior, machine process behavior or etc. The linguist rules describe a control surface, which defines an appropriate output value for every vector of input values. Some rules in the domain of speed control, for example, could have the form: ‘If the speed is very high and the next vehicle is very near, then apply break very strongly’. ‘If the speed is below average and road is empty, then apply accelerator’. The set of fuzzy control rules is applied as input-output stimuli’s to this fuzzy-neural network system. The multilayer feedforward neural network system is incompetent for classification of fuzzy information in the form of if-then rules with linguistic values. The architecture of presented fuzzy-neural network system is extended from the multilayer feedforward neural network. Here, in the present paper, we augmented one additional layer for this fuzzy-neural network system immediately following the input layer. This additional layer estimates the degree of membership [7] of various input-output stimuli’s for each class. This generated degree of membership of various input-output stimuli’s corresponding to each class is now being learned as input-output stimuli’s to the fuzzy-neural network system. So, the whole process can be described in two phases; in the first phase the degree of membership for

Mukul Jain, P. K. Butey and Manu Pratap Singh various input-output stimuli’s is generated, in the second phase the classification of fuzzy information is performed on the basis of degree of membership. Here we applied the approach to two well-known real world classification problems i.e. IRIS [27] and Postoperative Patient Data [28]. Initially the classification for both the problems is performed with backpropagation algorithm. The results obtained exhibit the inadequacy of backpropagation algorithm for classification. Now we performed the classification with proposed fuzzy-neural network system. We have investigated the adequacy for classification using 84 different architectures of hidden layers. Results exhibit that the artificial neural network is not able to generate adequate classification due to presence of fuzzy information. The adequate classification can be achieved using the presented fuzzy-neural network system. The fuzzy-neural network system envisages superior consequences in contrast with the feedforward neural network. The next section discusses the methodology and simulation design of the problem. In this section the mathematical formulation for the artificial neural network and fuzzy-neural network system has been discussed. The experimental analysis has been shown in section 3. The results and discussions are presented in section 4. The Section 5 concludes this paper with a summary, the conclusions of this study.

II.

Methodology and Simulation Design

The architecture of fuzzy-neural network system is extended from the multilayer feedforward neural network (figure - 1). The fuzzy-neural network system consists of the components of a conventional fuzzy system except that computation of degree of membership for each rule is performed by additional layer (DOM layer) and the neural network’s learning capacity is provided to enhance the system knowledge. Thus, we are extending one additional layer (DOM layer) i.e. used to generate the degree of membership for the input-output stimuli’s. Various inputoutput stimuli’s in the form of fuzzy if-then rules are presented to this fuzzy-neural network system. Here we are discussing the mathematical foundation of both the systems for the evaluation.

A.

Neural Network Architecture

Figure 1. Architecture of Artificial Neural Network

Classification of Fuzzy-Based Information

269

The input-output stimuli’s for a particular data set are trained with the neural network architecture as shown in figure-1. It has three layers: one input layer, one output layer and a combination of hidden layer(s). Classification of fuzzy information can’t be accomplished precisely with the help of conventional artificial neural network architecture. In backpropagation training algorithm, an input pattern vector P having n features as P1 , P2 , P3 ,......Pn . Classification of these patterns will be in M classes having the output pattern respectively C1 , C 2 , C3 ,......, C M . Output layer neuron’s activation

AKO and output function

OKO could be specified as follows, A

O

o k

∑

o k

io

i=0

= f

( A KO

where function

O

w

O

k

h i

(2.1.1)

 H = f )  ∑ w io k O ih   i=0 

(2.1.2)

f ( AKO ) is used as given as,

( )

= f AKO =

µi

C

H

=

O K

B. Generating Degree of Membership The building edifice of presented fuzzy-neural network system (figure-2) is diverse from the traditional artificial neural network. The additional layer, we call degree of membership layer (DOM layer), generates the degree of membership for each of the input-output stimuli’s. This DOM layer will have exactly the same number of neurons as in input layer. Suppose we have p input-output stimuli, each having n features and each stimuli belongs to one of M classes. In fuzzy artificial neural network, the degree of membership for ith pattern (i = 1 to P patterns) with the jth class (j = 1 to M classes) can be generated as follows,

1

(2.1.3)

1 + e − KA k

o

j

= e

−

1  Pik − c k    2 σ k 

(2.2.1)

where k = 1 to L pattern feature. ck and 1k are the center and width corresponding to the whole set of patterns for that particular feature across M classes. This method of generating degree of membership is taken from the standard Gaussian membership function (MF). A Gaussian MF is determined completely by c and 1; c represents the MFs center and 1 determines the MFs width. The following figure plots Gaussian membership function Gaussian (x; 50, 20).

Now similarly, the activation and output value for the neurons of hidden layers can be written as,

A Kh =

O

h k

=

N

∑

(2.1.4)

w ik O iI

i=0

1

(2.1.5)

1 + e − KA k

h

and output value of input layer neurons and

O

i k

= f ( Aki )

(2.1.6)

In the backpropagation learning algorithm the change in weight vector is being done according to the calculated error in the network, after iterative training. The error and change in weights in the network can be calculated as,

∂E + α ∆ w ho (n ) ∂w ho

(2.1.7)

∂E + α ∆ wih (n ) ∂wih

(2.1.8)

H

∆ w ho (n + 1) = −η ∑ i =1

N

∆ wih (n + 1) = −η ∑ i =1

(

o 1M E = ∑ Cmp − Omp 2 m=1 p

)

Figure 2. Architecture of Fuzzy-Neural Network

2

(2.1.9)

for m = 1 to M output pattern features and p = 1 to P presented input patterns and

(C

p m

−O

po m

)

2

is the squared

difference between the actual output value of output layer for pattern P and the target output value.

Figure 3. Gaussian MF DOM layer will generate a vector VDOM (p, m) of degree of membership corresponding to the relationship between various input-output stimuli’s as follows;

270

VDOM

Mukul Jain, P. K. Butey and Manu Pratap Singh

µ1 µ2 µ = i ... µP

=

µ1C2 µ1C3 ... µ 1Cmµ µ 2C2 µ 2C3 ...µ 2Cµm µ iC2 µ iC3 ...µ iµCm ... ... ... ... ... C1 C2 C3 µ Cpm p µ p µ p µ...

Similarly the partial derivative of

C1 1 C1 2 C1 i

(weight between input and hidden layer units) can be derived as follows, (2.2.2)

C C C C Here µi 1 , µi 2 , µi 3 ...µ m represents the input pattern i vector for training. For classification of this input pattern vector of degree of membership with the fuzzy-neural network system a target output corresponding each input pattern in the form of degree of membership may be defined as follows, C  C  C C i (2.2.3) µ max = max  µ i 1 , µ i 2 , µ i 3 ... µ m    i   For example in the IRIS problem we are considering 4 features, 3 classes and 30 fuzzy rules as input-output stimuli’s. So the DOM layer will generate the vector of degree of membership as VDOM (30, 3). This vector of degree of membership (VDOM) will be used as input-output stimuli’s for training to the fuzzy-neural network system in support of generating the appropriate classification using the backpropagation algorithm. The error in the network can be calculated as,

E

p

where

1 = 2

(µ

∑ (µ m =1

p max

)

2

M

− S

p max

− S mp

o

)

po m

(2.2.4)

2

is the squared difference

between the actual output value of output layer and the target output value in the form of degree of membership. To minimize the error signal, coupling-strength is updated by an amount proportional to the partial derivative of

E p with respect to wik (weight between hidden and output layer units) 2

 p  ph    µ max − f  ∑ wmk S m   m   •   p f ∑ wmk S mph  S mph = S mpo − µ max m  po p ph po = (S m − µ max )S m (1 − S m )S mph

∂E p 1 ∂ = ∂wik 2 ∂wik

{

}

∂E p = ∂ mpo S mph ∂wik where

(

(2.2.5)

) (

p S mph 1 − S mpo ∂ mpo = S mpo − µ max

)

ph

and S m

is the

output from hidden layer. Here ∂ m is the error term from po

output layer.

E p with respect to who

)[ (

){

(

)}]

)([

){

(

)}]

∂E p p S oi 1 − S oi wio S oh 1 − S oh µ hCi = ∑ S oi − µ max ∂who m ∂E (2.2.6) = ∂ oh S oi ∂who

(

where

(

p 1 − S oi wio S oh 1 − S oh µ hCi ∂ oh = ∑ S oi − µ max m

and this is the error removed from hidden layer. Change in weights in the network for optimization of weights can be calculated as, H

∆ w ho ( p + 1) = −η ∑ i =1

N

∆ wih ( p + 1) = −η ∑ i =1

III.

∂E + α ∆w ho ( p ) ∂w ho

(2.2.7)

∂E + α ∆ wih ( p ) ∂wih

(2.2.8)

Experiments and Results

In order to consistently validate our method, we performed two experiments for two different sets of data i.e. IRIS and Post-operative Patient Data. First we are attempting to classify both the real-world data with the conventional artificial neural network, later the classification is carried out using the proposed approach of fuzzy-neuro system. Both experiments were run with the varying neural network architectures for generating the appropriate classification. 84 different combinations of hidden layers for artificial neural network have been used for investigating the adequacy of the artificial neural network and fuzzy neutral network system. We have chosen three combinations of hidden layers i.e. one, two and three hidden layers for both the neural network system. Tolerance of neural network has  3DUDPHWHU VXVH GLQ taken for error i.e. (MAXE ” experiments are described in table-1. Consequences of classification have been exposed in table-2 and table-3. Parameter Value Backpropagation Learning 0.7 Rate ( η ) 0.7 Momentum Term ( α ) 1 Adaption Rate ( K ) Neural Network Error 0.00001 Tolerance (MAXE) Initial weights and biased term Randomly Generated values Values between 0 and 1 Table 1. Parameters used for Artificial Neural Network and Fuzzy-neural System A. Data Set – I (IRIS) Here we have performed the experiment for classification of IRIS data with varying hidden layers. IRIS data contains

Classification of Fuzzy-Based Information four fuzzy input constraints to decide classification in three classes named IRIS Setosa, IRIS Versicolor, and IRIS Virginica. Here we are considering only 30 out of 150 rules for classification. We have trained the sample data for around 100 times. Figure 4 depicts the results presented in table 2. The results clearly show the superiority of presented approach over artificial neural network. Neurons in Various Hidden Layers 1st 2nd 3rd 2 0 0 4 0 0 6 0 0 8 0 0 2 2 0 2 4 0 2 6 0 2 8 0 4 2 0 4 4 0 4 6 0 4 8 0 6 2 0 6 4 0 6 6 0 6 8 0 8 2 0 8 4 0 8 6 0 8 8 0 2 2 2 2 2 4 2 2 6 2 2 8 2 4 2 2 4 4 2 4 6 2 4 8 2 6 2 2 6 4 2 6 6 2 6 8 2 8 2 2 8 4 2 8 6 2 8 8 4 2 2 4 2 4 4 2 6 4 2 8 4 4 2 4 4 4 4 4 6 4 4 8 4 6 2 4 6 4

Epochs for Backpropagation Algorithm 0.032257 0.032258 0.032258 0.032258 1704 Error Error Error 1727 1591 Error Error 1757 1610 2096 Error 1779 1616 2053 2810 1680 Error Error Error 1676 1575 Error Error 1674 1576 2390 Error Error Error Error 5383 1680 Error Error Error 1678 1573 Error Error 1672 1577

Epochs for Fuzzyneural Network 0.032257 0.032258 0.032258 0.032258 756 520 527 1152 769 495 527 1146 778 505 723 802 783 501 738 487 736 520 527 1152 721 495 527 525 696 475 574 1472 660 745 621 516 754 521 1275 997 735 509 783 1239 697 472

271 4 6 6 2411 556 4 6 8 Error 652 4 8 2 Error 691 4 8 4 Error 453 4 8 6 Error 633 4 8 8 5413 510 6 2 2 1680 758 6 2 4 Error 468 6 2 6 Error 535 6 2 8 Error 1080 6 4 2 1682 760 6 4 4 1572 508 6 4 6 Error 536 6 4 8 Error 1214 6 6 2 1680 789 6 6 4 1572 506 6 6 6 2381 725 6 6 8 Error 656 6 8 2 1672 689 6 8 4 1574 456 6 8 6 2474 641 6 8 8 5430 493 8 2 2 1681 762 8 2 4 Error 536 8 2 6 Error 737 8 2 8 Error 1198 8 4 2 1682 789 8 4 4 1570 533 8 4 6 Error 582 8 4 8 Error 1435 8 6 2 1682 865 8 6 4 1568 519 8 6 6 2387 727 8 6 8 Error 309 8 8 2 1685 863 8 8 4 1568 556 8 8 6 2419 1048 8 8 8 5383 937 Table 2. Classification of IRIS data using Conventional Artificial Neural Networks and Fuzzy Artificial Neural Networks.

Figure 4. Comparison of Classification of IRIS data using Backpropagation algorithm and Fuzzy-neural network system

272

Mukul Jain, P. K. Butey and Manu Pratap Singh

B. Data Set – II (Post-Operative Patient Data) Here we have performed the experiment for classification of Post-Operative Patient Data with varying hidden layers. Post-Operative Patient data contains eight fuzzy input constraints to decide classification in three classes named I (patient should be send to intensive care unit) S (patient should be discharged) and A (patient should be shifted to general floor in hospital). Here we are considering all 90 rules for classification. We have trained the sample data for around 100 times. Figure 5 depicts the results presented in table 3. The results clearly show the superiority of presented approach over artificial neural network. Neurons in Various Hidden Layers 1st 2nd 3rd 2 0 0 4 0 0 6 0 0 8 0 0 2 2 0 2 4 0 2 6 0 2 8 0 4 2 0 4 4 0 4 6 0 4 8 0 6 2 0 6 4 0 6 6 0 6 8 0 8 2 0 8 4 0 8 6 0 8 8 0 2 2 2 2 2 4 2 2 6 2 2 8 2 4 2 2 4 4 2 4 6 2 4 8 2 6 2 2 6 4 2 6 6 2 6 8 2 8 2 2 8 4 2 8 6 2 8 8 4 2 2

Epochs for Backpropagation Algorithm 0.017045 0.017045 0.017045 0.017045 0.017045 Error Error Error 0.017045 0.017045 Error Error 0.017045 0.017045 0.017045 Error 0.017045 0.017045 0.017045 0.017045 1866 Error Error Error 1808 1860 Error Error Error 1951 2492 Error 1796 2054 2662 7412 1906

Epochs for Fuzzyneural Network 0.017045 0.017045 0.017045 0.017045 933 882 1145 0.017045 911 869 0.017045 1886 909 885 1227 0.005682 895 866 1219 3086 929 899 1132 2323 912 936 0.017045 2671 906 953 1274 1669 0.017045 1009 1240 3820 949

4 2 4 Error 865 4 2 6 Error 1113 4 2 8 Error 1903 4 4 2 1914 920 4 4 4 1790 917 4 4 6 Error 1170 4 4 8 Error 0.017045 4 6 2 Error 908 4 6 4 Error 945 4 6 6 2468 1279 4 6 8 Error 2518 4 8 2 Error 874 4 8 4 Error 996 4 8 6 2564 1292 4 8 8 Error 3006 6 2 2 1968 942 6 2 4 Error 891 6 2 6 Error 0.005682 6 2 8 Error 0.005682 6 4 2 2126 937 6 4 4 1846 909 6 4 6 Error 0.017045 6 4 8 Error 0.005682 6 6 2 2616 928 6 6 4 1862 899 6 6 6 5674 1238 6 6 8 Error 3410 6 8 2 Error 899 6 8 4 2034 985 6 8 6 Error 1267 6 8 8 7410 3673 8 2 2 1988 942 8 2 4 Error 839 8 2 6 Error 0.017045 8 2 8 Error 2576 8 4 2 2284 930 8 4 4 1914 888 8 4 6 Error 0.017045 8 4 8 Error 1644 8 6 2 6860 957 8 6 4 0.017045 912 8 6 6 0.017045 1292 8 6 8 Error 994 8 8 2 0.017045 988 8 8 4 0.017045 927 8 8 6 0.017045 1326 8 8 8 0.017045 4414 Table 3. Classification of Post-Operative Patient Data using Conventional Artificial Neural Networks and Fuzzy Neural Networks.

Classification of Fuzzy-Based Information

Figure 5. Comparison of Classification of Post-Operative Patient Data using Backpropagation algorithm and Fuzzyneural network system

IV.

Discussion

To test the performance of the presented fuzzy-neural network system, it has been employed to the real-world problem of IRIS and Post-Operative Patient Data. The proceedings described in this work have some advantages over the standard backpropagation algorithm. ƒ The artificial neural network is trained with fuzzy rules rather than standard real features values. ƒ The weights of the neural network has been allocated as same as in artificial neural network system i.e. they are real in nature. In the first experiment we are trying to classify the fuzzy ifthen rules using simple backpropagation algorithm of artificial neural networks. In the second experiment, we have augmented one more layer following input layer i.e. DOM layer for generating the degree of membership for various input-output stimuli’s. The role of fuzzy-neural network is to classify the vector of degree of membership into corresponding output classes. Table 2 and 3 demonstrates the results of both the approaches. In table 2 and 3 the term “Error” represents the misclassification of fuzzy if-then rules. The conventional artificial neural network is not able to classify the rules with the particular combination of hidden layers. The epochs are representing the convergence of the network. The epochs in decimal number is representing the network convergence up to 20000 iterations with the error. Results in table 2 and 3 indicate the superiority of fuzzy-neural network over conventional neural network. Figure 4 and 5 shows the comparison study of standard backpropagation algorithm and presented fuzzy-neural network. Results indicate that only 52% convergence achieved with conventional artificial neural network trained with simple Backpropagation algorithm, while the fuzzy-neural system is able to classify the rules with 95% convergence for IRIS data. Results indicate that only 54% convergence achieved with conventional artificial neural network trained with simple

273 Backpropagation algorithm, while the fuzzy-neural system is able to classify the rules with 80% convergence for PostOperative Patient data. The results demonstrated that, within the simulation framework presented above, large significant differences exist between the performances of backpropagation feedforward neural network and fuzzy-neural network system for the classification problem of IRIS and PostOperative Patient data. These results recommend that, in most of the cases the fuzzy-neural network is adequate for classification of fuzzy if-then rules. Results show classification of both real world problems performed with the methodology up to having the maximum limit of 20000 iterations (Decimal number indicates error exists after 20000 epochs). The artificial neural network has often failed to classify the information as it contains fuzzy rules for classification purposes. Some neural network architectures cannot be trained for fuzzy data i.e. IRIS and Post-Operative Patient data. The simulation program, which we have developed in VC++ 6.0, for testing the adequacy of presented methodology over the data set of IRIS and Post-Operative Patient data, generates the initial weights randomly through its random generator. So the epochs for the algorithms will be different every time with the same network structure and the same training data set. We have chosen the best suitable epochs for designing our results by testing the same training set on the same network structure repeatedly.

V.

Conclusion

Fuzzy-neural networks are based on the integration of two complementary theories. Purpose of their integration is to compensate weaknesses of one theory with advantages of the other. Based on the analysis of several fuzzy-neural network models, we tried to introduce uniform representation model for classification of fuzzy information. Fuzzy logic facilities reasoning based on incomplete and imprecise information, known as approximate reasoning. On the other hand, artificial neural networks are developed to ape biological neural systems in performing functions such as adaptation, pattern classification, pattern recalling, pattern association and lots of other functionalities. While fuzzy logic enables mechanism for reasoning based on incomplete and imprecise information, artificial neural networks provide some remarkable abilities such as learning, adaptation and generalization. The results demonstrated that, large significant differences exist between the performances of backpropagation feedforward neural network and fuzzy-neural network system for the classification problem of IRIS and PostOperative Patient data in the terms of accuracy, convergence and epochs. These results recommend the adequacy of fuzzy-neural network for classification of IRIS PostOperative Patient fuzzy data. In first experiment i.e. using feedforward neural network, it often failed to classify the IRIS and Post-Operative Patient data as fuzzy rules are used for classification purposes. In the second experiment i.e. using fuzzy-neural network system there are some

274

Mukul Jain, P. K. Butey and Manu Pratap Singh

advantages for classification. Here we perform the classification in two steps. First we are generating the degree of membership using DOM layer. Later on, we classify the vector of degree of membership. The fuzzyneural network is working with fuzzy information i.e. fuzzy if-then rules. This is the main advantage against common neural networks and the theme of this paper. The duration of training the fuzzy information in the form of if-then rules is often shorter. We require less epochs for training the fuzzy information to the fuzzy-neural network. The neural network is also capable for generating degree of membership and classifying any unknown stimuli, not used in the training process. In this research paper, we are also trying to integrate both the approaches i.e. fuzzy logic and artificial neural networks. We found that, in all cases the presented methodology performs adequately for the classification of well-known problem of IRIS. It is also obvious from the results that the training time is improved in comparison with feedforward neural networks. Here we perform the classification in two steps. First we are generating the degree of membership using DOM layer. Later on, we classify the vector of degree of membership. The fuzzy-neural network is working with fuzzy information i.e. fuzzy if-then rules. This is the main advantage against common neural networks and the theme of this paper. The duration of training the fuzzy information in the form of ifthen rules is often shorter. We require fewer epochs for training the fuzzy information to the fuzzy-neural network, i.e. degree of membership. Once trained with adequate fuzzy information, the neural network is also capable for generating degree of membership and classifying any unknown stimuli, even not used in the training process yet. The observations made from the experiments are clearly indicating the superiority of fuzzy-neural network trained with Backpropagation algorithm over the conventional artificial neural network in terms of accuracy, convergence and epochs.

References [1]

[2]

[3]

[4]

[5]

B. Muller and J. Reinhardt, Neural Networks: An Introduction, Physics of Neural Networks, New York: Springer-Verlag, 1991. F. Rosenblatt, Principles of Neuro-dynamics: perceptrons and the theory of brain mechanisms, Spartan, New York, 1962. P. A. Devijver and J. Kittler, Feature Extraction based on the Karhunen - Loeve expansion, Pattern Recognition-A Statistical Approach, NJ: Prentice Hall, Inc, 1982. R. Schalkoff, Pattern Recognition: Statistical, Structural and Neural Appraoches, New Work, John Wiley & Sons, 1992. J. C. Bezdek, A review of probabilistic, fuzzy and neural models for pattern recognition, Fuzzy Logic and Neural Network Handbook, USA, McGraw Hill, 1996.

[6]

[7] [8]

[9]

[10]

[11]

[12]

[13]

[14]

[15]

[16]

[17] [18]

[19]

[20]

[21]

[22]

LA Zadeh, Fuzzy logic, neural networks and soft computing, One-page course announcement of CS 294-4, Spring 1993, University of California at Berkeley, 1992. LA Zadeh, Fuzzy Sets, Information and Control, 1965. LA Zadeh, The concept of a linguistic variable and its application to approximate reasoning, Parts 1, 2 & 3, Information Sciences, 1975. D Nauck, Neuro-Fuzzy Systems: An Overview, Fuzzy systems in Computer Science, Artificial Intelligence, Wiesbaden, 1994b. D Nauck, R Kruse, Choosing Appropriate NeuroFuzzy Models, Proc. 2nd European Congress on Fuzzy and Intelligent Technologies (EUFIT94), 1994a. Y. Yuan and S. Suarga, On the Integration of Neural Networks and Fuzzy Logic Systems, International Conference on Systems, Man and Cybernetics, Canada, 1995. J. S. R. Jang, ANFIS: Adaptive-Network-Based Fuzzy Inference Systems. IEEE Trans. Systems, Man & Cybernetics, vol. 23, pp. 665-685, 1993. SK Halgamuge and M Glesner, The fuzzy neural controller FuNe-II with a new adaptive defuzzification strategy based on CBAD distributions, Proc. 1st European Congress on Fuzzy and Intelligent Technologies (EUFIT93), 1993. SK Halgamuge and M Glesner, Neural networks in designing fuzzy systems for real world applications, Fuzzy Sets and Systems, 1994. N Tschichold and Gürman, Generation and improvement of fuzzy classifiers with incremental learning using fuzzy RuleNet, Proc. of the 1995 ACM Symposium on Applied Computing, 1995. HR Berenji and P Khedkar, Fuzzy rules for guiding reinforcement learning. In Int. Conf. on Information Processing and Management of Uncertainty in Knowledge-Based Systems (IPMU'92), 1992. N. Nilsson, Learning Machines, New York, McGraw Hill, 1965. D Nauck and R Kruse. NEFCLASS - A neuro fuzzy approach for the classification of data, Proc. of the 1995 ACM Symposium on Applied Computing, 1995a. D Nauck, Building neural fuzzy controllers with NEFCON, Fuzzy Systems in Computer Science, Artificial Intelligence, 1994. D Nauck and R Kruse, A fuzzy neural network learning fuzzy control rules and membership functions by fuzzy error backpropagation. In Proc. IEEE Int. Conf. on Neural Networks, 1993. D Nauck and R. Kruse. NEFCON–I: An X-Window based simulator for neural fuzzy controllers, In Proc. IEEE Int. Conf. Neural Networks, 1994b. P. Deplanques, P. Vaija, P. and R. Zapata, Fuzzy Neural Networks: A Backpropagation Algorithm Specific to the Controller of Sugeno, Internat. Conf. on Systems, Man and Cybernetics, Vancouver, Canada, 1995.

Classification of Fuzzy-Based Information [23] Y. Wang and G. Rong, A self-organizing neuralnetwork-based fuzzy system, Fuzzy Sets and Systems, 1999. [24] Y. Shi and M. Mizumoto, Some consideration on conventional neuro-fuzzy learning algorithms by gradient descent method, Fuzzy Sets and Systems, 2000. [25] Y. Yang, X. Xu and W. Zang, Design neural networks based fuzzy logic, Fuzzy Sets and Systems, 2000. [26] Y. Shi, and M. Mizumoto, A new approach of neurofuzzy learning algorithm for tuning fuzzy rules, Fuzzy Sets and Systems, 2000. [27] E. Anderson, The Irises of the Gaspe peninsula, Bulletin America, IRIS Soc., 1935. [28] A. Budihardjo, J. Grzymala-Busse, L. Woolery. Program LERS_LB 2.5 as a tool for knowledge acquisition in nursing, Proceedings of the 4th Int. Conference on Industrial & Engineering Applications of AI & Expert Systems, pp. 735-740, 1991.

275 Author Biographies Mukul Jain received his Master of Computer Applications from Jiwaji University, Gwalior, Madhya Pradesh, India. He is currently a research scholar pursuing Ph. D. in Computer Science from Dr. B. R. Ambedkar University, Agra, Uttar Pradesh, India under the guidance of Dr. Manu Pratap Singh. His current research interests include artificial neural networks, fuzzy logic. P. K. Butey is currently a Reader at Department of Computer Science, Kamla Nehru Mahavidhyalaya, Nagpur, Maharashtra, India. He serves as committee member for various Universities and conferences across India. Manu Pratap Singh received the Ph.D. degree in Computer Science from Kumaun University, Nainital, India in 2001. He is currently a Reader at Department of Computer Science, ICIS, Dr. B. R. Ambedkar University, Agra, Uttar Pradesh, India. His current research interests include artificial neural networks, fuzzy logic, genetic algorithms, software reliability, optimizations and network security. He has published more than 35 research papers in various international / national journals. Dr. Manu Pratap Singh was honored Young Scientist award in 2005 by International Academy of Physical Sciences, Allahabad. He serves as a committee member and reviewer for various international journals and conferences.