A new classification scheme using Artificial Immune Systems Learning ...

A new classification scheme using Artificial Immune Systems Learning for Fuzzy Cognitive Mapping Arthi Kannappan

Elpiniki I. Papageorgiou

Director, RVS college of Computer Application Coimbatore, Tamilnadu, India [email protected]

Dept Informatics & Computer Technology, Technological Educational Institute of Lamia, Lamia, Greece [email protected]

Abstract—In this paper, an intelligent approach based on artificial immune systems (AIS) is proposed to perform the task of classification using fuzzy cognitive map learning. Fuzzy cognitive map is an approach to knowledge representation and inference including learning capabilities; it emphasizes the connections of concepts as basic units for storing knowledge, and the structure that represents the significance of system. One of the most useful aspects of the FCM is its prediction capability as a classification tool. Little research has been done on pattern classification using FCM approach. The proposed artificial immune algorithm inspired by theoretical immunology and observed immune functions, principles and models, was considered to learn FCM network providing a category after classification. Consequently, the proposed method provides an FCM learning methodology for pattern recognition. The proposed algorithm is implemented in a previous autism classification problem, as well as in some benchmark machine learning datasets to show its functionality. Keywords- learning; artificial immune system; fuzzy cognitive map, classification, clone

I.

INTRODUCTION

Fuzzy cognitive mapping (FCM) is a method for analyzing and depicting human perception of a given system. The method produces a conceptual model which is not limited by exact values and measurements, and thus is well suited to represent relatively unstructured knowledge and causalities expressed in imprecise forms. In 1986, Kosko first proposed this concept, which combined cognitive map with fuzzy logic [1]. FCM can be considered as an integration of multifold subjects, including neural network, graph theory and fuzzy logic. It is a dynamic tool involving feedback mechanisms [1]. The advantageous modeling features of FCMs, such as simplicity, adaptability and capability of approximating abstractive structures encourage us to use them for complex problems [2,3]. FCMs have been used in many different scientific fields for modeling, control, management and decision making [4]. The ability of FCMs to improve their operation on the light of experience (learning of the connection matrix) is a crucial issue in modeling and optimization. The adaptation of the connection matrix (known as weight matrix) can be done by diverse unsupervised and evolutionary type learning methods, such as unsupervised learning based on the Hebbian method

[5,6], supervised ones with the use of evolutionary computation [7,8,9] and/ or gradient-based methods [10,11]. In most known approaches to learning FCMs, the set of concept labels C is provided a-priori by expert, and only the weight matrix is drawn from raw data. Most of the FCM learning algorithms adapt the weight matrix using the available knowledge from historical data and only in few cases from experts. The produced FCM model after training follows system’s/ problem’s characteristics. In the case of evolutionary computation techniques, the FCM design is based on the minimization of an error/cost or fitness function. The fitness function for each population-based algorithm might be modified according to the problem type [12]. In a previous study, an immune algorithm for decision making was suggested, to find the initial state of system in given goal state. The proposed algorithm takes the error objective function and constraints as antigen, through genetic evolution; an antibody that most fits the antigen becomes the solution [13]. However, few studies have been accomplished proposing a classification approach based on FCM, using the FCM learning capabilities [14,15,16,17]. An intelligent approach based on artificial immune systems (AIS) [18,19] is proposed here as an alternative way to perform the task of FCM learning for classification. AIS inspired by the vertebrate immune systems, use the immunological properties to support a wide range of applications including pattern recognition [20], intrusion detection [21], clustering [22], and optimization [23]. AIS are nonlinear models, and hence are flexible in modeling complex real word relationships and they inherit the memory property of human immune system and can recognize the same or similar antigen quickly at different times [24,25]. AIS has strong capabilities of pattern recognition, learning and associative memory, hence it is natural to view AIS as a powerful information processing and problem-solving paradigm in both the scientific and engineering fields [25,26]. AIS possess nonlinear classification properties along with the biological properties such as positive and negative selection, clonal selection, and immune memory. Therefore, AIS, like genetic algorithms and neural nets, is an intelligent tool for

advanced pattern recognition [19], and in this work, it is used for FCM learning. II.

APPLICATIONS OF ARTIFICIAL IMMUNE SYSTEMS IN PATTERN RECOGNITION

The effectiveness of classification and recognition systems has improved in a great deal to help medical experts in diagnosing diseases using Artificial Immune Recognition System (AIRS) [27]. AIRS inspired by the vertebrate immune systems, use the immunological properties to support pattern recognition. Similar to the use of artificial neural networks, performing pattern recognition with an AIS usually involves three stages: 1) defining a representation for the patterns; 2) adapting (learning or evolving) the system to identify a set of typical data; and 3) applying the system to recognize a set of new patterns [20]. Some representative AIRS algorithms are presented for classification and diagnosis. In the work of Castro and Timmis (2002), a theoretical comparison between artificial immune systems and neural network models for pattern recognition was accomplished. Aspects such as the basic units composing each system, their respective types of adaptation mechanisms, the types of memory presented, and how they present generalization capabilities were stressed. The algorithms presented can be directly employed and/or adapted as alternatives to solve the same types of pattern recognition problems as neural networks, or to complement their potentialities [19]. A Tree Structured Artificial Immune Network (TSAIN) for data clustering and classification was explored in [28] where a topological link is setup between two antibodies immediately after one has reproduced by another with no need to set a threshold for this connection. A Chaos Immune Network (CIN) algorithm for multimodal function optimization was enhanced using the chaos variable to simulate proliferation mode of immune cells to improve searching accuracy [29]. Meryem SAIDI et al used a Modified AIRS2 (MAIRS2) to replace the K nearest neighbors’ algorithm with the fuzzy Knearest neighbors to improve the diagnostic accuracy of diabetes diseases. The performances of the AIRS2 and MAIRS2 are evaluated and the highest classification accuracy obtained when applying the AIRS2 and AIRS2 with fuzzy Knn algorithm [30]. Kodaz et al. (2009) proposed information gain based artificial immune recognition system (IG-AIRS) which minimizes the negative effects of taking into account all attributes in calculating Euclidean distance in shape–space representation for diagnosing thyroid function based on laboratory tests [31]. Polat et al. (2009) compared classifier algorithms including the C4.5 decision tree classifier, the least squares support vector machine (LS-SVM) and the artificial immune recognition system (AIRS) for diagnosing macular and optic nerve diseases from pattern electro-retinography signals [32]. Complementary to the role of artificial immune selection, clonal selection is the theory used to explain how an immune response is mounted when a non-self antigenic pattern is recognised by a B-cell [33]. Clonal selection algorithm has proved useful for pattern classification in several applications

[26]. The Clonal Selection principle is the whole process of antigen recognition, cell proliferation and differentiation into memory cell. Several artificial immune algorithms have been developed imitating the clonal selection theory [34]. A clonal selection algorithm named CLONALG for learning and optimization generates a population of N antibodies, each specifying a random solution for the optimization process. During each iteration, some of the best existing antibodies are selected, cloned and mutated in order to construct a new candidate population. New antibodies are then evaluated and certain percentage of the best antibodies is added to the original population. Finally a percentage of worst antibodies of previous generation are replaced with new randomly create ones [34]. An Adaptive Clonal Selection (ACS) algorithm as a modification of CLONALG suggests some modifications to the CLONALG based on an analysis of the operators for selecting the amount of mutation and number of clones to overcome the drawbacks of CLONALG such as the several parameters used and the binary representation and an Adaptive Immune Clonal Strategy Algorithm (AICSA) is proposed for solving numerical optimization problems [35]. An improved clonal selection algorithm was introduced where a learning operator was used to enhance the learning mechanism of CLONALG and also improve the detection efficiency [36]. A Clonal Chaos Adjustment Algorithm (CCAA) for Multi-modal Function Optimization enhances the global convergence performance of CLONALG and also it takes advantages of the dynamic properties of chaos system, and introduces the chaotic search mechanism into the CLONALG to improve its search efficiency [37]. III.

ARTIFICIAL IMMUNE SYSTEM

An artificial immune system (AIS) is a class of adaptive or learning computer algorithm inspired by function of the biological immune system, designed for and applied to difficult problems such as intrusion detection, data clustering, and classification, and search problems [26]. A. AIRS Algorithm Artificial Immune Recognition System is a biologically inspired computing paradigm such as Neural Networks, and Swarm Intelligence. Procedure for AIRS algorithm can be described as: Initialization: Dataset is normalized into [0,1] interval. Affinity threshold variable is computed. Training dataset is normalized to [0,1] range. Affinity Threshold (AT) variable is the average value of affinities between antigens. Memory cell pool and ARB pool are initialized. Antigen Training: Each data point in training set is provided to the memory pool to stimulate the recognition cells in memory pool. Stimulation values are assigned to the recognition cells and the cell, which has maximum stimulation value, is marked as the best memory cell. This cell is used for affinity maturation and cloned, then mutated. These clone cells are put into the Artificial Recognition Ball (ARB) pool.

Competition for limited resource: After mutated clones are added to the ARB pool, competition starts. Antigen is used to stimulate the ARB pool and limited resource is computed with respect to stimulation values. ARBs with very limited resource or no limited resource are deleted from ARB pool. This step continues until stopping criteria is met. Otherwise, mutated clones of ARBs are produced.

Step 2: For each pattern of P, present it to the population M and determine its affinity with each element of the population M; Step 3: Select n of the best highest affinity elements of M and generate copies of these individuals proportionally to their affinity with the antigen. The higher the affinity, the higher the number of copies, and vice-versa;

Memory cell selection: Candidate memory cell that has a maximum stimulation score from ARB pool is chosen. ARB is copied to the memory cell pool if ARB's stimulation value is better than the original best matching memory.

Step 4: Mutate all these copies with a rate proportional to their affinity with the input pattern: the higher the affinity, the smaller the mutation rate;

Classification: Memory cell pool is used for cross-validation and K-nearest neighbor approach is applied for classification.

Step 5: Add these mutated individuals to the population M and reselect m of these maturated individuals to be kept as memories of the systems;

B. Parallel AIRS The other interesting area of research for the AIRS algorithm is work into exploiting the parallelism inherent in the techniques base metaphor [19]. Few AIS algorithms exploit the distributed nature and parallel processing attributes exhibited in the mammalian immune system. The approach to parallelizing AIRS was simple, involving the following steps in addition to the standard training scheme: 1. Initialize the input and the parameters 2. Divide the obtained data set into np number of partitions, where np is the number of desired processes running AIRS 2. Allocate training partitions to processes and prepare memory pools 3. Gather the np number of memory pools 4. Use a merging scheme of affinity between memory cells for creating a master memory pool for classification C. Clon Algorithm The CLONALG algorithm is also a useful AIS algorithm, successfully applied to a number of machine learning and artificial intelligence problems such as binary character recognition, multimodal function optimization and the travelling salesperson problem. De Castro and Von zuben developed the Clonal Selection Algorithm on the basis of clonal selection theory of the immune system. It was proved that it can perform pattern recognition and involves the selection of antibodies (candidate solutions) based on affinity either by matching against an antigen pattern or via evaluation of a pattern by a cost function. Selected antibodies are subjected to cloning proportional to affinity, and hyper mutation of clones inversely proportional to clone affinity. The resultant clonal-set competes with the antibody population for membership in the next generation, and finally low-affinity population members are replaced by randomly generated antibodies. The pattern recognition variation of the algorithm includes a maintenance memory solution set which in its entirety represents a solution [34,35]. The CLONALG algorithm can be described as follows: Step 1: Randomly initialize a population of individual (M);

Step 6: Repeat steps 2 to 5 until a certain criterion is met III.

FUZZY COGNITIVE MAP

Fuzzy Cognitive Maps are fuzzy logic based graph structures for the representation of causal reasoning [1]. FCMs are applicable to soft knowledge domains as their structure helps systematic causal propagation. These graph structures describe the behavior of an intelligent system using concepts, which represent entities, states, variables or characteristics of the system. Each node in FCM represents a concept Ci, where C is the set of concepts. Similarly, each arc (Ci, Cj) is directed as well as weighted, and represents causal link between concepts, showing how concept Ci causes concept Cj. The arc weights are associated with a weight value matrix Wn, which is n x n square matrix with each Wij having values in [-1, ... , 1]. The positive and negative signs on the directed edge represent different properties of this effect respectively. The positive sign represents that Ci has positive effect to Cj, and the negative sign represents Ci has negative effect to Cj. The arc weights are used to represent either ‘expert opinion’, semi-quantitative or quantitative data on the relative degree to which one variable influences another [3]. The mathematical representation of FCMs has the following form: N

Ai( k +1) = ∫ ( Ai( k ) + ∑ A(j k ) .W ji

(1)

j ≠i j =1

where Ai(k) is the value of concept Ci at step k, Aj( k) is the value of concept Cj at step k, Wji is the weight of the interconnection from concept Cj to concept Ci and f is the threshold function that squashes the result of the multiplication in the interval [0,1]. The transformation function is used to reduce unbounded weighted sum to a certain range, which hinders quantitative analysis, but allows for qualitative comparisons between concepts [3]. A. Learning algorithms The learning approaches for FCMs are concentrated on learning the connection matrix E, based either on expert intervention and/or on the available historical data. According to the available type of knowledge, the learning techniques

could be categorized into three groups; Hebbian-based, population-based and hybrid combining the main aspects of Hebbian-based- and evolution-based- type learning algorithms [12]. The unsupervised Hebbian-based learning algorithms, are mainly based on experts’ knowledge, where in the case of evolutionary algorithms, the experts are substituted by historical data and these learning algorithms are oriented towards finding models that mimic the input data [9]. They are optimization techniques, and for this reason, they are computationally quite demanding. Several population-based algorithms for training FCMs, such as evolutionary strategies, genetic algorithms, real coded generic algorithm - RCGA, swarm intelligence, chaotic simulated annealing, Tabu search, and so on, have been proposed [12]. They have gained the interest in diverse applications by researchers. B. Previous FCM works on Classification Little research has been done on pattern classification using FCM. Four works have been presented recently to applying FCM with its learning capabilities for classification tasks. The first work is devoted to the FCM learning using unsupervised non-linear hebbian learning (NHL) algorithm which was developed using the hebbian algorithm on non-linear units for the autistic disorder prediction problem. The classification approach was based on human knowledge and experience, as well as on historical data (patterns). The proposed algorithm presented high classification accuracy of 82% [14]. Another very challenging learning category, which has also been applied for classification tasks and recently emerged, is the Ensemble learning [15]. This ensemble learning method inherits the main ideas of ensemble based learning approaches, such as bagging and boosting. FCM ensemble learning is an approach where the model is trained using non linear Hebbian learning (NHL) algorithm and further its performance is enhanced using ensemble techniques. The fuzzy cognitive map ensembles were used to learn the produced FCM by the already known and efficient data driven NHL algorithm. This new proposed approach of FCM ensembles, showed results with higher classification accuracy instead of the NHL alone learning technique [15]. Papakostas et al. (2008) implemented FCMs for pattern recognition tasks. In their study, a new hybrid classifier was proposed as an alternative classification structure, which exploited both neural networks and FCMs to ensure improved classification capabilities. A simple GA was used to find a common weight set which, for different initial state of the input concepts, the hybrid classifier equilibrate to different points [16]. Recently, Papakostas et al. (2012) presented some Hebbian-based approaches for pattern recognition, showing the advantages and the limitations of each one [17]. IV.

PROPOSED LEARNING ALGORITHM

AIRS algorithm has a number of user configurable parameters for fine-tuning the training schedule to specific problem domains. It was shown that the technique remains reasonably stable (in terms of classification accuracy) over a range of parameters on a number of standard machine learning

datasets [22]. In what follows, a description of each AIRS parameter is given. (1) Affinity Threshold Scalar (ATS): the mean affinity between a set of antigens from the training dataset. The affinity threshold scalar provides a means of adjusting the automatic threshold by making it softer (less than the mean) or harder (more than the mean). The effect of softening the threshold causes less replacement of best matching memory cells by candidate memory cells, and the reverse is true when the threshold is hardened. Common values for this user parameter are in the range [0.1, 0.3], which is a significant softening of the mean. (2) Clonal Rate: Firstly, it is used in conjunction with the hypermutation rate to determine the number of clones that a best matching memory cell can create to population the ARB pool. Secondly, it is used to determine the number of clones an each ARB can create during the ARB refinement stage. Finally, it is multiplied by an ARBs stimulation to determine its resource allocation. This means that the number of ARB clones created will be in the range of [0,clonalRate]. It also means that allocated resources for an ARB will also be in this range, which has an impact on the total number of resources that should be allocated. (3) Hypermutation Rate: Used with the clonal rate and cell stimulation, to determine the number of mutated clones a best matching memory cell can create. As mentioned, the number of clones will be in the range of [0,clonalRate] which is then increased by a factor of the hypermutation rate. (4) Total Resources: The total resources place a direct limit on the number of ARBs that can coexist in the ARB pool. (5) Stimulation Threshold: The stopping criterion to the ARB refinement process is when the average stimulation value is above the stimulation threshold. This parameter controls the amount of refinement performed on ARBs for an antigen. The range for the stimulation threshold lies in the range of [0,1]. (6) Number of Initialisation Instances: Is the number of randomly selected training instances used to seed the memory pool. This parameter can be in the range [0, total training instances], and is commonly set to low values such as zero or one. (7) k-Nearest Neighbours: The kNN parameter is only used during the read-only classification stage of the algorithm. It determines the number of best match memory cells used to vote by majority on the classification of unseen antigens (data vectors). A. Steps of the proposed FCM AIRS algorithm Given parameters: Mutation Rate(MR),Affinity threshold (AT), Stimulation Threshold(ST), Clonal Rates (CR), Hyper mutation rate (HMR), Knn, No. of resources (RES), total number of Instances(tot), memory cell (MC) Stage 1: Preprocessing step Step 1: FCM construction: Every expert (like psychologist, pediatricians, special educators) describes each interconnection between concepts with a linguistic fuzzy rule

to assign the weight. The fuzzy rules for each interconnection can be represented using the following form: IF a change B occurs in the value of concept Cj THEN a change D in the value of concept Ci is caused. Infer: The influence from Cj to Ci is E where B, D and E are fuzzy linguistic variables that experts use to describe the variance of concept values and the degree of influence from concept Cj to Ci. Step 2: The inferred fuzzy weights are combined using the SUM method as suggested by the experts. Step 3. Using defuzzification method of centroid, weight is calculated as

X* =

∫ μ ( x) xdx ∫ μ ( x)dx i

(2)

i

Where x is the input value and

μ i (x) is

the membership

value of the input variable x. Step 4. All the linguistic weights are transformed into a numerical weight w f which lies in the range [-1,1]. Step 5. Calculation of input to the AIRS algorithm: Input attributes = w f * d(x,y) n

where d(x,y) =

∑ (x

f

(3)

− y f )2

f =1

and

w f is the calculated weight of a feature f given in

appendix A . d is the distance between 2 data instances x and y. Stage 2: Seeding Phase Randomly chosen data vectors are used to form an initial population of memory cells which belongs to different class. Memory Pool (MP) for each class is defined: MP(i) = randomly selected vector(training data) where i= 1 to n, where n is the number of classes

(4)

Stage 3: Training Phase Step 1: Calculation of affinity and stimulation: Affinity= Euclidean distance / max possible distance (5) Stimulation = 1 – Affinity (6) Step 2: Process of cloning and mutation takes place on stimulation value. Step 3: Memory cell identification and ARB generation process: Clone and mutate the highest affinity memory cell and add them to the set of ARBs (P). No of mutated Clones = stimulation × CR× HMR (7) ARBs (P) = ARBs (P) + No of mutated Clones (8) Step 4: Competition for resources and development of a candidate memory cell: (i) For each ARBs(p), calculate resources = stimulation * CR (9) total(resources) = sum(resources) (10) (ii) IF total (resources) > RES

sort ARBs(p) by allocated resources descending, delete resources of ARBs(p) starting from end of the list and delete ARB with zero resources until (calculated no of resources = RES) and (average stimulation of each ARBs(p) ) > ST Step 5: Memory Cell (MC) introduction: MC = maximum stimulation value from ARB (11) IF stimulation(ARBs(P) > stimulation(memory pool data) THEN memory cell pool = ARBs(P) Stage 4: Classification Stage Now memory pool cells become core for AIRS classifier. The classification is performed with the k best matches to data instances located and the class is determined by using a majority vote [21]. B. Steps of the proposed FCM- Parallel AIRS algorithm For FCM- parallel AIRS, the steps are similar as FCM-AIRS algorithm and only for classification, a merging scheme of affinity between memory cells is used for creating a master memory pool in stage 4 (for details see [put a reference]). C. Steps of the proposed FCM- CLONALG algorithm Step 1. Initialization: Initialize the parameters like Antibody pool size, clonal factor, number of generations, remainder pool ratio, seed value, selection pool size and total replacement. Step 2. FCM construction using AIRS: FCM is constructed to get the antibody pool fixed size N using the steps from FCM AIRS algorithm as described from (1) to (5). Step 3. Loop: The algorithm then proceeds by executing a number of iterations of exposing the system to all known antigens. The number of iterations or generations G that the system executes is user configurable, though the system can use a problem specific stopping condition. Here, a stopping condition is the maximum number of iterations. a. Select Antigen: A single antigen is selected at random without replacement (for the current generation) from the pool of antigens b. Exposure: The system is exposed to the selected antigen. Affinity values are calculated for all antibodies against the antigen. Affinity is a measure of similarity, and is problem dependent. It is common to use Hamming distance. Calculate the affinity as L ⎧1, abi ≠ agi (12) D = ∑δ = ⎨ i =1 ⎩0, abi = agi where D is the Hamming distance, ab is the antibody, ag is the antigen and L is the length of the bit string. c. Selection: A set of n antibodies are selected from the entire antibody pool that have the highest affinity with the antigen. d. Cloning: The set of selected antibodies are then cloned in proportion to their affinity (rank based). The ordered list is then iterated, and the number of clones created from each antibody is calculated as follows: ⎡ βN ⎤ + 0.5⎥ (13) numclones = ⎢ ⎣ i ⎦ where β is a clonal factor, N is the size of the antibody pool, and i is the antibody current rank where i∈ [1,n].

e. Affinity Maturation (mutation): The clone (set of duplicate antigens) is then subjected to an affinity maturation process to better match the antigen in question. Here, the degree of maturation is inversely proportional to their parent’s affinity (rank based), meaning that the greater the affinity, the lower the mutation. f. Clone Exposure: The clone is then exposed to the antigen, and affinity measures are calculated. The total number of clones (NC) prepared for each antigen exposure to the system is thus calculated as: n

NC = ∑ numclones

(14)

i =1

g. Candidature: The antibody or antibodies with the highest affinity in the clone are then selected as candidate memory antibodies for placement into m. If the affinity of a candidate memory cell is higher than that of the highest stimulated antigen from the memory pool m, then it replaces said antigen. Group replacements occur in a similar, but batched manner. h. Replacement: Finally, the d individuals in the remaining r antigen pool with the lowest affinity are replaced with new random antibodies. Step 4: Finish: After the completion of the training regime, the memory m component of the antigen pool is then taken as the algorithms solution. Depending on the problem domain, the solution may be a single best individual antigen or the collective of all antigens in the pool. The flowchart of the FCM-AIRS learning algorithm is depicted in Figure 1. FCM construction: Initializing the input attributes and other parameters

V.

IMPLEMENTATION OF THE PROPOSED ALGORITHM

A. Dataset Description The proposed algorithms FCM-AIRS, FCM-Parallel AIRS and FCM-ClonALG were implemented to Autism dataset which contains 23 input attributes. The autism disorder dataset has been selected due to the previous application of FCMbased learning approaches on it [14,15]. A number of datasets were also retrieved from the well-known UCI machine learning repository [38] to show further the performance of the AIRS algorithms. There were 538 instances, 10 attributes and 2 classes for Breast cancer datasets, 120 instances, 8 attributes with 2 classes for Echocardiogram datasets, 332 instances ,8 attributes and 6 classes for Ecoli datasets,149 instances, 19 attributes and 2 classes for Hepatitis datasets and 345 instances, 7 attributes and 2 classes for Liver cancer datasets. The proposed FCM-based AIRS algorithms were not able to be exploited in these benchmark datasets, as there is no any previous knowledge about the interconnections between instances and classes. B. Classification Performance The performance of the Hebbian-like learning algorithms in pattern classification problems as well as the ensemble learning algorithms have been studied in previous studies [14-17]. It was observed that the FCMs which are enhanced by learning algorithms are able to be used for prediction tasks. This works further confirms these results, showing the performance of this new classification scheme based on AIRS algorithms for FCMs. TABLE I.

Calculate weight using SUM method

Defuzzify the linguistic weight into numerical weight

w* d(x,y) f

Input attributes = n

d(x,y) =

∑

(x

f

− y f )2

f =1

Find affinity = d/max (d), stimulation = 1 - affinity For each input : Generate ARB: ARBs(p) = ARBs((p) + no of mutated clones Competition for resources and Refine ARB

If Avg (stim)>Stim threshold and no of resources == RES

Find Memory Cell candidate and memory cell pool

Classification of autism via majority vote

Figure 1. Flowchart of the porposed approach

PARAMETERS USED IN AIRS AND PARALLEL AIRS

Parameters

Values

Mutation Rate Affinity threshold Stimulation Threshold Hyper mutation rate No.of resources Clonal Rates Knn

0.10 0.2 0.99 2.0 150 10 1

TABLE II.

PARAMETERS USED IN CLONALG CLASSIFICATION

Parameters

Values

Antibody Pool Size Clonal Factor Number of Generations Remainder Pool Ratio Seed Selection Pool Size Total Replacement

30 0.1 10 0.1 1 20 0

For this purpose, a set of appropriate experiments has been arranged, by using the pattern classification scheme presented in the section IV. The weights are initialized based on experts’ knowledge and the training data are provided with shuffle. The configuration parameters of the AIRS and CLONAL algorithms are summarized in the following Table I and Table II, where in the main properties of all the datasets used in this experimental study are summarized in Table III. The algorithms’ classification performance is presented in Table IV.

TABLE III.

DATASETS PROPERTIES

Datasets

Instances

Attributes

Autism

40

23

Classes 3

Breast Cancer

538

10

2

Echocardiogram

120

8

2

Ecoli

332

8

6

Hepatitis

149

19

2

Liver Cancer

345

7

2

Comparing the produced classification accuracy of the autism dataset, implementing the AIRS and the FCM-AIRS algorithms, as well as the parallel AIRS and Clonal Algorithm for FCMs, we observe that the proposed learning approaches outperform the operation of the AIRS, Parallel AIRS and CloALG. The use of the initially constructed FCM model in the AIRS learning algorithm increases the classification accuracy due to the consideration of an initial weight matrix, which is assigned by experts’ knowledge. This previous knowledge has been taken into consideration in the AIRS-based three learning configurations, and improves the classification performance. The results showed that the FCM-AIRS, FCM-Parallel AIRS and FCM-ClonALG methods are more efficient concerning the classification rate than the AIRS-based and clonALG algorithms. Their performance is inferior to that of the FCM learning under the same conditions. Comparing these results with the previous ones, using ensemble learning [15] and some Hebbian-based approaches for pattern recognition [16,17], we observe that this approach is more efficient in correctly categorizing patterns. Kappa is a chance-corrected measure of agreement between the classifications and the true classes. Kappa statistics refers to a measure of hard classification agreement, where Kappa=1 indicates perfect agreement and Kappa=0 no agreement. Along with the autism disorder dataset used in the previous investigation, five more datasets are selected from UCIMachine Learning Repository [38] in order to compare the performance of the AIRS algorithms with the configuration parameters derived from the preceding section. The use of FCM-AIRS approaches in these UCI datasets is not possible as there is no experts’ knowledge for constructing the initial weight matrix. Therefore only the AIRS algorithms and their configurations were implemented to show their functionality. To sum up, FCM AIRS, FCM-Parallel AIRS and FCMClonALG classifiers are able to categorize the data in the respective classes for the autism dataset, where in the case of UCI machine learning datasets, the AIRS and Parallel AIRS are able to classify the patterns with almost average accuraciesof 80%, depending on the type of data. VI.

capabilities in the FCM model for learning and the results encourage us to further exploit it in this task. Surely, more research work is needed to be done towards more investigation of the learning capabilities of FCMs and the effectiveness of FCM-based methods in pattern recognition. REFERENCES [1] [2] [3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]

[13]

[14]

[15]

[16]

CONCLUSION

It is concluded that the proposed AIRS-based learning algorithms for classification estimate autism disorder with reasonably high overall accuracy, sufficient for this application area and therefore, they are established as an efficient learning approach for FCMs. This work presents our first trial to implement the artificial immune system characteristics and

[17]

[18]

B. Kosko, “Fuzzy cognitive maps”, International Journal of ManMachine Studies, vol. 24, pp. 65-75, 1986. M. Glykas, Fuzzy Cognitive Maps-Theories, Methodologies, Tools and Applications, Springer, July 2010. C.D. Stylios, P.P. Groumpos, “Modeling complex systems using fuzzy cognitive maps”, IEEE Transactions on Systems, Man and Cybernetics Part A, vol. 34, pp. 155-162, 2004. E.I. Papageorgiou and J.L. Salmeron, “A Review of Fuzzy Cognitive Map research at the last decade”, IEEE Transactions on Fuzzy Systems vol. 21 91), pp. 66-79, Feb. 2013. E.I. Papageorgiou, C.D. Stylios and P.P. Groumpos, “Unsupervised learning techniques for fine-tuning Fuzzy Cognitive Map causal links”, Intern. Journal of Human-Computer Studies, vol. 64, pp. 727-743, 2006. W. Stach, L. Kurgan and W. Pedrycz, “Data-driven nonlinear Hebbian learning method for fuzzy cognitive maps”, in Proc. IEEE international conference on fuzzy systems (FUZZ–IEEE), USA, pp. 1975–1981, 2008. W. Stach, L. Kurgan, W. Pedrycz, M. Reformat, “Genetic learning of fuzzy cognitive maps”, Fuzzy Sets and Systems, vol. 53, pp. 371–40, 2005. S. Alizadeh, M. Ghazanfari, M. Jafari, S. Hooshmand, “Learning FCM by Tabu Search”, International Journal of Computer Science, vol. 2, pp. 143-149, 2008. E.I. Papageorgiou, W. Froelich, “Multi-step Prediction of Pulmonary Infection with the Use of Evolutionary Fuzzy Cognitive Maps”, Neurocomputing Journal, vol. 92 , pp. 28-35, September 2012. A. Yastrebov, K. Piotrowska, “Simulation Analysis of Multistep Algorithms of Relational Cognitive Maps Learning.” In: A. Yastrebov, B. Kuzminska-Sososnia, M. Raczynska (eds.) Computer Technologies in Science, Technology and Education. Institute for Sustainable Technologies - National Research Institute, pp. 126-137, 2012. S.S. Madeiro and F. J. Von Zuben, “Gradient-based Algorithms for the Automatic Construction of Fuzzy Cognitive Maps”, Proc. 11th International Conference on Machine Learning and Applications, Boca Raton, FL, pp. 344 – 349, Dec. 2012. E.I. Papageorgiou, “Learning Algorithms for Fuzzy Cognitive Maps-A review study”, IEEE Transactions on Systems Man and Cybernetics (SMC)-Part C, vol. 42, no.2, pp. 150-163, March 2012. C. Lin, “An immune algorithm for complex fuzzy cognitive map partitioning”, in Proc. Genetic and Evolutionary Computation Conference, GEC Summit 2009, Shanghai, China, pp. 17-23, 2009. K. Arthi, A. Tamilarasi and E.I. Papageorgiou, “Analyzing the performance of Fuzzy Cognitive Maps with non linear hebbian learning algorithm in predicting autistic disorder”, International Journal of Expert Systems with Applications, vol. 38 (3), pp.1282-1292, March 2011. E. I. Papageorgiou and K. Arthi, “Fuzzy Cognitive Map ensemble learning paradigm to solve classification problems: application to autism identification” International Journal of Applied soft computing, vol. 12 (12), pp. 3798-3809, Dec 2012. G. A. Papakostas, Y. S. Boutalis, D. E. Koulouriotis, and B. G. Mertzios, “Fuzzy cognitive maps for pattern recognition applications”, International Journal of Pattern Recognition and Artificial Intelligence vol. 22 (8), pp. 1461-1486, 2008. G.A. Papakostas , D.E. Koulouriotis, A.S. Polydoros, V.D. Tourassis, “Towards Hebbian learning of Fuzzy Cognitive Maps in pattern classification problems”, Expert Systems with Applications ,2012 ,in press. D. Dasgupta, Artificial Immune Systems and Their Applications. Germany, Springer, 1999.

[19] L. N. De Castro, and J. Timmis, “Artificial Immune systems: A New Computational Intelligence Approach”, London, Springer-Verlag, 2002. [20] D. Dasgupta, Y. Cao and C. Yang, “Immunogenetic approach to spectra recognition”. in: Proc GECCO’99, San Francisco, CA , USA, 149- 155, 1999. [21] J. Kim, and P. Bentley, “Towards an artificial immune system for network intrusion detection: An investigation of clonal selection with a negative selection operator”. In Proceedings of the congress on evolutionary computation (CEC), Seoul, Korea, vol. 2, pp. 1244-1252, 2001. [22] J. Timmis, M. Neal and J. Hunt, “An artificial immune system for data analysis.” Biosystem, vol. 55(1/3), pp. 143-150, 2000. [23] L.de.Castro and F. Zuben, “Learning and Optimization Using the Clonal Selection Principle”, IEEE Transactions on Evolutionary Computation, vol. 6(3), pp. 239-251, 2002. [24] J. Timmis and M. Neal, “A resource limited artificial immune system”, Knowledge Based Systems, vol. 14(3-4), pp 121-130, 2001. [25] A. Al-Enezi, M.F. Abbod and S. Alsharhan Al-Enezi “Artificial Immune Systems – Models, Algorithms and applications”, IJRRAS, vol. 3 (2), pp.118-131, May 2010. [26] E. Hart, and J. Timmis, “Applications areas of AIS: The past, the present and the future,” Applied soft computing, vol. 8(1), pp. 191-201, 2008. [27] J. Zheng, Y. Chen and W. Zhang “A Survey of artificial immune applications” Artif Intell Rev , vol. 34, pp. 19–34 , 2010. [28] C. Zhang and Z. Yi, “An Artificial Immune Network Model Applied to Data Clustering and Classification”, Proc. 4th International Symposium on Neural Networks, Nanjing, China, pp. 526-533, 2007. [29] J. Lv, “Study on Chaos Immune Network Algorithm for Multimodal Function Optimization”, Proc. Fourth International Conference on Fuzzy Systems and Knowledge Discovery (FSKD 2007), Haikou, vol. 3, pp. 684-689, 2007.

[30] M. Amine Chikh, Meryem Saidi, Nesma Settouti, “Diagnosis of diabetes using an Artificial Immune Recognition System(AIRS) with fuzzy Knearest neighbor”, Journal of Medical Systems, vol. 36(5), pp. 27212729, June 2011. [31] H. Kodaz, S. Özşen, A. Arslan, S. Güneş, “Medical application of information gain based artificial immune recognition system (AIRS): Diagnosis of thyroid disease”, International Journal of Expert systems with applications, vol. 36 (2), pp 3086-3092, March 2009. [32] K. Polat, S. Gunesa, and Sulayman Tosunb “Diagnosis of heart disease using artificial immune recognition system and fuzzy weighted preprocessing” Pattern Recognition, vol. 39, pp. 2186 – 2193, 2009. [33] F.M Burnet, The Clonal Selection Theory of Acquired Immunity, Cambridge University Press, 1959. [34] R. Liu, L. Jiao and H. Du ,”Adaptive Immune Clonal Strategy Algorithm”, Proc. 7th International Conference on Signal Processing (ICSP’04), vol. 2, pp. 1554- 1557, 2004. [35] Y. Yu and C. Hou ,”A Clonal Selection Algorithm By Using Learning Operator”, Proceedings of the Third International Conference on Machine and Cybernetics, Shanghai, vol.5, pp. 2924 – 2929, 2004. [36] J. Hunt and D. Cooke, “Learning Using an Artificial Immune System”, Journal of Network and Computer Applications, vol 19, pp.189-212, 1996. [37] V. Cutello, G. Narzisi, G. Nicosia and M. Pavone, “An Immunological Algorithm for Global Numerical Optimization.” Artificial Evolution: 7th Int. Conference, Evolution Artificielle, EA, France, Springer, LNCS vol. 3871, pp. 284-295, 2005. [38] UCI Machine Learning Repository. Irvine, http://archive.ics.uci.edu/ml/datasets.html.

TABLE IV. CLASSIFICATION PERFORMANCE OF DIFFERENT DATASETS Learning Algorithm

Correctly Classified %

AIRS Parallel AIRs Clon ALG

92.66 95.9 87.5

0.8251 0.9106 0.7558

FCM - AIRS

94.73

0.712

FCM-Parallel AIRS FCMclonALG

Kappa Statistics

Mean Absolute Error

Root mean Squared error

Autism (without FCM model) 0.0667 0.2582 0.0333 0.1826 0.0833 0.2887 Autism (with FCM model) 0.0833

0.2887

95.83

0.825

0.0211

0.1703

88.89

0.871

0.0278

0.1667

AIRS Parallel AIRS Clon ALG

97.21 96.09 96.84

0.94 0.91 0.93

AIRS Parallel AIRS Clon ALG

80.83 80.83 70

0.54 0.48 0.11

AIRS Parallel AIRS Clonal

86.14 89.45 82.30

0.80 0.85 0.75

AIRS Parallel AIRS Clon ALG

83.22 83.89 86.27

0.44 0.53 0

Breast Cancer 0.02 0.16 0.03 0.19 0.03 0.17 Echocardiogram 0.19 0.43 0.19 0.43 0.3 0.54 Ecoli 0.04 0.21 0.03 0.18 0.05 0.24 Hepatitis 0.16 0.40 0.16 0.40 0.13 0.37

AIRS Parallel AIRS Clon ALG

69.56 80 52.99

0.38 0.58 0.00

0.30 0.2 0.47

Liver Cancer 0.55 0.44 0.68

Relative Error %

Absolute

Root Relative Square Error

17.8054 8.9027 22.2567

60.0679 42.4744 67.158

32.4

83.966 %

11.2

49.956 %

10.8

48.4778 %

5.92 8.2 6.71

34.41 40.71 36.63

45.49 45.49 71.12

95.52 95.53 119.41

19.15 14.57 24.38

61.98 54.06 69.77

50.56 48.54 43.07

100.90 98.86 102.30

62.44 41.03 96.72

111.76 90.60 140.19