FCM theory was introduced by Kosko [1, 2] based on Axelord's work on ... Compared with traditional expert systems and neural networks, FCM has several.
Implementation of Fuzzy Cognitive Maps based on Fuzzy Neural Network and Application in Numerical Prediction of Time Series Song Hengjie , Miao Chunyan , Shen Zhiqi 2.
1. School of Computing Engineering, Nanyang Tech University, Singapore School of Electrical and Electronic Engineering, Nanyang Tech University, Singapore
Abstract - Fuzzy Cognitive Map (FCM) has gradually emerged as a powerful paradigm for knowledge representation and a simulation mechanism applicable to numerous research and application fields. However since efficient methods to determine the states of investigated system and to quantify causalities which are the very foundation of FCM theory are lacking, constructing FCMs for complex causal systems greatly depends on expert knowledge. The manually developed models have a substantial shortcoming due to the model subjectivity and difficulties with accessing its reliability. In this paper, we proposed a fuzzy neural network to enhance the learning ability of FCMs. Our approach incorporates the inference mechanism of conventional FCMs with the determination of membership functions, as well as the quantification of causalities. In this manner, FCM models of investigated systems can be automatically constructed from data, and therefore are independent of the experts. In the employed fuzzy neural network, the concept of mutual subsethood is used to describe the causalities, which provides more transparent interpretation for causalities in FCMs. To demonstrate the capability of our approach, simulations for many benchmarks in numerical predication of time series problem are conducted. The simulation studies confirm the effectiveness of the proposed approach. Index Terms - Fuzzy neural network, Fuzzy Cognitive Maps (FCMs), mutual subsethood, fuzzy rule identification
1
Introduction FCM theory was introduced by Kosko [1, 2] based on Axelord’s work on Cognitive Maps
[3]. FCM models the causal system as a collection of concepts and causal relations among concepts. As an inference network, FCM is a signed directed graph with feedback, which
consists of a collection of nodes and directed weighted arcs interconnecting nodes. By describing the behavior of a collection of concepts, FCM provides a more flexible and natural mechanism for knowledge representation and reasoning which are essential to intelligent systems [4]. Compared with traditional expert systems and neural networks, FCM has several desirable properties [5, 6], such as abstraction, flexibility, adaptability and fuzzy reasoning. Therefore, FCM has been extensively studied and applied in various application fields, for instance, decision making [7], systems control [8], geographic information [9], and other areas [10, 11 and 12]. Over the last ten years, several works on FCMs have been proposed in order to investigate the extensions of FCMs. Recently, Y. Miao and Liu et al. [4, 6, and 13] have made some extensive investigations on the inference properties of FCMs. Moreover, [4] proposed dynamic causal networks (DCN) to introduce a mechanism that can quantify the description of concepts with required precision as well as the strength of causality between concepts. In addition, Papageorgiou [7] proposed an integrated two-level hierarchical system based fuzzy cognitive map and applied it in decision making in radiation therapy. In additionally, Stach and Lukasz employed FCMs to predict time series in [14]. Besides these applied researches, [15] and [16] proposed decomposition theory for FCMs, which provides an effective framework for calculating and simplifying causal patterns in complicated real-world application. Although many research works have been done on FCMs, very few researches have investigated how to objectively determine the states of investigated systems and how to quantify the causalities involved in FCMs. Actually ever since FCM was proposed in [1,2], two significant drawbacks have existed in FCMs and the related research works. First, for fuzzy inference mechanism, the membership functions are crucial to the fuzzification and defuzzification processes. However as a kind of inference models based on
fuzzy theory, FCMs lack effective methods to provide membership functions which are absolutely necessary to determinate system states in FCMs. Therefore, the inference results generated by FCMs are always limited to the fuzzy domain, which makes it difficult to directly compare the inference results against real data. Consequently in most existing literatures about FCMs, the interpretation of inference result greatly depends on expert knowledge. The subjective interpretation is not more or less convincing. Therefore, how to improve FCMs so as to automatically identify membership functions for specific systems under investigation has become a very urgent and realistic issue. In original FCMs proposed by Kosko, the weights interconnecting the concepts that are extracted from the investigated systems quantify the causalities among concepts. Taking the inference mechanism of original FCMs [1, 2] into account, accurate quantification of causalities is the significant guarantee for generating reliable and accurate inference results. Therefore, the quantified causalities are very important for inference processes and inference results generated by FCMs. However in many research works on FCMs, the causalities are still quantified based on expert knowledge. In most cases, it is difficult for human experts to accurately pre-specify the causalities involved in a complex causal system. In order to cope with these difficulties, the first attempt to learn FCMs using Hebbian law was proposded by Dickerson and Kosko, and was referred to the Differential Hebbian Learnin [28]. This method was further extended into Nonlinear Hebbian Learing (NHL) [29]. Based on the previous research result, Stach proposed a novel extension to NHL method, called datadriven NHL [30]. Furthermore, Carvalho [32] proposed fuzzy boolean nets to qualitatively optimize the causalities in rule based FCM (RB-FCM) [23, 31]. Additionally, Koulouriotis [17, 18] and Song [19, 20] proposed learning algorithms for FCMs that base on evolution strategies and particle swarm optimization respectively.
However in these proposed approaches, the causalities needed to be quantified are not exactly consistent with the definition of causality in conventional FCMs. Therefore, the learned causalities lack clear mathematical interpretations. More recently, Stach [14] presented an application framework which employed FCMs to implement numerical and linguistic prediction of time series. However, the approach proposed in [14] introduces many procedures for pre-processing the raw dataset, which makes the prediction more complex. Additionally, in all documents about FCMs [17~20], the involved membership functions are still pre-specified based on the analysis of dataset [4, 8, 14] or expert knowledge. In this paper, we propose a novel fuzzy neural network which equips the inference mechanism of original FCMs proposed by Kosko [1] with the automatic determination of membership functions, as well as quantification of causalities. Our approach is able to identify the membership functions and causalities from real data, which makes the construction of FCM for complex systems independent of expert knowledge. In addition, our approach makes it possible to compare the inference results against real data directly, so it extends the application of FCMs to real problems, such as numerical prediction of time series. Furthermore, the employed fuzzy neural network makes use of mutual subsethood to describe the causalities in investigated systems. It provides clearer mathematical interpretation on causalities and makes the inference process easier to understand. The rest of this paper is organized as follows. Section 2 describes the representation of FCMs, and stresses the motivations of our works. Section 3 proposes the novel fuzzy neural network and discusses its corresponding operations. Section 4 gives the detailed description of learning algorithm which is employed to tune the related parameters in the proposed fuzzy neural network. In Section 5, the performance of our approach is verified by testing the model in two different applications in numerical prediction of time series. The comparisons
of the experiment results of our approach with other models are also given in Section 5. Finally, Section 6 presents the conclusions.
2
Fuzzy Cognitive Maps Intuitively, FCM is a signed directed graph which represents a causal system with
uncertain and incomplete causal information. The human experience and knowledge on the complex systems is embedded in the structure of FCMs and the corresponding inference process. 2.1
Origin of FCMs
FCM is developed from Cognitive Maps (CMs) [3]. The research about CMs sprung from findings in physic-psychological experiments tried to trace and interpret the functionalities of various mental and cognitive tasks, abilities and phenomena in animals and humans. Tolman [21] and Axelrod [3] described them in the formal and systemic manner. In detail, a CM represents a causal system which consists of a set of concepts and causal-effect relationships (causalities) among these concepts. By describing the causalities and state information of concepts, CM reveals impacts produced by changes in all elements on the entire system. As
far as CM structure is concerned, links between nodes may obtain only binary values, 1 and 1. The nodes may take the value 1, 0 and 1. Practically, node value 1 represents
an increase of the concept state and node value 1 means a decrease of the corresponding
concept state.
In order to overcome the drawbacks of the binary logic that CMs enclose, Kosko [1, 2] extended CMs to FCMs by introducing fuzzy values to quantify the concept states and the strength of causalities.
2.2
Basic Definitions and Structures
As an extension to CMs, FCMs were developed in order to overcome the drawbacks of the binary logic that plague CMs. By introducing fuzzy values to describe the concept states and the strength of causalities, FCMs combine CM theories and fuzzy logic principles and therefore provide a more realistic and accurate representations of the real causal systems than CMs do. A FCM is a signed directed graph with feedbacks as shown in Fig.1, which consists of a collection of nodes and directed weighted arcs interconnecting the nodes. In FCMs, nodes represent the concepts which are abstracted from real causal systems. Directed weighted arcs
denote the causal relationships among concepts. In FCMs, concepts 1,2, … , represent a set of research objects with semantic meaning that form the investigated causal
system, where denotes the number of concepts in a given FCM. The active degree of
concept is described by state value 0,1 which changes over time in inference process. Therefore, concept states describe the behavioral characteristics of the system. The
causal relationship between concept and in a FCM is represented by the directed arc pointing from to . In general, the weight associated with the arc connecting
with , takes the value in the interval 1,1, describing the type and magnitude of the
corresponding causality. The weights with high absolute value signify strong cause-effect relationships between the concepts. In practice,
--1 0, which indicates the increase (decrease) in the value of leads to increase
(decrease) of value of .
-- 0 1, which indicates the increase (decrease) in the value of leads to
decrease (increase) of value of .
-- 0, which indicates no causal relationship between and .
Fig. 1.
An example of FCMs
As mentioned previously, our approach is only proposed to improve original FCMs proposed by Kosko [1]. Therefore, a concept cannot cause itself and there is no causal relationship between a concept and itself, so for all the concepts of a FCM there will be
0. For the sake of simplicity, the weights in a FCM can be represented by a matrix. 0 ,
2.3
Inference Mechanism of FCMs
, 0
!
The detailed inference mechanism of the original FCMs was introduced in many publications [1, 10]. Briefly, the process is to keep updating the state values of concepts in
discrete time manner based on a given weight matrix and initial state information of a particular system. The inference mechanism of FCMs is described by following formula, # $ $! * " $ 1 %&∑ ( $ ! )
, + 1,2, … , ; $ 0,1,2, … , -
where $ is the iteration step; $ indicates state value of concept at iteration $ . Correspondingly, # $ indicates the system state at iteration $ . % is threshold
(transformation) function.
The inference process of a FCM can be regarded as an iterative process that applies the scalar product and threshold function to generate the discrete time series of system state until the following requirements on convergence are satisfied, --A fixed point equilibrium is reached. In this case,
# $. 1 # $. $. -, where # $. is the final state.
--A limited cycle is reached. In this case
# $. ∆- # $. $. - , where # $. is the final state. This case means that the
system falls in a loop of a specific period, and after a certain number of inference steps ∆-, it
reaches the same state # $. .
--Chaotic behavior is exhibited [22, 23]. 3
Structure of the Approach
From the above introduction, we can see that FCMs provide a more flexible and natural mechanism for knowledge representation and reasoning mechanism which are essential to intelligent systems [4]. Compared with the traditional expert systems and neural networks, FCMs have several desirable properties [24, 6], such as abstraction, flexibility, adaptability and fuzzy reasoning. In fact, a vast majority of FCM models were constructed manually, i.e., they are based on domain experts’ knowledge. The manually developed models have a substantial shortcoming due to the model subjectivity and difficulties in accessing its reliability. In order to handle with this problem, we proposed a four-layer fuzzy neural network based on the definition and description of the conventional FCMs. Our approach extends the inference process of FCMs with the identification of membership functions, as well as the quantification of causalities together. In this section, we introduce the basis structure of the proposed fuzzy neural network and discuss the operation in each layer.
The basic structure of the proposed four-layer fuzzy neural network is shown in Figure.2.1. In the fuzzy neural network, the inputs and outputs are represented by non-fuzzy vector
0 1 2 , 2 , … , 2 , … , 2 and 3 1 4 , 4 , … , 4 , … , 4 respectively, where denote the
numbers of the input and output variables. Taking the definition of FCMs into account, the
proposed fuzzy neural network always has the same numbers of input and output variables that both correspond to the concepts involved in a FCM model. Additionally, input variable
2 is characterized by a set of linguistic term 56 8 9 1, … , , where 56 8 is expressed as 7
7
a semantic symbol, such as ‘Small (S)’, ‘Medium (M)’ or ‘Large (L)’ etc. Each 567 8 is described by a fuzzy subset in the universe of discourse on 2 . Similarly, output variable 4
is also partitioned into the same set of fuzzy subsets :67 8 . We emphasize that input variable
2 and output variable 4 refer to the same concept involved in the investigated system; 56 8 7
and :67 8 represent the same linguistic term for input variable 2 and output variable 4
respectively. According to above introduction, the structure of FCM corresponding to the proposed fuzzy neural network is depicted as Fig. 2.2. Obviously in our approach, the linguistic terms of inputs/outputs are introduced into traditional FCMs, which make it possible to describe the states of the investigated systems more accurately. In the proposed fuzzy neural network (Fig.2.1), the weights connecting the neurons in layer 2 with the neurons in layer 3 quantify the causalities among linguistic terms of different concepts (Fig.2.2). In remaining parts of this section, we will describe the particular operations involved in the proposed fuzzy neural network.
3.1
Layer 1
This layer consists of input variables. Each input node 2 represents a concept in the
investigated system. Nodes in layer 1 directly transmit input values 0 1 2 , 2 , … , 2 , … , 2
to next layer. Thus, the net input % and net output 2 of the ith node are given in Eq. (1)
(in following discussion, the number between parentheses in super script represents the level number of proposed neural network). %
2
th
2
%
(1)
where xi the i element of input vector XT. ( xi = yi )
( x1 = y1 )
IL11 (OL11 )
IL1n1 (OL1n1 )
IL1i (OL1i )
ILnii (OLnii )
ILNii (OLNii )
IL1N1 (OL1N1 )
IL1N (OL1N )
ILnNN (OLnNN )
ILNN N (OLNN N )
(xN = yN )
Fig. 2.1 Basic Structure of the Proposed Fuzzy Neural Network 3.2
Fig. 2.2 Corresponding Fuzzy Cognitive Map
Layer 2
This layer realizes the fuzzification process. Nodes in layer 2 represent the linguistic term
set of inputs. The 9 th linguistic term of input variable 2 is denoted as 567 8 9 1, … , that is expressed as a semantic symbol, such as ‘Small (S)’, ‘Medium (M)’ or ‘Large (L)’ etc.
Each 567 8 represents a fuzzy subset in the universe of discourse on 2 . In the proposed fuzzy
neural network, The fuzzy set 567 8 is modeled by a symmetric Gaussian membership function with center ;;
