2013 IEEE 6th International Workshop on Computational Intelligence and Applications July 13, 2013, Hiroshima, Japan
Meta-heuristic Algorithms Applied to the Optimization of Type-1 and Type 2 TSK Fuzzy Logic Systems for Sea Water Level Prediction Nguyen Cong Long and Phayung Meesad Faculty of Information Technology King Mongkut’s University of Technology North Bangkok Bangkok, Thailand
[email protected],
[email protected] In the recent years, applications of Artificial Neural Network (ANN), Adaptive Neuron Fuzzy Inference System (ANFIS), Fuzzy Logic System (FLS) and Genetic Programming (GP) approaches in forecasting sea water level have become viable. Mohammad et al. [8] applied ANN and GP techniques for sea level variations prediction and found both methods perform satisfactorily. Makarynskyy et al. [9] used ANN for sea level prediction in Hillarys Harbor, Australia. The results showed feasibility of the ANN for sea water level prediction in term of root mean square errors (about 10% of tidal range). Bunchingiv et al. [10] predicted short term water level using ANN and neuron-fuzzy system and found that both ANN and neuron-fuzzy system have performed accurately for the purpose. Shiri et al. [11] forecasted short term operational sea water level using ANN and ANFIS and found that ANFIS outperforms ANN.
Abstract— This paper describes an approach using Firefly Algorithm, Particle Swarm Optimization and Genetic Algorithm to optimize the parameters of Takagi-Sugeno-Kang (TSK) fuzzy logic system (both type-1 and type-2) in order to find the optimal fuzzy logic system for sea water level prediction. The obtained results of the simulations performed are compared among these optimization algorithms in order to find which one is the best optimization algorithm for sea water level prediction. Keywords— Firefly Algorithm; Particle Swarm Optimization; Genetic Algorithm; Type-1 TSK Fuzzy Logic System; Interval Type2 TSK Fuzzy Logic System, Sea Water Level Prediction
I.
INTRODUCTION
Type-2 fuzzy set (T2 FSs) were first proposed by Zadeh in 1975 [1] as an extension of an ordinary fuzzy sets. T2 FSs provide additional free design parameters of fuzzy logic systems, which can be beneficial when lots of uncertainties are presented in the systems [2]. Because of the computational complexity of a general T2 FLS, most researchers are interested in interval type-2 fuzzy logic system (IT2 FLS), for their simplicity and reduced computational cost, which makes an IT2 FLS quite practical [2,3]. T2 FLSs have been successful in many application areas such as modeling and control, classification, prediction, etc. [4,5,6].
The main goal of this paper is to describe the approach of Firefly Algorithm (FA), Particle Swarm Optimization (PSO), and Genetic Algorithm (GA) as optimization methods for the optimal parameters the fuzzy logic system in order to find the best possible fuzzy logic system for sea water level prediction. In addition, the comparison results from the optimization of the fuzzy logic system using these algorithms with ANFIS are also investigated. II.
Generally, TSK fuzzy logic system design can be divided into three parts: determination of the number of rules, determination of rules structure, and tuning of rules antecedents and consequents parameters. The complexity in developing a fuzzy logic system can be found at the time of deciding which are the best number of rules and the best parameters of membership functions in each rule.
An interval type-2 TSK fuzzy logic system (IT2 TSK FLS) having p inputs , ,.., and one output can be described by fuzzy IF-THEN rules that present input-output relations of a system consisting of M rules, each having p antecedents, the th rule can be expressed in (1) [3] :
Predictions of hourly sea water level variations in the nearshore environments are very important to water resource engineering, protection of coast, low-lying region residents as well as alternative energy technologies based on both sea level variations and wave energies [7].
… (1)
where are interval type-2 antecedent fuzzy sets (A2), are consequent type-1 fuzzy set (C1), , , denotes the center (mean) of , and denotes the spread of ; , output of the th rule, is also interval T1 FS,
Sea water level variations are complex, determined by many environmental forces, such as lunar and solar gravitational attraction, ocean tides, atmospheric pressure and wind force, temperature, and salinity of sea water [8].
978-1-4673-5726-5/13/$31.00 ©2013 IEEE
INTERVAL TYPE-2 TSK FUZZY LOGIC SYSTEM
69
1… , 1… TSK FLS [3].
. This fuzzy logic system is called A2-C1 IT2
This section briefly explains the methods of firefly algorithm, particle swarm optimization and genetic algorithm.
For an input vector , ,…, , typical computations in an A2-C1 IT2 TSK FLS consist of the following steps: 1) Compute the membership interval of ,
,
on each
1. . . ,
1…
,
A. Firefly Algorithm The Firefly algorithm, is a population based meta-heuristic algorithm to find the global optimal in problem space, inspired by the flashing behavior of fireflies, and first proposed by Yang in 2007 [12]. In this algorithm, each firefly represents a potential solution and the attractiveness of a firefly is determined by its light intensity which is proportional to its objective function value. The main idea of FA is that the firefly with lower light intensity is attracted to the firefly with higher light intensity. FA begins with randomly generated positions of fireflies (population). In each generation, fireflies with lower light intensity move toward fireflies with higher light intensity by the following equation:
,
(2)
2) Compute the firing interval of the th rule, consequent of th rule [3]
and
, where …
(3)
…
(4)
POPULATION BASED META-HEURISTIC ALGORITHM
III.
1
(11) (12)
In this paper, the use of only the product t-norm is applied in (3) and (4). However, the minimum t-norm may also be used. The consequent of rule , , is also an interval set, that is: ,
and are position of firefly with lower light where intensity and firefly with higher intensity at time t respectively, is a random parameter which determines random behavior of movement, is a random number generator uniformly distributed in [0,1], is a light absorption is the attractiveness at r = 0, and is coefficient, Euclidean distance between any two fireflies and at and , respectively.
, where
∑
∑
∑
∑
(5) .
(6)
After movements, all fireflies move toward the best firefly and their light intensity improves. In the case that no firefly with higher light intensity is recognized, it will move randomly in multi-dimension search space. After stopping criteria met, the best solution is considered by the firefly with the highest light intensity.
3) Perform type-reduction to combine firing interval and corresponding rule consequents. There are many such methods. In this paper, the center-of-sets type reducer is applied [3]. ,
, ,
…
(7)
,
,
…
,
1
B. Particle Swarm Optimization Particle Swarm Optimization, is a population based metaheuristic optimization technique inspired by social behavior of birth of flocking or fish schooling, was first proposed by Kennedy and Eberhart in 1995 [13]. It is initialized with a population of random solutions and search for optimum by updating generations. In PSO, each particle, represents a possible solution, flies through multi-dimension search space following the current optimum particles, and keeps a memory of its previous best solution, called personal best (pbest). In the global version, particle swarm optimizer keeps track of the overall best value and its location, obtaining so far by any particle in the population, which called global best (gbest).
∑ ∑
where ∑
∑
∑ ∑
∑
∑
∑
∑
∑ ∑
∑
∑
(8) (9)
Hence, , is an interval type-1 set. To compute , , it is only necessary to compute its two end-points , and . and can also be computed more efficient by the KM algorithm. In (8) and (9), L and R are two possible switch points [3].
In every step, each particle updated its velocity and position towards its pbest and gbest according to (13) and (14) as follows [13,14]
4) Compute the defuzzified output [3] as: 1 ,
.
,
(10)
70
(13)
1
1 .
Suppose that there are M ruules, each rule has p antecedents, the th rule of the fuzzy logiic system is represented in the equation (1). The number of free design parameters for IT2 2 3 . Hence, there are TSK FLS are 5 5 2 dimensions in the search space (in the case of FA and PSO) and 5 2 genes (in the case of GA). Figure 2 illustrates the encodingg method of IT2 TSK FLS.
(14)
where and are velocity and position of at time particle in the dimension ( 1,2. . , respectively; and are possible accelerattion constants used to scale the contribution of the cognnitive and social components respectively; and and are random values in the range of [0, 1]. C. Genetic Algorithm The genetic algorithm, is a population baased meta-heuristic optimization technique based on the mecchanics of natural selection and natural genetics, first proposedd by John Holland in 1975 [15]. GA has been successfully appplied in variety of problems such as control and optimizatioon of fuzzy logic systems [4,5]. To solve a problem, the basic idea of a GA is to maintain a population of chromosomes, which represent potential solutions and applies several geneticc operators such as selection, crossover and mutation to findd the approximate optimal solution. IV.
Fig. 2. Encoding of an IT2 TSK FLS
b of parameters of In this method, the boundaries antecedents and consequents are determined using training data sets. The boundaries of paarameters are shown in Table 1. TABLE I. Boundary of parameters off interval type-2 Gaussian membership function ( , , is the minim mum, maximum, and standard deviation of training dataa, respectively).
DESIGN OPTIMIZATION OF FUZZY LOGIC SYSTEM
Parameters
The main issues in the optimization algorithms when approaching to design an optimization off the fuzzy logic system are representation of fuzzy logic (encoding) in the corresponding optimization paradigm and deetermining solution space (boundary of parameters to be optimizzed) as well as the fitness function.
Maximal value
, 0.001 ,
B. Fitness Evaluation In any model, it is essenntial to use some performance measurements criteria for evaluuation of each applied model as well as comparison to differennt applied models. In this paper, two statistic indices were used.
A. Encoding Method When applying these optimization algorrithms, the feature parameters of the fuzzy logic systems have to t be encoded into a form of firefly (in the case of FA), particcle (in the case of PSO) and chromosome (in the case of GA A). In this method, membership functions of each input variable are represent by Gaussian. In the encoding schemes for a tyype-1 TSK FLS, a type-1 membership function is represented as a mean and a standard deviation in a Gaussian. In the IT2-T TSK FLS case, the membership functions of each input variable are represented by Gaussian with uncertain mean and standdard deviation [3] shown in (15). ,
Minim mal value
•
The root mean squaree error (RMSE), defined in (16), is used as the fitness function to measure the quality of fuzzy logic moddel. Lower values of RMSE correspond to better performance of the system. RMSE
•
(15)
∑
(16)
The scatter index (SI) is defined by the equation (17). The perfect value of SI is zero representing the best fit of the model outputt to the observed value. RMSE
(17)
where, : Target output as giiven in data set. : Output of fuzzy system. : Number of data pooints for evaluation of the model. : The mean of obserrvational value. Fig. 1. Gaussian membership function with uncertain mean and standard deviation
C. Firefly Algorithm Approachh for Fuzzy Logic System The process for optimization of the fuzzy logic system using the firefly algorithm is reepresented in Fig. 3.
S is similar to the The encoding method for T1 TSK FLS method for IT2 TSK FLS. Therefore, onnly the encoding method for IT2 TSK FLS is represented.
The initial fireflies are crreated randomly respecting the range of the parameters of the fuzzy f logic system in Table 1 (in
71
the case of IT2 TSK FLS). The firefly algorithm a fits the parameters mentioned above according to Table T 1 in order to find the best IT2-TSK FLS using (16).
E. Genetic Algorithm Approacch for Fuzzy Logic System The process for optimization of the fuzzy logic system using generic algorithm is repreesented in Fig. 5. Each chromosome is initiallized respecting the range of the parameters of the fuzzy logic system in Table 1. The genetic algorithm finds the optimal values v of the parameters of the fuzzy model using (16) in orderr to get the best fuzzy model.
Fig. 3. FA process for optimization of fuzzy logic systeem
D. Particle Swarm Optimiazation Aprroach for f Fuzzy Logic System The process for optimization of the fuuzzy logic system using particle swarm optimization is represennted in Fig. 4. The initial particles respecting the range of the parameters of the fuzzy logic system in Table 1 is creatted randomly. The objective of the PSO is to find the best param meters of the fuzzy logic system using the fitness function inn (16) in order to obtain the desirable fuzzy logic system.
Fig. 5. GA process for optimization off fuzzy logic system
V.
EXPERIMENT RESULTSS AND PERFORM EVALUATION
A. Data Used The presented study is baseed on the record data of the time series of hourly sea water level l collected by Nha Trang Oceanography from Nha Tranng Sea in Vietnam, at latitude 12.205709 and longitude 109.2222318 from 1st January 2006 to 31st December 2007. This dataa was used to predict hourly sea water level variations. The reccoded data from the 1st January 2006 to 31st March 2007 was used u to train the applied methods while data from the 1st April 2007 to 31st December 2007 was s of the hourly sea water used to test the methods. The statistics level during the training and teesting period are shown in Table 2. TABLE II. Statistical parameters off used data sets , , , , ) denote the minimum, maximum, mean, m standard deviation, coefficient of variation and skewness coefficient, respectively).
Fig. 4. PSO process for optimization of fuzzy logic sysstem
72
Training Period
82
227
138.33
36.37
0.26
-0.06
Testing Period
82
191
129.59
28.21
0.22
-0.04
B. Results and Discussion The approaches of optimization algorithms for the fuzzy logic system were implemented in Matlab on a laptop with Intel Core 2 Due CPU 2 GHz, and 4 GB of RAM.
The overall evaluation of three optimization algorithms applied to optimal T1 TSK FLS and IT2 TSK FLS reveals that FA outperforms the other algorithms as well as ANFIS and IT2 TSK FLSs have lower RMSE and SI than T1 TSK FLSs. Figures 6a- 6g displays the target and simulated sea level values produced by the optimal fuzzy logic system using FA, PSO and GA as well as ANFIS during testing periods. The high accuracy of FA applied to IT2 TSK FLS can be clearly seen from the Figure below.
To obtain suitable parameters for algorithms, a large number of experiments for each model were simulated with different parameters settings. The final parameters that were obtained through experimentations are: number of generations and population for both algorithms equated 100 and 50 respectively; for FA, = 0.2, = 1, and = 1; for PSO, 1.4945; for GA, probability of crossover and mutation equated 0.9 and 0.1, respectively.
200
Table 3 shows the comparison of RMSE, SI and running time using FA, PSO and GA for training T1 TSK FLS and IT2 TSK FLS as well as ANFIS. It clearly illustrates that, the FA applied for training fuzzy logic systems outperforms the other algorithms, while PSO also outperforms GA. Furthermore, the optimization algorithms applied for training IT2 TSK FLSs are more accurate but slower than traditional T1 TSK FLSs. ANFIS is similar to PSO and GA for IT2 TSK FLS in terms of RMSE and SI but it is very fast.
Sea water level(cm)
160
140
120
100
80
60
TABLE III. Training results of different algorithms for T1 TSK FLS and IT2 TSK FLSs RMSE (cm)
SI
Running time
FA for T1-TSK FLS
8.54
0.062
02:40:08
PSO for T1-TSK FLS
10.23
0.074
02.55:38
GA for T1-TSK FLS
10.15
0.073
03:08:32
FA for IT2-TSK FLS
6.92
0.050
18:24:55
PSO for IT2-TSK FLS
7.25
0.052
19:03:30
GA for IT2-TSK FLS
7.76
0.056
20:12:23
ANFIS
7.64
0.055
0:0:04
FA for T1-TSK FLS
9.62
0.074
PSO for T1-TSK FLS
9.65
0.074
GA for T1-TSK FLS
10.48
0.080
FA for IT2-TSK FLS
7.54
0.058
PSO for IT2-TSK FLS
7.93
0.061
GA for IT2-TSK FLS
8.02
0.062
ANFIS
8.05
0.062
60
80
100 Time(h)
120
140
160
180
200
(a)
160
140
120
100
80
0
20
40
60
80
100 Time(h)
120
140
160
180
200
(b)
200 GA for IT2 TSK FLS Target 180
Sea water level(cm)
SI
40
180
TABLE IV. Testing results of different models for the 24h sea water levels prediction. RMSE(cm)
20
PSO for IT2 TSK FLS Target
After training of the models, they were then used to predict the 24h sea water level. The obtained results from different methods are shown in Table 4. From the table, it is clear that, FA applied to IT2 TSK FLS outperforms the other models with an IS of 0.058 and a RMSE of 7.54 (cm). Whereas, ANFIS has similar accuracy to other algorithms for IT2 FLS in terms of RMSE and SI.
Models
0
200
Sea water level(cm)
Models
FA for IT2 TSK FLS Target
180
160
140
120
100
80
73
0
20
40
60
80
100 120 Time(h)
140
160
180
200
(c)
VI. 180
This paper describes an FA, PSO and GA approach to optimization of type-1 and type-2 TSK fuzzy logic systems to forecast 24h sea water level. The hourly sea water level observations from Nha Trang Sea in Vietnam were used to train and test each model. The obtained results illustrate that FA approaches for IT2 FLS outperforms PSO, GA and ANFIS in term of RMSE and SI, but PSO, GA and ANFIS give similar RMSE and SI. Whereas IT2 TSK FSLs have higher accuracy than T1 TSK FLSs.
FA for T1 TSK FLS Target
170 160 150 Sea water level(cm)
CONCLUSION
140 130 120 110 100 90 80
REFERENCES 0
20
40
60
80
100 Time(h)
120
140
160
180
200
(d)
[1]
200 PSO for T1 TSK FLS Target
[2]
Sea water level(cm)
180
160
[3]
140
[4]
120
100
[5] 80
0
20
40
60
80
100 Time(h)
120
140
160
180
200
(e) [6]
180 GA for T1 TSK FLS Target
170 160
Sea water level(cm)
150
[7]
140 130
[8]
120 110 100
[9]
90 80
0
20
40
60
80
100 Time(h)
120
140
160
180
200
(f) [10]
200 ANFIS Target 180
[11] Sea water level
160
140
[12] 120
[13] 100
80
0
20
40
60
80
100 Time(h)
120
140
160
180
[14]
200
(g)
Fig. 6. Results of different optimization algorithms applied to the T1 TSK FLS, IT2 TSK FLS and ANFIS for the period between 1th and 200th 24h sea water levels prediction.
[15]
74
L. A. Zadeh, “The Concept of a Linguistic Variable and its Application to Approximate Reasoning,” Information Sciences, Vol. 8, pp, 199-249, 1975. Jerry M. Mendel, Robert I. John, and Feilong Liu, “Interval Type-2 Fuzzy Logic Systems Made Simple”, IEEE Transactions on Fuzzy Systems, Vol.14.No.6, December 2006. Jerry M. Mendel, Uncertain Rule-Base Fuzzy Logic Systems: Introduction and New directions. Upper Saddle River, NJ: Prentice-Hall, 2001. Yazmin Maldonado, Oscar Castillo and Patrica Melin, “Particle Swarm Optimization of Interval Type-2 Fuzzy System for FPGA application,” Application Soft Computing Val.13, pp. 496-508, 2013. Denisse Hidalgo, Patricia Melin and Oscar Castillo, “An optimization method for designing T2 FLS based on the footprint of uncertainty using genetic algorithms,” Expert Systems with Applications, Vol. 39, pp. 4590-4598, 2012. Abbas Khosravi, Saeid Nahavandi, Doug Creighton and Dipti Srinivasan, “Interval Type-2 Fuzzy Logic Systems for Load Forecasting: A Comparative Study,” IEEE Transaction on Power Systems, Vol. 27, No. 3, pp. 1274 -1282, 2012. Herbich, J.B. Handbook of coastal and ocean engineering. Gulf Professional Publishing, 1992. Mohammad Ali Ghorbani, Rahman Khatibi, Ali Aytek, Oleg Makarynskyy and Jalal Shiri, “Sea water level forecasting using genetic programming and comparing the performance with Artificial Neural Networks,” Computers & Geosciences Vol. 36, 620–627, 2010. O. Makarynskyy, D. Makarynska, M. Kuhn, and W.E. Featherstone, “Predicting sea level variations with artificial neural networks at Hillarys Boat Harbour, Western Australia,” Estuarine, Coastal and Shelf Science Vol. 61, pp.351–360, 2004. Bunchingiv Bazartseren, Gerald Hildebrandt, and K.P. Holz, “Shortterm water level prediction using neural networks andneuro-fuzzy approach,” Neurocomputing Vol. 55, pp. 439–450, 2003. J. Shiri, O.Makarynskyy, O. Kisi, W. Dierickx, and A.F. Fard, “prediction of short-term operational sea water level using an adaptive neuro-fuzzy inference system,” ASCE Journal of Waterway, Port, Coastal and Ocean Engineering Vol.137, 344-355, 2011. Yang-She Yang, “Firefly algorithms for Multimodel Optimization”, Foundations and Applications, SAGA 2009. Y. Shi and R. Eberhart, “A Modified Particle Swarm Optimizer: Evolutionary Computation Proceedings,” IEEE W cong on comp int, pp. 69-73, 1998. J. Kennedy and R. Eberhart, “Particle Swarm Optimization”, In Proceedings of IEEE Conference on Neural Networks, pp.1942-1948, Perth, WA, Australia, 1995. Melanie Mitchell, An Introduction to Genetic Algorithm, Cambridge, Massachusetts, London, England, fifth printing, 1999.
