Application of Type-1 Fuzzy Functions Approach for Time Series ...

TJFS: Turkish Journal of Fuzzy Systems (eISSN: 1309–1190) An Official Journal of Turkish Fuzzy Systems Association Vol.5, No.1, pp. 01-09, 2014.

Application of Type-1 Fuzzy Functions Approach for Time Series Forecasting Cagdas Hakan Aladag * Hacettepe University, Faculty of Science, Department of Statistics, 06800 Ankara, Turkey E-mail: [email protected] *Corresponding author I. Burhan Turksen TOBB Economics and Technology University, Department of Industrial Engineering, 06560 Ankara, Turkey E-mail: [email protected] Ali Zafer Dalar Ondokuz Mayis University, Faculty of Art and Science, Department of Statistic 55139 Samsun, Turkey E-mail: [email protected] Erol Egrioglu Marmara University, Faculty of Art and Science, Department of Statistic Istanbul, Turkey E-mail: [email protected] Ufuk Yolcu Ankara University, Faculty of Science, Department of Statistics, 06100 Ankara, Turkey E-mail: [email protected] Abstract Fuzzy inference systems have been used for prediction problems in the literature. Classical fuzzy inference systems are rule-based systems. The determination of the rules is important and difficult problem. Fuzzy functions were proposed as a good alternative for fuzzy inference systems. Fuzzy functions are not rule-based. This is a big advantage for them. Fuzzy functions were applied to obtain forecasting by using simultaneous variables of other time series as covariates. In this study, type-1 fuzzy functions approach has been applied to obtain forecasts of Australian beer consumption time series. The lagged variables of elementary time series have been used as covariates. The performance of type-1 fuzzy functions approach has been compared with some recent methods in the literature. Keywords: Type-1 fuzzy functions approach, forecasting, fuzzy C-means, fuzzy time series, artificial neural networks.

1

1. Introduction In recent years, alternative methods have been used for time series forecast as well as probabilistic models. Alternative methods generally based on fuzzy set theory or artificial neural networks approach. Artificial neural networks when applied to the time series do not contain any approximations to uncertainty. Fuzzy set theory based methods include fuzzy approximation to uncertainty. There are many real world applications which require fuzzy set theory as much as situations that require probability theory. Fuzzy set theory firstly proposed by Zadeh (1965) and this theory has been used in many areas. Fuzzy set theory based fuzzy inference systems which are similar to the human brain’s inference mechanism, work with linguistic variables. Fuzzy set theory based approaches for time series forecasting can be classified as fuzzy regression methods, fuzzy time series methods, fuzzy inference systems and fuzzy functions. In fuzzy regression approaches, parameters of probabilistic models such as traditional regression or autoregressive models are taken as fuzzy numbers. The aim of these methods is to achieve more accurate interval and point predictions. However, these methods have not often been used in literature because they use linear models and require computations complex mathematical programming problems. Fuzzy time series approaches have wider area than fuzzy regression techniques because it is easier to handle these methods. The techniques as artificial intelligent systems and artificial neural networks have been easily used in fuzzy time series approaches. One type of artificial neural networks is multiplicative neuron model that was presented by Yadav et al. (2007). In recent years, time series approaches based on multiplicative neuron model have been suggested by Zhao and Yang (2009), Wu et al. (2012), Wu et al. (2013), Yolcu et al. (2013b) and Egrioglu et al. (2014). Fuzzy time series approach was firstly introduced by Song and Chissom (1993a, 1993b). In last years, fuzzy time series approaches based on multiplicative neuron model and membership values have been suggested by Yu and Huarng (2010), Egrioglu (2012), Aladag et al. (2012), Aladag, S. et al. (2012), Aladag (2013), Cagcag Yolcu (2013) and Egrioglu et al. (2013). Yolcu (2012) introduce a high-order multivariate fuzzy time series forecasting model. These studies made a significant contribution to the literature. Fuzzy time series approaches which work with membership values are like fuzzy functions approaches. However, these kinds of fuzzy time series methods are first order approaches, and there are some problems with extending these approaches to high order models. In the literature, fuzzy time series methods have been frequently used for time series forecast, recently and these methods do not contain restrictions unlike classical time series methods. In fuzzy inference system, membership values of fuzzy time series approaches are not taken into account, and this is the main problem of the methods. Although there are many studies in recent years which take membership values into consideration, there is not yet sufficient literature in this subject. Many of fuzzy time series approaches are rule-based like classical fuzzy inference systems. Determining of the rules is a significant problem in a fuzzy inference system and also this is an important factor which effects performance of methods. Fuzzy inference systems can be designed from expert knowledge and learnt from data. Fuzzy inference systems have been widely used for prediction problem, but these

2

methods have not been sufficiently used for time series prediction problem. In literature, the most common used fuzzy inference system for time series forecasting problem is adaptive neuro fuzzy inference system (ANFIS) that was proposed by Jang (1993). Fuzzy inference systems are rule-based systems, but this is a disadvantage for them. Therefore, fuzzy functions were proposed. Inference system of fuzzy functions takes membership values into consideration, and there is no need the use of rule base. Fuzzy functions approach proposed by Turksen (2008), instead of fuzzy rule base approaches. Fuzzy functions were proposed for regression and clustering problems based on fuzzy set theory. Afterwards fuzzy functions were enhanced by using different kinds of artificial intelligent systems and fuzzy sets (Celikyilmaz and Turksen, [2008a, 2008b, and 2009], Turksen, 2009). Beyhan and Alci (2010) adapted fuzzy functions to time series forecasting and used an embedded model. In Beyhan and Alci (2010), the model was used as a linear ARX model and lagged variables were determined by trial and error methods. A hybrid fuzzy functions approach was proposed by Zarandi et al. (2013). In Zarandi et al. (2013), lagged variables were not used like in regression analysis. Aladag et al. (2013) proposed a type-1 fuzzy functions approach. In this approach, inputs of the system are lagged variables of time series and these variables are determined by binary particle swarm optimization. In the literature, fuzzy functions approaches were applied to time series forecasting problem by using simultaneus other time series as covariates. However, it is well known that many time series can be explained with its or other time series’ lagged variables and lagged variables should be used to obtain more accurate forecast. In this study, type-1 fuzzy functions approach which uses fuzzy C-means algorithm is implemented to a real world time series by using lagged variables of related time series for time series forecasting. The paper is organized as follows. Section 2 starts with the introduction of type-1 fuzzy functions approach, and presents an algorithm for the approach. Section 3 presents obtained results from experimental study. Finally, conclusions and discussions have been given in the last section. 2. Type-1 fuzzy functions approach Turksen (2008) proposed fuzzy functions approach instead of rule-based fuzzy inference systems. While a relation between input and output is established in rulebased fuzzy inference systems, a function is generated instead of a relation in fuzzy functions approach. There is no need to determine any rules in fuzzy functions approach. This is an important advantage of it. Algorithm for the type-1 fuzzy functions approach (T1FF) using fuzzy C-means (FCM) is given below step by step. Step 1. Inputs are lagged variables of time series. Matrix Z comprises of inputs and output of the system. Inputs and output of the system are clustered using FCM (Bezdek, 1981) clustering method. FCM clustering method can be applied by using the formulas given below.

3

n

vi 

   k 1 n

fi

ik

   k 1

zk

, i  1, 2, ..., c

(1)

fi

ik

2  c  f i 1     d z , v    k i   ik     d z k , v j     j 1  

1

, i  1, 2, ..., c ; k  1, 2, ..., n

(2)

where f is degree of fuzziness, zk is a vector whose elements are the elements compose of kth row of Z, and ik is degree of belongingness of kth observation to ith cluster. Also d(z, v) is Euclidian distance and is computed by using the formula (3).

d  z k , vi   z k  vi

(3)

Step 2. Membership values of the input space are constituted as below. 2  c  f i 1     d x k , vi     ik    d x , v    k j   j 1  

1

, i  1, 2, ..., c ; k  1, 2, ..., n

(4)

where x is input matrix which is generated for lagged variables. If  ik    cut , then

ik value will be taken as zero. Step 3. For each cluster i, membership values of each input data sample, and th original inputs are gathered together, and i fuzzy function is obtained from predicting Y (i )  X (i )  (i )   (i ) multivariate regression model. When the number of the inputs is p, X (i ) and Y (i ) matrices are given below.

X (i )

  i1  i2       in

x11 x12  x1n

 x p1   y1    x p 2  (i )  y 2  ,Y           x pn   yn 

(5)

Celikyilmaz and Turksen (2009) used mathematical transformations of membership values. Their research indicated that the exponential and various logarithmic transformations of membership values can improve the performance of the system models.

4

In this study, for each cluster i by using membership values, ik and/or their transformations such as i21 , expi1  and ln1  i1  i1  are taken. After adding the all transformations of membership values, matrix X (i ) reconstitutes as follow:

X (i )

  i1     i2      in

 i21 exp  i1  ln1   i1   i1  x11  x p1    i22 exp  i 2  ln1   i 2   i 2  x12  x p 2  

 in2





exp  in  ln1   in   in 

     x1n  x pn 

(6)

Step 4. Output values are calculated by using the results obtained from fuzzy functions as follow: c

yˆ i 

 yˆ i 1

ik

 ik

c

 i 1

, k  1, 2, ..., n

(7)

ik

3. Application T1FF approach was implemented to a well-known real world time series in literature. The time series is quarterly Australian beer consumption (Janacek, 2001) between 1956 Q1 and 1994 Q1 whose graph is given in Figure 1. This time series consists of 148 observations in total, and the last 16 observations of the time series were used as test set.

650 600 550 500 450 400 350 300 250 200

Figure 1. The graph of Australian beer consumption data

5

Australian beer consumption data was forecasted by using seasonal autoregressive integrated moving average (SARIMA), Winters’ multiplicative exponential smoothing method (WMES), Yolcu et al. (2013a) linear and nonlinear artificial neural network model (L&NL-ANN), Aladag (2013) multiplicative neuron model based fuzzy time series method (MNM-FTS) and T1FF approach. In this study, we determined the number of lagged variables (m) between 2 and 16, with an increment of 1. The number of fuzzy cluster (cn) experienced between 3 and 10, with an increment of 1. α-cut is taken as 0 and 0.1. All obtained results for Australian beer consumption time series are summarized in Table 1. The best model for SARIMA was obtained as SARIMA(0,1,1)(0,1,1)4. Linear trend component was utilized when WMES was applied to Australian beer consumption data. When the L&NL-ANN method was used, the order of the model was 8. In T1FF approach, the best result is obtained when cn, m and α-cut are taken as 5, 8 and 0, respectively. And besides, the result is obtained when matrix X (i ) is taken like formula (6). Table 1. All obtained results for Australian beer consumption time series Test Data 430.50

SARIMA 452.72

WMES 453.91

L&NL-ANN 449.92

MNM-FTS 437.50

T1FF 446.20

600.00

578.29

575.22

574.28

537.50

580.12

464.50 423.60

487.70 446.28

502.32 444.73

481.47 442.79

437.50 437.50

483.04 442.97

437.00

456.77

459.66

445.12

437.50

444.74

574.00

583.51

582.48

571.97

537.50

579.90

443.00 410.00

492.13 450.36

508.64 450.31

472.76 416.36

487.50 437.50

468.01 418.98

420.00

461.01

465.40

428.63

437.50

431.60

532.00

588.96

589.74

559.89

562.50

559.41

432.00

496.77

514.96

445.75

462.50

444.08

420.00

454.64

455.89

390.25

412.50

394.99

411.00

465.46

471.15

412.38

437.50

409.72

512.00

594.71

597.00

533.19

537.50

525.60

449.00 382.00

501.67 459.17 47.0367 0.0949 0.7333

521.28 461.46 53.3295 0.1072 0.6667

442.13 405.08 18.7888 0.0357 1.000

437.50 412.50 29.1381 0.0532 0.9333

438.91 409.07 17.3926 0.0345 1.000

RMSE MAPE DA

Root mean square error (RMSE), mean absolute percentage error (MAPE) and direction accuracy (DA) performance measures are calculated for each method, and values of these criteria are given in Table 1. According to Table 1, when T1FF approach is employed, the most accurate forecasts are obtained in terms of RMSE, MAPE and DA criteria. The forecasting performance of T1FF approach is also examined visually. The 6

graph of the real observations and the forecasts obtained from the approach for the test set is presented in Figure 2. According to this graph, it is clearly seen that the forecasts obtained from T1FF approach are very accurate.

700 650

Test Data

Forecast

600 550 500 450 400 350 300

Figure 2. The graph of the real observations and the forecasts obtained from the T1FF approach 4. Conclusions and future work The most important feature of fuzzy functions is that fuzzy functions are not needed rules in inference systems. In this study, we used Turksen’s (2008) T1FF approach for more accurate forecasting performance. And besides, in this study, T1FF approach is implemented to time series by using lagged variables. In the implementation, T1FF approach was applied to a real world time series. The lagged variables of elementary time series have been used as covariates. The performance of T1FF approach has been compared with some recent methods such as artificial neural networks and fuzzy time series methods available in the literature. As a result of the comparison, it was shown that T1FF approach produces the best forecasts in terms of RMSE, MAPE and DA performance measures. References Aladag, C.H., Yolcu, U., Egrioglu, E., Dalar, A.Z., A new time invariant fuzzy time series forecasting method based on particle swarm optimization. Applied Soft Computing, 12, 3291-3299, 2012.

7

Aladag, S., Aladag, C.H., Mentes, T., Egrioglu, E., A new seasonal fuzzy time series method based on the multiplicative neuron model and SARIMA. Hacettepe Journal of Mathematics and Statistics, 41 (3), 337-345, 2012. Aladag, C.H., Using multiplicative neuron model to establish fuzzy logic relationships. Expert Systems with Applications, 40 (3), 15, 850-853, 2013. Aladag, C.H., Yolcu, U., Egrioglu, E., Turksen, I.B., Type-1 fuzzy time series function approach based on binary particle swarm optimization. The 3rd International Fuzzy Systems Symposium, FUZZYSS’13, 24-25 October, Istanbul, Turkey, 2013. Beyhan, S., Alci, M., Fuzzy functions based ARX model and new fuzzy basis function models for nonlinear system identification. Applied Soft Computing, 10, 439–444, 2010. Bezdek, J.C., Pattern recognition with fuzzy objective function algorithms, Plenum Press, NewYork, USA, 1981. Cagcag Yolcu, O., A hybrid fuzzy time series approach based on fuzzy clustering and artificial neural network with single multiplicative neuron model. Hindawi Publishing Corporation Mathematical Problems in Engineering, vol. 2013, Article ID 560472, 2013. Celikyilmaz, A., Turksen, I.B., Enhanced fuzzy system models with improved fuzzy clustering algorithm. IEEE Trans. Fuzzy Syst., 16 (3), 779-794, 2008a. Celikyilmaz, A., Turksen, I.B., Uncertainty modeling of improved fuzzy functions with evolutionary systems. IEEE Trans. On system, Man and Cybernetics, 38 (4), 10981110, 2008b. Celikyilmaz, A., Turksen, I.B., Modeling uncertainty with fuzzy logic. Studies in Fuzziness and Soft Computing, 240, Springer, 2009. Egrioglu, E., A new time invariant fuzzy time series forecasting method based on genetic algorithm. Advances in Fuzzy Systems, vol. 2012, Article ID 785709, 2012. Egrioglu, E., Aladag, C.H., Yolcu, U., Fuzzy time series method based on multiplicative neruın model and membership values. American Journal of Intelligent Systems, 3 (1), 33-39, 2013. DOI: 10.5923/j.ajis.20130301.05 Egrioglu, E., Yolcu, U., Aladag, C.H., Bas, E., Recurrent multiplicative neuron model artificial neural network for non-linear time series forecasting. Procedia – Social and Behavioral Sciences, 109, 1094-1100, 2014. Janacek, G., Practical time series, Oxford University Press Inc, New York, 2001. Jang, J.S.R., ANFIS: Adaptive network based fuzzy inference system. IEEE Trans. On system, Man and Cybernetics, 23 (3), 665-685, 1993.

8

Song, Q., Chissom, B.S., Fuzzy time series and its models. Fuzzy Sets and Systems, 54, 269-277, 1993a. Song, Q., Chissom, B.S., Forecasting enrollments with fuzzy time series-Part I. Fuzzy Sets and Systems, 54, 1-10, 1993b. Turksen, I.B., Fuzzy function with LSE. Applied Soft Computing, 8, 1178-1188, 2008. Turksen, I.B., Fuzzy system models. Encyclopedia Complex. Syst. Sci., 4080-4094, 2009. Wu, X., Tan, Y., Wang, Y., Xiao, Y., Long-term prediction of time series with iterative extended Kalman Filter trained single multiplicative neuron model. Journal of Computational Information Systems, 8 (14), 5933-5940, 2012. Wu, X., Mao, J., Du, Z., Chang, Y., Online training algorithms based single multiplicative neuron model for energy consumption forecasting. Energy, 59, 126-132, 2013. Yadav, R.N., Kalra, P.K., John, J., Time series prediction with single multiplicative neuron model. Applied Soft Computing, 7, 1157–1163, 2007. Yolcu, U., The forecasting of Istanbul Stock Market with a high order multivariate fuzzy time series forecasting model. Turkish Journal of Fuzzy Systems, 3 (2), 118-135, 2012.

Yolcu, U., Egrioglu E., Aladag C.H., A new linear & nonlinear artificial neural network model for time series forecasting. Decision Support System Journals, 54, 1340-1347, 2013a. Yolcu, U., Aladag, C.H., Egrioglu, E., Uslu, V.R., Time series forecasting with a novel fuzzy time series approach: an example for Istanbul stock market. Journal of Statistical Computation and Simulation, 83 (4), 599-612, 2013b. Yu, T.H.K., Huarng, K.H., A neural network-based fuzzy time series model to improve forecasting. Expert Systems with Applications, 37, 3366–3372, 2010. Zadeh, L.A, Fuzzy Sets. Inform and Control, 8 (3), 338-353, 1965. Zarandi, M.H.F., Zarinbal, M., Ghanbari, N., Turksen, I.B., A new fuzzy functions model tuned by hybridizing imperialist competitive algorithm and simulated annealing. Application: Stock price prediction. Information Sciences, 222 (10), 213-228, 2013. Zhao, L., Yang, Y., PSO-based single multiplicative neuron model for time series prediction. Expert Systems with Applications, 36 (2), Part II, 2805-2812, 2009.

9