Solving the Sales Prediction Problem with Fuzzy ... - Semantic Scholar

WCCI 2012 IEEE World Congress on Computational Intelligence June, 10-15, 2012 - Brisbane, Australia

FUZZ IEEE

Solving the Sales Prediction Problem with Fuzzy Evolving Methods Dejan Dovˇzan

Vito Logar

ˇ Igor Skrjanc

Faculty of Electrical Engineering University of Ljubljana Trˇzaska 25 Email: [email protected]

Faculty of Electrical Engineering University of Ljubljana Trˇzaska 25 Email: [email protected]

Faculty of Electrical Engineering University of Ljubljana Trˇzaska 25 Email: [email protected]

Abstract—This paper presents the solving of the petrol sales volume estimation problem given by the Task Force on Competitions, Fuzzy Systems Technical Committee IEEE Computational Intelligence Society. The solution using eTS, SAFIS and eFuMo method is presented. The results are compared to linear ARX model identified using RLS method. Results using static and prediction model are given. The paper also presents parts of new eFuMo method, which is based on recursive Gustafson-Kessel clustering. Keywords: recursive Gustafson-Kessel clustering, evolving fuzzy model, eFuMo.

I. I NTRODUCTION Takagi-Sugeno fuzzy models are a powerful practical engineering tool for the modeling and control of complex systems. They expand and generalize the well known concept of gain scheduling. T-S models utilize the idea of linearization in a fuzzily defined region of the state space. Due to the fuzzy regions (clusters), the nonlinear system is decomposed into a multi-model structure consisting of linear models [1]. This enables the T-S fuzzy model to approximate virtually any nonlinear system within a required accuracy, provided that enough regions are given [2]. To build the T-S model the structure and the parameters of the local models must be identified [3]. Structure identification includes an estimation of the cluster centers (antecedent parameters), which is usually done by fuzzy clustering. Then for each cluster the local model’s parameters are estimated, which is usually done with the least-squares method [4]. The identification can be made off-line or on-line. On-line learning of fuzzy model has made significant progress in last few decades. Many methods already exist. Some of them are based on fuzzy logic (eTS [5], exTS [6], [7], simple TS [8], +eTS [9], FLEXFIS [10], FLEXFIS+ [11]), others use neural networks, which realize the behavior of fuzzy model (EFuNN [12], [13], DENFIS [14], ANFIS [15], GANFIS [16], SOFNN [17], SAFIS [18], SCFNN [19], NFCN [20], D-FNN [21], GDFNN [22], SONFIN [23], NeuroFAST [24], RAN [25], ESOM [26], Neural gas [27], ENFM [28], GAP-RBF [29]). The fuzzy model, that is usually used by the methods is first order Takagi-Sugeno (ANFIS, SONFIN, D-FNN, GD-FNN, DENFIS, eTS, NeuroFAST, SOFNN ...), zero order TakagiSugeno (SCFNN, SAFIS, GAP-RBF, EFuNN) or generalized fuzzy model (GANFIS).

U.S. Government work not protected by U.S. copyright

The methods differ also in the ability of adapting the fuzzy model and its structure. Some of the methods require an initial fuzzy model structure, which is then adapted. The adaptation includes only the adaptation of consequent and premise parameters (adaptive methods [16],[15]). Some of the methods include a mechanism for adding new clusters to the model structure (incremental methods [5]). Recently proposed methods include also mechanisms for merging, removing and splitting clusters (evolving methods). In this paper solving of the petrol sales volume prediction is discussed. The learning data was provided by the Task Force on Competitions, Fuzzy Systems Technical Committee IEEE Computational Intelligence Society and are real historic data. For solving the problem the eTS and SAFIS methods are considered. Their results are compared to linear ARX model, obtained by LSE and to results obtained by eFuMo toolbox. The eFuMo (evolving fuzzy model) is our humble contribution to evolving methods. It is based on recursive fuzzy c-means and Gustafson-Kessel (GK) clustering methods [30],[31],[32]. Evolving nature was achieved by adding the evolving mechanisms (adding, merging,...) to recursive clustering. The paper is organized in the following fashion. First a description of the eFuMo method is given. Because of the paper page limit only mechanisms that were used for solving this problem are presented. Next the problem is shortly discussed and solution is proposed. At the end results and conclusion are given. II. E VOLVING F UZZY M ODEL As stated above, the evolving fuzzy model method is our humble contribution to a set of evolving methods. It is included in a eFuMo toolbox, which is an in-house developed toolbox for Matlab environment. The method is based on c-means and GK recursive clustering proposed in [30]. The method includes adding, merging, deleting and splitting mechanisms. The parameters of local linear models are identified by fuzzy least squares. The method considers two different vectors. One is clustering vector, for clustering (xf ), and the other is regression vector, for local models’ parameters estimation (xk ). In most cases these two vectors can be the same. To cluster the input-output space, positions of cluster centers and variances of the data around the centers should be

308

calculated. Using the fuzzy c-means recursive algorithm this can be done by the following equations. First the change of centers positions is calculated as: η

∆vi (k + 1) =

(µik+1 ) (xf (k + 1) − vi (k)) , si (k + 1)

(1)

where vi (k) is the center position at previous sample, xf (k+1) is the current clustering data vector and si (k + 1) is the sum of past membership degrees, calculated as: si (k + 1) = γv si (k) +

µηik+1 .

(2)

The initial si (0) is usually set to one. With initialization of si (0) to higher value slows down the initial speed of adaptation. The γv is the forgetting factor. The centers are then moved for ∆vi (k + 1): vi (k + 1) = vi (k) + ∆vi (k + 1) i = 1, ..., c.

(3)

dFi = xf (k + 1) − vi (k + 1).

The equation for recursive determinant calculation is given as: ( )z si (k) det (Fi (k + 1)) = γc det (Fi (k)) (1 + A) si (k + 1) (13) η ) 1 µik+1 ( T −1 A= dFi [Fi (k)] dFi , (14) γc si (k) where z is the number of elements of the clustering vector. More detailed derivation of these equations can be found in [32]. Once the clusters are updated, the fuzzy recursive least squares are used to update the local linear models’ parameters. Different algorithms are proposed in [33], [5], [28], [34]. They are based on weighted recursive least squares. The equation for adaptation based on [34] are: ψi (k + 1) = βi [xk1 (k + 1), ..., xkz (k + 1), 1]T yi (k + 1) = βi xkz+1 (k + 1) i = 1, 2, ..., c. ( T

The index i is the index of the cluster and c is the number of clusters. Membership µik+1 of the current clustering vector xf is calculated as: µik+1 =

(

∑c j=1

1 dik+1 djk+1

, 2 ) η−1

T

d2ik+1 = (xf (k + 1) − vi (k)) Ai (xf (k + 1) − vi (k)) , (5) where Ai depends on a distance that we use. For Euclidian distance Ai is unit matrix, using Mahanalobis distance Ai is equal to inverse fuzzy covariance matrix. For the distance used with GK clustering algorithm the Ai is equal to fuzzy inverse matrix and determinant: Ai = [ρi det(Fi )]1/p F−1 i ,

Pi (k + 1) =

(4)

where dik+1 is the distance between clustering vector and the i-th cluster and η is the fuzziness (in most cases η = 2). The distance is defined as:

(6)

where ρi is usually set to one and p is the number of variables (number of elements of xf ). The fuzzy covariance is calculated by the following equation: µηik+1 si (k) Fi (k + 1) = γc Fi (k) + DF (7) si (k + 1) si (k + 1) i

B = [Fi (k)]

−1

−1

DFi [Fi (k)]

si (k) −1 + dTFi [Fi (k)] dFi C = γc η µik+1

(10) (11)

1 λr

Pi (k) −

(15)

Pi (k)ψi (k + 1)ψi (k + 1)Pi (k) λr + ψiT (k + 1)Pi (k)ψi (k + 1)

)

(

)

θi (k + 1) = θi (k) + Pi (k)ψi (k + 1) yi (k + 1) − ψiT (k + 1)θi (k)

(16) Parameters of i-th local model are denoted as θi , forgetting factor is denoted with λr and βi denotes membership degree of current clustering vector to the i-th cluster. For βi the membership degrees, calculated at cluster’s center update can be used. But in general they do not have a smooth transition between clusters. When identifying a process with a smooth non-linearity it’s better to recalculate the membership degrees using Gaussian functions. One can use radial base functions: −

µij = e

d2 ij 2ηm σ 2 ij

,

(17)

where dij is the distance between j-th component of i-th cluster and j-th component of clustering vector: ( )2 d2ij = xfj − vij , (18) 2 is the j-th diagonal element of fuzzy covariance matrix σij and ηm is the overlapping factor. Membership degree βi is calculated as a product of component membership degrees:

βi =

T

DFi = (xf (k + 1) − vi (k + 1)) (xf (k + 1) − vi (k + 1)) . (8) When using GK clustering algorithm recursive inverse fuzzy covariance matrix and recursive determinant calculation is needed. The equation for the inverse fuzzy variance matrix is following: [ ] B 1 si (k + 1) −1 −1 [Fi (k)] − (9) [Fi (k + 1)] = γc si (k) C

(12)

z ∏

µij ,

(19)

j=1

where z is the number of components of clustering vector xf . More accurate membership degrees are obtained using elliptical functions: D2 i

βi = e− 2ηm ,

(20)

where Di is calculated as: Di2 = (xf − vi ) F−1 i (xf − vi ) . T

(21)

Note that these membership degrees βi should be normalized. The above equations represent the adaptation algorithm of eFuMo method. To achieve the evolving nature of the method,

309

mechanisms for adding, removing, splitting and merging of clusters must be included. The method also employs a condition for preventing moving of clusters to which membership degree of the current clustering vector is lower than a certain threshold. This threshold is set somewhere between 0.1 and 0.3. The evolving mechanisms are shortly described in the following subsections. A. Adding the clusters Adding is usually based on some distance measure of the current clustering vector to existing clusters or on membership degrees. If a current clustering vector has lower membership degree than a predefined threshold a new cluster is added. The eFuMo adding criterion is based on a normalized distances between the current clustering vector and existing clusters. Two different distances are considered. One is the Mahanalobis normalized distance and other is the normalized Evklid distance. The second one is calculated for each component of clustering vector. For the Mahanalobis normalized distance the normalization vector is first defined. The vector consist of diagonal elements of fuzzy covariance matrix: √ √ √ dfinorm = [ fi11 fi22 ... fizz ]T , (22) The normalized distance is calculated as: 0.5 ((xf (k) − vi )T F−1 i (xf (k) − vi )) dinorm = , (23) 0.5 kn (dfTinorm F−1 i dfinorm ) √ where kn is user defined constant, normally set to 2 ηm , i is the index of the closest cluster. If the distance under Eq. 23 is greater than one, a new cluster can be added. The Evklid normalized distance is defined as: dij dijnorm = √ , kn fijj (24)

then a diagonal element is equal to a diagonal element of the closest cluster (fnewjj = fijj ). In the same manner a initial fuzzy covariance for the first cluster can be set considering the expected number of clusters and data range. Next the parameters of new local model can be initialized using weighted mean: ∑c i=1 ωij θij θi+1j = ∑ (26) c i=1 ωij where i is the index of cluster and j is the parameter index. Weights ωij can be equal to normalized membership degrees to other clusters, or equal to normalized membership degrees to other clusters combined with variance of parameters: ωij = βi

1 , σP2 i

(27)

jj

where σP2 i is the j-th diagonal element of least squares jj covariance matrix of i-th cluster. B. Removing the clusters Removing is in our case based on cluster’s support [9]. Support N si is defined as the number of samples that have the maximal membership to the certain cluster. The cluster is removed if in certain user defined time (kdelay ) does not receive a certain number of support samples. The cluster can also be deleted if its support, normalized with their age (Sn ), is lower than a fraction of mean. Age ai is defined as a number of samples from the creation ki and current sample k: ai = k − ki

(28)

N si . ai

(29)

Sni =

The condition for removing can be written as:

dij = |xfj − vij |, where i is the index of the closest cluster and j is the index of the component. Again the distance must be greater than one for adding the cluster. In this case there are several possibilities of defining the condition for adding. Either all component distances must satisfy the condition, either only one component distance satisfies the condition, or a certain number of component distances satisfy the condition. To prevent the adding of the clusters based on outliers, additional condition was introduced as in [35]. For the cluster to be added at least N consequent samples must satisfy the distance condition for adding. The number N is normally set from 3 to 10, depending on data. 1) Initialization of new cluster: The new cluster center is positioned at the current clustering vector. The fuzzy covariance matrix is initialized as diagonal matrix where the distances to closest cluster are considered. The diagonal elements are defined as: d2ij , (25) fnewjj = − 2ηm ln(ϵβ )

IF Sni < ε mean(Sn ) OR (N si < N strh AND k > ki + kdelay ) THEN remove i-th cluster.

(30)

C. Merging the clusters Merging algorithm is meant to merge the clusters that are either close or have the same local model’s parameters. The eFuMo method considers merging based on membership degrees of clusters to each other [28], and merging based on correlation method [11]. Basic merging is based on a normalized Mahanalobis distance. diknorm =

0.5 ((vk − vi )T F−1 i (vk − vi )) . 0.5 (dfTinorm F−1 i dfinorm )

(31)

Clusters are merged if the distances diknorm and dkinorm are below the threshold kd and the ratio between distances is lower than the threshold kr .

where ϵβ is a user defined constant, normally set to √ 0.15. If the distance dij is smaller than standard deviation ( fijj ),

310

IF diknorm < kd AND dkinorm < kd AND |1 −

diknorm | < kr dkinorm

THEN merge i-th and k-th cluster.

(32)

Variable ∆pmin ∆p pown vs

Threshold kd is defined based on the allowed maximal membership degree εβ of k-th cluster to i-th cluster and vice versa. √ kd = −2ηm ln(εβm ). (33) The kr should be set to around 0.1 and εβm to around 0.8. This rule merges only close clusters. In general, the clusters can be further apart, but still have the same shape and local model’s parameters. To merge this clusters, a correlation between membership degrees to clusters (clusters’ firing levels) (Cbetaik ) are monitored as in [11]. If the correlation is higher than a threshold value, the clusters are considered for merging. This condition is combined with the condition of angles between parameters (ϕik ) [11] and clusters’ support from the time of last change in the number of clusters (N sli ). Each time some of the evolving mechanisms changes the number of clusters the correlation value Cbetaik and support N sli are reset to zero. Clusters can also be merged if the angle between parameters are small and there are no other clusters between them. The initialization of new cluster position and variance is done as suggested in [28]. D. Splitting the clusters Splitting is currently meant to fine tune the evolving fuzzy model. It can add clusters in input-output space, where the output model error is higher than predefined threshold. The eFuMo method tracks the mean error for each cluster. At each sample, if the sample does not satisfy the distance condition for adding, the output error of the current model is calculated. Then the model error is divided among the clusters, depending on the membership degrees of the current clustering vector. If the error of one of the clusters exceeds the defined threshold, this cluster is split. The parameters of a new and old local model stay the same. Centers of new and old cluster are positioned based on a fuzzy covariance matrix. The new positions are calculated as: vi1 = vi + dfinorm vi2 = vi + dfinorm

(34)

This approach of initialization is presented also in [36]. Initialization can also be based on eigenvectors of fuzzy covariance matrix [37]. The fuzzy covariance matrix is divided by two. The error threshold is time decaying. The user specifies the maximal and the minimal value and decay constant for the threshold - the same as in [21]. The error is set to zero when number of clusters changes. The eFuMo method also considers a general time delay for the evolving mechanisms. The delay is user specified. If change in number of clusters occurs, the evolving mechanisms are stopped for this specified delay. III. P ROBLEM AND SOLUTION The goal of the competition is to solve the petrol volume sales estimation. The given data were station’s own price of the petrol, competitors’ prices and the day and the month of the data sample. The given data included 25 different petrol

k 7.5 5 3 8000−1

n 0.9 0.5 -2.7 0

TABLE I N ORMALIZATION CONSTANTS

stations, where the daily prices and sales of petrol for about one year were provided. First step in solving the problem was defining the least squares regressor and clustering vector. Based on our observations, the petrol sales is day dependent and relative price dependent. In general the consumption of gas doesn’t lower much with the price raise, because we highly depend on our car and therefore the petrol. This assumption probably doesn’t hold for a long period of time. If the prices will constantly grow, people will search for alternative sources of energy/feul. However since we are modeling petrol sales for one year period, the above assumption seems reasonable. Decision on which petrol station we buy gas is probably made by comparing the minimum price of petrol to the petrol price at each station. The decision is then made based on this difference and local position of the gas station. Based on that the regression vector for least squares was formed as: T

xk (k) = [d(k), ∆pmin (k − 1), ∆p(k), pown (k)] ,

(35)

where, d is the dummy vector for days. The vector has seven variables, each representing one day in a week. If the current data sample is from Monday, then the first element of vector d is one and others are zero. The ∆pmin is the difference between the minimal price and the price of own petrol station and ∆p is the difference between competitive prices and our own price (pown ). The output of the model is volume of sale vs . The variables differ in their maximal and minimal values. For clustering and evolving methods it is recommended that the variables are normalized. Usually this is done by using variance and mean value of the variable or minimal and maximal value of the variable. This is usually not possible when an online identification is made, since there is little prior knowledge of the system. However, one can estimate minimal and maximal value. In order to come as close as possible to a real life scenario, we decided on a fixed normalization of variables, meaning that all data sets were normalized with the same value as: xnorm = k · x − n. (36) Normalization constant for variables are given in Table I. Next step was applying the eFuMo method on the data sets using vector under Eq. 35 for clustering and least squares regressor. Using Mahanalobis distance for adding we found that very few clusters were added (one or two on average). This can be due the fact that either adding distance is not adequate, either clustering vector is not adequate or the system can actually be approximated with a linear model.

311

Data set eT Sg eT Sl SAF IS LSE eF uM ol

MAE Ncl MAE Ncl MAE Ncl MAE Ncl MAE Ncl

1 1150.65 7 1087.58 7 1031.31 21 982.05 1 957.74 6

2 1247.39 12 1162.95 12 1320.92 18 1195.14 1 1164.60 5

3 681.43 8 676.85 8 802.19 20 680.94 1 651.98 5

4 758.01 9 704.59 9 974.50 17 702.73 1 692.63 5

5 1044.74 11 711.38 11 800.35 8 577.48 1 627.79 3

Data set eT Sg eT Sl SAF IS LSE eF uM ol

TABLE II M EAN ABSOLUTE ERROR AND

Results of the modeling are shown in the next section. The problem was solved using eTS method, SAFIS method, eFuMo method and LSE method (for linear model identification). The results are compared. It should be noted that SAFIS was programmed from an article, therefore there could be some mistakes in the program. Also both methods were tuned as proposed in papers. Since we are no expert for this two methods we are convinced that they can give better results as presented in this paper. IV. R ESULTS As mentioned above the parameters of eTS method were set as in [5] and of SAFIS mathod as in [31]. For the eFuMo method Gustafson-Kessel on-line clustering was chosen. The clustering vector was chosen as: T

6 1149.34 11 1101.13 11 1797.37 19 1100.37 1 1219.75 6

7 823.27 10 789.98 10 877.36 15 767.43 1 773.53 6

8 1342.40 7 1357.44 7 1522.19 29 1360.36 1 1369.71 6

9 580.17 10 570.88 10 733.08 16 597.90 1 553.39 5

10 796.94 11 673.76 11 621.78 14 524.24 1 538.33 2

TABLE III M EAN ABSOLUTE ERROR AND NUMBER OF CLUSTERS FOR DATA SETS FROM 6 TO 10.

NUMBER OF CLUSTERS FOR DATA SETS FROM 1 TO 5.

xf (k) = [∆pmin (k − 1), pown (k), vs (k − 1)] .

MAE Ncl MAE Ncl MAE Ncl MAE Ncl MAE Ncl

(37)

The condition for adding the clusters was based on component distances. The condition for adding is satisfied if at least one component distance satisfies the condition under Eq. 24. A new cluster is added if three consequent samples satisfy the adding condition. The threshold for merging was set to kr = 0.1 and εβm = 0.8. The correlation threshold was 0.96 and angle threshold was 5 degrees. The removing of the cluster was done when its support was less than 1 percent of the mean support. Maximal threshold for splitting was set to 1 and minimal to 0.1 with the decay factor 0.99. The threshold for stopping the adaptation of cluster centers was set to 0.3. The initial value of Pi was a unit matrix and the value of si was 40. The forgetting factors were set to one. Local fuzzy least squares were used for linear parameter identification [5]. The mean absolute error for learning data sets and number of generated clusters are given in Tables II, III, IV, V, VI. Under eT Sg are the results, given by eT S method, using global fuzzy weighted least squares. Results using local fuzzy weighted least squares are under eT Sl . This algorithm was also used with the eFuMo method (eF uM ol ). The mean of MAE for all data sets are given in Table VII. It can be seen that the eFuMo method can approximate sales volume at least as good as eTS method. With the eFuMo method also different algorithms for linear parameters identification were tested. The results are in Table IX, where

312

Data set eT Sg eT Sl SAF IS LSE eF uM ol

MAE Ncl MAE Ncl MAE Ncl MAE Ncl MAE Ncl

11 1078.22 10 1140.16 10 1277.09 23 1121.71 1 1094.06 5

12 1337.24 10 1175.26 10 1573.80 21 1299.47 1 1177.44 5

13 945.13 10 931.75 10 994.50 21 898.19 1 937.14 5

14 374.87 15 362.93 15 445.26 14 394.34 1 366.70 4

15 810.07 12 757.27 12 830.53 20 738.29 1 679.68 6

TABLE IV M EAN ABSOLUTE ERROR AND NUMBER OF CLUSTERS FOR DATA SETS FROM 11 TO 15.

Data set eT Sg eT Sl SAF IS LSE eF uM ol

MAE Ncl MAE Ncl MAE Ncl MAE Ncl MAE Ncl

16 469.57 13 456.34 13 669.50 24 511.40 1 448.89 6

17 1012.07 9 932.12 9 1195.23 17 923.96 1 878.56 5

18 808.70 8 747.47 8 799.12 17 758.59 1 735.61 6

19 930.70 8 949.81 8 1158.47 19 990.30 1 982.97 5

20 432.88 16 429.98 16 486.16 16 460.35 1 420.45 4

TABLE V M EAN ABSOLUTE ERROR AND NUMBER OF CLUSTERS FOR DATA SETS FROM 16 TO 20.

Data set eT Sg eT Sl SAF IS LSE eF uM ol

MAE Ncl MAE Ncl MAE Ncl MAE Ncl MAE Ncl

21 645.91 12 528.34 12 628.40 14 529.52 1 546.05 5

22 755.65 10 716.42 10 998.55 16 708.66 1 715.13 4

23 777.67 7 741.26 7 1009.34 16 806.24 1 804.34 6

24 707.14 13 551.59 13 901.17 22 484.06 1 468.78 4

25 679.54 10 646.90 10 892.17 17 655.33 1 643.94 5

TABLE VI M EAN ABSOLUTE ERROR AND NUMBER OF CLUSTERS FOR DATA SETS FROM 21 TO 25.

1

mean MAE 793.82 858.22 973.61 790.76 777.97

method eT Sl eT Sg SAF IS LSE eF uM ol

0.9 5

0.8 4

0.7 0.6

TABLE VII OF

MAE

FOR ALL DATA SETS WITH THE STATIC MODEL .

pown

M EAN

0.5 3

0.4

method eF uM og eF uM ol eF uM olg eF uM ogg eF uM ost

mean MAE 792.38 777.97 791.31 785.04 870.29

0.3

0.1

1

2

0.2 0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1.1

1.2

1.3

vs

TABLE VIII M EAN OF MAE

Fig. 1.

FOR ALL DATA SETS FOR STATIC MODEL WITH DIFFERENT WEIGHTED LEAST SQUARES ADAPTATION .

Cluster positions for the third data set.

1.3 1.2

method eF uM og eF uM ol eF uM olg eF uM ogg eF uM ost

mean MAE 738.93 720.81 734.48 725.68 816.49 TABLE IX

M EAN

OF

MAE

FOR ALL DATA SETS FOR PREDICTION MODEL WITH DIFFERENT LEAST SQUARES ADAPTATION .

1.1 1

Δpmin

the mean values of MAE are given. The tested algorithms were global wlse (eF uM og ) and local wlse (eF uM ol ) [5], wlse presented in [28] (eF uM ogg ), in [31] (eF uM olg ) and in [33] (eF uM ost ). The obtained results with evolving methods are not as good as we expected. Partly because of outliers in data sets (the MAE was calculated including the outliers) and partly because of small numbers of data (evolving methods usually need a larger set of data to properly identify the fuzzy model). The accuracy of the fuzzy model can be further improved using one step prediction model. The results for the one step prediction model are given in Table IX. In Figure 4 the sales volume and its estimation using static models for the third data set is given. In Figure 5 normalized inputs for static fuzzy model are presented. The clusters for the same set are shown in Figures 1, 2 and 3. The clusters were generated with eFuMo method. In Figure 6 an error histogram for the third data set is given. Not considering a few large error amplitudes, which are the results of outliers in real data, the error distribution can be defined as the Gaussian. From this we can conclude that the model structure is adequate. It was noticed that the sales volume data have some spikes. Usually somewhere between the 70th and the 80th sample. The spikes can not be predicted using given prices. They are probably date dependent (probably some holidays). This assumption is however not certain. In order to be sure that the assumption holds we would

5

1

0.9 3

0.8 0.7

2

0.6

4

0.5 0.4 0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1.1

1.2

1.3

vs

Fig. 2.

Cluster positions for the third data set.

need data for at least three years. Adding the dummy variable for month and competitive prices to the regression vector, the accuracy of prediction model can be further improved for about one percent. However, the fuzzy model in this case is less transparent and more comprehensive. V. C ONCLUSION The paper describes solving of the volume sales estimation problem. Fuzzy models identified with evolving methods are compared to linear models identified with LSE method. The idea behind the solution was to use as few inputs as possible to make the fuzzy model transparent. The same method with the same setting was used on all data sets. Only one sweep trough data was made. The results show that the given data can be adequately approximated with a linear model. With the use of the fuzzy model not much is gained on accuracy. In our case the eFuMo method performed better than other methods. However, it can not be said that eFuMo method is better than others, since other methods were not tuned by an expert. The working data sets in the presented problem were quite small. In general, the eFuMo method is meant for larger data sets.

313

1 0.9 5

0.8 4

0.7

pown pown

0.6 0.5 3

0.4 0.3 1

2

0.2

1.1 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

50

100 150 200 250 300 350 400 sample

0.8

0.1 0.4

0.5

0.6

0.7

0.8 Δpmin

0.9

1

1.1

1.2

1.3

0.7 0.6

Fig. 3.

Cluster positions for the third data set. 0.5 Δp 0.4

real eFuMo model eTS model SAFIS model

10000

0.3 0.2 0.1

8000

0

50

100 150 200 250 300 350 400 sample

0

50

100 150 200 250 300 350 400 sample

vs

1.3

6000

1.2 1.1 1

4000 Δpmin

0.9 0.8

2000

0.7

50

Fig. 4.

100

150

200 sample

250

300

350

0.6 0.5 0.4

Sales volume and its estimation for the third data set.

Fig. 5.

However, the results show that it could also perform quite well on smaller data sets using proper settings. The eFuMo toolbox is currently in developer’s version. We still need to make some tests and experiments to evaluate the performance of the method. The user friendly final version of the toolbox should be available for download at the end of this summer. The toolbox will be downloadable from our web site http : //msc.f e.uni−lj.si/ with help files and settings suggestions. R EFERENCES [1] T. A. Johanson and R. Murray-Smith, Operating regime approach to nonlinear modeling and control, M.-S. R. Johanson T A, Ed. Taylor Francis, 1981. [2] C. L. Hwang and L. J. Chang, “Fuzzy neural-based control for nonlinear time-varying delay systems,” IEEE Trans. Syst. Man Cyber. part B, vol. 37, no. 6, 2007. [3] T. Takagi and M. Sugeno, “A fuzzy identification of systems and its applications to modelling and control,” IEEE Trans. on Syst., Man and Cyber., vol. 15, no. 1, pp. 116–132, 1985. [4] S. L. Chiu, “Fuzzy model identification based on cluster estimation,” Journal of Intellegent and Fuzzy Systems, vol. 2, pp. 267–278, 1994. [5] P. P. Angelov and D. P. Filev, “An approach to on-line identification of Takagi-Sugeno fuzzy models,” IEEE Trans. Syst. Man Cyber. Part B, vol. 34, no. 1, pp. 484–497, 2004.

Normalized inputs to static model for the third data set.

[6] M. Asif, P. Angelov, and H. Ahmed, “An approach to real-time colorbased object tracking,” in Evolving Fuzzy Systems, 2006 International Symposium on, sept 2006, pp. 86 –91. [7] P. Angelov and X. Zhou, “Evolving fuzzy systems from data streams in real-time,” in Evolving Fuzzy Systems, 2006 International Symposium on, 2006, pp. 29 –35. [8] P. Angelov and D. Filev, “Simple T S: A Simplified Method for Learning Evolving Takagi-Sugeno Fuzzy Models,” in Proceedings of The 2005 IEEE International Conference on Fuzzy Systems, Reno, Nevada, USA, 22-25 May 2005, pp. 1068 – 1073. [9] P. Angelov, EVOLVING INTELLIGENT SYSTEMS: Methodology and Applications. New Jersey: Wiley, 2010, ch. Evolving Takagi-Sugeno Fuzzy Systems From Streaming Data (eTS+), pp. 21 – 50. [10] E. Lughofer and E. P. Klement, “Flexfis: A variant for incremental learning of takagi-sugeno fuzzy systems,” in Proceedings of The 2005 IEEE International Conference on Fuzzy Systems, Reno, Nevada, USA, 22-25 May 2005, pp. 915 – 920. [11] E. Lughofer, J.-L. Bouchot, and A. Shaker, “On-line elimination of local redundancies in evolving fuzzy systems,” Evolving Systems, no. 2, p. 165187, 2011. [12] N. K. Kasabov, “Evolving fuzzy neural networks for supervised/unsupervised on-line knowledge-based learning,” IEEE Trans. Syst. Man Cyber. Part B, vol. 31, no. 6, pp. 902–918, 2001. [13] N. Kasabov, “Evolving fuzzy neural networks-Algorithms, applications and biological motivation,” in Methodologies for the Conceptation,

314

70 60

[31]

number

50

[32] 40

[33] 30 20

[34] 10 0

−12000

−8000

Fig. 6.

−4000 −2000 error amplitude

0

2000 4000

[35] [36]

Error histogram.

[37] [14]

[15]

[16]

[17]

[18]

[19]

[20] [21]

[22]

[23]

[24]

[25] [26]

[27] [28]

[29]

[30]

Design and Application of Soft Computing, Japan, 1998, pp. 271–274. N. K. Kasabov and Q. Song, “DENFIS: Dynamic Evolving NeuralFuzzy Inference System and its application for time-series prediction,” IEEE Trans. on Fuzzy Syst., vol. 10, no. 2, pp. 144–154, 2002. J. Shing and R. Jang, “ANFIS : Adaptive-Network-Based Fuzzy Inference System,” IEEE Trans. Syst. Man Cyber., vol. 23, no. 3, pp. 665–685, 1993. M. F. Azeem, H. Hanmandlu, and N. Ahmad, “Structure identification of generalized adaptive neuro-fuzzy inference systems,” IEEE Trans. on Fuzzy Syst., vol. 11, no. 5, pp. 666–681, 2003. G. Leng, G. Prasad, and T. M. McGinnity, “An on-line algorithm for creating self-organizing fuzzy neural networks,” Neural Networks, vol. 17, p. 1477 1493, 2004. H. J. Rong, N. Sundararajan, G. B. Huang, and P. Saratchandran, “Sequential adaptive fuzzy inference system (SAFIS) for nonlinear system identification and prediction,” Fuzzy Sets and Syst., vol. 157, no. 9, pp. 1260–1275, 2006. F. J. Lin, C. H. Lin, and P. H. Shen, “Self-constructing fuzzy neural network speed controller for permanent-magnet synchronous motor drive,” IEEE Trans. on Fuzzy Syst., vol. 9, no. 5, pp. 751–759, 2001. C. T. Lin, “A neural fuzzy control system with structure and parameter learning,” Fuzzy Sets and Syst., vol. 70, pp. 183–212, 1995. S. Wu and M. J. Er, “Dynamic fuzzy neural networksA novel approach to function approximation,” IEEE Trans. Syst. Man Cyber. Part B, vol. 30, no. 2, pp. 358–364, 2000. S. Wu, M. J. Er, and Y. Gao, “A fast approach for automatic generation of fuzzy rules by generalized dynamicc fuzzy neural networks,” IEEE Trans. on Fuzzy Syst., vol. 9, no. 4, pp. 578–594, 2001. C. F. Juang and C. T. Lin, “An on-line self-constructing neural fuzzy inference network and its applications,” IEEE Trans. on Fuzzy Syst., vol. 6, no. 1, pp. 12–32, 1998. S. G. Tzafestas and K. C. Zikidis, “NeuroFAST: On-line neuro-fuzzy ART-based structure and parameter learning TSK model,” IEEE Trans. Syst. Man Cyber. Part B, vol. 31, no. 5, pp. 797–802, 2001. J. Patt, “A resource allocating network for function interpolation,” Neural Computat., vol. 3, no. 2, pp. 213–225, 1991. D. Deng and N. Kasabov, “Evolving self-organizing maps for on-line learning, data analysis and modeling,” in Proc. IJCNN 2000 Neural Networks Neural Computing: New Challenges Perspectives New Millenium, vol. 6. New York: IEEE, 1991, pp. 3–8. B. Fritzke, “A growing neural gas network learns topologies,” Adv. Neural Inform. Processing Syst., vol. 7, pp. 845–865, 1995. H. Soleimani-B, C. Lucas, and B. N. Araabi, “Recursive gath-geva clustering as a basis for evolving neuro-fuzzy modeling,” Evolving Systems, vol. 1, no. 1, pp. 59 – 71, 2010. G.-B. Huang, P. Saratchandran, and N. Sundararajan, “A recursive growing and pruning rbf (gap-rbf) algorithm for function approximations,” in Proceedings of The Fourth International Conference on Control and Automation (ICCA03), Montreal, Canada, June 2003, pp. 10 – 12. ˇ D. Dovˇzan and I. Skrjanc, “Predictive functional control based on

315

an adaptive fuzzy model of a hybrid semi-batch reactor,” Control Engineering Practice, vol. 18, no. 8, pp. 979 – 989, 2010. ——, “Recursive fuzzy c-means clustering for recursive fuzzy identification of time-varying processes,” ISA Transactions, vol. 50, no. 2, pp. 159 – 169, 2011. ——, “Recursive clustering based on a gustafsonkessel algorithm,” Evolving Systems, vol. 2, pp. 15–24, 2011. J. de Jes´us Rubio, EVOLVING INTELLIGENT SYSTEMS Methodology and Applications. Hoboken, New Jersey: John Wiley & Sons, 2010, ch. STABILITY ANALYSIS FOR AN ONLINE EVOLVING NEUROFUZZY RECURRENT NETWORK, pp. 173–199. ˇ S. Blaˇziˇc, I. Skrjanc, S. Gerkˇsiˇc, G. Dolanc, S. Strmˇcnik, M. B. Hadjiski, and A. Stathaki, “Online fuzzy identification for an intelligent controller based on a simple platform,” Engineering Applications of Artificial Intelligence, vol. 22, p. 628638, 2009. L. Hartert, M. S. Mouchaweh, and P. Billaudel, “A semi-supervised dynamic version of fuzzy k-nearest neighbours to monitor evolving systems,” Evolving Systems, vol. 1, pp. 3–15, 2010. E. Lughofer, “Evolving Fuzzy Systems-Methodologies, Advanced Concepts and Applications,” ser. Studies in Fuzziness and Soft Computing Series, J. Kacprzyk, Ed. Berlin Heidelberg: Springer-Verlag, January 2011, vol. 266, p. 476. I. Skrjanc, B. Hartmann, O. Banfer, O. Nelles, A. Sodja, and L. Teslic, “Supervised hierarchical clustering in fuzzy model identification,” Fuzzy Systems, IEEE Transactions on, vol. PP, no. 99, p. 1, 2011.