INTEGRATION OF SOM NETWORK AND

3 downloads 0 Views 163KB Size Report
organizing map (SOM) network with genetic algorithm (GA) and particle swarm opti- ... ity and robustness of ANNs, typically in data-rich environment, can come in ... EAs [45] have also been applied to optimize the parameters defining RBF networks ... clustering method [5], and it is an unsupervised learning ANN that applies ...
International Journal of Innovative Computing, Information and Control Volume 7, Number 4, April 2011

c ICIC International ⃝2011 ISSN 1349-4198 pp. 1959–1970

INTEGRATION OF SOM NETWORK AND EVOLUTIONARY ALGORITHMS TO TRAIN RBF NETWORK FOR FORECASTING Ren-Jieh Kuo1 , Tung-Lai Hu2 and Zhen-Yao Chen3,∗ 1

Department of Industrial Management National Taiwan University of Science and Technology No. 43, Sec. 4, Keelung Rd., Taipei 10607, Taiwan [email protected] 2

3

Department of Business Management Graduate Institute of Industrial and Business Management National Taipei University of Technology No. 1, Sec. 3, Chung-hsiao E. Rd., Taipei 10608, Taiwan [email protected] ∗ Corresponding author: [email protected]

Received December 2009; revised May 2010 Abstract. This paper intends to propose an integrated method which combines selforganizing map (SOM) network with genetic algorithm (GA) and particle swarm optimization (PSO)-based (ISGP) algorithm to train the radial basis function (RBF) network for function approximation. The experimental results for three benchmark problems indicated that such integration can have better performance. In addition, using the proposed ISGP algorithm to exercise oil price forecasting also showed that the proposed algorithm is able to achieve more promising accuracy than the auto-regressive integrated moving average (ARIMA) model and four evolutionary algorithms (EAs) proposed in literatures. Keywords: Evolutionary algorithm, Self-organizing map network, Radial basis function network, Genetic algorithm, Particle swarm optimization

1. Introduction. Evolutionary algorithms (EAs) are heuristic and stochastic search procedures based on the mechanics of natural selection, genetics and evolution, which allow them to find the global solution for a given problem [48] (e.g., [41]). On the other hand, artificial neural networks (ANNs) are essentially a nonlinear modeling approach that provides a fairly accurate universal approximation to any function [56]. The learning capability and robustness of ANNs, typically in data-rich environment, can come in handy when discovering regularities from large datasets. This can be unsupervised as in clustering or supervised as in classification [46]. Among ANNs, radial basis function (RBF) network is a particular class of multilayer feed-forward ANNs [53]. It has several key advantages which includes finding the input to output map using local approximators, rapid learning while requiring fewer examples, good approximation and learning ability, and is easier to train [39]. The performance of RBF network depends upon the parameters of kernel functions, which are the nonlinear elements of the network. However, how to select a suitable number of hidden layer neurons remains as an open question. A common approach is to start with a predetermined number of hidden units, which is chosen by using a priori knowledge. It usually results in too many hidden units and poor generalization [50]. EAs have been developed to train ANNs which have been employed for many applications. EAs [45] have also been applied to optimize the parameters defining RBF networks [65]. On the other hand, since the self-organizing map (SOM) network can be seen as a 1959

1960

R. J. KUO, T.-L. HU AND Z.-Y. CHEN

clustering method [5], and it is an unsupervised learning ANN that applies neighborhood and topology to cluster associated data into one group [10]. Accordingly, the purpose for this paper is to employ SOM network first with autoclustering capability to determine the initial activation function parameters for RBF network. Secondly, the integration which combines SOM network with genetic algorithm (GA) and particle swarm optimization (PSO)-based (ISGP) algorithm is proposed to adjust these related parameters involved. In order to assess the adequateness of proposed ISGP algorithm for training RBF network, three benchmark problems are employed. Results show that the proposed algorithm is able to achieve more promising accuracy compared with four EAs: RBF network [11], GA-based [58], PSO-based [22] and hybrid of PSO and GA (HPG) [36] algorithms. Next, the exercising of oil price forecasting indicates that the proposed ISGP algorithm has higher accuracy than four EAs proposed in the literatures and the auto-regressive integrated moving average (ARIMA) models. The rest of this paper is organized as follows. Section 2 presents the relevant literatures for this study. Section 3 proposes the ISGP algorithm for training RBF network while the experiment results are shown in Section 4. Section 5 presents the evaluation results of this algorithm by exercising the oil price forecasting. Finally, the concluding remarks are made in Section 6. 2. Literature Review. In this section, we will present relevant literatures regarding to this study. 2.1. Forecasting methods. Time series data are often examined in hopes of discovering a historical pattern that can be exploited in the forecast. Basically, time series modeling is based on the assumption that the historical values of a variable provide an indication of its value in the future [73]. The applied time series forecasting models include linear and nonlinear regression approaches [55], ANNs [17], transfer functions [59] and hybrid algorithms that include a periodic component with SOM network [34]. A linear correlation structure is assumed among the time series data [57]. Furthermore, SOM network is an unsupervised ANN, and it has the ability to partition the space of input training data set into many subsets without prior knowledge about the classifying criteria, where each subset may be considered as stationary time-series data [21]. Due to its fast learning, self-organized and graphical inherence, literature indicates that the SOM network can be effectively employed to narrow down initial design options [68]. Box and Jenkins first developed the ARIMA methodology for forecasting time series events in 1976 [7]. However, when ARIMA models are applied to process data that are nonlinear, forecasting errors often increase greatly as the forecasting horizon becomes longer. To improve its forecasting on nonlinear time series events, researchers have developed alternative modeling approaches [23]. Several researchers have provided empirical evidence on the comparative advantage of one model over the other in various forecasting situations. For example, Prybutok et al. [54] compared ANNs with ARIMA and linear regression for maximum ozone concentrations and found that ANNs were superior to the linear models. On the other hand, nonlinear forecasting techniques such as RBF network [8] have shown good performance on short-term prediction [47]. The adjustable parameters of RBF network are the receptive field centers (the location of basis functions), the width (the spread), the shape of the receptive field and the linear output weights [9]. Next, Chen et al. [11] have employed the orthogonal least squares (OLS) algorithm for training and configuring the RBF network. The hidden units are allocated one by one based on an error reduction ratio [50].

INTEGRATION OF SOM NETWORK AND EVOLUTIONARY ALGORITHMS

1961

RBF network was proposed by Duda and Hart in 1973 [18], it is suited for applications such as pattern discrimination and classification, interpolation, prediction, forecasting and process modeling [52]. RBF networks have the ability to rapidly learn complex patterns and tend to present data then adapt to changes quickly [27]. Such characteristics make RBF network suitable for time series forecasting, especially those systems governed by linear and/or non-stationary processes [42]. In the field of prediction, RBF network has received considerable degree of attention recently due to its universal approximation properties and simple parameter estimation [70]. For example, Yu et al. [69] proposes an RBF network ensemble forecasting model to obtain accurate prediction results and improve prediction quality further. Also, a reliable price prediction model based on an advanced self-adaptive RBF network is presented and is used to auto-configure the structure of networks and obtain the model parameters [44]. Moreover, a RBF network model was developed to forecast the total ecological footprint (TEF) from 2006 to 2015 [38] as well. All of these represent the capability of RBF network for making good prediction in wide applications. Finally, it was demonstrated that the RBF network architecture is capable of achieving comparable or lower prediction errors, compared to traditional Box – Jenkins models [67]. Due to its universal approximation properties and simple parameter estimation [70], we are confident to conclude that RBF network has been recognized as a good approach for prediction. 2.2. Integration of EAs for RBF network learning. EAs have attracted much interest in various applications [26]. Guo et al. [28] proposed a multi-objective evolutionary algorithm constraint handling (EACH) method, it is able to find the feasible region in the search space, to obtain the jagged Pareto front and thereby to provide efficient schedule for aircraft landing. On the other hand, ANN performs well at learning when using some EAs such as GA [63] and PSO [49] for training. GA was pioneered in 1975 by Holland [29]. Its concept is to mimic the natural evolution of a population by allowing solutions to reproduce and to create new solutions, which then compete for survival in the next iteration [51]. The parallel searching mechanism is the main advantage of GAs since they cannot easily get trapped in local minima [16]. For example, Kuo et al. [35] aimed at developing a learning algorithm for fuzzy logic, and it’s consisted by a GA based fuzzy neural network (GFNN) then used it to capture the knowledge of stock market experts. Applying GAs to the topological definition of RBF network usually results in excellent solutions [6]. Sarimveis et al. [58] proposed a GA-based algorithm, whose objective is the minimization of an error function with respect to the structure of the RBF network, the hidden node centers and the weights between the hidden layer and the output layer. However, the algorithm does not make a proper decision in the value of width within the network. This may result in less accurate learning during the process of function approximation by RBF network. In addition to various GA improvements, recent developments in EAs have introduced quite some natural processes and one among them is PSO algorithm [19]. PSO is another adaptive algorithm based on the swarm cooperation and competition in the social-psychological science [31]. In PSO, the population is called swarm and the individuals are called particles. Each particle moves with an adaptable velocity within search space and retains the memory of the best position that it has ever encountered. This best position is shared with other particles in the swarm at each iteration [62]. Mu and Sheng [48] then also proposed a novel training algorithm based on evolutionary programming (EP) and PSO for evolutionary diagonal recurrent neural network (EDRNN).

1962

R. J. KUO, T.-L. HU AND Z.-Y. CHEN

Figure 1. The framework for the proposed ISGP algorithm In addition, Feng [22] proposed an evolutional PSO learning-based RBF network system to solve nonlinear control and modeling problems. The first positions and velocities of the algorithm are randomly initialized within a population; then, at the next epoch, positions and velocities are adjusted based on the corresponding best information in previous round. However, the randomly initialized method will increase the entropy of the individual solution and hard to converge. The performance of simple PSO greatly depends on its parameters, and it often encounters the problem of being trapped in local optimal due to premature convergence [2]. Therefore, combining more methodologies can obtain better accuracy in prediction than using a single methodology [40] (e.g., [64]). Considering different merits of some EAs, Kuo et al. [36] proposed the HPG algorithm, which applies EC learning and is designed to resolve the problem of network parameters training for RBF network. They supplement a practical application on the historical sales forecasting data of papaya milk to expound the superiority of the HPG algorithm. 3. Methodology. The proposed ISGP algorithm is designed to resolve the problem of parameters training for RBF network, and it provides the settings of parameters such as hidden node neuron, width and weight. In the proposed algorithm, SOM network is used to determine the number of centers and the corresponding position values through its auto-clustering ability. The results are then used as the number of neurons in RBF network. In which, the GA and PSO-based (GP) algorithm of this study is utilized to train the RBF network by further adjusting the network parameters. The framework for the proposed ISGP algorithm is illustrated in Figure 1. Consequently, the flowchart for the GP algorithm is depicted in Figure 2, where S represents the population size. The activation function that the RBF network hidden layer adopted is the most frequently used Gaussian basis function (Φ) with the following form [22]: )2 ( ∥x − cj ∥ , (1) Φ(∥x − cj ∥) = exp − σj where ∥x − cj ∥ is the Euclidean distance between an input vector x and a neuron cj , and σj represents the width of the j th RBF network hidden node.

INTEGRATION OF SOM NETWORK AND EVOLUTIONARY ALGORITHMS

1963

Figure 2. The flowchart of the GP algorithm Next, the Gram-Schmidt scheme [25] and Moore Penrose pseudo-inverse [14] methods of the basis matrix are used to calculate the weights between RBF network hidden and output layers. In addition, the inverse of root mean squared error (i.e., RMSE−1 ) [13] is used as the fitness function. The fitness values for all measured algorithms in the experiment are computed by maximizing the RMSE−1 [37] defined as: √ N F itness = RM SE −1 = ∑N , (2) ˆj )2 j=1 (yj − y where yi is the actual output and yˆj is the predicted output of the learned RBF network model for the j th testing pattern and N is the number of the testing set. The learning through the Maximum selection PSO learning [22] (i.e., PSO approach) would preserve better vector solution for the next epoch through the memory mechanism [66] of information sharing. Next, through some steps such as exchange mutation, uniform crossover [61] and addition/deletion [58] with GA learning (i.e., GA approach), they may further help individuals to share the solution of parameter values with other individuals within the same population. Additionally, due to the property of global search with GA, no matter what the fitness values of the individuals in population are, they all have chance to proceed with steps of genetic operators and enter into the next generation for population to evolve. The diversity of individuals results in higher chance to search in the direction of global minimum instead of being restrained to local minimum. Thus, the solution space in population could be changed gradually and converge toward the optimal solution. By adopting the advantages of GA and PSO approaches, the GP algorithm simultaneously implements two learning approaches targeting the first half and the second half of the initialized primitive population. The first half population is implemented with PSO approach and followed by GA approach while the second half is implemented with GA approach first and followed by PSO approach respectively. The GP algorithm then

1964

R. J. KUO, T.-L. HU AND Z.-Y. CHEN

Table 1. Three continuous test functions in the experiment Continuous test

Equation

function [60] Rosenbrock

RS(xj , xj+1 ) =

n−1 ∑

[100(x2j − xj+1 )2 + (xj − 1)2 ], n = 2

j=1

Griewank B2

GR(xj , xj+1 ) = B2(xj , xj+1 ) =

x2j

+

n ∑

x2j 4000

j=1 2 2xj+1 −



n ∏ j=1

x

j cos( √j+1 ) + 1, n = 1

0.3 cos(3πxj ) − 0.4 cos(4πxj+1 ) + 0.7, j=1

forms the [PSO + GA] and [GA + PSO] sub-populations respectively as a new generated population. The new generation is produced using the Roulette wheel selection [24] mechanism and the algorithm stops after a specific number of generations have been completed. The whole population makes better training for two sub-populations during evolutionary learning process for upcoming preceding evolution. In other words, by integrating the approaches based on PSO and GA mentioned above, the GP algorithm can increase the diversity of population-based solutions in the process of evolution, and thus increase the possibility of solving the global optimal solution. 4. Experiment Setting. This section will verify the proposed ISGP algorithm using three benchmark problems including Rosenbrock, Griewank, and B2 continuous test functions [60], which are defined in Table 1. For each problem one thousand randomly generated data are divided into three parts [43] to train RBF network where 65% is used for training, 25% is used for testing (i.e., learning stage: 90% dataset) and the rest 10% is used for validation, and here we also examine the learning status and adjust the parameter setting. This experiment was performed on a PC with Intel XeonTM CPU running at 3.40GHz symmetric multi-processing (SMP) with 2GB of RAM. Simulation was programmed in the Java 2 Platform, Standard Edition (J2SE) 1.5. 4.1. Setting of parameters. In this study, the settings for all parameters are obtained based on the literatures [29,31,33,58]. In the GA approach, it has recently gained its attention due to its powerful search for identification of optimum parameters [4]. In the PSO approach, the inertia weight k is designed as a tradeoff between the global and local search. Larger values of k facilitate global exploration while lower values encourage a local search [66]. The ISGP algorithm starts with the selection of the parameters’ setting shown in Table 2 to ensure consistent basis in the experiment. 4.2. Performance analysis of experimental results. As mentioned earlier, RBF network used 90% dataset during the learning stage. After one thousand epochs in the evolutionary process achieved, the optimal RBF network parameters solution was obtained. Lastly, it was validated by using the 10% validation set, which had not been utilized throughout the entire learning stage. The RMSE was used to measure performance. It randomly generated unrepeated 10% validation set to prove how the solution of individual parameters approximate to three benchmark problems and then recorded the RMSE values to confirm the learning situation of RBF network.

INTEGRATION OF SOM NETWORK AND EVOLUTIONARY ALGORITHMS

1965

Table 2. Setting of parameters for the proposed ISGP algorithm in the experiment Parameter Description Value E The maximum number of epochs 1000 G The maximum number of generations of SOM network 100000 C The number of centers of SOM network [1, 100] ε The learning rate of SOM network 0.9 σ The radius of SOM network 10 S Population size 30 Pm Exchange mutation probability [0.01, 0.05] Pc Uniform crossover probability [0.5, 0.8] Pa , Pd Addition and Deletion probability 0.005 k Inertia weight [0.4, 0.75] c1 , c2 Acceleration constants 2 Table 3. Comparison of results for all algorithms in the experiment Rosenbrock function Griewank function Training Validation Training Validation set set set set RBF 11880 ± 12731 ± 26.39 ± 27.85 ± network [11] 1343.40 2555.70 2.36 3.78 GA-based 6.93E-4 ± 8.84E-4 ± 5.04E-1 ± 5.27E-1 ± [58] 1.24E-4 2.12E-4 120.23E-4 165.51E-4 PSO-based 60.37E-4 ± 87.28E-4 ± 7.32E-1 ± 8.43E-1 ± [22] 31.49E-4 49.61E-4 1205.01E-4 1241.17E-4 4.94E-4 ± 6.17E-4 ± 5.69E-1 ± 5.42E-1 ± HPG [36] 1.43E-4 1.04E-4 94.29E-4 124.28E-4 1.12E-4 ± 1.37E-4 ± 4.13E-1 ± 4.28E-1 ± ISGP 0.62E-4 0.39E-4 11.42E-4 33.48E-4 Algorithm

B2 function Training Validation set set 5791.50 ± 5848.80 ± 403.70 673.66 24.92E-2 ± 30.85E-2 ± 4.84E-3 13.26E-3 23.89E-2 ± 38.48E-2 ± 3.17E-3 627.92E-3 20.81E-2 ± 24.17E-2 ± 1.74E-3 7.08E-3 7.24E-2 ± 6.91E-2 ± 2.18E-3 3.57E-3

The learning and validation stages mentioned above were implemented fifty times then average RMSE values were calculated. The values of the average RMSEs and standard deviation (SD) are shown in Table 3 for all algorithms. The results show that the proposed ISGP algorithm has the smallest average RMSE ± SD values for three benchmark problems and is very stable compared to other algorithms. Next, as shown in Table 3, the values of training and validation performance are consistently small, which means RBF network trained through the proposed ISPG algorithm provides certain stability. Thus over-fitting and over-training problems do not exist in the experiment adopting the ISPG algorithm. Such result not only suffices for the training set and validation set, a generalization could also be made with regards to other unseen dataset. Additionally, from the numerical results comparison in Table 3, the superiority of performance results obtained from the ISPG algorithm when verified under different datasets is shown. 5. Case Study and Results. Among the literatures of time series forecasting with ANNs, most studies [30,72] used the ARIMA models [7] as the benchmark to test the effectiveness of the ANN model with mixed results [71]. This section will utilize this ISGP algorithm to run the forecasting for oil price exercise. The data source of oil price in around one year period is from West Texas Intermediate (WTI) (currency in US dollar)

1966

R. J. KUO, T.-L. HU AND Z.-Y. CHEN

Table 4. The setting of parameters for all algorithms in the oil price forecasting exercise Parameter Description Value E The maximum number of epochs 1000 G The maximum number of generations of SOM network 100000 S Population size 30 Pm Exchange mutation probability 0.025 Pc Uniform crossover probability 0.65 Pa , Pd Addition and Deletion probability 0.005 k Inertia weight 0.5 c1 , c2 Acceleration coefficients 2 Table 5. The data for the oil price forecasting exercise Data set

The observations: month-day-year (number of samples) Total number of data Learning set (90%) Forecasting set (10%) 02/01/2008 ∼ 02/01/2008 ∼ 12/29/2008 ∼ Period 02/02/2009 (258) 12/24/2008 (233) 02/02/2009 (25) and is adopted as observations. Then all algorithms start with parameters setting shown in Table 4 to ensure consistency during the whole exercise. 5.1. Input data for RBF network learning. Most studies in literatures use convenient ratio of splitting for in- and out-of-samples such as 70:30%, 80:20% or 90:10% [74]. Here, we use the ratio of 90:10% as the basis of data division. The detailed data distribution of the oil price forecasting exercise is listed in Table 5. The learning stage of RBF network is based on daily observations; it contains 65% training set and 25% testing set. Accordingly, the following predicted values were generated in turn by the moving window method. The first 90% of the observations were used for model estimation while the remaining 10% were used for validation and one-step-ahead forecasting. 5.2. Forecasting performance and error measurement. The study carries out oil price forecasting exercise based on ARIMA models, and the procedures are divided into the following stages [51]: data identification, model estimation, model diagnosis and model forecasting. Meanwhile, EViewsTM 6.0 software was used in the analysis of ARIMA models. If the data is stationary, model estimation can be implemented directly, otherwise differencing must be implemented to make it stationary. This research precedes the data identification of ARIMA models through augmented Dickey-Fuller (ADF) testing [15]. Akaike information criterion (AIC) [1] criteria were employed to sift the optimal model out [20]. The results of model diagnosis reveal that the values of Q-statistic (i.e., LjungBox statistic) [32] are greater than 0.05 in ARIMA (2, 1, 2) model, in which the result is serial non-correlation (i.e., white noise) and has been suitably fitted. This study elaborates how data is entered into the RBF network for prediction through all algorithms, and then compare with the results from ARIMA models. The results of forecasting set for the exercise are shown in Figure 3. In addition, the RMSE, mean absolute error (MAE), and mean absolute percentage error (MAPE) are the most commonly used error measures in practice, and so were used to evaluate the forecast models [12]. Thus, the forecasting performances for all algorithms

INTEGRATION OF SOM NETWORK AND EVOLUTIONARY ALGORITHMS

1967

Figure 3. The comparison of forecasting results from the ISGP algorithm and ARIMA model in the oil price forecasting exercise Table 6. The errors comparison for all algorithms in the oil price forecasting exercise Error RMSE MAE MAPE (%) RBF network [11] 4.09E-1 4.07E-1 897.16 GA-based [58] 62.25E-12 56.41E-12 173E-10 PSO-based [22] 8.62E-12 6.43E-12 23.87E-10 HPG [36] 1.93E-12 5.78E-12 20.46E-10 ARIMA (2, 1, 2) model 2.9780 2.1570 5.1735 ISGP 1.69E-12 2.15E-12 4.22E-10 Algorithm

in this study alone with the data are presented in Table 6. It reveals that the ISGP algorithm has the most accurate forecast compared to other algorithms. 6. Conclusions. The proposed ISGP algorithm combining the automatically clustering ability of SOM network with the GA and PSO evolutionary algorithms provides the settings of RBF network parameters such as neuron, width and weight. With this algorithm, the GP algorithm improves the diversity of populations and also increases the precision of the results. The results from both benchmark problems and the oil price forecasting exercise have been compared with those obtained from the RBF network [11], GA-based [58], PSO-based [22], HPG [36] and the ARIMA (2, 1, 2) model algorithms, and the result of proposed ISGP algorithm significantly outperformed. In the future, the effort will be made to improve clustering for getting better initial activation functions for RBF network in order to speed up the training process. Also, other EAs, such as artificial immune system (AIS) and ant colony optimization (ACO) can be applied to further improve the accuracy.

1968

R. J. KUO, T.-L. HU AND Z.-Y. CHEN

REFERENCES [1] H. Akaike, A new look at the statistical model identification, IEEE Trans. on Automat. Control, vol.19, pp.716-723, 1974. [2] P. J. Angeline, Evolutionary optimization versus particle swarm optimization: Philosophy and performance differences, Evolutionary Programming, vol.7, pp.601-610, 1998. [3] G. Armano, M. Marchesi and A. Murru, A hybrid genetic-neural architecture for stock indexes forecasting, Information Sciences, vol.170, pp.3-33, 2005. [4] A. Azadeh and S. Tarverdian, Integration of genetic algorithm, computer simulation and design of experiments for forecasting electrical energy consumption, Energy Policy, vol.35, pp.5229-5241, 2007. [5] G. A. Barreto and A. F. R. Araujo, Identification and control of dynamical systems using the selforganizing map, IEEE Trans. on Neural Networks, vol.15, no.5, pp.1244-1259, 2004. [6] A. M. S. Barreto, H. J. C. Barbosa and N. F. F. Ebecken, GOLS – Genetic orthogonal least squares algorithm for training RBF networks, Neurocomputing, vol.69, pp.2041-2064, 2006. [7] G. E. P. Box and G. Jenkins, Time Series Analysis, Forecasting and Control, Holden-Day, San Francisco, 1976. [8] M. Casdagli, Nonlinear prediction of chaotic time series, Physica D, vol.35, pp.335-356, 1989. [9] H. B. Celikoglu and H. K. Cigizoglu, Modelling public transport trips by radial basis function neural networks, Mathematical and Computer Modelling, vol.45, pp.480-489, 2007. [10] C. C. H. Chan, Intelligent spider for information retrieval to support mining-based price prediction for online auctioning, Expert Systems with Application, vol.34, pp.347-356, 2008. [11] S. Chen, C. F. N. Cowan and P. M. Grant, Orthogonal least squares learning algorithm for radial basis function networks, IEEE Trans. on Neural Networks, vol.2, no.2, pp.302-309, 1991. [12] H. C. Co and R. Boosarawongse, Forecasting Thailand’s rice export: Statistical techniques vs. artificial neural networks, Computers & Industrial Engineering, vol.53, pp.610-627, 2007. [13] L. delaOssa, J. A. Gamez and J. M. Puetra, Learning weighted linguistic fuzzy rules with estimation of distribution algorithms, IEEE Congress on Evolutionary Computation, Vancouver, Canada, pp.900-907, 2006. [14] J. S. Denker, Neural network models of learning and adaptation, Physica D, vol.22, pp.216-232, 1986. [15] D. A. Dickey and W. A. Fuller, Likelihood ration statistics for autoregressive time series with a unit root, Econometrica, vol.49, no.4, pp.1057-1072, 1981. [16] P. Doganis, A. Alexandridis, P. Patrinos and H. Sarimveis, Time series sales forecasting for short shelf-life food products based on artificial neural networks and evolutionary computing, Journal of Food Engineering, vol.75, no.2, pp.196-204, 2006. [17] S. R. Dorling, R. J. Foxall, D. P. Mandic and G. C. Cawley, Maximum likelihood cost functions for neural network models of air quality data, Atmos. Environ., vol.37, pp.3435-3443, 2003. [18] R. O. Duda and P. E. Hart, Pattern Classification and Scene Analysis, John Wiley & Sons, New York, 1973. [19] E. Elbeltagi, T. Hegazy and D. Grierson, Comparison among five evolutionary-based optimization algorithms, Advanced Engineering Informatics, vol.19, pp.43-53, 2005. [20] R. F. Engle, F. Robert and B. S. Yoo, Forecasting and testing in cointegrated systems, Journal of Econometrics, vol.35, pp.588-589, 1987. [21] S. Fan and L. Chen, Short-term load forecasting based on an adaptive hybrid method, IEEE Trans. on Power Systems, vol.21, no.1, pp.392-401, 2006. [22] H. M. Feng, Self-generation RBFNs using evolutional PSO learning, Neurocomputing, vol.70, pp.241251, 2006. [23] M. Ghiassi, H. Saidane and D. K. Zimbra, A dynamic artificial neural network model for forecasting time series events, International Journal of Forecasting, vol.21, pp.341-362, 2005. [24] D. E. Goldberg, Genetic Algorithm in Search, Optimization & Machine Learning, Addison-Wesley, Reading, 1989. [25] G. H. Golub and C. F. V. Loan, Matrix Computations, 3rd Edition, Johns Hopkins Univ. Press, Baltimore, 1996. [26] S. U. Guan and F. Zhu, An incremental approach to genetic-algorithms-based classification, IEEE Trans. on Systems, Man, and Cybernetics – Part B: Cybernetics, vol.35, no.2, pp.227-239, 2005. [27] F. A. Guerra and L. D. S. Coelho, Multi-step ahead nonlinear identification of Lorenz’s chaotic system using radial basis neural network with learning by clustering and particle swarm optimization, Chaos, Solitons and Fractals, vol.35, pp.967-979, 2008.

INTEGRATION OF SOM NETWORK AND EVOLUTIONARY ALGORITHMS

1969

[28] Y. Guo, X. Cao and J. Zhang, Constraint handling based multiobjective evolutionary algorithm for aircraft landing scheduling, International Journal of Innovative Computing, Information and Control, vol.5, no.8, pp.2229-2238, 2009. [29] J. H. Holland, Adaptation in Natural and Artificial Systems, University of Michigan Press, Ann Arbor, 1975. [30] H. B. Hwang, Insights into neural-network forecasting time series corresponding to ARMA (p; q) structures, Omega, vol.29, pp.273-289, 2001. [31] J. Kennedy and R. C. Eberhart, Particle swarm optimization, Proc. of IEEE International Conference on Neural Networks, Perth, Australia, pp.1942-1948, 1995. [32] J. Kmenta, Elements of Econometrics, 2nd Edition, Macmillan Publishing Co., New York, 1986. [33] T. Kohonen, The self-organizing map, Proc. of IEEE, vol.78, no.9, pp.1464-1480, 1990. [34] M. Kolehmainen, H. Martikainen and J. Ruuskanen, Neural networks and periodic components used in air quality forecasting, Atmos. Environ., vol.35, pp.815-825, 2001. [35] R. J. Kuo, C. H. Chen and Y. C. Hwang, An intelligent stock trading decision support system through integration of genetic algorithm based fuzzy neural network and artificial neural network, Fuzzy Sets and Systems, vol.118, pp.21-45, 2001. [36] R. J. Kuo, T. L. Hu and Z. Y. Chen, Sales forecasting using an evolutionary algorithm based radial basis function neural network, Lecture Notes in Business Information Processing, vol.20, pp.65-74, 2009. [37] Z. J. Lee, A novel hybrid algorithm for function approximation, Expert Systems with Applications, vol.34, pp.384-390, 2008. [38] X. M. Li, R. B. Xiao, S. H. Yuan, J. A. Chen and J. X. Zhou, Urban total ecological footprint forecasting by using radial basis function neural network: A case study of Wuhan city, China, Ecological Indicators, vol.10, pp.241-248, 2010. [39] S. H. Liao and C. H. Wen, Artificial neural networks classification and clustering of methodologies and applications – Literature analysis from 1995 to 2005, Expert Systems with Applications, vol.32, pp.1-11, 2007. [40] F. Y. Lin and S. McClean, A data mining approach to the prediction of corporate failure, KnowledgeBased System, vol.14, pp.189-195, 2001. [41] C. Liu and Y. Wang, A new evolutionary algorithm for multi-objective optimization problems, ICIC Express Letters, vol.1, no.1, pp.93-96, 2007. [42] J. T. H. Lo, Multilayer perceptrons and radial basis functions are universal robust approximators, Proc. of Int. Joint Conf. on Neural Networks, Anchorage, AK, pp.1311-1314, 1998. [43] C. G. Looney, Advances in feed-forward neural networks: Demystifying knowledge acquiring black boxes, IEEE Trans. on Knowledge Data Eng., vol.8, no.2, pp.211-226, 1996. [44] K. Meng, Z. Y. Dong and K. P. Wong, Self-adaptive radial basis function neural network for shortterm electricity price forecasting, IET Gener. Transm. Distrib., vol.3, no.4, pp.325-335, 2009. [45] Z. Michalewicz, Genetic Algorithms + Data Structures = Evolution Programs, Springer-Verlag, New York, 1996. [46] S. Mitra and Y. Hayashi, Bioinformatics with soft computing, IEEE Trans. on Systems, Man, and Cybernetics – Part C: Applications and Reviews, vol.36, no.5, pp.616-635, 2006. [47] T. Miyano, S. Kimoto, H. Shibuta, K. Nakashima, Y. Ikenaga and K. Aihara, Time series analysis and prediction on complex dynamical behavior observed in a blast furnace, Physica D, vol.135, pp.305-330, 2000. [48] Y. Mu and A. Sheng, Evolutionary diagonal recurrent neural network with improved hybrid EPPSO algorithm and its identification application, International Journal of Innovative Computing, Information and Control, vol.5, no.6, pp.1615-1624, 2009. [49] N. Noman and H. Iba, Accelerating differential evolution using an adaptive local search, IEEE Trans. on Evolutionary Computation, vol.12, no.1, pp.107-125, 2008. [50] A. E. Omidvar, Configuring radial basis function network using fractal scaling process with application to chaotic time series prediction, Chaos, Solitons and Fractals, vol.22, pp.757-766, 2004. [51] C. S. Ong, J. J. Huang and G. H. Tzeng, Model identification of ARIMA family using genetic algorithm, Applied Mathematics and Computation, vol.164, pp.885-912, 2005. [52] J. B. Ordieres, E. P. Vergara, R. S. Capuz and R. E. Salazar, Neural network prediction model for fine particulate matter (PM2.5 ) on the US-Mexico border in El Paso (Texas) and Ciudad Juarez (Chihuahua), Environmental Modelling and Software, vol.20, pp.547-559, 2005. [53] T. Poggio and F. Girosi, A Theory of Networks for Approximation and Learning, AI memo/CBIP Paper 31, Massachusetts Institute of Technology, 1989.

1970

R. J. KUO, T.-L. HU AND Z.-Y. CHEN

[54] V. R. Prybutok, J. Yi and D. Mitchell, Comparison of neural network models with ARIMA and regression models for prediction of Houston’s daily maximum ozone concentrations, European Journal of Operational Research, vol.122, pp.31-40, 2000. [55] S. Robeson and D. Steyn, Evaluation and comparison of statistical forecast models for daily maximum ozone concentrations, Atmos. Environ., vol.24, pp.303-312, 1990. [56] I. Rojas, O. Valenzuela, F. Rojas et al., Soft-computing techniques and ARMA model for time series prediction, Neurocomputing, vol.71, pp.519-537, 2008. [57] E. Salajegheh and S. Gholizadeh, Optimum design of structures by an improved genetic algorithm using neural networks, Advances in Engineering Software, vol.36, pp.757-767, 2005. [58] H. Sarimveis, A. Alexandridis, S. Mazarakis and G. Bafas, A new algorithm for developing dynamic radial basis function neural network models based on genetic algorithms, Computers and Chemical Engineering, vol.28, pp.209-217, 2004. [59] U. Schlink, S. Dorling, E. Pelikan et al., A rigorous inter-comparison of ground-level ozone predictions, Atmos. Environ., vol.37, pp.3237-3253, 2003. [60] P. S. Shelokar, P. Siarry, V. K. Jayaraman and B. D. Kulkarni, Particle swarm and colony algorithms hybridized for improved continuous optimization, Applied Mathematics and Computation, vol.188, pp.129-142, 2007. [61] G. Syswerda, Uniform crossover in genetic algorithms, Proc. of the 3rd International Conference on Genetic Algorithms and Their Applications, San Mateo, CA, pp.2-9, 1989. [62] M. Tripathy, R. P. Maheshwari and H. K. Verma, Radial basis probabilistic neural network for differential protection of power transformer, IET Gener. Transm. Distrib., vol.2, no.1, pp.43-52, 2008. [63] J. T. Tsai, J. H. Chou and T. K. Liu, Tuning the structure and parameters of a neural network by using hybrid Taguchi-Genetic algorithm, IEEE Trans. on Neural Networks, vol.17, no.1, pp.69-80, 2006. [64] Z. Wang and M. Li, A hybrid coevolutionary algorithm for learning classification rules set, ICIC Express Letters, vol.4, no.2, pp.401-406, 2010. [65] B. A. Whitehead and T. D. Choate, Cooperative-competitive genetic evolution of radial basis function centers and widths for time series prediction, IEEE Trans. on Neural Networks, vol.7, no.4, pp.869-880, 1996. [66] R. Xu, G. K. Venayagamoorthy and D. C. Wunsch II, Modeling of gene regulatory networks with hybrid differential evolution and particle swarm optimization, Neural Networks, vol.20, no.8, pp.917927, 2007. [67] K. Xu, M. Xie, L. C. Tang and S. L. Ho, Application of neural networks in forecasting engine systems reliability, Applied Soft Computing, vol.2, pp.255-268, 2003. [68] W. Yan, C. H. Chen, Y. Huang and W. Mi, An integration of bidding-oriented product conceptualization and supply chain formation, Computers in Industry, vol.59, pp.128-144, 2008. [69] L. Yu, K. K. Lai and S. Wang, Multistage RBF neural network ensemble learning for exchange rates forecasting, Neurocomputing, vol.71, pp.3295-3302, 2008. [70] R. Zemouri, D. Racoceanu and N. Zerhouni, Recurrent radial basis function network for time-series prediction, Artificial Intelligence, vol.16, pp.453-463, 2003. [71] G. P. Zhang, Time series forecasting using a hybrid ARIMA and neural network model, Neurocomputing, vol.50, pp.159-175, 2003. [72] G. P. Zhang, E. B. Patuwo and M. Y. Hu, A simulation study of artificial neural networks for nonlinear time-series forecasting, Comput. Oper. Res., vol.28, pp.381-396, 2001. [73] M. Zhong, P. Lingras and S. Sharma, Estimation of missing traffic counts using factor, genetic, neural, and regression techniques, Transportation Research Part C, vol.12, no.2, pp.139-166, 2004. [74] H. F. Zou, G. P. Xia, F. T. Yang and H. Y. Wang, An investigation and comparison of artificial neural network and time series models for Chinese food grain price forecasting, Neurocomputing, vol.70, pp.2913-2923, 2007.