Comparing global models of neural networks,

Applied mathematics in Engineering, Management and Technology 1 (1) 2013:66-71 www.amiemt-journal.com

Comparing global models of neural networks, feedback and piecewise uniform for predicting yields in Tehran Stock Exchange Maryam Shabania*, MahsaPezeshkib, Samira Parsaiyanc, Mina Danaied a

Department of Industrial Engineering, Faculty of Industrial and Mechanical Engineering, Qazvin Branch, Islamic Azad University, Qazvin, Iran. b Department of Industrial Engineering, Faculty of Industrial and Mechanical Engineering, Qazvin Branch, Islamic Azad University, Qazvin, Iran. c Department of Industrial Management, Faculty of Management & Accounting, AllamehTabataba’i University, Tehran, Iran. d Department of Financial Engineering, Faculty of Industrial Engineering, University of Science & Culture, Tehran, Iran. *Corresponding author: [email protected]

Abstract This paper makes a comparison of global, feedback and smoothed-piecewise neural prediction models for financialtime series (FTS) prediction problem. Each model is implemented by various neural network (NN) architectures: global model by a multilayer perceptron (MLP), feedback model by a recurrent neural network (RNN) and smoothed-piecewise model by a mixture of experts (MoE) structure. The advantages and disadvantages of each model are discussed by using real Tehran exchange return data: 3 continuous years data of Tehran stock exchange (TSE) index (TEDIX) that collected from Tehran stock exchange corporation site. The comparison for each model is done based on well-known criterion mean squared error (MSE). Finally, it is observed that the smoothed-piecewise neural model have the best prediction among the other models and gives the outputs more near than real data. JEL Code: C45 Keywords: Financial time series (FTS) prediction; Global; Feedback; Smoothed-piecewise neural models.

1 Introduction: Artificial neural networks, which its initial concepts were introduced 40s A.D, is a network with interrelated elements. These elements are inspired through studies in the field of biological neural systems. The goal of neural networks is attempt to build machines that act like the human brain or in other words they have the ability to learn. In addition to classic models for neural networks, recently combination models become more practical in the field of machine learning researches. Totally, in the fields of classification, clustering, regression and etc. time series issue have been considered.Affordable tries in this filed is including of: support vector machines (SVM)Bayesian networks. ،mixture of experts ، ensembles of neural networks ،fuzzy models. The most famous people that had studies in this field were, Shafer and Vovk, 2001; Petridis and Kehagias, 1998; Petridis et al., 2001; Rao et al., 1997; Castillo and Melin, 2002; Shawe-Taylor and Crisitianini, 2004; Heckerman, 1999. This article has been taken from the research that has been done with the same title and purpose in Turkey, in this research it is tried to compare these models using data taken from Iran stock and to compare the obtained results with Turkey. Therefore, the present article research the use of world neural models, feedbacks and unformed piecewise in financial and time Series prediction and is a sample of choosing proper model for the specific use of time series prediction. We use a MLP as an overall predictor of financial and time series which uses taught databases obtained from the local parts of the time series. RNN network which uses a feedback model using a memory between local parts of time series To summarize the local series and then a MOE, finally, uniformed piecewise predictor which is smoothed from a general prediction structure is obtained through conditional probability values of each series or local experts. We attempt to have a comparison between known neural models and Tehran Stock Exchange data.

66

Applied mathematics in Engineering, Management and Technology 2013 M. Shabani et al 2 Research data: Index of yield of the Tehran Stock Exchange, which is the weighted average of yields of all stock in the stock market, has been used daily for three consecutive years; since comparing three described models in Tehran Stock Exchange is the goal of this assessing; which include 819 kinds of data from the beginning of 2009 to the end of November 2012 that has been extracted from the stock website. In order to avoid problems and sudden changes that happen politically in the statistics of stock indexes in the Tehran Stock Exchange resulted from the presidential election, we have used data from the last the last three years.

3 Predictive models: Generally, the problem of time series prediction will be solved by coordinating input and output with a model for the given data and an estimated mapping function. Thus, the model of basic patterns suggests trends and cycles which use typical and historical observed data. A time series xt, t = 1, 2, 3, ... for convenience xt is scalar, but a vector time series is used, too. Among the several predictors of k predictor is derived from the following path. ), ≅ = ( , ,… , ; = 1,2,3, … , (1) Real amount of xt as yt will be predict by predictive mistake of f(.,w) ، the predictor k belongs to which w is a parametric Vector can be obtained by putting w=w .its reasonable to assume ,if k th Predictive has Independent distributed random variables,) iid With zero mean and variance , the − = error amount of M is as Prediction horizon that is expected it would be as large as an amount that can be known by The prediction error is kept within reasonable area.Predictive regression linear (LP) as a predictor of input units (2) + ⋯+ + + + = And predictors with multiple entries such as the following two entries will be determined by : + + ⋯+ + + + + ,+ = (3) Training predictive means to obtain the coefficients a, b that is done by using least squares estimation.Lp has special limitation in time series withnon-static nature.Polynomial Predictor (PP) is polynomials of the t variables and nth order of Polynomial Predictors are shown as below + ⋯+ (4) + + + = In this context, as well as learning to calculate an index for each group occurs by the method of least squares regression.PP has some limitation is capturing the main process of general series. For example when the series getting longer ,the order of n may be getting longer and prediction getting harder as well.. )(5) Neural predictors are essentially the same linear predictors. The form: = ( , ,… , ; ) Or in two types of two varibles: = ( , ,… , ; , ,… , ; (6) Here wk is amatrix of weights or parameters .in the neural networks, learning algorithms are used to determine the parameters.

4 Financial time series prediction In general, the prediction of financial time series (Shafer and Vovk 2001; Petridis and Kehagias, 1998; Petridis et al., 2001) is a problem of hidden variables and the lack of available data to determine if there is underlying structure of a set of series. There are two popular approaches to the prediction of financial markets. A One of them is efficient financial market hypothesis in which current market price reflects all available data quickly. People believe they do not find evidence to forecast stock markets in the efficient market hypothesis. Another approach that has been adopted widely among traders, is the believing of predictably through fundamental analysis and forecasting method. In this study, we examine the prediction by assumption of the second approach which is the opposite efficient market hypothesis. Furthermore, we show using and comparing the neural predictors in the prediction of financial time series.in summary, variance id the criteria of finding changes in Assets returnin (Tino et al., 2000; Nelson, 1991; Engle and Ng, 1993; Black, 1976). Accurate prediction of the

67

Applied mathematics in Engineering, Management and Technology 2013 201 M. Shabani et al future stock needs to predicting variance of asset returns financial time series of stock revenues show the time timedependent variance which makes us to predict the distribution. Predicting a usance distribution may be used use on market risk management, portfolio selection, appropriate timing of market entry and exit, etc. Market risk management is critical in financial decisions. None of the players want the market which is not steady and manageable.VaR (Value at Risk), is a standard in the risk market management. Estimation of VaR is based on the prediction of distribution of risk factors in market.Consequently, estimating the usance distribution which is the primary risk factor for the stock market provides valuable informat information ion for risk in the market and this will force players to understand the high or low distribution which is expected in the market. We normalized the data of nervous predictive models, and by using logarithmic operators, we transmit price index series y to sequential compound efficient series . Normalizing process holds series price range steady. Price series that are unsteady and consistent with trends, cycles and seasonal repetitions, become sequential compound efficient series for obtaining acceptable constant series by using the formula given in equation number 7. ) (7) = ln( / Efficiency series have fixed boundaries even when we consider data of many years, but prices are very changeable and the comparison between assets ets by using efficiency series is more accurate that using price series. In this study, since utilized data are changing very few, there is no need to normalized the data.

5 Describing global models, feedback and smoothed piecewise In this study, we resolved neural predicting models of world, feedback and piecewise, for an instance we interpret financial time series of Tehran Stock Exchange. Neural predicting model can be explained as follows: Giving a set of input-output pairs = {( , ))} in which ε and ε are unknown from two distributions, path design f: ⇒ which minimize expected prediction error, in the form of squared errors which is explained E − ( , ) . The function ( , ) defines the response path for predictor. dictor. Amount of m about time series is 1. Global neural models, like Multi Multi-layer layer feed forward perceptron (figure 1) is used and then predict sample function . During implementation, we have selected conventional sigmoid function and we train weights of w from the function ( , )with Back Back-propagation propagation algorithm. Feedback neural models such as returnee neural network (RNN) (Figure 2) have returnee communications which is appropriate for modeling of temporary communications in time. Unlike Multilayer fee feed d forward Perceptron network, RNN introduces valuable bases for time series modeling. We implemented returnee neural network (Elman) for Recurrent Learning Algorithm in our studies. RNN Elman saves the values of hidden layer units (or internal components) components and returns them to the network again. Minimizing errors in this algorithm is done by measurement in units of the output sensitivity k in time T relatively small changes in the amount w (From hidden unit j to input unit i). The impact of change is considered idered in weight and distribute to whole network during time steps.

68

Applied mathematics in Engineering, Management and Technology 2013 M. Shabani et al

Totally, it can be stated that Elman networks are introduced as a type of returnee for dynamic systems modeling. Elman network is a three-layered network which only its middle layer is returnable. One of the advantages of such network in comparison with complete returnee networks is the possibility of using backpropagation for its training. It is because that communications to context layer or memory are constant; therefore context layers perform by delay with a sampling period. As an interpretation about these networks we could say that the inputs of hidden layer show the status of the network. Network outputs are a function of current status, previous status (which are provided by context layer) and current inputs. It means, when set of inputs is applied to the network, the network can be trained to provide proper outputs in mirror of previous status. Piecewise Unformed Neural model, such as MoE (Rao et al., 1998) (figure 3) is made from local expert systems with predictive function ( , ), in which w is a set of model parameters for local model j. Totally, local expert systems may be fixed, linear or non-linear function (polynomial with any degree) from X. total prediction function model is defined through below conditional statement: ( , )=∑ [ / ] , (8) in which [ / ] non-negative weightt is from relation between x and local model or expert system j and determine the degree that the expert system j participates in total output model. Sometimes these weightts are called input ties and have limitations [ / ] = 1 that parametric function is determined by w parameter set. The statistical model is expressed as follows: input-output pairs (x , y ) are determined randomly among input mass and through local model from the cumulative probability function P[j/x]. For the selected local model k, the output is set as random variable and based on algorithm type and its neural network which its

69

Applied mathematics in Engineering, Management and Technology 2013 M. Shabani et al average is , . By this description, ( , ) shows the expected output for given input x. the main advantage is that it improves minimizing squared errors in uniformed piecewise neural structure predict participation between output ( ) and predictor , while each independent predictor improves single predict between out put and each predictor , . Normalized structure, one hidden layered with RBF (Figures 1 and 3) (Petridis and Kehadias 1998) is the first example of uniformed piecewise neural structure in which we calculate the outputs of hidden layer on = exp − |x − m | /2σ that generally it is used functions by Gaussian basis with mean vectors m (Centers of the input space) and variance σ .K is the index for the specified node and the input space is covered by all nodes in the hidden layer structure.The second layer is a linear combination of all nodes in the hidden layer, which may interpret as a local expert system , and Output node weights may show the relation of { [ / ]} with responsive Local expert systems.

MoE is as a combing way of several learner of general structure.The structure is composed of two parts.The first part is a number of parallel local expert systems which is using the same input models. And the second part is the input Expert System which determines output by decisions of expert systems relevant to output pattern. Its suggested that, (figure 3) can predict every data with the possibility of local models by special exploration predictive MoE. , : As well as set of parameters of optimal model of {P[jl/x]} MMSE cost is selected for error optimization. Each selection from random branch of output tie leads us to one of the local expert systems. Totally piecewise models match inputs with local areas of the input space and attach local parameters with a sense of capture or division. Therefore, piecewise solutions which are created by singular parameters from singular sub-models are very easy to interpret. Such neural network can be considered to be trained as an example in which each one of single neural network units to meet solutions for local input areas. A uniformed piecewise model divides regression problem into learning sets of local models (expert system) but none of them claim area’s exclusive membership unlike piecewise models. By using [ / ] weights which are limited to{0,1}, the parameters would be added only when they need match improvements in local area. Therefore, the general issue will be simplified into learning statements and modeling.

6 Implementation and performances

70

Applied mathematics in Engineering, Management and Technology 2013 M. Shabani et al As it has already been stated, in order to implement and run the program, we used 819 data as system’s input data. From this 819 data, we selected 65% of these data as training data, it means 500 data. We have selected 7 data of input data for each network during training stage which is explainer of 7 days a week’s sequential data, and our goal in each one of the networks is to obtain the eighth day by using predictor models. The number of hidden layers is set 10 layers by using try and error in written algorithm via MATLAB software and the number of local experts in MOE model is set 9. Minimum square error was considered as the criteria used to measure network performance and the final comparison of their performance.

7 Conclusion The outcomes of written program run were very interesting for models in MATLAB software, since the total and final result was completely equal with the final result obtained from samples tested in the Turkish stock. As mentioned in reference article, and it was concluded from the criteria used to interpret and evaluate network performance, that uniformed piecewise model used from local experts, has better predictive power and the least average of squared error. Also in this study, after running program it was determined that the uniformed piecewise model has the best performance in prediction of Tehran Stock Exchange index yields and has the least squared error. However, the study of only two world Stock Exchanges seems to be inadequate for a comprehensive and definitive comment about the performance of the models but through these results and also thinking about the logic of unformed Piecewise, we can be hopeful that by studying and comparing these three models in other stock markets and predicting financial time series, most of the time uniformed piecewise approach would be the best one for prediction. References Atsalakis George s.; Valavaniskimon p.; (2008). Surveying stock market forcastingthechniques-part II: soft computing methods. Expert Systems with Applications. Devadoss, A.victo; Alphonnse, T.Antony; (2013). Stock Prediction Using Artificial Neural Networks; International Journal of Data Mining Techniques and Applications, 283-291 Ghazanfari, Mehdi, orkutJamal, (2004).Neural Networks, Principles and Functions. center of Iran University of Science and Technology. Yumlu, serdar; Gurgenfikret S.; Okay, nesrin;(2005).a comparison of global, Recurrent and smoothed –piecewise neural models for Istanbul stock exchange (ISE) prediction. Pattern Recognition Letters, 26, 2093–2103

71