Financial Forecasting Using Pattern Modeling and Recognition System Based on Kernel Regression Defu Zhang1,2, Yubao Liu3 and Yi Jiang1 1 Department of Computer Science, Xiamen University, 361005, China 2 Longtop Group Post-doctoral Research Center, Xiamen, 361005, China 3 School of Information Science and Technology, Sun Yat-sen University, 510275, China
[email protected] http://xmuzdf.ik8.com Abstract: - The increased popularity of financial time series forecasting in recent times lies to its great importance in predicting the best stock market timing. In this paper, we develop the concept of a pattern modeling and recognition system for predicting future behavior of time series using local approximation. In order to improve the performance of this system, we propose a systematic and automatic approach to technical pattern recognition using nonparametric kernel regression, and use this method for filtering the noise of the time series. The computational results on the well-known stock market indices reveal that kernel regression is an important tool for improving the performance of the proposed forecasting system, and the performance of the improved method can outperform the performance of advanced methods such as neural networks.
Key-Words: - financial time series; forecasting; pattern modeling and recognition system; kernel regression
1 Introduction Forecasting becomes more and more important in many domains. It has been one of the greatest challenges to predict the stock market. However, determining stock market timing, that is when to buy and sell, is a very difficult problem for humans because of the complexity of the stock market. The traders and investors in the stock market have come to need powerful assistants in their decisions making. Throughout the literature, many techniques have been implemented to perform time series forecasting. Artificial intelligence methods such as neural network [1], genetic algorithms [2], Markov models [3] and fuzzy methods [2] have been frequently used. Pattern modeling and recognition system (PMRS)[4] which was developed recently has received special interest. The reasons for its popularity are as follows [4]: 1) its algorithm is easy; 2) such a tool requires lesser number of parameters that need optimization; 3) PMRS works in real-time. However, there is a fatal shortcoming in this technique. When used to dealing with noise data, its performance usually discourages people. In this paper, we hope to realize the improvement of this forecasting method. Our goal is to identify regularities in the time series of prices by extracting nonlinear pattern from noisy data. Implicit in this goal is the recognition that some price movements are significant, they contribute to the formation of a specific pattern, and others are merely random
fluctuations to be ignored. Fortunately, there are a class of statistical estimators, called smoothing estimators [5], are ideally suited to this task because they extract nonlinear relations by “averaging out” the noise. The kernel smoothing estimator has been used to improve the performance of neural networks in [6]. Therefore, we propose to use these estimators to improve the performance of PMRS. We then study the usefulness of noise-filtering technique for improving the performance of the tool. The computational results show that PMRS based on kernel regression are highly accurate and outperform the performance of advanced methods such as neural networks.
2 PMRS From statistics to artificial intelligence, there are myriad choices of techniques for financial time series forecasting. One of the simplest techniques is to search a data series for similar past events and use the matches to make a forecast. PMRS is an example of this technique. This pattern recognition method is based on the philosophy that it is easier to approximate the chaotic behavior of time series at the local than global level [7]. The algorithm of PMRS is penetrable and easy to program into a computer as well. However, the prediction results are not always satisfying. People who would like to profit from time series forecasting can often be frustrated by the low
direction % success. In a sense, it’s somewhat like a voodoo. One explanation for this frustrating situation is that the stock prices are arbitrary, so the time series may contain random fluctuations which are not components of a specific pattern. In the next chapter, we will propose to apply a statistical method to filter noises.
3 Noise-filtering—Kernel Regression Although PMRS fails to identify regularities in the time series of prices by extracting nonlinear patterns from noisy data, a class of smoothing estimators is fit to this task. By “averaging out” the noise, they can extract nonlinear relations from time series. We choose one kind of smoothing estimators, namely nonparametric kernel regression to filter the noise in the time series. Kernel regression is one class of modeling methods that belongs to the smoothing methods family. It is a part of the nonparametric regression methods. Kernel regression allows you to base the prediction of a value on passed observations, and to weight the impact of passed observations depending on how similar they are compared with the current values of the explanatory variables. We will use the following formula [8] to filter noise.
1 T m ( x) = ∑ ω t ,h ( x)Yt = T t =1 ' h
K h ( x) =
1 h 2π
−
e
∑ ∑ T
t =1 T
K h ( x − X t )Yt
t =1
x
Kh (x − X t )
2
2h2
where h is the bandwidth that is an important aspect of any local-averaging technique, { Yt } is a time series, X t denotes the distance far from x ,
mh′ (x) is the smooth value of { Yt } at time x .
one data point. To forecast the next data point of current states, we need to smooth the time series before the current data point in the time window. When the current states move one data point forward, the smoothing process should be repeated, all data points in the time window besides the current data point should be treated with again, for some treated data points will be modified with the sliding of the time window. To deal with one data point yi , or filter the noise, we consider 80 data points ahead and at the back for reference. According to the weighted average principle, the closer the data point is to yi , the larger the weights it would have, we neglect the effect of other data points beyond [ y i − 80, y i + 80] within the time widow. In order to program the principles of PMRS into a computer, we need to define the mathematical representation of time series data for our purpose. Providing a time series is represented as a vector, y = { y1 , y 2 ,L , y n } , then the state of the time series represents its current value y n . The prediction process can be described as follows: first identifying the closest neighbor of y n in the past data, say y j , then predict yˆ n +1 on the basis of y j +1 . This approach may be extended, i.e. the current state of a time series may be extended to include more than one value. For example, take values of size three, the current state may be defined as { y n − 2 , y n −1 , y n } . For such a current state, the prediction will depend on the past state { y j − 2 , y j −1 , y j } . The optimal state size must be determined experimentally on the basis of achieving minimal errors on standard measures. A segment in the series may be defined as a different vector δ = (δ 1 , δ 2 , L, δ n −1 ) where
δ i = yi +1 − yi . A pattern includes one or more
segments
4 PMRS Based on Kernel Regression As we discuss before, noise seriously influences the performance of PMRS. Now we apply kernel regression to reduce the impact of noise. To illustrate the noise-filtering procedure, we need to describe the forecasting process at the same time. First we set a fixed length of time series, called time window. We will search for the similar past states to match the current states in the time window. As the current states move forward by one data point, the time window is also slide forward by
and
may
be
visualized
as
p = (δ i , δ i +1 , L , δ h ) for given values of i and h , 1 ≤ i, h ≤ n − 1 , provided that h > i . The technique
of matching structural primitives is based on the premise that the past repeats itself [4]. In detail, the algorithm of PMRS based on kernel smoothing can be described as follows: Step 0: Process data using kernel smoothing. Step 1: Define a pattern of size as k , here k =2 as the beginning value of k .
pattern of size k : p = (δ n − k , δ n − ( k −1) , L , δ n −1 ) at the end of the time
Step
2:
series. Step
Choose
3:
a
A
nearest neighbor p ′ = (δ j − k , δ j − ( k −1) , L , δ j −1 ) of pattern p is
determined from historical data on the basis of smallest offset ▽. Here, j is the marker position. Step 4: If δ j > 0 , it denotes to predict high, then predict value yˆ n +1 can be calculated as follows:
yˆ n +1 = y n + βδ j , here, β =
1 k δ n −i ∑ δ j −i ; If k i =1
δ j < 0 , it denotes to predict low, then predict value yˆ n +1 can be calculated as follows: yˆ n +1 = y n − βδ j , 1 k δ n −i ∑ δ j −i ; If δ j = 0 , it denotes to k i =1 predict same, then predict value yˆ n +1 = y n . here, β =
Step 5: We use the standard error measurement PMSE for calculating the accuracy of the forecast and record the error to select optimal k . Step 6: Increase k by one, and repeat the process from step 2 to step 5 until it reaches a predefined maximum allowed for the experiment (here is 5). Step 7: Choose a pattern size k which yields minimal error measurements. Step 8: Forecasting. Then apply this pattern size
k to forecasting according to the principle of moving time widow.
5 Experimental Results In this section, the performance of the pattern modeling and recognition system based on kernel regression (PMRSKR) is compared with PMRS, BPNN. BPNN is a Feedforward Backpropagation Network with three layer structure. In this paper, BPNN has 20 neurons in the input layer, 5 neurons in the hidden layer and one neuron in the output layer, f is typically taken to be a sigmoidal function, such as the logistic function
1 f ( x) = . The backpropagation algorithm 1 + e−x with momentum factor is adopted to perform steepest descent on the total mean squared error. The more detailed description about BPNN can be found in [9]. PMRSKR, PMRS and BPNN are implemented through an object oriented VC programming tool. The test data sets used for prediction are Shanghai
Composite Index (SSEC) and S&P 500 Index (SPX), HANG SENG Index (HSI) and NASDAQ Combined Composite Index (CCMP) from January 1993 to December 2003. We have tried to do experiment with as long time series as possible in order to eliminate the likelihood of luck. The overall procedure consists of dividing the data into two parts: estimation data and test data. The estimation data is to determine the optimal k. 16% of the total data, about 500 data is the estimation data, and the remaining 84% is test data. The evaluation criteria are the absolute error and the direction of series change: positive or negative relative to the current position. The latter is one of the most important measures in financial markets. The computational results are reported in Table 1~4. The forecasting results of PMRS and BPNN really make us frustrate, the direction % success of them are less than 52% from Table 1~4. Table 1. The prediction performance of three methods for SSEC stock market. Method Optimal k % success Absolute error PMRSKR 4 68% 15.65 PMRS 5 49% 20.84 BPNN / 51% 26.96 Table 2. The prediction performance of three methods for SPX stock market. Method Optimal k % success Absolute error PMRSKR 3 67% 55.53 PMRS 3 50% 24.86 BPNN / 49% 229.31 Table 3. The prediction performance of three methods for HSI stock market. Method Optimal k % success Absolute error PMRSKR 3 70% 159.63 PMRS 4 52% 668.12 BPNN / 51% 163.47 Table 4. The prediction performance of three methods for CCMP stock market. Method Optimal k % success Absolute error PMRSKR 3 69% 26.67 PMRS 2 52% 6.93 BPNN / 50% 45.34 It is known that if the hit ratio of the direction is highly above 51%, the technique for forecasting can
then be regarded as useful. Obviously, the performance of PMRS could not achieve this goal in dealing with these indices in the well-known stock markets. Also, the performance of neural networks (BPNN) is not good with our test data. By using kernel regression for filtering noise, the performance of PMRS is greatly improved and reaches 67~70% in % success. It means that investment according to this technique can result in a good profit. Note that h is an important parameter of kernel regression. If h is very small, the averaging will be done with respect to a rather small neighborhood around each data point. If h is too large, the averaging will be over larger neighborhoods of a data point. Therefore, controlling the degree of averaging amounts to adjusting the smoothing parameter h, also known as the bandwidth. In our experiment, the appropriate h is 2.7 for all indices. Although the absolute error of PMRSKR is not minimal, its direction % success is highest. In fact, the latter is most important for make a good profit. Therefore, throughout the ten years of test data, the usefulness of this improved technique is proven. In addition, the comparisons of the predict value and the real value about twenty days among PMRS, PMRSKR, BPNN are reported in Fig. 1. From Fig.1, we can observe the predict value drops behind real value.
6 Conclusion In this study, we develop PMRS based on kernel regression for predicting future behavior of dynamic time series using local approximation. The experimental results are highly encouraging. Thus, we could come to the conclusion that the kernel regression is a very useful preprocessing stage in making accurate forecasts, and this improved PMRS can be an efficient tool for stock market timing. We expect that this technology will be further
developed and realized by integrating the trading strategy for decision-making.
Acknowledgments This work has been supported by academician startup fund (Grant No. X01109), Xiamen University 985 information technology fund (Grant No. 0000X07204) and fund (Grant No. Y07025). References: [1] Azoff, M. E, Neural network time series forecasting of financial markets, John Wiley and Sons, 1994. [2] Chorafas, D. N, Chaos theory in the financial markets: Applying fractals, fuzzy logic, ge-netic algorithms, Swarm simulation & the Monte Carlo method to manage markets, Probus Publishing Company,1994. [3] MacDonald, I. L. and Zucchini, W, Hidden Markov and other models for discrete valued time series, London: Chapman and Hall, 1997. [4] Singh, S, A Long Memory Pattern Modeling and Recognition System for Financial Forecasting, Pattern Analysis and Applications, vol. 2, issue 3 ,1999, 264–273. [5] Hardle, Wolfgang, Applied Nonparametric Regression, Cambridge University Press, Cambridge, UK, 1990. [6] Defu Zhang, Qingshan Jiang, and Xin Li, A Hybrid Mining Model Based on Neural Network and Kernel Smoothing Technique, Lecture Notes in Computer Science, 3516, 2005, 801–805. [7] Farmer, J. D. and Sidorowich, J. J, Predicting chaotic dynamics, in Dynamic patterns in complex systems, J.A.S. Kelso. A. J. Mandell and M. F. Shlesinger(Eds.), Singapore: World Scientific, 1988, 265–292. [8] Andrew W. Lo, Harry Mamaysky, Jiang Wang, Foundations of Technical Analysis: Computational Algorithms, Statistical Inference, and Empirical Implementation. http://www.nber. org/papers/w7613. National Bureau of Economic Research, 2000. [9] Rao, V. and Rao H, C++ Neural Network and Fuzzy Logic, MIS press, New York, 1995.
