Can we predict the unpredictable?

Supplementary Information for

Can we predict the unpredictable? Abbas Golestani 1 , Robin Gras1,2,3

1

School of Computer Science, University of Windsor, ON N9B 3P4, Canada.

2

Department of Biology, University of Windsor, ON N9B 3P4, Canada.

3

Great Lakes Institutes for Environmental Research, University of Windsor, ON N9B 3P4,

Canada.

1. Prediction Methods 1.1. ARIMA An autoregressive integrated moving average (ARIMA) model is a generalization of an autoregressive moving average (ARMA) model. These models are fitted to time series data either to better understand the data or to predict future points in the series forecasting18. They are applied in some cases where data show evidence of non-stationarity, where an initial differencing step (corresponding to the "integrated" part of the model) can be applied to remove the nonstationarity. The model is generally referred to as an ARIMA(p,d,q) model where parameters p, d, and q are non-negative integers that refer to the order of the autoregressive, integrated, and moving average parts of the model respectively40.

The ARMA models, as described in (Box, 1994), provide a parsimonious description of a stationary stochastic process in terms of two polynomials, one for the auto-regression and the second for the moving average18. Given a time series of data Xt, the ARMA model is a method for predicting future values in this series. The model consists of two parts, an autoregressive (AR) part and a moving average (MA) part. The model is usually then referred to as the ARMA(p,q) model where p is the order of the autoregressive part and q is the order of the moving average part (as defined below). The notation AR(p) refers to the autoregressive model of order p. The AR(p) model is written p

X t = c + ∑ ϕi X t −i + ε t

(1)

i =1

ϕ ,...,ϕ p where 1 are parameters, c is a constant, and the random variable

ε t is white noise.

The notation MA(q) refers to the moving average model of order q: q

X t = µ + ε t + ∑ θ iε t −i i =1

(2)

where the θ1, ..., θq are the parameters of the model, µ is the expectation of X t (often assumed

ε to equal 0), and the ε t , t −1 ,... are again, white noise error terms. The stationary series Yt is said to be ARMA(p,q) if: Yt − φ1Yt −1 − ... − φ pYt − p = ε t + θ1ε t −1 + ... + θ q ε t − q

(3)

where ε t is white noise and there is no common factor between autoregressive polynomial, (

1 − φ1 L − φ2 L2 − ... − φ p Lp ), and moving average polynomial, ( 1 + θ1 L + ... + θ q Lq ), where L is a lag operator. Also, these polynomials can be represented by φ ( L ) and θ ( L ) , respectively:

φ ( L )Yt = θ ( L )ε t

(4)

If the series is difference-stationary, the integrated autoregressive moving average (ARIMA) model is implemented. The series Yt is said to be ARIMA(p,d,q) if

φ ( L )(1 − L ) d Yt = θ ( L )ε t

(5)

where d is the dth difference operator. In this paper, we used different configurations that have been reported as a good candidate for making good prediction such as: ARIMA(4,1,5), ARIMA(3,3,3), ARIMA(2,1,1), ARIMA(1,1,1), ARIMA(5,2,11), ARIMA(2,1,0), ARIMA(3,1,4). We used the one with best results which was ARIMA(4,1,5). 1.2. ARCH/GARCH Models A key feature of financial time series is that large (small) absolute returns tend to be followed by large (small) absolute returns, where there are periods, which display high (low) fluctuation. The class of autoregressive conditional heteroscedastic (ARCH) models, introduced by Engle (1982)41. The ARCH process explicitly recognizes the difference between the unconditional and the conditional variance allowing the latter to change over time as a function of past errors. ARCH models describe the dynamic changes in conditional variance as a deterministic (typically quadratic) function of past returns. Because the variance is known at time t-1, one-step-ahead forecasts are readily available. Next, multi-step-ahead forecasts can be computed recursively. A

more parsimonious model than ARCH is the so-called generalized ARCH (GARCH) model42 where additional dependencies are permitted on lags of the conditional variance. A GARCH model has an ARMA-type representation, so that the models share many properties. 2 The GARCH (p, q) model23 (where p is the order of the GARCH terms σ and q is the order of 2 the ARCH terms ε ) is given by

q

p

σ t2 = α 0 + α1ε t2−1 + ... + α q ε t2− q + β1σ t2−1 + ... + β pσ t2− p = ∑ α iε t2−i + ∑ β iσ t2−i i =1

i =1

(6)

α and β are parameters and have to be estimated. Let ε t denote a real-valued discrete time stochastic process, and σ t , the information set (a-field) of all information through time t. For financial time series prediction by GARCH, relative changes of DJIA index has been used, instead of the actual index values, since the performance of GARCH in this case is better43. We used different versions of GARCH such as: GARCH(1,1), GARCH(1,2), GARCH(2,1), GARCH(2,2). For comparison with our new method regarding to time series prediction, we picked the one with the best result which was GARCH(1,2). 2. Result of financial time series prediction The Dow Jones Industrial index (DJIA) time series was considered with respect to the daily closing values of the DJIA for three time periods: 1) September 1993-September 2001, 2) July 2001-July 2009 and 3) August 2004-August 2012. The prediction errors for GenericPred were significantly less than those for other methods in all three periods for any length of prediction (see Supplementary Table 1). Moreover, the GenericPred predictions were more stable with a consistently lower standard deviation, regardless of whether the target data lie before the

recession, during the recession, or after the recession. The prediction error for the first 200 steps is, in particular, smaller than that of the other methods. Supplementary Table 1. Comparing the performance of different methods using the mean absolute percentage error (MAPE)44. Comparison of MAPE values between several methods and the GenericPred method for the prediction of DJIA time series for the next 500 days. LFABS and MLP are methods dedicated to one-time-step prediction, and no values are given for the longer ranges. 1

10

50

100

200

300

400

500

Mean

Std.

(1-500)

(1-500)

DJIA 1993-2001 ARIMA

0.22%

0.5%

4%

5%

10%

11%

10%

18%

10%

5%

GARCH

0.22%

0.7%

6%

8%

15%

18%

20%

28%

16%

7%

VAR

0.25%

0.46%

5%

7%

15%

17%

19%

28%

15%

7%

0.14%

0.23%

1%

3%

3%

0.1%

3%

2%

3%

2%

L-FABS

0.57%

-

-

-

-

-

-

-

-

-

MLP

1.06%

-

-

-

-

-

-

-

-

-

ARIMA

0.15%

2.5%

3%

4%

7%

13%

41%

40%

19%

17%

GARCH

0.02%

1.5%

6%

9%

17%

25%

52%

54%

27%

20%

VAR

0.02%

2%

5%

8%

14%

23%

50%

52%

26%

19%

0.03%

0.8%

1.5%

3%

0.27%

7%

24%

15%

10%

8%

ARIMA

0.93%

3.5%

7%

12%

17%

8%

19%

17%

12%

5%

GARCH

0.65%

2%

12%

19%

37%

44%

95%

125%

47%

32%

VAR

0.93%

3%

9%

16%

26%

20%

40%

46%

24%

12%

0.43%

1.5%

0.3%

2%

4%

3%

13%

8%

7%

4%

GenericPred

DJIA 2001-2009

GenericPred DJIA 2004-2012

GenericPred

With respect to the DJIA period of 1993-2001, we also compared our results with the Learning Financial Agent Based Simulator (L-FABS)21 and the MLP model22, which have demonstrated high accuracy for predictions in this period. The GenericPred method outperformed F-FABS and

MLP for the first step prediction with these data. The GenericPred method still clearly outperformed the three other methods, with an overall error rate of 3% (see Supplementary Table 1). In the second considered period, the prediction accuracy for the first 200 steps of the GenericPred method is still high (less than 3% error), whereas the accuracy decreases significantly for the last 200 steps at the peak of the financial crisis, although its accuracy is still higher than that of the other methods (see Supplementary Table 1). The ARIMA method still outperformed the GARCH and VAR methods, although its performance is significantly worse than that of the GenericPred method for the 500 time steps. In the third period (August 2004-August 2012), the GenericPred method has a higher overall predictive accuracy (7% errors on average) than the other predictive methods (see Supplementary Table 1). For the same data, with respect to prediction accuracy, ARIMA outperforms the GARCH and VAR methods, although it performs significantly worse than GenericPred (12% error on average for ARIMA). To investigate the prediction performance and sensitivity of our new method with respect to the starting point of the prediction, GenericPred was applied to different consecutive intervals of the 2003-2011 DJIA time series with a difference of 2 months between each prediction (see Supplementary Figure 1). GenericPred always predicted a strong decrease in the stock market, whereas ARIMA, GARCH and VAR failed to do so. Only VAR predicted a decrease in the stock market for the last considered starting point, which corresponds to the very beginning of the crisis.

Supplementary Figure 1. Analysis of the sensitivity of GenericPred to the starting point for the prediction. Prediction of DJIA time series starting from (A) September 2001, (B) November 2001, (C) January 2002, (D) March 2002, (E) May 2002 and (F) July 2002. 3. Results of epileptic seizure prediction 3.1. Description of the EEG data used The electroencephalogram (EEG) is an extracellular recording of the summated electrical activity of groups of neurons in the brain (field potentials), achieved by placing a set of

electrodes on the scalp’s surface45. In neurology, the main diagnostic application of the EEG is with respect to epilepsy as epileptic activity can create clear abnormalities on a standard EEG46. EEG datasets of 21 patients were selected from the Epilepsy Center of the University Hospital of Freiburg47. In 11 patients, the epileptic focus was located in neocortical brain structures, whereas in eight of the patients, the focus was in the subcortical hippocampus. In the two remaining patients, the focus of the seizures involved both the neocortex and the hippocampus. The EEG data were acquired using a Neurofile NT digital video EEG system with 128 channels, 256 Hz sampling rate, and a 16-bit analogue-to-digital converter. For each of the patients, there were two types of datasets, viz., "ictal" and "interictal.” The former contains records of epileptic seizures and at least 50 min of pre-ictal data. The latter contains approximately 24 hours of EEG records without seizure activity48. The EEG signal is represented as a time series vector X={x1, x2,..., xN} comprising singlevoltage readings at various time intervals and expressed as a series of individual data points (single voltage readings by an electrode), where N is the total number of data points and the subscript indicates the time instant49. 3.2. Prediction results In practice, the detection of seizures during an EEG is difficult even for a trained neurologist because there is no obvious change in pattern during an epileptic seizure (see Supplementary Figure 2). The GenericPred method was applied to all three stages (before a seizure, during a seizure and after a seizure).

Supplementary Figure 2. EEG time series during different stages. The recorded EEG time series from electrode #5 of patient #1 over approximately 1 hour. The red colour illustrates the ictal part (indicating an epileptic seizure) of the EEG. There are many different methods of predicting epileptic seizures, although none of them has had significant success with a wide variety of data. One of the best current results only achieves an accuracy of 73% sensitivity and 67% specificity for 10 patients within a 1-10 minute range26. To illustrate the methodology we designed with GenericPred, we present an example of the prediction of an epileptic seizure. For this example, the EEG time series recorded by electrode #5 for the first patient has been considered. The EEG time series we considered, including before the seizure, during the seizure and after the seizure, has a length of 920,000 time steps (approximately 24 hours). According to the database, the seizure starts from the time 91,100 and lasts until 96,090 (approximately 8 minutes). To evaluate the GenericPred method for false detection, we considered the 10 ranges of the EEG time series before time step 91,100 and after

time step 96,090 (before and after the seizure) when there is no seizure. No peaks were predicted by the GenericPred method in any of these cases (see Supplementary Figure 3 for an example of one of these cases), while GenericPred predicted the peak of the P&H measure during the seizure 17 minutes before it occurred (see Supplementary Figure 4). The average P&H value during the seizure-free part of the EEG time series is -0.3 (±0.7), while the average P&H value during the seizure is 2.8 (±0.05). The same methodology has been applied to the data for all 21 patients to compute the sensitivity and specificity of GenericPred (see Table 1 in the main paper).

Supplementary Figure 3. No peak is predicted by GenericPred during a seizure-free part of the EEG.

Supplementary Figure 4. Prediction of epileptic seizure 17 minutes in advance.

References 40. Makridakis, S. & M. Hibon, “ARMA models and the Box--Jenkins methodology,” J. Forecast. 16,147–163 (1997). 41. Engle, R. F. “Autoregressive Conditional Heteroscedasticity with Estimates of the Variance of United Kingdom Inflation,” Econometrica 50, 987–1007 (1982). 42. Taylor, S. J. Forecasting the volatility of currency exchange rates. Int. J. Forecast. 3, 159– 170 (1987). 43. Omar, L. S., Kinyanjui, J. K. & Mutiso, J. M. GARCH Modelling In Share Prices For Specific Companies In Kenya. AM. J. MATH-S. 1, 5-12 (2013). 44. Hamilton, J. D. Time Series Analysis (Princeton University Press, Princeton, 1994).

45. Kandel, E. R., Schwartz, J. H., Jessell, T. M. Principles of Neural Science (McGraw-Hill, New York, 2000). 46. Abou-Khalil, B. & Misulis, K. E. Atlas Of EEG And Seizure Semiology (Elsevier, Philadelphia, 2006) 47. Schelter, B. et al. Do false predictions of seizures depend on the state of vigilance? A report from two seizure-prediction methods and proposed remedies. Epilepsia 47, 2058–2070 (2006). 48. Tafreshi, A. K., Nasrabadi, A. M. & Omidvarnia, A. H. Epileptic Seizure Detection Using Empirical Mode Decomposition, in 2008 IEEE International Symposium on Signal Processing and Information Technology, 238–242 (2008). 49. Adeli, H., Ghosh-Dastidar, S. & Dadmehr, N. A wavelet-chaos methodology for analysis of EEGs and EEG subbands to detect seizure and epilepsy. IEEE Trans. Biomed. Eng. 54, 205–211 (2007).