[9] Papacharalampous, G.A., Tyralis, H., and Koutsoyiannis, D., 2017a. Comparison of stochastic and machine learning methods for multi-step ahead forecasting ...
Univariate time series forecasting properties of random forests Hristos Tyralis, and Georgia Papacharalampous National Technical University of Athens (itia.ntua.gr/1826) Session IE4.1/NP4.3/AS5.13/CL5.18/ESSI2.3/GD10.6/HS3.7/NH11.14/SM7.03
1. Abstract
4. Errors for the ARMA(1, 0) with φ1 = 0.6
7. Rankings based on the absolute errors
The random forests’ univariate time series forecasting properties have remained unexplored. Here we assess the performance of random forests in one-step forecasting using two large datasets of short time series with the aim to suggest an optimal set of predictor variables. Furthermore, we compare their performance to benchmarking methods. The first dataset consists of 16 000 simulated time series from a variety of Autoregressive Fractionally Integrated Moving Average (ARFIMA) models. The second dataset consists of 135 mean annual temperature time series. The random forests performed better mostly when using a few recent lagged predictor variables. A possible explanation of this result is that increasing the number of lagged variables decreases the length of the training set and simultaneously decreases the information exploited from the original time series during the model fitting phase. Furthermore, the random forests were comparable to the benchmarking methods.
2. Introduction • • • • • • • • • •
• •
Mean of absolute errors
Median of absolute errors
5. Boxplots of absolute errors for the ARMA(p, q)
8. Rankings based on squared errors
The use of machine learning algorithms for time series forecasting has increased. Random forests (RF) is a machine learning algorithm introduced by Breiman (2001). A review and a simple presentation of the RF algorithm can be found in Biau and Scornet (2016) and Verikas et al. (2011). The aim here is to investigate how the performance of RF is related to the variable selection in one-step forecasting of short time series. Our contribution regards the optimization of the forecasting performance of the RF. We apply the RF using the caret (Kuhn 2008) and randomForest (Liaw and Wiener 2002) R packages. To this end we simulate 16 000 time series of length equal to 101 using 16 ARFIMA(p, d, q) models (each one 1 000 times) (Fraley et al. 2012). We also use 135 mean annual instrumental temperature time series (Lawrimore et al. 2011) spanning the period 1916-2016 (101 years). The RF are used to forecast the 101st value (test set) of each time series of both datasets (simulations and real data). The RF are also compared to two naïve methods (last value and average of the first 100 values), the ARFIMA method (Hyndman et al. 2017) and the Theta method (Assimakopoulos and Nikolopoulos 2000). The forecasting errors are computed for each method and are used for comparison reasons. Details on the methods and the results can be found in Tyralis and Papacharalampous (2017a), while supplementary information can be found in Tyralis and Papacharalampous (2017b).
3. Data and methods
10. Errors for the temperature dataset
11. Conclusions
Mean of squared errors.
Median of squared errors.
6. Simulation experiments and rankings
9. Errors for the temperature dataset
•
Random forests are amongst the most popular machine learning algorithms.
•
The present study is one of the first applying benchmarking to assess the forecasting performance of random forests in time series forecasting (see also Papacharalampous et al. 2017a, b).
•
The results of the simulation experiments and the real case study indicate that using a few recent variables as predictors during the fitting process leads to higher predictive accuracy for the random forests algorithm used here.
•
The results were obtained using short time series, while there may be better procedures to find the optimal sets of parameters.
•
The performance of the optimized random forests was equal to the performance of the benchmarking methods.
•
This is also an indication that there is an upper bound of predictability in univariate time series of geophysical processes (see also Papacharalampous et al. 2017c, 2018a, b, c).
•
Based on this fact, we would like to propose the use of a variety of algorithms in each forecasting application.
•
Particularly for the experiment conducted on real world time series, we observe that the minimum absolute error in forecasting next year’s temperature using RF was approximately equal to 0.6°C. This outcome is definitely important for geosciences, suggesting that temperature is a process difficult to forecast.
References [1]
Assimakopoulos, V., and Nikolopoulos, K., 2000. The theta model: a decomposition approach to forecasting. International Journal of Forecasting, 16 (4), 521–530. doi: 10.1016/S0169-2070(00)00066-2.
[2]
Biau, G., and Scornet, E., 2016. A random forest guided tour. Test 25 (2), 197–227. doi:10.1007/s11749-016-0481-7.
[3]
Breiman, L., 2001. Random Forests. Machine Learning 45 (1), 5–32. doi: 10.1023/A:1010933404324.
[4]
Fraley, C., Leisch, F., Maechler, M., Reisen, V., and Lemonte, A., 2012. fracdiff: Fractionally differenced ARIMA aka ARFIMA(p,d,q) models. R package version 1.4-2.
[5]
Hyndman, R., O'Hara-Wild, M., Bergmeir, C., Razbash, S., and Wang E., 2017. forecast: Forecasting Functions for Time Series and Linear Models. R package version 8.1.
[6]
Kuhn, M., 2008. Building Predictive Models in R Using the caret Package. Journal of Statistical Software , 28 (5). doi: 10.18637/jss.v028.i05.
[7]
Lawrimore, J.H., Menne, M.J., Gleason, B.E., Williams, C.N., Wuertz, D.B., Vose, R.S., and Rennie, J., 2011. An overview of the Global Historical Climatology Network monthly mean temperature data set, version 3. Journal of Geophysical Research, 116 (D19121). doi: 10.1029/2011JD016187.
[8]
Liaw, A., and Wiener, M., 2002. Classification and Regression by randomForest. R News, 2 (3), 18–22.
[9]
Papacharalampous, G.A., Tyralis, H., and Koutsoyiannis, D., 2017a. Comparison of stochastic and machine learning methods for multi-step ahead forecasting of hydrological processes. Preprints, 2017100133. doi: 10.20944/preprints201710.0133.v2.
Simulation experiments In this case, the role of the predictor variables is taken by previous values of the time series (lagged variables). Therefore, increasing the number of predictor variables, i.e. the selected lagged variables, results in reducing the length of the training set.
Temperature stations.
Experiment 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Model ARMA(1, 0) ARMA(1, 0) ARMA(2, 0) ARMA(2, 0) ARMA(0, 1) ARMA(0, 1) ARMA(0, 2) ARMA(0, 2) ARMA(1, 1) ARMA(1, 1) ARMA(2, 2) ARFIMA(0, 0.40, 0) ARFIMA(1, 0.40, 0) ARFIMA(0, 0.40, 1) ARFIMA(1, 0.40, 1) ARFIMA(2, 0.40, 2)
Parameters φ1 = 0.6 φ1 = -0.6 φ1 = 0.6, φ2 = 0.2 φ1 = -0.6, φ2 = 0.2 θ1 = 0.6 θ1 = -0.6 θ1 = 0.6, θ2 = 0.2 θ1 = -0.6, θ2 = -0.2 φ1 = 0.6, θ1 = 0.6 φ1 = -0.6, θ1 = -0.6 φ1 = 0.6, φ2 = 0.2, θ1 = 0.6, θ2 = 0.2 φ1 = 0.6 θ1 = 0.6 φ1 = 0.6, θ1 = 0.6 φ1 = 0.6, φ2 = 0.2, θ1 = 0.6, θ2 = 0.2
[10] Papacharalampous, G.A., Tyralis, H., and Koutsoyiannis, D., 2017b. Forecasting of geophysical processes using stochastic and machine learning algorithms. European Water, 59, 161–168. [11] Papacharalampous, G.A., Tyralis, H., Koutsoyiannis, D., 2017c. Error Evolution in Multi-Step Ahead Streamflow Forecasting for the Operation of Hydropower Reservoirs. Preprints, 2017100129. doi: 10.20944/preprints201710.0129.v1. [12] Papacharalampous, G.A., Tyralis, H., and Koutsoyiannis, D., 2018a. One-step ahead forecasting of geophysical processes within a purely statistical framework. Geoscience Letters, 5 (12). doi: 10.1186/s40562-018-0111-1. [13] Papacharalampous, G.A., Tyralis, H., and Koutsoyiannis, D., 2018b. Predictability of monthly temperature and precipitation using automatic time series forecasting methods. Acta Geophysica. doi: 10.1007/s11600-018-0120-7.
Rankings based on regression coefficients
[14] Papacharalampous, G.A., Tyralis, H., and Koutsoyiannis, D., 2018c. Univariate time series forecasting of temperature and precipitation with a focus on machine learning algorithms: A multiple-case study from Greece. In review.
rf05 uses 5 predictor variables
[16] Tyralis, H., and Papacharalampous, G.A., 2017b. Supplementary material for the paper "Variable selection in time series forecasting using random forests". Mendeley Data. doi: 10.17632/nr3z96jmbm.1.
[15] Tyralis, H., and Papacharalampous, G.A., 2017a. Variable selection in time series forecasting using random forests. Algorithms , 10 (4), 114. doi: 10.3390/a10040114.
[17] Verikas, A., Gelzinis, A., and Bacauskiene, M., 2011. Mining data with random forests: A survey and results of new tests. Pattern Recognition, 44 (2), 330–349. doi: 10.1016/j.patcog.2010.08.011.
