High-dimensional functional time series forecasting

High-dimensional functional time series forecasting Yuan Gao, Han Lin Shang, Yanrong Yang

Abstract In this paper, we address the problem of forecasting high-dimensional functional time series through a two-fold dimension reduction procedure. Dynamic functional principal component analysis is applied to reduce each infinite-dimension functional time series to a vector. We use factor model as a further dimension reduction technique so that only a small number of latent factors are preserved. Simple time series models can be used to forecast the factors and forecast of the functions can be constructed. The proposed method is easy to implement especially when the dimension of functional time series is large. We show the superiority of our approach by both simulation studies and an application to Japan mortality rates data.

1 Introduction Functional data are considered as realizations of smooth random curves. When curves are collected sequentially, they form functional time series Xt (u), u ∈ I . To deal with infinite dimensional functions, there is a demand for efficient data reduction techniques. Functional principal component ananlysis (FPCA) is the most commonly used approach that serves this purpose. FPCA performs eigendecomposition on the underlying covariance functions. Most of the variance structures captured in a vector called principal component scores (see [13] for details). Existing FPCA Yuan Gao College of Business and Economics, Australian National University, 26C Kingsley St. ACTON 2601, Australia, e-mail: [email protected] Hanlin Shang College of Business and Economics, Australian National University, e-mail: [email protected] Yanrong Yang College of Business and Economics, Australian National University, e-mail: [email protected]

1

2

Yuan Gao, Han Lin Shang, Yanrong Yang

method has been developed for independent observations, which is a serious weakness when we are dealing with time series data. In this paper, we adopt a dynamic FPCA approach ([10],[15]), where serial dependence between the curves are taken into account. With dynamic FPCA, functional time series are reduced to a vector time series, where the individual component processes are mutually uncorrelated principal component scores. It is often the case that we collect a vector of N functions at a single time point t. If these N functions are assumed to be correlated, multivariate functional models should be considered. Classical multivariate FPCA concatenates the multiple functions into one to perform univariate FPCA ([13]). [9] suggested normalizing each random function as a preliminary step before concatenation. [6] studied functional version of principal component ananlysis, where multivariate functional data are reduced to one or two functions rather than vectors. However, existing models dealing with multivariate functional data either fail to handle data with a large N (as in the classical FPCA approach), or are hard to implement practically (as in [6]). We propose a two-fold dimension reduction model to handle high-dimensional functional time series. By high-dimension, we allow that the dimension of the functional time series N to be as large as or even larger than the length of observed functional time series T . The dimension reduction process is straightforward and easy to implement: 1) Dynamic functional principal component analysis is performed on each set of functional time series, resulting in N sets of principal component scores of low dimension K (typically less than 5); 2) The N first principal component are fitted to a factor model, and is further reduced to a dimension of r (r  N). The same is done for the second principal component scores and thus K factor models are fitted. The vector of N functional time series is reduced to r × K vector time series. Factor models are frequently used for dimension reduction. Some early application of factor analysis to multiple time series include [1], [16] and [7]. Time series in high-dimensional settings with N → ∞ together with T are studied in [8], [4] and [11]. Among these, we adopt the method considered in [11], where the model is conceptually simple and asymptotic properties are established.

2 Research methods In this section, we introduce dynamic FPCA and factor model for the two-fold dimension reduction process. Estimation and asymptotic properties are discussed. We also suggest methods for forecasting.

High-dimensional functional time series forecasting

3

2.1 Dynamic functional principal component analysis We consider stationary N-dimensional functional time series X t : t ∈ Z, where X t = (Xt1 (u), . . . , XtN (u))> , and each Xt (u) takes values in the space H := L2 (I ) of realvalued square integrable functions on I . The space H is a Hilbert space, equipped with the inner product hx, yi := R i I x(u)y(u)du. For each i = 1, . . . , N, we assume Xt has a continuous mean function, i i µ (u) and an auto-covariance function at lag h, γh (u, v), where µ i (u)

= E[X i (u)],

γhi (u, v)

i (v)] = cov[Xti (u), Xt+h

(1)

The long-run covariance function is defined as ∞

ci (u, v) =

∑

γhi (u, v)

(2)

h=−∞

Using ci (u, v) as a kernel, we define the operator C by: Ci (x)(u) =

Z I

ci (u, v)x(v)dv,

u, v ∈ I

(3)

The kernel is symmetric, non-negative definite. Thus by Mercer’s Theorem, the operator C admits an eigendecomposition Ci (x) =

∞

∑ λk hx, υk iυk ,

(4)

k=1

where (λl : l ≥ 1) are C’s eigenvalues in descending order and (υl : l ≥ 1) the corresponding normalized eigenfunctions. By Karhunen-Lo`eve Theorem, X can be represented with Xti (u) =

∞

∑ βt,ki υki (u)

(5)

k=1 i = i i where βt,k I Xt (u)υk (u)du is the kth principal component score at time t. The infinite dimensional functions can be approximated by the first K principal component scores:

R

K

Xti (u) =

∑ βt,ki υki (u) + εti (u)

(6)

k=1

2.2 Factor model With the first step dimension reduction, we now have principal component scores i , where i = 1, . . . , N. Following the early work by [10] and [5], we consider the βt,k

4

Yuan Gao, Han Lin Shang, Yanrong Yang

1 , . . . , β N )> . For each k = 1, . . . , K, let following factor model for β i,t = (βt,k t,k

β t,k = A k ω t,k + e i,t ,

t = 1, . . . T,

(7)

where β t,k is the vector that contains the kth principal component score of all N functional time series. ω t,k is an r × 1 unobserved factor time series. A k is an N × r unknown constant factor loading matrix, and et,k is idiosyncratic error with mean 0 and variance σi,t2 . K factor models are fitted to the principal component scores. With two-fold dimension reduction, the original functional time series can be approximated by K

Xti (u) =

∑ [AAk ω t,k ]i υki (u) + θti (u),

(8)

k=1

Ak ω t,k ]i is the ith element in the vector A k ω t,k , and θti (u) is the error term where [A from two steps of approximation.

2.3 Estimation We want to estimate A k , ω t,k and υki (u) in (8). In the dynamic FPCA step, the longrun covariance function ci (u, v) can be estimated by cbi (u, v) =

∞

h W ( )γbhi (u, v) q h=−∞

∑

(9)

where ( γˆhi (u, v) =

1 T 1 T

−h ¯ (X ji (u) − X¯ j (u))(X j+h (v) − X(v)), h≥0 ∑Tj=1 T i ¯ ¯ h m for some m > 0, and W is continuous on [−m, m]. Some possible choices include Bartlett, Parzen, Tukey-Hanning, Quadratic spectral and Flat-top functions ([2, 3]). q is a bandwidth parameter. [14] proposed a plug-in procedure to select q. cbi (u, v) b with which we can estimate υbi (u) by peris used as the kernel of the operator C, k b υbi (u) is the normalized eigenfunction that corforming eigendecomposition on C. k responds to the kth largest eigenvalue. The empirical principal component scores R i = i bki (u)du, can be calculated by numerical integration. βbt,k I Xt (u)υ i are fitted to a factor model. The estimation of latent factors The estimates βbt,k for high-dimensional time series can be found in [11]. A natural estimator for A k is b k = (b b k , and defined as A a1, . . . , b a r ), where b a j is the jth eigenvector of Q h0

bk = Q

>

∑ Σb h,k Σb h,k , h=1

Σb h =

1 T −h b ∑ (β t+h,k − β˜ k )(βb t,k − β˜ k )> , T − h h=1

(10)

High-dimensional functional time series forecasting

where β˜ k =

1 T

5

T b β t,k . Thus we estimate the kth factor by ∑t=1

b > βb b t,k = A ω k t,k

(11)

The estimator for the original function Xti (u) is K

Xbti (u) =

∑ [Ab k ωb t,k ]i υbki (u),

i = 1, . . . , N,

t = 1, . . . , T

(12)

k=1

bkω bkω b t,k ]i is the ith element of the vector A b t,k . where [A

2.4 Forecasting With two-fold dimension reduction, information of serial correlation is contained in the factors ω t,k . To forecast N-dimensional functional time series, we could instead make forecast on the estimated factors. Scalar or vector time series models could be applied. We suggest univariate time series models, autoregressive moving average (ARMA) models, for instance, since the factors are mutually uncorrelated. Consequently, we need to fit r × K ARMA models on the factors. The prediction of the functions could be calculated: K i E[Xbt+h|t (u)] =

∑ [Ab k ωb t+h|t,k ]i υbki (u),

i = 1, . . . , N,

t = 1, . . . , T,

(13)

k=1 i where Xbt+h|t (u) is h-step ahead forecast at time t.

3 Empirical studies Japanese sub-national mortality rates in 47 prefectures is used to demonstrate the effectiveness of our proposed method. Available at [12], the data set contains yearly age-specific mortality rates in a span of 40 years from 1975 to 2014. The dimension of the functional time series is 47, which is greater than the sample size 40. With the two-fold dimension reduction model, we use the first three principal component scores for each population, and the first three factors. The problem of choosing the appropriate number of scores and factors will be discussed in detail. We compare the forecast accuracy of our proposed method with the independent functional time series model, where each sub-national population is forecast individually. Both point forecast and interval forecast errors are calculated and it is found that the proposed method outperforms the independent model in most of the prefectures.

6

Yuan Gao, Han Lin Shang, Yanrong Yang

References 1. Anderson, T.: The use of factor analysis in the statistical analysis of multiple time series. Psychometrika. 28, 1-25 (1963) 2. Andrews, D.: Heteroskedasticity and autocorrelation consistent covariance matrix estimation. Econometrica. 59, 817-858 (1991) 3. Andrews, D., Monahan, J.C.: An improved heteroskedasticity and autocorrelation consistent covariance matrix estimator. Econometrica. 60, 953-966 (1992) 4. Bai, J.: Inferential theory for factor models of large dimensions. Econometrica. 71, 135-171 (2003) 5. Bai, J.: Panel data models with interactive fixed effects. Econometrica. 77, 4, 1229-1279 (2009) 6. Berrendero, J., Justel, A. Svarc, M.: Principal components for multivariate functional data. Computational Statistics and Data Analysis. 55, 9, 2619-2634 (2011) 7. Brillinger, D.: Time Series Data Analysis and Theory, extended ed. Holder-Day, San Francisco (1981) 8. Chamberlain, G.: Funds, factors, and diversification in arbitrage pricing models. Econometrica. 70, 191-221 (1983) 9. Chiou, J.M., Chen, Y.T., Yang, Y.F.: Multivariate functional principal component analysis: A normalization approach. Statistica Sinica. 24, 4, 1571-1596 (2014) 10. H¨ormann, S., Kidzi´nski, L., Hallin, M.: Dynamic functional principal components. Journal of the Royal Statistical Society, Statistical Methodology, Series B. 77, 2, 319-348 (2015) 11. Lam, C., Yao, Q., Bathia, N.: Estimation of latent factors for high-dimensional time series. Biometrika. 98, 4, 901-918 (2011) 12. National Institute of Population and Social Security Research: Japan Mortality Database. Available at http://www.ipss.go.jp/p-toukei/JMD/index-en.html. (2016) 13. Ramsay, J.O., Silverman, B.W.: Functional Data Analysis. Springer, New York (2005) 14. Rice, G., Shang, H.L.: A plug-in bandwidth selection procedure for long run covariance estimation with stationary functional time series. Journal of Time Series Analysis. in proceeding (2016) 15. Panaretos, V.M., Tavakoli, S.: Cram´er-Karhunen-Lo`eve representation and harmonic principal component analysis of functional time series. Stochastic Processes and their Applications. 123, 7, 2779-2807 (2013) 16. Priestley, M., Rao, T. Tong, J.: Applications of principal component analysis and factor analysis in the identification of multivariable systems. IEEE Trans. Auto. Contr. 19, 6 730-734 (1974)