Author's Copy
Random Operators / Stochastic Eqs. 17 (2009), 125–130
c de Gruyter 2009
DOI 10.1515 / ROSE.2009.008
Sum of the sample autocorrelation function Hossein Hassani
Abstract. In this paper, the sum of the sample autocorrelation function, found in many standard time series textbooks and software, at lag h ≥ 1 is considered. It is shown that this sum is always − 12 for any stationary time series with arbitrary length T ≥ 2. As an application of this quantity, it is shown that the sample spectral density of a stationary process fluctuates violently about the theoretical spectral density. Key words. Sample autocorrelation function; stationary process; spectral density; periodogram. AMS classification. 37M10.
1. Introduction The autocovariance function (ACF) of a wide sense stationary process {Yt , t ∈ N} at lag h is: R(h) = E [(Yt+h − µY )(Yt − µY )], h ∈ Z, (1.1) where E is the expected value operator, µY is the expected value of the variable Y . In practical problems we only have a set of data YT = (y1 , . . . , yT ); the following estimator can be considered as an estimate of R(h): PT −|h| ¯ t −y) ¯ t=1 (yt+|h| −y)(y h = 0, ±1, . . . , ±(T − 1), ˜ T −|h| R(h) = (1.2) 0 |h| ≥ T, where y¯ =
T 1X yt T t=1
is the sample mean which is an unbiased estimator of µ. If we ignore the effect of estimating µ by y¯ (that is if we replace y¯ by µ in (1.2)), then it is easy to show that ˜ R(h) is an exactly unbiased estimator of R(h). In the general case, it can be shown ˜ (see Priestly (1981)) that R(h) is asymptotically unbiased. There is an alternative estimate of R(h) which is suggested by some authors (for example Parzen (1961) and Schaerf (1964)): P Tt=1−|h| (yt+|h| −y)(y ¯ t −y) ¯ h = 0, ±1, . . . , ±(T − 1), T γ(h) ˆ = (1.3) 0 |h| ≥ T.
Author's Copy
Author's Copy
Author's Copy
Communicated by N. N. Leonenko
Author's Copy
Hossein Hassani
γ(h) ˆ is biased on the use of the divisor T rather than T − |h| and also has larger bias ˜ than R(h). It has been asserted that, in general, γ(h) ˆ has smaller mean squared error ˜ than R(h). For example Parzen (1961) has shown that for a particular AR(1) process, γ(h) ˆ has smaller mean squared error for all |h| > 0, the difference increasing significantly as h increases. It should be noted that γ(h) ˆ is much more popular than ˜ R(h) among time series analysts and they prefer to use γ(h) ˆ (see, for example, Brockwell and Davis (2002), Priestley (1981), Shumway and Stoffer (2006) and Pourahmadi (2001) among others) and also time series packages e.g. Minitab, SPSS, Eviews, Stata, ˆ Splus, R and SAS use R(h). Priestley (1981) discussed in detail the advantages of us˜ ing γ(h) ˆ rather than R(h) from several points of view. The main point why γ(h) ˆ ˜ ˜ is preferred to R(h) is that γ(h) ˆ has the positive semi-definite property while R(h) does not, this is an important issue in the spectral domain estimation. Therefore, we consider γ(h) ˆ as an estimator of R(h). The autocorrelation function, ACF, is given by ρ(h) =
R(h) , R(0)
h ∈ Z,
(1.4)
h = 0, ±1, . . . , ±(T − 1).
(1.5)
and an estimate of ρ(h) is ρ(h) ˆ =
γ(h) ˆ , γ(0) ˆ
ˆ The sequence ρ(h) ˆ is positive semi-definite (this follows from the property of R(h)). ˜ ˜ But this does not necessarily hold for sequence ρ(h) ˜ = R(h)/R(0). Moreover, it is easy to show that |ρ(h)| ˆ 6 1 for all h and, again, this property does not necessarily hold for ρ(h) ˜ (Priestley (1981), p. 331). In the rest of the paper, we only consider ACF at lag h > 0, since ρ(h) = ρ(−h), and also use ρ(h) ˆ as its estimator. Note also that ρ(0) = 1. The sum of the sample autocorrelation function at lag h ≥ 1 for any stationary time series has been considered as a characteristic of interest in Brockwell and Davis (1991) (see Problem 7.3 of Brockwell and Davis (1991))1 . This equality has been partly examined, and several of its properties also considered. The structure of the paper is as follows. In Section 2, we will prove that the sum of the sample autocorrelation function, based on ρ, ˆ is always − 21 for any time series with arbitrary length T ≥ 2. A useful application using this quantity is illustrated in Section 3. Some conclusions are given in Section 4.
2. Sum of the sample ACF Theorem 2.1. The sum of the sample autocorrelation function, Sacf , at lag h ≥ 1 is always − 21 for any stationary time series with arbitrary length T ≥ 2; that is Sacf =
T −1 X h=1
1
1 ρ(h) ˆ =− . 2
The author is grateful to Professor Richard A. Davis for his comments on this matter.
Author's Copy
Author's Copy
Author's Copy
126
Author's Copy
127
Sum of the sample autocorrelation function
Proof. Sacf =
T −1 X
ρ(h) ˆ =
h=1
T −1 X
(yt − y)(y ¯ t+h − y) ¯ PT 2 ¯ t=1 (yt − y)
h=1
t=1 P T
!
t=1
h=1
PT −1 PT −h =
PT −h
(yt − y)(y ¯ t+h − y) ¯
t=1 (yt
− y) ¯ 2
PT −1 PT −h
¯ t+h − y) ¯ h=1 t=1 (yt − y)(y = PT PT −1 PT −h 2 ¯ t+h − y) ¯ ¯ − 2 h=1 t=1 (yt − y)(y ( t=1 (yt − y)) PT −1 PT −h
¯ t+h − y) ¯ 1 h=1 t=1 (yt − y)(y =− . P T −1 PT −h 2 ¯ t+h − y) ¯ −2 h=1 t=1 (yt − y)(y
(2.1) 2
Sacf has the following properties: (i) It does not depend on the time series length T ; Sacf = − 12 for T ≥ 2. (ii) The value of Sacf is equal to − 12 for any stationary time series. Thus, for example, Sacf for ARMA(p, q) of any order (p, q) is equal to a Gaussian white noise process and both are equal to − 12 . (iii) The values of ρ(h) ˆ are linearly dependent: T −1 X 1 ρ(i) ˆ =− − ρ(j), ˆ 2
i = 1, . . . , T − 1.
(2.2)
j6=i=1
(iv) There is at least one negative ρ(h) ˆ for any stationary time series even for AR(p) with positive ACF.
3. An application Let {Yt , t ∈ N} be a real valued wide sense stationary stochastic process with autocovariance function γ(·) and zero expectation. The spectral distribution function F (ω) of γ(·) or of the process is defined as a right-continuous, non-decreasing, bounded function on (−π, π] with F (−π) = 0 satisfying Z γ(h) = eihv dF (v) for all h = 0, ±1, . . . . (3.1) (−π,π]
The spectral density function f (w) is defined by Z ω F (ω) = f (v)dv, ω ∈ (−π, π] −π
Author's Copy
(3.2)
Author's Copy
Author's Copy
=
Author's Copy
128
Hossein Hassani
and can be written as (Brockwell and Davis (2002), p. 124) f (ω) =
∞ 1 X −ihω e γ(h) for all ω ∈ (−π, π]. 2π
(3.3)
h=−∞
The autocovariance function, therefore, is a Fourier coefficient of the spectral density function, and vice versa the spectral density function is a Fourier coefficient of the autocovariance function. The sample autocovariance function γ(·) ˆ yields an intuitive estimate of the spectral density by replacing γ(·) in the definition of the spectral density, as in (3.3), by its estimate γ(·). ˆ 1 2π
T −1 X
e−ihω γ(h) ˆ for all ω ∈ (−π, π].
(3.4)
h=−(T −1)
I(·) is called the periodogram and is very often defined on Fourier frequencies only. It has been shown that the periodogram is not a consistent estimator of the spectral density (Priestley (1981), p. 425) in the sense that Var(I(ω)) does not converge to zero as T → ∞. Also I(ω) does not converge to f (ω), the true density, in mean square. Here we consider this problem from another point of view. Let us consider I(ω) at ω = 0. T −1 T −1 X X 1 1 I(0) = ρ(h) ˆ . (3.5) γ(h) ˆ = γ(0) ˆ 1+2 2π 2π h=−(T −1)
h=1
The right side of (3.5), therefore, is always equal to zero, as Sacf = − 12 , for the case ω = 0. This is an alternative proof which shows that the sample spectral density of a stationary process fluctuates violently about the theoretical spectral density (for more information see, for example, Priestley (1981) and Jenkins and Watts (1968)). Let us consider the spectral density of a white noise process as an example. If {Yt } ∼ WN(0, σ 2 ), then γ(0) = σ 2 and γ(h) = 0 for h ≥ 1. This process has a flat spectral density σ2 f (ω) = , −π ≤ ω ≤ π. (3.6) 2π A process with this spectral density is called white noise, since each frequency contributes equally to the variance of the process. Therefore, the theoretical spectral σ2 density, f (ω) of a white noise process with variance σ 2 at ω = 0 is 2π while its estimate, I(ω), is always zero. A smoothed version of the periodogram, though, may be shown, under some conditions, to be a mean square consistent estimate of the true spectral density. By using a modified or smoothed estimate (Brockwell and Davis (2002), p. 354, Priestley (1981), p. 434), 1 fˆ(ω) = 2π
T −1 X
λ(h)γ(h) ˆ e−ihω
for all ω ∈ (−π, π],
h=−(T −1)
Author's Copy
(3.7)
Author's Copy
Author's Copy
I(ω) =
Author's Copy
Sum of the sample autocorrelation function
129
where λ(·) is a so-called lag window, it is possible to increase bandwidth of the estimate and to obtain a smoother estimate of the spectrum. Let us consider fˆ(ω) at ω = 0. T −1 T −1 X X 1 γ(0) ˆ fˆ(0) = λ(h)γ(h) ˆ = 1+2 λ(h)ρ(h) ˆ . (3.8) 2π 2π h=1
The smoothed version, therefore, is a function of λ(·) and γ(·) ˆ rather than γ(·). ˆ Thus, choosing of the bandwidth of some specified lag window or spectral window employed for smoothing the periodogram is very important in spectral density estimation. There exist several nonparametric methods to choose the bandwidth; the cross-validation based methods described by Hurvich (1985) and Hurvich and Beltrao (1990), an iterative procedure due to B¨uhlmann (1996), and a bootstrap approach followed by Franke and H¨ardle (1992).
4. Conclusion In this paper, we have introduced a quantity for the sum of the sample autocorrelation, Sacf , of any linear stationary process. We have proved that Sacf is always equal to − 12 for any stationary process with arbitrary length T > 2. It is therefore proven here, in an alternative form, that we require a smooth version of spectral density. The results have shown, using this quantity, that the sample spectral density of a stationary process fluctuates violently about the theoretical spectral density confirming result obtained by Priestley (1981) and Jenkins and Watts (1968)). In particular, it has been illustrated that the periodogram of a white noise process is always zero if we use ρ. ˆ Acknowledgments. The author is grateful to Professor Nikolai Leonenko for his constructive comments on an earlier version of this paper.
References 1. P. B¨uhlmann, Locally adaptive lag-window spectral estimation, J. Time Ser. Anal. 17(3) (1996), pp. 247–270. 2. P. J. Brockwell and R. A. Davis, Time Series: Theory and Methods, 2nd edition, Springer, New York, 1991. 3. P. J. Brockwell and R. A. Davis, Introduction to Time Series and Forecasting, 2nd edition, Springer, 2002. 4. J. Franke and W. H¨ardle, On bootstrapping kernel spectral estimates, Ann. Stat. 20(1) (1992), pp. 121–145. 5. G. M. Jenkins and D. G. Watts, Spectral Analysis and its Applications, Holden-Day, San Francisco, 1968.
Author's Copy
Author's Copy
Author's Copy
h=−(T −1)
Author's Copy
130
Hossein Hassani
6. C. M. Hurvich, Data-driven choice of a spectrum estimate: Extending the applicability of cross-validation methods, J. Am. Stat. Assoc. 80 (1985), pp. 933–940. 7. C. M. Hurvich and K. I. Beltrao, Cross-validatory choice of a spectrum estimate and its connection with AIC, J. Time Ser. Anal. 11 (1990), pp. 121–137. 8. E. Parzen, Mathematical considerations in the estimation of spectra, Technometrics 3 (1961), pp. 167–190. 9. M. Pourahmadi, Foundations of Time Series Analysis and Prediction Theory, Wiley, New York, 2001.
11. M. C. Schaerf, Estimation of the covariance and autoregressive structure of a stationary time series, Technical Report (1964), Dep. of Statistics, Standford Univ., Stanford, CA. 12. R. Shumway and D. Stoffer, Time Series Analysis and its Applications. With R Examples, 2nd edition, Springer, New York, 2006. Received 12 October, 2008 Author information Hossein Hassani, Centre for Optimisation and its Applications, School of Mathematics, Cardiff University, CF24 4AG, UK. Email:
[email protected]
Author's Copy
Author's Copy
Author's Copy
10. M. B. Priestley, Spectral Analysis and Time Series, Academic Press, London, 1981.
