Frequency domain bootstrap for time series Efstathios Paparoditis Abstract. The paper discusses frequency domain bootstrap methods for time series including some recent developments. Attention is focused on nonparametric resampling methods of the periodogram and their application to statistical inference in the frequency domain.
1
Introduction
It is common in time series analysis that the (limiting) distribution of some statistics of interest depends in a complicated way on certain characteristics of the underlying stochastic process making statistical inference based on large sample approximations difficult to implement. In such situations bootstrap based methods offer a potentially useful alternative. Working for instance in the time domain, one can use bootstrap methods by resampling whole blocks of consecutive observations (K¨ unsch (1989), Liu and Singh (1992)) or by applying parametric models together with i.i.d. resampling of estimated residuals (Freedman (1984), Swanepoel and van Wyk (1986), Kreiss and Franke (1992)). See also Radulovi´c (2001) in this volume. On the other hand, bootstrap methods for time series have been also used successfully in the frequency domain and in particular in resampling periodogram ordinates. The advantage of such an approach is that in order for a bootstrap procedure to generate replicates of the periodogram it is not necessary first to generate replicates of the observed series. Such a bootstrap procedure can operate directly in the frequency domain bypassing, therefore, the difficult problem of mimicing the (potentially very complicated) dependence structure of the underlying process in the time domain. This possibility is due to a basic nonparametric property of the periodogram according to which for quite a general class of stochastic processes, the periodogram ordinates behave for large sample sizes and for any fixed set of frequencies, like independent exponentially distributed random variables, the parameters of which equal the value of the spectral density function at the corresponding frequencies. Several approaches have been proposed for bootstrapping time series in the frequency domain. For Gaussian processes frequency domain resampling methods have been investigated by Nordgaard (1992), Theiler et al. (1994) and Braun and Kulperger (1997). Hartigan (1980) proposes perturbed periodogram estimates of variance. In this paper, 1
2
FREQUENCY DOMAIN BOOTSTRAP
however, we are interested in nonparametric, frequency domain methods of bootstrapping the periodogram. The periodogram is a basic tool in spectral analysis and several statistics of interest can be expressed as functionals of the periodogram; cf. the paper by Dahlhaus and Polonik (2001) and Soulier (2001) in this volume. Apart from a discussion of the different resampling methods proposed, our interest is focused on those classes of periodogram-based statistics to which they can be successfully applied. In the following we consider a stochastic process X = {Xt , t ∈ Z} that is generated by P∞ P 2 (A1) Xt = j=−∞ ψj εt−j , j∈Z j |ψj | < ∞, where (A2) {εt , t ∈ Z} is a sequence of independent, identically distributed random variables satisfying E(εt ) = 0 and E(ε8t ) < ∞. Denote by f the spectral density of X which can be expressed as f (λ) = (2π)−1 σε2 |Ψ(eiλ )|2 where Ψ(z) =
P j∈Z
ψj z j and σε2 = E[ε2t ]. We assume that
(A3) inf λ∈[0,π] f (λ) > 0. Let X1 , X2 , . . . , Xn be observations from X and denote by In (λ) the periodogram defined by n ¯2 1 ¯¯ X ¯ In (λ) = Xt e−iλt ¯ . (1.1) ¯ 2πn t=1 The periodogram In (λ) is usually calculated at the so-called Fourier frequencies λj = 2πj/n, j = 0, 1, . . . , N,
where N = [n/2].
Under assumptions (A1) and (A2) we have E[In (λj )] = f (λj )+O(n−1 ), Cov[In (λj ), In (λk )] = f 2 (λj ) + O(n−1 ) for 0 < λj = λk < π and Cov[In (λj ), In (λk )] = n−1 η4 f (λj )f (λk ) + o(n−1 ) if λj 6= λk , where η4 = κ4 /σε4 and κ4 = E[ε4t ] − 3σε4 is the fourth cumulant of εt ; cf. Anderson (1971), p. 476. In the sequel, we denote quantities (expectation, covariance, etc.) taken with respect to the corresponding bootstrap probability measures P ∗ or P + with an asterisk ∗ or with a cross + respectively. Furthermore, in order to metrize the distance between probability measures we use Mallow’s d2 metric on the space {P : P is a probability measure on (R, R B), |x|2 dP < ∞}. This metric is defined by d2 (P1 , P2 ) = inf{E|Y1 − Y2 |2 }1/2 where the infimum is taken over all real-valued random variables (Y1 , Y2 ) which have marginal distributions P1 and P2 respectively. Note that convergence in the d2 metric implies weak convergence and convergence of the first two moments; cf. Bickel and Freedman (1981).
2
Nonparametric residual-based bootstrap
The first nonparametric approach to bootstrapping the periodogram which we consider is based on the expression In (λj ) = f (λj )
In,ε (λj ) + Rn (λj ), 2πσε2
(2.1)
EFSTATHIOS PAPARODITIS
3
Pn where In,ε (λ) = (2πn)−1 | t=1 εt exp{−iλt}|2 and the remainder term Rn (λj ) satisfies under assumptions (A1) and (A2), maxλj ∈[0,π] E[Rn2 (λj )] = O(n−1 ), cf. Brockwell and Davis (1991), Proposition 10.3.1. For bootstrap purposes, the interesting aspect of expression (2.1) is that the relation between the periodogram of the observed series on the one hand and of the unknown spectral density and the (rescaled) periodogram of the unobserved i.i.d. sequence on the other hand, can be approximately described by means of a multiplicative regression model. Taking into account the asymptotic independence of the periodogram ordinates and ignoring the asymptotically negligible term Rn (λj ), this expression suggests that bootstrap replicates of the periodogram can be obtained by replacing the unknown spectral density f by an estimator fˆ and mimicing the sampling behavior of In,ε (λj )/σε2 by means of an i.i.d. resampling of appropriately defined frequency domain residuals. To implement this idea let fˆ be an estimator of f satisfying (B1) supλ∈[0,π] |fˆ(λ) − f (λ)| → 0 a.s. The algorithm used to generate bootstrap periodogram replicates In∗ (λj ), j = −N, −N + 1, . . . , N can then be described by the following two steps. bj = In (λj )/fˆ(λj ) and define the rescaled residuals STEP 1. Let U ej = U
N −1
bj U PN s=1
bs U
,
j = 1, 2, . . . , N.
e1 , U e2 , . . . , U eN . Denote by Fn,U the empirical distribution function of U STEP 2. Let Uj∗ , j = 1, 2, . . . , N be i.i.d. random variables with cumulative distribution function Fn,U (conditionally on the given observations X1 , X2 , . . . , Xn ). Define bootstrap periodogram values as In∗ (λj ) = In∗ (−λj ) = fˆ(λj )Uj∗ , j = 1, 2, . . . , N and In∗ (0) = 0. Like centering in an additive regression model, the rescaling in Step 1 ensures that E ∗ [Uj∗ ] = 1 which avoids an unnecessary bias. Elementary calculations yield E∗ [In∗ (λj )] = fˆ(λj ) and Cov∗ [In∗ (λj ), In∗ (λk )] = δj,k fˆ(λj )fˆ(λk ), where δj,k denotes Kronecker’s delta. The above procedure is called a nonparametric residual-based frequency bootstrap procedure because it resamples the periodogram by means of an i.i.d. resampling of ej , j = 1, 2, . . . , N . the (nonparametrically estimated) frequency domain “residuals” U Such an approach for bootstrapping the periodogram was proposed by Hurvich and Zeger (1987) under more restrictive (Gaussian) assumptions on the underlying process and by Franke and H¨ardle (1992) for inferring properties of kernel estimators of the spectral density. Note that exploiting our knowledge about the asymptotic distribution of the rescaled periodogram In,ε (λ)/σε2 , we may modify the above bootstrap procedure by replacing the Uj∗ , j = 1, 2, . . . , N by a a set of N independent standard exponentially distributed random variables, Vj∗ . However, for the class of statistics considered in the sequel, all results obtained hold if the bootstrap periodogram In∗ (λj ) is replaced by the modified bootstrap statistic I˜n∗ (λj ) = fˆ(λj )Vj∗ ; cf. Franke and H¨arlde (1992) and Dahlhaus and Janas (1996).
4
FREQUENCY DOMAIN BOOTSTRAP
2.1
Spectral density estimators
The first application of the above bootstrap procedure we discuss is in inferring properties of nonparametric (kernel) estimators of the spectral density. For this consider the estimator N 1 X Kh (λ − λj )In (λj ) (2.2) fˆh (λ) = n j=−N
of f (λ), where Kh (·) = h
−1
K(·/h), h > 0, and the real-valued function K satisfies
(A4) K is a symmetric, non-negative andR Lipschitz continuous kernel with compact support [−π, π] and which satisfies (2π)−1 K(u)du = 1. Furthermore, the Fourier transform κ(u) of K(u) is locally quadratic around u = 0, i.e., limu→0 {κ(0) − κ(u)}/u2 exists, is finite, and is different from zero. To ensure consistency of fˆh we require that the smoothing bandwidth h satisfies (A5) h → 0 and nh → ∞ as n → ∞.
√ Interest is now focused on estimating the distribution of the statistic Ln (λ) = nh (fˆh (λ) − f (λ))/f (λ). For this, the residual-based bootstrap algorithm can be adapted appropriately by replacing the estimator fˆ used to resample the periodogram by a kernel estimator fˆb (λ). For the smoothing bandwidth b, b > 0, we require that (B2) b → 0 and nb → ∞ as n → ∞. To estimate the distribution of Ln (λ) the bootstrap statistic L∗n (λ) = fˆb (λ))/fˆb (λ) is used, where fˆh∗ (λ) is defined by
√
nh(fˆh∗ (λ) −
N 1 X fˆh∗ (λ) = Kh (λ − λj )In∗ (λj ). n j=−N
Writing Ln (λ) as the sum of a stochastic and a bias term, i.e., Ln (λ) =
√
nh
³ fˆ (λ) − E[fˆ (λ)] ´ h h f (λ)
+
√
nh
³ E[fˆ (λ)] − f (λ) ´ h f (λ)
=: Dn (λ) + Bn (λ),
with an obvious notation for Dn (λ) and Bn (λ), we have, under assumptions (A1)(A5), that the stochastic term Dn (λ) ⇒ N (0, σf2 (λ)) as n → ∞, where σf2 (λ) = R (2π)−1 K 2 (u)du. Furthermore, if nh5 → 0 then the bias term vanishes, i.e., Bn (λ) → 0, while for h ∼ n−1/5 , which is the optimal rate from the viewpoint of minimizing an integrated mean square error criterion, the bias term Bn (λ) R is asymptotically non 00 negligible. In this case we get Bn (λ) → (4π)−1 f (λ)f −1 (λ) u2 K(u)du as n → ∞. From the previous discussion it is clear that in order for the bootstrap to provide a valid approximation of the distribution of Ln (λ), the statistic L∗n (λ) has to be able to estimate correctly the asymptotic behavior of the stochastic term Dn (λ) and of the bias term Bn (λ). To achieve the second goal in case of an asymptotically nonvanishing bias, it is essential that the smoothing bandwidth b used in the resampling stage behaves in a particular way compared to the smoothing bandwidth h used to estimate the spectral density f . The following theorem summarizes the asymptotic behavior of the nonparametric residual-based bootstrap proposal in estimating the distribution of Ln (λ); cf. Theorem 1 of Franke and H¨ardle (1992).
EFSTATHIOS PAPARODITIS
5
Theorem 2.1. Suppose that assumptions (A1)-(A5) and (B2) are satisfied. (i) If nh5 → 0 as n → ∞, then ³ ´ d2 L{Ln (λ)}, L{L∗n (λ)|X1 , X2 , . . . , Xn } → 0 in probability. (ii) If h ∼ n−1/5 and hb−1 → 0 as n → ∞, then the same result as in (i) holds true. The condition that h tends to zero faster than b stated in the second part of the above theorem, implies that the kernel estimator fˆb (λ) used for resampling purposes should be somewhat smoother compared to the kernel estimator fˆh (λ) used for estimation purposes. This kind of “oversmoothing” condition is common in applications of the bootstrap to estimate the bias in nonparametric estimation; cf. for instance Romano (1988).
2.2
Spectral means and ratio statistics
An interesting question concerning the applicability of the nonparametric residual-based bootstrap is whether apart from the class of nonparametric estimators considered, there are other classes of periodogram statistics to which this procedure can be successfully applied. To answer this question it is essential to understand the extent to which certain sampling properties of the periodogram In (λ) which are not appropriately captured by its bootstrapped version In∗ (λ), affect the sampling properties of the statistics of interest. To understand this point, consider for instance the important class of so-called spectral means. Spectral means are integrated periodogram statistics defined by Z π M (In , φ) = φ(λ)In (λ)dλ (2.3) 0
where (A6) φ : [0, π] → R is a bounded function of bounded variation. We assume that φ is extended to the real line with φ(−λ) = φ(λ) and φ(λ + 2π) = φ(λ). Several statistics useful in time series analysis are obtained as special cases ofR Mn (In , φ). π For instance, for φ(λ) = 2 cos(λh), h ∈ N0 , we get M (In , 2 cos(·h)) = 2 0 cos(λh) √ In (λ)dλ =: γˆ (h) which is a n-consistent estimator of the true underlying autocovariance γ(h) = E(Xt Xt+h ). For φ(λ) = 1[0,x] (λ), x ∈ [0, π], we get M (In , 1[0,x] ) = Rx √ I (λ)dλ =: Fn (x) which is a n-consistent estimator of the spectral distribution 0 n Rx function F (x) = 0 f (λ)dλ. Discretizing the integral in (2.3) along the Fourier frequencies leads to a computationally more tractable version of M (In , φ) which is given by N 2π X MD (In , φ) = φ(λj )In (λj ). (2.4) n j=1 Under some smoothness assumptions on φ the difference between the above statistic and (2.3) is asymptotically negligible. From Dahlhaus (1985) it is known that ´ √ ³ 2 n M (In , φ) − M (f, φ) ⇒ N (0, σM (φ)) (2.5)
6
FREQUENCY DOMAIN BOOTSTRAP
where
Z 2 σM (φ) = 2π
π
φ2 (λ)f 2 (λ)dλ + η4
0
³Z
π
φ(λ)f (λ)dλ
´2
(2.6)
0
and η4 = κ4 /σε4 . The interesting fact here is that the second term on the right hand side of (2.6) is due to the (asymptotically vanishing) covariance of the periodogram ordinates, i.e., for the class of spectral means (2.3), the slight dependence of the periodogram 2 ordinates sums up to a non-vanishing contribution to the asymptotic variance σM (φ). ∗ Now, let MD (In , φ) be the bootstrap analogue of MD (In , φ) obtained by replacing In by the bootstrap periodogram In∗ . By the independence of the periodogram ordinates and the uniform convergence (B1) it easily follows that Z π ∗ ∗ φ2 (λ)f 2 (λ)dλ, nVar [MD (In , φ)] → 2π 0
in probability. Comparing this with (2.6) shows that apart from two specific cases where (i) κ4 = 0, which is for instance true if εt is Gaussian, or (ii)
Rπ 0
φ(λ)f (λ)dλ = 0,
the nonparametric residual-based periodogram bootstrap fails to estimate correctly the limiting variance of this class of statistics. Since the above failure of the nonparametric residual-based bootstrap is due to the fact that the bootstrap periodogram ordinates are independent while the ordinary periodogram ordinates are not, we expect that this bootstrap proposal will work for statistics for which the weak and asymptotically vanishing dependence of the periodogram ordinates does not affect characteristics of their (limiting) distribution. There are some different classes of statistics for which this is true. One such class is that of nonparametric (kernel) spectral density estimators considered in the previous section. Here the O(n−1 )-vanishing covariance of the periodogram does not affect the limiting variance σf2 (λ) because of the slower, namely n−1/2 h−1/2 , convergence rate of these estimators. We expect that the same will be true for other linear statistics based on the periodogram −1/2 provided their convergence . An interesting question is, however, √ rate is less than n if there exist classes of n-consistent statististics to which the nonparametric residualbased periodogram bootstrap can be successfully applied also in the non-Gaussian case. Dahlhaus and Janas (1996) showed that one important class of such statistics is that of the so-called ratio statistics. Ratio statistics which are defined as the ratio of two spectral means, i.e., M (In , φ) R(In , φ) = , (2.7) M (In , 1) have the desired property that their asymptotic distribution is not √ affected by the weak dependence of the periodogram. In particular, since the statistic n(R(In , φ)−R(f, φ)) can be written as a normalized, integrated periodogram type statistic, i.e., Z π ´ √ √ ³ 1 Rπ n R(In , φ) − R(f, φ) = n R π w(λ)In (λ)dλ, f (λ)dλ 0 In (λ)dλ 0 0 Rπ Rπ Rπ with w := φ 0 f − 0 φf , we have because 0 w(λ)f (λ)dλ = 0 that the asymptotic variance of (2.7) does not depend on the fourth cumulant of the error process. In fact, ´ √ ³ 2 n R(In , φ) − R(f, φ) ⇒ N (0, σR (φ)) (2.8)
EFSTATHIOS PAPARODITIS as n → ∞, where
7 Z
2 σR (φ) = 2π
0
π
w2 (λ)f 2 (λ)dλ/
³Z
π
f 2 (λ)dλ
´4
;
0
cf. Dahlhaus and Janas (1996). Note that some statistics useful in time series analysis are obtained as special cases of (2.7). For instance, γ (0) =: ρˆ(h) is the sample autocorrelation at √ R(In , 2 cos(·h)) = γˆ (h)/ˆ lag h, which is a n-consistent estimator of√ the lag-h autocorrelation ρ(h) = γ(h)/γ(0). Furthermore, R(In , 1[0,x] ) =: Gn (x) is a n-consistent estimator of the normalized spectral distribution function G(x) = F (x)/F (π). By comparing Edgeworth expansions for the ratio statistics and their bootstrap versions it was shown by Dahlhaus and Janas (1996) that for this class of statistics the nonparametric residual-based periodogram bootstrap outperforms the classical approach based on the large sample Gaussian approximation (2.8). To make this statement precise, the following assumptions are imposed. (A7) E[ε3t ] = 0. b (A8) The filter coefficients {ψj } and the Fourier coefficients {φ(u)} of φ satisfy |j| |u| b |ψj | ≤ τ and for large u, |φ(u)| ≤ τ , where τ ∈ (0, 1) fixed. (A9) (εt , ε2t ) satisfies Cram´er’s Condition, i.e., δ > 0 and d > 0 exist such that for 0 0 all ktk > d, |E[exp{it (εt , ε2t ) }| ≤ 1 − δ. Pn (A10) Let dn (λ) := t=1 Xt exp{−iλt} be the finite Fourier transform of X1 , X2 , . . . , Xn . For the finite Fourier transform ´0 1 ³ 2π 2π 2π Yn := √ dn ( j(1)), dn ( j(2)), . . . , dn ( j(8)) , j(s) ∈ {1, 2, . . . , n/2 − 1} n n n n R and for Vn := (φ(λ), 1)In (λ)dλ, the limits of the corresponding dispersion matrices R 0 exist and are positive definite. Furthermore, (φ(λ), 1) (φ(λ), 1) f 2 (λ)dλ is positive definite. The following theorem summarizes the asymptotic properties of the bootstrap statis√ √ tic Tn∗ = n(RD (In∗ , φ)−RD (fˆ, φ)) in estimating the distribution of Tn = n(R(In , φ)− R(f, φ)). Here RD (In∗ , φ) := MD (In∗ , φ)/MD (In∗ , 1) and RD (fˆ, φ) := MD (fˆ, φ)/MD (fˆ, 1), where MD (·, ·) is defined in (2.4). Theorem 2.2. Suppose that the assumptions (A1)-(A3), (A6)-(A10) and (B1) hold. Then for almost all samples X1 , X2 , . . . , Xn , ¯ ¯ ¯ ˆ n Tn∗ ∈ C)¯¯ = o(n−1/2 ), sup ¯P (Dn Tn ∈ C) − P ∗ (D C∈C
2 −1 ˆ 2 ˆ−1 where C is the class of convex √ measurable sets√C ⊆ R,∗ Dn = Sn , Dn = Sn and Sn ˆ and Sn are the variances of nR(In , φ), and nRD (In , φ) respectively.
2.3
Whittle estimators
Whittle estimators are obtained by minimizing the distance Z πn In (λ) o 1 log f (λ, θ) + dλ Ln (θ) = 2π 0 f (λ, θ)
8
FREQUENCY DOMAIN BOOTSTRAP
between the periodogram In and the parametric spectral density f (·, θ). We assume that f (λ, θ) = τ 2 g(λ, ϑ) where θ = (τ 2 , ϑ), f (·, θ) ∈ F and F = R{f (·, θ) : θ ∈ Θ}. π Furthermore, we require that Kolmogorov’s formula holds, that is, −π log f (λ, θ)dλ = 2π log(τ 2 /2π); cf. Brockwell and Davis (1991), p. 191. In the following we do not assume that f ∈ F, i.e., we allow the model to be misspecified. In this case the Whittle ˆ converges to θ0 = (τ 2 , ϑ0 ) where the later is the minimum of estimator θˆ = (ˆ τ 2 , ϑ) 0 Z πn 1 f (λ) o L(θ) = log f (λ, θ) + dλ. 2π 0 f (λ, θ) The estimator θˆ fulfills the equations Z π ¯ ∂ ¯ In (λ) f −1 (λ, θ)¯ dλ = 0 ∂ϑ θ=θˆ 0
Z and
π
π −1
ˆ In (λ)f −1 (λ, θ)dλ =1
0
while its asymptotic limit θ0 satisfies the equations Z π Z π ¯ ∂ ¯ dλ = 0 and π −1 f (λ) f −1 (λ, θ)¯ f (λ)f −1 (λ, θ0 )dλ = 1. ∂ϑ θ=θ 0 0 0 Now, apart from the class of ratio√statistics, the innovation free part of the Whittle ˆ that is ϑ, ˆ forms another n-consistent statistic to which the residual-based estimator θ, periodogram bootstrap can be successfully applied. Note that this is not the case for the variance estimator τˆ2 since the limiting variance of√this estimator depends on the ˆ fourth cumulant κ4 . To approximate the distribution of n(ϑ−ϑ) the bootstrap statistic √ ˆ∗ ˜ ∗ ∗2 ˆ∗ ˆ n(ϑ − ϑ) is used, where θ = (ˆ τ , ϑ ) is the minimizer of N
L∗n (θ) =
1 1 X ∗ log(τ 2 /2π) + I (λj )f −1 (λj , θ) 2 2n j=1 n
˜ the minimizer of L(θ), ˜ and θ˜ = (˜ τ 2 , ϑ) which is defined as L∗n (θ) after replacing the ∗ bootstrap periodogram In (λj ) by the spectral density estimator fˆ(λj ). To deal with the class of Whittle estimators the following assumptions are needed. (A11) Θ is a compact subset of R+ ×Rp and the parameter ϑ satisfy the identifiability condition that if ϑ1 6= ϑ2 then f (·, ϑ1 ) 6= f (·, ϑ2 ) on a set with positive Lebesgue measure. (A12) The function g(λ, ϑ) is three times continuously differentiable with respect to ϑ and every component of its derivative is continuous in λ. 0
(A13)R Let cϑ (·) = ((∂/∂ϑ)g −1 (·, ϑ), (∂/∂ϑ)2 g −1 (·, ϑ)) . The limit of dispersion R the 0 0 0 0 0 matrix (cϑ (λ), 1) In (λ)dλ exists and is positive definite. Furthermore, (cϑ (λ), 1) (cϑ (λ), 1)f 2 (λ)dλ is positive definite. ˜ the following result can then Using a Taylor series expansion of L∗n (θ) around θ = θ, be established; cf. Dahlhaus and Janas (1996), Theorem 6. Theorem 2.3. If (A1)-(A3), (A11)-(A13) and (B1) are satisfied, then ³ √ ´ √ ˜ 1 , X2 , . . . , Xn } → 0 d2 L{ n(ϑˆ − ϑ)}, L{ n(ϑˆ∗ − ϑ)|X in probability.
EFSTATHIOS PAPARODITIS
9
Improving the above result for the case where the dimension of ϑ equals one (i.e., p = 1), it was shown by Dahlhaus and Janas (1996) under some additional assumptions that √ ˜ of the distribution of √n(ϑ−ϑ) ˆ the bootstrap approximation n(ϑˆ∗ − ϑ) is second order correct, i.e., that the nonparametric residual-based periodogram bootstrap outperforms the asymptotic Gaussian approximation.
3
Local bootstrap
An alternative way to bootstrap the periodogram and which does not require initial estimation of the unknown spectral density f and i.i.d. resampling of (estimated) innovations, can be motivated by the following observation. Recall that for large n, In (λ) is approximately exponentially distributed with parameter f (λ). Assuming that the spectral density f (λ) is a smooth function of λ, we expect that the sampling behavior of the periodogram at any particular frequency λs will be very similar for periodogram ordinates corresponding to frequencies in a small neighborhood of λs . This implies that periodogram replicates at a frequency λs can be obtained by locally resampling the periodogram, i.e., by choosing with replacement between periodogram ordinates corresponding to frequencies ‘close’ to the frequency λs of interest. Borrowing ideas from nonparametric estimation, quantification of closeness in such a resampling scheme can be done by means of a resampling bandwidth. Consistency of this resampling procedure for interesting classes of periodogram statistics can then be established by allowing the neighborhood around λs (from which replicates are obtained) to get narrower as the sample size increases but in such a way that the number of periodogram ordinates falling in this neighborhood increases to infinity. The algorithm used to implement the above idea of generating bootstrap replicates of the periodogram and which are denoted by In+ (λj ), j = −N, −N + 1, . . . , N , can be described by the following three steps. STEP 1. Select a resampling width Bn where Bn = B(n) ∈ N and Bn ≤ [N/2]. STEP 2. Define i.i.d. discrete random variables J1 , J2 , . . . , JN taking values in the set {−Bn , −Bn +1, . . . , Bn } with probability mass function pn,s , i.e., P (Ji = s) = pn,s for s = 0, ±1, . . . , ±Bn . STEP 3. The bootstrap periodogram is then defined by In+ (λj ) = In+ (−λj ) = In (λJj +j ),
for j = 1, 2, . . . , N
and In+ (0) = 0. Such an approach for bootstrapping the periodogram was proposed by Paparoditis and Politis (1999). The idea underlying the above ‘local’ resampling scheme can be considered as a modification of the ordinary bootstrap idea of ‘global’ resampling, i.e., resampling from the whole set of observations and can be applied when the distribution of the sequence of random variables to be resampled changes smoothly with respect to their indexing; see Shi (1991), Falk and Reiss (1992) and Paparoditis and Politis (2000).
10
FREQUENCY DOMAIN BOOTSTRAP
The following condition states our requirements on the limiting behavior of the resampling width Bn and of the resampling probabilities pn,s . (L1) Bn → ∞ as n → ∞ such that Bn /n → 0. Furthermore, the resampling probabilities {pn,s : s = −Bn , −Bn + 1, . . . , Bn } satisfy Bn X
pn,s = 1,
pn,s = pn,−s ,
and
s=−Bn
Bn X
p2n,s → 0, as n → ∞.
s=−Bn
Note that one way to obtain the resampling probabilities pn,s of the above resampling PBn scheme is by pn,s = K(πs/Bn )/ s=−B K(πs/Bn ) where K is any commonly used n kernel satisfying assumption (A4). If assumptions (A1)-(A3) and (L1) are fulfilled then we get as n → ∞, that sup d2 (L{In+ (λj )|X1 , X2 , . . . , Xn }, L{In (λj )}) → 0
λj ∈(0,π)
in probability. This suggests that sampling properties of statistics based on the periodogram In can be approximated by the corresponding properties of the same statistic based on the local bootstrapped periodogram In+ . Note, however, that the idea of local resampling relies on the asymptotic independence of the periodogram ordinates. In fact the bootstrap periodogram ordinates In+ (λj ) are independent. Simple calculations yield E+ [In+ (λj )] = f˜(λj ) and Cov+ [In+ (λj ), In+ (λk )] = δj,k f˜(λj )f˜(λk ) where PBn pn,s In+ (c(n, λ) + λs ) and c(n, λ) denotes the multiple of 2π/n closest to f˜(λ) = s=−B n λ (the smallest one if there are two). If λ ∈ [−π, 0) we set c(n, λ) = c(n, −λ). Because of the independence of the ordinates In+ (λj ) and as in the case of the nonparametric residual-based bootstrap, we expect that inference based on the local resampling procedure can lead to correct answers only as long as the existing weak dependence between the periodogram ordinates In (λj ) does not affect the sampling characteristics of the class of statistics considered. Consistency properties for such classes of periodogram-based statistics are briefly discussed in the next sections.
3.1
Spectral density estimators
Using the local bootstrap approach of the periodogram, the distribution of the statistic Ln (λ) considered in Section 2.1 √ can be approximated by means of the distribution of + the bootstrap statistic Ln (λ) = nh(fˆh+ (λ) − f˜(λ))/f˜(λ), where fh+ (λ)
N 1 X = Kh (λ − λj )In+ (λj ). n j=−N
We then have the following result; cf. Paparoditis and Politis (1999), Theorem 3.2. Theorem 3.1. Suppose that (A1)-(A5) and (L1) are satisfied. (i) If nh5 → 0 as n → ∞, then ³ ´ d2 L{Ln (λ)}, L{L+ n (λ)|X1 , X2 , . . . , Xn } → 0 in probability.
EFSTATHIOS PAPARODITIS (ii) If h ∼ n−1/5 and nh holds true.
PBn s=−Bn
11 p2n,s → 0 as n → ∞, then the same result as in (i)
PBn Note that the assumption nh s=−B p2n,s → 0 imposed in the second part of the n above theorem is an oversmoothing assumption similar to the one imposed in the second part of Theorem 2.1. To see this, consider the simple case of uniform resampling probabilities, that is pn,s = 1/(2Bn + 1). In this case the above condition simplifies to nh/(2Bn + 1) → 0. Taking into account the fact that nh are approximately the number of periodogram ordinates effectively used in fˆh (λ) to estimate f (λ), this implies that 2Bn + 1 increases faster than nh, i.e., f˜(λ) is somewhat smoother than fˆh (λ).
3.2
Ratio statistics
√ For the class of ratio statistics the distribution of Tn = n(R(In , φ)−R(f, φ)) is approxi√ mated by that of Tn+ = n(RD (In+ , φ)−RD (f˜, φ)) where RD (In+ , φ) = MD (In+ , φ)/MD (In+ , 1) and MD (·, ·) is defined in (2.4). The following result can then be established; cf. Paparoditis and Politis (1999), Theorem 3.1. Theorem 3.2. Suppose that assumptions (A1)-(A3), (A6), (A10) and (L1) hold true. As n → ∞, ³ ´ d2 L{Tn }, L{Tn+ |X1 , X2 , . . . , Xn } → 0 in probability.
3.3
Parametric fits in the frequency domain
To approximate the distribution of the innovation √ √ free part of the Whittle estimator, i.e., of n(ϑˆ − ϑ), the local bootstrap estimator n(ϑˆ+ − ϑ˜+ ) can be used, where ϑˆ+ is obtained by minimizing the function N
L+ n (θ) =
1 1 X + log(τ 2 /2π) + I (λj )f −1 (λj , θ) 2 2n j=1 n
˜ + (θ), where L ˜ + (θ) is defined as L+ and ϑ˜ by minimizing L n (θ) but by replacing the + local bootstrap periodogram In (λj ) by f˜(λj ). We then have the following result; cf. Paparoditis and Politis (1999), Theorem 3.3. Theorem 3.3. If (A1)-(A3), (A11)-(A13) and (L1) are satisfied, then ³ √ ´ √ d2 L{ n(ϑˆ − ϑ)}, L{ n(ϑˆ+ − ϑ˜+ )|X1 , X2 , . . . , Xn } → 0 in probability. We mention here that apart from the class of Whittle estimators, the local bootstrap can be successfully applied to approximate the distribution of the innovation free part of more general classes of frequency domain parameter estimators, a result which can be also established for the nonparametric residual-based periodogram bootstrap. Such estimators are obtained by fitting a model from the parametric class F = {f (λ, θ) : θ ∈ Θ} to the process X via minimizing the objective function Z D(f (λ, θ)/f n (λ))dλ. L(f (·, θ), f n ) = [−π,π]
12
FREQUENCY DOMAIN BOOTSTRAP
Here f n denotes an estimator of f and D(x) is a three times continuously differentiable function on (0, ∞) which has a unique minimum at x = 1. Note that for f n (λ) = In (λ) and D(x) = log(x)+x−1 , we are in the case of the Whittle estimators. Taniguchi (1987) investigated properties of such parameter estimators for different choices of the function D(·) and by allowing f n to be a nonparametric estimator of f .
4
Capturing periodogram dependence
From the previous discussion it is clear that in order to extend the class of periodogrambased statistics for which frequency domain bootstrap methods work, it is necessary to take into account the weak dependence structure of the periodogram ordinates. In fact, the statistics considered so far and for which validity of the nonparametric periodogram bootstrap procedures has been established, either obey a convergence rate of −1/2 order less than R π can be expressed as normalized, integrated periodogram √ n R Ror they statistics, n ω( f, In ) 0 ψ(λ)In (λ)dλ for some function ψ(·) which satisfies the Rπ condition 0 ψ(λ)f (λ)dλ = 0. This condition ensures that the asymptotic variance of the corresponding statistic does not depend on the fourth cumulant of the error process. Thus, the independence of the bootstrap periodogram ordinates In∗ and In+ which was justified by the asymptotic independence of the ordinary periodogram, turns out to provide a major limitation in terms of classes of statistics to which these methods can be successfully applied. One approach to extend the class of statistics for which frequency domain bootstrap methods work, is by modifying appropriately the bootstrap procedures considered in the previous sections. For instance, and for the residual-based bootstrap, such an approach was proposed by Janas and Dahlhaus (1994). Using a nonparametric estimator of the quantity η4 , the bootstrap periodogram In∗ is modified in a way that emulates also the covariance of In . In particular, for j = 1, 2, . . . , N , a modified bootstrap periodogram is defined by N ³ ´ X p Ien∗ (λj ) = fˆ(λj ) Uj∗ + { 1 + ηˆ4 /2 − 1}n−1 (Us∗ − 1) , s=1
where
Z ³ ηˆ4 = 2π fˆ2 (0) − 4π
π −π
´ ³Z fˆ2 (λ)dλ /
π
´2 fˆ(λ)dλ .
−π
Here f2 (·) denotes the spectral density of the squared process {Xt2 , t ∈ Z} while fˆ and fˆ2 are consistent estimators of f and f2 respectively. Note that apart from the problems associated with estimating nonparametrically the complicated functional η4 , such a procedure has some additional drawbacks in that it does not lead to correct estimates of the variance if one wants to apply an asymptotically nonvanishing data taper and of the skewness; see Janas and Dahlhaus (1994) and Dahlhaus and Janas (1996). A different approach to the bootstrap of the periodogram, called the autoregressive aided periodogram bootstrap, has been recently proposed by Kreiss and Paparoditis (2001). This method combines a time domain and a frequency domain resampling approach, where a parametric fit in the time domain is used to generate periodogram ordinates that imitate the essential features of the data and the weak dependence structure of the periodogram while a nonparametric (kernel based) correction in the frequency domain is applied in order to catch features not represented by the parametric
EFSTATHIOS PAPARODITIS
13
fit. The bootstrap procedure used to generate pseudo-replicates of the periodogram can be described along the following five steps. STEP 1. Fit a linear autoregressive process of order P to the observations X1 , X2 , . . . , Xn , 0 ˆ(P ) = (ˆ where P may depend on the sample. Let a a1 (P ), a ˆ2 (P ), . . . , a ˆP (P )) and σ ˆ 2 (P ) be the estimators of the autoregressive coefficients and of the error variance respectively. Consider the estimated residuals εˆt = Xt −
P X
a ˆj (P )Xt−j ,
t = P + 1, P + 2, . . . , n,
j=1
and let Fˆn be the empirical distribution function of the centered Pn quantities ε˜t , t = P + 1, P + 2, . . . , n, i.e., ε˜t := εˆt − ε where ε = (n − P )−1 t=P +1 εˆt . STEP 2. Generate bootstrap observations X1† , X2† , . . . , Xn† , according to the autoregressive model P X † Xt† = a ˆj (P )Xt−j + ε†t , j=1
where {ε†t } is a sequence of i.i.d. random variables with distribution function Fˆn (conditionally on the observations X1 , X2 , . . . , Xn ). Note that the bootstrap process X† = {Xt† , t ∈ Z} possesses the spectral density fˆAR (λ) = (2π)−1 σ ˆ 2 (P )|1 − PP ˆj (P ) exp{−ijλ}|−2 for λ ∈ [0, π], which is always well-defined if, for inj=1 a 0 stance, (ˆ a1 (P ), a ˆ2 (P ), . . . , a ˆP (P )) are the Yule-Walker estimators. STEP 3. Compute the periodogram of the bootstrap observations, i.e., In† (λ) =
n 1 ¯¯ X † −iλt ¯¯2 X e ¯ ¯ , λ ∈ [0, π]. 2π t=1 t
STEP 4. Let q(λ) := f (λ)/fAR (λ) where fAR (λ) = (2π)−1 σ 2 (P )|1 −
P X
aj (P ) exp{−ijλ}|−2 .
j=1
Here a1 (P ), a2 (P ), . . . , aP (P ) are the coefficients of Xt−1 , Xt−2 , . . . Xt−p in the ˆ t of Xt on the closed span sp{Xt−1 , Xt−2 , . . . Xt−p } and orthogonal projection X 2 2 ˆ σ (P ) := E[Xt −Xt ] . Let qˆ(λ) be an estimator of q(λ) satisfying supλ∈[0,π] |ˆ q (λ)− q(λ)| → 0 a.s., as n → ∞. For instance, qˆ(λ) could be the kernel estimator qˆ(λ) =
N 1 X In (λj ) , λ ∈ [0, π]. Kh (λ − λj ) ˆ n fAR (λj ) j=−N
STEP 5. Using In† (λ) and qˆ(λ), the bootstrap periodogram is defined by In? (λj ) = qˆ(λj )In† (λj ),
for j = 1, 2, . . . , N.
14
FREQUENCY DOMAIN BOOTSTRAP
To see the differences between the nonparametric bootstrap procedures considered so far and the new proposal, recall that using f (λ) = q(λ)fAR (λ), we have under the assumptions made that In (λj ) = q(λj )fAR (λj )
In,ε (λj ) + Rn (λj ), 2πσε2
see (2.1). A similar expression is also valid for In? (λj ), i.e., In? (λj ) = qˆ(λj )fˆAR (λj )
In,ε† (λj ) + Rn? (λj ), 2πˆ σ 2 (P )
Pn where In,ε† (λj ) = (2πn)−1 | t=1 ε†t exp{−iλt}|2 is the periodogram of the i.i.d. series ε†1 , ε†2 , . . . , ε†n and Rn? (λj ) satisfies maxj E? [Rn? (λj )]2 = OP (n−1 ); cf. Lemma 7.3 of Kreiss and Paparoditis (2001). Furthermore, since the periodogram ordinates In† (λj ) are not independent, the same is also true for In? (λj ). In fact it can be shown that (conditional on the observed series), E? [In? (λj )] = f˘(λj ) + OP (n−1 ), Cov? [In? (λj ), In? (λk )] = f˘2 (λj )+OP (n−1 ) for 0 < λj = λk < π and Cov? [In? (λj ), In? (λk )] = n−1 ηˆ4 (P )f˘(λj )f˘(λk )+ oP (n−1 ) for λj 6= λk , where f˘(λ) = qˆ(λ)fˆAR (λ) and ηˆ4 (P ) = (E† [˜ εt ]4 − 3ˆ σ 4 (P ))/ˆ σ 4 (P ). Note that P f˘(λ) is a uniformly consistent estimator of f (λ) and E † [˜ εt ]4 − 3ˆ σ 4 (P ) = n σ 4 (P ) is the fourth cumulant of ε˜t . (n − P )−1 t=P +1 ε˜4t − 3ˆ The essential feature of this new periodogram bootstrap proposal is that for periodogram statistics for which the dependence of the periodogram ordinates does not affect characteristics of their asymptotic distribution (like nonparametric spectral density estimators, ratio statistics, parametric fits in the frequency domain), the procedure works under the same set of process assumptions as those required for the nonparametric bootstrap methods considered in this paper. Furthermore, for important classes of stochastic processes the procedure leads to asymptotically valid approximations for more general classes of periodogram statistics including spectral means; see Kreiss and Paparoditis (2001) for details.
5
Goodness-of-fit testing
Frequency domain bootstrap procedures have also enjoyed success in testing the fit of a parametric time series model. The basic problem considered here is that of testing the composite hypothesis H0 : f ∈ F Θ
against
H1 : f 6∈ FΘ ,
where FΘ = {f (·, θ), θ ∈ Θ} denotes a class of parametric spectral density models and Θ is a parameter set. Interesting classes of test statistics can be obtained by comparing nonparametric estimators of certain spectral characteristics of the observed process with those of the parametric class postulated under the null hypothesis. For instance, in time series analysis, goodness-of-fit tests based on comparisons between hypothesized and postulated spectral distribution functions have a long history; cf. Priestley (1981) and Anderson (1993). Although a well developed asymptotic theory exist for such tests, the asymptotic distribution obtained is hard to implement. Bootstrap methods offer an alternative and potentially more powerful way to estimate the sampling behavior of the corresponding test statistics under the null.
EFSTATHIOS PAPARODITIS
15
In conjunction with the bootstrap, several goodness-of-fit statistics have been considered in the literature which can be expressed as functionals of the periodogram and in particular of the empirical process Dn (λ) =
ˆ o √ n Fn (λ) F (λ, θ) n − , ˆ Fn (π) F (π, θ)
λ ∈ [0, π]
or of the (kernel) smoothed deviations dn (λ) =
N ³ I (λ ) ´ 1 X n j Kh (λ − λj ) −1 , ˆ n f (λj , θ)
λ ∈ [0, π].
j=−N
ˆ denotes the estimated parametric spectral density obtained by replacing Here f (λ, θ) R √ ˆ = x f (λ, θ)dλ ˆ θ by a n-consistent estimator θˆ and F (x, θ) is the corresponding es0 timated parametric spectral distribution function. Test statistics based on Dn (·) and using a parametric time domain bootstrap with resampling of estimated residuals, was considered by Chen and Romano (1999). Note that such a time domain bootstrap requires that the parametric model for the spectral density under the null can be appropriately translated to the time domain. Alternatively, one can use directly a frequency domain bootstrap approach. Hainz and Dahlhaus (1999) proposed a frequency domain bootstrap for test statistics which are smooth functionals of Dn (·) when θˆ is the Whittle estimator. Paparoditis (2000) uses a frequency domain bootstrap in order to approximate the distribution of L2 -type statistics based on dn (·). Since Dn (·) and dn (·) are scale free, i.e., they are not affected by the innovation variance, and dn (λ) = OP (n−1/2 h−1/2 ), an appropriately modified frequency domain periodogram bootstrap, like the nonparametric residual-based bootstrap, which generates independent periodogram replicates can be successfully applied. The modification uses the estimated parametric spectral ˆ together with standard exponentially distributed innovations to gendensity f (λ, θ) erated the pseudo-replicates of the periodogram instead of using the nonparametric ej as in Section 2. This is estimated spectral density fˆ(λ) and the rescaled residuals U important for a good power performance of the methods; cf. Hainz and Dahlhaus (1999) and Paparoditis (2000). Such a frequency domain periodogram bootstrap leading to independent periodogram ordinates is, however, not valid if the test statistic used, depends on the innovation variance. This is for instance true if instead of the normalized process Dn (·) the test statistic √ ˆ An appropriate modused is a functional of the empirical process n(Fn (λ) − F (λ, θ)). ification for this case has been proposed by Hainz and Dahlhaus (1999) along the lines suggested in Janas and Dahlhaus (1994).
16
FREQUENCY DOMAIN BOOTSTRAP
Bibliography [1] T. W. Anderson: The Statistical Analysis of Time Series. John Wiley, N. York, 1971. [2] T. W. Anderson: Goodness of fit tests for spectral distributions. Annals of Statistics 21 830-847. [3] P. J. Bickel and D. A. Freedman: Some asymptotic theory for the bootstrap. The Annals of Statistics 9 (1981), 1196-1217. [4] J. W. Braun and R. J. Kulperger: Properties of a Fourier bootstrap method for time series. Communications in Statistics, Theory and Methods 276 (1997), 1329-1336. [5] P. Brockwell and R. Davis: Time Series: Theory and Methods. Springer-Verlag, N. York, 1991. [6] H. Chen and J. Romano: Bootstrap-assisted goodness-of-fit tests in the frequency domain. Journal of Time Series Analysis 20 (1999), 619-654. [7] R. Dahlhaus: Asymptotic normality of spectral estimates. Journal of Multivariate Analysis 16 (1985), 412-431. [8] R. Dahlhaus and D. Janas: A frequency domain bootstrap for ratio statistics in time series analysis. The Annals of Statistics 24 (1996), 1934-1963. [9] R. Dahlhaus and W. Polonik: Empirical spectral processes and nonparametric maximum likelihood estimation for time series. In: Empirical Process Techniques for Dependent Data. (H. Dehling, T. Mikosch, M. Søresen, eds.) Birkh¨auser: Boston, 2001. [10] M. Falk and R.-D. Reiss: Bootstrapping conditional curves. In: Bootstrapping and Related Techniques, (K. H. J¨ockel, G. Rothe and W. Sendler, eds.). Lecture Notes in Economics and Mathematical Systems, 376, Springer Verlag: Berlin, (1992), 173-180. [11] J. Franke and W. H¨ardle: On bootstrapping kernel spectral estimates. The Annals of Statistics 20 (1992), 121-145. [12] D. A. Freedman: On bootstrapping two-stage least-squares estimates in stationary linear models. The Annals of Statistics 12 (1984), 827-842 [13] G. Hainz and R. Dahlhaus: Spectral domain bootstrap tests for stationary time series. Beitr¨ age zur Statistik 61, Universit¨ at Heidelberg, (1999). 17
18
FREQUENCY DOMAIN BOOTSTRAP
[14] J. A. Hartigan: Perturbed periodogram estimates of variance. International Statistical Review 58 (1990), 1-7. [15] C. M. Hurvich and S. L. Zeger: Frequency domain bootstrap methods for time series. Technical Report # 87-115, (1987), Graduate School of Business Administration, New York University. [16] D. Janas and R. Dahlhaus: A frequency domain bootstrap for time series. In: Computationally Intensive Statistical Methods. Proceedings of the 26th Symposium on the Interface (J. Sall and A. Lehman, eds.). Interface Foundation of North America, Faifax Station, VA, (1994), 423-425. [17] J.-P. Kreiss and J. Franke: Bootstrapping stationary autoregressive moving-average models. Journal of Time Series Analysis 13 (1992), 297-313. [18] J.-P. Kreiss and E. Paparoditis: Autoregressive aided periodogram bootstrap. Discussion Paper No 60, SFB 373, Humboldt Universit¨ at Berlin, (2001). [19] H. K¨ unsch: The jackknife and the bootstrap for general stationary observations. The Annals of Statistics 17 (1989), 1217-1241. [20] R. Liu and C. Singh: Moving blocks jackknife and bootstrap capture weak dependence. In: Exploring the Limits of the Bootstrap (R. LePage and L. Billard, eds.), Wiley: New York, (1992), 225-248. [21] A. Nordgaard: Resampling stochastic processes using a bootstrap approach. In: Bootstrapping and Related Techniques. (K. H. J¨ockel, G. Rothe and W. Sendler, eds.). Lecture Note in Economics and Mathematical Systems, 376, Springer, Berlin, (1992), 181-185. [22] E. Paparoditis and D. N. Politis: The local bootstrap for periodogram statistics. Journal of Time Series Analysis 20 (1999), 193-222. [23] E. Paparoditis: Spectral density based goodness-of-fit tests for time series models. Scandinavian Journal of Statistics 27 (2000), 143-176. [24] E. Paparoditis and D. N. Politis: The local bootstrap for kernel estimators under general dependence conditions. Annals of the Institute of Statistical Mathematics 52 (2000), 139-159. [25] D. Radulovi´c: On the bootstrap and empirical processes for dependent sequences. In: Empirical Process Techniques for Dependent Data. (H. Dehling, T. Mikosch, M. Søresen, eds.) Birkh¨auser: Boston, 2001. [26] J. P. Romano: Bootstrapping the mode. Annals of the Institute of Statistical Mathematics 40 (1988), 565-586. [27] S. G. Shi: Local bootstrap. Annals of the Institute of Statistical Mathematics 43 (1991), 667-676. [28] P. Soulier: Empirical process techniques for spectral estimation of fractional processes. In: Empirical Process Techniques for Dependent Data. (H. Dehling, T. Mikosch, M. Søresen, eds.) Birkh¨auser: Boston, 2001.
EFSTATHIOS PAPARODITIS
19
[29] J. W. H. Swanepoel and J. W. Y. van Wyk: The bootstrap applied to power spectral density function. Biometrika 73 (1986), 135-141. [30] M. Taniguchi: Minimum contrast estimation for spectral densities of stationary processes. Journal of the Royal Statistical Society, Series B 49 (1987), 315-325. [31] J. Theiler, L. S. Paul and D. M. Rubin: Detecting nonlinearity in data with long coherence times. In: Time Series Prediction. (A. Weigend and N. Gershenfeld, eds.). Reading, MA: Addison-Welsey, 1994.
Efstathios Paparoditis Department of Mathematics and Statistics University of Cyprus P.O.Box 20537 CY-1678 Nicosia Cyprus
[email protected] 
