an invariance principle for sieve bootstrap in time ... - Semantic Scholar

Econometric Theory, 18, 2002, 469–490+ Printed in the United States of America+ DOI: 10+1017+S0266466602182090

AN INVARIANCE PRINCIPLE FOR SIEVE BOOTSTRAP IN TIME SERIES JO O N Y. PAR K Seoul National University This paper establishes an invariance principle applicable for the asymptotic analysis of sieve bootstrap in time series+ The sieve bootstrap is based on the approximation of a linear process by a finite autoregressive process of order increasing with the sample size, and resampling from the approximated autoregression+ In this context, we prove an invariance principle for the bootstrap samples obtained from the approximated autoregressive process+ It is of the strong form and holds almost surely for all sample realizations+ Our development relies upon the strong approximation and the Beveridge–Nelson representation of linear processes+ For illustrative purposes, we apply our results and show the asymptotic validity of the sieve bootstrap for Dickey–Fuller unit root tests for the model driven by a general linear process with independent and identically distributed innovations+ We thus provide a theoretical justification on the use of the bootstrap Dickey–Fuller tests for general unit root models, in place of the testing procedures by Said and Dickey and by Phillips+

1. INTRODUCTION The bootstrap has become increasingly popular in econometrics and also in statistics and many other applied fields+ This is in part due to the decreased computational cost but appears to be mostly because of its practical success and wide applicability+ For a nice nontechnical introduction with an extensive survey of the bootstrap methodology, see Horowitz ~1999!+ The method of bootstrap used to be applied mainly for the analysis of cross-sectional data, but recently it has also become popular in time series applications+ In this paper, we establish an invariance principle for bootstrap samples from linear processes with independent and identically distributed ~i+i+d+! innovations+ The underlying time series is approximated by a finite autoregressive process of order increasing with the sample size, and the bootstrap samples are drawn from the centered fitted residuals+ Such a bootstrap is called sieve bootstrap by Bühlmann ~1997!, because it is based on an approximation of an infiniteI thank a co-editor and two anonymous referees for useful comments and the Department of Economics at Rice University, where I am an adjunct professor, for its continuing hospitality and secretarial support+ I am also very grateful to Don Andrews and Yoosoon Chang for helpful discussions and constructive suggestions, which have greatly improved the exposition of an earlier version of this paper+ This research was supported by the Statistical Research Center for Complex Systems at Seoul National University+ Address correspondence to: Joon Y+ Park, School of Economics, Seoul National University, Seoul 151-742, Korea+

© 2002 Cambridge University Press

0266-4666002 $9+50

469

470

JOON Y. PARK

dimensional and nonparametric model by a sequence of finite-dimensional parametric models+ The sieve bootstrap has been studied earlier by Kreiss ~1992!, Bühlmann ~1997, 1998!, and Bickel and Bühlmann ~1999!+ Along with the block bootstrap by Künsch ~1989!, it provides a standard tool for the bootstrap from dependent samples+ Our work is most closely related in its aim to Bühlmann ~1997! and Bickel and Bühlmann ~1999!+ The former proves the bootstrap consistency for a class of nonlinear estimators, whereas the latter derives a bootstrap functional central limit theorem under a bracketing condition+ They are of the weak form that holds only in probability+ The invariance principle we establish in the paper concerns the weak convergence of the bootstrap partial sum process to Brownian motion+ None of the work cited previously deals with this type of invariance principle+ It is of the strong form and holds almost surely for all sample realizations+ With the continuous mapping theorem, it can be used to obtain asymptotic distributions of various bootstrapped statistics without making parametric assumptions on the underlying model+ Our approach is built upon the strong approximation and the Beveridge–Nelson representation of linear processes+ The invariance principles for the i+i+d+ innovation and its bootstrap version are first developed using the strong approximation of the partial sum process by the standard Brownian motion, and subsequently the invariance principles for the general linear process with i+i+d+ innovations and the corresponding bootstrapped process are established by their Beveridge–Nelson representations as in Phillips and Solo ~1992!+ For the purpose of illustration, we apply our results to analyze bootstrap asymptotics for Dickey–Fuller unit root tests+ In particular, we prove the asymptotic validity of bootstrap Dickey–Fuller tests for models generated by linear processes with i+i+d+ innovations+ The Dickey–Fuller test statistics have the same limiting distributions as the corresponding bootstrap test statistics obtained by the sieve bootstrap procedure+ One can therefore use Dickey–Fuller tests even when the underlying models are driven by innovations that are serially correlated, if the tests are based on the bootstrapped critical values+ Thus bootstrap Dickey–Fuller tests can be an alternative to the tests by Said and Dickey and by Phillips, which were proposed to test for a unit root in general unit root models+ The rest of the paper is organized as follows+ Section 2 presents some preliminary results on invariance principles for i+i+d+ innovations and their bootstrap samples+ In Section 3 we use the results from Section 2 to establish invariance principles for linear processes with i+i+d+ innovations and for the samples obtained from the sieve bootstrap+ Section 4 provides an application of our results to the test of a unit root+ In particular, we develop the bootstrap asymptotic theory for Dickey–Fuller tests, and we show that they are asymptotically valid for unit root models driven by linear processes with i+i+d+ innovations+ Section 5 contains concluding remarks, and the mathematical proofs are given in Section 6+ Finally, a word on notation+ The standard notation used in probability and measure theory is used without reference, i+e+, ra+s+ , rp , rLr , and

SIEVE BOOTSTRAP IN TIME SERIES

471

rd imply, respectively, convergence almost surely, in probability, in L r , and in distribution+ A time series is denoted by ~z t !, and whenever necessary the range ` n and ~z t ! t⫽1 + of time index will be specified as in ~z t ! t⫽1 2. PRELIMINARY RESULTS Throughout this section, we let ~«t ! be a sequence of i+i+d+ random variables with mean zero+ It is thought to be a sequence driving a model under investigation+ Also, in the context of the bootstrap applications, it is supposed that the realization of ~«t ! itself or its estimates provides the values from which we resample+ In this section, we first establish an invariance principle for ~«t ! and its bootstrap version+ Primarily, this is a preliminary step toward an invariance principle for the sieve bootstrap in time series developed in the next section+ Our results presented here, however, have a wider applicability and hence should be of independent interest+ Though the existence of higher moments for ~«t ! will be required later for the main result of the paper, we simply let E«t2 ⫽ s 2 ⬍ ` at the moment+ Let a partial sum process of ~«t ! be defined by Wn ~t ! ⫽

1 sM n

@nt #

( «k +

(1)

k⫽1

Here and elsewhere in the paper @x# denotes the maximum integer that does not exceed x+ Then by the classical Donsker’s theorem ~see, e+g+, Billingsley, 1968! we have Wn rd W

(2)

in the space D@0,1# of cadlag functions, where W is the standard Brownian motion+ The space D@0,1# is usually equipped with the Skorohod topology+ However, it is more convenient to consider the space endowed with the uniform norm 7{7 in the subsequent development of our theory+ We may actually have a stronger result than the weak convergence in ~2!+ As a result of what is known as the Skorohod representation theorem ~see, e+g+, Pollard, 1984!, we know that there exists a probability space ~V, F, P! supporting a process Wn' , say, and W such that Wn' has the same distribution as Wn and Wn' ra+s+ W+

(3)

Indeed, it follows from the result in Sakhanenko ~1980! that we may choose Wn' satisfying the following condition+ LEMMA 2+1+ Let E6«t 6 r ⬍ ` for some r ⬎ 2. Then we have for any d ⬎ 0 P$7Wn' ⫺ W7 ⱖ d% ⱕ n 1⫺r02 Kr E6«t 6 r, where Kr is an absolute constant depending only upon r.

472

JOON Y. PARK

The result in Lemma 2+1, which is often called the strong approximation, is very useful in developing an invariance principle for bootstrap samples+ This will be explained subsequently+ In what follows, we will not distinguish Wn' from Wn , and we assume that Wn converges a+s+ to W uniformly+ This causes no loss in generality, because we are only concerned with distributional results+ n from the sample of For the bootstrap, we first obtain or estimate ~«t ! t⫽1 n size n and get ~ «[ t ! t⫽1 , say+ Then we resample from the empirical distribution of n , i+e+, the distribution with point probability mass 10n on their n observed ~ «[ t ! t⫽1 n n + We may thus regard ~«t* ! t⫽1 as the values, to get the bootstrap sample ~«t* ! t⫽1 n i+i+d+ samples from the empirical distribution of ~ «[ t ! t⫽1+ Both ~ «[ t ! and ~«t* ! are dependent upon the sample size n, and we may more precisely denote them as triangular arrays ~ «[ nt ! and ~«nt* !+ However, we will continue to write them as ~ «[ t ! and ~«t* ! in the subsequent discussions, following the usual convention+ ` is given and consider the bootNow we suppose that a realization of ~ «[ t ! t⫽1 ` * * * + strap probability space ~V , F , P ! conditional on the realization of ~ «[ t ! t⫽1 ` * n For each realization of ~ «[ t ! t⫽1 , bootstrap samples ~«t ! t⫽1 are regarded formally as random variables defined on this probability space+ As mentioned earlier, n n are obtained for each n from ~ «[ t ! t⫽1 + Under the the bootstrap samples ~«t* ! t⫽1 * n * probability P , the bootstrap samples ~«t ! t⫽1 become i+i+d+ with the underlying n distribution given by the empirical distribution of ~ «[ t ! t⫽1 + Naturally, the ex* * pectation with respect to P is signified by E + We also use the conventional notations rp * and rd * to denote the convergences in probability and in distribution, respectively, for the functionals of bootstrap samples defined on ~V *, F *, P * !+ In particular, if the convergences occur a+s+ for all realizations of ` , then we write them, respectively, as rp * a+s+ and rd * a+s+ Of course, ~ «[ t ! t⫽1 n is the former implies the latter+ Whenever the sample mean «S n of ~ «[ t ! t⫽1 n nonzero, we assume that bootstrap samples are drawn from ~ «[ t ⫺ «S n ! t⫽1 so that E * «t* ⫽ 0 a+s+ ` , we consider For each realization of ~ «[ t ! t⫽1 Wn* ~t ! ⫽

1 s[ nM n

@nt #

«k* , ( k⫽1

(4)

n which corresponds to Wn defined in ~1!, where s[ n2 ⫽ E * «t*2 ⫽ ~10n! ( t⫽1 «[ t2+ *' * As before, we let Wn be the distributionally equivalent copy of Wn in an expanded probability space rich enough to support a standard Brownian motion W * such that

P * $7Wn* ' ⫺ W * 7 ⱖ d% ⱕ n 1⫺r02 Kr E * 6«t* 6 r

(5)

for any d ⬎ 0 and r ⬎ 2, where Kr is the absolute constant introduced in Lemma 2+1+ This is possible, again due to the result by Sakhanenko ~1980! as in Lemma 2+1+ Following our earlier convention, we will not distinguish Wn* in ~4! from Wn* ' , and we simply assume that Wn* and W * are defined on the common probability space+

SIEVE BOOTSTRAP IN TIME SERIES

473

It is now obvious from the result in ~5! that we can state the following theorem+ THEOREM 2+2+ If E * 6«t* 6 r ⬍ ` a+s+ and n 1⫺r02 E * 6«t* 6 r ra+s+ 0

(6)

for some r ⬎ 2, then Wn* rd * W

a+s+

(7)

as n r `. Thus we only need to show ~6! to establish the invariance principle ~7! for the bootstrap sample ~«t* !+ Note that ~6! implies Wn* rp * W * a+s+ as a result of ~5! and therefore Wn* rd * W * a+s+ Clearly, the latter convergence can be represented simply by Wn* rd * W a+s+ as in ~7!, because the limit process W * is ` + distributionally independent of the realizations of ~ «[ t ! t⫽1 In the simple case that ~«t ! are observable and the bootstrap samples are n , where «S n is the taken directly from the empirical distribution of ~«t ⫺ «S n ! t⫽1 n sample mean of ~«t ! t⫽1 , we have E * 6«t* 6 r ⫽

1 n

n

( 6«t ⫺ «S n 6 r+ t⫽1

Therefore, it suffices to require E6«t 6 r ⬍ ` for some r ⬎ 2 to derive the bootstrap invariance principle ~7!+ Note that condition ~6! is satisfied under such a n 6«t ⫺ «S n 6 r ra+s+ E6«t 6 r by the strong law moment condition, because ~10n! ( t⫽1 of large numbers+ For the simple case considered earlier, Basawa, Mallik, McCormick, Reeves, and Taylor ~1991b! establish the bootstrap invariance principle ~7! under a weaker assumption E«t2 ⬍ `+1 They derive ~7! directly by showing that the finite-dimensional distribution converges @in the Mallows metric ~for details, see Bickel and Freedman, 1981!# and that the tightness condition holds+ However, their approach does not readily extend to more complicated procedures such as the sieve bootstrap, which we will consider in the next section+ We also refer to Ferretti and Romo ~1996! for an invariance principle that is comparable to ours+ Let w be a function on D@0,1# continuous a+s+ with respect to the Wiener measure+ Once the invariance principle ~2! is established, it follows immediately that w~Wn ! rd w~W !

(8)

as a result of the continuous mapping theorem+ It is obvious from our development that a similar result holds for Wn*; i+e+, we may easily deduce from the bootstrap invariance principle that w~Wn* ! rd * w~W !

a+s+

(9)

474

JOON Y. PARK

In particular, if we apply ~8! or ~9! with ~w~ x!!~t ! ⫽ x~t ! ⫺ tx~1! for x 僆 D@0,1# , then the limit process becomes the Brownian bridge U~t ! ⫽ W~t ! ⫺ tW ~1!+ Such limit process appears when we deal with the partial sum of ~«t ⫺ «S n ! or ~«t* ⫺ «S n* !, where «S n and «S n* are, respectively, the sample means of ~«t ! and ~«t* !+ Also, we need to consider the detrended Brownian motion Wt ~t ! ⫽ W~t ! ⫺ ~*01 W~s!t~s! ' ds!~*01 t~s!t~s! ' ds!⫺1 t~t ! with t 僆 D@0,1# m for the asymptotic analysis of unit root models with deterministic trends+ An invariance principle with limit process Wt can be obtained from the application of ~8! or ~9! with ~w~ x!!~t ! ⫽ x~t ! ⫺ ~*01 x~s!t~s! ' ds!~*01 t~s!t~s! ' ds!⫺1 t~t ! for x 僆 D@0,1# + As is well known, ~8! or ~9! yields the usual central limit theorem, because for x 僆 D@0,1# ~w~ x!!~t ! ⫽ x~1! is a continuous functional+ 3. INVARIANCE PRINCIPLE FOR SIEVE BOOTSTRAP We consider a general linear process ~u t ! given by u t ⫽ p~L!«t , where ~«t ! is an i+i+d+ random sequence and `

p~z! ⫽

pk z k+ ( k⫽0

More specifically, we let ~«t ! and p~z! satisfy the following conditions+ Assumption 3+1+ We assume that ~a! ~«t ! are i+i+d+ random variables such that E«t ⫽ 0, E«t2 ⫽ s 2 , and E6«t 6 r ⬍ ` for some r ⬎ 4, ` ~b! p~z! ⫽ 0 for all 6z6 ⱕ 1, and ( k⫽0 6k 6 s 6pk 6 ⬍ ` for some s ⱖ 1+

The conditions in Assumption 3+1 are not stringent and are routinely assumed in stationary time series analysis+ Yet, they are sufficient to establish an invariance principle for the sieve bootstrap from the general linear process ~u t !+ The invariance principle that we will develop is of the strong form, which holds almost surely for all sample realizations+ For a weak invariance principle that we only require to hold in probability, we may allow r ⫽ 4+2 Following Phillips and Solo ~1992!, we use the Beveridge–Nelson representation and write ~u t ! as u t ⫽ p~1!«t ⫹ ~ uI t⫺1 ⫺ uI t !, where `

uI t ⫽

( pJ k «t⫺k

k⫽0

(10)

SIEVE BOOTSTRAP IN TIME SERIES

475

` with pJ k ⫽ (i⫽k⫹1 pi + Under the condition in Assumption 3+1~b!, we have ` 6 p J 6 ⬍ ` as shown by Phillips and Solo ~1992, p+ 973!+ The time series ( k⫽0 k ~ uI t ! is therefore well defined in both the a+s+ and L 1 sense under Assumption 3+1~a!+ If we let

Vn ~t ! ⫽

1 Mn

@nt #

1

( uk ⫽ ~sp~1!!Wn ~t ! ⫹ M n ~ uI 0 ⫺ uI @nt # !, k⫽1

(11)

then the invariance principle for ~u t !, Vn rd V ⫽ ~sp~1!!W, follows immediately from the continuous mapping theorem, because we have under Assumptions 3+1 that max 6n⫺102 uI k 6 rp 0

1ⱕkⱕn

as shown in Phillips and Solo ~1992, p+ 978!+ Under Assumption 3+1~b!, we may write ~u t ! as a~L!u t ⫽ «t , where `

a~z! ⫽ 1 ⫺

( ak z k k⫽1

with ~ak ! satisfying (6k 6 s 6ak 6 ⬍ `, as shown in, e+g+, Brillinger ~1975!+ Therefore, it may be reasonable to approximate ~u t ! by an autoregressive process of finite order, i+e+, u t ⫽ a1 u t⫺1 ⫹ {{{ ⫹ ap u t⫺p ⫹ «p, t +

(12)

In the subsequent development of our theory, we let p ⫽ pn be a function of sample size n+ More precisely, we can state the following assumption+ Assumption 3+2+ pn satisfies pn r ` and pn ⫽ o~~n0log~n!! 102 !+ We do not impose any lower limit to the divergence rate for pn , and we may thus let it increase as slowly as we want+ If we also assume pn 0~log n! 104⫹d r ` for some d ⬎ 0, all of our subsequent results hold also for r ⫽ 4 in Assumption 3+1~a!+ When there is some p0 such that ak ⫽ 0 for all k ⬎ p0 and ~u t ! is generated by a finite autoregression, the conditions in Assumption 3+2 are of course not necessary+ We only need to require pn ⱖ p0 for all large n+ For the sake of notational brevity, we will denote by p as before instead of pn the order of the approximated autoregression+

476

JOON Y. PARK

The parameters ~ak ! and s 2 can be consistently estimated from the regression ~12!+ To show this, we let ~ a[ p, k ! be the ordinary least squares ~OLS! estimators of ~ak ! for k ⫽ 1, + + + , p and define p

a[ n ~1! ⫽ 1 ⫺

( a[ p, k + k⫽1

Also, let s[ n2 be the usual error variance estimator+ LEMMA 3+1+ Let Assumptions 3.1 and 3.2 hold. Then it follows that max 6 a[ p, k ⫺ ak 6 ⫽ O~~log n0n! 102 ! ⫹ o~ p ⫺s !

a+s+

1ⱕkⱕp

for all large n. Moreover, we have s[ n2 ⫽ s 2 ⫹ O~~log n0n! 102 ! ⫹ o~ p ⫺s !

a+s+

a[ n ~1! ⫽ a~1! ⫹ O~ p~log n0n! 102 ! ⫹ o~ p ⫺s !

a+s+

as n r `. The OLS estimators ~ a[ p, k ! are therefore consistent for ~ak !, k ⫽ 1, + + + , p+ Moreover, as long as we let p r ` as the sample size increases, the autoregressive coefficients ~ak ! for k ⬎ p become negligible in the limit, because ` (k⫽p⫹1 ak ⫽ o~ p⫺s ! under our condition in Assumption 3+1~b!+ For the sieve bootstrap, we first fit ~u t ! with an approximated autoregression ~12! of order p by OLS, i+e+, u t ⫽ a[ p,1 u t⫺1 ⫹ {{{ ⫹ a[ p, p u t⫺p ⫹ «[ p, t to get the fitted residuals ~ «[ p, t !+ At this stage, we may use the usual order selection criterion such as the Akaike information criterion ~AIC! or the Bayesian information criterion ~BIC!+ Under AIC and BIC, respectively, p is selected such that log s[ n2 ⫹ 2p0n and log s[ n2 ⫹ p log n0n are minimized+ If ~u t ! is believed to be generated by a finite autoregression, BIC might be preferred because it yields a consistent estimator for p+ See, e+g+, An, Chen, and Hannan ~1982!+ If not, AIC may be a better choice, because it leads to an asymptotically efficient choice for the optimal order of some projected infinite-order autoregressive process, as shown by Shibata ~1980!+ Our subsequent theory allows for such data-dependent selection rules+ From Lemma 3+1, we have log s[ n2 ⫽ log s 2 ⫹ O~~log n0n! 102 ! ⫹ o~ p ⫺s !

a+s+

Therefore, if we use AIC to select p, then p satisfies p ⫽ o~n 10~1⫹s! ! a+s+, and the condition in Assumption 3+2 holds a+s+ if s ⬎ 1+ Likewise, BIC selects p such that p ⫽ o~~n0log n! 10~1⫹s! ! a+s+, and the condition in Assumption 3+2 is

SIEVE BOOTSTRAP IN TIME SERIES

477

satisfied a+s+ for s ⱖ 1+ Note that p r ` a+s+ under both selection rules, unless ~u t ! is generated by a finite autoregression+ n from the empirical distribution of The next step is to resample ~«t* ! t⫽1

冉

«[ p, t ⫺

1 n

n

( «[ p, t t⫽1

冊

n

t⫽1

and generate recursively from ~«t* ! an autoregressive process ~u t* ! of order p as * * u t* ⫽ a[ p,1 u t⫺1 ⫹ {{{ ⫹ a[ p, p u t⫺p ⫹ «t* +

For the actual generation of the bootstrap samples ~u t* !, their initial values must be set+ The choice of the initial values of ~u t* ! may have an important effect on the actual performance of the bootstrap in finite samples+ Also, to get stationary samples for ~u t* !, we must impose the appropriate stationarity conditions for the initial values, or we may repeat sampling until stationarity is achieved and throw the first generated values away+ However, it is irrelevant in our subsequent asymptotic analysis and will not be discussed any further+ The bootstrapped process ~u t* ! may not be invertible in finite samples, though it should have small probability unless the sample size is very small+ Given our results in Lemma 3+1, the problem of noninvertibility in ~u t* ! should vanish almost surely as the sample size increases+ We may however prefer to use the Yule–Walker estimators ~see, e+g+, Brockwell and Davis, 1991, Secs+ 8+1 and 8+2!+ It is well known that any finite autoregression fitted by the Yule– Walker estimators is invertible+ All our subsequent results are also applicable for the bootstrap based on the Yule–Walker estimation of the autoregression ~12!+ Let Wn* ~t ! ⫽

1 s[ nM n

@nt #

«k* ( k⫽1

be the bootstrap analogue of the process Wn introduced in the previous section+ Then we have the following lemma+ LEMMA 3+2+ Let Assumptions 3.1 and 3.2 hold. Then condition (6) is satisfied and Wn* rd * W a+s+, as n r `. The bootstrap invariance principle for ~u t* ! can be obtained from the Beveridge–Nelson representation, as in ~10!+ If we let p[ n ~1! ⫽ 10a[ n ~1! and uI t* ⫽

1 a[ n ~1!

p

冉 冊 p

( i⫽k ( a[ p, i

k⫽1

* u t⫺k⫹1 ,

478

JOON Y. PARK

then we may write Vn* ~t ! ⫽

1 Mn

@nt #

1

* # !+ ( uk* ⫽ ~ s[ n p[ n ~1!!Wn* ~t ! ⫹ M n ~ uI 0* ⫺ uI @nt k⫽1

(13)

Note that s[ n2 ra+s+ s 2 and p[ n ~1! ra+s+ p~1! as n r `, as shown in Lemma 3+1+ Therefore, if we can show that

再

冎

P * max 6n⫺102 uI t* 6 ⬎ d ra+s+ 0 1ⱕtⱕn

(14)

holds for any d ⬎ 0, the bootstrap invariance principle for ~u t* ! Vn* rd * V ⫽ ~sp~1!!W

a+s+

would follow+ THEOREM 3+3+ Let Assumptions 3.1 and 3.2 hold. Then condition (14) holds and Vn* rd * V a+s+, as n r `. The bootstrap invariance principle for the sieve bootstrap has now been established for general linear processes with i+i+d+ innovations+ It can be applied to obtain bootstrap asymptotics without making parametric assumptions on the underlying model+ In particular, the limiting distributions of various bootstrap statistics from linear time series can be found simply by the application of the continuous mapping theorem+ 4. SIEVE BOOTSTRAP FOR DICKEY–FULLER TESTS We now show how the bootstrap invariance principle developed in the previous section can be applied to testing the unit root hypothesis+ Let a time series ~ yt ! be generated as yt ⫽ Dt ⫹ x t , where ~Dt ! and ~ x t ! denote, respectively, the deterministic and stochastic components of ~ yt !+ We specify ~Dt ! as Dt ⫽ ct' b,

(15)

where ct ⫽ ~c1t , + + + , cmt ! ' is a vector of deterministic functions of time and b ⫽ ~ b1 , + + + , bm ! ' is a parameter vector+ Most commonly used such trends are ct ⫽ 1 and ct ⫽ ~1, t ! ' + Furthermore, we let ~ x t ! be given by x t ⫽ ax t⫺1 ⫹ u t

(16)

with a ⫽ 1 and let ~u t ! be a linear process specified as in the previous section+3 We may allow the initial value x 0 of ~ x t ! to be any random variable, and therefore we set x 0 ⫽ 0 in the subsequent development of our theory+

SIEVE BOOTSTRAP IN TIME SERIES

479

For expositional simplicity, we will mainly consider ~ yt ! with no deterministic trends, i+e+, ~ yt ! [ ~ x t !+ It is easy to accommodate the presence of deterministic trends+ For instance, our subsequent bootstrap methodology applies, without any modification, to ~ yt ! with deterministic trends if we detrend ~ yt ! by running an OLS regression yt ⫽ ct' bZ n ⫹ x[ t and use the fitted residuals ~ x[ t ! in place of ~ x t !+4 Using ~ x[ t ! instead of ~ x t ! would, however, result in different asymptotic distributions for the test statistics+ This will be explained in detail later+ To test the hypothesis H 0 : a ⫽ 1 in the model ~16!, we consider the statistics proposed and studied by Dickey and Fuller ~1979, 1981!+ Their coefficient and t-statistics will be denoted by Sn and Tn , respectively+ To present them more explicitly, we let a[ n be the OLS estimator of a, and we let s~ a[ n ! be the stann 2 x t⫺1 !⫺102 , where v[ n2 is the usual error varidard error for a[ n given by v[ n ~ ( t⫽1 ance estimator from regression ~16!+ Then, we may write Sn and Tn as

冉

n

~10n! x n2 ⫺ ( u t2 Sn ⫽ n~ a[ n ⫺ 1! ⫽

t⫽1

n

~20n ! ( 2

a[ n ⫺ 1 ⫽ Tn ⫽ s~ a[ n !

冊

t⫽1

冉

n

~10n! x n2 ⫺ ( u t2

冉

,

2 x t⫺1

t⫽1

n

冊 冊

102

2 2 v[ n ~10n 2 ! ( x t⫺1 t⫽1

under the unit root hypothesis+ We now have Sn ⬇

Vn ~1! 2 ⫺ vn2 2

冕

1

V~1! 2 ⫺ v 2

rd

Vn ~t ! dt 2

2

0

冕

(17)

1

V~t ! dt 2

0

and similarly, Vn ~1! 2 ⫺ vn2

Tn ⬇ 2 v[ n

冉冕

0

1

冊

102

Vn ~t ! dt 2

V~1! 2 ⫺ v 2

rd 2v

冉冕

0

1

冊

102

,

(18)

V~t ! dt 2

n where vn2 ⫽ ~10n! ( t⫽1 u t2 and other notation is defined earlier+ The convergence in distribution in ~17! and ~18! can be easily deduced from the laws of large numbers applied to vn2 and v[ n2 and the continuous mapping theorem ap-

480

JOON Y. PARK

plied to the weak convergence Vn rd V because both V~1! 2 and *01 V~t ! 2 dt are continuous functionals of V in D@0,1# + Let ~u t* ! be the bootstrap sample for ~u t !, u t ⫽ x t ⫺ x t⫺1 , obtained as described in the previous section, and denote by ~ x t* ! the bootstrap sample for ~ x t ! generated by * ⫹ u t* x t* ⫽ x t⫺1

with initial value x 0 , which is assumed to be zero to simplify the exposition+ It is important to impose the unit root hypothesis when we generate the bootstrap * ⫹ u t* using the estimated samples ~ x t* !+ If we generate them by x t* ⫽ a[ n x t⫺1 value a[ n of a in ~16!, then they would not behave as a unit root process+ This is so, even though a[ n is superconsistent and converges to unity at a faster rate n+ See Basawa et al+ ~1991a!, Datta ~1996!, and Kreiss and Heimann ~1996!+ Let a[ n* be the OLS coefficient estimator from the first-order autoregression * !, and s * ~ a[ n* ! be the bootstrap standard error for a[ n* , which is of ~ x t* ! on ~ x t⫺1 n *2 102 ! + Now we define given by v[ n ~ ( t⫽1 x t⫺1

冉

n

~10n! x n*2 ⫺ ( u t*2 Sn*

Tn*

⫽

n~ a[ n*

⫺ 1! ⫽

a[ n* ⫺ 1 ⫽ * * ⫽ s ~ a[ n !

t⫽1

冊

n

,

*2 ~20n 2 ! ( x t⫺1 t⫽1

冉

n

冊 冊

~10n! x n*2 ⫺ ( u t*2

冉

t⫽1

n

2 v[ n ~10n ! ( 2

102

*2 x t⫺1

t⫽1

,

which are the bootstrap versions of the statistics Sn and Tn + Also, we define n vn*2 ⫽ ~10n! ( t⫽1 u t*2+ LEMMA 4+1+ Let Assumptions 3.1 and 3.2 hold. Then we have, for any d ⬎ 0, P * $6vn*2 ⫺ v 2 6 ⱖ d% ra+s+ 0 as n r `. THEOREM 4+2+ Let Assumptions 3.1 and 3.2 hold. Then we have Sn* rd *

V~1! 2 ⫺ v 2 2

冕

a+s+

1

V~t ! dt 2

0

V~1! 2 ⫺ v 2

Tn* rd * 2v

冉冕

0

as n r `.

1

冊

102

V~t ! dt 2

a+s+

SIEVE BOOTSTRAP IN TIME SERIES

481

Theorem 4+2 shows that the bootstrap statistics Sn* and Tn* have the same asymptotic distributions as the corresponding sample statistics Sn and Tn + Therefore, it establishes the asymptotic validity of the bootstrap Dickey–Fuller tests+ It shows in particular that if based on the bootstrap critical values then the Dickey–Fuller tests can be used to test for the presence of a unit root in models driven by innovations that are correlated+ The bootstrap Dickey–Fuller tests can therefore be an alternative to the testing procedures by Phillips ~1987! and Said and Dickey ~1984! ~which are more often called augmented Dickey–Fuller tests!+ The limiting distributions of the sample statistics Sn and Tn are not pivotal+ They depend on the parameters s 2, v 2 , and p~1!+ Note that V ⫽ ~sp~1!!W, where W is standard Brownian motion+ It is thus not expected that the bootstrap statistics Sn* and Tn* provide asymptotic refinements+ Therefore, the Dickey– Fuller tests based on the bootstrapped critical values are not necessarily better than the tests relying on the asymptotic critical values+ However, for the model considered here, the Said–Dickey or Phillips statistics have limiting distributions that are free of nuisance parameters+ Consequently, they may provide tests with more accurate finite sample sizes if based on the bootstrapped critical values+ The bootstrap consistency established previously assumes the absence of deterministic trends+ However, it can be extended in a trivial manner to allow for the presence of deterministic trends+ For the trend specification in ~15!, we just assume that there exists ki such that, if we define n ci, k⫺1 1$~k ⫺ 1!0n ⱕ t ⬍ k0n%, then tni rL2 ti for some tni ~t ! ⫽ n⫺ki ( k⫽1 ti : @0,1# r R that is of bounded variation for i ⫽ 1, + + + , m+ Also, we let ti ’s be linearly independent in L 2 @0,1# + Under these assumptions, we may show that the asymptotic distributions of Sn and Tn constructed from ~ x[ t ! are identical to those in ~17! and ~18!, except that V ⫽ ~sp~1!!W is replaced by Vt ⫽ ~sp~1!!Wt , where Wt is the standard detrended Brownian motion introduced in Section 2 ~see, e+g+, Park, 1992!+ Moreover, it can also be shown without difficulty that the limiting distributions of their bootstrap counterparts Sn* and Tn* have the same limiting distributions+ 5. CONCLUDING REMARKS In this paper, we have developed an invariance principle for the sieve bootstrap from linear process with i+i+d+ innovations+ It can be used, in various contexts, to obtain the limiting distributions of bootstrap statistics without making parametric assumptions on the underlying model+ As an illustration, we used the invariance principle to derive the limiting distributions of the Dickey–Fuller unit root tests and show their asymptotic validity for a model driven by a general linear process that is serially correlated+ It appears that the bootstrap invariance principle established in the paper has wide applicability and can be used to analyze the asymptotic behavior of bootstrap samples in stationary and nonstationary time series models+ Some of the applications are under way by the author and co-researchers+

482

JOON Y. PARK

6. MATHEMATICAL PROOFS The proofs of Lemma 2+1 and Theorem 2+2 are omitted+ Lemma 2+1 is due to Sakhanenko ~1980!, and the stated result in Theorem 2+2 follows immediately from ~5! as mentioned in the following remarks+ Proof of Lemma 3+1+ We write ~u t ! as u t ⫽ ap,1 u t⫺1 ⫹ {{{ ⫹ ap, p u t⫺p ⫹ ep, t ,

(19)

where the coefficients ~ap, k ! are defined so that ~ep, t ! are uncorrelated with ~u t⫺k ! for k ⫽ 1, + + + , p+ We have max 6 a[ p, k ⫺ ap, k 6 ⫽ O~~log n0n! 102 !

a+s+,

(20)

6ak 6 ⫽ o~ p ⫺s !, ( k⫽p⫹1

(21)

1ⱕkⱕp

p

( 6ap, k ⫺ ak 6 ⱕ c

k⫽1

`

where c is some constant+ For the results in ~20! and ~21!, see, e+g+, Hannan and Kavalieris ~1986, Theorem 2+1! and Bühlmann ~1995, proof of theorem 3+1!+ The proof of the first part is immediate, because 6 a[ p, k ⫺ ak 6 ⱕ 6 a[ p, k ⫺ ap, k 6 ⫹ 6ap, k ⫺ ak 6+ Similarly, it follows that p

6 a[ n ~1! ⫺ a~1!6 ⱕ

( 6 a[ p, k ⫺ ap, k 6 ⫹

k⫽1

p

( 6ap, k ⫺ ak 6 ⫹

k⫽1

`

(

6ak 6

k⫽p⫹1

⫽ O~ p~log n0n! 102 ! ⫹ o~ p ⫺s ! as was to be shown for the result on a[ n ~1!+ For the result of s[ n2 , see Bühlmann ~1995, proof of theorem 3+2!+ He establishes the result for the Yule–Walker estimator, but it is easy to see that his result readily extends to the least squares 䡲 estimator+ Proof of Lemma 3+2+ We will show n 1⫺r02 E * 6«t* 6 r ⫽ n 1⫺r02

冉

1 n

(冨 n

t⫽1

«[ p, t ⫺

1 n

n

( «[ p, t

t⫽1

冨 冊r r

as n r `+ Note that 1 n

(冨 n

t⫽1

«[ p, t ⫺

1 n

n

( «[ p, t

t⫽1

冨

r

ⱕ c~A n ⫹ Bn ⫹ Cn ⫹ Dn !,

a+s+

0

(22)

SIEVE BOOTSTRAP IN TIME SERIES

483

where c is some constant, and n

An ⫽

1 n

( 6«t 6 r, t⫽1

Bn ⫽

1 n

( 6«p, t ⫺ «t 6 r, t⫽1

Cn ⫽

1 n

Dn ⫽

冨

n

n

( 6 «[ p, t ⫺ «p, t 6 r,

t⫽1

1 n

冨

n

r

( «[ p, t +

t⫽1

Therefore, to deduce ~22!, we only need to show that n 1⫺r02 A n , n 1⫺r02 Bn , n 1⫺r02 Cn and n 1⫺r02 Dn ra+s+ 0+ This is what we now set out to do+ By the strong law of large numbers, A n ra+s+ E6 «t 6 r , and therefore, 1⫺r02 A n ra+s+ 0+ Next, we show that n E6«p, t ⫺ «t 6 r ⫽ o~ p ⫺rs !

(23)

Bn ra+s+ 0+ Note that if holds uniformly in t, from which it follows that n p0~log n! 104⫹d r ` for some d ⬎ 0 then n 1⫺r02 Bn ra+s+ 0 also for r ⫽ 4, as we remarked following Assumption 3+2+ To obtain ~23!, we write 1⫺r02

`

(

«p, t ⫽ «t ⫹

ak u t⫺k

(24)

k⫽p⫹1

and apply Minkowski’s inequality and use the stationarity of ~u t ! to get

冉( 冊

r

`

E6«p, t ⫺ «t 6 r ⱕ E6u t 6 r

6ak 6

⫽ o~ p ⫺rs !+

k⫽p⫹1

Note that, due to the Marcinkiewicz–Zygmund inequality ~see, e+g+, Stout, 1974, Theorem 3+3+6!,

冉( 冊

r02

`

E6u t 6 r ⱕ c

pk2

E6«t 6 r ⬍ `,

(25)

k⫽0

where c is some constant, and therefore (p⫹1 ak u t⫺k is well defined in the L r sense+ We now prove that n 1⫺r02 Cn ra+s+ 0+ Write `

p

«[ p, t ⫽ u t ⫺

( a[ p, k ut⫺k

k⫽1

p

⫽ «p, t ⫺

( ~ a[ p, k ⫺ ap, k !ut⫺k ⫺

k⫽1

p

( ~ap, k ⫺ ak !ut⫺k ,

k⫽1

(26)

484

JOON Y. PARK

where ~ap, k ! are defined in ~19!+ It follows that

冉冨 (

冨

p

6 «[ p, t ⫺ «p, t 6 r ⱕ c

r

~ a[ p, k ⫺ ap, k !u t⫺k ⫹

k⫽1

( ~ap, k ⫺ ak !ut⫺k冨 冨 k⫽1 p

r

冊

for c ⫽ 2 r⫺1 + Therefore, if we define C1n ⫽

1 n

( 冨 ( ~ a[ p, k ⫺ ap, k !ut⫺k冨 , t⫽1 k⫽1

C2n ⫽

1 n

(冨

n

r

p

n

冨

p

r

( ~ap, k ⫺ ak !ut⫺k ,

t⫽1 k⫽1

then it suffices to show that n 1⫺r02 C1n , n 1⫺r02 C2n ra+s+ 0+ Note that C1n is majorized by

冉 max 6 a[ 1ⱕkⱕp

ⱕ

冉

冊 n1 ( ( 6u n

p, k

⫺ ap, k 6 r

p

t⫺k 6

r

t⫽1 k⫽1

max 6 a[ p, k ⫺ ap, k 6 r

1ⱕkⱕp

冊冉 p n

n⫺1

1⫺p

t⫽0

t⫽⫺1

( 6ut 6 r ⫹

(

6u t 6 r

⫽ O~~log n0n! r !~ p0n!O~n! ⫽ O~ p~log n0n! r !

冊

a+s+

by ~20! and ~25! and therefore, C1n ra+s+ 0+ Moreover, we have

( ~ap, k ⫺ ak !ut⫺k冨 冨 k⫽1 p

E

r

ⱕ E6u t 6 r

冉

p

( 6ap, k ⫺ ak 6 k⫽1

冊

r

⫽ o~ p ⫺rs !

by Minkowski’s inequality and the stationarity of ~u t !+ Consequently, it follows that n 1⫺r02 C2n ra+s+ 0 for r ⬎ 4+ If p0~log n! 104⫹d r `, r ⫽ 4 works also+ Finally, to deduce n 1⫺r02 Dn ra+s+ 0, we show 1 n

n

1

n

( «[ p, t ⫽ n t⫽1 ( «p, t ⫹ o~1! t⫽1

a+s+ ⫽

1 n

n

( «t ⫹ o~1! t⫽1

a+s+,

which hold if 1 n 1 n 1 n

n

`

( (

ak u t⫺k ra+s+ 0,

(27)

( ( ~ap, k ⫺ ak !ut⫺k ra+s+ 0, t⫽1 k⫽1

(28)

n

n

t⫽1 k⫽p⫹1

p

p

( ( ~ a[ p, k ⫺ ap, k !ut⫺k ra+s+ 0 t⫽1 k⫽1

as a result of ~24! and ~26!+

(29)

SIEVE BOOTSTRAP IN TIME SERIES

485

To establish ~27!–~29!, we first define n

Sn ~i, j ! ⫽

n

( «t⫺i⫺j

Tn ~i ! ⫽ ( u t⫺i +

and

t⫽1

t⫽1

Then it follows that n

Tn ~i ! ⫽

`

`

( ( pj «t⫺i⫺j ⫽ j⫽0 ( pj Sn ~i, j !+ t⫽1 j⫽0

In what follows, we use c to denote a constant, which is not always the same+ By successive applications of Doob’s inequality and Burkholder’s inequality ~see, e+g+, Hall and Heyde, 1980, Theorems 2+2 and 2+10!, we have

冉

冊

E max 6Sm ~i, j !6 r ⱕ cn r02 1ⱕmⱕn

uniformly in i and j+ Moreover, because

冋冉

E max 6Tm ~i !6 r 1ⱕmⱕn

冊册

10r

max 6Sm ~i, j !6 r 冊册 ( 6pj 6冋E 冉 1ⱕmⱕn `

ⱕ

10r

j⫽0

we have

冉

冊

E max 6Tm ~i !6 r ⱕ cn r02 1ⱕmⱕn

uniformly in i+ We now let `

Ln ⫽

(

k⫽p⫹1

n

ak ( u t⫺k + t⫽1

Then it follows that

冋冉

E max 6L m 6 r 1ⱕmⱕn

冊册

10r

6ak 6冋E 冉 max 6Tm ~k!6 r 冊册 ( 1ⱕmⱕn k⫽p⫹1 `

ⱕ

10r

ⱕ cp ⫺s n 102

and consequently,

冉

冊

E max 6L m 6 r ⱕ cp ⫺rs n r02+ 1ⱕmⱕn

Therefore, we may show for any d ⬎ 0 L n ⫽ o~ p ⫺s n 102 ~log n! 10r ~log log n! ~1⫹d!0r ! ⫽ o~n!

a+s+

exactly as in Móricz ~1976, pp+ 309–310, below ~4+8!!+ This proves ~27!+

486

JOON Y. PARK

Similarly, if we let p

Mn ⫽

n

( ~ap, k ⫺ ak ! t⫽1 ( ut⫺k k⫽1

then we have

冋冉

E max 6Mm 6 r 1ⱕmⱕn

冊册

10r

max 6Tm ~k!6 r 冊册 ( 6ap, k ⫺ ak 6冋E 冉 1ⱕmⱕn k⫽1 p

ⱕ

10r

ⱕ cp ⫺s n 102+

Therefore, we have Mn ⫽ o~ p ⫺s n 102 ~log n! 10r ~log log n! ~1⫹d!0r ! ⫽ o~n!

a+s+,

which establishes ~28!+ Finally, we consider Nn ⫽

p

n

k⫽1

t⫽1

( ~ a[ p, k ⫺ ap, k ! ( ut⫺k +

If we let

( 冨 ( ut⫺k冨 k⫽1 t⫽1 p

Qn ⫽

n

then Nn is dominated by Qn max 6 a[ p, k ⫺ ap, k 6+ 1ⱕkⱕp

Moreover, we may deduce precisely as before that

冉

冊

E max 6Qm 6 r ⱕ cp r n r02 1ⱕmⱕn

and therefore for any d ⬎ 0 Qn ⫽ o~ pn 102 ~log n! 10r ~log log n! ~1⫹d!0r !

a+s+

Consequently, Nn ⫽ O~~log n0n! 102 !Qn ⫽ o~ p~log n! ~r⫹2!02r ~log log n! ~1⫹d!0r ! ⫽ o~n! as was to be shown for ~29!+ Therefore, the proof is complete+ Proof of Theorem 3+3+ To show ~14!, we note that

再

冎

n

P * max 6n⫺102 uI t* 6 ⬎ d ⱕ ( P * $6n⫺102 uI t* 6 ⬎ d% 1ⱕtⱕn

t⫽1

⫽ nP * $6n⫺102 uI t* 6 ⬎ d% ⱕ ~10d r !n 1⫺r02 E * 6 uI t* 6 r+

a+s+

䡲

SIEVE BOOTSTRAP IN TIME SERIES

487

The first inequality is obvious+ The second equality follows from the stationarity of ~ uI t* !, conditional on the realization of ~ «[ p, t !, and the third inequality is due to the Tchebyshev inequality+ To complete the proof, it now suffices to prove that n 1⫺r02 E * 6 uI t* 6 r ra+s+ 0, which follows immediately if we show n 1⫺r02 E * 6u t* 6 r ra+s+ 0 because E * 6 uI t* 6 r ⱕ

冉

1 6 a[ n ~1!6

(30)

p

( k 6 a[ p, k 6 k⫽1

冊

r

E * 6u t* 6 r

by Minkowski’s inequality and the conditional stationarity of ~u t* !+ Note that a[ n ~1! ra+s+ a~1! as shown in Lemma 3+1 and that p

(

p

`

(

k 6 a[ p, k 6 ⫽

k⫽1

k 6ap, k 6 ⫹ o~1! ⫽

k⫽1

( k 6ak 6 ⫹ o~1!

a+s+

k⫽1

as shown in the proof of Lemma 3+1+ We may write for large n @for all n if we use the Yule–Walker method to estimate regression ~12! ~see, e+g+, Brockwell and Davis, 1991, p+ 240!# `

u t* ⫽

* ( p[ p, k «t⫺k k⫽0

(31)

with ~ p[ p, k ! such that ( k⫽0 6 p[ p, k 6 ⬍ ` a+s+ Therefore, due to the Marcinkiewicz–Zygmund inequality, `

冉( 冊

r02

`

E * 6u t* 6 ⱕ c

p[ p,2 k

E * 6«t* 6 r

k⫽0

for some constant c+ We therefore have ~30! whenever ~22! holds, which was 䡲 shown earlier in the proof of Lemma 3+1+ Proof of Lemma 4+1+ Let v*2 ⫽ E * 6u t* 6 2 and note that 6vn*2 ⫺ v 2 6 ⱕ 6vn*2 ⫺ v*2 6 ⫹ 6v*2 ⫺ v 2 6 We first show that v*2 ra+s+ v 2

(32)

488

JOON Y. PARK

From the Yule–Walker equations, we have v*2 ⫽ ~s*20s[ n2 !vn2 where s*2 ⫽ E * 6«t* 6 2 and other notations were defined earlier+ Because vn2 ra+s+ v 2 by the strong law of large numbers, it suffices to show that s*20s[ n2 ra+s+ 1 to deduce ~32!+ This, however, is immediate because s[ n2 ra+s+ s 2 as shown in Lemma 3+1 and s*2 ⫽ s[ n2 ⫹

冉

1 n

n

( «[ p, t

t⫽1

冊

2

the second term of which is of order o~1! a+s+, as we have shown in the proof of Lemma 3+2+ We now show P * $6vn*2 ⫺ v*2 6 ⬎ d% ⱕ ~10d! r E * 6vn*2 ⫺ v*2 6 r ra+s+ 0

(33)

To deduce ~33!, we write ~u t* ! as in ~31!+ Then we have vn*2 ⫺ v*2 ⫽

1 n

`

(

n

`

* * «t⫺j ⫺ dij s*2 ! ( p[ p, i p[ p, j ( ~«t⫺i

i⫽0 j⫽0

t⫽1

where dij is the Kronecker delta, and it follows that E * 6vn*2 ⫺ v*2 6 r02 ⱕ n⫺r02

冉

`

冊

r

( 6 p[ p, k 6 E *

k⫽0

冨

n

* * «t⫺j ⫺ dij s*2 ! ( ~«t⫺i

t⫽1

冨

r02

by Minkowski’s inequality+ By the successive applications of Burkholder’s inequality and Minkowski’s inequality, we have E*

冨

n

* * «t⫺j ⫺ dij s*2 ! ( ~«t⫺i

t⫽1

冨

r02

ⱕ KE *

冨

n

* * «t⫺j ⫺ dij s*2 ! 2 ( ~«t⫺i

t⫽1

冨

r04

* * ⱕ Kn r04 E * 6«t⫺i «t⫺j ⫺ dij s*2 6 r02,

where K is an absolute constant depending only upon r+ Moreover, * * E * 6«t⫺i «t⫺j ⫺ dij s*2 6 r02 * * ⱕ ~r ⫺ 2!02~E * 6«t⫺i «t⫺j 6 r02 ⫹ 6s*2 6 r02 ! ⱕ ~r ⫺ 2!E * 6«t* 6 r

SIEVE BOOTSTRAP IN TIME SERIES

489

as a result of the Cauchy–Schwarz inequality and Jensen’s inequality+ It follows that E * 6vn*2 ⫺ v*2 6 r02 ⱕ n⫺r04 K

冉( 冊 `

r

6 p[ p, k 6 E * 6«t* 6 r,

k⫽0

where K is an absolute constant depending upon r, and ~33! would thus follow if n ⫺r04 E * 6«t* 6 r ra+s+ 0+ This can be shown exactly as in the proof of Lemma 3+2+ The stated result now 䡲 follows from ~32! and ~33!+ Proof of Theorem 4+2+ The stated results follow immediately from Theorem 3+3 and Lemma 4+1+ NOTES 1+ They assume that ~«t ! has a continuous distribution function+ This condition, however, is not used in their proof for the bootstrap invariance principle+ 2+ Our assumptions here are comparable to those made in related works+ Kreiss ~1992! considers the model with ~pk ! that decays exponentially+ Bühlmann ~1997! proves the weak form of bootstrap consistency for a class of nonlinear estimators under the condition r ⱖ 4 and s ⱖ 1+ Bickel and Bühlmann ~1999! establish a weak invariance principle for the bootstrap empirical distribution function for the case r ⱖ 4 and exponentially decaying ~pk !+ 3+ Notice that we do not allow for unit root models generated by general martingale difference innovations+ 4+ Elliott, Rothenberg and Stock ~1996! show that the power of the unit root test can be substantially improved by the generalized least squares detrending+ Though we will not present the details here, the bootstrap asymptotics of their tests can also be analyzed using our method here+

REFERENCES An, H+-Z+, Z+-G+ Chen, & E+J+ Hannan ~1982! Autocorrelation, autoregression and autoregressive approximation+ Annals of Statistics 10, 926–936+ Correction: 11, 1018+ Basawa, I+V+, A+K+ Mallik, W+P+ McCormick, J+H+ Reeves, & R+L+ Taylor ~1991a! Bootstrapping unstable first-order autoregressive processes+ Annals of Statistics 19, 1098–1101+ Basawa, I+V+, A+K+ Mallik, W+P+ McCormick, J+H+ Reeves, & R+L+ Taylor ~1991b! Bootstrap test of significance and sequential bootstrap estimation for unstable first order autoregressive processes+ Communications in Statistics—Theory and Methods 20, 1015–1026+ Bickel, P+J+ & P+ Bühlmann ~1999! A new mixing notion and functional central limit theorems for a sieve bootstrap in time series+ Bernoulli 5, 413– 446+ Bickel, P+J+ & D+A+ Freedman ~1981! Some asymptotic theory for the bootstrap+ Annals of Statistics 9, 1196–1217+ Billingsley, P+ ~1968! Convergence of Probability Measures+ New York: Wiley+ Brillinger, D+R+ ~1975! Time Series: Data Analysis and Theory+ New York: Holt, Rinehart and Winston+ Brockwell, P+J+ & R+A+ Davis ~1991! Time Series: Theory and Methods+ New York: Springer-Verlag+ Bühlmann, P+ ~1995! Moving-average representation of autoregressive approximations+ Stochastic Processes and Their Applications 60, 331–342+

490

JOON Y. PARK

Bühlmann, P+ ~1997! Sieve bootstrap for time series+ Bernoulli 3, 123–148+ Bühlmann, P+ ~1998! Sieve bootstrap for smoothing in nonstationary time series+ Annals of Statistics 26, 48–83+ Datta, S+ ~1996! On asymptotic properties of bootstrap for AR~1! processes+ Journal of Statistical Planning and Inference 53, 361–374+ Dickey, D+A+ & W+A+ Fuller ~1979! Distribution of estimators for autoregressive time series with a unit root+ Journal of the American Statistical Association 74, 427– 431+ Dickey, D+A+ & W+A+ Fuller ~1981! Likelihood ratio statistics for autoregressive time series with a unit root+ Econometrica 49, 1057–1072+ Elliott, G+, T+J+ Rothenberg, & J+H+ Stock ~1996! Efficient tests for an autoregressive unit root+ Econometrica 64, 813–836+ Ferretti, N+ & J+ Romo ~1996! Unit root bootstrap tests for AR~1! models+ Biometrika 83, 849–860+ Hall, P+ & C+C+ Heyde ~1980! Martingale Limit Theory and Its Application+ New York: Academic Press+ Hannan, E+J+ & L+ Kavalieris ~1986! Regression, autoregression models+ Journal of Time Series Analysis 7, 27– 49+ Horowitz, J+L+ ~1999! The Bootstrap+ Mimeo, Department of Economics, University of Iowa+ Kreiss, J+P+ ~1992! Bootstrap procedures for AR~`!-processes+ In K+H+ Jöckel, G+ Rothe, & W+ Sender ~eds+!, Bootstrapping and Related Techniques, Lecture Notes in Economics and Mathematical Systems 376+ Heidelberg: Springer+ Kreiss, J+P+ & G+ Heimann ~1996! Bootstrapping general first order autoregression+ Statistics and Probability Letters 30, 87–98+ Künsch, H+R+ ~1989! The jackknife and the bootstrap for general stationary observations+ Annals of Statistics 17, 1217–1241+ Móricz, F+ ~1976! Moment inequalities and the strong law of large numbers+ Zeitschrift für Wahrscheinlichkeitstheorie und Verwandte Gebiete 35, 298–314+ Park, J+Y+ ~1992! Canonical cointegrating regressions+ Econometrica 60, 119–143+ Phillips, P+C+B+ ~1987! Time series regression with a unit root+ Econometrica 55, 277–301+ Phillips, P+C+B+ & V+ Solo ~1992! Asymptotics for linear processes+ Annals of Statistics 20, 971–1001+ Pollard, D+ ~1984! Convergence of Stochastic Processes+ New York: Springer-Verlag+ Said, S+E+ & D+A+ Dickey ~1984! Testing for unit roots in autoregressive-moving average models of unknown order+ Biometrika 71, 599– 608+ Sakhanenko, A+I+ ~1980! On unimprovable estimates of the rate of convergence in invariance principle+ In Nonparametric Statistical Inference, Colloquia Mathematica Societatis János Bolyai 32+ pp+ 779–783+ Budapest, Hungary+ Shibata, R+ ~1980! Asymptotically efficient selection of the order of the model for estimating parameters of a linear process+ Annals of Statistics 8, 147–164+ Stout, W+F+ ~1974! Almost Sure Convergence+ New York: Academic Press+