Department of Economics
Multiple Change-Point Detection in Linear Regression Models Via U-Statistic Type Processes Department of Economics Discussion Paper 13-13
Burcu Kapar William Pouliot
Multiple Change-Point Detection in Linear Regression Models via U -Statistic Type Processes Burcu Kapar, Department of Economics, City University London William Pouliot, Department of Economics, University of Birmingham∗ This Version MAY, 2013
Abstract Many procedures have been developed that are suited to testing for multiple changes in parameters of regression models which occur at unknown times. Most notably, Brown, Durbin and Evans [11] and Dufour [15], have developed or extended existing techniques, but said extensions lack power for detecting changes (cf. Kramer, Ploberger, Alt [24] and Pouliot [32]) in the intercept parameter of linear regression models. Orasch [26] has developed a stochastic process that easily accommodates testing for many change-points that occur at unknown times. A slight modification of his process is suggested here which improves the power of statistics fashioned from it. These statistics are then used to construct tests to detect multiple changes in intercept in linear regression models. It is also shown here that this slightly altered processes, when weighted by appropriately chosen functions, is sensitive to detection of multiple changes in intercept that occur both early and later on in the sample, while maintaining sensitivity to changes that occur in the middle of the sample.
JEL Codes: C1, C2. Keywords: Structural Breaks, U -Statistics, Brownian Bridge, Linear Regression Model.
1
Introduction
The problem of detecting abrupt parameter changes in models that are frequently employed in widely different disciplines of science and social science research has led to a variety of methods to detect such changes. To date, a number of papers have proposed and developed statistical methods that are capable of detecting changes in parameters of regression models that occur at unknown times. Brown, Durbin and Evans (BDE) [11], Dufour [15] and Ploberger, Kramer and Kontrus (hereafter PKK) [31] are three of the many papers. BDE uses the sum of recursive residuals and their square to fashion statistics to detect structural change, these statistics are referred to here and in the literature as CUSUM tests, that test for multiple changes in slope parameters. Dufour [15] extends the framework of Chow [12] by developing test to detect multiple changes in slope parameters of regression models that occur at unknown times. Kramer, Ploberger and Alt [24], however, show that the power of the tests developed by Dufour can, in certain situations, fall below the nominal significance level of the test. ∗
Corresponding Address: Department of Economics, University of Birmingham, Birmingham West Midlands, B5 2TT. E-mail:
[email protected]
1
PKK extend the fluctuation test, a test based on successive parameter estimates, rather than recursive residuals as in the CUSUM test of BDE. They propose a test for the null hypothesis of parameter constancy whenever estimates of the parameters fluctuate too much, just how much parameters can fluctuate is measured via a test statistic that requires taking two maximum, the first over estimators of the parameters of a linear regression model and the second over the recursive parameter estimates. Their test is complicated to implement and, moreover, Ploberger and Kramer [30] show, via Monte Carlo results, that their fluctuation test fails to dominate the CUSUM test in small samples. Alternative procedures such as those developed by Altissimo and Corradi [1] can also detect the presence of multiple changes in a time-series. Their test, however, requires large sample sizes as well as large changes in the parameters of regression models. There are non-parametric tests that have recently been developed by Kristensen [22] that can also detect changes in parameters of regression models. Given that implementation of many of these tests is complicated as well as lack of clear superiority of any one method, there still remains the need for testing procedures that are capable of testing for multiple changes in these parameters that occur early and later on in the sample, while, also maintain power to detect changes in small samples as well as for small changes in these parameters. Here a test statistic is constructed that displays considerable power to detect a change in the intercept of linear regression models. Most of the current tests display a lack of power when it comes to detecting changes in the intercept parameter (cf. Olmo and Pouliot [25]). Since most of the widely used tests lack sensitivity to detect changes in the intercept, it is imperative then to develop tests with power for detecting changes for changes in this parameters. The purpose here is to develop one such test statistic. In what follows, Section 2 provide details on a weighted two parameter process and lists a number of its asymptotic properties. Section 3 uses the results detailed in Section 2 to construct test statistics. Section 4 describes the testing procedure developed by Bai and Perron [9] and implemented in Bai and Perron [7]. Section 5 details the results of a Monte Carlo exercise and Section 6 concludes. Tabulated cumulative distribution functions for many of the statistics constructed in Section 3 are detailed in Section 7.
2
Weighted Two Parameter Stochastic Processes
To set the stage for this research consider the following situation: a large dataset is collect and the investigator is interested in estimating a model similar in nature to that described in equation (1). 0
Yt = β0 + β Xt + t ,
(1)
for t = 1, . . . , T . The sequence {(Yi , Xi )}Ti=1 is a random sample, β is k x 1 vector of slope parameters and Xt is a k x 1 vector of covariates. Here, concern is with detecting at-most-two changes in intercept which can be parameterised in the following way; (1) 0 ? β0 + β Xt + σεt 1 < t ≤ k1 0 (2) Yt = β0 + β Xt + σεt k2? < i ≤ k2? 0 (3) β0 + β Xt + σεt k3? < i ≤ T
2
(2)
where t are independent and identically distributed random variables with IIE1 = 0, IIE2t = 1 and IIE|t |4 < ∞, (1)
(2)
t = 1, . . . , T.
(3)
(3)
0
The values of the parameters β0 , β0 , β0 , β , σ k1? and k2? are unknown. Assuming that at least (1) (2) (3) one of the following inequalities holds β0 6= β0 6= β0 , the no change in intercept null hypothesis can be formulated as HO : k1? ≥ T versus the at-most-two change (AMTC) in variance and or slope alternative HA : 1 ≤ k1? ≤ k2? < T. The task here is to construct a statistics that displays power for detecting multiple changes in intercept or variance. Section 2 adapts the framework of Orasch [26] to the model considered in (2) through appropriate choice of kernel, then derives the asymptotic distributions of these slightly altered process under conditions when the parameters are known and then unknown. It will be shown here that there is no estimation risk associated with substitution of estimators for unknown parameters. Orasch [26] points out that within the context of testing for at most one change (AMOC) in parameters of a distribution a one-time parameter stochastic process proved useful in constructing a statistic capable of testing the null hypothesis of no change in the parameter under investigation. It would seem natural, then, that in order to test for at most two changes in the parameter to fashion a two-time parameter stochastic process. Orasch [26] provides some motivation for this and then provides a rigorous geometrical argument to justify this approach. The insight that can be taken from his theory is that if one wishes to test for at most s changes in a parameter, then one should base a test statistic on an s-parameter stochastic process. Returning to construction of an appropriate two parameter stochastic process, consider the following stochastic process:
NT (τ1 , τ2 ) =
[T τ1 ]
[T τ2 ]
X
X
t=1 j=[T τ1 ]+1
[T τ1 ]
h(Xt , Xj ) +
X
T X
t=1 j=[T τ2 ]+1
[T τ2 ]
h(Xt , Xj ) +
X
n X
h(Xt , Xj ),
t=[T τ1 ]+1 j=[T τ2 ]+1
(4) where 1 ≤ τ1 ≤ τ2 < T . Here, interest is with h(x, y) = f (x; Θ) − f (y; Θ), where Θ is a vector of kx1 parameters. After substitution for h(· ; ·) in (4), the equation reduces to NT (τ1 , τ2 ) MT (τ1 , τ2 ) = T 3/2 P[T τ2 ] P[T τ ] P[T τ ] P T t=1 f (Xt ; Θ) − [T τ1 ] t=12 f (Xt ; Θ) + [T τ2 ] t=11 f (Xt ; Θ) − [T τ2 ] Tt=1 f (Xt ; Θ) = T −3/2 P P PT PT [T τ2 ] [T τ1 ] f (X ; Θ) − τ f (X ; Θ) f (X ; Θ) − τ f (X ; Θ t 1 t t 2 t t=1 t=1 t=1 + τ2 = (1 − τ1 ) 1/2 T T 1/2 = (1 − τ1 )MT (τ2 ) + τ2 MT (τ1 ). (5) 3
Equation (5) is what is called a two sample U -statistic type process in τ1 and τ2 for each T . In the case τ = τ1 = τ2 , the the family of statistics in (4) reduces to a one sample U -statistic type process that can be used to test for at most one change. Here, interest is in model (2) and testing for at most two changes in intercept or variance. To fashion a statistic, similar in nature to the statistics for testing for AMOC, we must select the appropriate kernel. Before the asymptotic properties of process (5) can be advertised in a series of propositions, further notations and definitions need to be introduced. Definition 2.1. Let Q be the class of positive functions on (0, 1) which are non-decreasing in a neighbourhood of zero and non-increasing in a neighbourhood of one, where a function q(·) defined on (0,1) is called positive if inf
δ≤τ ≤1−δ
q(τ ) > 0
for all δ ∈ (0, 1/2).
Definition 2.2. Let q(·) ε Q. Then define Z
1
I(q, c) := 0
c 1 − exp (τ (1−τ ))q2 (τ ) dτ τ (1 − τ )
for some constant c > 0. Let {Xi }Ti=1 be a sequence of i.i.d rvs with finite mean and variance, i.e., 0 < ∆ = IIE[(Z1 − IIE[Z1 ])2 ] < ∞,
(6)
IIE[f (Z1 , Θ)] = Λ, 2
0 < IIE[f (Z1 , Θ)] − Λ
2
(7)
= Ψ < ∞.
(8)
It is now possible to make the following conclusions regarding appropriately normalized versions of (5). Proposition 2.1. Let {Xt }Tt=1 be a sequence of i.i.d rvs that satisfy (6), assume (7) and (8) hold. D
Then a sequence of Gaussian processes can be defined such that {ΓT (τ1 , τ2 ); 0 ≤ τ1 ≤ τ2 ≤ 1} = {Γ(τ1 , τ2 ); 0 ≤ τ1 ≤ τ2 ≤ 1} holds for each T ≥ 1, such that the following hold: a) Let q(τ1 , τ2 ) = g(τ1 ) + h(τ2 ), q, h ∈ Q. If I(g, c), I(h, c) < ∞ for all c > 0 then, as T → ∞, sup 0≤t1 ≤t2 ≤1
1 MT (τ1 , τ2 ) − ΓT (τ1 , τ2 )| | Ψ1/2
q(τ1 , τ2 )
= oP (1).
b) Let q(τ1 , τ2 ) = g(τ1 ) + h(τ2 ), q, h ∈ Q. If I(g, c), I(h, c) < ∞ for some c > 0 then, as T → ∞, sup 0≤t1 ≤t2 ≤1
1 | Ψ1/2 MT (τ1 , τ2 ) − ΓT (τ1 , τ2 )|
q(τ1 , τ2 )
4
= OP (1).
c) Let q(τ1 , τ2 ) = g(τ1 ) + h(τ2 ), q, h ∈ Q. If I(g, c), I(h, c) < ∞ for some c > 0 then, as T → ∞, 1 | Ψ1/2 MT (τ1 , τ2 )|
sup
q(τ1 , τ2 )
1 ≤τ1 ≤τ2 ≤ T T+1 T +1
D
−→
|Γ(τ1 , τ2 )| . 0
