Invariance of the likelihood-ratio test for time-varying autoregressive ...

Invariance of the likelihood-ratio test for time-varying autoregressive (TVAR) model order estimation under the H0 hypothesis Yuri Abramovich1 and Nicholas Spencer*2 1

Defence Science & Technology Organisation (DSTO), Adelaide, Australia 2 Adelaide Research & Innovation Pty Ltd (ARI), Adelaide, Australia

N. Spencer is supported by DSTO through the Centre of Expertise in Phased Array and Microwave Radar Systems (CEPAMiR), established in partnership between DSTO and the University of Adelaide, Australia.

1 Introduction Methods for order estimation and parameter estimation of a stationary autoregressive (AR) model of order m, AR(m), given a set of T independent identically distributed (i.i.d.) N-variate Gaussian samples, are well established. These methods usually require a complete identification of the model, which can be achieved only approximately in the maximum-likelihood (ML) sense. In some applications, the strict stationarity of the observed training data is questionable, so a general problem is to select between a stationary and a time-varying model.

2 Order estimation of an AR/TVAR model In Abramovich et al. (2007a) we demonstrated that the necessary and sufficient condition for an N-variate complex vector x ≡ [x1 , . . . , xN ]T to be a sample of the TVAR(m) process xj =

m X

a∗kj xj−k + ηj

j = m + 1, . . . , N,

for

E{ηj ηk∗ } = σ02 δjk ,

a0 = 1 (1)

k=1

(m)

is that its positive-definite (p.d.) Hermitian covariance matrix RN “band-inverse” property (m)

{[RN ]−1 }jk = 0

for

 ≡ E xxH satisfies the

|j − k| > m

(2)

ie. the elements of its inverse are zero outside the (2m + 1)-wide diagonal band. Since the p.d. Toeplitz covariance matrix of the stationary AR(m) model has this same property, a test for “band-inverseness” may be seen as a unified test for estimating the model order m, irrespective of its stationary or time-varying nature. Such a test can be constructed from the properties of the probability density function (p.d.f.) of a certain likelihood ratio (LR). We consider a set of T i.i.d. N-variate training data (j)

(j)

xj ≡ [x1 , . . . , xN ]T

for

j = 1, . . . , T

(3)

that are samples of a complex Gaussian random process whose p.d.f. is CN (0, RN ), where ˆ = RNPis an N-variate p.d. Hermitian matrix. The conventional sample covariance matrix R T 1 H ˆ j=1 xj xj is rank deficient for T < N, and the matrix T R is described by the anti-Wishart T 91

complex distribution ACW (T < N, N, RN ) (Janik & Nowak, 2003). Yet, for T ≥ m + 1, all ˆ q(m) of R ˆ are p.d. (Anderson, 1958), ie. (m + 1)-variate central block matrices R   rˆqq . . . rˆq,q+m  . . . .. ˆ q(m) ≡  R for q = 1, . . . , N − m. (4)  .. >0 . . rˆq+m,q . . . rˆq+m,q+m In Abramovich et al. (2007a) we demonstrated that this condition is necessary and sufficient for the existence of an accurate nondegenerate ML estimate of a TVAR(m) covariance matrix that ˆ q(m) using the Dym–Gohberg formula (Dym & Gohberg, is calculated directly from the blocks R 1981): ˆ (m) = [Vˆ (m)H ]−1 [Vˆ (m) ]−1 R (5) TVAR where Vˆ (m) is a lower-triangular matrix whose elements are defined as ( 1 (m) (m)− for j ≤ i ≤ L(j) vˆij vˆjj 2 (m) ˆ Vij ≡ 0 otherwise

(6)

where 

(m) vˆqq





   .. =  . (m) vˆL(q),q

rˆqq ... .. .. . . rˆL(q),q . . .

−1

rˆq,L(q) .. . rˆL(q),L(q)

 

    

1 0 .. . 0



  ˆ (L) ]−1 eL(q)−q+1  ≡ [R q 

(7)

are the time-varying AR coefficients, with L(q) ≡ min{N, q + m}, and ez is the z-variate unit vector. This ML TVAR(m) covariance matrix is uniquely specified by the remarkable properties o n ˆ (m) = rˆij for |i − j| ≤ m R TVAR ij

h i−1  (m) ˆ RTVAR =0

for

|i − j| > m.

(8)

ij

For a stationary AR(m) model, the inverse of its ML-optimal Toeplitz covariance matrix estiˆ (m) ]−1 , and of the same bandwidth, but its elements cannot mate is also a band matrix, like [R TVAR ˆ only numerical solutions are currently available for be directly and simply obtained from R; ML Toeplitz covariance matrix estimation, and so suboptimal solutions are usually suggested (Grigoriadis et al., 1994). ˆ (m) , the Gaussian likelihood function For the ML estimate R TVAR LF (X, R) =

ˆ exp[− tr(T R−1 R)]

(9)

[det R]T

evaluates to (m)

ˆ max LF (X, R) = LF (X, R TVAR ) = exp[−NT ] R

where

N Y q=1

(m) ˆ (L) ]−1 eL(q)−q+1 . vˆqq ≡ eTL(q)−q+1 [R q

92

(m) vˆqq

!T

(10)

(11)

In fact, (8) follows directly from the ML equation ∂ log LF (X, R)/∂(R−1 )ij = 0 subject to the ˆ ≡ [R ˆ (m) ]−1 we get TVAR(m) constraint (R−1 )ij = 0 for |i − j| > m. According to (8), for B TVAR ˆ R] ˆ = tr[B ˆR ˆ (m) ] = NT, tr[B TVAR

ˆ = det[Vˆ (m)H Vˆ (m) ] = det B

∴

N Y

(m) vˆqq

(12)

q=1

by (5). Let mmax be the maximum admissible order of a TVAR(m) model that is identifiable for the sample volume T , then mmax + 1 ≤ T . From the “nested” property of the model-order testing problem, and directly from (10), it is evident that ˆ (m1 ) ) ≥ LF (X, R ˆ (m2 ) ) LF (X, R TVAR TVAR

m1 > m2

for

(13)

and so our hypothesis test for the TVAR(m) order can be based on the likelihood ratio ˆ (m) ] ˆ (m) ] LF [X, R LF [X, R TVAR TVAR = LR(m) = = (µ) (m ˆ ˆ max ) ] max LF [X, R LF [X, R TVAR ] TVAR µ≤mmax

N Y q=1

(m)

vˆqq (mmax ) vˆqq

!T

(14)

where, according to (7) and (11), (m ) ˆ (K) ]−1 eK(q)−q+1 vˆqq max ≡ eTK(q)−q+1 [R q (m

(15)

)

with K(q) ≡ min{N, q + mmax }. Now since vˆqq = vˆqq max for q ≥ N − m, the LR is (m)

LR(m) =

N −m−1 Y q=1

ˆ q(K) is Note that the dimension of the matrix R  mmax + 1 (K) ˆ dim Rq = m+2 We introduce the notation  mmax µ≡ N −q

for for

(m)

vˆqq (mmax ) vˆqq

!T

.

q ≤ N − mmax q = N − m − 1.

for for

q < N − mmax N − mmax ≤ q ≤ N − m − 1.

(16)

(17)

(18)

Instead of LR(m), we can deal with LR0 ≡ [LR]1/T ; and so we investigate the p.d.f. of the LR LR0 (m) =

N −m−1 Y q=1

(m)

vˆqq

(µ)

vˆqq

.

(19)

Theorem 1. Let m0 be the true order of the AR or TVAR input data, then for all m ≥ m0 , the p.d.f. of LR0 (m) does not depend on scenario, ie. is completely defined by the parameters {N, T, mmax , m}. Specifically,   mmax −m,...,mmax −m (N −m−1),0 (T −mmax −1) f (x) = C(N, T, mmax , m) x G(N −m−1),(N −m−1) x mmax −m−1,...,0,...,0 (20) 93

where Ga,b c,d (·) is Meijer’s G-function (Gradshteyn & Ryzhik, 2000), and N −m−1 Y

Γ(T − m) . Γ(T − µ)

(21)

Γ(T − m)Γ(T − µ + p) . Γ(T − µ)Γ(T − m + p)

(22)

C(N, T, mmax , m) =

q=1

The pth moment of LR0 (m ≥ m0 ) is p

E{x } =

N −m−1 Y q=1

Moreover, the p.d.f. of LR0 (m) can be expressed as the p.d.f. of a product of (N − m − 1) independent random numbers βq : LR0 (m) =

N −m−1 Y

(T −µ−1)

βq ,

with

βq ∼

q=1

βq

(1 − βq )(µ−m−1) . B[µ − m, T − µ]

(23)

For the proof, see Appendix I of Abramovich et al. (2007b).

3 Final remarks The “scenario-invariant” nature of the LR allows us to derive a test to estimate the order of an AR or TVAR model et al. (2007b). Computationally, it may be easier to deal with the expression (23) than the somewhat obscure analytic formula (20).

References Abramovich, Y., Spencer, N., & Turley, M. (2007a). Time-varying autoregressive (TVAR) models for multiple radar observations. IEEE Trans. Sig. Proc., 55, 1298–1311. Abramovich, Y., Spencer, N., & Turley, M. (2007b). Order estimation and discrimination between stationary and time-varying autoregressive (TVAR) models. IEEE Trans. Sig. Proc., 55, 2861-2876. Anderson, T. (1958). An Introduction to Multivariate Statistical Analysis.New York, Wiley. Dym, H. & Gohberg, I. (1981). Extensions of band matrices with band inverses. Linear Algebra Appl., 36, 1-24. Gradshteyn, I. & Ryzhik, I. (2000). Tables of Integrals, Series, and Products.New York, Academic Press, 6th edition. Grigoriadis, K., Frazho, A., & Skelton, R. (1994). Application of alternating convex projection methods for computation of positive definite Toeplitz matrices. IEEE Trans. Sig. Proc., 42, 1873-1975. Janik, R. & Nowak, M. (2003). Wishart and anti-Wishart random matrices. J. Phys. A: Math. Gen., 36, 3629–3637.

94