Fast Recursive Low-Rank Linear Prediction Frequency Estimation ...

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 44, NO. 4, APRIL 1996

834

Fast Recursive Low-Rank Linear Prediction Frequency Estimation Algorithms Peter Strobach, Senior Member, IEEE

Abstract-A class of fast recursive low-rank linear prediction algorithms for the tracking of time-varying frequencies of multiple nonstationary sinusoids in noise is introduced. Realizations with O ( N n ) and O ( N n 2 )arithmetic operations per time step are described, where N is the model order and n is the number of independent sinusoids. The key step towards an operations count that depends only linearly on the model order is fast eigensubspace tracking, and the property that the coefficients of a high-order N prediction filter itself constitute a perfectly (or almost perfectly) predictable sequence that can be annihilated using a low-order 2n prediction error filter that carries the desired signal frequency information in its roots. In this concept, root tracking is limited to a low-order filter polynomial, even if the overmodeling factor N / n is much larger than 1 for optimal noise suppression. Extraneous roots are not computed explicitly. Detailed simulation results confirm the excellent tracking capabilities of the new algorithms.

I. INTRODUCTION LASSICAL high-resolution spectral estimation algorithms based on linear prediction, overmodeling, and rank reduction using eigendecomposition [ 11-[ 101 assume that the frequencies of a multiple sinusoid sequence are constant in time. This is, of course, a very limiting assumption because, in many practical cases, the sinusoid frequencies will be slowly and smoothly or even abruptly changing functions of time. A prominent example is the tracking of a set of maneuvering emitters in space. Even if the emitted frequencies are constant, a linear equispaced array of sensors will convert the received signal into a sum of spatial sinusoids with timevarying frequencies. Other examples of multiple time-varying sinusoids appear in communications, particularly in carrier frequency systems, and in biomedical signal analysis. In this paper, we describe a class of time-recursive adaptive algorithms for tracking the time-varying frequencies of nonstationary sources. The algorithms can be viewed as extensions of the widely discussed classical low-rank linear prediction frequency estimation algorithms [2]-[9]. For a good survey, see [lo]. Just like the classical batch algorithms, our timerecursive algorithms are based on a low-rank approximation of an oversized covariance matrix, linear parameterization and root determination of a linear prediction polynomial. Fast sequential low-rank approximation is accomplished using the O ( N n ) and O ( N n 2 ) subspace trackers known from the recently introduced fast low-rank or eigensubspace adaptive Manuscript received March 11, 1995; revised October 16, 1995. The associate editor coordinating the review of this paper and approving it for publication was Prof. Michael D. Zoltowski. The author is with Fachhochschule Furtwangen, Furtwangen, Germany. Publisher Item Identifier S 1053-587X(96)02766-1.

filters [11], [12]. These subspace trackers have proven outstanding tracking capability and robustness. They are based on a particularly useful fast sequential variant of simultaneous orthogonal iteration [131. Once the signal subspace is known, the classical approach would require that a high-order prediction error filter polynomial is formed, and the desired frequencies are determined as the angles of the dominant complex conjugate root pairs which lie on the unit circle. Unfortunately, though, the computation of these root pairs is extremely time consuming because the dominant roots can only be found when all the roots of the polynomial (including a large number of “extraneous” roots) are calculated explicitly. The situation is dramatic when the overmodeling factor N / n is large for optimal noise suppression. Of course, in a fast time-recursive algorithm, we cannot expend a significant amount of computations just to determine useless extraneous roots. Therefore, a new approach that completely circumvents the computation of extraneous roots has been developed. The basic idea is that a filter that annihilates a multiple sinusoid plus noise process can be decomposed into a high-order prediction filter followed by a low-order prediction error filter. The high-order (order N ) prediction filter provides the “cleaned” sinusoids which can be “annihilated” almost perfectly in a subsequent prediction error filter of order T = 2n. The loworder prediction error filter finally carries the desired frequency information in its roots. Thus, only the roots of a minimum degree prediction error filter polynomial must be tracked explicitly in our algorithms. We show that the high-order prediction filter that “cleans” the input process from noise can be computed in a relatively straightforward manner using the output of a dominant signal subspace tracker. The “trick” in the approach is the determination of the subsequent low-order prediction error filter without computing the cleaned signal explicitly. The method utilizes the property that in the ideal case, the coefficients of the large prediction filter vector itself constitute a perfectly predictable sequence. Thus, it turns out that the coefficients of the frequency revealing low-order prediction error filter can be determined as the solution of an overdetermined “Prony-type” problem in the high-order prediction filter coefficient domain. This step, again, is computationally efficient and requires only O ( N n ) O ( n 3 )arithmetic operations per time step. The paper is organized as follows. In Section 11, we summarize some basic principles of frequency estimation using linear prediction and rank reduction. These principles are required in the further sections. Section 111describes the design

+

1053-587X/96$05.00 0 1996 IEEE

~

STROBACH: FAST RECURSIVE LOW-RANK LlNEAR PREDICTION FREQUENCY ESTIMATION ALGORITHMS

of the frequency revealing low-order prediction error filter and the principles of root computation which we use in this paper. In Section IV, three time-recursive frequency estimation algorithms based on this concept are developed. In Section V, we extend the algorithms to the more general case of correlated Gaussian noise. In Section VI, the tracking charactiristics of the sequential frequency estimation algorithms are studied and demonstrated experimentally. Section VI1 lists ithe main conclusions of this paper. 11. LINEARPREDICTION, OVERMODELING, AND RANKREDUCTION Consider the class of discrete-time processes z ( t ) ,which can be modeled as a sum of incoherent sinusoids s ( t ) in zero-mean white Gaussian noise ~ ( t as ) , follows:

4 t ) = s(t)+ V ( t )

(14

,=I

where {A,, 1 5 I, 5 n } is a set of constant amplitude factors and {pV,1 5 I, 5 n } is a set of initial phase angles. The problem widely encountered in signal processing is the estimation of the normalized frequencies {U,, 1 5 v 5 n } of the sinusoids which add up in the signal s ( t ) from a probably very large number of noisy observations z ( t ) . We shall first introduce some essential principles for estimating these normalized frequencies using linear prediction, overmodeling, and rank reduction. Proofs are partly omitted because they can be looked up easily in the basic literature about linear prediction frequency estimation. See [11-[9] or [lo] and the references listed therein. Define a data snapshot vector ~ ( tof) dimension N as follows: Z(t) =

[ z ( t )ic(t , - l ) ,. . , z ( t - N '

+ 1)]T

(2)

+

Hence, ~ ( t=)s ( t ) ~ ( twhere ) s ( t ) and ~ ( tare) signal and noise snapshot vectors, respectively. Define the data, signal, and noise covariance matrices piX, pi,, and pi, as folllows: pi, = E ( Z ( t ) Z T ( t ) )

@TI

= E(V(t)VT(t)).

+ pi,.

(3c)

(4)

In the outset, we further assume that the noise is white, as follows: pi, = a21N

The signal covariance matrix @, obeys the following characteristic eigenvalue deomposition (EVD) [9], [ 131:

where V, is an N x N real orthonormal matrix of eigenvectors and V$.) is an N x T real submatrix with orthonormal columns. A:) is an T x T diagonal matrix of eigenvalues. Note that we have precisely rank(@,) = T = 2n

(7)

in this problem because a set of n arbitrary phased sinusoids occupies at most a subspace of dimension 2n. Using (4)-(6), the elements of the EVD of piX can be expressed in terms of the EVD of pi, and the noise as follows:

Clearly, from (8), it follows that the dominant siignal eigenvector matrix V f ) is not perturbed by additive white Gaussian noise; therefore, range ( v f ) ) = range ( ~ 5 ) ) .

(94

Furthermore, a comparison of the terms in (8) reveals that we must have

A;) = A$.) + a21T

(9b)

AY-') = c r 2 I ~ - ' .

( 1Ob)

(34

Suppose that signal and noise are statistically independent. Then the data covariance matrix can be represented a!; the sum of the signal and the noise covariance matrices as follows: @, = pis

835

(5)

where o2 is the noise variance and I N is an identity matrix of dimension N x N . This whiteness assumption will later be relaxed (see Section V).

Introduce a minimum variance rank follows:

2,

T

approximant

= Vg)AF)VF)'

and the associated Moore-Penrose follows:

2,

as

(1 la)

-+ pseudoinverse pi, [13], as

c,t IJp@rlIJpT

Ulb)

Define a projection operator P , that projects vectors orthogonally onto the (dominant signal subspace spanned by the columns of v): as follows:

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 44, NO. 4, APRIL 1996

836

Using this projection operator, we may reconstruct the signal component from an observed snapshot vector ~ ( tvia) projection of ~ ( tonto ) the dominant signal subspace spanned by the column vectors of v:): q t ) = P,z(t).

(13)

Investigate the covariance matrix @s of s(t),as follows:

Prediction Filter

Fig. 1. Predictor-annihilator filter.

111. THE PREDICTOR-ANNIHILATOR APPROACH A. Basic Concept

to verify that the reconstruction still contains noise. Define the signal-to-noise ratio (SNR) pz of the raw data

As pointed out above, classical linear prediction frequency estimation based on overmodeling and rank reduction will x(t): mainly suffer from the necessity to calculate the complete set pz = t r (Qs)/tr ( Q V ) = t r ( A f ) ) / ( N a 2 ) (15a) of roots of a possibly very large prediction error filter polynomial, even if only a few frequencies are sought. The problem and the corresponding SNR pa of the reconstructed signal s(t). can be eliminated using the following “predictor-annihilator’’ as follows: concept illustrated in Fig. 1. A “predictor-annihilator’’ filter pa = t r ( A F ) ) / ( r g ’ ) (15b) is defined as a cascade of the high-order prediction filter w and an order r prediction error filter [I,-aTIT. Clearly, to see that the reconstruction step improves the SNR by a the predictor w reduces the noise in x ( t ) by a factor of factor of N I T , as follows: NIT. In practical cases where N / r is large, the prediction filter suppresses the noise almost perfectly, and produces an N pa = -pz. almost “clean” output signal 8 ( t ) which consists of the sum r of the n sinusoids plus a negligible noise residual. Such a The projection operator P, in (13) may be interpreted as a sequence, however, is almost perfectly predictable. Hence, matrix-valued prediction filter for the signal s ( t ) . Hence, each the subsequent low-order prediction error filter annihilates the column (or row) vector of P , constitutes a valid prediction predictable sinusoids and produces a very small final noise filter vector. In the following, we assume that N is odd and use residual ~ ( t The ) . desired frequencies of the sinusoids in ~ ( t ) a center pinning vector 7rc = [O . ’ 0 , 1. o . . OIT to select a are obtained as the angles of the n complex conjugate root prediction filter vector w of dimension N as the center column pairs of the annihilator polynomial E,(%),as follows: of P , :

w = [wI, ~ 2 , . .. . W

N ] ~ = P,T,

= V g ) V g ) T ~ c (17) .

At each time step, the center component i(t) = 7rTs(t) of the reconstructed signal vector s(t) may hence be computed using an order N transversal prediction filter with coefficient vector w as follows:

E,(%) = 1 p=1

The significant advantage is that in such a concept, only a degree-r polynomial must be rooted. Thus, the complexity in the final frequency determination process involving nonlinear root finding becomes independent of the overmodeling factor i(t) = W T Z ( t ) . (18) N/n. A large overmodeling factor can be selected for optimal Usually, in linear prediction based frequency estimation, the noise suppression. Thus, the predictor-annihilator approach desired normalized frequencies { w V , 1 5 7 1 5 n } of the is also a constructive basis for the fast sequential sinusoid sinusoids in s ( t ) are determined by the roots of the following frequency tracking algorithms which are introduced in the following sections. linear prediction (prediction error filter) polynomial E, (2): As the computation of w according to (17) presents no real E,(z) = 1 - Pw(.) (194 problem even for very large N , the question arises how the coefficient vector a of the low-order annihilator filter can be N determined. Of course, it would be very unwise to quantify the P W ( %=) Wv%-v. 1901 “cleaned’ signal i(t) explicitly for this purpose. A clever dev=l sign rule for a can be established directly using the properties The complete set of roots of E , ( z ) must be computed, and the of the prediction filter coefficient vector w. Recall that w is just normalized frequencies are determined by the angles of the n a weighted linear combination of the dominant eigenvectors “dominant” complex-conjugate root pairs which lie on the unit in V $ ) .These dominant eigenvectors, however, represent the circle. The complete rooting of E,(z) is a numerically and fundamental modes of the signal s ( t ) . This signal consists computationally burdensome problem, particularly in cases entirely of sinusoidal components according to (lb). Thus, it where the model order N is much larger than the number n follows immediately that in the ideal case, the elements of w of incoherent sinusoids in ~ ( t In ) .practice, it is often desired satisfy a “Prony law” [9], and therefore constitute a perfectly to select the overmodeling factor N / n much larger than 1 for predictable sequence. Moreover, the fundamental modes in improved noise suppression in the formal prediction filtering w represent the frequencies of the sinusoids in s ( t ) . Hence, the low-order prediction error filter [I, -uTlT that annihilates step (18).

STROBACH: FAST RECURSIVE LOW-RANK LINEAR PREDICTION FREQUENCY ESTIMATION ALGORITHMS

the signal will also annihilate the fundamental modes in w. Conversely, we may exploit this key property to compute the desired coefficients a of the annihilator filter directly as the solution of an overdetermined "Prony-type" problem in the prediction filter coefficient domain. For this purpose, construct an ( N - T ) x (T 1) data matrix T , which contains ithe shifted elements of TU as follows:

+

rw,+i

Wr

L WN

UN-1

...

WN-2

w1

" '

831

The results obtained using this approach are generally far better than the results from the usually recommended combined forwardhackward linear prediction approach [ 101.

C. Root Computation Finally, use the elements of the coefficient vector a to construct a companion matrix C as follows:

1

WN--rl

The Prony law for an ideal predictor w can be formulated as follows:

Note that the roots

E,(z) = 1 Hence, the large towel matrix T , in (22) will have al singular value decomposition (SVD) with one vanishing singular value. Thus, the desired solution [l, -uTIT is formally determined by the minimum right singular vector associated with the vanishing singular value in the SVD of Tu>. A practical computation of the coefficient vector a can be based on the least squares solution of the following; overdetermined system of linear equations:

{zk,

1 5 IC

5

r}

of E,(z)

apz-'l

can be computed as the eigenvalues of the associated companion matrix C . The computation of the eigenvalues of C is based on the real Schur theorem [ 131, which states that any real square matrix C (not necessarily restricted to companion shape) is orthogonally similar to an upper-right Hessenberg matrix H that displays the eigenvalues of C in blocks of size 1 x 1 or 2 x 2 on the main tridiagonal of H as follows:

H = UTCU.

This is a standard linear prediction problem [l] in the coefficient domain which can be solved in O(Nr)+O(r3)arithmetic operations taking into account the special banded structure of T w.

(29)

Real eigenvalues correspond to 1 x 1 blocks and complex conjugate pairs of eigenvalues correspond to 2 x 2 blocks on the main tridiagonal of H . Fig. 2 shows three typical configurations of H in an example of r = 4. The comB. Implementation Aspects plex conjugate eigenvalue pairs can be computed in closed Several important implementation aspects may lbe taken form once the corresponding 2 x 2 Schur blocks are known. into account in the practical least squares solutioin of the Thus, in harmonic retrieval problems, the eigenvalue revealing overdetermined system for the desired annihilator coefficients Hessenberg matrix H exhibits a block structure as shown a according to (23). First, note that the annihilator polynomial in Fig. 2(a). The orthonormal transform matrix U in (29) is E,(z) will be symmetric (or almost symmetric) belcause its called a linear invariant matrix with respect to (7. Orthonormal roots are located exactly (or almost exactly) on the unit circle. linear invariant matrices can be computed iteratively using On the other hand, it is fairly unwise to constrain the (extended the following two-term recurrence known as simultaneous coefficient vector [I, -aTIT in (23) to a symmetric shape a orthogonal iteration [ 131: priori. By far the best results have been obtained using the U ( 0 )= I following concept called "centered prediction." The centered for p = 1, 2,3 , . . .until convergence iterate: prediction approach is based on a "centered' intermediate (30) 4 P L ) = CU(P - 1) solution vector U(') of the type A ( p ) = U ( p ) R ( p ) : QR factorization.

[

We first solve the intermediate least squares problem as follows: T , ~ ( c ) 1e:

eTe = min -+ ukJ

(25)

and obtain the desired solution via rescaling from afi as follows:

Modern eigenvalue routines for constant matrices usually avoid the direct computation of U because each iteration in (30) requires O ( r 3 )arithmetic operations. The so-called QR algorithm (see [113], ch. 71) is probably the rnost prominent method that iterates directly on f1 and avoids an explicit quantification of U.The QR algorithm requires only O ( r 2 ) arithmetic operations per iteration. Thus, the QR algorithm is the appropriate method as long as C is fixed. In the case of frequency tracking, as treated in the following sections, C will be time varying. In this case, it becomes very difficult to

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 44, NO. 4, APRIL 1996

838

TABLE I LOW-RANK LINEAR PREDICTION FREQUENCY ESTIMATION ALGORITHM BASEDON THE PREDICTOR-ANNIHILATOR EQUATIONS NUMBERED AS THEYAPPEAR IN TEXT

CONCEPT.

(a)

(b)

(C)

Estimate d a t a covariance matrix Ox

Fig. 2. Three possible configurations of Schur blocks of size 1 x 1 and 2 x 2 on the main tridiagonal of an eigenvalue revealing real 4 x 4 Hessenberg matrix H : (a) Two complex conjugate pairs of eigenvalues; (b) two real eigenvalues and one complex conjugate pair of eigenvalues; (c) four real eigenvalues.

O x = E(X(tiXT(t1)

(34

C o m p u t e dominant eigenvectors IT =

[w,' w z , .

. , WN]T

C o n s t r u c t towel matrix

update the QR algorithm because the QR algorithm does not iterate on G directly. Thus, in the time-varying case, it will be more expedient to return to the orthogonal iteration principle (30). In [14], a fast sequential 0 (v 2 ) orthogonal iteration algorithm for tracking of the eigenvalues of a time-varying companion matrix C ( t ) has been introduced. Some further remarks about root calculation using the companion matrix method seem to be in order. We know from numerical analysis that the necessary condition for convergence of any root-finding algorithm based on eigenvalue computation of a companion matrix is that the roots are all different in their absolute values [13], [14]. But just this condition is violated here because all the roots of the loworder linear prediction polynomial E, ( 2 ) are placed exactly (or almost exactly) on the unit circle in the z-plane, and therefore these roots are all identical in terms of their absolute values. Fortunately, there exists a very clever ".trick" to circumvent this fundamental difficulty. In [14], we have demonstrated in detail that a linear prediction polynomial which has its roots all placed on the unit circle in the z plane can be transformed into an s domain using the familiar bilinear transformation [15]. The effect is that the roots are mapped from the unit circle in the z domain into the imaginary axis in the s domain. As a direct consequence, all roots with identical absolute value and different angle in the z domain will appear as roots with different absolute values in the s domain. Hence, the method is to transform a given polynomial E, ( 2 ) from the z domain into the s domain, and apply the companion matrix root-finding algorithm in the s domain. Finally, convert the roots back into the original z domain. The method has shown excellent results in practice. Details are described in a recent paper on the subject of root tracking [14]. In fact, clever root-tracking techniques are the key to fast sequential frequency estimation algorithms. D. Batch Algorithm Summa0 In Table I, we summarize the computations which are necessary to extract the normalized frequencies {wh,1 5 IC 5 T } from a given data covariance matrix Pi, using the described batch variant of the predictor-annihilator algorithm.

E. A Batch Experiment The operation of the predictor-annihilator based frequency estimator of Table I is demonstrated in the following experiment. Two sinusoids with normalized frequencies w1 = 70.0" and w2 = 71.0" are buried in white noise. Fig. 3 shows 4000 samples of the time series in this experiment. Fig. 3(a) is the

solve

T ,

T,,, a(=)= a'c)= e

e :

, T

V y a n d optimal prediction f i l t e r w:

= v:'V;Tx,

(17)

according t o (21).

f o r a(=) = [a:, a;,

eTe = min

. . . , a:,

1 , a,:

. ..,

ai, a?lT

$A

+

:

(25) (26)

Bi-linear transformation aZ

, , ,

[a;,

(r-domain representation)

az '

'I

. . . ar

(s-domain representation)

Create s-domain companion matrix.

(27)

C o m p u t e real Schur decomposition of

C

and eigenvalue revealing

Hessenberg matrix H:

,

1

i ,

H = UTC U

(29)

Extract Schur b l o c k s f r o m main tri-diagonal of H and determine t h e associated complex conjugate eigenvalue pairs in the s-domain. Apply inverse bi-linear transformation to obtain associated eigenvalue pairs in the z-domain {uk. 1

I

k

I

Determine t h e desired normalized frequencies

r} as t h e angles of t h e eigenvalue pairs in t h e z-domain.

signal s ( t ) consisting of the sum of the two sinusoids. Fig. 3(b) shows the artificial data sequence generated as the sum of s ( t ) and the noise process. In this experiment, the logarithmic signal-to-noise ratio (SNR) defined as 10 log,, (p,) is -7.1 dB. The time series shown in Fig. 3(b) thus constitutes the raw data x ( t ) used in this experiment. The number of snapshots was L = 3500 and the model order was set to N = 501. Thus, the overmodeling factor is N / n = 250 (!). Fig. 3(c) shows the reconstructed signal 6 ( t ) using a prediction filter w of order N = 501 (recall Fig. 1). The prediction filter output i ( t ) shown in Fig. 3(c) is an almost perfectly clean reconstruction of s ( t ) . This is demonstrated in Fig. 3(d) which shows the reconstruction error s ( t ) - 6 ( t ) . An overall quantificationof the reconstruction quality is provided in terms of an overall relative reconstruction error defined as ER = (E( s( t ) - s ( t))') '1' / (E( s( t )) ") 1/2 where the summation comprises all 4000 samples of this experiment. The resulting relative reconstruction error is ER = 0.26. This reconstruction error, however, is mainly caused by amplitude errors, and not by frequency errors. This becomes apparent in the next step of the predictor-annihilator based frequency estimator, where the coefficients of the order T = 4 annihilator filter E,(z) are computed as the least squares solution of the overdetermined system (25), (26) constructed using the elements of w. Fig. 3(e) shows the output ~ ( tof) the annihilator filter. Indeed, it is seen that the reconstructed signal a(t) is annihilated almost perfectly, as expected. The attenuation achieved by

STROBACH: FAST RECURSIVE LOW-RANK LINEAR PREDICTION FREQUENCY ESTIMATION ALGORITHMS

839

Discrete Time (Samples)

Fig. 3. (a) Linear superposition of two sinusoids with fixed frequencies (v'l = 70.0' and w a = 71.0'; (b) white Gaussian noise added to the sinusoids of (a). SNR is -7.1 dB; (c) reconstruction of the sinusoids nsing a prediction filter of order N = 501; (d) reconstruction error; (e) annihilation residual. 4000 samplcs displayed.

the annihilator filter in this experiment is 37.7 d13. Thus, predictor-annihilator concept as described and summarized in the system is remarkably well matched on the signad, and it Table I of the previous section. These sequential algorithms should be possible to compute the two frequencies with high comprise the following three fundamental computation steps. accuracy as the roots of the small annihilator filter pollynomial 1) Fast recursive tracking of a time-varying dominant data E,(%).For this purpose, transform E,(%) into the s domain covariance eigenvector matrix ~ $ (1t ) . using the bilinear transformation, form the companion matrix 2) A computation of the parameters of the frequency reof dimension 4 x 4, determine the complex conjugate root vealing time-varying annihilator polynornial Ea(z, t ) as pairs in the s domain, and finally convert the roots back basically described in Section 111. into the z domain using the inverse bilinear transformation. 3 ) A tracking of the n complex conjugate root pairs of Determine the normalized frequencies as the angles of the root Ea(x, t ) . The angles of these root pairs are the desired pairs in the x domain. The following normalized frequency normalized frequencies. Algorithms for root tracking of estimates were obtained in this experiment: LQ = 70.062' time-varying polynomials are described in [141. and G2 = 71.052'. Thus, we conclude that both the natIn this section, we primarily discuss step 1).The results are ural modes in 20 as well as the reconstructed signal i ( t ) finally compiled into algorithm summaries. represent the desired frequencies with a very high accuracy, thanks to the natural signallnoise separation capability of A. Subspace Tracking Using Sequential Orthogonal Iteration large eigendecompositions and extreme overmodeling factors. We first discuss the fast recursive tracking of the domiAnother important insight is that the amplitude errors in nant data covariance eigenvector matrix V g ) ( t ) A . class of the reconstructed signal B ( t ) do not affect the estimated frequencies or root angles Lj. They do affect, however, the subspace trackers based on sequential orthogonal iteration is radius or magnitude of the estimated root pairs. Thiis is the described. These subspace trackers have first been seriously deeper reason why we should not constrain the coefficient studied in the context of adaptive filtering [ I l l , [12]. Consider the following sequential data covariance estimator: vector in of the annihilator filter in (22) to a strictly symmetric shape. The roots need the freedom of a slight radial deviation !PZ(t)== a@,(t - 1) (1 - a ) z ( t ) z T ( t ) (31) from the unit circle to accommodate amplitude errors in highaccuracy angle (frequency) estimation. where 0 5 a 5 1 is a positive exponential forgetting factor close to 1 and z ( t ) is a real data vector of dimension N as IV. FASTSEQUENTIAL FREQUENCY TRACKING ALGORITHMS defined in (2). Introduce a strictly orthonormal N x T recursion matrix In this section, we describe three algorithmic realizations of a sequential frequency estimation algorithm based on the Q ( t ) . Update this matrix recursively in time using the fol-

+

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 44, NO. 4, APRIL 1996

840

I

I

Discrete Time (Samples) ‘

Fig. 4. Jump scenario raw data generator: (a) First sinusoidal source signal: (b) second sinusoidal source signal; (c) white Gaussian noise process; (d) raw data for experiment = source 1 source 2 noise. SNR is 0.9 dB.

+

+

the dominant eigenvalue matrix AF)(t - 1). Hence, a matrix $z(t - 1) defined as

lowing simultaneous orthogonal iteration [1 11, [13]:

A@)= @Z(t)Q(t- 1)

(324

&T(t- 1) = Q(t -

1)R(t- l)QT(t- 2 )

(35)

will converge against the true rank T approximant PS, (t- 1) = V ~ ) ( t - l ) A ~ ) ( t - l ) V(t-1) ~ ) T of the data covariance matrix When updated in time using simultaneous orthogonal iteration Q Z (t 1).Recall the discussion in Section 11, and note that a according to (32a), (32b), the recursion matrix Q(t) will minimum variance rank T approximant can be interpreted as almost always converge rapidly against the true dominant an enhanced representation of the complete signal information eigenvector matrix V$)( t ) .See [ 111 for details. that is available in a data covariance matrix !PZ(t- 1). As long as the noise is white and forms a spherical noise subspace, only B. A First Sequential Algorithm this signal information, which is almost perfectly concentrated The recursions (32a), (32b) deduced from classical orthog- in 4, (t - l),is finally required in the updating process of the onal iteration are seldom used for sequential processing in estimated eigenvector matrix Q(t). Thus, we can replace the practice because they require a tremendous amount of O (N’T) covariance matrix PSZ(t- 1) in (33) by its estimated low-rank t 1) without any degrading effect of the operations each time step. Therefore, we introduce a fast approximant @ z ( sequential variant of orthogonal iteration based on a low-rank recursion. This yields the following modified recursion: approximation of the data covariance matrix. For this purpose, A ( t ) = a&,(t - l)Q(t - 1) (1 - a)x(t)hT(t) substitute the covariance time update (31) into the “mapping = aQ(t l ) R ( t- l ) Q T ( t- 2 ) Q ( t I) equation-- (32a) of orthogonal iteration to obtain (1 - a ) x ( t ) h T ( t ) A(t) = [aQZ(t - 1) (1 - a ) z ( t ) x T ( t ) ] Q-( t1) = aA(t - l ) e ( t - 1) (1 - a ) x ( t ) h T ( t ) (36) = a@,(t - l)Q(t - 1) (1 - a)z(t)hT(t) (33) A(t) = Q(t)R(t): “skinny” QR factorization.

(32b)

+

~

+

+

~

+

+

where

where

h(t)= QT(t- l)x(t)

e(t)= Q’(t (34)

is a “compacted” data vector. Note that Q(t - 1) converges against the dominant eigenvector matrix V F ) ( t- 1) and R(t - 1) converges against

-

l)Q(t)

(37)

is a matrix of cosines of angles that characterize the “distance” between Q ( t ) and Q ( t - 1). Surprisingly enough, we have here a fairly practical, fast, and powerful direct O ( N r 2 )timeupdating scheme for the auxiliary matrix A ( t ) of orthogonal

84 I

STROBACH: FAST RECURSIVE LOW-RANK LINEAR PREDICTION FREQUENCY ESTlMATlON ALGORITHMS

TABLE I1 FASTRECURSIVE FREQUENCY ESTIMATION ALGORITHM 1. EQUATIONS NUMBERED AS THEY APPEARIN TEXT.ROOT-TRACKING OF A TIME-VARYING POLYNOMIAL OF DEGREE T = 2n APPEARS AS A SUBPROBLEM THAT IS DlSCtiSSED IN [14]

PI

Initialize: Q(t-1) = --

Wt-1) = Ir ;

i

0

d

a s 1 ;

r

FOR EACH TIME STEP DO:

Input: x(t) Subspace ~.

-p

tracking section:

h(t) = QT(t-l) x(t) A(t)

s

d ( t - l ) e ( t - 1 ) + (l-a)x(t)hT(t)

A(t) = Q(t)R(t)

"skinny" Q R factorization

:

e ( t ) = QT(t-l) Q(t) Compute prediction filter w t t ) and averaged filter wA(& w ( t ) E [w,(t), w,(t),

. . . , w,(t)IT

w,(t)

Recursive averaging

= A(w(t))

.~ Construct

T,,,,(t)

:

= Q(t)QT(t)nc

according t o (21) usingwA(t).

Solve least squares problem for desired annihilator parameters:

T,,,,(t)a(C)(t)= e ( t )

c1,

-a,(t), -a,(t),

:

eT(t)e(t)= min

+

. . . , - a r ( t ) l T = (af,=(t))

Lz(t)

-1

Lz(t)

Track the roots of the annihilator filter polynomial $,&E Ea(z,t) = 1

- a,(t)z-' -

az(t)z-' -

, ,

. - ar(t)z-'

,

Frequency Computation Section: Extract instantaneous normalized frequencies{ok(t), 1 s k angles of complex conjugate root-pairs.

i

d as

iteration. The computationally burdensome time updating of the estimated covariance matrix according to (3 1) is replaced by a direct time-updating scheme for the auxiliary N 2: r matrix

4t). In the sequential case, a high-order predictor parameter vector w ( t ) is computed according to (17) where we only ) the replace the true dominant eigenvector matrix V g ) ( f by estimated dominant eigenvector matrix &(t ). Note, Ihowever, that we do not use w ( t ) directly to construct a data matrix T,,(t) according to (21) at each time step. In the sequential case, we first apply recursive averaging on the parameter vector sequence. Thus, at each time step, a data matrix T w( t )~is constructed from the recursively averaged parameter sequence w ~ ( t = ) A ( w ( t ) )where the operator A(.) denotes recursive sliding or growing window averaging. Table I1 summarizes a first variant of a fast recursive low-rank linear prediction frequency estimation algorithm called Algorithm 1.

Discrete Time (Samples)

Fig. 5. Jump scenario frequency estimates. Dashed line means ideal frequency trajectory. Solid line means estimated frequency trajectory: (a) Tracking result for hatch reference algorithm; (h) tracking result for Algorithm 1; (c) tracking result for Algorithm 2. Algorithm parameters : T = 4, N = 91, Q = 0.98. Ten independent runs plotted into one diagram.

Surprisingly, we can show that explicit QR factorization is, in fact, completely unnecessary. We develop a time-recursive scheme to update the Q and R factors of a QR decomposition directly in item. Note that this scheme for direct Q R factor C. A Second Sequential Algorithm tracking does not require any further simplifications. Nevertheless, there are two good reasons to introduce the main iideas Note that the subspace tracking section in Table I1 is of direct QR factor tracking using a simplified time-updating characterized by the heavy use of a compressed (dimension r ) data vector h(t).The initial data compression step h(t) = recursion for A ( t ) .First, a simplified time update for A ( t )will QT(t - l)z(t)is, in fact, the key to fast recursive !subspace result in an extremely fast high-performance O(N r ) subspace tracking. Note that this compression does not cause any loss in tracker. Second, we can easily extend the simplified recursions performance because the data are, in fact, compressible since to obtain a fast direct QR update subspace tracker whlch is we work with overmodeling. The recursion matrix &(t) always perfectly equivalent to the explicit QR factor subspace tracker tends to approximate an optimal data compressor for ~ ( t ) . of Table 11. Begin with a simplification and set e(t- 1) = I , . This A large number of operations in the subspace tracker in Table 11, however, must be expended in the compuitation of implies that the data subspace does not change in time anymore Q ( t ) via explicit QR factorization of A ( t ) in each time step. or that the algorithm has converged. In fact, it tums put

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 44, NO. 4, APRIL 1996

842

I

Discrete Time (Samples)

'

Fig. 6. Intersection scenario raw data generator: (a) First sinusoidal source signal: (b) second sinusoidal source signal; (c) white Gaussian noise process; (d) raw data for experiment = source 1 source 2 noise. SNR is 0.9 dB.

+

+

in practice that this is not a ,very limiting assumption in many cases (see the experiments provided in Section VI). But the update equation for the auxiliary matrix A ( t ) simplifies considerably as A ( t ) = aA(t - 1)

+ (I - a)z(t)hT(t).

(38)

Note that now the QR factors of A ( t ) depend on the QR factors of A(t - 1) plus a rank 1 layer as follows: Q(t)R(t)= a&(t - l ) R ( t- 1) (1 - a ) z ( t ) h T ( t ) . We shall show that this updating problem has a fast solution. Only 37- - 3 Givens row rotations are finally required to compute the actual QR factors from their predecessors. The auxiliary matrix A ( t ) will not be formed explicitly anymore. Introduce a projection operator PQ(t- 1) that projects vectors orthogonally onto the subspace spanned by the columns of Q ( t - 1) and its orthogonal complement P&(t- 1) as follows:

+

P Q (-~ 1) = Q ( t - l)QT(t - I)

P&(t- 1) = I N

-

PQ(t- 1).

(394

where X ( t ) = z y ( t ) z l ( t ) It . becomes apparent that we may use (41) to decompose the actual data vector z ( t ) into a component that can be represented in the "old" subspace spanned by the columns of Q ( t - 1) and an orthogonal (decoupled) innovation as follows:

~ ( t=)X 1 / 2 ( t ) Z l ( t+ ) Q ( t - 1)h(t).

(42)

Thus, (38) together with (42) yields the following decomposition:

Q(t)R(t) = aQ(t - l)R(t- 1) + (1 - OI)X1/2(t)Zi(t)hT(t)

+ (1

-

a ) Q ( t- l ) h ( t ) h T ( t ) .

(43)

The key trick is that this decomposition can be expressed equivalently as the product of two augmented matrices as follows:

(39b)

The complement z ~ ( tof) the orthogonal projection of z ( t ) onto Q(t - 1) is then expressed as follows:

z l ( t )= P&(t- l)z(t)= ~ ( t-)Q(t - l ) h ( t ) . Normalize

(40)

21 (t) :

ZL(t) = X-l/2(t)z&)

= X - ' / ' ( t ) z ( t ) - X - 1 / 2 ( t ) Q ( t- l ) h ( t ) (41)

Define an orthonormal multiple plane rotation matrix G(t). Insert the rotor product GT(t)G(t)= IT+l between the augmented right-hand side matrices in (44) and split the expression into the following two recursions:

[Z]= w [

(I - * ) X ' / " t ) h T ( t )

]

aR(t - 1)+ (1 - a)h(t)hT(t)

(454

STROBACH: FAST RECURSIVE LOW-RANK LINEAR PREDICTION FREQUENCY ESTIMATION ALGORITHMS

843

TABLE I11 follows: FASTRECURSIVE FREQUENCY ESTIMATION ALGORITHM 2. EQUATIONS NUMBERED AS THEYAPPEAR IN TEXT. ROOT-TRACKING OF A TIME-VARYING I’OLYNOMIAL Q(t)R(t) = aQ(t - l ) L ( t ) (1 - a).;(t)hT(t) (461) OF DEGREE 7‘ = 27l APPEARS AS A SUBPROBLEM THAT IS DISCUSSED IN [14] where L ( t ) = R ( t - l)e(t- 1). Restore the QR structure

+

r1

Initialize: Q(t-1) = -~ ; R(t-1) = 0 ;

0

i U i1 ;

on the right side of (46) via rotation using an intermediate sequence of Givens plane rotations G L ( ~as) follows:

r

FOR EACH TIME STEP DO:

R’(t) = G L ( ~ ) L ( ~ )

Input x ( t )

~

(474

Subspace tracking section:

Q’(t) = Q(t- 1)GE(t)

h(t) = d ( t - 1 ) x(t) x,(t) = x(t) - Q(t-l)h(t)

(47b)

where R’(t) and Q’(t) are intermediate QR factors. Using these intermediate QR factors, the exact time update recursion for A ( t ) according to (36) can be expressed in the form of the following basic decomposition:

X(t) = X?(t)X,(t) %,(t)= x-l’z(t) x , ( t )

Q(t)R(t)= aQ’(t)R’(t)+ ( 1 - a).;(t)hT(t).

(48)

It turns out that the Q R time update (48) can now be factorized into the following direct time updating recursions: Compute prediction filter w ( t ) and averaged filter wA(& w ( t ) = [w,(t), w 2 ( t ) , . w,(t)

= A(w(t1)

:

, ,

, w,(t)IT = Q(t)QT(t)xc

Recursive averaging

Construct TwA(t) according to (21) u*wA(t). Solve .

least squares problem for desired annihilator parameters:

T w A ( t ) a ( c ) ( t ) =e(t)

:

eT(t)e(t)= min

3

e(t)

[ I , - a , ( t ) , - a , ( t ) , . . . , - a r ( t ) l T = (e.f,Ls(t))-l k c i ( t ) ~Track the roots of the annihilator filter polynomial

E,(z.t) = 1

-

a,(t)z-’ - a2(t)z-’ -

(49b) Clearly, (49a) is the exact version of the triangular matrix. update, and is therefore the counterpart to irecursion (45a).. Equation (49b) in turn is the exact version of the orthogonal1 matrix update, and is thus the counterpart to (45b). Note further that

&(,E

. . . - a.,.(t)z-’

Frequency Computation Section: Extract instantaneous normalized frequencies{ok(t), 1 angles of complex conjugate root-pairs.

L

k

i

n) as

Consider recursion (45a). The Givens plane rotations in G ( t ) Therefore, we must have must be detrmined so that the augmented and updated matrix R(t - 1) in (45a) is transformed into a strictly upper-right triangular matrix R(t) by rotation. Investigate the principal nature of this problem. Although expression aR(t - 1) (1a)h(t)hT(d) constitutes a “full” matrix, the update hr(i)hT(t) has only rank 1, and therefore the lower submatrix in (45a) can be reduced to an upper Hessenberg matrix in only r - 2 row rotations. This constitutes a first step in the transformation (45a). In a second step, the upper Hessenberg form is transformed into an upper triangular matrix using r - 1 Hessenberg Q R steps. A sequence of r Hessenberg QR steps finally produces R(t).See [ l l ] for a detailed explaination of these transformation steps. Table I11 summarizes a fast O ( N r ) frequency tracker called Algorithm 2 based on these simplified recursions for direct Q R factor tracking. This is just another representation of the orthogonal matrix update (49b). Apply the rules of partitioned matrix multipliD. A Third Sequential Algorithm cation to demonstrate that from (52), the following recursion We continue establishing a QR factor updating a.lgorithm for Q ( t ) can be derived: for the exact time recursion (36). Note that this algorithm is complicated by a nonidentity We may express (36) as (53) Q(t)= Q(t - l)e(t) +Zl(t)ZT(t)$(t).

+

lEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 44, NO. 4, APRIL 1996

TABLE IV FASTRECURSIVE FREQUENCY ESTIMATION ALGORITHM 3. EQUATIONS NUMBERED AS THEYAPPEAR I\ TEXT ROOTTRACKING OF A TIME-VARYING POLYNOMIAL OF DEGREEr = 2n APPEARS AS A SUBPROBLEM THAT IS DISCUSSED IN [14]

! i

Initialize:

Qit-1) =

[a]

;

Rit-I)

=0

;

Wt-1) = J-

;

0

S

a

S

1 ; r

FOR EACH TIME STEP D O .

Input:

X(t)

Subspace tracking section:

' O I

I

h ( t ) = QTit-l) x ( t ) x - i t ) = xit) - Q(t-l)h(t) X(t1 = x:(t)

X,itl

ji-it) = X-'/?t)

x,(t)

Lit1 = R(t-1)eit-1) R'itl = G,(t)L(t)

(55)

Q i t ) = Q(t-1) 0it) + ji,(t) f T ( t ) Compute prediction f i l t e r w ( t ) and averaged f i l t e r wA(t). w i t ) = [ w , ( t ) , w,it), w,(t)

= A(w(t))

C o n s t r u c t Tw,(t)

. . . , w,(t)IT

(17)

= Q(t)QT(t)xc

Recursive averaging

:

according t o (21) usingwA($

Solve l e a s t s q u a r e s problem for desired annihilator parameters:

Tw,(t)e.(C)(tl = e i t )

:

[I, -a,it), -a,(t), .

. .

e T ( t ) e ( t ) = min

(25)

:;(t)

Track t h e r o o t s of t h e annihilator f i l t e r polynomial Ea(& E,(z.t) = 1 - a,it)z-' - az(t)z-* -

. . . - ar(t)z-'

Frequency Computatlon Section: E x t r a c t i n s t a n t a n e o u s normalized frequencies{i$(t), angles of complex conjugate root-pairs.

Discrete Time (Samples)

Fig. 7. Intersection scenario frequency estimates. Dashed line means ideal frequency trajectory. Solid line means estimated frequency trajectory: (a) Tracking result for hatch reference algorithm; (b) tracking result for Algorithm 1; (c) tracking result for Algorithm 2. Algorithm parameters: r = 4,LY= 161. cy = 0.995. Ten runs in one diagram.

Clearly, the term zT(t)Q(t)must not be computed explicitly. This vector is directly available from the partioning scheme shown in (51). Thus, define an intermediate vector f ( t ) as

f(t)= Q T ( t ) z i ( t )

(54)

and compute (53) as follows:

Q(t)= Q(t - l)@(t)+ zi(t)fT(t).

(55)

Finally, investigate the principal nature of this recursion. Clearly, 8 ( t ) = Q'(t - l)Q(t) and P g ( t - 1) = Q(t l)Q'(t - 1); therefore Q(t - l)e(t) = P Q (-~l)Q(t).

(56)

This means that Q(t- l)e(t) constitutes the "old" information that can be represented (via orthogonal projection) in the old

15 k

S

n) a s

subspace spanned by the columns of Q ( t - I), and the outer product s,(t)fT(t) can be interpreted as an orthogonal rank 1 innovations matrix A ( t ) ,as follows:

A ( t )= % l ( t ) f T ( t ) .

(57)

Table IV summarizes a fast frequency estimation algorithm using sequential orthogonal iteration based on direct QR factor tracking. This method was called Algorithm 3.

v.

FREQUENCY TRACKING IN CORRELATED

NOISE

Until now, the discussion was based on the limiting assumption that the noise in the data is white (5). This assumption is frequently violated in practice. Therefore, we will next derive the necessary relationships for low-rank linear prediction frequency estimation of signals in stationary Gaussian noise with known covariance matrix @.I Define a whitening matrix @;'I2 as the inverse symmetric matrix square root of @.I. Then the following transformation whitens the noise in ~ ( t ) :

STROBACH: FAST RECURSIVE LOW-RANK LINEAR PREDICTION FIREQUENCY ESTIMATlON ALGORITHMS

1

923 257 3% 513 E41 7 9 897 ~

~

1

1

~

~

~

{

~

~

1

~

3

845

7 2 618 9 ~~i ; ; ~~3 0 7 ~ 33a1 I1 3 3~2 9 3i3 5 7~3 5 825 3 7 1 3 3 474 1 37

~

2

~

~

- >

Discrete Time (Samples)

Fig. 8. Bottleneck scenario raw data generator: (a) First sinusodal source signal; (b) second sinusoidal source signal; (c) white Gaussian noise process; (d) raw data for experiment = source 1 source 2 noise. SNR is 1.0 dB.

+

+

Hence, the vector x, ( t ) represents a transformed signal in white noise. We may extract this transformed signal vector via projection of x , ( t ) onto the dominant eigensubspace of the whitened data covariance matrix Pi,, = E(x,(t)x:(t)). Note that the subspace trackers introduced in Section IV can be used for estimation of the dominant eigensubspace matrix of CP,,. When operated on the whitened data sequence, these subspace trackers produce estimates Q, ( t ) of the dominant eigensubspace matrix. Hence, the following operation yields a reconstruction i,( t )of the transformed signal in x ( t ) :

the noise covariance matrix is known and the input data are whitened appropriately prior to processing.

VI. EXPERIMENTAL RESULTS

The computer experiments in this section are organized as follows. We show tracking results from three different scenarios. In each scenario, the tracking characteristics are shown for two independently time-varying sinusoids in white noise. Each experiment comprises 4000 samples. Tracking results are shown for Algorithm 1 and Algorithm 2. Algoi T i ( t ) = ~ , ( t ) ~ ; ( t ) @ ; ~ / ~ x ( t ) . (59) rithm 3 performs absolutely identically to Algorithm l , and therefore it is unnecessary to display these results explicitly. This is not yet the desired result. In a final step, the whitening As a reference, we computed frequency estimates using the transformation of the signal vector must be reversed via exact batch algorithm of Table I (comprising exact minimum an inverse transformation i ( t ) = @;I2i,(t). Thus, signal variance low-rank approximation using true EVD and exact reconstruction from observations in correlated noise requires eigensubspace computation) in each time step. As a surprising the application of a generalized projection of the type result, we shall see from these experiments that after an initial q t ) = @l,/2i,(t)= @ ~ / 2 ~ & , , ( t ) ~ ~ ( t ) ~ ;(60) 1 j 2 xstart-up ( t ) . phase, our fast frequency trackers perform very close to the exact and computationally much more demanding batch Therefore, we can define the high-order prediction filter wTI( t ) scheme. The experiments also indicate that the exponential forgetting factor Q (or the effective window length) of the as follows: covariance estimator and the order N of the algorithms mainly w T - T 1/2 T (61) determine the tracking delay. - rc4, Q, (t)4L1’2+

,

Finally, note that in the serialized data case, it is usually unnecessary to work with the inverse symmetric matrix squareroot whitening operator. Whitening can be carried out equivalently using a time-invariant linear prediction whitening filter [l]. This means that all the sequential frequency estimation algorithms described in Section IV can be used for frequency estimation of signals in correlated noise if only an eslimate of

A. Jump Scenario

In the jump scenario, we employ two sources whose normalized frequencies are constant in intervals of duration 800 samples. The frequencies change abruptly at the segment boundaries. The following normalized frequencies have been used:

IEEE TRANSACTlONS ON SIGNAL PROCESSING, VOL. 44, NO. 4, APRIL 1996

846

Segment I : w1 = 85"; Segment 2: w1 = 39"; Segment 3: w1 = 75"; Segment 4: w1 = 65"; Segment 5: w1 = 50";

w2

= 10" = 35"

w2

= 25"

w2

= 61" = 15"

w2

w2

Fig. 4 shows the corresponding processes. Fig. 4(a) is the output of sinusoidal source 1 with frequency w1.Fig. 4(b) is the output of sinusoidal source 2 with frequency w2. Fig. 4(c) shows one realization of the noise process. Fig. 4(d) shows the artificial raw data process created as the sum of the two sinusoids plus the noise process. Each experiment comprises ten independent runs with different noise realizations. The logarithmic signal-to-noise ratio 10 loglo ( p z ) in each run is 0.9 dB. In this experiment, the algorithms have been operated using the following parameter configuration: T = 4, N = 91, Q = 0.98. We use sliding window averaging of the highorder prediction filter coefficient vectors with an averaging window length of 80. This averaging is carried out recursively. Thus, only a negligible amount of 2N additions is required to perform coefficient averaging in each time step. Fig. 5(a) shows the frequency tracks obtained from the exact batch algorithm of Table I as a reference. The corresponding frequency tracks for Algorithm 1 and for Algorithm 2 are shown in Fig. 5(b) and (c), respectively. Estimated frequency trajectories are displayed as solid curves. True frequency trajectories are displayed as dashed curves. The tracking results of the ten independent runs are plotted into one diagram for comparison. We see that the tracking behavior is characterized by a certain delay until the algorithm reacts on a frequency jump. This delay is mainly determined by the effective memory length of the algorithm, which is controlled by the exponential forgetting factor Q and the number of recursively averaged high-order prediction filter coefficient vectors. Of course, the tuning of these parameters is governed by a certain tradeoff between estimation accuracy in segments of constant frequency and an acceptable delay. Moreover, we see that after convergence, the algorithms produce almost perfectly unbiased estimates, as expected.

Discrete Time (Samples)

>

Fig. 9. Bottleneck scenario frequency estimates. Dashed line means ideal frequency trajectory. Solid line means estimated frequency trajectory: (a) Tracking result for hatch reference algorithm; (b) tracking result for Algorithm 1: (c) tracking result for Algorithm 2. Algorithm parameters: T = 4, N = 261, n = 0.993. Ten funs in one diagram.

B. Intersection Scenario

The intersection scenario is constituted by two sinusoidal sources whose frequencies are changing smoothly with time. Source 1 is operated with a decreasing frequency, while source 2 is operated with increasing frequency. Fig. 6 shows the corresponding processes. Again, the logarithmic SNR is 0.9 dB. It is interesting to study the behavior of the algorithms in the area where the frequency trajectories intersect. The algorithms have been operated using the following parameter configuration: r = 4, N = 161, Q = 0.995. 80 subsequent prediction filter coefficient vectors have been averaged. Fig. 7 shows the resulting estimated frequency trajectories. Again, estimated frequency trajectories are displayed as solid curves, whereas the true frequency trajectories are displayed as dashed curves for comparison. Estimation results of ten independent runs are plotted into one diagram. The behavior of the algorithms around the intersection point

is characterized by the property that estimated frequency trajectories cannot intersect. The trajectories come close to each other and "reflect" at the intersection point. The deeper theoretical explanation for this effect is long, and appears in ~41.

C. Bottleneck Scenario

In this experiment, we discuss the behavior of the algorithms in situations where the frequencies of two sources come very close, but do not intersect. The minimum gap between the frequencies of the two sources in this experiment is only 1". Fig. 8 shows the processes used to generate the raw data for this experiment. The logarithmic SNR is 1.0 dB. The bottleneck scenario is much more difficult than expected. The algorithms suffer from a considerable "proximity effect" when root pairs come very close to each other.

STROBACH: FAST RECURSIVE LOW-RANK LINEAR PREDICTION FREQUENCY ESTIMATION ALGORITHMS

Therefore, a reliable identification of very closely spaced sources makes it necessary to increase the prediction filter order N . In this experiment, the algorithms have been operated using the following parameter configuration: T = 4, N = 261, a = 0.995. 80 subsequent prediction filter coefficient vectors have been averaged. Fig. 9 shows the resulting estimated frequency trajectories. Again, ten independent realizatiom have been plotted into one diagram, and true frequency trajlectories are provided as dashed lines for comparison. VII. CONCLUSIONS

Three sequential low-rank linear prediction algorithms for frequency tracking have been introduced. The computational complexity of these algorithms depends only linearly on the model order N . This reduction in complexity became possible using the predictor-annihilator concept for fast root tracking and a new class of fast subspace trackers. Excellent experimental results have been obtained using the new algorithms. Throughout all experiments, we realized the experience that the overmodeling factor N / n should be several 10-100 times greater than 1 for robust tracking of sinusoidal sources in heavy noise.

ACKNOWLEDGMENT The author wishes to thank the reviewers (particularly reviewer A) for their useful comments and suggestions on an early version of this paper. REFERENCES [I] P. Strohach, Linear Prediction Theory: A Mathematical Basis f o r Adaptive Systems, Springer Ser. in Inform. Sci., Vol. 21. Heidelberg, New York: Springer-Verlag, 1990. [2] D. W. Tufts and R. Kumaresan, “Frequency estimation of multiple sinusoids: Making linear prediction perform like maximum likelihood,” Proc. IEEE, vol. 70, pp. 975-989, Sept. 1982. [3] J. A. Cadzow, “Spectral estimation: An overdetermined rational model equation approach,” Proc. IEEE, vol. 70, pp. 907-939, Sept. 1982. [4] R. Kumaresan and D. W. Tufts, “Estimating the angle of arrival of multiple plane waves,” IEEE Trans. Aerosp. Electron. Sysf., vol. AES-19, Jan. 1983. [SI R. Knmaresan, D. W. Tufts, and L. L. Scharf, “A Prony method for noisy data: Choosing the signal components and selecting the order in exponential signal models,” Proc. IEEE, vol. 72, pp. 230--233, Feb. 1984.

847

[6] L. L. Scharf and D. W. Tufts, “Rank reduction for signal modelling and parameter estimation,” in Proc. 19th Annual Asilomar Con$ Circuits, Sysr. Comput., Pacific Grove, Monterey, CA, Nov. 1985, pp. 467471. “Rank reduction for modeling stationary signals,” IEEE Trans. [7] -, Acoust., Speech, Signal Processing, vol. ASSP-35, pp. 350-355, Mar. 1987. [SI M. A. Rahman and K. B. Yu, “Total least squares approach for frequency estimation using linear prediction,” ZEEE Trans. Acoust., Speech, Signal Processing, vol. ASSP-35, pp. 1440-1454, Oct. 1987. [9] L. L. Scharf, Statistical Signal Processing: Detection, Estimation, and Time Series Analysis. Reading, MA: Addison-Wesley, 1990. [lo] B. D. Rao and K. S. Arun, “Model based processing of signals: A state space approach,” Proc. IEEE, vol. 80, pp. 283-309, Feb. 1992. [ 111 P. Strobach, “Low rank adaptive filters,” IEEE Trans. Signal Processing, submitted for publication, Aug. 1994. [ 121 -, “Fast recursive eigensubspace adaptive filters,” in Proc. ICASSP’YS, Detroit, MI, May 1995. [13] G. H. Golub and C. F. VanLoan, Matrix Computations, 2nd ed. Baltimore, MD: Johns Hopkins Univ. Press, 1989. [14] P. Strobach, “The recursive companion matrix root tracker,” IEEE Trans. Signal Processing, submitted for publication. [15] J. Heinhold, Einfuhrung in die Hiihere Mathematik, Teil IV: Funktionentheorie. Munich, Vienna: Carl Hanser, 1980.

Peter Strobach (M’86-SM’91) was born in Passau, Germany, on February 6, 1955. He received the engineer’s degree in electrical engineering from Fachhochschule Regensburg in 1978, the DiplomIngenieur (M S.) degree from Technical University Munich, Germany in 1983, and the Dr -1ng degree from Bundeswehr University, Munich, Germany, in 1985. From 1976 to 1977, he was with CERN Nuclear Research, Geneva, Switzerland From 1978 to 1982, he was a systems engineer at MesserschmittBoelkow-Blohm GmbH, Munich, Germany. From May 1986 to December 1992, he was with Sieineiis AG, Zentralabteilung Forschnng und Entwicklnng (ZFE), Munich His last position at Siemens was head of the Signal and Image Processing Laboratory In January 1993, he joined the Faculty of Fachhochschule Furtwangen (Black Forest), Germany His current research interest is the theory of recursive algorithms He has industrial experience in high-energy physics data processing, high-speed digital hardware, aircraft iadar and guided missile systems, image processing, communication and coding systems, and noninvasive biomedical investigation systems. He IS the sole author of over 50 reviewed papers, and has authored the book Linear Prediction Theory. A Mathematical Basis for Adaptive Systems (Springer Senes in Information Sciences, Vol 21, 1990) He also wrote a 600-page unpublished memorandum entitled “Adaptive Filter uud Parameterschatzung” (in Geman) In the spnng of 1990, he was a Guest Lecturer at the Technical University of Tallin, Estonia, and in the summer of 1990, he held the first adaptive filter course ever held in Germany at the University of Erlangen-Nuernberg, Germany Dr. Strobach is a member of the IEEE Signal Processing Society, an Editonal Board member of Signal Processing, and received a 1988 National Paper Prize Award