Blind Input, Initial State, and System Identification of ... - IEEE Xplore

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 52, NO. 12, DECEMBER 2004

3357

Blind Input, Initial State, and System Identification of SIMO Laguerre Systems Jacob H. Gunther and Roberto López-Valcarce, Member, IEEE

Abstract—Laguerre filters have infinite impulse responses (IIRs) but with finite tapped delay-line parameterizations. This paper investigates subspace-based blind identification of Laguerre filter tap coefficients, the internal filter state, and the input, given only noisy observations of the output. This paper deals only with single-input, multiple-output (SIMO) Laguerre models. A state space model for the SIMO Laguerre system is derived from which blind estimation algorithms are developed. As in the finite impulse response (FIR) case, the Laguerre filter taps coefficients can be estimated from the column space of a certain Hankel matrix constructed from noisy output observations, whereas the internal state and input can be estimated from the row space by exploiting state space structure. While not exactly uniquely identifiable, conditions are given for which the tap coefficients, the internal state, and the input can be determined to within a multiplicative scalar factor. Index Terms—Blind identification, blind input estimation, Laguerre systems.

I. INTRODUCTION

F

INITE impulse response (FIR) systems have received much attention for applications involving modeling of unknown systems or their inverses [1]–[3]. The input–output relationship of FIR systems is linear in the parameters, which leads to efficient algorithms for parameter estimation. Additionally, the manner in which the input data shifts sequentially through the memory elements of discrete-time FIR systems gives additional structure that can also be exploited for estimation. In applications where the impulse response decays very slowly, FIR approximations require a large number of parameters. In such situations, infinite impulse response (IIR) systems may be able to approximate slowly decaying impulse responses with only a few parameters. However, parameters in the feedback path of IIR models make estimation more complex than in FIR models. Furthermore, issues relating to the stability of IIR models must be factored into the estimation [4]. Laguerre filters offer a compromise between FIR and IIR models and inherit some of the advantages of both structures. Roughly speaking, a Laguerre filter is constructed by ) replacing the delay elements (having transfer function in an FIR filter by allpass IIR filters (having transfer function Manuscript received November 13, 2002; revised November 13, 2003. The work of R. López-Valcarce was supported by a Ramón y Cajal grant from the Spanish Ministry of Science and Technology. The associate editor coordinating the review of this paper and approving it for publication was Prof. Zhi Ding. J. H. Gunther is with the Department of Electrical and Computer Engineering, Utah State University, Logan, UT 84322-4120 USA (e-mail: jake@ ece.usu.edu). R. López-Valcarce is with the Department of Signal Theory and Communications, University of Vigo, 36200 Vigo, Spain (e-mail: [email protected]. uvigo.es). Digital Object Identifier 10.1109/TSP.2004.837438

). The allpass filters give Laguerre models IIR characteristics. The adjustable parameters in Laguerre filters are the FIR-like feedforward filter tap coefficients and the pole position , which is often estimated from measured data and preset before estimating the filter tap coefficients. The . Finite stability of Laguerre filters is guaranteed by impulse response filters are an instance of Laguerre filters with . An important difference between FIR and Laguerre filters is that the output of FIR filters is a linear combination of time shifts of the input, whereas this is not true in Laguerre filters. Still, Laguerre filters possess a certain generalized shift structure that will be exploited in this paper for estimation of the filter tap coefficients, the input, and the initial state of the allpass sections using only observations of the output (i.e., blind estimation). The generalized shift structure of Laguerre filters was used to great benefit by Merched and Sayed [5], [6] in the development of order-recursive, recursive least-squares (RLS) Laguerre lattice filters. By exploiting the generalized shift structure, Merched and Sayed arrived at an algorithm with order computational complexity, which was an improvement of den complexity fixed-order RLS Laguerre Brinker’s [7] order algorithm. Fejzo and Lev-Ari extended the gradient adaptive lattice technique to Laguerre filters [8]. Their algorithm also computational complexity per sample. achieved order Wahlberg [9] studied the identification of Laguerre system parameters using input and output data, derived persistence of excitation conditions and error bounds, and studied the asymptotic statistical properties of least-squares estimates as the amount of data and the model order tend to infinity. The accuracy of Laguerre models depends on a correct choice of the pole parameter . Therefore, a large part of the work on Laguerre approximations is devoted to pole estimation. In [10] and [11], online algorithms are proposed for learning the optimal pole location. Offline estimation algorithms have been proposed that search an error performance surface for a global optimum [12]–[14] or derive the pole position to match given impulse response data [15]–[17]. Laguerre filters have also been applied with success to the problems of blind deconvolution of acoustic and digital communication signals. Stanacevic et al. use Laguerre filterbanks in an online adaptive gradient algorithm for source separation [18]. Hansson and Wahlberg exploited shift structure for blind estimation of Laguerre filter taps in a blind deconvolution application for communications [19]. Using the tap estimates, minimum mean square error estimates for the input signal were computed. Hansson and Wahlberg’s work was done in the continuous-time domain.

1053-587X/04$20.00 © 2004 IEEE

3358

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 52, NO. 12, DECEMBER 2004

TABLE I DIMENSIONS OF MODEL COMPONENTS

Fig. 1. SIMO Laguerre system.

In this paper, we investigate closed-form estimation of the Laguerre tap coefficients, the input signal, and the initial state of the allpass sections using only a finite record of output data (i.e., blind estimation) (see Section III). Our work differs from that of Hansson and Walhberg in that our investigation takes place on discrete-time signals and systems, and we provide a means for directly estimating the input signal and initial states without the need for tap coefficient estimates. In [19], the initial states were taken to be zero. The identifiability of the discrete-time SIMO Laguerre model is studied in Section IV. The paper concludes with simulations in Section V.

and allpass

The state equations of the lowpass systems are

(1) (2) (3) (4)

II. DATA MODEL The single-input, multiple-output (SIMO) Laguerre system that is the subject of study here is illustrated in Fig. 1. In this paper, all the signals and systems will be taken to be real valued. The results can easily be generalized to the complex case. The are input signal is a scalar process and the tap coefficients 1 vectors. Let denote the output of the allpass section, be its internal state. The mathematical relationship and let between input and output for the Laguerre system in Fig. 1 is most easily described in the -transform domain by

(5) (6) (7) where

where

and are the -transform of , and are the discrete Laguerre functions defined by

, , and the tap co. Then, the SIMO

Define the Laguerre tap input vector the state vector efficient matrix Laguerre system admits the model

,

,

, and

are given by

for

for . The discrete Laguerre functions are or, where is the thonormal in the sense that inverse -transform of . The Laguerre model used here differs slightly from Laguerre systems studied to date in that one additional connection between the input and the output has been added. In typical is zero so that does Laguerre filters, the tap coefficient not contribute directly to the output. The connection has been added here to allow unique identification of the initial state and tap, there is ambiguity between the the input. Without the input and the state vector . A state space description of the SIMO Laguerre filter that will be used in the remainder of the paper is developed next.

.. .

.. .

.. .

..

.

.. .

.. .

.. .

..

.

and where . Table I gives the dimensions of important signals and system parameters. Note that so that is invertible for .

GUNTHER AND LÓPEZ-VALCARCE: BLIND INPUT, INITIAL STATE, AND SYSTEM IDENTIFICATION OF SIMO LAGUERRE SYSTEMS

In the rest of the paper, we will use the blanket is controllable, implying that assumption that is full rank. This is a difficult condition to prove analytically but appears to be true in practice . for all In order to use subspace based methods for blind identification, a low-rank signal plus noise model is needed. This can be obtained by stacking the measured data. In order to stack the tap must be written in terms inputs , the sequence of a set of common inputs. Recursively substituting (5) into (6) times leads to

3359

III. BLIND ESTIMATION OF INPUT, STATE, AND TAP COEFFICIENTS The problem of blind estimation of SIMO Laguerre systems is, given and , to find a structured factorization . The leading matrix must have Kronecker structure, and the trailing matrix has two structured submatrices. The upper submatrix satisfies the state vector recursion (5). The lower submatrix has Hankel structure. Following [20], which dealt with the MIMO-FIR case, the factorization problem can be divided into two simpler subproblems. The fundamental to be a low rank fac-

idea is to require

, , and can be determined uniquely torization. Then, from the column and row spaces of by exploiting structure in the model. A low-rank factorization is possible if the matrices .. .

and

are tall and wide , respectively. The matrix

can be made wide by choosing Now, stacking

large enough

leads to (10)

.. .

.. .

.. .

..

.

..

.

.. .

(8) Define to be the block matrix is completely defined by the single scalar in (8). Note that parameter , which is assumed to be known. Observe that always has more rows than columns so that it is a tall matrix . The matrix also has full column rank for . The first columns are linearly independent because the appear in the first columns of the full rank matrix rows. The shift structure in the last columns is sufficient to guarantee their linear independence. The last columns must columns. Otherwise, be linearly independent of the first would not be full rank. samples so that (8) We assume that is constant over holds with the same from time to time . Form the block Hankel matrix as .. .

.. .

and form , , and in the same way as but in terms of the measurements , the noise , and the input , respectively. Collect corresponding state vectors into the matrix . The stacked data admits the signal plus noise model

The matrix is guaranteed to be tall and full column rank based . Therefore, the product on its structure provided will be tall if has at least rows, which leads to the lower bound for the stacking factor (11) is also necessary. Evidently, a minimum of two Note that to be outputs are required. The constraint (11) forces is wide. Conditions (10) and (11) are nectall even if and

essary but not sufficient conditions for have full rank

to

. Conditions sufficient to insure full rank

and

would have to include knowledge of the

and the structure of the input signal . For example, rank of could lead to a rank-deficient . a rank-deficient Similarly, if

was constant, then clearly,

would be rank

deficient. More will be said about identifiability in Section IV. The immediate focus is on estimation of , , and . In the following, the vector spaces spanned by the columns matrix will be referred to as col and rows of an and row , meaning for all for all

col row

For the moment, assume a noise free observation . Suppose that (10) and (11) are satisfied and that and

(9) where is a product.

identity matrix, and

is the Kronecker

have full rank

. Then, the

following subspace relationships hold: row

row

(12)

col

col

(13)

3360

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 52, NO. 12, DECEMBER 2004

This paper shows how relations (12) and (13) can be applied to blind identification of the Laguerre class of IIR systems. A. Estimating Of the two problems, estimating from col is more . Therefore, straight forward than estimating , from row this development begins with estimating . Assume has rank , and let (14) is , and be an SVD of , where is . Then, the columns of are an orthonormal basis for the column space of . Since the columns of form a basis for the same space, there must be an invertible matrix that converts one basis into the other. When noise is and the transformation can be present, the tap coefficients estimated by solving the least-squares problem: Minimize with respect to and , where is the Frobenius norm. Minimizing first with respect to leads to . Substituting back into the Frobenius norm and expanding in terms of the trace leads to the optimization problem for tr s.t.

(15)

where the constraint was added to avoid the trivial solution. The , are orthogonal projecmatrices could be used to replace tion matrices. By orthogonality, . Essentially, the problem reduces to choosing so that is orthogonal to . The cost function in (15) is a . Therefore, it can be quadratic function of the elements of rewritten more simply as s.t.

(16)

that windowed versions of the input sequence are contained in and are thus orthogonal to row row row .. . row

row

row

(17)

A similar set of relationships can be derived for the state vectors. From (12), it is clear that the rows of are contained in row . Using the state vector recursion (5), additional subspace relations can be developed. In particular, rewrite (5) at multiple times as follows:

(18) Therefore, the rows of contained in row since they are nations of the rows of , which are both Continuing in this fashion, is is possible to are in rows of . In summary, we have row .. . row

are linear combiand in row . show that the row for

row

row

(19)

With (17) and (19), it is possible to develop subspace relationships for the extended input and state sequences and . Define the matrices

where vec is a column scanned version of . An explicit expression for the symmetric matrix can be derived and but is rather complicated. Instead, in terms of we note that the elements of can be systematically determined by evaluating the quadratic function in (15) on the unit vectors vec

(20)

tr where is a vector of zeros, except for a one in the th povec . The solution to (16) is sition, and vec the unit-length eigenvector of corresponding to the smallest eigenvalue. B. Estimating

and

Next, consider the problem of estimating the input and the from . As in [20], two related approaches can state vector be taken using the union of null spaces or the intersection of row spaces. Both approaches will be summarized here. and (14) are, reIn the absence of noise, the rows of spectively, orthonormal bases for the row space and null space indicate of . Equation (12) and the Hankel structure of

for . Note that the columns of span the null space of , . The subspace relations (17) and (19) can be summarized as follows: row

col

(21) As indicated, the input and state sequences are contained in the intersection of the row spaces of . Equivalently, they are orthogonal to the union of the column spaces of . However, neither row space intersections nor null space unions provide enough information to uniquely characterize the input and state sequences. Linear combinations of the input and state vectors are also contained in this vector space. Equation (21) can only

GUNTHER AND LÓPEZ-VALCARCE: BLIND INPUT, INITIAL STATE, AND SYSTEM IDENTIFICATION OF SIMO LAGUERRE SYSTEMS

be used to estimate the input and state sequences up to an unknown mixture. Section IV proves that (21) uniquely specifies -dimensional space that contains the input and state sea quences. The ambiguity can be eliminated by exploiting state space structure of the Laguerre system in (18), as explained below. Either the row space intersection or the null space union can be used to calculate an orthonormal basis for the input/state left singular vectors of space. For example, the corresponding to the smallest singular values right singular vectors provide a basis. Equivalently, the of corresponding to the largest singular values could be used. Let the rows of be an orthonormal basis for the input/state space computed using an SVD as described . Because the above. The dimensions of are rows of span the same space as the input and state sequences, that relates the two bases there must be an invertible matrix (22)

C. Exploiting State Space Structure can be eliminated by exThe uncertainty presented by ploiting the state space structure of the Laguerre system (18). and are the same Observe that matrix except for a shift

3361

identifiable from the row space of . To begin, note that in the noise-free data model (25) fixed, a constant nonzero scalar factor can be exwith and the state/input changed between the coefficient matrix without changing the observation . That is, the pair and both lead to the triples same . Aside from this ambiguity, a necessary requirement for unique identification is that (25) is a low-rank factorization of

:

is tall with full column rank, and

is

wide with full row rank. These considerations led to (10) and , the matrix is guaranteed to (11). In the product . have full column rank by construction provided is times the rank of . Therefore, The rank of will have full column rank if (11) holds and is full rank. If rank , then will have full . While it may column rank, provided be possible to identify a rank-deficient , in the following, is assumed to be full rank. Readers interested in implementation issues may want to skip the technical details in the remainder of this section. For a brief overview, the main ideas are conveyed in Assumption 1 and the statements of Theorems 1 and 2. The following definitions will be useful in the statements and to be the matrix proofs of the theorems below. Define

The subscript indicates the number of columns. Note with that the matrix that we have used up to this point is . Setting gives the complete length state to be the Hankel matrix sequence. In addition, define Therefore,

can be determined from the equations

.. .

(23) which are linear in

. Vectorizing both sides of (23) leads to

with rows and columns. Setting gives the matrix , which we have dealt with up to this point. The row space intersection and the null space union criterion in (21) and : the case. applies to Underlying all that has been said to this point is the assumption that the input signal has rich enough structure to assure that the matrix

(24) where

vec . This is an overdetermined system of equations in unknowns. The least-squares be the right singular vector corresponding solution is to let to the smallest singular value of the coefficient matrix in (24). IV. BLIND IDENTIFIABILITY OF THE COEFFICIENT MATRIX, THE INPUT, AND THE STATE This section investigates the conditions under which and identifiable from the column space of , and

is are

.. .

has full row rank

. Physically,

this assumption means that the input cannot be a constant or a sum of a few sinusoids, for example. In the discussion that follows, we make the following blanket assumption that is assumed to hold, even though it is not explicitly stated. Assumption 1: The input signal is sufficiently exciting for rank for . The following theorem states the conditions under which and are identifiable from the row space of . The theorem and the accompanying lemma follow the pattern set forth in [20]

3362

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 52, NO. 12, DECEMBER 2004

for the FIR case. Here, the added complexity of the state vector must be dealt with. The main contribution of the sequence following theorem is to show how to proceed in the presence of the state vector. Therefore, this extends a result in [20] to general IIR systems expressed in state space form with known system matrices , . output, Theorem 1: Consider the single-input, orderLaguerre system (5)–(7) with known pole location . Assume that and satisfy (10) and (11) and that . Let

rank factorization of structure of

times rows

of . Rows of appear explicitly in the same rows and of . This leaves questions only about rows of , which are as in (26), shown at the top of the of (the last next page, stacked beneath rows columns are not shown since they are zero). The boxed

row rank in Assumption 1. The boxed vector on the second to last row of (26) is linearly independent of the shaded rows above it. The last row in (26) cannot be a linear combination of the rows in the upper block. For if it was, then the boxed rows in the upper block could not be linearly independent. Therefore, rows and of cannot be synthesized by taking linear combinations of the rows of , and as a result, neither of row . these rows can appear in the intersection row Similar arguments show that all but rows and of can by synthesized by linear combinations of the rows of . Counting the number of linearly independent row vectors in . the intersection leads to To complete the proof, this demonstration must be repeated . In each of these steps, as in the step for shown above, the proof boils down to arguments about the linear independence of subsequences of . These questions can be answered by appealing to Assumption 1. The conclusion is that row is the dimension of the intersection in general, which completes the proof. Proof of Theorem 1: Lemma 1 fully utilized the state space structure of the Laguerre system and the Hankel structure of to conclude that the -fold intersection row is

be a structured

. The state space structure (18), the Hankel

, and the condition row

row

specify, up to a nonzero scaling factor, the state vector sequence and the input . The following lemma will be utilized in the proof of the theorem. be an orthonormal basis for Lemma 1: Let the rows of , and define as in (20). Under the conditions of Therow row is diorem 1, the intersection and with mensional and contains the rows of arbitrary extensions for . case. Let Proof: The statement clearly holds in the , and consider the dimension of row row . In

,

may be replaced by

since they have the same

and are row space. With this replacement, the matrices shown in the equation at the bottom of the page, where row indexes have been added for reference. The row vectors that aprow must be linear compear in the intersection row binations of the rows of both and . First, note that rows of can be synthesized by taking times rows of via (18). In addition, of can be synthesized by taking rows

dimensional and contains the By the rank condition on , the rows of

.. .

.. .

.

to a full-rank

.. .

.. .

.. .

.. .

.. .

rows of

.

are linearly

row . independent and, therefore, provide a basis for To complete the proof, it remains to be shown that there cannot be another state vector sequence and input that generates the same subspace, satisfies the dynamic (5), and corresponds

.. .

.. .

, which is assumed to have full

submatrix in (26) is

.. .

.. .

.. .

.. .

.. .

.. .

GUNTHER AND LÓPEZ-VALCARCE: BLIND INPUT, INITIAL STATE, AND SYSTEM IDENTIFICATION OF SIMO LAGUERRE SYSTEMS

.. .

.. .

.. .

Let the rows of be an orthonormal basis for the row space intersection. If there are two different state vector and input sequences, they can each be produced by different invertible transof , as in (22). Therefore, there must formations be two different solutions to (24). However, the coefficient matrix in (24) has only a one-dimensional null space, provided . Therefore, we must have , and the state vector and input sequences are unique. The foregoing argument does not prove that the matrix in (24) has a one-dimensional null space. Therefore, we offer the and be following alternative argument. Let two different sequences providing bases for the row space intersection that satisfy the same dynamic (5) and that satisfy rank

rank

.. .

3363

.. .

(26)

Although it is true, it is more difficult to show that Recursing times on the state equation gives the formula

.

(31) as well. which is true for the primed variables Substituting the relations (30) into (31) written down for primed variables gives

(32) Now, use (31) to replace

wherever it appears:

, as required

in the statement of the theorem. Because these sequences span the same space, they must be related by an invertible transformation for all

(27) (33)

We will show that this transformation is a scaled identity so that and differ only by a scalar. In this transformation, . Substituting (27) it is easy to show that we must have into leads to

The next step is to equate coefficients of and . Let us begin with the coefficient of equation

, , which gives the

(34) Inserting yields

wherever

(28) appears in (28)

. Recall that Using this equation, we can show that is . By the Cayley–Hamilton theorem [21], can be written as a linear combination of

(29) Equating coefficients of

(35)

gives the three equations Write down (34) for the case (35). This produces

where the fact that was used to eliminate the term from the second equation. In conclusion, then, means that

(30)

Substitute (34) into (36) wherever This leads to

, and then, substitute in

(36) appears.

3364

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 52, NO. 12, DECEMBER 2004

The structure of this equation is more visible when written out in matrix form as follows:

Proof: The implication is obvious in one direction. The main difficulty lies in showing that Statement 1 implies Statement 2. Assume Statement 1 is true. Then, there exists an inmatrix that satisfies vertible (38)

.. .

..

.

..

.

(37)

The first matrix in (37) is the controllability matrix, which has full rank. The second matrix is a full-rank Toeplitz matrix so that the product of the first two matrices is full rank. The last matrix must be zero since it nulls a full-rank matrix. The only . way this can happen is if , (33), in the case , simplifies to Using

The following proof consists of showing that is a scalar mulis a scalar multiple tiple of the identity and, as a result, that . Recall that is tall with full column rank. Therefore, of it has a nontrivial left null space. That there can be no row of in the left null space of can be seen by examining . Rewriting this equation the block elements of leads to

The matrix containing combinations of , row rank. Therefore, the only solution is Partition in (38) as follows: Finally, equating coefficients of

, and .

has full

gives .. .

Since the matrix on the right is the controllability matrix and is the diagonal matrix has full rank, it must be the case that . The next theorem establishes the uniqueness of estimated is tall from the column space of . The two cases where versus wide could be considered separately. The following theorem applies in either case. In the course of the proof, there are occasions in which the truth of the statement could be concluded were brought to bear. Howearly if the assumption of a tall ever, this would lead to restricted application. In the statement as a wide matrix with full row rank. and proof, think of Theorem 2: Consider the single-input output, order Laguerre system (5)–(7) with known pole location . Assume that satisfies (10), , and that rank . Let

,

be a structured factoriza-

tion of . The state space structure (5) and (6) and the condition col specify the filter taps up to a col constant scalar factor. To manage the presentation of the proof, it has been organized around several lemmas whose statements and proofs are mixed in with the proof of the theorem. and be two different sets of filter tap Proof: Let col coefficients, and suppose that col col . Clearly, , where is sufficient for the column spaces to be the same. It remains to show that it is also necessary. Lemma 2 states col if and only if that col . Lemma 2: Under the conditions of Theorem 2, the following two statements are equivalent. 1) col col 2) .

.. .

.. .

where and are 1, is , and the are scalars. columns of (38) involving and First, consider the last for . Recalling (8), the product appears times in different columns and rows on the left of (38). Therefore, there are different expressions for as follows:

(39) (39a) (39b) .. . (39c) , there are different expansions Similarly, for the term , , , , , and different expansions in terms of , and so on, down to the term , for for which there is a single expression. For the moment, focus on and columns of : expansions involving the (40) (40a) (41) (41a)

GUNTHER AND LÓPEZ-VALCARCE: BLIND INPUT, INITIAL STATE, AND SYSTEM IDENTIFICATION OF SIMO LAGUERRE SYSTEMS

.. .

Using the recursion, all degenerate to a single equation instance

3365

of the equations in (39)–(39c) . For

(42) (42a) (43) The values that appear on the right-hand side of the equations above are values in the impulse response of the system the first . Equations (39)–(43) compute the output of the system that is the convolution of the input sequence with the impulse response plus a term that characterizes the influence of the initial state on the output. On the other hand, (39a), (40a), (41a), and (42a) show that the same output sequence can be computed for the same system , and a different input but with a different initial state sequence . Given that the outputs of a system are equal, what are necessary and sufficient conditions for the inputs and the internal state to be the same? The answer were is contained in Lemma 3. The vectors , first introduced by partitioning the matrix . As suggested by the equations above and in the following lemma, they play the role of state vectors. Lemma 3: Let be a single-input, -state, -output system. Let initial state and input lead to outputs . Let initial state and input lead to outputs . If , then the following two statements are equivalent. for

1) 2)

and

for

.

Proof: That statement 2 implies statement 1 is clear. Asand sume statement 1 is true. Equating outputs for exploiting the state recursion leads to the system of equations

.. .

.. .

..

.

..

.

.. .

(44) The coefficient matrix has full column rank if it is tall, which or . leads to the condition Under this condition, the vector in (44) must be zero, leading for and through the state vector recursion to for . to Apply Lemma 3 to the system to conclude that the solution to (39)–(43) and others that can be generated from (38) is to let the “ ” block of have Toeplitz structure and to let vectors satisfy the recursion for , where for and for . Roughly speaking, what we have concluded so far is that the transformation in (38) taking one block Toeplitz matrix to another must itself have Toeplitz structure.

Similarly, there are single equations for each of the other prod. The independent ucts equations implied by (38) can be summarized by

(45) The left-hand side is simply the response of the Laguerre system with tap coefficients to an impulse applied at time and with zero initial state. The right-hand side is the response of to the input signal a Laguerre system with tap coefficients , and with initial state . Thus, the question of uniqueness reduces to the question of whether or not a La, initial state , guerre system with tap coefficient matrix can produce the impulse response of another Laand input but with a tap coefguerre system with identical ficient matrix . This issue is the subject of Lemma 4, which states the conditions under which output equality for two different systems is the same as input equality. and be two Lemma 4: Let output systems with the same single-input, -state, dynamics but possibly different measurement equations. Asis controllable. Let be the sume that outputs of the first system with zero initial state and driven by ). Let an impulse at time ( be outputs of the second system with initial state and . Suppose that for and input , ; then, , , and , i.e., the two systems have the same input, initial state, and the same measurement equations up to a scalar factor. columns of (45) can be Proof: Note that the first rewritten as

.. .

.. .

..

.

..

.

.. .

(46) The question relating to these equations is as follows. If the first outputs of are zero, must the initial state and inputs be zero also? The answer depends on the size of relative to the number of states and the output dimension . The matrix in (46) has full column rank if it is tall, which or . Under leads to the condition

3366

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 52, NO. 12, DECEMBER 2004

this condition, the solution is The state vector recursion leads to

,

. (47)

By throwing out the null equations and using (47), (45) reduces to

(48) Using the state vector recursion (47), note that the right-hand side of (48) can be rewritten as

need to match the transformation of the rows rows of by . Clearly, and can be of very different and yet still produce identical output sequences. One possible solution that is easy to compute is , which can be checked by substituting into (50)

since . Finally, consider the case . The matrix is now be the Moore–Penrose “wide” with full row rank. Again, let inverse, which satisfies . Multiplying (50) on the leads to right by (52) This, however, does not solve (50) in general, which can be checked as follows:

..

Define the Toeplitz matrix

matrix as follows:

..

.

.

.. .

(53)

(49)

However, there is a condition under which (52) does give a solution and that is precisely when since in that case, . Then, . Although , , and is one possible solution, it must be shown that it is the only one. In other words, we must show that having equality in (53)

and the

.. .

(54) , implies that For (54) to be satisfied for any

Then, (48) can be written more compactly as

. , we must have

(50)

(55)

The question relating to (48) and (50) is as follows. Can the system produce the first (for ) samusing the possibly ples of the impulse response of nonimpulsive input when for any . The answer depends on the size of relative to . It , , and is enlightening to consider the three cases . Start with the simplest case by assuming , in which is invertible. Multiplying (50) on the right by case, gives the solution . That this solves (50) can be checked by substitution. Note that and may be chosen arbitrarily, that, in general, will not be a scalar multiple of , and that need not be an impulsive sequence. A similar conclusion may be reached in the case. , in which case, is “tall” with full column Suppose be the Moore–Penrose generalized inverse [21] of rank. Let . Multiplying (50) on the right by leads to

Since is an orthogonal projection matrix that projects , (55) says that the rows of are onto the row space of since they are unchanged by projection. in the row space of Let the columns of the matrix be a basis for the null space of , . The matrix is . Since is in the row space of , it must be orthogonal to

(51) Note that is an orthogonal projection matrix. Therefore, according to (51), there is a great deal of freedom in choosing and since only the orthogonal projection of the

(56) We want to continue to examine the structure of . Since the last row of is , the first row of is zero. By the , there are coefficients Cayley–Hamilton theorem, for all such that . Using this idea, the blocks in can be written as

(57) where the blocks of

, and involving

for

. Taking all leads to the partition

GUNTHER AND LÓPEZ-VALCARCE: BLIND INPUT, INITIAL STATE, AND SYSTEM IDENTIFICATION OF SIMO LAGUERRE SYSTEMS

3367

where an

is controllability matrix that is invertible, and is . Now, it is easy to see that the matrix can be taken to be

Writing out (56) with partitioned matrices leads to (58), shown at the bottom of the page. Multiplying leads to the two equations (59) (60) Finally, using the Toeplitz structure of the matrices, we can see that . For example, consider the element on both sides of (60):

The only solution is . Next, equate the th to show that elements on both sides, and use . This procedure can be continued to show that . Then, (59) can be used to show that . With , the matrix , and (50) can be rearranged to becomes . Since has full row rank, the only solution is , which completes the proof. Apply Lemma 4 to the two systems and . The conclusion is that or . Since is square or . and full rank, we must have

Fig. 2.

Blind RMS estimation errors versus SNR averaged over 100 trials.

ance Gaussian-distributed independent variables. An indepenwith variance dent Gaussian noise sequence was generated and added to to form the noisy observation . The signal-to-noise ratio (SNR) was defined to be

SNR The algorithms from Section III were applied to estimate , , and from the noisy observations. The root mean-square estimation errors (RMSE) over trials were calculated at each SNR according to RMSE

RMSE

V. SIMULATIONS The main purpose of this paper has been to investigate algorithms for blind identification of single-input, multiple-output Laguerre system parameters, internal state, and input, and to address the associated identifiability questions. This section briefly examines the performance of the estimation algorithms presented in Section III. output Laguerre system with A single-input, internal states was simulated. The pole parameter was set to . A total of ( , ) noise-free observations were generated according to (5)–(7), where the , and the input coefficient matrix , the initial state were all generated as zero-mean, unit-vari-

RMSE where is the Frobenius norm. Due to the scale ambiguity in the estimates, the scalar parameter was chosen to give the best fit prior to calculating the RMS errors. New sets of parameters were generated at each trial. Fig. 2 shows RMS errors trials. versus SNR averaged over The next experiment is an initial attempt to understand how matrix are in blind estimaimportant the dimensions of the easier to estimate than a wide one? The mation. Is a tall . We simulated the situation where trix is

(58)

3368

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 52, NO. 12, DECEMBER 2004

Fig. 3.

Blind RMS estimation errors versus

m.

Fig. 5. Blind RMS estimation errors versus . The true

a

Fig. 6.

Fig. 4. Blind RMS estimation errors versus .

and was swept from to . When , matrix is wide, and it is tall when . The SNR the was held fixed at 30 dB. The results in Fig. 3 indicate that estimates are somewhat better when is wide. The state and input estimates do not seem to be sensitive to the dimensions . of Throughout the foregoing development, it has been assumed that the pole position of the Laguerre system was known. The performance of the estimation algorithms as approaches 0 or 1 was investigated by letting take on the values . The RMSE performance shown in Fig. 4 indicates that the quality estimate seems to be fairly insensitive to . The error of the in the state estimate appears to grow as approaches 1 while the error in the input estimate increases but levels off. Next, the accuracy of blind estimation when using an incorrect pole position was evaluated. The true system pole was . The blind estimation algorithms were given placed at incorrect pole positions between 0.1 and 0.9. Fig. 5 shows the RMSE versus the incorrect value. These results indicate that the pole position is very important in blind estimation. Finally, the sensitivity to knowledge of the system order was investigated. The true system order was set to ,

a

Blind RMS estimation errors versus Mismatch

a is at 0.5.

= M^ 0 M .

whereas the blind identification algorithm was given values in . Fig. 6 plots the RMS estimation the range error for the input versus the model order mismatch, which is . Mismatch values less than (greater than) defined to be zero correspond to under (over) modeling. Similar to subspacebased methods for FIR identification, the technique developed in this paper is also strongly dependent on accurate knowledge of the system order. VI. CONCLUSIONS A state space model for a general single-input, multipleoutput Laguerre filter was developed. Subspace-based blind estimation algorithms were derived that estimate the Laguerre coefficient matrix, the state vector sequence, and the input given only noisy observations. Conditions for blind identifiability of the input, state, and coefficient matrix from noise-free observations were presented. Future work will address extensions of the estimation algorithms and identifiability conditions to multiple-input, multiple-output Laguerre systems. Laguerre filters are a special case of generalized feedforward (GF) filters proposed by Principe et al. [22], in which the delays of an FIR filter are replaced by identical low-order stable

GUNTHER AND LÓPEZ-VALCARCE: BLIND INPUT, INITIAL STATE, AND SYSTEM IDENTIFICATION OF SIMO LAGUERRE SYSTEMS

IIR filters . Principe et al. studied a particular instance of GF filters called Gamma filters. Williamson and Zimmermann FIR introduced an even more general structure in which delays are replaced by different low order IIR filters with fixed pole locations [23]. Future work will also examine the possibility of applying the techniques in this paper to blind identification of SIMO and MIMO generalized feedforward and fixed pole filters. ACKNOWLEDGMENT The author would like to thank the anonymous reviewers for their careful reading of the manuscript and their suggestions, which significantly improved this paper. REFERENCES [1] L. Tong and S. Perreau, “Blind channel estimation: From subspace to maximum likelihood methods,” Proc. IEEE, vol. 86, pp. 1951–1968, Oct. 1998. [2] H. Liu, G. Xu, L. Tong, and T. Kailath, “Recent developments in blind channel equalization: From cyclostationarity to subspaces,” Signal Process., vol. 50, pp. 83–99, 1996. [3] K. Abed-Meraim, W. Qiu, and Y. Hua, “Blind system identification,” Proc. IEEE, vol. 85, pp. 1310–1322, Aug. 1997. [4] P. A. Regalia, Adaptive IIR Filtering in Signal Processing and Control. New York: Marcel Dekker, 1995. [5] R. Merched and A. H. Sayed, “Order-recursive RLS Laguerre adaptive filtering,” IEEE Trans. Signal Processing, vol. 48, pp. 3000–3010, Nov. 2000. [6] , “RLS-Laguerre lattice adaptive filtering: Error-feedback, normalized and array-based algorithms,” IEEE Trans. Signal Processing, vol. 49, pp. 2565–2576, Nov. 2001. [7] A. C. den Brinker, “Laguerre-domain adaptive filters,” IEEE Trans. Signal Processing, vol. 42, pp. 953–956, Apr. 1994. [8] Z. Fejzo and H. Lev-Ari, “Adaptive Laguerre-lattice filters,” IEEE Trans. Signal Processing, vol. 45, pp. 3006–3016, Dec. 1997. [9] B. Wahlberg, “System identification using Laguerre models,” IEEE Trans. Automat. Contr., vol. 36, pp. 551–562, May 1991. [10] H. J. W. Belt and H. J. Butterwerk, “Laguerre filters with adaptive pole optimization,” in Proc. IEEE Symp. Circuits Syst., 1996, pp. 37–40. [11] N. Tanguy, R. Morvan, P. Vilbe, and L. C. Calvez, “Online optimization of the time scale in adaptive Laguerre-based filters,” IEEE Trans. Signal Processing, vol. 48, pp. 1184–1187, Apr. 2000. [12] H. J. W. Belt and A. C. den Brinker, “Optimality condition for truncated generalized Laguerre networks,” Int. J. Circuit Theory Applicat., vol. 23, no. 3, pp. 227–235, May-June 1995. [13] A. C. den Brinker, “Optimality conditions for a specific class of truncated Kautz series,” IEEE Trans. Circuits Syst. II, vol. 43, pp. 597–600, Aug. 1996. [14] T. O. e Silva, “On the determination of the optimal pole position of Laguerre filters,” IEEE Trans. Signal Processing, vol. 43, pp. 2079–2087, Sept. 1995. [15] Y. Fu and G. A. Dumont, “An optimum time scale for discrete Laguerre networks,” IEEE Trans. Automat. Contr., vol. 38, pp. 934–938, June 1993.

3369

[16] M. A. Masnadi-Shirazi and N. Ahmed, “Optimal Laguerre networks for a class of discrete-time systems,” IEEE Trans. Signal Processing, vol. 39, pp. 2104–2108, Sept. 1991. [17] N. Tanguy, R. Morvan, P. Vilbe, and L. C. Calvez, “Improved method for optimum choice of free parameter in orthogonal approximations,” IEEE Trans. Signal Processing, vol. 47, pp. 2576–2678, Sept. 1999. [18] M. Stanacevic, M. Cohen, and G. Cauwenberghs, “Blind separation of linear convolutive mixtures using orthogonal filter banks,” in Proc. Third Int. Conf. Independent Component Anal. Blind Signal Separation, 2001. [19] A. Hansson and B. Wahlberg, “Continuous-time blind channel deconvolution using Laguerre shifts,” Math. Contr., Signals, Syst., vol. 13, pp. 333–346, 2000. [20] A.-J. van der Veen, S. Talwar, and A. Paulraj, “A subspace approach to blind space-time signal processing for wireless communication systems,” IEEE Trans. Signal Processing, vol. 45, pp. 173–190, Jan. 1997. [21] R. A. Horn and C. R. Johnson, Matrix Analysis: Cambridge Univ. Press, 1991. [22] J. C. Principe, B. de Vries, and P. G. de Olivieira, “The Gamma filter—A new class of adaptive IIR filters with restricted feedback,” IEEE Trans. Signal Processing, vol. 41, pp. 649–656, Feb. 1993. [23] G. A. Williamson and S. Zimmermann, “Globally convergent adaptive IIR filters based on fixed pole locations,” IEEE Trans. Signal Processing, vol. 44, pp. 1418–1427, June 1996.

Jacob H. Gunther received the B.S., M.S., and Ph.D. degrees in electrical engineering from Brigham Young University (BYU), Provo, UT, in 1994, 1994, and 1998 respectively. From 1992 to 1994, he was a Research Assistant with the Microwave Earth Remote Sensing Laboratory, BYU. From 1994 to 1995, he was with Lockheed-Martin, Manassas, VA, where he worked on target detection algorithms, sonar systems, and satellite digital communication systems. From 1998 to 2000, he worked at Merasoft, Inc., Provo, where he worked on speech recognition, speaker identification, and acoustic echo and noise cancellation using microphone arrays. He joined the faculty of the Department of Electrical and Computer Engineering, Utah State University, Logan, in 2000. His research interests include array signal processing for wireless communications, blind deconvolution and source separation, and system identification.

Roberto López-Valcarce (M’01) was born in Spain in 1971. He received the telecommunications engineer degree from Universidad de Vigo, Vigo, Spain in 1995 and the M.S. and Ph.D. degrees in electrical engineering from the University of Iowa, Iowa City, in 1998 and 2000, respectively. From 1995 to 1996, he was a systems engineer with Intelsis. He was the recipient of a Fundación Pedro Barrié de la Maza fellowship for continuation of studies. He is currently a research associate (Ramón y Cajal fellow) with the Signal Theory and Communications Department, Universidad de Vigo. His research interests are in adaptive signal processing, communications, and traffic monitoring systems.