Blind Identification of ARMA Models with Periodically Modulated Inputs

Blind Identification of ARMA Models with Periodically Modulated Inputs1 Erchin Serpedin and Georgios B. Giannakis Dept. of Electrical Eng., Univ. of Virginia, Charlottesville, VA 22903-2442 e-mail: georgios@virginia edu, tel: (804) 924-3659, fax: (804) 924-8818

.

Abstract Recent results have pointed out the importance of inducing cyclostationarity at the transmitter for blind identification of FIR communication channels. Present paper considers the blind identification problem of an M A @ , q) channel, by exploiting the cyclostationarity induced at the transmitter through periodic encoding of the input. It is shown that causal and s t a b l e M A ( p , q ) channels can be uniquely identified from the output second-order cyclic statistics, irrespective of the location of channel poles and zeros, and color of additive stationary noise, provided that the cyclostationary input has at least q + 2 cycles.

1 Introduction In [13], [6], [lo], [5], a new approach was introduced for blind identification and equalization of finite impulse response (FIR) communication channels by inducing cyclostationarity (CS) at the transmitted sequence (as opposed to the received sequence, as is the case with all fractionallyspaced (FS) approaches [9], [ll], [14]). For transmitter induced CS, blind channel identification can be performed with no restrictions on the channel zeros, color of additive noise, and channel order over-estimation errors. In the present work, we consider the problem of blind identificationof an ARMA@, q ) channel with the input symbol stream modulated by a strictly periodic sequence. We show that such a precoding of the input induces CS in the transmitted sequence and guarantees channel identifiability from the output second-order statistics, irrespective of the location of channel zeros/poles, and color of additive stationary noise, provided that the CS periodically precoded input possesses at least q 2 cycles. Blind identification of an ARMA channel is understood modulo a complex scalar factor, which can be overcome by automatic gain control and differential encoding. Blind ARMA identification based on second-order cyclic statistics has been addressed when the received data are fractionally sampled [7]. Parametric and nonparametric approaches for estimating thc ARMA transfer function are proposed in [7]. However, both approaches

+

'Theresearchwassupportedby ONRGrant No N00014-93-

1058-6393/98 $10.00 0 1998 IEEE

1-0485.

impose rather restrictive conditions on the channel zeros; namely, no zeros are allowed to be equispaced on a circle separated by an angle of 47r/P, where P denotes the period of CS. The parametric approach, which relies on extracting common zeros and poles of different cyclic spectra, is sensitive to errors due to finite sample effects/noise. In [12], a description of the subset of ARMA systems identifiable from the output second-order cyclic spectra is presented. It is stated that the maximum-phase ARMA filters with equispaced zeros are not identifiable. The present paper shows that by inducing CS at the input all stable and causal ARMA channels can be identified from the output secondorder cyclic statistics, and proposes estimating the AR and MA coefficients by solving two systems of linear equations. Thus, the numerical problems associated with common root finding are avoided.

2 Motivation and Problem Statement It is well known that in the stationary case, blind ARMA identification from the output second-order statistics is possible within all-pass factors. Even in the absence of all-pass factors, the recently developed FS-based approaches [11], [ 9 ] , [14], cannot, in general, be used to identify FIR approximations, resulted by truncating the impulse response of an ARMA channel. Extensive simulations have shown that if the FIR truncation contains sufficiently many lags (say 15t 2 0 lags), then the zeros of the resulting FIR channel appear to be equispaced on a circle. Thus, the FS-subchannels tend to have zeros close to each other, making the FS-based identification ill-conditioned numerically. As an illustration, we consider first the AR(1) model H ( t ) = 1/(1- a z - ' ) , with la/ < 1, and its FIR truncation

where N is an arbitrary integer. Note that the two subchannels resulting from downsampling by a factor P = 2 are, respectivcly: Go(.) = (1 - u * ~ z - ~ ) / ( a~' z - l ) , and G , ( z ) = uGo(z),and are identical except for a scaling factor. The zeros of the truncated model G ( z ) are equispaced by 2 x / 2 N on a circle of radius IQ/. Thus, even if we

1633

Figure 2. ARMA(p, Q) Channel

Figure 1. Zero Plots: All-Pass Filter

Let u s consider the ARMA channel in Fig. 2, where the zero mean, independently and identically distributed (iid) information stream s(n) is modulated by the deterministic and P-periodic sequence f(n),to obtain the input w ( n ) = s ( n ) f ( n ) . The channel output is corrupted by stationary noise ~ ( n assumed ), to be uncorrelated with the inaccessible s( n). The receiver samples z( n ) are modeled as:

increase the downsampling factor P, we end up with common or nearly common zeros among the FS-subchannels. This feature is encountered when using FIR truncations of arbitrary ARMA channels. If the transfer function of a genhnz-n, then from eral ARMA channel is H ( z ) = a certain order on the decay of its impulse response is almost exponential, i.e., similar to that of an AR(1) system. In addition, the decay rate is dictated by the pole of largest magnitude. It is this exponential decay which forces the zeros of the FIR-truncation to be uniformly spaced on a circle with radius approximately equal to the decay rate. For example,

considertheall-passfilterN(z) = (z-'-0.5)/(1-0.5z-'), with G ( z )denoting its 20-lag FIR approximation. In Figs. la-b, we plot the zeros of the subchannels when P = 2 and P = 3, respectively. It is clear that almost all roots are equispaced, and thus G ( z )cannot be equalized with the standard FS-approaches. We are thus motivated to treat the blind ARMA identification problem separately from the FIR FS-based framework. In many practical systems, ARMA filters are introduced in the communication link for special purposes, e.g., all-pass ARMA factors are introduced as prefilters at the receiver in order to reduce severe distorsions (like leadinghailing echoes [4, p. 304]), or, to shorten the impulse response of the channel to some manageable length in order to use a Viterbi detector [4, p. 331 and p. 3191. We conclude that FS-equalization is not applicable for such ARMA channels. The use of ARMA models for communication systems has recently received interest in modeling high-bit-rateDigita1 Subscriber Loops (DSL) on twisted copper lines. It has been shown that at high data speeds, the length of the channel is very large (hundreds of taps), and real time implementation of an FIR equalizer becomes very costly [l].Such reduced-order ARMA representations are also useful for implementing reduced complexity Viterbi detectors. Research interest has also been devoted to the problem of approximating a long FIR filter by a reduced-parameter ARMA filter [ 2 ] . Blind ARMA identification may represent a viable solution for tracking slow variatiocs of such channels, or, for correcting misadjustments resulting from finite training sequence, and quantization effects [8, p. 16621.

a

D

c u I z ( n - I ) = C b l w ( n - [)

r=o

+ w(n),

= 1 . (1)

The time-varying correlation of w ( n ) at time n and lag T is defined as c,,(n; 7-):=E[w(n)w*(n T ) ] and satisfies c,,(n; T ) = c,w(n IP; 7 ) = .51f(n)12S(7-), V I , 7- E z, ~ * denoting complex conjugation. with ~ : : = E l s ( n ) 1and Thus, w ( n ) is CS with period P, provided that the sequence is periodic with period P . We assume for the rest of paper that sequence lf(n)I has period P. The CS of w ( n ) is preserved at the output z ( n ) . The time-varying correlation of z ( n ) is given by c,,(n; 7-):=E[z(n)x*(n T ) ] = If(n-m)12h(m)h*(m+r)+c,,(7-),where cU,(7-):=E[w(n)w*(n T ) ] . We deduce that c,,(n; T ) = cz2(n T ;T ) , V n. Being periodic, c,,(n; T) accepts a Fourier series expansion over the set of complex exponentials with harmonic cycles Agz = {2nk/P, k = 0 , . . ., P - l}. Its Fourier coefficients C,,(k; T):= 1/P Cfzi c,,(n; T ) exp (-j27kn/P), called cyclic correlations, are given by C,,(~;T) = a ~ F z ( k ) C ~ = - , h(m)h*(m T ) e - j Z a k n / P c v , , ( ~ ) S ( k ) where , Fz(k>:= P-' C~~~lf(n)12e-jznk~/p. The Z-transform of thecyclic for a fixed cycle k , correlation sequence {Cz,(k; T)}?=-,, is called cyclic spectrum, and is factorized as (IC # 0)

+

+

If(.)\

a0

I=O

+

+

+

+

+

S,,(IC; z ) = u;F@)H(ej2nk/P z - I ) H * ( z * ) . (2) Given {z(n)}rii,and knowing the modulating sequence f(n), wewish to estimate: {b~}y=~.

3 AR-Parameter Estimation A procedure for estimating the order p and the vector of AR-coefficients aT = [ U , . . . u p ] from an extended cyclic Yule-Walker (YW) system of equations is proposed next. Multiplying(1) by z*(n - Q - 7-), with 7- 2 1, and taking expected values, we arrive at

f:

urcz2(n;-Q - T

l=O

1634

+ E)

= E[V(n)x*(n- y - 7-11 . (3)

A(exp (j2,k/P>~-~)A*(z')S,~(k; 2). Making the change of variables z +, e j 2 r r k / P / z and z H e j 4 a k l P z * in B 2 ( k ; z ) , and considering the ratio &(k; e i 2 * k l P / z ) / B ;(k;e i 4 s k l P z * ) ,the following crossrelation is obtained F; ( k ) ~ 2 ( ke ;j 2 x k l P / z ) ~ ( e - j 4 x k l P

The R.H.S. of (3) is time-invariant with respect to n (since s ( n ) is stationary and H ( z ) is time-invariant), while the L.H.S. sum is periodic. Considering the discrete Fourier Series representation of (3), we obtain for k = 1, . . . , P - 1 P

Cale-mk1lPCz , ( k ; -q - 7

+ 1) = 0 ,

7

2 1. (4)

l=O

=F~(~E.)B e j ;4 (" ~ ' I;P z * ) ~ ( 2 ) , k = I , . . . , P - 1. (7)

For a fixed k, we concatenate equations (4) for 7 = 1,. . . , p 1 to obtain the system of equations C k ( p , q ) a = 0, where c k ( p , q ) is the ( p 1) x (p 1) Toeplitz matrix with first column and row

+

+

+

T

given by [C,,(k; -q - 1).. . C,,(k; -q - p - l)] and [C,,(k; -q - 1 ) .. .exp ( - - m k P / P ) c z z ( k ; -Q - 1 p ) ] , respectively. Note that in (4)the contribution of stationary noise is cancelled out by using non-zero cycles. We collect all such matrix equations for k = 1,. . . , P - 1, to obtain

+

C(P,4)" = 0 , C(P7 d:= [CT(P, !I).. . CT,-I(P,

T

d]

'

+

Similar to C k ( p ,q ) and C ( p ,q ) , we introduce the (p(5) 1) x (p 1) Toeplitz matrix C k ( p , q ) and the matrix T C ( p ,q):= [ C T ( p ,q) . . . C$-l(F, q)] , where known upper bounds (j7, q) replace the unknown @, q ) orders. The extended cyclic Yule-Walker system of eqs. (5)is re-written as C(F,q)z = 0, (6) where Z stands for the extended AR-parameter vector. We have the result (see Appendix for a proof):

+

Proposition 1. Suppose we select p 2 p, 2 q, and q - jj 2 q - p. (a). If for a fixed ko E [ l , P - 11, A ( z ) , B ( z ) do not contain (pole,zero) pairs of the form ( p , p*-' exp (-j2nko/P)), then rank(C(j7, q)) =: p . (b). If the CS inputpossesses at least q + 2 cycles, then it is guaranteed that rank(C(p, @) = p, and hence (6) has a unique solution. 0

-

Thus, the AR order p can be in principle determined via SVD from the rank of C(j7,q). Once the AR-order p is determined, Proposition 1 guarantees unique identifiability of the AR-vector a from (6), by considering j7 = p. Remark If the additive noise ).(U is white (or, in general, an MA(q) process), then the R.H.S. term of (3) cancels out ( E v ( n ) z * ( n- q - T ) = 0 for V T 2 1). Thus, the cycle k = 0 can be used in the Yule-Walker system of eqs. (5). Cyclic correlations C,, (0; T ) become the ordinary secondorder correlations in the case when w ( n ) is stationary and iid. Because in the CS-case the YW-system of eqs. uses also cycles k # 0, as opposed to the stationary case where only k = 0 is used, we expect that inducing cyclostationarity at the input, besides allowing identification of ARMA models, also improves accuracy of AR-parameter estimates.

4 MA-Parameter Estimation Having recovered the AI3 part, the MA cyclic specB * actrum &(k; z):=cr:~2(k)~(exp( ~ z T ~ / P ) z - ' )(z*), cording to ( 2 ) , can be recovered as B Z ( k ; z ) =

In order to re-write (7) in a matrix form, let g:=[g(O) g( 1) . . .g(L,)IT,and define the (E+ L, + 1) x ( E + 1)Toeplitz matrix 'Tc;(L), having as first column and row the vectors [g(O) g(1) . . . g ( L,) O...OIT and [g(O) O . . . O ] , respectively. Denote by 7Bk,,(E)and 7Bk,*(Z)the

(z+

( E + 2q + 1) x 1) Toeplitz matrices associated to the 2qth-order polynomials F,"(k)z-qB,(k; e j 2 . r r k l P / z ) and Fz(k)z-QBI(k;e j 4 * k / p z * ) , respectively. Define 1) x ( E -I- 1) diagonal matrix Dk(z) := the ( E diag{ 1,exp (j27rk/P), . . . , exp ( j 2 7 r k z / P ) } and , the MAvector b:=[bo...b,lT. The vector of coefficients bk of B ( e - j Z x k l P z )is given by: bk = Dk(q)b. Equating the coefficients of both sides of (7), we deduce T ~ ~ , , ( q )=b k 7~,,,(q)b, and, thus

+

z('?)b = 0 %(q):='TEk,,(q)Dk(q) - %,2(Q) . (8) We have shown in [lo, Theorem 11that the MA-vector b can be uniquely recovered from (S), if P 2 q 1. Moreover, if the MA-order q isover-estimated in the range q 5 < P+q, then again the identifiability of MA-vector b from (7) is guaranteed [lo, Theorem 21. This result lessens accuracy requirements on the MA-order determination problem, and the MA-order can be determined simply by inspecting the memory of B2(k; z ) ; i.e, the "effective length" of b. Rigurous statistical tests are needed to select ARMA orders, but are beyond the scope and size of this paper. We remark that the MA-order determination approach from [15] can be adapted to the cyclic domain using techniques similar to those described in the proof of Proposition 1. In summary, only one cycle together with P 2 q 2 are sufficient to identify uniquely the AR and MA-vectors. By taking advantage of all the cycles present in the CS input, better estimates for the MA-vector may be obtained. By collecting eqs. (S), corresponding to all cycles k = 1 , .. ., P - 1, we deduce the eqs. I ( q ) b = 0 , where I(q ) = [qT ( 9 ) . . .7F- ( q ) I T . MA-parameter estimation can be further improved by taking into account crossrelations between two different cyclic spectra &(k; z ) and B2(l;z ) (k # I). It is easy to check also that 1

+

+

~2

(11~2 ( I C ;z>B (ej2*1/P/ z ) = ~2 (r~ ) ~2

( I ; z ) B (e j 2 x k l P/z).

(9) Equation (9) can be brought to a form similar to (8), and thus the MA-vector can be estimated by stacking together all the equations (7) and (9), corresponding to all possible cycles. In practice, the output cyclic correlations are consistently

1635

6 Conclusions

estimated using the sample cyclic correlation computed as: ~

N-1

+

Multiplying s ( n ) by the periodic f(n) possessing q 2 cycles we are assured by Proposition 1that channel identifiability is guaranteed. Because the cyclic correlation coefficients are consistently estimated, we infer that the estimates of ARMA coefficients are consistent.

5 Simulation Results We simulated an ARMA(2,2) channel with transfer func( 10.7071~-' + tion H ( z ) = (1 - 1.752-' - 0 . 5 ~ - ~ ) / 0 . 2 5 ~ - ~ and ) , compared the performance of the proposed cyclic YuIe-Walker (CYW) approach with the FS-based approaches MDCM 191, and XLTK [14]. The modulating sequence f ( n ) has period P = 8, and within one period it takes the values { 1, 1, 1, 1, 1,2,2,2}. The channel is specially chosen with a cyclic all-pass factor at cycle 2 ~ / 8 (k = 1). The set of cycles used in the CYW approach is { 2 n k / 8 , le = 1,. . . ,7}. Thus, the channel identifiability condition is satisfied since 7 2 q 2 = 4. For the FSapproaches, the channel was considered to be a 16-lag FIR truncation of the ARMA(2,Z) channel, and the oversampling factor equal to 2. Input s ( n ) was an iid QPSK sequence, and additive noise U(.) was white and normally distributed. As channel estimation performance measures, we plot the normalized root-mean-square error (RMSE) [I 11, and the average bias (Avg. Bias) [9] of channel estimates versus signal-to-noise ratio (SNR) and number of samples ( N ) . For all simulations, we define SNR at the equalizer input as: SNR:=E{lz(n)12}/E{lv(n)12}, and use M C = 100 Monte-Carlo runs. In Figs. 3a-b, we plot RMSE/Avg. Bias of channel estimates versus N, at S N R = 10 dB, while in Figs. 3c-d the channel RMSE/Avg. Bias is plotted versus SNR, for a fixed number of samples N = 2,000. We note that FS-based approaches fail to provide good channel estimates even for large values for N and SNR, because the truncated FIR channel is close to being non-identifiable. This followsfrom the zero-plots of the two subchannels shown in Fig. 4a. We tested also the performance of MDCM, XLTK, and CYWbased Wiener equalizers with respect to the probability of symbol error after equalization for different SNR levels. In all experiments, the lengthof the Wiener equalizer was fixed to 21 lags, and the channel was estimated at each MonteCarlo run using N = 2,000, and N = 3,000 samples, respectively. The output of the ARMA channel was first convolved with the AR estimate, and then deconvolved with the mmse Wiener equalizer. The resulting symbol error curves are plotted in Fig. 4b, where 300 Monte-Carlo runs were averaged.

+

The paper showed that any ARMA@, Q ) channel can be identified by exploiting the cyclostationarity induced at the transmitter by periodically encoding the input. The identification can be performed from the output second-order cyclic statistics, without assuming any restrictive condition on the channel zeros and poles. The only condition required is the number of cycles to be greater than or equal to q + 2. Some additional problems remain to be solved: performance analysis of a cyclic decision-feedback equalizer, derivation of asymptotically optimal algorithms, selection of optimal periodic sequences f(n),modeling of FIR long channels with reduced-order ARMA representations, investigating other approaches for estimating the MA-vector. Acknowledgment. The authors wish to thank Prof. Phillip Loubaton for his remark concerning the selection of number of cycles. Appendix: Proof of Proposition I . First we obtain a characterization of the null space of C(p,q). Consider the (I + p ) x 1vector g:=[go. . . g ~such ] that ~ C ( p ,q) g = 0. Elementwise, we have for k = 1, . . . , P - 1: P c g l e - j 2 T k ' / p C x x ( k-;q - ~ + l ) = 0, r = 1,. . .,pi+l. l=O

(10)

For a fixed le, define the sequence P

Yk(P):=

-&Ire

-j2nkl/Pc X z ( k

-T-

T

+I).

l=O

The Z-transform of y k ( T ) is given by

Yk( 2 ) = c:F2( k)zp H*( 1/ z * ) H(ej2T k / P z)G(e j 2Tk'P

211

(11) where G(z):=ETzog l z - ' . According to (lo), we have Y ~ ( T )= 0, for T = 1,.. . , p + 1 and k = I , . . ., P 1. Since the inverse Z-transform of Y k ( z ) is given by 1/(2~$ j )Y k ( Z ) z r - ' d z , (10) and (11) imply that for k = 1, . . . , P - I 2nj

f

B*(I/z*) .zTB(ej2Tk/Pz) A*(l/z*)

From the assumption of Proposition l a , it follows that for a given ko, B * ( l / z * ) and z P A ( e J 2 " k o / Pare z ) coprime(i.e., no cyclic all-pass factor is present at cycle ko). All the poles inside the unit circle of the integrand in (12), corresponding to k = ko, are given by the p zeros of z P A ( e j 2 T k " / P z ) . According to [3, Lemma 11, it follows that the integrand in (12) must be analytic inside the unit circle, i.e., all the poles inside the unit circle must be cancelled by appropri) divide ate zeros. Thus, necessarily z p A ( e J 2 " k o I P zmust ~ ~ G ( e j ~ ~We~ can ~ lwrite ~ z G) (.z ) = A ( z ) K ( z ) ,where K ( z ) is an arbitrary polynomial of degree p - p such that

1636

z q - q K ( z ) is analytic inside the unit circle. It is easy to check that this solution satisfies (12) for any other value of IC, and thus, it spans the null space of matrix C(F,@.Hence, dim[n/(C(jj, q))]= jj - p , and rank[(=@, U)] = p (where N stands for null space). Next, we show that if the CS ~ ( npossesses ) at least q 2 cycles, then rank(C(p,U)) = p . The poles of the integrand (12) located inside the unit disk are given by the zeros of A ( e j Z T k I P zwhich ) are not cancelled by the zeros of B*(l/z*). Since B*(l/z*) has only q zeros, and there are P - 1 2 q 1cyclic spectra, it follows that all the zeros ofA(z) are to befoundamong thepolesoftheintegrand(12) located inside the unit disk. According to [3, Lemma 11, it follows that A(%)must divideG(z), and rank(C(p, q)) = p .

+

+

[12] L. Tong, “Identifiability of Minimal, Stable, Rational, Causal Systems Using Second-Order Output Cyclic Spectra,”IEEE Trans. on Autom. Control, no. 5 , pp. 959-962, May 1995. [13] M. K. Tsatsanis and G. B. Giannakis, “Transmitter Induced Cyclostationarity for Blind Channel E ualization,” IZEE Trans. on Signal Proc., no. 7, pp. 1785-1594, July 1997. [14] G. Xu, H. Liu, L. Tong, and T. Kailath, “A least-squares approach to blind channel identification,” IEEE Trans. on Signal Proc., pp. 2982-2993, Dec. 1995. [15] X.-D. Zhang and Y.-S.Zhang, “Determination of the MA order of an ARMAprocess using sample correlations,”IEEE Trans. SignalProc., vol. 41, no. 6, June 1993. R

References N. Al-Dhahir, A. Sayed, and J. M. Cioffi, “A high-

performance cost-effective pole-zero MMSE-DFE,” in Proc. Allerton Con5 Commun., Control, Computing, pp. 1 1661175, Sept. 1993.

Ibd-

ma-

N. AI-Dhahir, A. Sayed, and J. M. Cioffi, “Stable PoleZero Modeling of Long FIR Filters with Application to the MMSE-DFE,” IEEE Trans. on Communications, vol. 45, no. 5, pp. 508-514, May 1997. K. L. htrom and T. Soderstrom, “Uniqueness of the Maximum Likelihood Estimates of the Parameters of an A R M Model,” IEEE Trans. on Automatic Control, vol. AC-19, no. 6, pp. 769-773, Dec. 1974. J. A. C. Bingham, The Theory and Practice of Modem Design, John Wiley & Sons, Inc., 1988. YYI 149

A. Chevreuil and P. Loubaton, “Blind second-order identification of FIR channels: forced cyclostationarity and structured subspace method,” IEEE Signal Processing Letters, vol. 4, no. 7, pp. 204-206, July 1997.

NO

( . I

Figure 3. Channel RMSE and Avg. Bias

G. B. Ciannakis, “Filterbanks for Blind Channel Identification and Equalization,” IEEE Signal ProcessingLetters, vol. 4, no. 6 , pp. 184-187,June 1997.

Y. Li and 2. Ding, “ A R M System Identification Based on Second-Order Cyclostationarity,” IEEE Trans. Signal Proc., vol. 42, no. 12, pp. 3483-3494, Dec. 1994. P. J. W. Melsa, R. C. Younce, and C. E. Rohrs, “Impulse Response Shortening for Discrete Multitone Transceivers,” IEEE Trans. on Communications, vol. 44, no. 12, pp. 16621672, Dec. 1996.

Figure 4. (a). Zero Plot; (b). BER vs. SNR

E. Moulines, P. Duhamel, J. Cardoso, and S. Mayrargue, “Subspace Methods for the Blind Identification of Multichannel FIR Filters,” IEEE Trans. on Signal Proc., vol. 43, no. 2, pp. 516-525, Febr. 1995.

E. Serpedin and G. B. Giannakis,“Blind Channel Identification and Equalization with Modulation lnduced Cyclostationarity,IEEE Trans. SignalProc., Apri1797(submitted);see also, Proc. CISS, vol. 11, pp. 792-797, March 1997, Baltimore,MD. L. Tong, G. Xu, and T. Kailath, “Blind identification and equalization basedon second-orderstatistics: A time domain approach,” IEEE Trans. Inform. Theory, vol. 40, no. 3, pp. 340-349, March 1994.

1637