globally convergent algorithms for blind source separation - Core

GLOBALLY CONVERGENT ALGORITHMS FOR BLIND SOURCE SEPARATION Constantinos B. Papadias

Bell Labs Research, Lucent Technologies 791 Holmdel-Keyport Rd., NJ 07733-0400, U.S.A. [email protected]

ABSTRACT

We present a novel class of adaptive algorithms for the blind separation of non-Gaussian mutually independent source signals that can be modeled as independent identically distributed (i.i.d.) discrete random processes. The signals are assumed to be transmitted through a m  p narrow-band (instantaneous linear mixture) channel. The original algorithm, called the Multi-User Kurtosis (MUK) algorithm was rst presented in [1] and was derived from a set of conditions that were previously found to be necessary and sucient for the recovery of all the sources [2]. The analysis presented in [1], [3] has shown that the MUK algorithm is globally convergent to a zero-forcing { ZF (decorrelating) solution both in the absence of noise and in the presence of additive white Gaussian noise (AWGN), provided that the received signals are perfectly pre-whitened. In this paper, we propose other constant-modulus (CM) type variants of the MUK algorithm. These inherit the global convergence behavior of the MUK due to its de ation structure and moreover, they allow for increased convergence speed. These variants of the MUK are particularly useful in cases where the number of received signal snapshots is limited, such as in wireless communication applications.

1 INTRODUCTION

We assume that p i.i.d. and mutually independent zeromean discrete-time sequences aj (k), j =1; : : : ; p, that share the same statistical properties are transmitted through a pm MIMO linear memoryless channel which introduces inter-user interference (IUI). The m channel outputs (m  p) are subsequently ltered by an m  p receiver lter whose outputs zj (k), j =1; : : : ; p, should ideally match the transmitted signals ai (k). The receiver outputs can then be written as:

zj (k) =

p X l=1

gjl al (k) = GTj A(k) ; j =1; : : : ; p (1)

where A(k) = [a1 (k)


ap (k)]T . Gj is the channel/receiver cascade that contains the contribution of all the p channel inputs to the j -th receiver output, and

g11

- ?h -

a1 (k)

......... . .

. . .

.

g1p ..

z1(k)

-

P

- ?h -

ap (k)

......... . .

. . .

. . .

. . .

gp1

- ?h ......... . .

.

gpp ..

zp (k)

-

P

- ?h ........ . .

Figure 1: A multiuser instantaneous mixture setup

T denotes matrix transpose. Eq. (1) can be also written in the familiar form

z(k) = GT A(k) where

2

g11


gp1

G = G


Gp = 64 ... 



1

(2)

.. .

.. .

g1p


gpp

3 7 5

(3)

The above-described linear mixture setup is depicted in Figure 1 and can be used to model narrowband antennaarray systems, CDMA systems, or oversampled systems. In [2] (and references therein), it was shown that

Theorem 1 If each ai(k), i=1; : : : ; p, is an i.i.d. zeromean sequence, fai (k)g, faj (k)g are statistically independent for i = 6 j and share the same statistical properties, then the following set of conditions are necessary and sucient for the recovery of all the transmitted signals at the receiver outputs:

(C1) jK (zj (k))j = jKa j ; j = 1; : : : ; p (C2) E jzj (k)j2 = a2 ; j = 1; : : : ; p (C3) E (zi (k)zj (k)) = 0 ; i 6= j ;




;




;




where K (x)=E jxj4 ;2E 2 jxj2 ;jE x2 j2 is the (unnormalized) kurtosis of x, a2 , Ka are the variance and kurtosis of any aj (k), respectively and  denotes complex conjugate. A straightforward criterion can be then drawn from conditions (C1), (C2), (C3) of Theorem 1 in order to achieve blind source separation (BSS): 8 > < > :

max F (G) =

G

subject to: G G H

p X

j =1 = p

jK (zj )j

I

(4)

where Ip is the p  p identity matrix and H denotes Hermitian transpose. The constraint in (4) comes ;
from the fact that, according to (C2) and (C3), E zzH = a2 Ip (in (4) we have assumed for simplicity that a2 = 1). We call (4) the Multi-User Kurtosis (MUK) maximization criterion. Based on this criterion, an adaptive stochastic-gradient algorithm, called the MUK algorithm was developed in [1]. Due to its inherent de ation structure (see [5] for an early de ation-type BSS technique), the MUK algorithm was shown to be globally convergent to a zero-forcing (decorrelating) solution both in the absence [1] and in the presence [3] of AWGN, provided that the received signal is perfectly pre-whitened. In this paper we will present some CM-type variants of the original MUK algorithm. As will be shown, these techniques inherit the global convergence characteristic from the de ation structure of the MUK, however they may allow for increased convergence speed. The remainder of the paper is organized as follows. In Section 2 we review the MUK algorithm. In Section 3 we present its CM-type variants, namely normalized CMA and leastsquares CMA versions. In Section 4 we present some computer simulation results that show the increased convergence speed of the proposed techniques and nally Section 5 contains our conclusions.

2 THE MUK ALGORITHM In the following we denote the m  p channel matrix by H and the m  1 channel output vector by Y (k). The received signal model is then:

Y (k) = HA(k) + n(k)

(5)

where n(k) is the m  1 vector of additive noise samples. The receiver output can then be written as

z(k)=WT (k)Y (k)=WT (Tk)HA(k)+n (k) (6) =G (k )A(k )+n (k ) where W(k)=[W (k)


Wp (k)] and GT (k)=WT (k)H 0 0

1

are the m  p receiver matrix and p  p global response

k = 0: initialize W(0) = W0 for k > 0 Obtain W0 (k +1) from (7) Obtain W1 (k +1)=W10 (k +1)=jW10 (k +1)j for j = 2 : p Compute Wj (k + 1) as the ratio

1. 2. 3. 4. 5. 6.

Wj0 (k+1);

kWj (k+1); 0

7. 8. 9.

j ;1 X l=1

(WlH (k+1)Wj0 (k+1))Wl (k+1)

j ;1 X l=1

(WlH (k+1)Wj0 (k+1))Wl (k+1)k

Go to 5

W(k + 1) = [W (k+1)


Wp (k+1)] Go to 2

Table 1:

1

The MUK algorithm

matrix, respectively, and n0 (k)=WT (k)n(k) is the noise at the receiver output, all at time instant k. A stochastic-gradient algorithm for the constrained criterion (4) can be obtained as follows. We rst perform an update of W(k) in the direction of the instantaneous gradient of F (G) as follows W0 (k+1)=W(k) +  sign(Ka) Y (k)Z (k) (7) where Z (k) = [jz1 (k)j2 z1 (k)


jzp (k)j2 zp (k)] and  is the step size. We now have to satisfy the orthogonality constraint at the next iteration of the algorithm, GH (k+1)G(k+1) = Ip, which, assuming H to be unitary (corresponding to an assumed spatial pre-whitening of the received signals), reduces to WH (k+1)W(k+1) = Ip (8) As in general there is no guarantee that W0 (k+1) will satisfy the constraint (8), we have to transform it to a unitary W(k+1) = f (W0 (k+1)). We propose to choose W(k+1) as an m  p matrix which is as close as possible to W0 (k+1) in the Euclidean sense. As it turns out, this results in performing a Gram-Shmidt orthogonalization of W(k+1). The resulting algorithm is depicted in Table 1. Regarding the performance of the algorithm in Table 1, the following theorem was shown in [1]: Theorem 2 If the conditions of Theorem 1 are satis ed and H is full column rank and unitary, then the MUK algorithm described in Table 1 is globally convergent to a setting that recovers all the input signals, up to an arbitrary unit-norm complex scalar each. That is, after convergence, 2 3 ei1 0


0 6 . . . . . . .. 77 6 . 77  =  (9) 6 0 G = 66 . . . . . . . . 0 75 4 . 0


0 eip

where 1 , : : :, p are arbitrary p phases,  is a p  p permutation matrix and i = ;1. Moreover, regarding its behavior in the presence of AWGN, the following theorem was shown in [3]: Theorem 3 In the presence of additive white Gaussian and circularly symmetric noise with independent equivariate complex components, the only stable maxima of the MUK algorithm correspond to a decorrelating (zeroforcing) detector, i.e. to a solution of the type (9). These two results make the MUK algorithm attractive for the BSS of instantaneous mixtures. With the use of the LDU factorization of the received signal covariance matrix [6], the algorithm can actually be even applied to the case of convolutive mixtures. However, due its stochastic gradient nature, the algorithm is relatively slow converging (on the order of thousands of iterations). Our focus in the remainder of the paper will be on the development of faster converging variants of the MUK.

3 CM-TYPE VARIANTS OF THE MUK

The issue of convergence speed of adaptive blind deconvolution algorithms is important, especially in wireless systems whose time slot typically spans a rather small number of symbols (on the order of at most a couple hundred symbols in cellular TDMA). In [9] (and references therein), a number of variants of the CMA algorithm were proposed for single-user blind equalization. These are normalized, least squares and aneprojection algorithms that allow for increased convergence speed compared to the stochastic-gradient CMA. In this section we show how similar improved convergence variants of the MUK algorithm can be obtained. In the rest of the paper we will limit ourselves to subGaussian inputs (i.e. Ka < 0) { as is the case in typical communication signals. We begin by observing that the MUK algorithm of Table 1 is a two-step technique: it rst performs the stochastic-gradient adaptation of (7) to obtain W0 (k+1) from W(k) and then it performs a Gram-Shmidt orthogonalization of W0 (k+1) to obtain W(k+1). The adaptation (7) corresponds to the adaptation along the gradient of the kurtosis function F (G), whereas the GramShmidt orthogonalization (steps 4-8 in Table 1) to the satisfaction of the constraint in (4). If now we consider modi ed criteria of the type in (4) wherein only the cost function F (G) changes but not the constraint, then the algorithm in Table 1 will still apply, by a mere modi cation of Eq. (7). By assuming for these modi ed criteria too that the channel is perfectly pre-whitened, steps 4-8 will ensure that the constraint GH G = Ip is satis ed.

3.1 The CMA 2-2 variant

As an obvious step towards the derivation of faster converging schemes, we rst present the stochastic-gradient

NCMA

Wj0 (k+1)=Wj (k)+ Ek Y  (k)(sgn(zj (k));zj (k)) RLS-CMA

Wj0 (k+1)=Wj (k)+Rk;1 Y  (k)(sgn(zj (k));zj (k)) NSWCMA

Wj0 (k+1)=Wj (k)+Y (k)Pk;1 (sgn(Zj (k));Zj (k)) Table 2:

Three different CMA variants

CMA 2-2 variant of the original MUK algorithm. Having assumed that Ka < 0, the CMA 2-2 version of the criterion (4) is 8 > < > :

min J (G) =

G

subject to: G G H

p X

j =1 = p

(jzj (k)j2 ; 1)2

I

(10)

The corresponding algorithm is given again in Table 1, with the adaptation (7) of step 3 being replaced by

W0(k+1) = W(k) ; Y (k)(k) (11) where (k) = [(jz (k)j ;1)z (k)


(jzp (k)j ;1)zp (k)]. 1

2

1

2

Based on the facts that (i) each column of (11) is a standard single-user CMA; (ii) the single-user CMA and the single-user kurtosis algorithms are equivalent for Ka < 0 on GHj Gj = 1 [7]; (iii) the channel H is assumed perfectly pre-whitened and (iv) both the CMA and the single-user kurtosis stationary points are noise-free for orthogonal channels [8], one can show that Theorems 2 and 3 apply equally well to the CM-variant of the MUK that uses (11) instead of (7). That is, Theorem 4 Both in the absence of noise and in the presence of circularly symmetric complex AWGN and assuming Y (k) to be perfectly pre-whitened and Ka < 0, the only stable minima of the CMA 2-2 variant of the MUK algorithm correspond to a decorrelating (ZF) solution of the type (9). Due to HH H = Ip , this solution also minimizes the mean squared error (MMSE) (similarly to [8]).

3.2 Other CM-type variants

In (11) each component Wj (k) of W(k) is updated to Wj0 (k+1) through a standard CMA 2-2 iteration. We may now apply the so-called \separation principle" presented in [9] to replace the adaptation of each column in (11) by a faster converging CMA-type algorithm. Table 2 shows some options that are borrowed from [9]. We expect the MUK variants that use these recursions instead of (7) or (11) to (i) inherit the global convergence property of the original MUK algorithm; (ii) oer increased convergence speed, at the cost of extra computational complexity. In particular, NCMA is a normalized version of CMA that nulls the posterior error after each iteration. Ek =kY (k)k2 =Y H (k)Y (k) is the

energy at iteration k and sgn(z ) = z=jz j. RLS-CMA is a least-squares version of CMA, that uses the sample covariance matrix Rk = E (Y (k)Y H (k)), where E denotes sample expectation. Finally, NSWCMA is a normalized sliding window version, wherein a number of L posterior output errors are nulled (Y(k) = [Y (k)


Y (k;L+1)], Pk = YH (k)Y(k), Zj (k) = YT (k)Wj (k) and the sgn of a vector is obtained by element-wise sgn operations).

4 COMPUTER SIMULATIONS

A number of computer simulation results have shown that, as expected, the proposed CM-type variants of the MUK algorithm maintain its globally convergent behavior, while achieving higher convergence speeds. Figure 2 shows an example where a user's 4-QAM signal is split into 4 independent sub-streams which are transmitted each on one of 4 antenna elements through a narrowband channel to a 6-element receiver antenna array. The entries of the 6  4 channel matrix are independent random complex scalars which are computed with the use of the Jakes model at a 1.9 GHz frequency, assuming a slowly moving receiver (10 km/h). The SNR is 18 dB and we run the NCMA variant of MUK with =0:3 for 100 iterations. We then use the last setting to lter all the received data. The corresponding outputs after convergence (and phase removal) are shown at the top of the gure, whereas the lower part shows the corresponding squared errors. Notice how the algorithm has been able to separate well the four sub-streams, within only 100 iterations. Several other similar results have been obtained for dierent variants of the MUK.

5 CONCLUSIONS

1st sub−stream

4th sub−stream

1

1

0.5

0.5

0.5

0.5

0

0

0

0

−0.5

−0.5

−0.5

−0.5

−1

−1 −1

0

1

−1 −1

2

0

1

−1 −1

2

|z1−a1|

0

1

−1

2

|z2−a2|

4

4

3

3

3

3

2

2

2

2

1

1

1

1

0

50 samples

100

0

0

50 samples

100

0

0

50 samples

1

|z4−a4|

4

0

0

2

|z3−a3|

4

100

0

0

50 samples

100

Figure 2: Convergence after 100 iterations

[3] [4]

[5]

[6]

ACKNOWLEDGEMENTS

[7]

References

[8]

[1] C. B. Papadias. \A multi-user kurtosis algorithm for blind source separation". IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2000), Istanbul, Turkey, June 5-9, 2000. [2] C. B. Papadias. \Kurtosis-based criteria for adaptive blind source separation". In IEEE International Conference on Acoustics, Speech, and Signal

3rd sub−stream

1

We have proposed a number of variants of the recently presented MUK algorithm. These are based on accelerated convergence CM-type algorithms and inherit the MUK algorithm's globally convergent behavior while offering improved convergence speed. These techniques seem to be appealing for blind source separation in wireless communication systems. The author would like to thank professor L. Tong for his useful feedback and for pointing out the paper [8], as well as Dr. L. Deneire for pointing out the paper [6].

2nd sub−stream

1

[9]

Processing (ICASSP '98), pages 2317{2320, Seattle, WA, USA, May 12-15, 1998. C. B. Papadias \Blind signal separation in narrow band BLAST systems". In CISS'2000 Conference, pp. TP3:7{10, Princeton, USA, March 15-17, 2000. O. Shalvi and E. Weinstein. \New criteria for blind deconvolution of non-minimum phase systems". IEEE Transactions on Information Theory, 36:312{321, March 1990. N. Delfosse and P. Loubaton. \Adaptive separation of independent sources: A de ation approach". In IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '94), vol. IV, pages 41{44, Adelaide, Australia, April 1994. L. Deneire and D. Slock. \A Schur method for multiuser multichannel blind identi cation". In IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '99), Phoenix, AZ, USA, March 15-19, 1999. Y. Li and Z. Ding. \Global convergence of fractionally spaced Godard (CMA) adaptive equalizers". IEEE Trans. on Signal Processing, 44(4):818{826, April 1996. M. Gu and L. Tong. \Geometrical characterizations of constant modulus receivers". IEEE Trans. on Signal Processing, 47(10):2745{2756, Oct. 1999. C. B. Papadias and D. T. M. Slock. \Normalized sliding window constant modulus and decisiondirected algorithms: a link between blind equalization and classical adaptive ltering". IEEE Trans. on Signal Processing, Special Issue on Signal Processing for Advanced Communications, 45(1):231{ 235, Jan. 1997.