An Unsupervised Hybrid Network for Blind ... - Semantic Scholar

TO APPEAR IN JOURNAL OF COMMUNICATIONS AND NETWORKS, 1999

1

An Unsupervised Hybrid Network for Blind Separation of Independent Non-Gaussian Source Signals in Multipath Environment Seungjin CHOI, Andrzej CICHOCKI

Abstract | This paper is concerned with the problem of recovering multiple source signals that are transmitted through a linear Multiple Input Multiple Output (MIMO) system, without resorting to any prior knowledge. Source signals are assumed to be spatially independent and temporally i.i.d. non-Gaussian sequences. We present an unsupervised hybrid network (a linear feedback network with FIR synapses followed by a linear memoryless feedforward network) which is able to recover multiple source signals blindly. A simple criterion for multichannel blind deconvolution and an associated learning algorithm are presented. Extensive computer simulation results con rm the validity and high performance of the proposed method. Keywords |Blind signal separation, Hebbian/anti-Hebbian learning, independent component analysis, multichannel blind deconvolution/equalization, neural networks, spatiotemporal decorrelation, unsupervised learning.

M

I. Introduction

ULTIPLE Input Multiple Output (MIMO) models arise frequently in digital communication systems, antenna array processing, and biomedical signal processing where multiple sensors are involved. In multiaccess communications, the received signals at an m element array is an m dimensional vector process with its component representing the mixture of n independent source signals. We assume that n source signals (or user signals) impinge on an antenna array through a multipath channel. The channel between n source signals and m sensor signals can be modeled as a linear MIMO system. Referring to (1), the transfer function Hij (z ) represents the composite eects of channel distortion (such as multipath eect), transmitter, and receiver between the j th source signal and the ith sensor signal. Depending on applications one of the inputs to (1) may be regarded as a desired signal and the others as interferences. Typically, the channel distortion is unknown a priori. Source signals are also unknown in advance. Therefore, it is desirable to recover all source signals from their convolutive mixtures without the knowledge of channel distortion nor source signals. Separating all source signals blindly is useful in combating interuser interference and intersymbol interference. The blind reconstruction of source signals in multipath environment is often called as multichannel blind deconvolution. The term blind means S. Choi is with School of Electrical and Electronics Engineering, ChungBuk National University, KOREA, Email: [email protected] . A. Cichocki is with Lab for Open Information Systems, RIKEN Brain Science Institute, JAPAN. He is on leave from the Warsaw University of Technology, Warsaw, POLAND, Email: [email protected] .

that channel impulse responses and source signals are not known a priori. Consider a discrete-time FIR MIMO system with n inputs and m outputs. The ith sensor signal xi (k) (the received signal at the ith sensor) is given by

xi (k) =

n X j =1

Hij (z )sj (k) + vi (k);

(1)

where sj (k) is the j th source signal and vi (k) is additive white Gaussian noise that is assumed to be statistically independent of source signals and

Hij (z ) =

M X p=0

hij;p z ?p ;

(2)

where z ?p is the delay operator, i.e., z ?psj (k) = sj (k ? p). The vector of m sensor signals is de ned by

x(k) = [x1 (k)


xm (k)]T ;

(3)

and similarly the n dimensional source signal vector s(k) and the m dimensional noise vector v (k) are de ned by

s(k) = [s1(k)


sn (k)]T ; (4) T v(k) = [v1 (k)


vm(k)] : (5) De ne the m  n FIR polynomial matrix H (z ) whose

(i; j )th element is Hij (z ). With this de nition, (1) can be rewritten in a compact form, i.e.,

x(k) = H (z)s(k) + v(k);

(6)

or equivalently

x(k) =

M X p=0

H ps(k ? p) + v(k);

(7)

where H p is the m  n matrix whose (i; j )th component is hij;p . The source signals fsj (k)g are assumed to be spatially independent and temporally i.i.d. non-Gaussian stochastic sequences. The task of multichannel blind deconvolution is to recover the source signal vector s(k) from the observation vector x(k), up to possibly scaled, reordered, and delayed estimates, i.e., s^(k) = P D(z )s(k), where P 2 IRnn is a permutation matrix,  2 IRnn is a nonsingular diagonal matrix, and D(z ) = diagfz ?d1 ;


; z ?dn g.

TO APPEAR IN JOURNAL OF COMMUNICATIONS AND NETWORKS, 1999

Methods for determining sj (k) include both indirect and direct methods. In indirect methods, the channel transfer function H (z ) is identi ed from the received signals fxi (k)g and is used to calculate each sj (k) [1]. Such methods usually process the signals in blocks, which increase memory requirements and system latency. In direct methods, the source signals fsj (k)g are estimated without identifying the channel transfer function H (z ). In this paper, we consider a direct approach to multichannel blind deconvolution task using an unsupervised hybrid network (see Figure 1). Existing solutions in direct methods include successive estimation approaches [2], [3] and the extension of the Godard-family of algorithms [4], [5], [6] that impose additional constraints on the cross-correlations between extracted output signals [7], [8], [9]. These approaches require either relatively expensive computation or a prior knowledge about the number of source signals [7], [8]. Most approaches [8], [10] were restricted to the case where the number of sensors are equal to the number of sources. However, in general, the number of sources are not known in advance. In this paper, we propose a hybrid network which is able to recover source signals without a prior knowledge about the number of source signals. Delfosse and Loubaton [11] pointed out that the multichannel blind deconvolution problem can be decomposed into two sub-tasks which are (1) estimating an innovation sequence vector and (2) separation of instantaneous mixtures, provided that the given channel has a stable inverse. It was explained from the viewpoint of spectral factorization. We give an alternative view and show that spatio-temporal decorrelation can deconvolve MIMO channels up to the instantaneous mixtures of source signals which can be further separated by blind source separation (BSS) or independent component analysis (ICA) [12], [13], [14], [15], [16], [17], [18], [19], [20]. For the case of instantaneous mixtures of source signals, the standard principal component analysis (PCA) [21] or singular value decomposition (SVD) can be used to estimate the number of source signals. This paper is organized as follows. In Section II, we discuss the separability conditions and a criterion for multichannel blind deconvolution. The unsupervised hybrid network is presented in Section II. The associated unsupervised learning algorithms are derived in Sections III and IV. In Section V, several exemplary computer simulation results are provided to con rm the validity of the proposed method. Conclusions are drawn with discussions in Section VI.

2

A. Separability Conditions Let C (z ) be the inverse of MIMO FIR channel H (z ). The global system which combines the channel H (z ) and the separating system C (z ) is denoted by G(z ) = C (z)H (z). The zero-forcing condition for blind deconvolution of MIMO channels is given by

G(z) = P D(z): (8) The channel transfer function H (z ) is said to be signalseparable if the G(z ) has the decomposition of (8). Certainly, every channel H (z ) is not signal-separable.

The separability conditions were investigated by MasseySain [22], Delfosse-Loubaton [11], and Tugnait [2]. The transfer? function H (z) is an m  n polynomial matrix
having m distinct n  n submatrices. Let
i (z ), i = ?n
1; 2;


; mn , denote the determinants of these submatrices. In terms of these quantities, the condition for the existence of an inverse C (z ) was given by Massey and Sain [22]. Theorem 1 (Massey and Sain [22]) An FIR inverse exists, if and only if

GCD[
1 (z );
2 (z );


;
(mn) (z )] = z ?d;

(9)

for some d  0. Note that this condition cannot be satis ed for m = n (equal number of sensors and sources), except for trivial cases. This theorem states that H (z ) has a left inverse which is causal and stable if (9) is satis ed. This exactly implies that H (z ) is minimum phase under this condition. Similar conditions on the channel H (z ) were also discussed in [11], [2]. Suppose that the following assumptions are satis ed: (B1) There are more sensors than sources, i.e., m > n. (B2) H (z ) is causal and rational. (B3) H (z ) is full rank for all z . These hypotheses imply that H (z ) has a rational leftinverse which is causal and stable, which means that H (z ) is minimum phase. Throughout this paper, we consider a class of MIMO channels that is signal-separable. B. Separation Criterion In general, it is impossible to nd an exact inverse of the channel since any knowledge of the channel and source signals is not available in advance. Instead of nding an inverse which has the decomposition (8) at one step, we rst design an system Cb (z ) that satis es the generalized zero-forcing condition given by

Cb (z)H (z) = ?D(z); (10) where ? is an m  n matrix with full-column rank. Let us denote the output signal of the system Cb (z ) by y(k), i.e., In this section, we discuss separability conditions. Then, y(k) = Cb (z )x(k). It will be shown that the generalized a separation criterion for multichannel blind deconvolution zero forcing condition (10) is achieved if fyi (k)g are uncoris introduced. Throughout this paper, we assume that the related in temporal domain as well as in spatial domain. additive noise v (k) is negligible. Suppose that the system Cb (z ) satis es the generalized zero II. A Criterion for Multichannel Blind Deconvolution

TO APPEAR IN JOURNAL OF COMMUNICATIONS AND NETWORKS, 1999

forcing condition (10). Then one can easily see that the signals fyi (k)g are instantaneous mixtures of source signals fsi (k)g. Instantaneous mixtures can be separated by blind source separation or independent component analysis. Theorem 2: Let the channel H (z ) satisfy the condition (9) (or (B1)-(B3)). Suppose that source signals fsi (k)g are spatially independent and temporally i.i.d. sequences. Then the generalized zero-forcing condition (10) is satis ed if y(k) satis es E fy(k)yT (l)g = ?kl ; (11) where kl is Kronecker delta equal to 1 for k = l, otherwise it is 0. The proof is given in Appendix A. The result in Theorem 2 explains that MIMO channels can be deconvolved up to instantaneous mixtures by spatio-temporal decorrelation. We have to mention that the result in Theorem 2 is closely related with Delfosse-Loubaton's [11]. If H (z ) is minimum phase, the normalized innovation sequence e(k) of x(k) is equal to source signal s(k) up to an orthogonal matrix Q which can be identi ed by blind source separation. Multivariate linear prediction method was used to nd an innovation sequence of x(k) [11], [23]. In present paper, however, we do not look for a normalized innovation sequence exactly. The signals fyi (k)g are not required to have unit variance. This avoids the degeneracy of multivariate linear prediction for the overdetermined case (m > n). C. The Unsupervised Hybrid Network The task of spatio-temporal decorrelation and blind source separation is performed by an hybrid network as shown in Figure 1. For spatio-temporal decorrelation, we extend the anti-Hebbian learning rule [24] that was shown to be ecient in spatial decorrelation task. The rst stage of the network is learned by spatio-temporal anti-Hebbian rule (25) and produce the output y(k) whose components are spatio-temporally uncorrelated. The output y(k) is fed into the second stage which is implemented by a linear memoryless feedforward network and is transformed into z(k) whose components are statistically independent. III. Spatio-temporal Decorrelation

We rst brie y review the anti-Hebbian rule [24] that was shown to decorrelate two associated signals without instability problem if the learning rate is small enough. Then we extend the anti-Hebbian rule to spatio-temporal domain and derive the spatio-temporal anti-Hebbian rule from an information-theoretic viewpoint. A. Anti-Hebbian Rule Let us consider a linear memoryless feedback network whose output yi (k) is described by X yi (k) = xi (k) + wij yj (k); (12) j 6=i

where xi (k) is the input to the network and wij is the synaptic connection between yi (k) and yj (k). For spatial

3

decorrelation task, the synaptic weight wij is updated in such a way that cross-correlation between yi (k) and yj (k) is minimized. The anti-Hebbian rule [24] suggested that the synaptic weight at time k + 1, wij (k + 1) is updated using the synaptic weight at time k, wij (k) and signals yi (k); yj (k), i.e., wij (k + 1) = wij (k) ? k yi (k)yj (k); (13) where k > 0 is a learning rate. B. Spatio-temporal Anti-Hebbian Rule For the task of spatio-temporal decorrelation, it is desirable for the network to have capability of temporal processing as well as spatial processing. The FIR synapses are introduced in a linear feedback network (see Figure 1). The output y(k) of the linear feedback network with FIR synapses is described by

y(k) = x(k) +

L X p=1

W p y(k ? p)

= x(k) + W (z )y(k); (14) where fW p 2 IRmm g are synaptic weight matrices and PL W (z) = p=1 W p z?p. One natural choice of optimization function for the minimization of statistical correlations, is the Kullback-Leibler divergence between joint probability density function and the product of marginal density functions. The algorithm is derived for the minimization of statistical correlations and spatio-temporal decorrelation is treated as a special case. Let us consider m observations from sensors over a (N + 1)-point time block and the corresponding m outputs over the same time block, de ned by the following vectors: X = [xT (0)


xT (N )]T ; (15) T T T Y = [y (0)


y (N )] : (16) We consider the following risk function R(fW p g) (expectation of loss function L(fW p g)), R(fW p g) = E fL(fW p g)g Y) g; (17) = N 1+ 1 E flog Qm pN(+1 i=1 qi (yi (k )) where p(Y ) is the joint probability density function of Y and qi (yi (k)) is the marginal density function. Note that the assumption of i.i.d. sources was used in (17). The input-output relation in terms of X and Y is given by X = WY ; (18) where 2 I 0


0 3 6 ?W 1 I


0 77 W = 664 .. (19) .. 75 : . .

?W N ?W N ?1


I

TO APPEAR IN JOURNAL OF COMMUNICATIONS AND NETWORKS, 1999

4

W11 (z) PSfrag replacements

x1(k)

y1(k)



U 11 U n1

Wm1 (z) W1m (z) xm(k)

ym(k)





U 1m U nm



z1(k)

zn(k)

Wmm (z) Fig. 1. The structure of the hybrid network for multichannel blind deconvolution task is shown. It consists of the linear feedback network with FIR synapses followed by the linear memoryless feedforward network.

The length of delay, L in our spatio-temporal decorrelation W p that has the form network is less than N , i.e., W L+1 =


= W N = 0. @L(fW p g) According to relations (18) and (19), one can easily show W p (k + 1) = W p (k ) ? k @ W that the joint density of the observation X and the joint p density of Y has the following relation = W p (k) ? k y(k)y T (k ? p): (25) p(X ) = p(Y ): (20) The algorithm given in (25) is the spatio-temporal antiHebbian rule. The spatio-temporal anti-Hebbian learning Then the loss function L(fW p g) becomes rule was also applied to the task of blind equalization of Single Input Multiple Output (SIMO) channels [27]. m X 1 L(fW p g) = N + 1 log p(X ) ? log qi (yi (k)): (21) IV. Blind Source Separation: Separation of i=1

Note that log p(X ) does not depend on parameters fW p g. The minimization of the loss function (21) can be viewed as an extension of minimum entropy coding (or factorial coding) [25], [26] in spatio-temporal domain. De ne (22) 'i (yi (k)) = ? d logdyqi ((yki)(k)) : i

Instantaneous Mixtures

The task of blind source separation is to recover independent signals (sources) from their instantaneous mixtures without resorting to any prior knowledge. After the outputs fyi (k)g are successfully decorrelated in spatiotemporal domain, fyi (k)g are instantaneous mixtures of sources, fsi (k)g. Blind source separation nds a linear mapping that transforms y(k) to z(k) whose elements are statistically independent. The linear mapping is implemented by a linear feedforward network (in the second With this de nition, the gradient of the loss function is stage). The output of the linear feedforward network is given by given by @L(fW p g) = '(y (k))y T (k ? p); z(k) = Uy(k); (26) (23)

@W p

where U 2 IRnm is called a \demixing matrix". Here where '(y (k)) 2 IRm is an element-wise function, the ith we assume that the number of source signals are already element of which is given by (22). For the purpose of estimated by PCA [21]. spatio-temporal decorrelation, Gaussian density model for Typical learning algorithm for blind source separation qi (yi (k)) might be useful, i.e., has the form [14], [16], [18] U (k + 1) = U (k) + k fI ? f (z(k))zT (k)gU (k); (27) (24) qi (yi (k)) = p1 e? 21 yi2 (k) : 2 where f (z (k)) = [f1 (z1 (k))


fn (zn (k)]T is a properly choOne can easily see that 'i (yi (k)) = yi (k) with Gaussian sen nonlinear function. The choice of this function depends density model. With Gaussian density model, stochastic on the probability distributions of source signals. For exgradient descent method results in the updating rule for ample, it is known that fi (zi (k)) = jzi (k)j2 zi (k) is a good

TO APPEAR IN JOURNAL OF COMMUNICATIONS AND NETWORKS, 1999

choice for sub-Gaussian source signals (negative kurtosis) and fi (zi (k)) = tanh(zi (k)) is a good selection for superGaussian source signals (positive kurtosis). Since we are dealing with communication signals (which are sub-Gaussian), we choose fi (zi (k)) = jzi (k)j2 zi (k). The cubic nonlinear function has been shown to be ecient for sub-Gaussian signals [17] or communication signals [28]. We will not go into further details on blind source separation, because it is not the main contribution of this paper. V. Computer Simulations

Two exemplary computer simulations results are presented here. The rst simulation was conducted with binary source signals. Complex-valued source signals were assumed in the second simulation. A. Simulation 1 In this simulation, we have used binary pulse-amplitudemodulated (PAM) signals. Three i.i.d. sources consist of random variables that are uniformly distributed over the binary set f?1; +1g. Five sensor signals were generated from a multivariate FIR lter, i.e.,

5

trained by the algorithm (25) in an unsupervised manner and the second stage was learned by the algorithm (27). Synaptic weight matrices fW p g for p = 1; : : : ; L were initialized as a zero matrix. The delay length was L = 15. The initial condition for U (k) was randomly generated. We have used a constant learning rate k = :001. We have checked singular values of the covariance matrix E fy(k)yT (k)g to estimate the number of source signals, after fW p g converged. In descending order, the singular values were 2.39, 1.1, .99, .02, and .004. One can easily decide that there are three source signals. The output signals fzi (k)g are shown in Figure 2. It can be observed that three source signals are well separated after 25000 iterations. B. Simulation 2 In this test, we have used the same channels as used in Simulation 1. As unknown sources, 4 quadratureamplitude-modulated (QAM) signals were used. For 4 QAM signals, each source belongs to one of four complex values given by f1 + j; ?1 + j; ?1 ? j; 1 ? j g. Through the channels in Simulation 1, ve complex sensor signals were generated. One exemplary mixture in the complex plane is shown in Figure 3. The other 4 mixtures are very similar to one as shown in Figure 3. The initial conditions for the network, the delay length L, the learning rate were identical to those in Simulation 1. Three recovered signals for k 2 [45000; 50000] are shown in Figure 4.

x(k) = H 0s(k) + H 1s(k ? 1) + H 2 s(k ? 2) + H 5 s(k ? 5) + H 10 s(k ? 10); (28) where fH p g 2 IR53 for p = 0; 1; 2; 5; 10 are given by 2 3 ?0:8244 ?0:5281 0:2690 6 0:2790 ?0:4718 0:9545 7 6 7 H 0 = 66 ?0:8027 0:2088 0:6510 77 ; 4 0:3813 ?0:1638 ?0:4125 5 ?0:3169 ?0:7274 ?0:2183 2 3 ?0:6839 ?0:2140 0:7871 6 0:6521 ?0:2039 ?0:2467 7 6 7 H 1 = 66 0:0446 ?0:3388 0:3113 77 ; 4 0:8187 0:1666 0:6545 5 ?0:0648 ?0:2153 0:7130 2 3 0:2744 ?0:0576 0:2720 Fig. 3. An exemplary pattern of complex-valued sensor signal. 6 0:0314 0:7132 ?0:6183 77 6 6 7 H 2 = 6 0:2965 ?0:0549 ?0:0654 7 ; 4 ?0:7559 ?0:7838 ?0:5693 5 VI. Conclusion ?0:6820 0:3525 ?0:2852 We have presented a new unsupervised hybrid network 2 3 0:3381 ?0:5663 0:1081 model for multichannel blind deconvolution task. The hy6 0:9698 ?0:2780 0:1904 7 brid network consists of a linear feedback network with 6 7 H 5 = 66 ?0:0709 0:1803 0:3893 77 ; FIR synapses (performing spatio-temporal decorrelation) 4 ?0:1861 ?0:5072 ?0:9113 5 followed by a linear memoryless feedforward network (per0:8387 ?0:7378 0:3675 forming blind source separation). The proposed approach 2 3 employs very simple learning algorithms. This paper has ?0:0577 ?0:0087 0:3402 6 0:2289 two main contributions: (1) it was shown that multichan0:2103 ?0:3030 77 6 6 7 nel blind deconvolution task can be decomposed into two H 10 = 6 ?0:1541 ?0:1671 ?0:0578 7 : sub-tasks such as spatio-temporal decorrelation and blind 4 0:0683 ?0:1201 0:1050 5 source separation; (2) we derived a new spatio-temporal ?0:4733 0:4854 0:0988 4 3 2

Im[x1]

1 0

−1 −2 −3

−4 −4

−3

−2

−1

0

1

2

3

4

Re[x1]

decorrelation algorithm rigorously from an informationThey were assumed to be completely unknown in advance. theoretic approach. Computer simulation experiments We have used the unsupervised network as shown in Fig- demonstrated validity and high performance of our proure 1 in this simulation. The rst stage in the network was posed approach.

6 4

3

3

2

2

2

1

1

1

0

z

0

3

4

3

2

4

z

z

1

TO APPEAR IN JOURNAL OF COMMUNICATIONS AND NETWORKS, 1999

0

−1

−1

−1

−2

−2

−2

−3

−3

−3

−4 0

−4 0

1

2

3

4

5

number of iterations

1

2

3

4

5

number of iterations

4

x 10

−4 0

1

2

3

4

5

number of iterations

4

x 10

4

x 10

2

2

1.5

1.5

1

1

1

0.5

0.5

0.5

0

−0.5

Im[z3]

2 1.5

Im[z2]

Im[z1]

Fig. 2. Three recovered signals, z1 (k), z2 (k), and z3 (k) in Simulation 1.

0

−0.5

0

−0.5

−1

−1

−1

−1.5

−1.5

−1.5

−2 −2

−1.5

−1

−0.5

0

0.5

1

1.5

2

−2 −2

−1.5

−1

−0.5

Re[z1]

0

0.5

1

1.5

2

−2 −2

−1.5

−1

−0.5

Re[z2]

0

0.5

1

1.5

2

Re[z3]

Fig. 4. Three recovered signals, z1 (k), z2 (k), and z3 (k) in Simulation 2.

Acknowledgements

We gratefully acknowledge the helpful comments of the [1] anonymous reviewers. We also acknowledge many helpful early discussions of this research with Prof. S. Amari at Brain Science Institute in RIKEN, JAPAN and Prof. R. [2] Liu at University of Notre Dame in Indiana. This work was supported by grant No. 981-0913-063-1 from the Basic [3] Research Program of the KOSEF in KOREA. Appendix A

[4]

In this section, the proof of Theorem 2 is provided. Since both channel and its inverse are causal and stable, the [5] global system G(z ) is also causal and stable. Then

y(k) =

1 X i=0

Gi s(k ? i):

(29)

Suppose that Gd is the leading nonsingular coecient matrix, i.e., G0 =


= Gd?1 = 0. Then,

y(k) =

1 X i=d

Gi s(k ? i):

(30)

Invoking E fy(k)yT (k +  )g = 0 for 8 6= 0 we have 1 X i=d

Gi E fs(k ? i)sT (k ? i)gGTi+ = 0; 8 6= 0: (31)

[6] [7] [8] [9] [10]

Since E fs(k ? i)sT (k ? i)g for 8i and Gd are nonsingular [11] matrices, the condition (11) implies that Gi = 0 for i = d +1; : : : ; 1. Therefore, y(k) = Gd s(k ? d) if the condition [12] (11) is satis ed.

References L. Tong, G. Xu, and T. Kailath, \Blind identi cation and equalization based on second-order statistics: A time domain approach," IEEE Trans. Information Theory, vol. 40, pp. 340{ 349, 1994. J. Tugnait, \Identi cation and deconvolution of multichannel linear non-gaussian processes using higher order statistics and inverse lter criteria," IEEE Trans. Signal Processing, vol. 45, pp. 658{672, 1997. S. Choi and A. Cichocki, \Cascade neural networks for multichannel blind deconvolution," Electronics Letters, vol. 34, no. 12, pp. 1186{1187, 1998. D. N. Godard, \Self-recovering equalization and carrier tracking in two-dimensional data communication systems," IEEE Trans. Communications, vol. 28, pp. 1867{1875, 1980. J. R. Treichler and B. G. Agee, \A new approach to multipath correction of constant modulus signals," IEEE Trans. Acoustics, Speech, and Signal Processing, vol. 31, pp. 459{472, 1983. O. Shalvi and E. Weinstein, \New criteria for blind deconvolution of nonminimum phase systems," IEEE Trans. Information Theory, vol. 36, pp. 312{321, 1990. C. B. Papadias and A. J. Paulraj, \A constant modulus algorithm for multiuser signal separation in the presence of delay spread using antenna arrays," IEEE Signal Processing Letters, vol. 4, no. 6, pp. 178{181, 1997. Y. Inouye, \Blind deconvolution of multichannel linear timeinvariant systems of nonminimum phase," in Statistical Methods in Control and Signal Processing, T. Katayama and S. Sugimoto, Eds., pp. 375{397. Marcel Dekker, 1997. A. Touzni, I. Fijalkow, M. G. Larimore, and J. R. Treichler, \A globally convergent approach for blind MIMO adaptive deconvolution," in Proc. ICASSP, 1998, pp. 2385{2388. S. Amari, S. C. Douglas, A. Cichocki, and H. H. Yang, \Multichannel blind deconvolution and equalization using the natural gradient," in The First Signal Processing Workshop on Signal Processing Advances in Wireless Communications, Paris, France, 1997, pp. 101{104. N. Delfosse and P. Loubaton, \Adaptive blind separation of convolutive mixtures," in Proc. ICASSP, 1996, pp. 2940{2943. C. Jutten and J. Herault, \Blind separation of sources, part i: An adaptive algorithm based on neuromimetic architecture," Signal Processing, vol. 24, pp. 1{10, 1991.

TO APPEAR IN JOURNAL OF COMMUNICATIONS AND NETWORKS, 1999

[13] P. Comon, \Independent component analysis, a new concept?," Signal Processing, vol. 36, no. 3, pp. 287{314, 1994. [14] A. Cichocki, R. Unbehauen, and E. Rummert, \Robust learning algorithm for blind separation of signals," Electronics Letters, vol. 43, no. 17, pp. 1386{1387, 1994. [15] A. Bell and T. Sejnowski, \An information maximisation approach to blind separation and blind deconvolution," Neural Computation, vol. 7, pp. 1129{1159, 1995. [16] S. Amari, A. Cichocki, and H. H. Yang, \A new learning algorithm for blind signal separation," in Advances in Neural Information Processing Systems, D. S. Touretzky, M. C. Mozer, and M. E. Hasselmo, Eds. 1996, vol. 8, pp. 757{763, MIT press. [17] J. F. Cardoso and B. H. Laheld, \Equivariant adaptive source separation," IEEE Trans. Signal Processing, vol. 44, pp. 3017{ 3030, 1996. [18] A. Cichocki and R. Unbehauen, \Robust neural networks with on-line learning for blind identi cation and blind separation of sources," IEEE Trans. Circuits and Systems - I: Fundamental Theory and Applications, vol. 43, pp. 894{906, 1996. [19] S. Choi, A. Cichocki, and S. Amari, \Flexible independent component analysis," in IEEE Workshop on Neural Networks for Signal Processing, Cambridge, UK, 1998, pp. 83{92. [20] S. Choi, \Adaptive blind signal separation for multiuser communications: An information-theoretic approach," Journal of Electrical Engineering and Information Science, 1999, in press. [21] E. Oja, \Neural networks, principal component analysis, and subspaces," International Journal of Neural Systems, vol. 1, pp. 61{68, 1989. [22] J. Massey and M. Sain, \Inverses of linear sequential circuits," IEEE Trans. Computers, vol. 17, pp. 330{337, 1968. [23] S. Icart and R. Gautier, \Blind separation of convolutive mixtures using second and fourth order moments," in Proc. ICASSP, 1996, pp. 3018{3021. [24] P. Foldiak, \Adaptive network for optimal linear feature extraction," in International Joint Conference on Neural Networks, 1989, pp. 401{405. [25] H. B. Barlow, \Unsupervised learning," Neural Computation, vol. 1, pp. 295{311, 1989. [26] J. P. Nadal and N. Parga, \Nonlinear neurons in the low-noise limit: A factorial code maximize information transfer," Network: Computation in Neural Systems, vol. 5, pp. 565{581, 1994. [27] S. Choi, A. Cichocki, and S. Amari, \Blind equalization of SIMO channels via spatio-temporal anti-Hebbian learning rule," in IEEE Workshop on Neural Networks for Signal Processing, Cambridge, UK, 1998, pp. 93{102. [28] S. Choi, \Linear neural networks with FIR synapses for blind deconvolution and equalization," Journal of Electrical Engineering and Information Science, 1999, in press.

Seungjin CHOI was born in Seoul, Korea,

on October 26, 1964. He received the B.S. and M.S. degrees in electrical engineering from Seoul National University, Korea, in 1987 and 1989, respectively and the Ph.D degree in electrical engineering from the University of Notre Dame, Indiana, in 1996. After he spent the fall of 1996 as a Visiting Assistant Professor in the Department of Electrical Engineering at University of Notre Dame, Indiana, he was with the Laboratory for Arti cial Brain Systems, RIEKN in Japan. In August 1997, he joined the School of Electrical and Electronics Engineering at Chungbuk National University where he is a Full-time Lecturer now. He also has been an Invited Research Fellow at Brain-style Information Systems Research Group in Brain Science Institute, RIKEN in Japan. His current research interests include brain information processing, statistical (blind) signal processing, independent component analysis, multiuser communications, and unsupervised learning.

7

Andrzej CICHOCKI was born in Poland.

He received the M.Sc.(with honors), Ph.D., and Habilitate Doctorate (Dr.Sc.) degrees, all in electrical engineering and computer science, from Warsaw University of Technology (Poland) in 1972, 1975, and 1982, respectively. Since 1972, he has been with the Institute of Theory of Electrical Engineering and Electrical Measurements at the Warsaw University of Technology, where he became a full Professor in 1995. He is the co-author of two international books: MOS SwitchedCapacitor and Continuous-Time Integrated Circuits and Systems (Springer-Verlag, 1989) and Neural Networks for Optimization and Signal Processing (J Wiley and Teubner Verlag,1993/94) and author or co-author of more than hundred fty (150) scienti c papers. He spent at University Erlangen-Nuernberg (GERMANY) a few years as Alexander Humboldt Research Fellow and Guest Professor. In 1995-96 he has been working as a Team Leader of the Laboratory for Arti cial Brain Systems, at the Frontier Research Program RIKEN (JAPAN), in the Brain Information Processing Group directed by professor Shun-ichi Amari. Currently he is head of the laboratory for Open Information Systems in the Brain Science Institute, Riken, Wako-schi, JAPAN. He is reviewer of several international Journals, e.g. IEEE Trans. on Neural Networks, Signal Processing, Circuits and Systems, Biological Cybernetics, Electronics Letters, Neurocomputing, Neural Computation. He is also member of several international Scienti c Committees and the associated Editor of IEEE Transaction on Neural Networks (since January 1998). His current research interests include signal and image processing (especially blind signal/image processing), neural networks and their electronic implementations, learning theory and algorithms, independent and principal component analysis, optimization problems, circuits and systems theory and their applications, arti cial intelligence.