AN ITERATIVE INVERSION METHOD FOR BLIND SOURCE SEPARATION Sergio Cruces and Luis Castedo
Andrzej Cichocki
Universities of Seville and La Coru~na Camino de los Descubrimientos 41092-Seville, Spain
[email protected]
Laboratory for Open Information Systems Brain Science Institute, RIKEN Wako-Shi, Saitama 351-01, JAPAN
[email protected]
ABSTRACT
In this paper we present an Iterative Inversion (II) approach to derive the generalized learning rule for Blind Source Separation (BSS) proposed in 10]. The approach consists in the diagonalization of a non-linear spatial correlation matrix of the outputs and is rst presented for instantaneous mixtures. It is shown how existing algorithms for BSS can be derived as particular cases of the resulting generalized rule. The II method is also extended to the separation of convolutive mixtures of temporally i.i.d. sources. Finally, necessary and su cient asymptotic stability conditions for the method to converge are given.
1. INTRODUCTION The separation of mixtures of unknown signals arises in a large number of signal processing applications such as array processing, multiuser communications and voice restoration. This problem is known as Blind Source Separation (BSS) and it has been shown 11] that can be solved if the sources are non-Gaussian and statistically independent. Since the pioneering work of Jutten and Herault 16], a lot of novel, e cient and robust adaptive algorithms for blind source separation have been rigorously derived and their properties have been investigated. Algorithms are developed from a di erent points of view such as contrast functions 11], mutual information maximization 5], Kullback-Leibler minimization using natural gradient approach 3] and nonlinear principal component analysis 17]. In this paper we present a new approach, the Iterative Inversion (II) method, and show that most of the existing algorithms can be obtained as particular cases of this method. The BSS separation problem is typically formulated as follows. Assume that an array of sensors provides a vector of N observed signals xn] = x1n] x2n] xN n]]T that are mixtures of N random processes si n]
i=1 2 N termed sources. The exact probability density function of the sources is unknown: we only assume that they are complex-valued, zero-mean, nonGaussian distributed and statistically independent. In the convolutive mixture case we consider that the sources samples are temporally independent and identically distributed (i.i.d.), and the observations are related to the sources as follows
xn] =
1 X
k=;1
Hk]sn k]
(1)
;
where sn] = s1n] s2n] sN n]]T is the vector of sources and Hn] is the sequence of N N impulse response matrices corresponding to the mixing system. To recover the sources, the observations are processed by a Multiple Input Multiple Output (MIMO) system with memory to produce the outputs
yn] =
1 X
k=;1
Wk]sn k] ;
(2)
where Wn] is the sequence of N N impulse response matrices corresponding to the separating system. We will denote Gn] = Wn] Hn] the matrix impulse response of the mixing and separation system. The aim in BSS is to nd or estimate Wn] such that the separating system retrieves the original sources with some specic indeterminacies. The signal model is considerably simplied when instantaneous mixtures are considered. In this case Hn] = Hn] and Wn] = Wn], and equations (1) and (2) reduce to x = Hs (3) y = Wx (4) (Indices with respect to n are omitted in the instantaneous case to simplify notation). This paper is structured as follows. Section 2 presents the II method for instantaneous mixtures and a new
derivation of the generalized and exible learning algorithm for instantaneous BSS is given. Section 3 shows that many existing algorithms for BSS can be derived as special cases of the II method. Section 4 extends the method to convolutive mixtures. Section 5 presents the conditions to ensure algorithm stability. Section 6 illustrates the performance through computer simulations and Section 7 is devoted to the conclusions.
2. INSTANTANEOUS MIXTURES 2.1. Criterion
According to the Darmois-Skitovich theorem 11, 6], sources are recovered if and only if y is a vector of statistically independent signals. When this occurs, it is apparent that the nonlinear spatial correlation matrix Rfg (W) = Ef (y)gH (y)] (5) will be diagonal. Here f (y) and g(y) represent nonlinear functions of the outputs. Without loss of generality we will consider that this correlation matrix is equal to the identity at the optimum separation matrices W , i.e., Rfg (W ) = I (6) Next, let us dene a new non-linear function of the outputs F(y) = W;1 f (y) and a new spatial correlation matrix RFg (W) = EF(y)gH (y)] = W;1Rfg (W) (7) This allows us to rewrite condition (6) as Rfg (W ) = I W = R;Fg1(W) (8) Therefore, we may interpret that solving the BSS problem is equivalent to inverting the matrix RFg (W ). This inversion, however, cannot be carried out directly since we do not have access to F(y) and g(y) at the optimum separating solution W . Instead, we will develop in the sequel an iterative procedure that overcomes this limitation. Let us denote W(n) the separating matrix used in the nth iteration. According to the above discussion the best approximation to W at the nth iteration will be the inverse of RFg (W(n) ), i.e., W(n+1) = R;Fg1(W(n) ) (9) In order to avoid the computational complexity caused by matrix inversions, we propose that W(n+1) be the solution to the following minimization problem arg min = R (W(n) ) W;1 (n+1) 2 (10) )
W;1 (n+1)
k
Fg
;
k
F
where F denotes the Frobenius norm of a matrix. Let us resort now to a Newton-Raphson recursive technique to solve the minimization problem (10). Following an argumentation similar to the development of the Bussgang techniques for blind equalization 15] and to the continuation method 14], to extend the domain of convergence of the Newton-Rapshon method we will interpret RFg (W(n) ) as a rough estimate of the mixing matrix H at iteration n and, therefore, with a constant value. Computing the gradient and the Hessian of with respect to W;1 (n+1) it is obtained that @ W@ ;H = (RFg (W(n) ) W;1 (n+1)) and @ @ @ W;1 @ W;H = I. Therefore the resulting NewtonRapshon recursion that minimizes is k
k
;
;
W;1 (n+1) = W;1 (n) + (R(Fgn) W;1 (n+1)) where to simplify notation R(Fgn) = RFg (W(n)). Den ;
ing = 1+ we can rewrite the above iteration as
W;1 (n+1) = (1 ) W;1 (n) + R(Fgn) (11) Note from this equation that W;1 (n+1) is an estimate ;
of an exponential windowed version of the spatial correlation matrix, and is in accordance with our interpretation of R(Fgn) as an estimate of the mixing system H. Moreover, it also provides some clues to select the nonlinearities F( ) and g( ) as those that provide a good estimate of H from the outputs vector y. Next, let us describe explicitly the recursion (11) in terms of W(n) rather than W;1 (n)
W(n+1) = I + R(fgn) I ;
;1
W(n) (12)
If (n) is chosen in such a way that 1 (n)
