Blind Separation of Cyclo-stationary Signals using Generalized Eigenvalue Approach Tarik Aoufi1 , M’hamed Bakrim1,2 , Driss Aboutajdine1 {1} GSCM-LRIT, Faculté des Sciences, av. Ibn Battouta, BP 1014, Rabat, Maroc {2} LP2M2E, Faculté des Sciences et Techniques, Marrakech, Maroc E-mail:
[email protected],
[email protected],
[email protected]
Abstract— This communication addresses the problem of the blind separation of cyclo-stationary source signals in the case of an instantaneous linear mixture. We show that the separation can be realized by the generalized eigenvectors that simultaneously diagonalize the cyclic correlation matrices of the observation signals. The mixture matrix is estimated by the eigenvectors of the pencil matrix constructed with two different cyclic correlation matrices of the observation signals. We also discuss the extension of some existing second-order blind source separation methods. The main advantages of the proposed method are that a pre-whitening stage is dropped, the cyclic frequencies are unknown and it is consistent in noisy case. Numerical simulation results are presented to illustrate the performance of the proposed method in digital communications context.
Index Terms— Blind sources separation, instantaneous linear mixture, simultaneous diagonalization, generalized eigenvalue problem, cyclo-stationary signals.
cyclic frequencies and in the second step, we use a matrix pencil formed by output cyclic correlation matrices at different cyclic frequencies to solve BSS problem. We also show that it is possible to solve the blind source separation problem by the non unitary simultaneous diagonalization by using the algorithm proposed in [13]. The rest of this paper is organized as follows. The problem statement, assumptions and definitions are presented in section II. In the section III, we present the existing methods based on second order statistic and related to our proposed methods. In section IV we describe the proposed methods. Finaly, numerical simulations are provided in the section V to illustrate the effectiveness of the proposed methods. II. DATA MODEL , A SSUMPTIONS AND D EFINITIONS
I. INTRODUCTION Blind source separation (BSS) is a fundamental problem encountered in many practical applications such as telecommunications, radar, sonar, speech processing, biomedical, mechanics, image processing, etc... The aim of the BSS is to recover the unobserved input signals called sources from their observed unknown mixtures. In the signal processing community, many solutions have been proposed to solve this problem and several of them are based on second-order statistics (SOS) because they are sufficient to achieve a successful source separation. In the literature, the well-known approaches are AMUSE [12] and SOBI [3] algorithms applied to stationary sources, and pencil approach [4] [5] [6] exploiting the non-stationarity and temporal structure of sources to estimate the mixing matrix (or the demixing matrix) in the presence of spatially correlated but temporally white noise (not necessarily Gaussian). Such a problem has been already considered in [1] [2] [10] [11] for the cyclo-stationary signals in order to solve the interferences problem in digital communication. The theory of linear and quadratic processing of cyclostationary signal has been first introduced in [8]. In this paper we present a novel method which exploit the particular structure of cyclic correlation matrix of sources to estimate the mixing matrix in the presence of stationary white noise. Also, we show that the blind sources separation can be reformulated as a generalized eigenvalue problem. Therefore, the mixture matrix will be estimated by the eigenvectors of the pencil matrix constructed by two different cyclic correlation matrices of the observation signals. When the cyclic frequencies are unknown, we use the estimator proposed in [7] to estimate them. Our method consists of estimating in the first time the
In the simple form, the BSS problem can be formulated as follows : several linear instantaneous mixture of n source signals emitted are received by m sensors. The input/output relationship of the mixing system is given by : x(t) = As(t) + b(t)
(1)
where A, s(t), x(t) and b(t) represent the m×n mixing matrix, the n × 1 source signals vector, the m × 1 observation signals vector and the m × 1 noise signals vector respectively. Our proposed methods are based on the following assumptions : 1) The components of source signals are unobservable, statistically independent, zero-mean and cyclostationary. Thus, their autocorrelation functions Rsi (t, τ ) = E{si (t)s∗i (t− τ )} are Ti -periodic for all i = 1, ..., n. E{.} stands expectation operator and Ti stands the cyclic-period of the i-th source signal si (t). The cyclic correlation coefficient function of si (t) at the cyclic frequency α (α ∈ { Tki , k ∈ Z}) is defined as 1 Ti →∞ Ti
Rs(α) (τ ) = lim i
Z T /2 i
Rsi (t, τ ) exp(−2ıπαt)dt −Ti /2
(2) √ where ı = −1. 2) A is a full rank m × n matrix ( the number of sensors is bigger than the number of sources) so A is invertible. 3) The components of noise signals vector b(t) are stationary white zero-mean random signals, mutually uncorrelated and independent from the source signals. The correlation function of bj (t) for all j = 1, ..., m is Rbj (τ ) = σb2j δ(τ )
(3)
where σb2j stands the variance of noise signal bj (t) and δ(τ ) is the dirac function. We show that the cyclic correlation coefficient function of bj (t) at cyclic frequency
α is written as
where D is diagonal matrix.
(α)
Rbj (τ ) = Rbj (τ ) lim
Tj →∞
1 Tj
Z T /2 j
exp(−2ıπαt)dt −Tj /2
(4) Hence, if α 6= 0 then the correlation coefficient function of bj (t) is null. 4) The cyclic frequencies of source signals are all distinct (∀i, j = 1, ..., n, i 6= j, T1i 6= T1j ).
ˆ of A, The overall objective of BSS is to obtain an estimate A up to the standard BSS indeterminacies on ordering, scale and ˆ is known, the sources are estimated by : phase. Once A ˆ♯ x(t) = P Ds(t) sˆ(t) = A
(5)
Where (.)♯ denotes pseudo-inverse matrix, P is a permutation matrix (corresponding to an arbitrary order of restitution of the sources) and D is a diagonal matrix (corresponding to arbitrary attenuations for the restored sources). Thus the condition of separation can be put in the following form : Aˆ♯ A = P D
(6)
III. SOS- BASED SOURCE SEPARATION In this section, we briefly present the existing SOS based methods to solve the BSS problem having a link with the proposed methods. A. AMUSE method The method AMUSE (Algorithm for Multiple Unknown Signals Extraction) introduced by Tong in [12], is realized by the diagonalization of two symmetric matrices M0 and M1 , which one, say M0 is invertible. This method requires a whitening stage. Thus, the orthogonal matrix V is estimated in order to check : V M0 V T = I and V M1 V T = D (7) where I is the identity matrix and D is a diagonal matrix. The matrix V is obtained by V = U B with BM1 B T = U T D1 U , B and U are respectively the whitening and orthogonal matrices. We note that the matrices M0 and M1 are the correlation matrices.
Theorem 1 : Let Λ1 , D1 ∈ Cn×n be diagonal matrices with positive diagonal entries and Λ2 , D2 ∈ Cn×n be diagonal matrices with nonzero diagonal entries. We assume that G satisfies the following decompositions : D1 = GΛ1 GH , D2 = GΛ2 GH where (.)
H
stands the complex conjugate, transpose.
Then the matrix G is the generalized permutation matrix, i.e., G = P D (P is a permutation matrix and D is a diagonal matrix) if D1 D2−1 and Λ1 Λ−1 have distinct diagonal entries. 2 Proof : see [6] for proof. The linear transformation G which satisfies (10) is the eigenvector matrix of Λ1 Λ−1 2 . In other words, the matrix G is the generalized eigenvector matrix of the matrix Λ2 − λΛ1 called pencil matrix, where λ is the generalized eigenvalue vector. Among the SOS-based source separation methods, a matrix pencil method [4] employs only two time-delayed correlation matrices to estimate the mixing matrix in the presence of temporally white noise. In this method, the choice of time delays is very important in order to estimate the mixing matrix A. In the next section, we propose to use the cyclic correlation coefficient because it is insensitive to noise (for all time delay, the cyclic correlation coefficient of noise is null if the cyclic frequency is different to zero). IV. P ROPOSED METHODS A. principle The cyclic correlation coefficient matrix of observation signals vector is given by : (α)
Rx(α) (τ ) = ARs(α) (τ )AH + Rb (τ )
(11)
If α 6= 0 (see the assumption (3)) then
Rx(α) (τ ) = ARs(α) (τ )AH
B. SOBI method
(10)
(12) H
The SOBI method [3] extends AMUSE to several matrices. This method requires two steps (whitening followed by an orthogonal transformation) : −1 – First, the whitening matrix is estimated by B = D 2 U T where D and U are respectively the eigenvalue and eigenvector matrices of the Rx (0) (Rx is the correlation matrix of observation signals) Rx (0) = U DU T
(8)
– Second, the orthogonal matrix V = BA is estimated by the orthogonal simultaneous diagonalization of the correlation matrices at different time delays.
Our aim is to find the matrix W such as W A = I. We pose M1 = Rx(αi ) (0) = ARs(αi ) (0)AH , ∀i = 1, ..., n and M2 =
n X i=1
C. Generalized eigenvalue problem
M1 x = λM2 x (or M1 V = M2 V D)
(9)
n X
Rs(αi ) (0)AH
(14)
i=1
where P the αi represents the cyclic frequency of the source signal (αi ) (α ) si (t), n (0) is diagonal invertible matrix and Rs i (0) i=1 Rs is diagonal matrix, thus we constitute a generalized eigenvalue problem as follows ; we multiply the equations (13) and (14) with W M1 W
Let consider two n × n matrices M1 and M2 , the generalized eigenvalue problem consist of finding the scalar λ and the vector x such as x 6= 0 (or to find the matrix V ) and :
Rx(αi ) (0) = A
(13)
M2 W
=
ARs(αi ) (0)AH W
=
ARs(αi ) (0)
= =
A A
n X i=1 n X i=1
(15)
Rs(αi ) (0)AH W
(16)
Rs(αi ) (0)
(17)
n
P
(α )
n i By replacing A = M2 W (0) i=1 Rs we obtain M1 W = M2 W Λ
n
where Λ =
Pn
(αi )
i=1 Rs
o−1
(0)
(αi )
Rs
o−1
in (15) ; (18)
(0).
Since Λ is a diagonal matrix then the equation (18) represents the generalized eigenvalue problem described in section III. The matrix of the form M2 − λM1 (λ ∈ R) is said to be a pencil. This property is frequently encountered when both M1 and M2 are hermitian and invertible. Pencils of this kind of matrices are referred to as hermitian-definite pencils [9]. Theorem 2 (pp. 468 in [9] : If M2 − λM1 is hermitian-definite, then there exists a nonsingular matrix U such as U H M1 U
=
D1
(19)
U H M2 U
=
D2
(20)
where D1 and D2 are diagonal matrices. For the requirement of hermitian matrices , we can use o 1n M1 + M1H (21) 2n o 1 M2 = M2 + M2H (22) 2 Hence the matrix W is the generalized eigenvector matrix of the pencil M2 − λM1 .
M1
=
When the cyclic frequencies are unknown, we estimate them by maximizing the following criteria [7] : J(α) =
2 (α) Rx (τ )
τm X
1) Estimate the cyclic correlation coefficient of observation signals, 2) Estimate the cyclic frequencies by the procedure presented in the precedent subsection, 3) Calculate the set matrices cyclic n M for each estimated o (α ) (α ) frequency αi : M = Rx i (0), ..., Rx i (τmax ) , 4) Diagonalize simultaneously the set matrices M by the algorithm proposed in [13], 5) Use the demixing matrix to estimate source signals.
V. C OMPUTER S IMULATED EXPERIMENTS We present simulations to illustrate the effectiveness of the proposed methods in the BSS context and to compare them with the SOBI method. In fact, the SOBI method needs a pre-whitening stage. Thus we apply the simultaneous diagonalization of the whitening correlation matrix of observation signals. Our methods, Simultaneous Diagonalization of the set Matrices (SDM) based on non-unitary simultaneous diagonalization and the method based on pencil approach (PM) are directly applied to the cyclic correlation coefficient matrices of the observations. We consider m = 2 mixtures of n = 2 digital communication source signals defined as X a(k)h(t − kT ) (24) s(t) = k∈Z
where a(t) is an i.i.d. zero-mean complex random binary sequence, T represents a period symbol and h(t) is a deterministic waveform signal. The cyclic frequency of the first source signal (resp. the second source signal) is 14 (resp. 16 ). These sources are mixed by the following mixture matrix :
A= (23)
τ =−τm
J(α) = 0 if α 6∈ {−αi , 0, αi }, i = 1, ..., n (αi is the cyclic frequency of si (t)) else J(α) > 0, thus the cyclic frequency αi is estimated by αi = argmax]0, 1 [ {J(α)}
1 0.5
0.25 , 1
Firstly, we estimate the cyclic frequencies of the source signals from their mixtures in the case SN R = 1 dB and SN R = 30 dB, using the estimator presented in the section IV. The results are presented in Fig. 1.
2
B. Summary of the proposed methods The first proposed method based on matrix pencil approach can by summarized in the following five points : 1) Estimate the cyclic correlation coefficient of observation signals, 2) Estimate the cyclic frequencies by the procedure presented in the precedent subsection, 3) Calculate the two matrices defined in (21) and (22), 4) Find the generalized eigenvectors matrix W which satisfies the equation (18) 5) Estimate source signals by sˆ(t) = W H x(t).
Secondly, for the proposed method using non unitary simultaneous diagonalization, we estimate the set matrices of cyclic correlation coefficient matrices at different estimated cyclic frequencies and for all time delays τ ∈ {0, · · · , 10}, but for the first proposed, we use only M1 (21) and M2 (22) to estimate the mixing matrix. In this case, the demixing matrix W use the matrix pencil method (18). In order to measure the performance of our two proposed methods, we use the performance index (PI) [14] defined by 2
PI
=
1 n(n − 1) 0
To solve the BSS problem, we also propose to diagonalize simultaneously the set matrices of cyclic correlation coefficient at different time delays and for all cyclic frequencies. In this respect, we use the non unitary simultaneous diagonalization algorithm proposed in [13]. We summarize this second proposed approach as follows
+
n X i=1
n X
k=1
0 n X
n X
i=1
k=1
4
1
|gik | − 1A max |gij | j 13
|gki | − 1A5 max |gji | j
where gij is the (i, j)-element of the global system matrix G = W H A. The term maxj gij represents the maximum value among the elements in the i − th row vector of G. Whereas the term maxj gji designate the maximum value among the elements in the i − th column vector of G. When the perfect
SNR=1 dB
diagonalization to set cyclic correlation coefficient matrices at different time delays. Numerical experiments show that our two methods are less sensitive to noise and more efficient than SOBI method based on whitening stage.
35
30
25
R EFERENCES
20
15
10
5
0 −0.1
0
0.1
0.2 0.3 Estimated frequencies
0.4
0.5
0.6
0.4
0.5
0.6
SNR=30 dB 35
30
25
20
15
10
5
0 −0.1
0
0.1
0.2 0.3 Estimated frequencies
Figure 1. below : The estimated cyclic frequencies for SN R = 1 dB, down : The estimated cyclic frequencies for SN R = 30 dB
separation is achieved, the performance index is zero. In practice, a performance index around 10−2 indicates quite a good performance. On the Fig. 2, the obtained performance index PI is displayed versus the SNR. The result show that our proposed SDM and PM methods have better performances than the SOBI method. 0.013 SOBI PM SDM
0.0115 0.0105 0.0095
Performance index
0.0085 0.0075 0.0065 0.0055 0.0045 0.0035 0.0025 0.0015 0.0005 0
1
6
11
16
21
26
31
36
41
46 51 56 SNR (dB)
61
66
71
76
81
86
91
96 100
Figure 2. P I versus the SN R
VI. C ONCLUSION In this paper, we have proposed a new approach to blind cyclostationary sources separation in instantaneous mixtures context by considering the generalized eigenvalue problem. We have also proposed to use the non unitary simultaneous
[1] A. Ferréol, P. Chevalier and L. Albera, “Second-order blind separation of first- and second-order cyclostationary sources Application to AM, FSK, CPFSK and deterministic sources”, IEEE Transactions on Signal Processing, vol. 52, no. 4, pp. 845–861, April 2004. [2] A. Ferréol, P. Chevalier and L. Albera, “On the behavior of current second and higher order blind source separation methods for cyclostationary sources - Application to AM, FSK, CPFSK and deterministic sources”, IEEE Transactions on Signal Processing, vol. 48, no. 6, pp. 1712–1725, June 2000. [3] A. Belouchrani, K. Abed-Merain, J. -F. Cardoso, and E. Moulines. A “blind source separation technique using second order statistics”. IEEE Trans. Signal Processing, 45 :434–444, Feb. 1997. [4] C. Chang, Z. Ding, S. F. Yau, and F. H. Y. “Chan. A matrixpencil approach to blind separation of colored nonstationary signals”. IEEE Trans. Signal Processing, 48(3) :900–907, Mar. 2000. [5] S. Choi, A. Cichocki, and A. Belouchrani. “Blind separation of second-order nonstationary and temporally colored sources”. In Proc. IEEE Workshop on Statistical Signal Processing, pages 444–447, Singapore, 2001. [6] S. Choi, A. Cichocki, and A. Belouchrani. “Second order nonstationary source separation”. Journal of VLSI Signal Processing, 32 :93–104, 2002. [7] A.V. Dandawaté and G.B. Giannakis. “Statistical tests for presence of cyclostationarity”. IEEE trans signal processing, 42 :2355–2369, Sept. 1995. [8] W. A. Gardner, “Cyclostationarity in communications and signal processing”, IEEE press, W. A. Gardner Editor, 1993. [9] G. H. Golub and C. F. Van Loan. “Matrix Computations”, 2nd edition. Johns Hopkins, 1993. [10] P. A. Jallon, A. Chevreuil, P. Loubaton, P. Chevalier, “Separation of convolutive mixtures of cyclostationary sources : a contrast function based approach”, IEEE Transactions on Signal Processing, vol. 45, pp. 434–444, February 1997. [11] S. Rhioui, N. Thirion-Moreau, and E. Moreau, “Underdetermined blind identification of cyclo-stationary signals with unknown cyclic frequencies”, in Proc. IEEE International Conference on Acoustic Speech and Signal Processing (ICASSP’2006), Toulouse, France, May 2006 [12] L. Tong, V. C. Soon, Y. F. Huang, and R. Liu. “AMUSE : a new blind identification alogrithm”. In Proc. ISCAS, pages 1784–1787, 1990. [13] A. Yeredor, A. Ziehe and K.R. Müller, “Approximate joint diagonalization using a natural gradient approach”, ICA 2004, LNCS 3195, pp. 89–96, 2004. [14] S. Amari, A. Cichocki, and H. Yang, “A new algorithm for blind signal separation”, Advances in Neural Information Processing Systems, 8 :757–763, 1996.
