Turbo Source Extraction Algorithm and Noncancellation ... - IEEE Xplore

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 54, NO. 8, AUGUST 2006

2929

Turbo Source Extraction Algorithm and Noncancellation Source Separation Algorithms by Kurtosis Maximization Chong-Yung Chi, Senior Member, IEEE, and Chun-Hsien Peng

Abstract—The kurtosis maximization criterion has been effectively used for blind spatial extraction of one source from an instantaneous mixture of multiple non-Gaussian sources, such as the kurtosis maximization algorithm by Ding and Nguyen, and the fast kurtosis maximization algorithm (FKMA) by Chi and Chen. By empirical studies, we found that the smaller the normalized kurtosis magnitude of the extracted source signal, the worse the performance of these algorithms. In this paper, with the assumption that each source is a non-Gaussian linear process, a novel blind source extraction algorithm, called turbo source extraction algorithm (TSEA), is proposed. The ideas of the TSEA are to exploit signal temporal properties for increasing the normalized kurtosis magnitude, and to apply spatial and temporal processing in a cyclic fashion to improve the signal extraction performance. The proposed TSEA not only outperforms the FKMA, but also shares the convergence and computation advantages enjoyed by the latter. This paper also considers the extraction of multiple sources, also known as source separation, by incorporating the proposed TSEA into the widely used multistage successive cancellation (MSC) procedure. A problem with the MSC procedure is its susceptibility to error propagation accumulated at each stage. We propose two noncancellation multistage (NCMS) algorithms, referred to as NCMSFKMA and NCMS-TSEA, that are free from the error propagation effects. Simulation results are presented to show that the NCMSTSEA yields substantial performance gain compared with some existing blind separation algorithms, together with a computational complexity comparison. Finally, we draw some conclusions. Index Terms—Blind source separation (BSS), kurtosis maximization, multistage successive cancellation (MSC) procedure, noncancellation multistage (NCMS) algorithms, turbo source extraction algorithm (TSEA).

I. INTRODUCTION

B

LIND source separation (BSS) (or independent component analysis) has been a widely known problem in science and engineering areas such as array signal processing, wireless comManuscript received January 31, 2005; revised September 26, 2005. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Ercan E. Kuruoglu. This work was supported by the National Science Council, R.O.C., under Grant NSC 93-2213-E-007-079 and Industrial Technology Research Institute under Grant S5-93002. This work was partly presented at the Third IEEE International Symposium on Signal Processing and Information Technology, Darmstadt, Germany, December 14–17, 2003 and partly presented at the Third IEEE Sensor Array and Multichannel Signal Processing Workshop, Barcelona, Spain, July 18–21, 2004. C.-Y. Chi is with the Department of Electrical Engineering and Institute of Communications Engineering, National Tsing Hua University, Hsinchu, Taiwan 30013, R.O.C. (e-mail: [email protected]). C.-H. Peng is with the Department of Electrical Engineering and Institute of Communications Engineering, National Tsing Hua University, Hsinchu, Taiwan 30013, R.O.C. (e-mail: [email protected]). Digital Object Identifier 10.1109/TSP.2006.877640

munications, and biomedical signal processing [1]–[5]. With sensor measurements, denoted as a a given set of vector , the BSS problem unknown source signals, denoted as a is to extract vector , based on the following instantaneous (or memoryless) multiple-input multipleoutput (MIMO) model

(1) is an unknown mixing mawhere trix and is a noise vector. There have been a number of BSS algorithms reported in the open literature basically including algorithms using secondorder statistics (SOS) (known as correlations) [3]–[5], [7], [8], algorithms using higher order statistics (HOS) (known as cumulants) [1], [2], [4], [6], [9]–[16], [18] and a variety of linear and nonlinear algorithms using principles such as maximum-likelihood method and maximum entropy [1], [2], [19], or using characteristics and features of either of source signals and the mixing matrix such as nonstationarity and nonnegativity or their combinations [20]–[24]. and some Some statistical assumptions about the sources conditions about the unknown mixing matrix are required by most existing algorithms using either SOS or HOS. For instance, are required to be temporally colored the source signals and spatially uncorrelated with different power spectra by SOS based algorithms such as the algorithm for multiple unknown signals extraction (AMUSE) proposed by Tong et al. [4], [7], the second-order blind identification (SOBI) algorithm proposed by Belouchrani et al. [5], and the matrix-pencil approach proposed by Chang et al. [8]. The AMUSE and SOBI algorithm further and the noise correlation matrix given or esrequire timated in advance, while the matrix-pencil approach requires instead of without need of the noise correlation matrix in the mean time [8]. On the other hand, algorithms using HOS generally do not require temporal conditions or as. For instance, Cardoso’s [10] sumptions about the sources fourth-order blind identification (FOBI) algorithm (which ignores the noise effect) and Tong, Liu, Soon, and Huang’s [4] extended FOBI (EFOBI) algorithm (which takes the noise effect into account) using HOS require that source signals be mutually independent with distinct fourth-order moments, and that . Hyvärinen and Oja’s [2], [11] fast fixed-point algorithm (also called the FastICA) using kurtosis (a fourth-order

1053-587X/$20.00 © 2006 IEEE

2930

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 54, NO. 8, AUGUST 2006

cumulant), and Ding and Nguyen’s kurtosis maximization algorithm (KMA) [12], [14] require that source signals be non-Gaussian (either temporally independent identically distributed (i.i.d.) or colored) and mutually independent, and that . The KMA works well, but it is basically a gradient-type iterative algorithm which is not very computationally efficient. Chi and Chen [15], [16] then proposed a fast kurtosis maximization algorithm (FKMA) which is computationally efficient with a super-exponential convergence rate. Both KMA and FKMA only involve spatial processing, regardless of whether the source signals are colored or not. As reported in [12], we also empirically found that the smaller the normalized kurtosis magnitudes of source signals, the worse the performance of the KMA and FKMA. This observation motivates us to use an additional process, namely temporal process (linear filter), to transform the original source signals into temporally processed source signals with higher normalized kurtosis magnitudes, thereby improving the signal extraction performance. In the paper, we propose a novel blind source extraction algorithm also by kurtosis maximization, referred to as turbo source extraction algorithm (TSEA), which extracts one source signal through a spatial processing (for source extraction), and a temporal processing (for conversion of the extracted source into a filtered source with larger normalized kurtosis magnitude) cyclically. For the extraction of all the unknown sources using either the FKMA or the proposed TSEA, the widely used multistage successive cancellation (MSC) procedure [15], [16], [25]–[27] has been an effective approach in spite of error propagation effects accumulated from stage to stage [27]. Furthermore, two new noncancellation multistage (NCMS) BSS algorithms, referred to as NCMS-FKMA and NCMS-TSEA, which outperform the corresponding BSS algorithms involving the MSC procedure, are also proposed together with some simulation results including a computational load comparison, in addition to a performance comparison with some existing BSS algorithms through simulation. The remaining parts of this paper are organized as follows. A review of the FKMA and the MSC procedure is presented in Section II together with some performance observations of the FKMA. Section III presents the proposed TSEA. Then the proposed NCMS-FKMA and NCMS-TSEA are presented in Section IV together with some analytic results. Then, some simulation results are provided to verify the efficacy of the proposed algorithms in Section V. Finally, some conclusions are drawn in Section VI. II. REVIEW OF FKMA AND MSC PROCEDURE For ease of later use, let us define the following notations: convolution operation of discrete-time (scalar, vector, or matrix) signals; expectation operator; Euclidean norm of vectors or matrices; zero vector; identity matrix;

Superscript “ ”

complex conjugation;

Superscript “T”

transpose of vectors or matrices;

Superscript “H”

complex conjugate transpose (Hermitian) of vectors or matrices; range space of matrix null space of matrix dimension of

; ;

;

fourth-order joint cumulant of random , and variables ; kurtosis of random variable ; signal-to-noise ratio.

SNR

Some general assumptions that are also made in the paper for the MIMO system model given by (1) are as follows: A1) The unknown mixing matrix is of full column (i.e., ), and is known rank with a priori. , is a nonA2) Each source signal Gaussian linear process, i.e., can be modeled as the output of a causal stable linear time-invariant system as follows:

(2) is a stationary zero-mean non-Gaussian where i.i.d. process with nonzero (real) kurtosis [9] given by

(3) and are statistically independent for all . is zero-mean Gaussian, and statistically indepenA3) dent of . source extraction filter (a spatial filter) with Let be a . Its output is then the input being and

(4) The source separation process includes the design of a set of source extraction filters, each of which extracts a distinct source signal. Next, let us briefly review the FKMA proposed by Chi and Chen [15], [16], which is basically for the extracsource signals, i.e., where tion of one of the

CHI AND PENG: TSEA AND NONCANCELLATION SOURCE SEPARATION ALGORITHMS

, followed by some observations about its performance.

2931

Fact 1: With the assumptions A1) and A2), and the noisegiven in (5) attains maximum (either free assumption, locally or globally) if and only if

A. FKMA The FKMA is an iterative algorithm for finding the optimum spatial filter by maximizing the following objective function [12], [14]:

(5) which is also the magnitude of normalized kurtosis of . and obtained at the th iteration, Given at the th iteration is obtained through the following two steps. by Step 1) Update

(6) where

(11) is an unknown nonzero constant and the unknown where [14]. integer Fact 2: The FKMA is also a hybrid algorithm using the superexponential algorithm [17], [18] in Step 1 for fast convergence (basically with super-exponential convergence rate) and a gradient-type algorithm in Step 2) for the guaranteed convergence for finite data length and finite SNR. and , and In practice, sample cumulants sample correlation matrix , which are consistent estimates regardless of the SNR [28], are used by the FKMA. It is widely known that the performance of any source extraction (or separation) algorithms, including the FKMA is better either for higher SNR or for larger data length . Furthermore, we empirically found that performance of the FKMA is also dependent upon temporal properties of source signals to be presented next. It can be easily shown by A2) that

(7) and

(12) which leads to (8)

. Then obtain the associated , go to the next iteration; othStep 2) If erwise update through a gradient-type optimization algorithm, i.e.,

by (5)

(13)

where

(14) (9) where is a step size such that parameter, and then obtain the associated . (which can be arbitrarily chosen An initial condition with ) is needed to initialize the FKMA, and is usually updated in Step 1) before convergence. It can be easily shown, from (5), that

(10) required for comNote, from (8), (9), and (10), that puting (see (10)) in Step 2) has been obtained in is the same at each itStep 1), and the correlation matrix eration, indicating simple and straightforward computation for in Step 2). The efficacy of the FKMA is supported updating by the following two facts.

which is also an “entropy measure” of the stable sequence [27] (with larger indicating smaller entropy of , for , a unit sample sequence, and for all and integer ). Note that the larger the value , the larger the value of . In other words, of can be thought of as a measure of distance of from a Gaussian process [29]. These characteristics of the nonGaussian source signal account for the following empirical observation [12]. Observation 1: The FKMA itself is an exclusive spatial (i.e., processing algorithm. The smaller the value of the smaller the distance of from a Gaussian process), the worse the performance of the FKMA for finite SNR and (i.e., the lower estimation accuracy of the finite data length ), although the asymptotic performance extracted source of the FKMA as and SNR is independent of the by Fact 1. value of Some simulation results to justify Observation 1 will be presented in Section V later. Next, let us review the MSC procedure for the extraction of all the source signals using the FKMA.

2932

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 54, NO. 8, AUGUST 2006

B. MSC Procedure

in which

(possibly in a nonsequenEstimates tial order) of all the source signals can be obtained using the FKMA through the MSC procedure [15], [16], [25], [27] that includes the following two steps at each stage: (where is unT1) Obtain a source signal estimate known) using a source extraction algorithm, and then obtain the associated channel estimate (see (1)) given by

(15) by , namely, cancellation of T2) Update from . the contribution of Note that the above MSC procedure is also equivalent to the Gram–Schmidt orthogonalization method reported [14] which, however, never involves the estimation of the mixing matrix . The resultant BSS algorithm that uses the FKMA in T1) is referred to as MSC-FKMA. Note that imperfect cancellation in T2) usually results in error propagation effects accumulated in the ensuing stages. Therefore, as any other source separation algorithms involving the MSC procedure, an inevitable disadvantage of the MSC-FKMA is given in the following remark: s obtained at later stages are usually R1) The estimates less accurate due to error propagation effects from stage to stage.

(18) (19) Note from (17) and (19) that the spatial filter is used to ex(i.e., removing the spatial tract a colored source signal is a filter interference due to the mixing matrix ), and such that its output is a better apto further process proximation to an i.i.d. non-Gaussian process than (i.e., ). given by (5) is used for the deAgain, the criterion . sign of the optimum source separation filter Let . However, it is almost formidable to find a closed-form solution for the optimum by . Surely, one can resort to solving gradient-type iterative algorithms (which are not very computationally efficient) for finding the optimum . Instead, the proposed TSEA is a cyclically iterative algorithm as shown in Fig. 1, which basically makes use of the computationally effi(denoted as cient FKMA for the design of and and in Fig. 1). As shown in Fig. 1, the proposed TSEA consists of two steps at the th cycle as follows. S1) a) Temporal Processing: Compute

(20) III. TURBO SOURCE EXTRACTION ALGORITHM By Observation 1, the FKMA, which only involves spatial processing, performs better for larger , and meanwhile characterizes the temporal property of the source about its distance from an i.i.d. non-Gaussian signal can lead to the process. Advisable temporal processing of processed source with . This motivates that the source extraction processing should include not only spatial processing but also temporal processing, although given by (1) is merely a spatial the measurement vector mixture of non-Gaussian sources. Instead of the source extraction filter (spatial filter) used by source separation filter: the FKMA, the following

using the b) Spatial Processing: Process as the initial condition for FKMA (with ) to obtain the optimum and

by the first line of (17) (21)

S2) a) Spatial Processing: Compute

(22) (16) spatial filter (source extraction is considered where is a is a single-input single-output (SISO) temporal filter) and filter of order equal to . Then the output of the source is separation filter

(17)

where b) Temporal

. Processing:

Find

the optimum and

(23) (by (22) and the second line of (17)) using the as the initial condition for ), FKMA (with where . (spatial processing) and one for An initial condition for (temporal processing), which can be arbitrarily chosen with

CHI AND PENG: TSEA AND NONCANCELLATION SOURCE SEPARATION ALGORITHMS

2933

Fig. 1. Signal processing procedure of the proposed TSEA.

unity norm, are needed to initialize the proposed TSEA. Two worthy remarks regarding the proposed TSEA are as follows: R2)

R3)

The FKMA is employed in a cyclic fashion like a turbine for finding both to process the measurement vector of the optimum spatial filter in S1) and temporal filter in S2), and meanwhile the objective function increases whenever either the spatial filter or the temporal filter is updated. Therefore, for each cycle, both of the FKMA and FKMA also converge at super-exponential rate by Fact 2. The obtained signal estimate given by (22), the output of the spatial filter in S2), is the signal of interest rather than the output of the source separation filter . Moreover, the proposed TSEA reduces to the and S2) is FKMA (i.e., FKMA in S1)) as removed.

Next, let us discuss why the cyclic operation of spatial–temporal processing of the proposed TSEA performs better than the FKMA. The output of the optimum source separation filter obtained by the proposed TSEA at the th cycle can be approximated as follows: by (22) and (23) by (2) and (11) (24) where

is an unknown nonzero constant and (25)

is the overall signal model associated with in (S2) is equivalent to increasing

. Increasing

by (24) and (14)

Therefore, the SISO temporal filter performs like a “higher, order whitening” filter because the larger the value of to an i.i.d. non-Gaussian signal the better approximation of to ). As is (or the better approximation of sufficiently large, becomes a deconvolution filter or a linear ), equalizer (which tries to remove the effects of “channel” turns out to be an estimate of the i.i.d. signal and its output (see (2)). Another interpretation why the performance of the proposed TSEA is superior to that of the FKMA is as follows. Note that given by (18) is actually a mixture of filtered sources with the same mixing associated with (see (1)). In the spatial promatrix cessing of S1) (part b) of S1)), the extraction of by processing is much more effective than the extraction of by processing using the FKMA because of , thus leading to more accurate estimate in S2). Let us conclude this section with the following two remarks: R4) The performance gain of the TSEA reaches the maximum as long as the order (a parameter under our choice) of the temporal filter is sufficiently large. The chosen value for is sufficient as does not increase significantly be, the better approximacause the larger the value of tion of to an i.i.d. non-Gaussian signal. On the other hand, the asymptotic performance of FKMA approaches and SNR by Fact 1. that of the TSEA as R5) The proposed TSEA can surely be employed to extract all the source signals through the MSC procedure. The resultant BSS algorithm that uses the TSEA in T1) of the MSC procedure (see Section II-B) is referred to as MSCTSEA, and it also outperforms the MSC-FKMA, at the extra expense of the temporal processing at each stage. IV. NONCANCELLATION MULTISTAGE SOURCE SEPARATION ALGORITHMS In view of the performance degradation of BSS algorithms involving the MSC procedure due to error propagation effects

2934

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 54, NO. 8, AUGUST 2006

as mentioned in R1), this section presents two multistage source separation algorithms without involving cancellation, referred to as NCMS-FKMA and NCMS-TSEA, respectively. Next let us concentrate on the NCMS-FKMA.

and were defined by (5) and (26), respectively. where given by (30) can be conFinding the constrained optimal verted into an unconstrained optimization problem by the following transformation:

A. Proposed NCMS-FKMA At each stage, the proposed NCMS-FKMA basically comprises a preprocessing of constrained source extraction and a processing of unconstrained source extraction using the FKMA. The former (preprocessing) designs the source extraction filter, denoted , to extract a distinct source by imposing some coninstead of cancellation processing where the substraint on script “ ” denotes the stage number. The latter is nothing but as the initial the source extraction using the FKMA with condition of the source extraction filter. Next, let us present the preprocessing. , we have extracted Assume that at the end of stage source signals and obtained the associated column estimates of using (15), denoted as . Let be a matrix defined as

(31) where

(32) in which

(33) (26) and be a projection matrix for which is orthogonal to , and . Assume that is of full column rank, i.e., . By singular value can be expressed as [30] decomposition (SVD),

(27) where

is given by

(34) The optimum given by (34) obtained in the preprocessing is supported by the following theorem. be the set of all the extracted source sigTheorem 1: Let . With the assumptions 1), 2), and the nals up to stage noise-free assumption, the optimum given by (34) is

is a

is an matrix with the The projection matrix

unitary matrix, unitary matrix, is a th entry , and (singular values of ). is known as [30]

The associated optimum

(28) where

(29) which is also a full-rank ). The optimum spatial filter cessing is

matrix (i.e., to be designed in the prepro-

(30)

(35) where is an unknown nonzero constant and . The proof of Theorem 1 is presented in Appendix I. The proposed NCMS-FKMA is summarized as follows. NCMS-FKMA: F1) Set and . F2) Update by . If , obtain by (26) followed by its SVD, and then obtain and by (28) and (33), respectively. F3) a) Preprocessing (constrained source extraction): Obtain and using the FKMA. Then obtain using (15) (with replaced by in (15)), and obtain using (31). b) Unconstrained source extraction: Obtain the optimum and using the FKMA (with as the initial condition for ). F4) If , go to 2); otherwise, all the source signal estimates have been obtained. Three remarks about the proposed NCMS-FKMA are worth mentioning as follows.

CHI AND PENG: TSEA AND NONCANCELLATION SOURCE SEPARATION ALGORITHMS

R6) The FastICA using kurtosis [2], [11] obtains the through eigendeprewhitened signal composition (or SVD) of such that and then performs source separation by kurtosis maximization. Contrasting with the FastICA using kurtosis, the SVD in F2) is for obtaining the prorather than any prewhitening process. jected data R7) The proposed NCMS-FKMA with only preprocessing F3a) alone (i.e., removal of F3b)) itself is also a BSS algorithm which can extract a distinct source (by Theorem 1). However, its performance depends on the estimation accura(used in the cies of the channel estimates (see (30))). Although these constraint of channel estimates are never perfect for finite SNR and data length, F3a) provides a well-designed initial condition for the unconstrained source extraction filter in F3b). R8) The unconstrained source extraction filter in F3b) with obtained by F3a) accorda suitable initial condition obtained ingly leads to one distinct source estimate at each stage neither involving cancellation nor imposing any constraints on the source extraction filter, as well as faster convergence of F3b) than F3a). Therefore, the proposed NCMS-FKMA outperforms the MSC-FKMA due to no error propagation effects at each stage, but moderately extra computational expense for the constrained source extraction F3a) is required. Moreover, only FKMA is employed for the source extraction in F3a) and F3b) at each stage, the proposed NCMS-FKMA is also a fast multistage BSS algorithm.

2935

R9)

Remarks R7) and R8) regarding the constrained source extraction F3a) and unconstrained source extraction F3b) of the proposed NCMS-FKMA also apply to T3a) and T3b) of the proposed NCMS-TSEA which also shares the fast convergence of the proposed TSEA at each stage as presented in R2), so the performance of the proposed NCMS-TSEA is also superior to the MSC-TSEA at the extra computational expense for the constrained source extraction T3a). R10) As the proposed MSC-TSEA performs better than the MSC-FKMA (see R5)), the proposed NCMS-TSEA also performs better than the proposed NCMS-FKMA for the same reasons as presented in Section III at the moderate expense of extra computational load for the temporal processing of the TSEA. V. SIMULATION RESULTS To justify the efficacy of the proposed BSS algorithms, MSCTSEA, NCMS-FKMA, and NCMS-TSEA, two parts of simulation results are to be presented. Section V-A focuses on the performance of the proposed NCMS-FKMA and NCMS-TSEA to manifest the superiority of the proposed TSEA to the FKMA, and Section V-B focuses on performance comparison of the proposed BSS algorithms with the existing MSC-FKMA, FastICA using kurtosis, SOBI algorithm, and AMUSE. s used for generating s were equiprobThe i.i.d. and the noise vector able random binary sequences of was real, zero-mean, spatially independent, and white Gaussian . The synthetic was generated acwith covariance cording to (1) and then processed by the BSS algorithms under test. The convergence criterion [32] used was

B. Proposed NCMS-TSEA The NCMS-TSEA, which is basically obtained by replacing the FKMA used in 3-a) and 3-b) of the proposed NCMSFKMA by the proposed TSEA, is given as follows. NCMS-TSEA: and . T1) Set T2) Update by . If , obtain by (26) followed by its SVD, and then obtain and by (28) and (33), respectively. T3) a) Preprocessing (constrained source extraction): Obtain , and using the proposed TSEA. Then obtain using (15) (with replaced by in (15)), and obtain using (31). b) Unconstrained source extraction: Obtain the optimum , and using the proposed TSEA (with and as the initial conditions for and , respectively). T4) If , go to 2); otherwise, all the source signal estimates have been obtained. Two worthy remarks regarding the proposed NCMS-TSEA are as follows.

whenever the FKMA was employed, where was the spatial filter obtained at the th iteration, and that for the TSEA was

where and were the obtained in S1) and S2), respectively, at the th cycle. Moreover, the initial conditions for the spatial filter and the temporal filter used were , and , respectively, whenever they were not specified by the BSS algorithm under test. Fifty independent runs were conducted for performance evaluation of each BSS algorithm. At the th independent run, an optimum spatial filter and the associated source estimate were obtained. The estimate can be expressed as (36) where and for the BSS algorithms without involving the MSC procedure, and that involving the MSC procedure can be found in [27, App. B]. The average

2936

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 54, NO. 8, AUGUST 2006

output signal-to-interference-plus-noise ratio (SINR) associated over the 50 independent runs can be calculated as with

output SINR

(37)

is the th entry of the where averaged output SINR defined as

output SINR

vector . The total

output SINR

(38)

Fig. 2. Simulation results (output SINR versus SNR) of Case 1).

is used as the performance index of each BSS algorithm. A. Performance of NCMS-FKMA and NCMS-TSEA The performance comparison of the proposed TSEA and the FKMA was conducted by virtue of the proposed NCMS-TSEA and NCMS-FKMA without involving any constraints and cancellation processes. In the simulation, a 5 4 real mixing matrix and four MA models ’s taken from [8] were used which are given as follows:

(39) (40) Four cases are considered as follows: Case 1) Output SINR versus SNR for different data length ( , and ), (or ) for all , and (the ). order of the temporal filter Case 2) Output SINR versus different data length for (or ) for SNR 30 dB, . all , and Case 3) Output SINR versus (or ) for , and . all , for SNR 30 dB, Case 4) Output SINR versus , and the average of versus , for SNR 30 dB, , and as as well as well as 0.5 (i.e., 0.2368) for all . The simulation results for the four cases are shown in Figs. 2 through 5, respectively. One can observe, from Figs. 2 and 3, that the proposed NCMS-TSEA ( ) performs better than the proposed NCMS-FKMA ( ) for different values of SNR

Fig. 3. Simulation results (output SINR versus data length

N ) of Case 2).

and data length . Their performance differences are larger for either higher SNR (by Fig. 2) or smaller (by Fig. 3), and the performance of the proposed NCMS-TSEA has saturated for , and the asymptotic performance of the proposed NCMS-FKMA approaches that of the proposed NCMS-TSEA for high SNR (SNR 30 dB) (by Fig. 3). These as results are consistent with Observation 1 and R4). On the other hand, one can observe, from Fig. 4, that the proposed NCMS-TSEA outperforms the proposed NCMS-FKMA for all the values of , and their performance differences are larger for smaller . Moreover, the performance of the former is robust to the value of , while that of the latter is sensitive to the value of as mentioned in Observation 1. Finally, one can see, from Fig. 5(a), that the performance of the proposed imNCMS-TSEA has basically reached a plateau for plying that the optimum performance has been attained as long as is sufficiently large, and from Fig. 5(b), that the average of increases with while its increase is nonsignificant

CHI AND PENG: TSEA AND NONCANCELLATION SOURCE SEPARATION ALGORITHMS

2937

Fig. 4. Simulation results (output SINR versus ) of Case 3).

for , implying that is sufficient (see R4)) for all the simulations in Section V-A. Therefore, the above simulation results verify that the performance of the proposed TSEA is superior to that of the FKMA. As a remark, in spite of the same for all ) for each source power spectra (i.e., the same of the above four cases, the proposed TSEA works well with better performance than the FKMA. B. Performance Comparison The simulation results to be presented include performance tests to the proposed MSC-TSEA, NCMS-FKMA, NCMS-TSEA, and performance comparison with the MSC-FKMA, FastICA using kurtosis [11], AMUSE [7], and SOBI algorithm [5]. Both of the AMUSE and SOBI algorithm design the source separation matrix (demixing matrix) using SOS, where is a whitening matrix and is a unitary matrix. All the sources are extracted simultaneously without involving temporal processing as follows:

Fig. 5. Simulation results of Case 4). (a) Output SINR versus the order of the J (" [n]) versus L. temporal filter L and (b) (1=K )

given by (39) and the The same 5 4 mixing matrix ’s given by (40) with four MA models (or ) were used in the simulation. The following three cases are considered. Case A) Case B)

Output versus SNR for (data length) and (the order of the temporal filter ). and . Output SINR versus SNR for

Case C)

Output SINR versus

(41) is composed of where the prewhitened source signals. The whitening matrix is obtained through eigen-decomposition of the correlation matrix . On the other hand, the unitary matrix obtained by the AMUSE is through eigen-decomposition of the of the prewhitened correlation matrix for a chosen [7], while that obtained by the SOBI signal algorithm is through joint diagonalization of a set of [5]. The simulation results to be presented below were obtained with for the AMUSE and with for the SOBI algorithm.

for SNR

20 dB and

.

Fig. 6 shows some simulation results for Case A), where only source 1 is of interest. Note that source 1 was extracted by both of the proposed MSC-TSEA and the MSC-FKMA at in most of the 50 independent runs. One can see stage from this figure that the proposed NCMS-TSEA ( ) performs best (highest SINR ), the proposed MSC-TSEA ( ) second, the proposed NCMS-FKMA ( ) third and their performances are

2938

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 54, NO. 8, AUGUST 2006

Fig. 8. Simulation results (output SINR versus data length Fig. 6. Simulation results (output SINR versus SNR) of Case A).

Fig. 7. Simulation results (output SINR versus SNR) of Case B).

superior to those of the MSC-FKMA ( ), FastICA using kurtosis ( ), AMUSE ( ), and SOBI algorithm ( ). Some more observations from Fig. 6 are as follows. The proposed NCMS-TSEA ( ) performs much better than the proposed NCMS-FKMA ( ) (without cancellation processes for both of them). The proposed MSC-TSEA ( ) also performs much better than the MSC-FKMA ( ) (through cancellation processes for both of them) (as stated in R5)). In other words, the proposed TSEA outperforms the FKMA no matter whether the cancellation procedure is involved or not as mentioned in R10). Moreover, the performance of the proposed NCMS-TSEA ( ) outperforms the proposed MSC-TSEA ( ) and the same observation happens to the proposed NCMS-FKMA ( ) and the MSC-FKMA ( ). In other words, the BSS algorithm (using either TSEA or FKMA) without involving the cancellation procedure performs better than involving the cancellation procedure as stated in R8) and R9).

N ) of Case C).

Fig. 7 shows some simulation results (output SINR versus SNR) for Case B). One can see from this figure that the proposed NCMS-TSEA ( ) performs best (highest SINR), the proposed MSC-TSEA ( ) second, FastICA using kurtosis ( ) third, the proposed NCMS-FKMA ( ) fourth and their performances are superior to those of the MSC-FKMA ( ), AMUSE ( ), and SOBI algorithm ( ). All the other observations from Fig. 6 also apply to Fig. 7, except for smaller performance differences between the proposed NCMS-TSEA ( ) and MSC-TSEA ( ) for this case because all the sources except source 1 were exin most of the 50 independent tracted by the latter at stage runs (i.e., output SINR output SINR for ). Due to the same reason, performance differences between the proposed NCMS-FKMA ( ) and MSC-FKMA ( ) for Case B) are also smaller than those for Case A). Fig. 8 shows some simulation results (output SINR versus for Case C)). From Fig. 8, one can observe that data length all the seven BSS algorithms perform better for larger . Both of the proposed NCMS-TSEA and MSC-TSEA significantly outperform the other BSS algorithms, while the FastICA using kurtosis ( ) performs third. The performance of the proposed NCMS-FKMA is slightly superior to the MSC-FKMA and much better than that of the AMUSE, while the SOBI algorithm performs worst. Next, let us show some simulation results for source power spectra with different overlaps. The 3 2 mixing matrix used was the one by removing the last two rows and last two columns of the 5 4 mixing matrix given by (39), and the transfer func’s used were as follows: tions of the two MA models

Note that the larger the parameter , the smaller the spectral overlap between the two source power spectra. The simulation results for the following case are shown in Fig. 9.

CHI AND PENG: TSEA AND NONCANCELLATION SOURCE SEPARATION ALGORITHMS

2939

TABLE I COMPLEXITY AND PERFORMANCE COMPARISON OF THE MSC-FKMA, MSC-TSEA, NCMS-FKMA, AND NCMS-TSEA WITH THE SIMULATION RESULTS IN SECTION V-B

Fig. 9. Simulation results (output SINR versus spectral shift parameter  ) of Case D).

Case D)

Output SINR versus , and .

for SNR

30 dB,

From Fig. 9, we can observe that the five BSS algorithms using kurtosis perform well for all and significantly outperform the AMUSE ( ) and SOBI algorithm ( ) whose performance degradation is larger for smaller , and meanwhile the proposed NCMS-TSEA and MSC-TSEA perform best. In order to compare the computational complexities of the four kurtosis maximization based BSS algorithms, namely, MSC-FKMA, MSC-TSEA, NCMS-FKMA, and NCMS-TSEA, spent averages of the total number of iterations per source spent by FKMA for the simulation by the FKMA and results in Section V-B are shown in Table I together with whether the MSC and SVD are involved, and performance rank. Some observations from the results shown in Table I are associated with the MSC-FKMA as follows. The value of

is 7.35 and that associated with the NCMS-FKMA is 14.93 (about double of the former) which is the sum of for F3a) and for F3b), implying that the preprocessing of constrained source extraction F3a) is of benefit to the convergence speed of the unconstrained source extraction cycle and cycle F3b). On the other hand, the values of associated with the MSC-TSEA are 4.18/cycle and 3.09/cycle, respectively, and those associated with the NCMS-TSEA are cycle and cycle for T3a), and cycle and cycle for T3b), also implying the preprocessing of the constrained source extraction T3a) is of benefit to the convergence speed of the unconstrained source extraction T3b). Moreover, cycle, and cycle are basically much these values of less than 10 indicating the fast convergence of the FKMA (as stated in Fact 2). These results are also consistent with R8) and R9). One can also observe from Table I that the MSC-FKMA per(i.e., the lowest computational exforms worst with pense) and that the NCMS-TSEA performs best at the highest and computational expense with (about 3.5 times the value associated with the MSC-FKMA), the and MSC-TSEA performs second with (about 1.6 times the value associated with the MSC-FKMA), the (about double NCMS-FKMA performs third with of the value associated with the MSC-FKMA). The computational expenses associated with the MSC and SVD are basically not significant compared with that associated with the FKMA. Overall speaking, Table I provides a comparison of performance and the computational complexities of the four BSS algorithms (using the kurtosis maximization criterion) based on the simulation results presented in Section V-B, although these comparisons also depend on the values of , and used. These results also support the efficacy of the proposed NCMS-TSEA, NCMS-FKMA, and MSC-TSEA with moderately higher computational load than the MSC-FKMA as mentioned in R8) and R10).

2940

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 54, NO. 8, AUGUST 2006

VI. CONCLUSION

where

Chi and Chen’s FKMA only involves spatial processing for extraction of a single non-Gaussian (i.i.d. or colored) source. It performs well with super-exponential convergence rate, but its (see performance depends on the parameter (14)) as stated in Observation 1. By taking the effects of on the source extraction performance into account, we have presented a novel blind source extraction algorithm, TSEA shown in Fig. 1, which operates cyclically using the FKMA for both of the temporal processing and spatial processing. The proposed TSEA outperforms the FKMA for in addition to sharing convergence speed and computational efficiency of the latter at each cycle. Note that the proposed TSEA is only applicable for sources that can be modeled as non-Gaussian linear processes. As any other source extraction algorithms, it is straightforward to apply the proposed TSEA through the MSC procedure for the extraction of all the sources leading to the proposed MSC-TSEA. Because of performance degradation resultant from the error propagation in the MSC procedure as stated in R1), we further presented two noncancellation BSS algorithms, namely, NCMS-FKMA and NCMS-TSEA, that can extract a distinct source at each stage. The proposed NCMS-FKMA and NCMS-TSEA perform better than the existing MSC-FKMA and the proposed MSC-TSEA, respectively, with moderately higher computational complexities, as stated in R8) and R9). of sources was assumed known a Note that the number priori in the proposed BSS algorithms. The determination of based on normalized kurtosis is left as a future research. We also provided some simulation results to show the efficacy of the proposed MSC-TSEA, NCMS-FKMA, and NCMSTSEA together with a performance comparison with some existing BSS algorithms, and a computational load comparison with the MSC-FKMA (Table I). Two final concluding remarks from the simulation results are as follows. R11) The proposed TSEA outperforms the FKMA no matter whether or not the cancellation procedure is involved. R12) The proposed BSS algorithms using either TSEA or FKMA without involving the cancellation procedure performs better than involving the cancellation procedure, and meanwhile all the sources are not required to be temporally colored with different power spectra as required by most of SOS based algorithms.

APPENDIX I PROOF OF THEOREM 1 For ease of later use, let the mixing matrix be partitioned as signal vector

and the source

(I.1) (I.2)

(I.3) (I.4) (I.5) An inequality regarding matrix rank which is needed in the proof below is as follows [31]:

(I.6) where is a matrix and is an matrix. Moreover, by the assumption A1), one can easily prove the following fact (also needed in the proof below). defined by (I.3) and defined Fact 3: The matrices , and by (I.4) are of full column rank with , respectively. Moreover, . Assume that, without loss of generality, all the source estith stage are mates obtained at the end of the under ), ), and the noise-free assumption. Therefore, the matrix (defined by channel estimates (26)) which consists of the associated obtained by (15) is given by

by (1) and (15)

(I.7)

Let

(I.8) By (33) and (I.8), we have

by (1) by (I.1) by (I.2) and (I.8)

(I.9)

where in the derivation of (I.9), we have used the fact that because of . , denoted , by maximizing The optimum without any constraint on is known to be

by (I.9) and Fact 1 (I.10) mixing matrix associated if the unknown (see (I.9)) is of full column rank (i.e., with ), where is an unknown nonzero constant and is one of the source signals in . Therefore, what . remains to prove is

CHI AND PENG: TSEA AND NONCANCELLATION SOURCE SEPARATION ALGORITHMS

Substituting (28) into (I.8), one can obtain

REFERENCES (I.11)

where

by (I.4) (I.12) and by (28), (I.7) and

Again, by the fact of Fact 3, one can easily show that

(I.13) because the nullity of is equal to (by (29)) and (by Fact 3). matrix Next, let us prove that the given by (I.12) is of full column rank, i.e., . The matrix (see (I.12)) is of full column rank if i.e.,

by (I.12) and (29)

(I.14)

. Beleads to a unique solution of cause of (by (I.13)), it can be easily inferred that

(I.15) are scalars. Note, from (I.15), that the where while vector on the left-hand side of (I.15) belongs to . Because that on the right-hand side of (I.15) belongs to (by Fact 3), one can infer, from (I.15), of that

(I.16) which implies that

2941

because of (by Fact 3). Thus, we have proven that

. By (I.11) and (I.6) (with ), it can be easily inferred that the matrix must be of full column rank with because of (by (29)) and . Thus, we have completed the proof. ACKNOWLEDGMENT The authors would like to sincerely thank Dr. W.-K. Ma, Dr. C.-C. Feng, Dr. C.-Y. Chen, and Dr. C.-H. Chen for their advice and suggestions during the preparation of the manuscript.

[1] P. Comon, “Independent component analysis, a new concept?,” Signal Process., vol. 36, pp. 287–314, Apr. 1994. [2] A. Hyvärinen, J. Karhunen, and E. Oja, Independent Component Analysis. New York: Wiley-Interscience, 2001. [3] A. Cichocki and S. Amari, Adaptive Blind Signal and Image Processing: Learning Algorithms and Applications. New York: Wiley, 2002. [4] L. Tong, R.-W. Liu, V. C. Soon, and Y.-F. Huang, “Indeterminacy and identifiability of blind identification,” IEEE Trans. Circuits Syst., vol. 38, pp. 499–509, May 1991. [5] A. Belouchrani, K. Abed-Meraim, J.-F. Cardoso, and E. Moulines, “A blind source separation technique using second-order statistics,” IEEE Trans. Signal Process., vol. 45, no. 2, pp. 434–444, Feb. 1997. [6] N. Delfosse and P. Loubaton, “Adaptive blind separation of independent sources: A deflation approach,” Signal Process., vol. 45, pp. 59–83, Feb. 1995. [7] L. Tong, V. C. Soon, Y.-F. Huang, and R.-W. Liu, “AMUSE: A new blind identification algorithm,” in Proc. IEEE Int. Symp. Circuits Systems, New Orleans, LA, May 1–3, 1990, pp. 1784–1787. [8] C. Chang, Z. Ding, S. F. Yau, and F. H. Y. Chan, “A matrix-pencil approach to blind separation of non-white sources in white noise,” in Proc. IEEE Int. Conf. Acoustics, Speech, Signal Processing (ICASSP), Seattle, WA, May 12–15, 1998, pp. 2485–2488. [9] C. L. Nikias and A. P. Petropulu, Higher-Order Spectral Analysis: A Nonlinear Signal Processing Framework. Englewood Cliffs, NJ: Prentice-Hall, 1993. [10] J.-F. Cardoso, “Source separation using higher order moments,” in Proc. IEEE Int. Conf. Acoustics, Speech, Signal Processing (ICASSP), Glasgow, U.K., May 23–26, 1989, pp. 2109–2112. [11] A. Hyvärinen and E. Oja, “A fixed-point algorithm for independent component analysis,” Neur. Comput., vol. 9, pp. 1483–1492, 1997. [12] Z. Ding, “A new algorithm for automatic beamforming,” in Proc. 25th Asilomar Conf. Signals, Systems, Computers, Pacific Grove, CA, Nov. 4–6, 1991, pp. 689–693. [13] Y. Inouye and K. Tanebe, “Super-exponential algorithms for multichannel blind deconvolution,” IEEE Trans. Signal Process., vol. 48, no. 3, pp. 881–888, Mar. 2000. [14] Z. Ding and T. Nguyen, “Stationary points of a kurtosis maximization algorithm for blind signal separation and antenna beamforming,” IEEE Trans. Signal Process., vol. 48, no. 6, pp. 1587–1596, Jun. 2000. [15] C.-Y. Chi and C.-Y. Chen, “Blind beamforming and maximum ratio combining by kurtosis maximization for source separation in multipath,” in Proc. IEEE Workshop Signal Processing Advances Wireless Communications, Taoyuan, Taiwan, R.O.C., Mar. 20–23, 2001, pp. 243–246. [16] C.-Y. Chi, C.-Y. Chen, C.-H. Chen, and C.-C. Feng, “Batch processing algorithms for blind equalization using higher-order statistics,” IEEE Signal Process. Mag., vol. 20, no. 1, pp. 25–49, Jan. 2003. [17] O. Shalvi and E. Weinstein, “Super-exponential methods for blind deconvolution,” IEEE Trans. Inf. Theory, vol. 39, pp. 504–519, Mar. 1993. [18] K. Yang, T. Ohira, Y. Zhang, and C.-Y. Chi, “Super-exponential blind adaptive beamforming,” IEEE Trans. Signal Process., vol. 52, no. 6, pp. 1549–1563, Jun. 2004. [19] J.-F. Cardoso, “Infomax and maximum likelihood for blind source separation,” IEEE Signal Process. Lett., vol. 4, no. 4, pp. 112–114, Apr. 1997. [20] K. Matsuoka and M. Kawamoto, “A neural net for blind separation of nonstationary signal sources,” in Proc. IEEE Int. Conf. Neural Networks, Orlando, FL, Jun. 27–Jul. 2, 1994, vol. 1, pp. 221–232. [21] S. Choi and A. Cichocki, “Blind separation of nonstationary and temporally correlated sources from noisy mixtures,” in Proc. IEEE Signal Processing Society Workshop Neural Networks Signal Processing, Sydney, NSW, Australia, Dec. 11–13, 2000, vol. 1, pp. 405–414. [22] ——, “Blind separation of nonstationary sources in noisy mixtures,” IEEE Electron. Lett., vol. 36, pp. 848–849, Apr. 2000. [23] D.-T. Pham and J.-F. Cardoso, “Blind separation of instantaneous mixtures of nonstationary sources,” IEEE Trans. Signal Process., vol. 49, no. 9, pp. 1837–1848, Sep. 2001. [24] J. Zhang, L. Wei, and Y. Wang, “Computational decomposition of molecular signatures based on blind source separation of non-negative dependent sources with NMF,” in Proc. IEEE Workshop Neural Networks Signal Processing, Sep. 17–19, 2003, pp. 409–418.

2942

[25] J. K. Tugnait, “Identification and deconvolution of multichannel linear non-Gaussian processes using higher order statistics and inverse filter criteria,” IEEE Trans. Signal Process., vol. 45, no. 3, pp. 658–672, Mar. 1997. [26] C.-Y. Chi and C.-H. Chen, “Cumulant based inverse filter criteria for MIMO blind deconvolution: Properties, algorithms, and application to DS/CDMA systems in multipath,” IEEE Trans. Signal Process., vol. 49, no. 7, pp. 1282–1299, Jul. 2001. [27] C.-Y. Chi, C.-H. Chen, and C.-Y. Chen, “Blind MAI and ISI suppression for DS/CDMA systems using HOS-based inverse filter criteria,” IEEE Trans. Signal Process., vol. 50, no. 6, pp. 1368–1381, Jun. 2002. [28] C.-Y. Chi, C.-Y. Chen, C.-H. Chen, C.-C. Feng, and C.-H. Peng, “Blind identification of SIMO systems and simultaneous estimation of multiple time delays from HOS-based inverse filter criteria,” IEEE Trans. Signal Process., vol. 52, no. 10, pp. 2749–2761, Oct. 2004. [29] J. M. Mendel, “Tutorial on higher-order statistics (spectra) in signal processing and system theory: Theoretical results and some applications,” Proc. IEEE, vol. 79, pp. 278–305, Mar. 1991. [30] G. H. Golub and C. F. V. Loan, Matrix Computations, 3rd ed. Baltimore, MD: The Johns Hopkins Univ. Press, 1996. [31] R. A. Horn and C. R. Johnson, Matrix Analysis. New York: Cambridge Univ. Press, 1985. [32] C.-Y. Chi and C.-H. Chen, “Two-dimensional frequency-domain blind system identification using higher order statistics with application to texture synthesis,” IEEE Trans. Signal Process., vol. 49, no. 4, pp. 864–877, Apr. 2001.

Chong-Yung Chi (S’83–M’83–SM’89) was born in Taiwan, R.O.C., on August 7, 1952. He received the B.S. degree from the Tatung Institute of Technology, Taipei, Taiwan, R.O.C., in 1975, the M.S. degree from the National Taiwan University, Taipei, Taiwan, R.O.C., in 1977, and the Ph.D. degree from the University of Southern California, Los Angeles, in 1983, all in electrical engineering. From July 1983 to September 1988, he was with the Jet Propulsion Laboratory, Pasadena, CA, where he worked on the design of various spaceborne radar remote sensing systems, including radar scatterometers, SARs, altimeters, and rain mapping radars. From October 1988 to July 1989, he was a visiting Specialist at the Department of Electrical Engineering, National Taiwan University. He has also been with National Tsing Hua University, Hsinchu, Taiwan, R.O.C., as a Professor with the Department of Electrical Engineering since August 1989 and the Institute of Communications Engineering (ICE) since August 1999 (as well as the Chairman of ICE from August 2002 to July 2005). He was a visiting Researcher at the Advanced Telecommunications Research (ATR) Institute International, Kyoto, Japan, in May and June 2001. He has published more than 120 technical papers in radar remote sensing, system identification

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 54, NO. 8, AUGUST 2006

and estimation theory, deconvolution and channel equalization, digital filter design, spectral estimation, higher order statistics (HOS)-based signal processing, and wireless communications. His current research interests include signal processing for wireless communications, statistical signal processing and digital signal processing and their applications. Dr. Chi is an active member of Society of Exploration Geophysicists, a member of European Association for Signal Processing, and an active member of the Chinese Institute of Electrical Engineering. He was a Technical Committee Member of both the 1997 and 1999 IEEE Signal Processing Workshop on HOS and the 2001 IEEE Workshop on Statistical Signal Processing (SSP). He was also a member of International Advisory Committee of TENCON 2001 and a Member of International Program Committee of the Fourth International Symposium on Independent Component Analysis and Blind Source Separation (ICA 2003). He was a Co-Organizer and a general Co-Chairman of 2001 IEEE SP Workshop on Signal Processing Advances in Wireless Communications (SPAWC 2001), the International Liaison of SPAWC 2003, and the Technical Committee Member and International Liaison (Asia and Australia) of SPAWC 2004. He was a Program Committee Member of the International Conference on Signal Processing (ICSP 2003) and a Technical Committee Member of the Third IEEE International Symposium on Signal Processing and Information Technology (ISSPIT 2003). He was also the Chair of Information Theory Chapter of IEEE Taipei Section (July 2001 to June 2003). He was Technical Program Committee (TPC) Member of the Emerging Networks, Technologies & Standards Symposium, WirelessCom 2005, Track TPC (TTPC ) Member of the 2005 International Conference on Communications, Circuits and Systems, and TPC Member of First IEEE International Workshop on Computational Advances in Multisensor Adaptive Processing (IEEE CAMSAP 2005). Currently, he is an Associate Editor for the IEEE TRANSACTIONS ON SIGNAL PROCESSING and an Editorial Board Member of JASP (The EURASIP Journal on Applied Signal Processing), a member of Editorial Board of EURASIP Signal Processing Journal, a member of IEEE Technical Committee on Signal Processing Theory and Methods, TPC Member of the 2006 International Conference on Communications (ICC) Wireless and Ad hoc Sensor Networks, and TPC Member of the 2006 International Wireless Communications and Computing Conference (IWCCC 2006).

Chun-Hsien Peng was born in Taiwan, R.O.C., on November 21, 1978. He received the B.S. degree from the Department of Electrical Engineering, National Taipei University of Technology, Taipei, Taiwan, R.O.C., in 2001. He is currently working towards the Ph.D. degree at the Institute of Communications Engineering, Department of Electrical Engineering, National Tsing Hua University, Hsinchu, Taiwan, R.O.C. His research interests include statistical signal processing, digital signal processing, and wireless communications.