Downsizing data management for embedded ... - Semantic Scholar

Egyptian Computer Science Journal

Generalized Mixture Models for Blind Source Separation Amr Goneid Computer Science & Engineering Dept., the American University in Cairo, Cairo, Egypt Abeer Kamel and Ibrahim Farag Faculty of Computers and Information, Cairo University, Cairo, Egypt

Abstract Neural Independent Component Analysis (ICA) algorithms based on unimodal source distributions provide acceptable performances in the case of Blind Source Separation (BSS) of super-gaussian sources. However, their convergence profiles are significantly slower in the case of sub-gaussian sources. In some situations it is necessary to deal with sub-gaussian signals in the form of noise or others. In this case, one needs an algorithm that can deal efficiently with mixtures of both super- and sub-gaussian signals. In this paper, we introduce generalized mixture models for super-and sub-gaussian sources based on the Exponential Power Distibution (EPD). The kurtosis and stability profiles of these models are investigated and the corresponding non-linearities are derived. A switching algorithm is designed for the blind source separation of mixtres of super- and sub-gaussian sources. Experimental results are presented on the application of these models to homogeneous and mixed sources using a modified fast ICA algorithm.

1. Introduction Most of the Blind Source Separation (BSS) problems can be represented by the basic ICA instantaneous linear noiseless mixing model given by X = A S. In this model, X is a random matrix of observations, S is a random matrix of hidden sources with mutually independent components, and A is a nonsingular mixing matrix. There is no prior knowledge about the linear generative model A or the source signals S except that they are statistically independent. Computationally, the basic problem is to find an estimate Y of S and the mixing matrix A such that Y = W X ≈ S, where W = A-1 is the unmixing matrix. Practically, the estimate of the unmixing matrix W is not exactly the inverse of the mixing matrix A and the sources are recovered only up to scaling and permutation [1]. ICA techniques have been successfully used in problems of such nature as ,for example, in speech recognition [2], 3-D object recognition [3], image analysis [4], unsupervised classification [5], bioinformatics and biomedical research [6,7], texture segmentation [8], EEG [9], functional Magnetic Resonance Imaging (fMRI) [10], face recognition [11, 12], Geophysics and Remote Sensing [13,14] and Management [15]. While there exists no closed form estimate of the unmixing matrix W, the solution methods are based on optimizing the independence of Y [16,17,18]. The most two famous methods [1,19] seek an estimate of W either based on maximizing the negentropy (negative entropy) or by using Maximum Likelihood Estimation (MLE). Learning from the data is required in stepwise convergence procedures leading to essentially neural unsupervised learning algorithms. In such algorithms, it is common to use non-linearities F(x) that are derived from assumed source models such that F(x) = - log p(x), where p(x) is the PDF of the source. The non-1-

Egyptian Computer Science Journal linearities are essential in the optimization process and for the learning rules that update the estimates of the unmixing matrix W and, overall, they are important for the stability and robustness of the convergence process. Representative source models with symmetrical unimodal PDF’s are simple to analyze statistically and lead to computationally efficient algorithms. As an example, the source PDF p(x) = 1/cosh x leads to the general purpose function F(x) = log cosh x with derivative f (x) = tanh(x). Neural ICA algorithms based on unimodal source distributions provide acceptable performances in the case of super-gaussian sources. However, their convergence profiles are significantly slower in the case of sub-gaussian sources [20]. The reason why a non-linearity is inefficient in the case of sub-gaussian signals is that it is not representative of the source model. Therefore, there is a need for algorithms that can deal efficiently with mixtures of both super- and sub-gaussian signals. Such algorithms either monitor the statistics (e.g. kurtosis) of the recovered signals [21,22,23] or use an adaptive approach to apply flexible non-linearities during the separation process [24]. Another approach is to switch between fixed nonlinearities corresponding to separate super- and sub-gaussian source models [25]. In this approach, Lee et al [25] use a unimodal PDF to model super-gaussian sources, while subgaussian sources are modeled using a bimodal mixture model with Gaussian densities. In this paper, we present generalized mixture models for super-and sub-gaussian sources based on the EPD distribution. These models have different parameters that can be adjusted for efficiency and for which the models of Lee et al [25] are considered as a special case. Based on stability requirements, it would be possible to switch between non-linearities derived from the source models. The present paper is organized as follows: In Section 2 we present a generalized supergaussian model and the corresponding non-linearity. Section 3 presents our generalized mixture model for sub-gaussian sources and derives the corresponding non-linearity. Section 4 examines the stability profiles of the generalized models and Section 5 presents the neural learning algorithm to be used in the source separation. Section 6 gives experimental results on the application of the generalized models to homogeneous sources. Section 7 presents a switching algorithm for mixed sources and gives experimental results on the application of such algorithm to super- and sub-gaussian mixtures. Finally, Section 8 gives the summary and conclusions.

2. A generalized strictly super-gaussian mixture model The single EPD with zero mean is given by [26]:  ( x)  c exp{  x } ,   s b b

(1)

where c  1 /( 20 ) 

1 1 ,   , s, b  0 2 s( ) b

(2)

In the above, s = scale parameter, b = shape parameter and Г(..) is the gamma function. The quantities φn are given by: 

n   y n exp(   y b )dy   s n1 (  ( n  1 ))

(3)

0

They relate to the EPD central moments μn such that: for n odd 0  n   n ( ( n  1)) for n even  n / 0  s ( )  -2-

(4)

Egyptian Computer Science Journal

We may introduce a strictly super-gaussian mixture density that can be expressed as a product of an EPD with a hyperbolic term: (5) p( x )  c exp{   | x |b } sec h2 ( ax ) 2 The sech function is usually used to represent ultra short pulses and, compared to the Gaussian or Lorentzian distributions, it has the smallest FWHM. Its effect is to make the distribution more spiky and hence more super-gaussian. The generalized model given by equation (5) generally depends on the EPD parameters (ρ) and (b) and the sech parameter (a). For such density, the normalized kurtosis is given by: (6) K  E{ x 4 } /( E{ x 2 })2  3 With unit dispersion, K will depend only on (b) and (a). Figure (1) shows the normalized kurtosis of the model as a function of (b) for different values of the sech parameter (a).

Figure (1): Kurtosis profile for the generalized strictly super-gaussian mixture model It can be seen that the present super-gaussian mixture model represents a strictly supergaussian (K > 0) density for values of a > 1. It follows that we can introduce a strictly supergaussian non-linearity that has the form: F ( x )   log p( x )  const   | x |b 2 log sec h ( a x ) (7) f ( x )  b | x |b 1 sign( x )  2a tanh( ax ) f ' ( x )  b( b  1 ) | x |b  2 2a 2 ( 1  tanh 2 ( ax )) For the special case of a = 1, b = 2 and unit dispersion (ρb = 1), the above non-linearity reduces to: f(x)  x  2 tanh (x) (8) f '(x)  1  2 ( 1  tanh 2 (x)) which is the gaussian model given by Lee et al [25].

3. A generalized strictly sub-gaussian mixture model A bimodal EPD mixture density may be used to represent a generalized symmetric mixture model in the form: (9) p( x,b,  )  ( 1 / 2 ){ ( x,b,  )  ( x,b, )} where  ( x, b,  ) is the single EPD with mean μ. The central moments are given by:

-3-

Egyptian Computer Science Journal

0  1, 1  0,

2  ( 2 / 0 )   2

(10) 3  0, 4  ( 4 / 0 )  6 2 ( 2 / 0 )   4 Notice that when μ = 0, the above moments reduce to the case of the single EPD (equation (4)).

Figure (2): The symmetric EPD mixture PDF for different values of b and μ Figure (2) shows our computations for the symmetric EPD mixture PDF for different values of the shape parameter b and the location parameter μ. The top left figure shows a Laplacian mixture density with (b = 1). Such mixture density is bimodal for all values of μ > 0. The top center figure shows a Gaussian mixture density that is unimodal when μ < 1, bimodal when μ > 1 and Gaussian when μ = 0. Similar features are shown in the top right figure with b = 4 but with flatter tops. The bottom two figures show the densities for μ = 1 and μ = 2. The bottom right figure shows that when μ =2, the mixture density is bimodal for all values of b > 1. The normalized kurtosis for the EPD symmetric mixture is given by:  (    322 )  202  4 (11) K  42  3  4 0 2 ( 2  0  2 )2 Notice that when μ = 0, the above expression reduces to the case of the single EPD kurtosis. Another special case is the Gaussian mixture model given by Lee et al [25] with unit variance and b = 2. In this case, the general expression (11) reduces to:  2 4 (12) K 2 (   1) 2 Hence, K will always be between 0 and -2, i.e. this mixture model describes strictly subgaussian symmetric density for μ > 0. In general, the generalized EPD model given by equation (9) represents a strictly sub-gaussian (K < 0) density for all values μ > 0 and b ≥ 2 as can be seen from Figure (3) which shows the results of K for various values of the shape parameter b and location parameter μ.

-4-

Egyptian Computer Science Journal

Figure (3): Kurtosis profile for the generalized strictly sub-gaussian mixture model The above EPD mixture can be used to derive a strictly sub-gaussian non-linearity f(x). Let b (13)     ( x ,b,   )  C exp{   x   } ,   a b ,   0 Hence f ( x)  

p( x) / x (  / x)  (  / x)      p ( x)     g  g

(14)

where

g   exp {  ρ|x  μ|b } η  g  /x  ρ b |x  μ|b1 g  sign(x  μ), b  1

(15)

The derivative of f(x) can now be obtained as: f '( x )

D  D  f 2( x ) g  g

(16)

where D  b{(b  1) | x   |b2 g   | x   |b1 sign( x   ) }

(17)

The special case of the Gaussian mixture model can be obtained by taking b = 2, μ > 0 and ρ = 1/b = ½ to obtain: g   g   2 exp {  (x 2  μ 2 )/ 2 } cosh (μ x) η  η  2 exp {  (x 2  μ 2 )/ 2 }{x cosh (μ x)  μ sinh (μ x)} D  g   (x  μ)η  g ( 1  (x  μ)2 ) This leads to: f(x)  x   tanh(  x )

f ' ( x )  1   2   2 tanh 2 (  x ) which is the gaussian model given by Lee et al [25].

(18)

4. Stability profiles of generalized mixture non-linearities In principle, if one assumes certain distributions of the sources, then it is possible to use the kurtosis of a distribution to choose the corresponding non-linearity. In addition to the fact that in practical situations the source distributions are unknown, there is the uncertainty due to the mixing process. It is therefore argued that kurtosis is not always well defined and the determination of the optimal non-linearity is better achievable through a stability condition for a stability parameter that depends on the non-linearity and the signal itself. For a given recovered source xi, such stability parameter is defined as [27]: -5-

Egyptian Computer Science Journal vi  E{ xi2 }E{ f i' ( xi )}  E{ xi fi ( xi )}

(19)

For the estimate of independent sources to be stable, the condition is:

vi  0

for i  1...n

(20) We have made calculations of the stability parameter for our generalized mixture models derived in the previous two sections. For the generalized super-gaussian model, the nonlinearity f(x) and its derivative f‘(x) are those given by equation (7) while those for the generalized sub-gaussian model are given by equations (14) and (16). The expectations were computed as: 

E{ g( x )}   g( x )S ( x ,  ) dx

(21)



where S(x,β) is a zero mean EPD source with unit variance and shape parameter β. The expectations have been evaluated by numerical integration to obtain the dependence of stability parameter v on the source shape parameter β and non-linearity parameters. Figure (4) shows such dependence for non-linearities with b = 2 and unit dispersion. The left figure is for the super-gaussian mixture model for different values of the sech parameter (a). The right figure is for the sub-gaussin model for different values of the location parameter μ.

Figure (4): Stability parameter for the generalized EPD mixture models It can be seen that the stability condition is always valid for β < 2 for the super-gaussian model. This means that the present generalized mixture non-linearity (7) is strictly supergaussian and that the stability is higher with higher values of non-linearity sech parameter a. Hence, the present generalized super-gaussian mixture model can provide higher stability for the corresponding non-linearities by using values of a > 1. In case of the generalized sub-gaussian model, Figure (4) shows that the stability condition is always valid for β ≥ 2 and μ > 0. This means that the present generalized mixture nonlinearity (14) is strictly sub-gaussian and that the stability is higher with higher values of nonlinearity location parameter μ. Hence, the present generalized sub-gaussian mixture model can provide higher stability for the corresponding non-linearities by using values of μ > 1.

5. The neural learning algorithm In order to test the performance of the mixture models, we have adopted a modified algorithm based on the Fast ICA algorithm originally given by [28]. This modified algorithm is described in detail in a previous paper [20]. Basically, the algorithm uses a fixed-point iteration method to maximize the negentropy using a Newton iteration method. It assumes -6-

Egyptian Computer Science Journal that the observation matrix X of m components and n samples has been preprocessed by centering followed by whitening or sphering to remove correlations. Centering removes means via the transformation X←X-E{X} and whitening is done using a linear transform (PCA like) Z = VX where V is a whitening matrix. A popular whitening matrix is V = D-1/2 ET, where E and D are the eigenvector and eigenvalue matrices of the covariance matrix of X, respectively. The resulting new matrix Z is therefor characterized by E{ZZT} = I and E{Z}=0. After obtaining the unmixing matrix W from whitened data, the total unmixing matrix is then W ← W V. The algorithm estimates several or all components in parallel using symmetric orthogonalization by setting W ← (W WT)-1/2 W in every iteration. In our modified version of the algorithm [20], the matrix G = W Ao, where Ao is a randomly chosen initial value for the mixing matrix, is considered as decomposable in the form G = Q P, where P is a positive definite stretching matrix and Q is an orthogonal rotational matrix. The cosine of the rotation angle is to be found on the diagonal of Q so that a convergence criterion is taken as Δ |diag(Q)|min < ε, where ε is a threshold value. Also, In this algorithm, we use the performance (error) measure, E3 we introduced earlier in [29]:

E3 

m m m m 1 { g  M  M  1 }  {   ij   gij  M j  M j  1} i i 2m(m  1) i 1 j 1 j 1 i 1

(22)

where gij is the ijth element of the matrix G of dimensions m x m, Mi = maxk | gik | is the absolute value of the maximum element in row ( i ) and Mj = maxk | gkj | is the corresponding quantity for column ( j ). It is shown in our work [29] that the index E3 is more precise than the commonly used E1 and E2 indices [18, 30, 31] and is independent of the matrix dimensions. It is also normalized to the interval {0,1}, the greater the value of E3, the worse is the performance. The algorithm is summerized in the following steps:     

Preprocess signals X to get Z Choose random initial orthonormal vectors wi to form intial W. Compute G = W A0 and do polar decomposition of G Compute initial polar parameters (r,θ) and E3 Iterate: 1. Do Symmetric orthogonalization of W by setting W ← (W WT)-1/2 W 2. Compute new G = W A0 and do polar decomposition of G 3. Compute new polar parameters (r,θ) and E3 4. If not the first iteration, test for convergence: Δ | diag(Q) |min < ε 5. If converged, break. 6. Set Wold ← W 7. Update W using learning rule of the chosen non-linearity.

 

After convergence, dewhiten using W ← W V Compute independent components Y = W Z Algorithm (1): Modified ICA algorithm for many components

Step 7 in the iteration loop performs the updating of W using the learning rule for a given nonlinearity. For a given component wi from W this rule is given by:

-7-

Egyptian Computer Science Journal wi  E{z f (wi z)} – E{ f ’ (wi z)} wi

(23)

where z is a column vector representing one sample from the whitened matrix Z, f (y) is the non-linearity function, f ‘(y) is its derivative and the expectation is taken as the average over the n samples in Z. This is the basic fixed-point iteration in Fast ICA subject to the normalization w ← w / ‖w‖.

6. Experimental results for homogeneous sources A group of experiments has been done in order to investigate the performance of the mixture models under the condition of homogeneous sources, i.e., when the mixed sources are all either super-gaussian or sub-gaussian. In these experiments, we used our modified Fast ICA algorithm and the performance was measured in terms of the number of iterations required by the algorithm to converge. Each experiment represented a different number of homogeneous sources m (m = 2….30) with a length of 2000 random samples for each source. Each experiment was run 10 times with different random signals and a different random mixing matrix A0 in each run in order to obtain an average of the results. For generating the source signals, an EPD random signal generator was used with shape parameters β of 1 and 4 for super-gaussian and sub-gaussian sources, respectively. The above experimental setup has been used to measure the performance of the present mixture models. The number of iterations needed for the algorithm to converge with an error E3 < 0.02 is shown in Figure (5) where it is plotted against the number of super-gaussian sources.

Figure (5): Performance of the generalized mixture models The left figure compares between two models: the one given by Lee at al [25] as a strictly super-gaussian model (equation (8)) and present generalized super-gaussian mixture model as represented by equation (7). For our model, we have used a shape parameter b = 2 and a sech parameter a = 2. It can be seen from that figure that the two models exhibit almost similar performance profiles in that they both are almost linear in the number of super-gaussian sources. However, the present mixture model is able to perform better by requiring less iteration in order to separate the sources with the same precision. This can be achieved as shown in the figure by changing the value of the sech parameter (a). Over the region 20-40 sources, the enhancement is found to be about 13%. This represents an advantage of the present generalized strictly super-gaussian mixture model over the Lee et al model which does not contain the a and b adjustable parameters. -8-

Egyptian Computer Science Journal The graphs on the right in Figure (5) compare the performance of the strictly sub-gaussian mixture model of Lee et al [25] represented by equation (18) with a location parameter μ = 1 and the present generalized sub-gaussian mixture model represented by equations (14) and (16). For the generalized mixture model, we have used a shape parameter b = 2, a location parameter μ = 3 and a scale parameter a = 3. As in the super-gaussian case, the two models show close performances profiles. However, the present mixture model is able achieve source separation with the same precision using a smaller number of iterations particularly for large numbers of sub-gaussian sources. Over the region 20-30 sources, the enhancement is found to be about 12.5%. Again, this can be achieved by changing the values of present generalized model parameters.

7. A generalized switching model for mixed sources A comparison of the performance profiles in Figure (5) shows that the number of iterations required for convergence for sub-gaussin sources is significantly higher than that for the super-gaussian case, particularly as the number of sources increases. If the sources are mixed between super- and sub-gaussian, then the switching model originally put forward by Lee et al [25] seems to offer a solution to performance enhancement. Here we follow that switching model but use instead our own generalized mixture models represented by equations (7) and (14). For this purpose, we use the stability parameter (19) for switching between the superand sub-gaussian generalized non-linearities. When we compute such parameter for random sources with different source shape parameter, we obtain the results shown in Figure (6).

Figure (6): Stability parameter for the present generalized non-linearities

It can be seen from the figure that the stability condition (20) is satisfied when we use the super-gaussian mixture model with a recovered signal having v > 0 and the sub-gaussian mixture model with a recovered signal having v < 0. Such strategy is implemented by modifying our Fast ICA Algorithm (1) given in Section 5 to include the switching mechanism. The resulting modified algorithm is shown in Algorithm (2).

-9-

Egyptian Computer Science Journal

Update W using learning rule for switching non-linearities (NL): For each component i of m components: - Select ith row wi of W - Select recovered independent source Xi = wi Z - Compute stability parameter v ( Xi) = E{xi2} E{sech2(xi)} – E{ xi tanh(xi)} where xi is one sample from Xi - If (v > 0) choose f and f‘ to be the generalized super-gaussian NL, else choose them for the generalized sub-gaussian NL Set wi ← E{ z f (wi z)} - E{ f ‘(wi z)} wi where z is a column vector representing one sample from Z

Algorithm (2): Update rule for Fast ICA switching algorithm The update rule given in Algorithm (2) for the switching non-linearities has been used to obtain the performance profiles for our generalized mixture models using signals mixed between super- and sub-gaussian sources. The random super-gaussian and sub-gaussian signals were generated using an EPD generator with unit variance and shape parameters β = 1 and β = 4, respectively.

Figure (7): Performance of switching NL’s for mixed sources In case of mixed sources, the mixing matrix has been applied only after randomly permuting the order of the input sources. Figure (7) shows samples of the results for a 50/50% mixture with the extreme cases of 100% super-gaussian and 100% sub-gaussian sources also shown for comparison. In that figure, the graph marked (1) is for the case of 100% super-gaussian sources and has been obtained using our strictly super-gaussian non-linearity (7) with a shape parameter b = 2 and a sech parameter a = 2. The graph marked (2) is for case of 100% sub-10-

Egyptian Computer Science Journal gaussian sources and has been obtained using our strictly sub-gaussian non-linearity (14) with a shape parameter b = 2, a location parameter μ = 3 and a scale parameter a = 3. These two graphs represent the two extreme cases of homogeneous sources as given before. The graph marked (3) is for the case of 50/50% mixture of super- and sub-gaussian sources and using the super-gaussian NL as in the case of graph (1). In other words, it represents the case of mixed sources without the use of a switching NL. Finally, graph (4) represents the 50/50% mixed sources with the switching NL’s given by Algorithm (2). Examination of Figure (7) shows that, as expected, the cases of mixed sources fall somewhere between the two extreme cases of homogeneous sources. It also shows that the switching algorithm based on our generalized non-linearities actually enhances the performance of the separation process. The enhancement in the region 20-40 mixes sources is found to be about 17%.

8. Conclusions The symmetric mixture models we presented in this paper are generalizations of the Gaussianbased mixture models used by other workers, notably Lee et al [25]. The generalized models are based on an Exponential power Distribution (EPD) density with the adjustable parameters of scale, shape and location. For super-gaussian sources, the mixture model is EPD with a hyperbolic function and yields a unimodal source density that is strictly super-gaussian. For sub-gaussian sources, the mixture model is symmetric bimodal and yields a strictly subgaussian density. A mathematical analysis is used to derive the statistical properties of these models, including their various moments and kurtosis profiles. Such analysis is also used to prove that the Gaussian mixture models are only a special case of the present mixture models. We have derived the non-linearities corresponding to the present generalized mixture models for both cases of super- and sub-gaussian sources. Experimental results using these nonlinearities and homogeneous sources show that our generalized models are able to enhance the convergence profiles of the blind source separating algorithm by a factor of about 13%. The stability of the derived generalized non-linearities has also been analyzed and it shown that in the case of mixed sources, a stability parameter can be used to switch between the super- and sub-gaussian non-linearities. An algorithm utilizing such switching mechanism is designed and used for the blind source separation of mixtures of super- and sub-gaussian sources. Experimental results show that an enhancement of about 17% is achieved by using such switching algorithm.

-11-

Egyptian Computer Science Journal

References: 1. Hyvarirnen, A., Karhunen, J and Oja, E., "Independent Component Analysis", John Wiley and sons, New York , NT, 2001 2. Kwon, O.W and Lee, T.W., "Phoneme recognition using ICA-based feature extraction and transformation", Signal Processing, 84 (6), 1005-1019, 2004 3. Sahambi, H.S. and Khorasani, K., "A neural network appearance-based 3-D object recognition using Independent Component Analysis", IEEE Trans. on Neural Networks, 14(1), 138-148, 2003 4. Benlin, X., Fangfang, L., Xingliang, M. and Huazhong, J., “Study on Independent Component Analysis application in classification and change detection of multispectral images” The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol.XXXVII , Part B7, Beijing 2008 5. Lee, T.W., Lewicki, M.S. and Sejnowski, T.J., "ICA mixture models for unsupervised classification of non-gaussian classes and automatic context switching in blind signal separation", IEEE Trans. on Pattern Analysis and Machine Intelligence, 22 (10), 1078– 1089, 2000 6. Lee, S.I. and Batzoglou, S., "Application of Independent Component Analysis to microarrays", Genome Biol., 4 (11), R76, 2003 7. Kong, W., et al., “Independent Component Analysis of Alzheimer’s DNA microaray gene expression data”, Molecular Neurodegeneration, 4:5 2009 8. Jenssen, R. and Eltoft, T., "Independent Component Analysis for texture segmentation", Pattern Recognition, 36, 2301-2315, 2003. 9. Ashtiyani, M., Asadi, S., Birgani, P.M. and Khordechi, E.A., “EEG classification using neural neworks and Independent Component Analysis”, Biomed 2008, Proceedings 21, 179 - 182, 2008 10. McKeon, M.J., Makeig, S., Brown, G.G., Jung, T.P., Kindermann, S.S., Bell, A.J. and Sejnowski, T.J., "Analysis of fMRI by decomposition into independent spatial components", Human Brain Mapping , 6 (3), 160 – 188, 1998 11. Kwak, K.C., "Face recognition using an enhanced Independent Component Analysis approach", IEEE Trans. on Neural Networks, 18 (2), 530 - 541, 2007 12. Li, S.Z., et al., “Learning multiview face subspace and facial pose estimation using Independent Component Analysis”, IEEE Trans. Image Processing, 14 (6), 705 - 712, 2005 13. Bottiglieri, M., et al., "Independent Component Analysis as a tool for ground deformation analysis", Geophysical Journal International, 168 (3), 1305 - 1310, 2007 14. Hui he, T. Zhang, Xian-chuan Yu, Wang-lu peng, “Application of fast Independent Component Analysis on extracting the information of remote sensing imagery”, Proceedings of the Fifth International conference on Machine Learning and Cybernetics, Dalian, 1066 - 1071, 2006 15. Wehr, S., et al., "Post processing for BSS algorithms to recover spatial cues," IWAENC 2006 Paris – September 12 – 14, 2006. 16. Everson, R. and Roberts, S., "ICA: A flexible non-linearity and decorrelating manifold approach", Neural Computation.11, 1957-1983, 1999 17. Amari, S. and Cichocki, A. "Adaptive blind signal processing- Neural Network approaches". Proc. of the IEEE, 86 (10), 2026 - 2048, 1998 18. Giannakopoulos, X., Karhunen, J., and Oja, E., “Experimental comparison of neural algorithms for independent component analysis and blind separation”, Int. J. of Neural Systems, 9 (2), 651 - 656, 1999

-12-

Egyptian Computer Science Journal 19. Hyvarinen, A., “One-Unit contrast functions for independent component analysis: A statistical analysis”, Proc. IEEE workshop on Neural Networks for Signal Processing, VII, 388 - 397, 1997 20. Goneid, A., Kamel, A., and Farag, I., “An analysis of non-linearities in neural ICA algorithms”, Egyptian Computer Science Journal, to appear in January issue, 2010 21. Douglas, S.C., Cichocki, A., and Amari, S., “Multichannel blind separation and deconvolution of sources with arbitrary distributions”, IEEE Workshop on Neural Networks for Signal Processing, 436-445, 1997 22. Choi, S., Cichocki, A. and Amari, S., “Flexible Independent Component Analysis”, Proc. of the IEEE Workshop on Neural Networks for Signal Processing (NNSP’98), 83-92, 1998 23. Choi, S., Cichocki, A. and Amari, S., “Flexible Independent Component Analysis”, Journal of VLSI Signal Processing, 26(1/2), 25-38, 2000 24. Abu-Taleb, A., Zayed, E., El-Sayed, W., Badawy, A., and Mohamed, O., “Multichannel blind deconvolution using a generalized Gaussian source model”, Mathematical and Computational Applications, 12(1), 1-9, 2007 25. Lee, T., Girolami, M., and Sejnowski, T., “Independent Component Analysis using an extended Infomax algorithm for mixed subgaussian and supergaussian sources”, Neural Computation, 11, 417-441, 1999 26. Mineo, A., and Ruggieri, M., “A software tool for the Exponential Power Distribution: The normalp package”, J. Statistical Software, 12(4), 1-24, 2005 27. Cardoso, J.-F., and Laheld, B., “Equivariant adaptive source separation”, IEEE Trans. on Signal Processing, 45(2), 434-444, 1996 28. Hyvarinen, A., “Fast and robust fixed-point algorithms for independent component analysis”, IEEE Trans. on Neural Networks, 10(3), 626-634, 1999. 29. Goneid, A., Kamel, A., and Farag, I., “New convergence and performance measures for Blind Source Separation algorithms”, Egyptian Computer Science Journal, 31 (2), 13 -24, 2009 30. Amari, S.I, Cichocki. A and Yang. H., "A new learning algorithm for Blind Source Separation", Advances in Neural Information Processing Systems, MIT Press, Cambridge, MA, vol. 8, pp. 757 – 763, 1996. 31. Yang, H., and Amari, S.I., “Adaptive on-line learning algorithms for blind separation: Maximum entropy and minimum mutual information”, Neural Computation, 9(7), 1457 1482, 1997.

-13-