non-linear mixing functions prove the efficacy of the proposed algorithm when compared to ... as a mean to measure the efficacy of the separation performance.
Neural Network based Blind Source Separation of Nonlinear Mixtures Athanasios Koutras, Evangelos Dermatas and George Kokkinakis WCL, Electrical and Computer Engineering, University of Patras 26100 Patras, HELLAS.
NRXWUDV#JLDSLZFOHHXSDWUDVJU
Abstract. In this paper we present a novel neural network topology capable of separating simultaneous signals transferred through a memoryless non-linear path. The employed neural network is a two-layer perceptron that uses parametric non-linearities in the hidden neurons. The non-linearities are formed using a mixture of sigmoidal non-linear functions and present greater adaptation towards separating complex non-linear mixed signals. The network’s weights and the non-linearities parameters are estimated adaptively using the Maximum Likelihood Estimation criterion. Simulation results using complex forms of non-linear mixing functions prove the efficacy of the proposed algorithm when compared to similar networks that use standard non-linearities, achieving excellent separation performance and faster convergence rates.
1 Introduction The problem of Blind Source Separation (BSS) primarily consists of recovering a set of statistically independent signals or sources given only a number of observed mixture data. The term ''blind'' is justified by the fact that the only a-priori knowledge that we have for the signals is their statistical independence and no information about the mixing model parameters and the transfer paths from the sources to the sensors is available beforehand. The BSS problem has been intensively studied for the linear instantaneous (memoryless) mixture case [1,2] and more recently for the linear convolutive case [1,3] showing very good and promising results. However, up to now, presentation of algorithms that deal with more realistic mixture models, especially non-linear, has been very limited. For the non-linear mixture case, the linear BSS methods are not applicable and fail to extract the independent sources therefore new non-linear BSS (NLBSS) methods must be introduced. A simple approach towards solving the NLBSS problem would be the implementation of a non-linear function that would transform the mixtures and make them statistically independent. Still, such a solution although clear and straightforward, would be difficult - if not impossible - to be applied without properly limiting the function class for the de-mixing transforms. It is known that any random variable with a continuous probability density function (pdf) can be transformed to a uniformly distributed random variable (using more than
one transformations) and multiple uniform random variables are always independent. In this case we would have statistically independent outputs that tell nothing about the original sources. Under these facts, to limit the function class for the de-mixing functions is equivalent to assume some prior knowledge about the mixing function. In the last years a few algorithms that deal with the NLBSS problem have been presented, some of them based on neural networks [4-6]. Lee et al. [7] have addressed the problem of non-linear separation based on information maximization criteria using a parametric sigmoidal non-linearity and higher order polynomials. Taleb and Jutten [8] have dealt with a special case of post non-linear mixtures (PNL) using entropy minimization criteria. Finally, Yang et al. [9] have proposed a two-layer perceptron with parametric sigmoidal non-linearities to separate non-linear mixtures. In this paper, we present a neural network based approach towards solving the nonlinear signal separation problem of unknown sources. The employed neural network is a two-layer perceptron that uses a generalized form of a mixture of parametric sigmoidal non-linearities in the hidden neurons. The proposed form of these non-linearities presents better fitting manner when dealing with more complex non-linear mixing functions, showing greater speed of convergence and satisfactory separating performance. The separation network’s learning rules have been derived using the Maximum Likelihood Estimation (MLE) criterion and show great efficacy when used to separate non-linearly mixed signals. The structure of this paper is as follows: In section 2, we present the general nonlinear BSS model and derive the learning rules for the neural networks parameters using the MLE criterion. Additionally, the network’s performance index is introduced as a mean to measure the efficacy of the separation performance. In section 3 our experiments are presented and finally in the last section some conclusions and remarks are given.
2
Non-linear Blind Source Separation
Let us suppose we have N statistically independent sources {s1, s2,..,sN} that are mixed by a non-linear memoryless model. The mixed signals {x1,x2,..,xM} are formulated by the following equation: x(t)=f(s(t)),
(1)
where f is the unknown non-linear mixing function. The task of the NLBSS is to recover the N original sources without any prior knowledge about the mixing matrix, the non-linear function f or the original signals s(t). However, without any knowledge about the mixing function f, the problem of recovering the source signals using only statistical independence criteria cannot be considered efficient for separation without distortions. Particularly, consider two independent random variables X and Y and f1, f2 two bijective derivable functions. The random variables f1(X) and f2(Y) are also independent which leads us to the conclusion that the non-linearly mixed sources can be recovered not only up to the scaling and permutation factor, but up to any non-linear function as well. This indeterminacy will probably lead to strong distortions that
damage the quality of the separating signals. So in general, NLBSS is impossible using only the sources independence assumption without any prior knowledge about the non-linear functions. x 1 (t )
W
u 1 (t )
(2 ) 11
W
. . .
v 1 (t ) + W
(2 ) 12
K
(1 ) 11
W
g ( u ) = ∑ tanh(a i ⋅ u + b i )
(1 ) 12
i =1
W W
W
(2 ) 21
(2 ) 22
+
x N (t )
W
J
u N (t )
+ ,' ' (1 /$< (5 )LJ
y 1 (t )
J
+
(1 ) 21
. . .
(1 ) 22
+
v N (t )
y N (t )
2 8 7 387 /$
