IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL 44, NO. 8, AUGUST 1996
2055
System Identification Using Nonstationary Signals Ofir Shalvi and Ehud Weinstein, Fellow, IEEE
Abstract-The conventional method for identifying the transfer function of an unknown linear system consists of a least squares fit of its input to its output. It is equivalent to identifying the frequency response of the system by calculating the empirical cross-spectrum between the system’s input and output, divided by the empirical auto-spectrum of the input process. However, if the additive noise at the system’s output is correlated with the input process, e.g., in case of environmental noise that affects both system’s input and output, the method may suffer from a severe bias effect. In this paper we present a modification of the cross-spectral method that exploits nonstationary features in the data in order to circumvent bias effects caused by correlated stationary noise. The proposed method is particularly attractive to problems of multichannel signal enhancement and noise cancellation, when the desired signal is nonstationary in nature, e.g., a speech or an image.
I. INTRODUCTION
E CONSIDER the following model
multichannel system identification and signal reconstruction 161, 191. As an example, consider the following model:
+
.(t) = s ( t ) u ( t ) y ( t ) = h(t)0 s ( t )
+w(t)
where in the multichannel signal enhancement and noise cancellation problem, z ( t ) and y ( t ) represent the signals measured by the primary and reference sensors, s(t) is the desired signal (e.g., speech), h(t) represents the coupling of the desired signal to the reference sensor, and u(t) and w ( t ) are the additive sensor noises. In passive time delay estimation z ( t )and y ( t ) represent the signals observed at the two sensor outputs, where h ( t ) represents the relative delay between the two propagation paths. It is easy to verify that ~ ( tand ) y(t) in the model of ( 2 ) and (3) satisfy (1) where,
v(t) = w(t) - h(t)0 u(t).
+ v(t)
y(t) = h ( t )0 z ( t )
where h ( t ) represents the impulse response of a linear timeinvariant (LTI) system that we want to identify, the symbol o denotes the convolution operation, z ( t ) and y ( t ) are the observed input and output processes, and v(t) represents additive noise, modeling errors, etc. The conventional method for identifying the unknown system consists of a least squares fit of its input to its output. Under stationary conditions it is equivalent to identifying the frequency response of the system by calculating the cross-spectrum between y ( t ) and z ( t ) ,divided by the auto spectrum of z ( t ) . If w(t) is a zero-mean Gaussian process that is statistically independent of z ( t ) then this method is asymptotically efficient, i.e., it is asymptotically unbiased and its error variance approaches the Cramer-Rao lower bound (see [4]). However, if v(t) is correlated with z ( t ) ,the crossspectral method may suffer from a severe bias effect. ) v(t) may occur in Statistical correlation between ~ ( tand a variety of application contexts including identification of error-in-variable models [3], [4], 161, identification of systems with feedback [ 11, passive time-delay estimation [ 5 ] , and Manuscript received May 6, 1994; revised October 20, 1995. This work was supported in part by the Wolfson Research Award administrated by the Israel Academy of Science and Humanities at Tel-Aviv University, in part by the Wolfson Foundation, in part by the Fulbright Foundation, and in part by the Charles Clore Foundation. The associate editor coordinating the review of this paper and approving it for publication was Prof. Daniel Fuhrman. 0. Shalvi is with Libit Signal Processing, Herzelia 46766, Israel (e-mail:
[email protected] 1). E. Weinstein is with the Woods Hole Oceanographic Institution, Woods Hole, MA 02543 USA. Publisher Item Identifier S 1053-587X(96)05287-7.
(2) (3)
(4)
Since both z ( t ) and v(t) contain a component of u(t) they are generally statistically correlated even if u(t) and w(t) are uncorrelated. In addition, u(t) and w(t) may also be statistically correlated if the observed signals are contaminated by the same noise source, e.g., an environmental noise such as a directional interference. In [l], [ 5 ] , [6] it is suggested to identify the system using high-order statistics (HOS). If the joint statistics of z ( t ) and v ( t ) are Gaussian while the statistics of z ( t )are non-Gaussian then these methods yield a bias-free estimate of the system’s transfer function. However, small error analysis presented in [6] indicated that in using these methods lack of bias is compensated by an increase in error variance. Another approach that is designed to overcome the bias problem is the instrumental variable approach [4], [SI. This method is based on an instrument process, that is a process which is uncorrelated with v(t) but correlated with the observed signals. However, in many cases the attempt to generate an instrument that is uncorrelated with the noise may yield an instrument that is weakly correlated with the observed signals, which in turn yields poor results. In this paper, we reconsider the problem under a presumably less restrictive set of assumptions. Specifically, we assume that the correlation function of z ( t )is nonstationary, but the crosscorrelation of z ( t ) with v(t) is stationary. It is easy to verify that these assumptions are satisfied for the model in (2) and (3) if s ( t ) is a sample function from a nonstationary process (e.g., speech) while u(t) and w(t) are sample functions from jointly stationary processes that are statistically independent of s(t). More generally, this will be the case with the model
1053-587X/96$05.00 0 1996 IEEE
IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 44, NO. 8 , AUGUST 1996
2056
in (1) if the correlation between z ( t )and v(t) results from the stationary components of these signals. As we shall see, by exploiting the nonstationarity it becomes possible to obtain a bias-free estimate of the system’s transfer function with only insignificant increase in error variance.
To do that, let the observation interval be divided into M subintervals. Then, similar to (6), for each subinterval we obtain S$’(w)
11. THE PROPOSED METHOD Invoking (l),
= H(w)S$;)(w)
+ S$F)(w) + S,,(w) + d m ) ( w ) ,
= H(w)LQ)(w) m = 1,2,...,M
(10)
dm)w = S & q W )
(1 1)
where
where hap( T ) is the empirical cross-correlation (autocorrelation when a = p), defined by 1 Rap(7) = ____ -
min{T,T--.r}
4 t + .)P*(t)
(6)
t=max{l,l--.T)
where * denotes the complex conjugate. In case of continuoustime signals the summation is replaced by the appropriate integration. Define the empirical cross-spectrum by
- Svz(w).
Since z ( t ) is assumed to be nonstationary, its empirical auto-spectrum may vary significantly from one subinterval to another, generating linearly independent equations. Thus with M = 2 we obtain two equations that can be used to estimate both H ( w ) and SvZ(w). With M > 2 we obtain an overdetermined set of equations, and we may use a weighted least-squares (WLS) approach to solve for the two unknown quantities. Concatenating the equations in (lo), we obtain
(7) where w ( r ) is a pre-selected window function. Ignoring truncation effects due to the finite length of the window. Sy&)
= %4%&4
+ Swx(w).
(8)
If Swr(w)x 0 ‘d w , then
H(w)
” s,,( wo) SYZ
Then, the WLS estimate of 0 is given by
(9)
provided that S c z ( w ) # O’dw. This is, in fact, the classical = (A+wA)-lA+wz (13) cross-spectral method for identifying H ( w ) . However, if v ( t ) and z ( t ) are statistically correlated then Swvp(w)may be whereW is a positive Hermitian matrix, denotesnthe consignificantly different from zero, in which case the ratio jugate transpose, and where we have assumed thatA+WA is Syr(w)/Szr(w) does not yield the correct H ( w ) . This is the invertible. bias problem associated with the cross-spectral method. A possible choice for the weighi matrix is Let z (t) and v ( t )be possibly correlated random processes, but suppose that their cross-correlation is stationary, while the autocorrelation function of z ( t ) is nonstationary. This may be the case if z ( t ) is a nonstationary process, but the so that the longer subintervals obtain higher weights. correlation between z ( t ) and v(t) results only from stationary The choice of W that minimizes the error variance under components. We claim that nonstationarity, which is usually certain assumptions will be given in the following section. regarded as an undesirable feature, is in fact a very useful feature. It implies that by dividing the observation interval to 111. ANALYSIS several subintervals, the joint statistics of z ( t ) and y ( t ) may To analyze the asymptotic bias and error variance of the vary significantly from one subinterval to another, providing additional information that can be used to identify both the proposed method, suppose that the observation interval T is unknown system and the cross-correlation between z ( t ) and divided intoMM disjointed subintervals T,, m = 1 , 2 , . . . , M , = T , and let us make the following v ( t ) and by that to circumvent the indicated bias effect. The so that ~ , = , T , assumptions. idea was first presented in [7]. A different approach that 1) Observations of z ( t ) associated with disjointed subinexploits nonstationarity in the limited context of the dualtervals are statistically independent. channel signal separation problem is mentioned in [91.
+
SHALVI AND WEINSTEIN: SYSTEM IDENTIFICATION USING NONSTATIONARY SIGNALS
2) Over each subinterval x ( t ) is a wide-sense stationary (WSS) zero-mean process with spectral density S g ' ( w ) ,m = 1 , 2 , . . * ,M . 3) ~ ( tis)a WSS zero-mean process with spectral density
Thus
2057
COV(€)
is the diagonal matrix:
Svv(w). Substituting (12) into (13), we obtain
Substituting (21) into (17), we obtain
where we define: and where in the transition to the second line of (15) we have assumed that S i p ' ( w ) , m = 1 , 2 , . . . , M are sufficiently close to S$z)(u)so that to a first order approximation A can be substituted by A . If the bandwidth of the window function is sufficiently narrow, and it normalized so that w (0) = I, then Sig)( U ) is an unbiased estimate of S,,(U) (see Appendix A), and hence,€ is zero mean and 8 is an asymptotically unbiased estimate of 8. In particular, we assert that the method yields an asymptotically unbiased estimate & ( U ) of H ( w ) . Squaring (15) and taking expectation, the error covariance is given by
M
E
=
(.(U))
+&)(U).
m=l
The estimator in this case is obtained by substituting W = [ C O V ( ~ ) ] - ~into (13), where since the actual spectra Si;)((w) are not available, they are replaced by their sample estimates S$;)((w). The result is:
1
= (szZ(w))(l/Szz)- 1
is!,.
(1@ZZ
(U))
( S m(4) (&Z
(U) - (SYZ )A
(U)
( w )/@z
(U>>
/SZZ ( U ) ) - (SVZ (U)) (24)
= (A+WA)-~A+W~~~(€)W+A(A+WA (17) )-~.
It can be verified (the Gauss-Markov theorem) that the choice of W that minimizes the right hand side of (17) is given by
W = [COV(E)]-1
(18)
in which case COV(
8) = (A+[cov(e)]-'A)-'.
By [2, Th. 7.4.31, the elements of totically by (see Appendix A):
CO.(.)
(19)
The error covariance in (22) may also be calculated from the data by replacing SzZ(w)by SzZ(w),and where instead of Sww(y) we use the empirical spectrum S c ~ ( w of ) 6(t) =
?At)- h ( t ) 0 4 t ) . For the purpose of comparison we note that under the assumptions (1)-(3), the bias and error variance of the conventional cross-spectral method that ignores statistical correlation between ~ ( tand ) w(t) are given, respectively, by (see Appendix B):
are given asymp-
where by using the proposed method
E(&(w)} - H ( w ) = 0 and
for 0 < w < T , where
B=
1
CW"4 T
is related to the window's bandwidth. We note that lack of correlation between the spectral estimates, as indicated by the second line of (20), results from assumption (a) that disjointed time segments of z ( t ) are statistically independent.
The ratio between the variances (relative efficiency) is
(27)
IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 44, NO. 8, AUGUST 1996
2058
Thus in using our method lack of bias as indicated by (27) is compensated by an increase in error variance as indicated by (29). is close to 1 then p becomes very If (Szx(u))(l/Sxz(u)) large, indicating a very significant increase in error variance. We note that if Sg)(u)= SE) ( w ) = . . . = SLT)((w)then (Szz(w))(l/Szz(w)= ) 1 and the variance is infinite. This is since in this case z(t) is effectively a stationary process, in which case ~ ( tand ) y ( t ) become jointly stationary, all the equations in (10) are nearly the same, and the problem of estimating both N ( w ) and Svz(u)becomes ill-conditioned. On the other hand, if (Szx(w))(l/Szx(w)) >> 1, indicating that the data are nonstationary, p x 1 and the proposed method yields a bias-free estimate with only insignificant degradation in error variance. To incorporate the effects of bias and variance, we may consider the mean squared error (MSE). In case of our method the MSE is equal to the variance given by (28). In case of the conventional method it is given by
It corresponds to the model given by (2) and (3) where the unknown system is a pole-zero, or ARMA, system. It can be represented in the form of (32) where X ( t ) = (Y(t
11,. . , Y(t - PI,4 t ) ,4 t - I), . ' . > 4 i - 4 ) Y (36) (37) . . , u p , h ,b 2 , * . ,b J T
-
cp = (Ul,
'
and where P
C
v ( t ) = ~ ( t- )
+
MSE{I?(~)} = l ~ { i f ( w ) -) H ( W ) I ~ var{&(w)]
Q
Q W ( ~-
k=l
IC) -
C b k u ( t - IC).
(38)
k=l
Since both v(t) and ~ ( tcontains ) components from both u(t) and w(t), they bound to be statistically correlated. Consewhere rnz(u) is the coherence function between v(t) and ~ ( t ) ,quently, the conventional least squares approach may suffer from a severe bias effect (e.g., see [4]). defined by To apply our method, we shall assume, in complete analogy, snz ( w ) t )nonstationary, while ~ ( tand ) u(t)are jointly WSS that ~ ( is (31) = processes. These assumptions are satisfied for the model given J&&J)(SZX(4) by (33)-(35) if s ( t ) is a nonstationary signal (e.g., speech), If BT >> 1, i.e., large bandwidth-time product, the bias term while u(t) and w ( t ) are jointly WSS processes. may be dominant, and the overall MSE of the conventional Invoking (32), cross-spectral method may be drastically inferior to that of the proposed method. This will be demonstrated in the simulation section.
rVzw
Iv.
IDENTIFICATION OF
LINEARREGRESSION MODELS
In this section we shall present an equivalent time-domain approach for the identification of the parameters of linear regression models of the form:
Y(t) = X T ( t ) c p
+v(t)
(32)
where y(t) and v(t) are as in the model given by (l),~ ( tis ) the observed regression vector, and cp is a vector of unknown regression coefficients (system parameters) that we want to identify. This model covers a variety of commonly used models [4]. For example, consider the model illustrated in Fig. 1 in which the observed signals ~ ( tand ) y ( t ) are given by
z(t) = s ( t )
+ u(t)
(33) (34)
Y ( t ) = r ( t )+ w ( t )
where s ( t ) and ~ ( tsatisfy ) the following difference equation: T(t)
=
UkT"(t
k=l
- k)
+
bkS(t
k=O
t=1
- k).
(35)
-
t=l
As in the previous development, the key idea is to divide the observation interval into M subintervals and invoke the nonstationarity of the data. Similar to (IZ), we obtain the following vector equations:
SHALVI AND WEINSTEIN: SYSTEM IDENTIFICATION USING NONSTATIONARY SIGNALS
2059
0.8
0.7
i
0.61
' . ' . proposed method
frequency [radians] Fig. 2. Magnitude of estimation errors in 20 different trials.
Then the WLS estimate of 0 is given by
e = (A+wA)-lA+wz.
(41)
Once again, the optimal weight matrix is given by (18), in which case the error covariance is given in (19). Explicit expression for the error covariance matrix can also be derived here. Recursive and sequential time-domain algorithms can be obtained by computing (41) recursively. We may also incorporate exponential weighting, in which case we may obtain adaptive algorithms that are capable of identifying and tracking time-varying parameters. V. SIMULATION RESULTS
To verify the results of the analysis, we have considered the model in (2) and (3), rewritten here for reference:
z ( t ) = s ( t ) + .(t) y ( t ) = h ( t )0 s ( t ) w ( t )
+
where, in addition,
where g ( t ) is the unit sample response of some LTI system. This may be the case when u ( t ) and w(t) result from environmental noise source, e.g., a directional interference. The first set of experiments was performed on synthetic signals generated by the computer. The observation interval was divided into 111 = 5 subintervals, each having 1000 data points. The signal s ( t ) was a quasi-stationary moving average (MA) process, generated by
s ( t ) = n(t)+ b(")n(t - 1)
where n(t)was a computer generated normalized (zero-mean, unit-variance) white Gaussian process, and b(") = m - 3 is the MA parameter associated with the mth subinterval. The signal u(t) was also a computer generated normalized white Gaussian process, and
h ( t )= 2 6 ( t ) , g ( t ) = - S ( t ) where 6 ( t ) is the Kronecker delta function (Le, memoryless constant gain systems). The window function used to generate the spectral estimates was a Bartlett (triangular) window with 7 taps. In Fig. 2 we have plotted the magnitude of the estimation error I f i ( w ) H ( w ) ( of the conventional method given by (9), and of the proposed method given by (24), as calculated using 20 different trials. As we can see, the proposed method yields smaller errors in all trials. In Fig. 3 we have plotted the magnitude of the empirical bias, that is the magnitude of the error averaged over 150 independent trials, and in Fig. 4 we have plotted the empirical mse. The upper and lower dotted lines in Fig. 4 represent the approximate analytical results as given by (28) and (30), respectively. As one can see, the proposed method is essentially unbiased while the conventional method suffers from a severe bias effect and therefore the mse of the proposed method is a factor of 30-100 lower than the mse of the conventional approach. The good fit between the analysis and experiment indicates fast convergence to the asymptotic results. We have also applied the algorithm obtained by using the weight matrix (14), in which the weight of each subinterval is its duration, in the estimate proposed by (13), and the results were close to those obtained using the algorithm in (24) with the optimal weight matrix. In the second set of experiments, s ( t ) was a speech signal (male speaker) sampled at 8 kHz, u(t) is a stationary zero-
IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 44, NO. 8, AUGUST 1996
2060
_________________________-_-----
conventional method proposed method
10'.
IO2:
1o
-~
C f O Q Q0 8
,
I
I
I
e
I
I
I
€3-0-0-Q-L38800 0 0 4 3 ~ ~ - 0 4 + ~ 5 @ @ @
I
I
!
I
Fig. 4. Mean square estimation error, E{I&(w) - H ( W ) ~ ~ } .
mean Gaussian process whose average power is a factor of 2 larger than the average power of the speech. In Fig. 5 we have plotted the empirical spectrum of s ( t ) and the empirical both averaged Over the spectrum Of data segment, to show that the signal and the noise share the Same frequency band. The impulse responses of the linear systems in these
experiments were:
+
h(t) = S ( t ) O.lS(t - 1) + 0.2S(t - 2) g ( t ) z= - S ( t ) - 0.5S(t - 1)+ O.lS(t - 2) In Fig. 6 we have plotted the observed signals ~ ( tand ) y(t). we see, the noise level is so high that one hardly recognize the presence of speech in the measured signals.
SHALVI AND WEINSTEIN: SYSTEM IDENTIFICATION USING NONSTATIONARY SIGNALS
2061
empirical spectra of speech and noise signal
500
1000
1500
2000
2500
3000
4000
3500
frequency [Hz] Fig. 5. Empirical spectra of speech signal and noise signal.
We have applied the conventional algorithm (see (9)) and the nonweighted algorithm obtained by substituting (14) into (13) using a 5-s observation interval (i.e., 40 000 data samples) that was arbitrarily divided into disjointed subintervals, each having 128 data samples. To generate the spectral estimates, we have used a Bartlett window with 15 taps. The ultimate objective in this example is to estimate, or recover, the speech signal. In Appendix C we develop the linear unbiased minimum variance estimate of s ( t ) based on the two-channel data and the estimate of the unknown system. The result is
q t ) = Z ( t ) + f ( t )0 6 ( t )
(42)
6 ( t ) = y(t) - h(t)0 z ( t )
(43)
noisy obervalion x(1) a
7 EO a E
I 0 05
0 15 Time noisy ObeNallOn y(1)
01
0.2
2r
0 25
I
I
0
0 05
0.1
0.15
0.2
0.25
Tme
Fig. 6. The observed signals z ( t ) and y ( t ) .
where
and f ( t ) is the unit sample response of the filter whose frequency response is
32%(U)
F ( w )= -___ $66 ( U )
(44)
VI. CONCLUSIONS We have presented an approach for system identification that exploits the nonstationarity of the data in order to circumvent the bias effect caused by correlated stationary noise. The method can easily be extended to MIMO system, and to multidimensional signals, e.g., images. We may also incorporate higher order statistics, as suggested in [7].
In Fig. 7(a) we have plotted the original signal for reference, in Fig. 7(b) we have plotted the reconstructed signal when h(t) is obtained using the proposed method, and in Fig. 7(c) we have plotted the reconstructed signal using the conventional APPENDIX method. THEASYMPTOTIC BIASAND VARIANCE OF .(")(W) As one can see, the proposed method yields a signal estimate that is very close to the original speech. By actually Under the asymptotic conditions stated in ( [ 2 ] ,Th. 7.4.1) listening to the reconstructed signal we hear a very significant improvement in speech clarity and intelligibility. Finally, we note that faster convergence can be achieved by dividing the observation interval more carefully so that where W ( Y )= C, w(T)e-Jv.'. Assuming that the bandwidth of the spectral window is sufficiently narrow and that Suz(w) different subintervals correspond to different phonemes.
IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 44, NO 8, AUGUST 1996
2062
APPENDIXB DERIVATION OF (25) AND (26) The conventional cross-spectral method is given by: I
0
01 0 15 Time [Seconds]
0 05
02
0 25
2(w)
%X ( w ) = ___
(B.1)
s x x (w)
(a)
Substituting (8) into (A.1),
1 a,
3 f
g
2 5
1
Time (Seconds]
Thus
(b)
E{swx(u)}
E { H ( w ) }- H ( w )
I
(B.3)
E{ s x x (U)}
1 m
var{ svz (U)I-
TJ 2
5 0
var{B(w)}
E
4
"N
E2{
sm (U)>
*
(B .4)
5
1
Now,
Time [Seconds]
(c)
F I ~ 7 The restored signals versus the onglnal slgnal (a) Onglnal speech signal (b) Signal restored by the proposed method (c) Slgnal restored by the conventional method.
T
1
)&?I
z(t
=-
T
+
T).*(t,
t=l
M
is continuous, we may approximate
T,
m=l
Tm
T
t=l
E { d m ) ( w ) } 2 E{S$:)(w)} - SVX(W)
= ( w ( 0 )- l)Svx(u) =0
(A.1)
provided that w (0) = 1.Thus the proposed method is unbiased up to truncation effects. Under the asymptotic conditions stated in [2, Th. 7.4.31,
. W(W2 + v)Svx:(v)S,*,(~) dvl.
(A.2)
Assuming that the bandwidth of the spectral window is sufficiently narrow, (A.2) is asymptotically approximated by [2, Cor. 7.4.31:
where d m ) ( t )represents
~ ( t over ) the mth subinterval. M
,!?xx(u) = w(T)k,,(r)e-3wT =
Tm ?S:i)(w)
(B.6)
Similarly, L ( T )
1
T
=r
v(t
+
T)Z*(t)
t=l
+V(Wl
where
+
I
w2)/Svx(w1)I2
1, lw[= 0 , 2 ~4, ~. . , 0, elsewhere
Hence M ,!?Vx((w)
and (20) follows.
w(r)k,x(T)e-3wT =
= T
m=l
Tm y S , $ T ) ( w ) (B.9)
SHALVI AND WEINSTEIN: SYSTEM IDENTIFICATION USING NONSTATIONARY SIGNALS
2063
Thus invoking (7) and (lo), and assumption (a) that observations of%@)associated with disjoint sub-periods are statistically independent,
where in the transition to the second line of (C.8) we have used (1). The estimate in (C.8) is now approximated by replacing v ( t ) by C ( t ) given by (43) and by replacing the spectral functions ( ~ . 1 0 ) appearing in (C.7) by the corresponding empirical spectra.
m=l
REFERENCES
M m=l
M
=c m=l
M
-
[l] H. Akaike, “On the use of non-gaussian process in the identification of a linear dynamic system,” Ann. Inst. Statist. Math., vol. 18, pp. 269-276, 1966. [2] D. R. Brillinger, Time Series, Data Analysis and Theory. San Francisco, CA: Holden-Day, 1981, [3] M. Diestler, “Linear dynamic error-in-variables models,” 1986. Englewood Cliffs, [4] L. Ljung, System Identification Theory . ”for the User. NJ: Pre%ice-Hall, 1987. [SI C. L. Nikias and R. Pan, “Time Delay Estimation in Unknown Gaussian Spatially correlated Noise,” IEEE Trans. Acoust. Speech, Signal Processing, vol. 36, Nov. 1988. [6] 0. Shalvi and E. Weinstein, “Higher order cross-covariance methods for system identification with non-gaussian inputs,” Tech. Rep. EE-S-92-10, Tel-Aviv Univ. Faculty of Engineering, Tel-Aviv, Israel, July 1992. “System Identification and signal separation based on non[7] -, stationarity,” Tech. Rep. EE-S-93-14, Dep. Elect. Eng.-Syst., Faculty of Engineering, Tel-Aviv Univ.. Israel, Aug. 1993. [8] T. Soderstrom and P. Stoica, “Comparison of instrumental variable methods-Consistency and accuracy aspects,” Automatica, vol. 17, pp. 101-115, 1981. [9] E. Weinstein, M. Feder, and A.V. Oppenheim, “Multi-channel signal separation based on decorrelation,” IEEE Trans. Speech Audio Processing, vol. 1, Oct. 1993. ~~
(B.ll) Substituting (B.7), (B.10), and (B.ll) into (B.3) and (B.4), we obtain (25) and (26), respectively.
APPENDIXC SIGNAL ESTIMATION We consider a linear estimate for s ( t ) of the general form:
+
a(t) = k ( t ) 0 z ( t ) f ( t ) 0 y ( t )
(C.1)
Using (2) and (3),
+
S(t) = [ k ( t ) f ( t ) 0 h(t)]0 s ( t ) + O u(t)+ f ( t )0 4 t ) l .
(C.2)
If u(t) and w ( t ) are statistically independent of s ( t ) , then i ( t ) is conditionally unbiased estimate of s ( t ) provided that k(t)
+f(t)
0
h ( t )= S ( t )
(C.3)
in which case
e(t) =S ( t ) = .(t)
- s ( t ) = k ( t ) o u(t)
+f(t)
0
+ f ( t )o w ( t )
v(t)
(C.4)
where v(t) is defined by (4). The resulting mean square estimation error is 1
E { e 2 ( t ) }= 2:
Ofir Shalvi was born in Israel in 1963 He received the B.Sc. degree in physics and mathematics in 1984 from the Hebrew University, Israel, and the M.Sc. and Ph.D. degrees in electncal engineering from Tel-Aviv University, Israel, in 1988 and 1995, respectively. During the years 1984-1989, he served in an R&D unit in the Israeli Defense Force. During 1995 be was in the Department of Electncal Engineering and Computer Sciences in Massachusetts Institute of Technology, Boston, where he did his Post-Doctoral research. He is a co-founder of Libit Signal Processing Ltd., where he is currently a vice president for R&D His research interests are in the fields of digital communications and signal processing. Dr. S h a h has received the Israeli Ministry of Communications Research Award, the Charles Clore Fellowship, the Wolfson Fellowship, and the Fnlbright Fellowship.
r+r
IT
S,,(w) dw
where
Ehud Weinstein (M’82-SM’86-F’94) was born in
+
+
See(w)= Suu(w) 2 R e { F ( w ) S u u ( w ) } IF(w)12S,,(~)
(C.6)
The choice of F ( w ) that minimizes See(w)‘d(w),and therefore minimizes E { e 2 ( t ) }is:
Substituting (C.3) into (C.l)
q t ) = 4 t ) + f ( t ) 0 Mt>- h ( t )O 4 t ) l = z ( t ) f ( t )0 v ( t )
+
(c.8)
Tel-AVIV,Israel, on May 9, 1950 He received the B.Sc degree from the Technion-Israel Institute of Technology, and the Ph.D degree from Yale University, New Haven, CT, both in electrical engineermg, in 1975 and 1978, respectively. In 1980 he joined the Department of Electncal Engineering-Systems, Faculty of Engineering, TelAviv University, where is a Professor. Since 1978 he has been affiliated with the Woods Hole Oceanographic Institute He IS also a Research Affiliate in the Research Laboratory of Electronics at the Massachusetts Institute of Technology since 1990. H i s research interests are in the general areas of estimation theory, statistical signal processing, array processing, and digital communications. Dr. Weinstein is a co-recipient of the 1983 Senior Award of the IEEE Acoustics, Speech, and Signal Processing Society.
