Chapter 2 - Discrete Time Signals and Systems

2. Discrete Time Signals and Systems • We will review in this chapter the basic theories of discrete time signals and systems. The relevant sections from our text are 2.0-2.5 and 2.7-2.10.

• The only material that may be new to you in this chapter is the section on random signals (Section 2.10 of Text)

2.1 Discrete Time Signals • A discrete-time (DT) signal is signal that exists at specific time instants. The amplitude of a discrete-time signal can be continuous though. • When the amplitude of a DT signal is also discrete, then the signal is a digital signal. • A DT signal can be either real or complex. While a real signal carries only amplitude information about a physical phenomenon, a complex signal carries both amplitude and phase information. • Throughout this course, we use square brackets [•] to denote a DT signal and round brackets ( g ) to denote a continuous time signal. Example: If the n-th sample of the DT signal x[n ] is the value of the analog signal xa (t ) at t = nT , then x[ n] = xa ( nT )

2-1

• Some common DT signals are 1. Unit sample

1 δ [ n] =  0

n=0 otherwise

2. Unit step

1 u[ n] =  0

n≥0 n 0 and 0 < α < 1 , then x[n ] decreases as n increases; see figure below.

4. Sinusoidal

x[n] = A cos ( ω o n + φ ) where A is the amplitude, ω o is the frequency, and φ is the phase.

5. Periodic

x[n + N ] = x[n] 2-3

for any time index n . Here N denotes the period.

• Exercise: Is the sinusoidal signal defined above periodic in general?

• Example: Express the unit step function in terms of the unit-impulse Answers: (a)

u[n ] − u [n − 1] = δ [n ] ∞

(b)

u[ n] = ∑ δ [ n − k ] k =0

• Example: Express

1 x[ n] =  0

n = 2,3,...,10 otherwise

in terms of the unit step function. Answer:

x[ n] = u[ n − 2] − u[n − 11]

2-4

• Example: Express the sinusoidal signal in terms of complex exponential signals Answer:

x[n] = A cos ( ω o n + φ ) =A

e

j(ω o n +φ )

+e 2

− j(ω on +φ )

= A1 (α1 ) + A2 (α 2 ) n

n

where

Ae jφ A1 = = A2* 2 α1 = e jωo = α 2* It is also possible to express the signal as

{

x[n] = 2 Re A1 (α1 )

n

}

• Example: Express an arbitrary DT signal in terms of the unit impulse Answer x[n ] =

∞

∑ x[k ]δ [n − k ]

k =−∞

2-5

2.2 Discrete Time Systems • Let T [•] represents the transformation a discrete time system performed on its input x[n ] . The corresponding output signal of the system is

y[n] = T [ x[n]] .

T [•]

x[ n]

y[ n]

• The system is linear if

T [ ax1[n] + bx2 [n]] = ay1[n] + by2 [n] where y1[ n] and y2 [ n] are the responses of the system to inputs of

x1[n ] and x2 [ n] respectively. The above equation illustrates the principle of superposition.

• Assume the system is linear and let hh [n] be the output of the system when the input is

pk [ n] = δ [ n − k ] (i.e. a unit-impulse at time n = k ). Then according to the linearity property, 2-6

when the input is ∞

∑ x[k ]δ [n − k ]

x[n ] =

k =−∞ ∞

=

∑

k =−∞

x[k ] pk [n]

,

the output will be y[ n] =

∞

∑ x[ k ]h [n]

k =−∞

k

• The system is time-invariant if

hk [ n] = h[n − k ] , i.e. the output is delayed if the input is delayed. In this case y[ n] =

∞

∑ x[ k]h[n − k ]

k =−∞

= x[n ] ⊗ h[n] The signal h[ n] is called the impulse response of the time-invariant system.

• While the focus of this course is on linear, time-invariant (LTI) system, there are many real-life applications where the system is non-linear and time-variant. A good example is a digital FM demodulator operating in the mobile radio environment.

2-7

• Example: Provide a physical interpretation of a LTI system whose impulse response is

h[ n] = u[ n ] Answer: The output of the system is y[ n] =

∞

∑ x[ k] h [n − k ]

k =−∞

=

∞

∑ x[ k]u[n − k ]

k =−∞

=

n

∑ x[k ]

k =−∞

 n −1  =  ∑ x[k ]  + x[ n]  k =−∞  = y[ n − 1] + x[k ] Thus the system is an integrator.

• A system is casual if and only if the output at time n depends only on the input up to time n . According to the equation y[ n] =

∞

∑ h[ k] x[n − k ] ,

k =−∞

this means the impulse response h[n ] is zero when n < 0 .

2-8

• Example: A moving averager computes the signal M 1 y[n ] = ∑ x[n − k ] M 1 + M 2 + 1 k =− M 2

1

from its input x[n ] . Here M 1 and M 2 are positive integers. What is the impulse response of the system? Is the system casual? Answer: The output can be rewritten as n+ M 1 y[n] = ∑ x[m] M 1 + M 2 + 1 m =n −M 1

2

n − M −1  n+ M  1 =  ∑ x[m] − ∑ x[m]  M 1 + M 2 + 1  m =−∞ m =−∞  1

2

Compared to the output of the integrator, we can deduce that the impulse response of the system is

h[ n] =

1 ( u [ n + M1 ] − u [ n − M 2 − 1] ) M1 + M 2 + 1

Since the impulse response is non-zero when n < 0 , so the system is not casual.

• A real physical system can not be non-casual, i.e. it can not generate an output before there is an input. So in practice what a non-casual system 2-9

means is that there is a processing delay. For example, you can view the moving averager as a device that computes the local mean of the signal x[n ] at time n after it observes the sample x[ n + M 1] . So M 1 is the delay.

• A system is stable if a bounded input results in a bounded output. The requirement for having a stable system can be derived from the input/output relationship of a LTI system, which states that y[ n] =

∞

∑ x[ k] h[n − k ]

k =−∞

This means y[ n] =

∞

∑ x[ k] h[n − k ]

k =−∞

≤

∞

∑

k =−∞

≤ xmax

x[k ] h [n − k ] ∞

∞

∑ h [ n − k ] = x ∑ h [ m] max

k =−∞

m =−∞

where • is the absolute value operator and xmax is the largest magnitude of the input signal. So if the impulse response of the system is absolute-summable, i.e. when S=

∞

∑ h [k ] < ∞

k =−∞

then the system is stable.

2-10

• Example: Is the integrator a stable system? Answer: Since h[n] = 1 for n ≥ 0 and zero otherwise, the impulse response is not absolute-summable. Consequently the system is not stable.

• Example: Is the moving averager a stable system? Answer: Yes, because the impulse response consists of only a finite number of non-zero samples.

• Finite Impulse Response (FIR) and Infinite Impulse Response (IIR): FIR => An impulse response of finite duration, hence a finite number of non-zero samples. Always stable. IIR => The impulse response is infinitely long. Can be unstable (for example the integrator).

• Example: Comment on the stability of a LTI system with the exponential impulse response

an h[n ] =  0

n≥0 otherwise

Solution: ∞

∑

k =−∞

∞

∞

h[k ] = ∑ a = ∑ a k

k =0

k

k =0

2-11

This is summable if a < 1 . In this case,

S=

∞

∑

k =−∞

h[ k ] =

1 1− a

• Cascading of LTI systems – serial connection of two or more systems; see the example below.

As far as the input/output relationship is concerned, it really does not matter what the order of the concatenation is. For the example above, both possibilities yields the same combined impulse response of

h[n] = h1[n] ⊗ h2 [n]

2-12

• In many applications, we have to concatenate a system to an existing one so that the combined system yields the desired response. A good example is the equalizer used in a digital communication system. Many communication channels introduces intersymbol interference (ISI) ef. This means the received signal r[ n] depends not only on the data bit b[ n ] , but also on some adjacent bits. For example,

r[ n] = h1[0]b[n] + h1[1]b[ n − 1] where h1[ n] represents the impulse response of the channel. The objective of equalizer design is to find a digital filter with an impulse response h2 [ n] so that the combined response of the channel and the equalizer,

h[ n] = h1[ n] ⊗ h2 [ n] , is the unit-impulse function. This means after

equalization, we have y[n ] = b[n ] , i.e. the ISI is removed.

• Exercise: Consider an ISI channel with h1[n] = a u[n ] , where 0 < a < 1, and u[ n] is the unit step function. Determine the equalizer that completely removes the ISI. n

• Systems governed by the Linear Constant Coefficient Difference Equation (LCCDE): N

M

k =1

j =0

y[ n] = ∑ ak y[ n − k ] + ∑ bj x[ n − j] The above equation suggests that current output of the system depends on the previous output as well as the current and previous input.

2-13

• In analyzing the above system, we assume the input is applied at time n = 0 (i.e. x[n] = 0 for negative n ) and the initial state of the system is defined as Y[0] = ( y[− 1], y[− 2],..., y[− N ])

(a)

Zero State Response (ZSR) – response of the system to an unit impulse applied at time n = 0 , under the condition that Y[0] is the all-zero vector.

(b)

Zero Input Response (ZIR) – response of the system due to a nonzero initial state but no input.

• Example:

y[n ] = ay[n −1] + x[ n]

Let the initial state be Y[0] = y[− 1] = b , then

y[0] = ab + x[0] y[1] = a 2b + ax[0] + x[1] y[2] = a3b + a2 x[0] + ax[1] + x[2] or in general n

y[ n] = a b + ∑ a n −k x[k ] n +1

k =0

The ZIR is

h1[ n] = a n +1bu[n ] and the ZSR is

2-14

h2 [ n] = an u[ n] It is clear that the ZIR corresponds to the bias term in y[ n ] . Since it is independent of the input, the system can NOT be classified as a linear system. Note that the response of the system to

x3[ n] = w1 x1[n] + w2 x2[ n] is n

y[ n] = a b + ∑ a n −k x3[ k ] n +1

k =0 n

= a b + ∑ a n −k { w1 x1[ k ] + w2 x2 [ k ]} n +1

,

k= 0

which is different from

y3[n ] = w1 y1[n ] + w2 y 2[n ] , where n

y1[ n] = a b + ∑ a n −k x1[ k ], n +1

k =0 n

y2 [n ] = a b + ∑ a n− k x2 [ k ] n+1

k =0

2-15

2.3 Fourier Transform of Discrete Time Signals • Consider the sinusoidal signal

x[n ] = A cos (ω n + φ ) It can be written in terms of two complex exponential functions as

e j (ω n +φ ) + e − j(ω n +φ ) x[n] = A = A1e jω n + A2 e− jω n 2 where

A1 =

A jφ e = A2* 2

jω n

• The complex signal e is an important signal in discrete time signal processing – it is an eigenfunction of a linear system and it leads us to the concept of Fourier Transform of a discrete-time signal. Again let us use T [•] to represent the operation a discrete time system performs on its input. A signal f [ n] is an eigenfunction of the system if

T [ f [ n ]] = a f [n ] ,

a is called an eigenvalue. This definition is consistent with that in matrix theory where the eigenvector v and the eigenvalue b of where the constant

a matrix A is defined as

Av = bv . 2-16

Here the matrix A is analogous to our linear system.

• As shown in Section 2.2, the transformation performed by a LTI on its input x[n ] is described by the convolution formula: ∞

∑ h[ k] x[ n − k] ,

y[ n] =

k =−∞

where h[n] is the impulse response of the system and y[n ] is the transformed signal or output of the system. If

x[ n] = e jω n , then the output signal becomes

y[ n] =

∞

∑ h[ k]e

jω (n − k )

k =−∞

=

∞

∑ h[ k ]e

jω n − jω k

e

k =−∞

=e

jω n

 ∞ − jω k  , h [ k ] e ∑   k =−∞ 

= e jω n H ( e jω ) where ∞

( ) = ∑ h[ k]e

H e

jω

k =−∞

− jω k

.

2-17

jω n

It is clear from the above analysis that e is indeed an eigenfunction of a discrete-time LTI system with H ( e jω ) being the corresponding eigenvalue. In the linear system literature, H ( e jω ) is called the frequency response of a discrete-time LTI system.

• In general, the expression ∞

( ) = ∑ x[ k ]e

X e

jω

− jω k

k =−∞

is called the Fourier Transform of the discrete-time signal x[n ] .

• One important property of the Fourier Transform of a discrete time signal is that it is periodic in ω with a period of 2π . This is quite different from the Fourier Transform of a continuous time signal, which in general is not periodic.

• Example: Express the output of a LTI system in terms of its frequency response when the input is the sinusoid x[ n] = A cos (ω n + φ ) . Assume the impulse response of the sytem is a real signal. Solution: - The sinusoidal input can be written as a weighted sum of two complex exponential functions as

x[n] = A1e jωn + A2e− jωn 2-18

jφ

where A1 = Ae / 2 = A2 are the weighting coefficients. *

- Since the system is linear, the sinusoidal response is

y[n] = A1 y1 [n] + A2 y2 [n] where

( )

y1[n ] = H e jω e jω and

(

)

y 2 [ n] = H e − jω e − jω , jω n are the outputs of the system when the inputs are x1 [n ] = e and x 2 [n ] = e− jω n respectively.

- Since the impulse response is real,

(

∞

) ∑ h[ k ]e

H e− jω =

*

jω k

k =−∞

 ∞  =  ∑ h[ k ]e − jω k  = H * e jω .  k =−∞ 

( )

This means we can write the output of the system as

( )

(

)

y[n] = A1 H e jω e jωn + A2 H e − jω e − jω n 14243 14 4244 3 y1 [ n ]

( )

y2 n ]

( )

= A1 H e jω e jω n + A1* H * e jω e − jω n

{

( )

= 2Re A1 H e jω e jω n

( )

}

= A H e jω cos (ω n + φ + ψ (ω ) )

2-19

Note that H ( e jω ) and ψ (ω ) are respectively the magnitude and phase of the frequency response, i.e.

H ( e jω ) = H ( e jω ) e jψ ( ω ) In conclusion, when the input is a sinusoid, the output is also a sinusoid at the same frequency but with the amplitude scaled by H ( e jω ) and with the phase shifted by an amount ψ (ω ) .

• Example: Determine the frequency response of a delay element described by the impulse response

h[n ] = δ [n − d ] Solution

H (e

jω

∞

) = ∑ h[n ]e

− jω n

n =−∞

=

∞

− jω n − jω d δ [ n − d ] e = e ∑

n =−∞

This means

H ( e jω ) = 1 and

ψ (ω ) = −ω d

(constant magnitude response) (linear phase)

• Example: Determine the Fourier Transform of the one-sided exponential signal

2-20

x[ n] = a nu[ n] where 0 < a < 1 and u[ n ] is the unit-step function. Solution:

X (e

jω

∞

) = ∑ x[n]e

− jω n

n =−∞

∞

= ∑a e

n − jω n

n= 0

∞

= ∑ ( ae− jω ) = n

n= 0

1 1 − ae− jω

Since 1 − ae− jω = (1 − a cos(ω ) ) + ja sin(ω )  = (1 − a cos(ω ) ) + a sin (ω )    2

=

(1 − a cos(ω ) )

2

=

(1 − a cos(ω ) )

2

2

2

  +j 2 2 2 2 2 2 (1 − a cos(ω ) ) + a sin (ω ) (1 − a cos(ω ) ) + a sin (ω )  1 − a cos(ω )

a sin(ω )

+ a 2 sin 2 (ω ) {cos (θ (ω ) ) + j sin (θ (ω ) )} + a 2 sin 2 (ω )exp ( jθ (ω ) )

where

θ (ω ) =

a sin(ω ) 1 − a cos(ω ) ,

this means the magnitude of the Fourier transform is

( )

X e jω =

1

(1 − a cos(ω ) )2 + a 2 sin 2 (ω )

and the phase is simply

ψ (ω ) = −θ (ω ) .

2-21

• Existence of the Fourier Transform: - If we set the parameter a in the above example to unity, then the signal becomes a unit-step. The Fourier Transform in this case, however, does not exist in the finite magnitude sense. - A sufficient condition for the existence of the Fourier transform (in the finite-magnitude sense) is that the signal is absolute-summable, i.e.

S=

∞

∑

x[ k ] < ∞

k =−∞

The proof is the same as that we used to proof the stability of a LTI system. - We can deduce from the above that the Fourier Transform always exists for signals with finite duration.

• Example: Determine the Fourier Transform of the signal

1/( M + 1) x[n ] =  0

0≤ n≤ M otherwise

Solution

X (e

jω

∞

) = ∑ x[ n]e n =−∞

1 e = M +1 =e

− jω M / 2

− jω n

1 M − jω n 1 1 − e− jω ( M +1) = ∑ e = M + 1 1 − e− jω M + 1 n=0

− jω ( M +1)/ 2

{e

jω ( M +1) / 2

− e − jω ( M +1)/ 2

e− jω / 2 {e jω / 2 − e− jω / 2 }

}

1 sin ( ω ( M2+1 )) M + 1 sin ( ω2 )

2-22

The magnitude of the transform is

X (e

jω

)

1 sin (ω ( M2+1 )) = M + 1 sin ( ω2 )

At a first glance, the phase of the Fourier Transform is ψ (ω ) = −M ω / 2 . However, the sin(g) / sin(∗) function can take on either + or – ve value. When there is a sign change in this function, that corresponds to an additional 180 degree phase shift. Plots of the magnitude and phase of the transform for the case of M = 4 are shown below.

2-23

• The Inverse Fourier Transform is defined mathematically as 1 x[ n] = 2π

π

∫ X (e ) e jω

jω n

dω

−π

Proof: 1 2π

π

∫ X (e ) e jω

−π

jω n

1 dω = 2π 1 = 2π

π

 ∞  x[ k ]e − jω k  e jω n d ω ∫−π  k∑ =−∞  π

∞

∫ ∑ x[ k ]e

jω ( n −k )

dω

− π k =−∞

 1 = ∑ x[ k ]  k =−∞  2π ∞

 jω ( n −k ) e d ω  ∫−π  π

Since

1 2π

ωc

ω

c 1 jω m ( jω m ) e d ω = e d ( jω m ) ∫−ω ∫ j 2π m −ωc c

=

1  ( jω m )  ωc e   −ωc j 2π m

1 e ( jω c m ) − e − ( jω c m ) = πm 2j = =

sin ( mω c ) mπ

ω c sin ( mω c ) = π mω c

ωc  mω c  sinc   π  π 

this means 2-24

1 2π

π

∫ ( ) X e

−π

jω

e

jω n

 1 dω = ∑ x[ k ]  k =−∞  2π ∞

=

 jω ( n − k ) e d ω  ∫ −π  π

∞

∑ x[ k ] sinc ( n − k )

k =−∞

=

∞

∑ x[ k ] δ [n − k ]

k =−∞

= x[ n]

• Physically, the inverse Fourier transform states that the time domain signal is the sum of infinitesimally small complex sinuoids of the form

X ( e jω ) e jω n d ω where X ( e jω ) denotes the relative amount of each complex sinusoidal component. Consequently, it is a synthesizing formula.

• Example: Determine the impulse response of a LTI system whose frequency response is given by

( )

H e jω =

1 1 − 12 e − j 4ω

Solution: We first rewrite the frequency response as

2-25

( )

H e jω =

1 1 − 12 e − j 4ω ∞

=∑ k =0 ∞

(

1 2

e

)

− j 4ω k

= ∑ ( 12 ) e − j 4 kω k

k =0

Comparing the above infinite series with the definition of a Fourier transform, we come to the conclusion that

( 1 )n / 4 h[ n] =  2 0

n = 0,4,8,12,... otherwise

• Example: The frequency response of an ideal low pass (i.e. brickwall) filter is

1 H lp ( e jω ) =  0

ω ≤ ωc otherwise

What is the corresponding impulse response?

( )

H lp e jω 1 −ωc

ωc

ω 2-26

Solution:

1 hlp [ n] = 2π 1 = 2π =

π

∫

( )

X lp e jω e jωn dω

−π ωc

∫

e jω n d ω

−ωc

sin ( nω c ) nπ

Observation: the impulse response of an ideal low pass filter is NOT absolute summable. Question: But then why does the frequency response exist? Answer: The absolute-summability of a signal is a sufficient condition for H ( e jω ) < ∞ , not a necessary condition.

• If we impose the constraint that the magnitude of a valid Fourier Transform must be finite, is there any reason why we shouldn’t impose the constraint that the derivative(s) of a Fourier Transform should also be finite? After all, any paremeter associated with a real world signal should be finite, right? If we impose this additional constriant on the derivative, then the ideal low pass filter is not a valid frequency response because of the discontinuities in the spectrum.

2-27

• Exercise: Show that a sufficient requirement for

d H ( e jω ) < ∞ dω is ∞

∑

k h( k ) < ∞

k =−∞

Verify the result using the ideal low pass example.

• It appears that if we were to be able to deal with a wide variety of signals in our analysis, we should relax on the requirement that magnitude of a valid transform or its derivative(s) must be finite. This leads us to the impulse function in the frequency domain δ (ω ) . Some important properties of this function are: 1.

δ ( 0) is undefinied by infinitely large,

2.

δ (ω ) = 0 for

ω ≠ 0 , and

∞

3.

∫ δ (ω ) f (ω )dω = f (0)

−∞

With the introduction of this function, we can now have proper definitions for the Fourier transforms of signals such as a DC signal, a complex sinusoidal, and the unit-step signal. The fact that these signals exist at discrete frequencies is consistent with the above properties of the impulse function.

2-28

• The Fourier Transform of the DC signal

x[n ] = 1 is

X (e

jω

∞

) = ∑ x[n]e

− jω n

n =−∞

=

∞

∑e

− jω n

n =−∞

= 2π

∞

∑ δ ( ω + 2π n)

n =−∞

Proof: The function X ( e jω ) can be treated as an “analog signal” in ω . Since this “analog signal” has a period of P = 2π , it can be represented by the complex Fourier series X (e

jω

∞

)= ∑ X k =−∞

 2π  exp j k ω k    P 

(1)

where

1  2π  jω Xk = X e exp − j k ω dω ( )  ∫ P −P/ 2 P   P/2

(2)

is the k-th complex Fourier coefficient. Substituting

X (e

jω

∞

) = 2π ∑ δ ( ω + 2π n ) n =−∞

2-29

into (2) yields 1  2π  jω Xk = X e exp − j k ω  dω P − P∫/ 2 P   P/2

1 = 2π

( )

π

∞    2π  2 π δ ω + 2 π n exp − j k ω ( ) ∑     dω ∫−π  n=−∞ 2 π   

=1 Substituting X k = 1 into (1) yields X (e

jω

 2π  exp  j kω   P  k =−∞ ∞  2π  = ∑ exp  j kω   2π  k =−∞ ∞

)= ∑ X

=

k

∞

∑ exp ( − jkω )

k =−∞

• The Fourier Transform of the complex sinuoid

x[ n] = e jω on is

X (e

jω

∞

)= ∑e

jωo n − jω n

e

n =−∞

=

∞

∑e

− j (ω −ω 0 ) n

n =−∞

2-30

This is simply the Fourier Transform of the DC signal shifted to the frequency ω = ωo . Consequently,

X (e

jω

∞

) = 2π ∑ δ ( ω − ω n =−∞

o

+ 2π n )

• It can be shown that the Fourier Transform of the unit-step function is ∞ 1 U (e ) = + ∑ πδ (ω + 2π n ) 1 − e− jω n=−∞ jω

2.4 Properties of Fourier Transforms • Linearity: If

( ) x [n ] ↔ X ( e )

x1[n ] ↔ X 1 e jω , jω

2

then

2

ax1[ n] + bx2 [ n] ↔ aX 1 ( e jω ) + bX 2 ( e jω )

2-31

• Time Shifting and Frequency Shifting

x [n − d ] ↔

∞

∑ x [ n − d ]e

− jω n

n =−∞

∞

x [ n − d ]e − jω (n −d ) e− jω d

∑

=

n =−∞ ∞

∑ x [ m]e

=

− jω m − jω d

e

m =−∞

= e − j ω d X ( e jω )

e

jω o n

x [n ] ↔

∞

∑ x [ n]e

jω o n − jω n

e

n =−∞

∞

∑ x [ n]e

=

− j ( ω −ω0 ) n

n =−∞

(

= X e j (ω −ω0 )

)

• Time reversal

x [ −n] ↔

∞

∑ x [ −n]e

− jω n

n=−∞

=

∞

∑ x [ −n]e

− j ( −ω )( −n )

n =−∞

=

∞

∑ x [ m]e

− j ( −ω ) m

m =−∞

= X ( e − jω ) 2-32

We showed earlier that if x[n ] is real, then

X ( e− jω ) = X * ( e jω ) . So if x[n ] is real, then

x [ − n ] ↔ X * ( e jω ) • Differentiation in Frequency

d d  ∞ jω − jω k  X (e ) = x k e [ ] ∑  dω dω  k =−∞  ∞ d − jω k = ∑ x [k ] e dω k =−∞ =

∞

∑ x [ k ] ( − jk ) e

− jω k

k =−∞

∞

= − j ∑ kx [ k] e− jω k k =−∞

The above implies

nx[ n] ↔ j

• Convolution: If

d X ( e jω ) dω

x3[ n] = x1[ n] ⊗ x2[ n] , then

X 3 ( e jω ) = X 1 ( e jω ) X 2 (e jω ) 2-33

Proof: Since

x3 [n ] = x1[n ] ⊗ x2 [n ] =

∞

∑ x [k ]x [n − k ] ,

k =−∞

1

2

then

X 3 (e

jω

∞

) = ∑ x [n ]e n =−∞

=

∞

∞

∑ ∑ x [k ]x [n − k ]e

n =−∞ k =−∞

=

− jω n

3

∞

∞

∑∑

n =−∞ k =−∞

1

− jω n

2

x1[k ] x2 [ n − k ]e

− jω ( n − k ) − jω k

e

 ∞ − jω n − k  = ∑ x1[ k ]  ∑ x2 [ n − k ]e ( ) e − jω k k =−∞  n =−∞  ∞

 ∞  = ∑ x1[ k ]  ∑ x2 [m ]e− jω m e − jω k k =−∞  m =−∞  ∞

=

∞

∑ x [k ]X ( e )e

k =−∞

jω

1

− jω k

2

 ∞ − jω k  = X 2 ( e )  ∑ x1[k ]e   k =−∞  jω

= X 1 ( e jω ) X 2 ( e jω )

The result is known as the Convolution Theorem.

2-34

• Energy of a signal and the Parseval’s Theorem The energy of a discrete-time signal is defined as

E=

∞

∑

n =−∞

1 x[ n ] = 2π 2

π

∫

−π

X ( e jω ) d ω 2

The result is known as the Parseval’s Theorem.

Proof: Let

h[ n] = x* [ −n]

This means

H ( e jω ) = X * ( e jω ) .

If

y[ n] = x[n ] ⊗ h[ n] =

∞

∑ x[ k]h[ n − k ]

k =−∞

=

∞

∑ x[ k] x [ k − n]

,

*

k =−∞

then

( ) = X (e ) H (e ) = X (e )

Y e

jω

jω

jω

jω

2

jω Taking the inverse Fourier Transform of Y ( e ) at n = 0 yields

2-35

∞

y[0] =

1 x[ k ] = 2π

∑

π

∫ Y ( e )dω

2

k =−∞

1 = 2π

jω

−π π

∫

−π

X ( e jω ) dω 2

• The Windowing Theorem

y[ n] = x[n]w[ n]  1 =  2π  1 =  2π

∫ ( )

 1 e dθ     2π

 jφ jnφ W e e d φ  ∫−π 

π

 1 e dθ     2π

 jφ jnφ W e e d φ  ∫ − π −θ 

π

X e

jθ

−π

∫ ( ) X e

jθ

−π π

π −θ

π

π

jnθ

jnθ

π

( )

π −θ

( )

( ) ( )

1 1 jn(θ +φ ) jθ jφ = X e W e e dφ dθ 2π θ =−∫ π φ = −∫π −θ 2π

( ) (

)

1 1 j (ω −θ ) jθ jnω = X e W e e dθ dω ∫ ∫ 2π θ =−π ω =−π 2π 1 = 2π 1 = 2π

 π 1  jnω j (ω −θ ) jθ X e W e d θ  ∫ e d ω ∫ 2π ω =− π  θ =− π  π

( ) (

π

∫ Y ( e )e jω

jnω

)

dω

ω =− π

where

Y (e

jω

π

)= ∫

(

)

1 X ( e jθ ) W e j (ω −θ ) dθ θ =− π 2π 2-36

It is important to realize that the above is a circular convolution in the frequency domain. • Example: Use the Windowing Theorem to determine the Fourier Transform of the signal

 sin (nωπc n ) y[ n] =   0

−M ≤n≤ M otherwise

Solution This signal can be considered as the product of the ideal low pass signal

hlp [ n] =

sin ( nωc ) nπ

and the rectangular window

1 w[n ] =  0

−M ≤ n≤ M otherwise

As shown earlier,

1 H lp ( e jω ) =  0

ω ≤ ωc otherwise .

It can also be shown that (please verify)

1 − e − jω ( M +1) 1 − e + jω ( M +1) W (e ) = + −1 − jω + jω 1− e 1− e jω

2-37

Results for different values of M are shown below

Note that

H M ( e jω )

is this figure is equivalent to Y ( e jω )

Observations: - the oscillation (also known as the Gibbs phenomenon) is more rapid for larger M, - the amount of ripples does not decrease though

• Exercise: Plot W ( e jω ) and find out what causes the ripples in these diagrams.

2-38

2.5 Sampling of an Analog Signal • Reference: Sections 4.0-4.3, 4.6 of Text • Now that we know what the Fourier Transform of a discrete time signal x[n ] is, we want to relate it to the Fourier transform of its continuous-time counterpart xc (t ) , under the condition that

x[ n] = xc (nT ) , where T is the sampling period and fs =

1 T

is the sampling rate.

• Let

X c ( jΩ ) =

∞

∫

xc (t )e− jΩt dt

−∞

be the continuous-time (CT) Fourier transform of xc (t ) , where Ω is the analog frequency. This means

1 xc (t ) = 2π

∞

∫

X c ( jΩ ) e jΩt d Ω

−∞

(inverse Fourier Transform) .

Since x[n] = xc (nT ) , this means

2-39

1 x[ n] = xc ( nT ) = 2π =

1 2π

1 = 2π 1 = 2π 1 = 2π 1 = 2π 1 = 2π

∞

∫ X ( jΩ ) e c

jΩnT

dΩ

−∞

∞

( k + 12 ) 2Tπ

∑ ∫ (

k =−∞ k − 1 2 ∞

) 2Tπ

π /T

∑ ∫

k =−∞ − π / T ∞

π /T

∑ ∫

k =−∞ − π / T

X c ( jΩ ) e jΩnT d Ω

Xc ( j (Θ + k

2π T

)) e

(

j Θ +k

2π T

)nT

d ( Θ + k 2Tπ )

X c ( j ( Θ + k 2Tπ ) ) e jΘnT d Θ

π /T

 ∞  jΘnT 2π X j Θ + k dΘ ( ) ( )  e ∑ c T ∫  −π / T  k =−∞ π /T

 ∞  jΩnT 2π X j Ω + k dΩ ( ) ( )  e ∑ c T ∫  −π / T  k =−∞ π

1 ∫−π  T

∞

∑ X ( j( c

k =−∞

ω T

 + k 2Tπ ) )e jω nd ω 

In comparing the above with the inverse Fourier transform of a DT signal, we come to the conclusion that

1 ∞ X ( e ) = ∑ X c ( j ( ωT + k 2Tπ ) ) T k =−∞ jω

jω • Exercise: Show that the X ( e ) shown above is periodic in ω with a

period of 2π .

2-40

jω • To construct X ( e ) from X c ( jΩ ) , we can adopt the following procedure

1. Divide X c ( jΩ ) into intervals of width 2π / T according to the formula

k

k-th interval:

2π π 2π π − ≤Ω 0 . If the Nyquist sampling criterion is satisfied, this means

1  2π  1 1   k − = k − Ω ≥ k −     s   2W ≥ W 2 T 2 2       This means the spectral segments X k ( jΩ ) , k=1,2,…, are all zero. Similarly, it can be shown that all the spectral segments with negative k are zero. Consequently,

( )

X e jω =

1  ω Xc  j  , T  T

i.e. the shape of X c ( jΩ) is preserved after sampling. In this case, the Fourier transform of the sampled signal is simply a (frequency and amplitude) scaled version of X c ( jΩ) .

For the previous example involving the triangular X c ( jΩ) , a sampling frequency of 640 rad/s was used. This is less than the Nyquist frequency of 2W = 960 rad/s. Consequently there is aliasing in the sampled signal. When the sampling frequency is at or above the Nyquist frequency, say Ω s = 960, 2 × 960, 3 × 960 rad/s, then the aliasing disappears and X ( e jω ) becomes

2-44

Fig: Effect of varying the sampling frequency on X ( e jω ) : (a) the sampling frequency is at the Nyquist rate of 960 rad/s, (b) the sampling frequency is twice the Nyquist rate or 1920 rad/s, and (c) the sampling frequency is three times the Nyquist rate or 2880 rad/s.

jω It is evident from the diagram that the spectral width of X ( e ) in the interval [ −π , π ] is proportional to the bandwidth to sampling-rate ratio

ρ=

2W Ωs

The larger Ωs is, the narrower the spectrum of the digital signal.

2-45

2.5.1 Down Sampling by an Integer Factor • Suppose x c (t ) was sampled at a rate of 1/ T Hz to obtain

x[n ] = xc (nT ) A down sampler will convert x [ n] to another DT-signal xd [n ] according to

xd [n ] = x [ nM ]

x[ n]

M ↓ down sampler

xd [ n]

• Example: M=2

x[2] x[0] x[4]

n 0

1

2

3

4

5

xd [1] xd [0] xd [2]

n 0

1

2 2-46

The samples x[1], x[3], x[5],... are not used in forming the signal x d [ n] , i.e. they are decimated. Consequently a down sampler is also known as a decimator. jω jω • What is the relationship between X c ( jΩ ) , X ( e ) , and X d ( e ) ?

From our earlier discussion we know

1 ∞ X ( e ) = ∑ X c ( j ( ωT + k T k =−∞ jω

2π T

))

Since the down-sampled signal xd [n] can be viewed as a sampled version of xc (t ) with a sampling period of

T ' = MT , jω so X d ( e ) must be

1 ∞ X d (e ) = X c ( j ( Tω' + r 2Tπ' ) ) ∑ T ' r=−∞ 1 ∞ ω 2π = X c ( j ( MT + r MT )) ∑ MT r=−∞ jω

1 = M 1 = M

M −1

1 ∑ i =−0 T

∞

∑ X ( j(

k =−∞

c

ω MT

2π + [ kM + i] MT ))

1 ∞  ω 2π 2π X j + k + i ( )  ∑ c ( MT ∑ T MT )  T i =0  k =−∞ 

M −1

2-47

1 = M 1 = M

1 ∞  ω +i 2π 1 2π X j + k   ( )  ∑ c  ∑  M T T T i =0  k =−∞ 

M −1

(

)

M −1

j [ ω +i 2π ]/ M X e ( ) ∑ i =0

This equation says that the FT of the down-sampled signal is the jω superposition of M frequency shifted and scaled copies of X ( e ) .

• It should be emphasized that the above expression should only be used to jω determine X d ( e ) in the range −π ≤ ω < π . For ω outside this range, jω the value of X d ( e ) can be determined by making use of the fact that it is periodic with a period of 2π .

On the other hand if you blindly apply the formula above, you will end up jω with a X d ( e ) with a period of 2π M .

• Example: for M = 2 , we have

X d ( e jω ) =

{

}

1 X ( e jω / 2 ) + X ( e j (ω −2π ) / 2 ) 2

There will be no aliasing effect in xd [ n] if the sampling frequency of x[ n] is at least twice the Nyquist frequency, i.e.

Ωs =

2π ≥ 2 ( 2W ) . T 2-48

If this is the case, then the effective sampling frequency of xd [ n] is

Ω 's =

Ωs ≥ ( 2W ) 2

which satisfies the Nyquist criterion for zero-aliasing. Alternatively you can prove this using the diagram below.

Fig: (a) X ( e jω ) when the sampling frequency is twice the Nyquist frequency, (b) 1 2

X ( e jω / 2 ) , (c)

1 2

X ( e j (ω − 2π ) / 2 ) , and (d) X d ( e jω ) .

• In general, the sampling frequency of x[ n] must be at least M times the Nyquist rate in order to avoid aliasing in xd [n] . 2-49

2.5.2 Up Sampling by an Integer Factor • When there is no aliasing in the sampled signal, then theoretically it is possible to compute xc (t ) exactly at any value of t from x[n ] . This process is known as interpolation. Argument:

( )

x[ n] => X e jω => X c ( j Ω ) => xc (t ) Derivation:

1 xc (t ) = 2π 1 = 2π 1 = 2π 1 = 2π 1 = 2π

∞

X c ( jΩ ) e jΩt d Ω

∫

−∞ π /T

∫

X c ( jΩ ) e j Ωt d Ω

−π / T π

∫

−π π

 ω ω  X c  j  e jωt /T d    T T 

∫ X (e ) e jω

=

dω

−π π

 ∞ − jω n  jωt / T x [ n ] e dω e ∫−π  n∑ =−∞ 

 1 = ∑ x[n]  n =−∞  2π ∞

jωt / T

π

∫e

jω (t / T − n )

−π

 dω  

t  x [ n ]sinc − n ∑   T  n =−∞ ∞

2-50

• In practice, it is not possible to calculate xc (t ) exactly from its samples because the sinc function is, straightly speaking, infinitely long. Interpolators that are commonly used in practice are: - Lagrange, - Cubic spline

• Note that sinc(t / T ) is the inverse Fourier transform of the following ideal low pass spectrum:

H lp ( jΩ ) T −π / T

π /T

Ω

Thus ideal interpolation is equivalent to feeding the discrete time signal x[n ] as a sequence of (continuous-time) impulses to an ideal low pass filter, i.e.

 ∞  t  xc (t ) =  ∑ x[ n]δ (t − nT )  ⊗ sinc   T   n=−∞  Again, since the ideal low pass filter can never be implemented exactly because of the abrupt transitions, we can never interpolate the analog signal exactly from its samples.

2-51

• Let

xi [n] = xc (nT ')

be the DT signal obtained by sampling xc (t ) at a rate of f s' =

1 1 L = = Hz T ' (T / L) T

Since this signal can be obtained from the signal x[ n] = x c ( nT ) according

 nT '  x [ k ]sinc − k ∑    T  k =−∞ ∞  nT  = ∑ x[k ]sinc  −k  LT  k =−∞ ∞ n  = ∑ x[k ]sinc  − k , L  k =−∞

xi [n] =

∞

it is called an up-sampled signal.

x[ n]

L ↑ up sampler

xi [ n]

2-52

• What is the Fourier Transform of this up-sampled signal? - Since the sampling rate fs = 1/ T used in generating x[n ] satisfies the Nyquist criterion ,

X ( e jω ) =

1  ω Xc  j  T  T

- Since fs = 1/ T satisfies the Nyquist criterion, the new sampling rate x i [n ] will also satisfy the Nyquist fs ' = L / T used in generating criterion. This means

1  ω  L  ωL  Xc  j  = X c  j  T'  T' T  T  LX (e jω L ) ω ≤π /L = π/L< ω ≤π 0

X i ( e jω ) =

jω jω In otherword, X i ( e ) is simply a compressed version of X ( e ) .

jω • The existence of the two distinct frequency bands in X i ( e ) suggests (again) a low pass filtering effect in interpolation. This can be traced back to the sinc function

n  sinc  − k  L  in xi [n ] .

2-53

2-54

• Exercise: What is the Fourier Transform of the signal

xi [n] =

∞

n  x [ k ]sinc ∑  −k L  k =−∞

in general? Do not assume that x[k ] is obtained from sampling an analog signal.

2.5.3 Changing the Sampling Rate by a Non-Integer Factor • Suppose we want to change the sampling rate of the system by a factor of f . How?

• Approximate f as a rational number

f ≈

L M

and then up-sample the signal by a factor of L , followed by downsampling (the up-sampled signal) by a factor of M .

x[ n]

L ↑ up sampler

M ↓ down sampler

y[ n]

2-55

2.6 Discrete Time Random Processes • A random variable is a parameter whose value can not be predicted exactly.

• Associated with a random variable is its probability density function (pdf). Example 1: Uniform variable x :

1/(b − a) px (v) =  0

a≤v ≤b otherwise

Fig: pdf of a uniform random variable.

2-56

Example 2: Gaussian random variable x :

 ( v − mx )2  px ( v) = exp  −  2 2   σx 2πσ x   1

where

mx = E [ x]

and 2 σ x2 = E ( x − mx )   

are respectively the mean and variance of x , with E [•] being the average operator.

Fig: pdf of Gaussian random variables.

2-57

• Exercise: Determine the mean and variance of a uniform random variable.

• Note that b

Pr [ a ≤ x ≤ b] = ∫ px (v )dv a

with ∞

∫

−∞

px (v )dv = 1 .

• A discrete time signal x[n ] is a random process when every x[n ] is a random variable. Notations: 1. xn = x[n ] 2. The pdf of x n is p xn (v ) 3. The mean of x n is mxn 4. The variance of x n is σ xn 2

• If the random process x[n ] is (wide-sense) stationary, then 1. the pdf pxn ( v) is independent of the time index n , and 2. the autocorrelation function, defined as

2-58

φ xx [ n, m ] = E  x[ n] x*[ m]  , depends only on the time difference

n − m , i.e.

φ xx [ n, m ] = E  x[ n] x*[ m]  = φxx [ n − m ]

Basically what we are saying is that the first and second other statistics of a (wide-sense) stationary random process do not depend on absolute time. In this course, we will focus only on these random processes.

• Exercise: Show that

φ xx [ −n ] = φ xx* [ n ] Subsequently show that for a real random process

(

)

( )

Φ e − jω = Φ e jω

• The autocorrelation function at a time difference of n = 0 is 2 φ xx [ 0 ] = E  x[ n] x*[ n]  = E  x[ n]   

and is called the average power of the random process x[ n] . If the random process has zero mean, i.e. mxn = 0 , then

2-59

2 2    φ xx [ 0] = E x[n] = E  x[n] − mxn      = σ x2 , 2 where σ x is the variance of x[ n] .

We will only focus on zero-mean processes in this course.

• The autocorrelation function provides information as to how fast a random process varies with time. A “fast’ process will have a relatively narrow autocorrelation function. The exact frequency contents of a random process can be obtained from its power spectral density (psd) function, defined as

Φ xx ( e

jω

∞

)= ∑φ n =−∞

− jω n [ n ] e xx

Since this equation is simply the Fourier transform of φ xx [n ] , consequently

1 φ xx [n] = 2π

π

∫

Φ xx (e jω )e jωn d ω

−π

Note that

1 φxx [0] = 2π

π

∫

Φ xx (e jω )d ω

−π

= area under psd function = average power

2-60

• White noise: A random process is white if there is no correlation between the values of the process at different time instants. Mathematically this means φ xx [n ] = σ x2δ [n] and

Φ xx ( e

jω

∞

)= ∑φ

xx [ n]e

n =−∞

− jω n

=

∞

2 − jω n σ δ [ n ] e ∑ x

n =−∞

= σ x2 • Exercise: Convince yourself that a white noise must have zero mean.

• A commonly encountered random process is the white Gaussian noise. Remember, “white” refers to the spectral shape, and “Gaussian” refers to the pdf.

• Let the (stationary) random process x[ n] be the input to a LTI system with an impulse response h[n ] . Then the output of the system is

y[n ] =

∞

∑ x[k]h[n − k ] .

k =−∞

We want to examine the statistical properties of the output process.

• The mean value of y[ n] is 2-61

 ∞  m yn = E [ y[ n]] = E  ∑ x[ k]h[ n − k ]  k =−∞  =

∞

∑ E [ x[k ]] h[n − k ]

k =−∞

=

∞

∑ m h[n − k ] x

k =−∞

= mx

∞

∑ h[n − k ]

k =−∞

= mx H ( e jω )

ω =0

= my , where

H (e

jω

∞

) = ∑ h[m]e

− jω m

m =−∞

is the frequency response of the system.

Basically, what the result is saying is that if the mean of the input is stationary, then the mean of the output is also stationary.

• How about the autocorrelation function? By definition

φ yy [ n, m] = E  y[ n] y *[ m] *  ∞  ∞   = E  ∑ h[ k] x[ n − k ]   ∑ h[ r]x[ m − r ]     r =−∞    k =−∞

2-62

 ∞  ∞ *  * = E   ∑ h[ k] x[ n − k ]  ∑ h [ r ] x [ m − r ]    r =−∞    k =−∞  ∞  ∞ *  = E  ∑ h[ k ]  ∑ h [ r]x[n − k ] x*[ m − r ]   r =−∞   k =−∞ = =

∞

∞

∑ h[ k ] ∑ h [ r] E  x[n − k ] x [ m − r ] *

k =−∞

r =−∞

∞

∞

*

∑ h[ k ] ∑ h [ r ]φ [ n − m− k + r] *

k =−∞

xx

r =−∞

= φ yy [ n − m] Thus the autocorrelation function of the output process is also independent of absolute time.

• Since both the first and second order statistics of the output process are independent of absolute time, the process is (wide-sense) stationary.

• Let d = n − m , then the autocorrelation function of the output process can be written as

φ yy [ d ] =

∞

∑

k =−∞

∞

h[k ] ∑ h*[r ]φxx [d − k + r ] r =−∞

Taking Fourier transform of both sides yields

2-63

Φ yy ( e

jω

∞

) = ∑ φ [d]e

− jω d

yy

d =−∞

=

∞

∞

∞

∑ ∑ h[ k] ∑ h [ r ]φ [ d − k + r ] e *

− jω d

xx

d =−∞ k =−∞

r =−∞

 ∞ *  ∞   = ∑ h[ k ]  ∑ h [r ]  ∑ φ xx [ d − k + r ] e − jω ( d − k + r )  e jω r  e− jω k k =−∞  d =−∞  r =−∞  ∞

 ∞ *  ∞   = ∑ h[ k ]  ∑ h [r ]  ∑ φ xx [ m] e− jω m  e jω r  e− jω k k =−∞  m=−∞  r =−∞  ∞ ∞   jω = ∑ h[ k ] Φ xx ( e ) ∑ h*[ r ]e jω r  e − jω k k =−∞  r =−∞  ∞

= Φ xx ( e = Φ xx = Φ xx

∞

) H ( e ) ∑ h[ k ]e (e ) H (e ) H (e ) (e ) H (e ) jω

*

jω

− jω k

d =−∞

jω

*

jω

jω

jω

jω

2

• According to an earlier result, if the input process x[n ] has zero mean, then the output process y[n ] will also have zero mean. This means the variance of the output process equals the output power:

1 σ 2y = E  y[n]  = φ yy [0] =   2π 2

1 = 2π

π

∫ Φ ( e )dω jω

yy

−π π

∫

−π

H ( e jω ) Φ xx ( e jω ) dω 2

2-64

jω If H (e ) is a very narrow band filter centered at ω = ±ωc , then

(

σ ≈∆ H e 2 y

jω c

) {Φ ( e ) + Φ ( e )} , 2

jω c

− jω c

xx

xx

where ∆ → 0 is the bandwidth of the filter. For simiplicity assume a real x[n ] 1. Then φxx [ −n] = φxx [ n] (see an earlier − jω jω exercise) and Φ ( e ) = Φ ( e ) . Consequently,

(

σ ≈ 2∆ H e 2 y

or simply

jωc

(

)

2

(

)

Φ xx e jωc ≥ 0

)

Φ xx e jω c ≥ 0 . jω This property of Φ xx ( e power density.

c

)

is consistent with the fact that it represents a

• Example: The input/output relationship of a LTI system is given by

y[n ] = ay[n −1] + x[ n] where a is a positive constant less than 1. Determine the psd and the pdf of y[n ] if x[n ] is a white Gaussian process with zero mean and unit variance.

1

The proof is a bit lengthy for complex

x[n ] but the end result is the same. 2-65

Solution 1 - Since the input has zero mean, the output must also have zero mean. - Since the input is Gaussian, the output must also be Gaussian. - Taking the expectation of the square of both sides of y[n ] yields

σ 2y = E  y 2 [n] = E  (ay[n − 1] + x[n])2  = E a 2 y 2[ n − 1] + x 2[n] + 2ay[n − 1]x [n ]

= a 2 E  y 2 [n − 1] + E  x2 [n]  + 2aE [ y[n − 1]x [n ]] = a 2σ 2y + σ x2 + 2aE [ y[n − 1]] E [ x[ n]] = a 2σ 2y + σ x2 or

σ x2 1 σ = = 1− a2 1− a2 2 y

- The pdf of y[n ] is simply

 v2  py ( v) = exp  − 2  2  σy  2πσ y   1

- The output y[n ] can be expressed recursively as

y[n ] = ay[n −1] + x[n] = a ( ay[n − 2] + x[ n − 1] ) + x[n]

2-66

= a 2 y [ n − 2] + ax[ n − 1] + x[n] = a 3 y[ n − 3] + a 2 x[ n − 2] + ax[ n − 1] + x[n] m−1

= a y[n − m ] + ∑ a k x [n − k ] m

k =0

Multiplying both sides by y[ n − m ] , m > 0 , and taking expectation yields m −1  m   E [ y[ n] y[ n − m] ] = E  a y[ n − m] + ∑ a k x[ n − k ]  y[ n − m ]  k=0   

 m  m−1 k  = E a y [n − m ]y [n − m ] +  ∑ a x [n − k ]y [n − m ]    k =0   m −1

= a E  y2 [n − m ] + ∑ a k E [ x[n − k ]y [n − m ]] m

k =0

= a mσ y2 = φ yy [ m ] Since the process y[n ] is real, φ yy [ −m] = φ yy [ m] . So in general

φ yy [m ] = a m σ y2 - The psd is

Φ yy ( e

jω

∞

)= ∑φ

yy

[m]e− jω m

m =−∞

=σ

2 y

∞

∑a

m

e − j ωm

m =−∞

2-67

=σ

0

2 y

∑a

m

− jω m

e

+σ

∞

∑a

2 y

m =−∞

=σ

∞

2 y

∑a e n

= = = =

e− jω m −σ 2y

m=0 jω n

+σ

∞

2 y

n= 0

σ 2y

m

+

∑a

m − jω m

e

−σ 2y

m= 0

σ 2y

− σ 2y

1 − ae jω 1 − ae− jω σ 2y (1 − ae − jω ) + σ 2y (1 − ae jω ) − σ 2y (1 − ae jω )(1 − ae− jω )

(1 − ae ) (1 − ae ) jω

σ y2 (1 − a 2 )

− jω

1 + a2 − 2a cos (ω ) 1 1 + a2 − 2a cos (ω )

Solution 2 - The LTI has an impulse response of

h[n ] = a n u[n] where u[n] is the unit-step function.

- The corresponding frequency response is

H (e

jω

∞

) = ∑ h[n]e n=−∞

− jω n

∞

= ∑ an e− jω n = n= 0

1 1 − ae − jω

This means 2-68

( )

H e jω

2

= = =

1 1 − ae

− jω 2

1

(1 − a cos(ω ) ) + ( a sin(ω ) ) 2

2

1 1 + a 2 − 2a cos(ω )

- Since x[ n] is white and has a variance of unity, its psd is simply

Φ xx ( e jω ) = σ x2 = 1 - The psd of y[n ] is

( ) = Φ (e ) H (e )

Φ yy e

jω

jω

xx

jω

2

=

1 1 + a 2 − 2a cos(ω )

2.7 Linear Predictiive Coding and the Autocorrelation Function • Consider the last example in Section 2.6 where we have a random process y[ n] governed by the equation

y[n ] = ay[n −1] + x[ n]

2-69

Here a is a positive constant less than 1 and x[n ] is a white process with zero mean and unit variance. As shown in Section 2.6, the variance of y[ n] is related to the variance of x[n ] according to the equation

σ x2 1 σ = = 1− a2 1− a2 2 y

• Suppose y[ n] is a sampled-speech signal and we want to send this speech signal digitally through a communication channel. A simple way is to quantize each sample into By bits and feed the resultant bit stream to a digital modulator to generate the transmitted signal

By − bit

y[n ]

Quantizer

Digital Modulator

This encoding method, which is referred to as Pulse Code Modulation (PCM) in the literature, is not very efficient because y[ n] has a relatively large dynamic range, at least compared to x[n ] .

• Suppose the parameter parameter a (which will vary from speaker to speaker) is known to both the encoder and the decoder. Then a more efficient encoding scheme can be obtained by first subtracting

y p [ n] = ay[ n −1] from y[ n] to obtain

2-70

x [n ] = y [n ] − y p [ n ] = y [n ] − ay [n − 1] and then quantize x[n ] using a Bx -bit quantizer. The resultant bits, together with the parameter a (also in quantized form), are then sent to the receiver. Upon receiving these information, the decoder can approximate y[n ] according to

% ˆ[n − 1] + x%[ n] , yˆ[n ] = ay where x%[n ] and a% are respectively the quantized versions of x[n ] and a . This encoding scheme, known as linear predictive coding (LPC), is more efficient than PCM because x[n ] has a smaller dynamic range than y[n ] . In the speech coding literature, y p [ n] = ay[ n − 1] is referred to as the predicted value of y[ n] and x[ n ] = y[ n] − y p [ n ] is called the residual or excitation.

x[n ]

Speech Model 1 H ( e jω ) = 1 − ae− jω

y[ n]

+

x[n ] −

Prediction Filter

( )

F e jω = ae− jω

ay[ n − 1]

Eqv. response = 1 − ae− jω

2-71

x[n ]

Q [ •]

x%[n ]

yˆ[ n]

Quantizer

ˆ ˆ[ n − 1] ay

ˆ −1 az

Synthesizing filter

m 2 • As shown earlier, the autocorrelation function of y[ n] is φ yy [ m] = a σ x . This means

φ yy [1] = aφ yy [0] or

a=

φ yy [1] φ yy [0]

So as long as φ yy [0] and φ yy [1] are known, then the parameter a will be known to both the encoder and decoder. In practice, φ yy [0] and φ yy [1] can be estimated according to

1 φˆyy [ d ] = N

N−d

∑ y[ n] y[n + d ]. k =1

This suggests that in order to implement the LPC encoder, the signal y[ n] must first be analyzed to obtain its autocorrelation function. Once the autocorrelation function is estimated, then the information will be used to obtain the prediction filter at the encoder and the synthesizing filter at the decoder.

2-72

• As the parameter a → 1 , the variance of y[ n] is going to much greater than that of x[ n] . Consequently LPC will be much more efficient than PCM. As a → 0 , the variance of y[ n] approaches that of x[ n] and LPC provides little improvement to the encoding efficiency. Since a → 0 implies little correlation between samples in y[ n] while a → 1 corresponds to high correlation, we can conclude that LPC achieves a higher encoding efficiency because it removes the redundancy in y[ n] through linear prediction.

• The expression y[ n] = ay[ n − 1] + x[ n] represents a first-order model. In general, speech signals can be modeled accurately using a N-th order model (typical value of N is 10): N

y[ n] = ∑ ak y [n − k ] + x[ n ] k =1

where x[ n] is a white noise. The model also represents a system described by a Linear Constant Coefficient Difference Equation with x[n ] being the excitation and y[n] being the output.

• For the N-th order model, the prediction filter in the encoder computes N

y p [ n] = ∑ ak y[ n − k] k =1

and subtract it from y[ n] to obtain the residual (excitation)

2-73

N

x[n ] = y[n ] − y p [n ] = y [n ] − ∑ a k y [n − k ] k =1

The residual, as well as the filter coefficients,

 a1  a  A= 2 M   aN  are then quantized and transmitted to the receiver. Upon receiving these information, the decoder synthesizes the original speech signal according to N

yˆ[n ] = ∑ a%k yˆ[n − k ] + x%[ n] , k =1

where x%[n ] and a%k are respectively the quantized versions of x[n ] and

ak .

• How to determine linear predictor A? If we multiply both sides of y[n ] by y[ n − 1] and taking average, we obtain

N  E [ y[ n] y[ n − 1]] = E ∑ ak y [ n − k ]y [ n − 1] + x[ n] y[n − 1]  k =1  N

= ∑ a k E [ y[ n − k ]y [n − 1]] + E [ x[ n] y [n − 1]] k =1 N

= ∑ a kφ yy [k − 1] k =1

2-74

or simply N

φ yy [1] = ∑ ak φ yy [k − 1] k =1

Similarly if we multiply both sides of y[n ] by y[ n − 1] and taking average, we obtain

N  E [ y[ n] y[ n − 2]] = E ∑ a k y [ n − k ] y[ n − 2] + x[n] y[ n − 2]  k =1  N

= ∑ ak E [ y[ n − k ]y [ n − 2]] + E [ x[ n] y [n − 2]] k =1 N

= ∑ akφ yy [k − 2] k =1

or N

φ yy [2] = ∑ ak φ yy [k − 2] k =1

• It is evident that in general N

φ yy [ m] = ∑ ak φ yy [k − m]; k =1 N

= ∑ akφ yy  k − m ;

m = 1,2,..., N

k =1

2-75

These equations can be written in matrix form as 2 φ [1] φ [2]  φ [0]  φ[1] φ[0] φ[1]   φ[2] φ [1] φ[0]  M M M  φ[ N − 2] φ [ N − 3] φ[ N − 4]   φ[ N − 1] φ[ N − 2] φ[ N − 3]

L φ [ N − 2] φ [N − 1]   a1   φ [1]  L φ[ N − 3] φ[ N − 2]  a2   φ[2]  L φ[ N − 4] φ[ N − 3]   a3   φ[3]   =  M M M M M         L φ[0] φ[1] aN −1 φ[ N − 1]     L φ[1] φ[0]   a N   φ[ N ] 

or

U N A N = VN , where φ [1] φ [2]  φ [0]  φ[1] φ[0] φ[1]   φ [2] φ [1] φ [0] UN =  M M M  φ [ N − 2] φ [ N − 3] φ [ N − 4]   φ[ N − 1] φ[ N − 2] φ[ N − 3]

 a1   a N ,1   a   a  N ,2 2      a   aN ,3  AN=  3  =   M M      aN −1   aN ,N −1       a N   a N , N 

2

and

L φ [ N − 2] φ [N − 1]  L φ[ N − 3] φ[ N − 2] L φ [N − 4] φ [N − 3]  M M M  L φ [0] φ [1]   L φ[1] φ[0] 

 φ[1]   φ[2]     φ[3]  VN =    M  φ[ N − 1]    φ[ N ] 

For convenience, we drop the subscript yy in φ yy [ m] . 2-76

So the coefficients for the prediction filter can be obtained by solving

A N = U−N1VN , provided that the autocorrelation function φ[ m] is known.

• A brute-force computation of A N results in a complexity in the order of O ( N 3 ) . However, since the matrix is Toeplitz, i.e. all elements along any diagonal are identical, a more efficient algorithm called the Levison and Durbin (LD) algorithm can be used. The LD algorithm only has a complexity of only

O (N 2 ) .

• The LD algorithm is a order-recursive algorithm, i.e. the m -th order predictor A m can be obtained from the (m − 1) -th predictor A m−1 . Specifically, we rewrite A m as  am,1  a   A  d  A m =  m,2  =  m−1  +  m−1   M   0   km    am,m 

(1)

where the vector d m−1 and the scaler k m are quantities to be determined.

• The covariance matrix Um itself can be written in terms of U m−1 and Vm−1 as:

2-77

 U m −1 Um =  r t ( Vm−1 )

Vmr −1   φ [0] 

(2)

where

Vmr −1

φ [m − 1]  φ[m − 2]   φ[ m − 3]  =   M   φ[2]     φ [1] 

(3)

is the correlation vector Vm−1 arranged in reverse order, and [ •]t stands for the transpose of a matrix.

• Substituting (1)-(3) into the equation U m A m = Vm implies

 U m−1  t ( Vmr −1 )

Vmr −1    A m−1  d m−1    Vm−1    + =    φ[0]    0   km    φ[ m ] 

(4)

• One of the equation we can obtain from (4) is

Vm −1 = Um −1 A m −1 + Um −1dm −1 + k m Vmr −1 = Vm −1 + U m −1d m −1 + km Vmr −1 or

2-78

dm −1 = −km Um−1−1 Vmr −1

(5)

Since Vm−1 = U m−1A m−1 , Vmr −1 = PVm−1 = PU m−1A m−1 = PU m−1PA rm−1 ,

(6)

= Wm−1 A mr −1

where

0 L 0 1  0 L 1 0   P= M M M M    1 L 0 0 

is the permutation matrix representing vector reversal, and Wm−1 = PU m−1P

Because of the property of U m−1 , Wm −1 = U m −1

(7)

Combining (5)-(7) implies

dm −1 = − km Um−1−1 Vmr −1 = − km U −m1−1Wm −1 A mr −1 = − km U −m1−1U m −1A rm −1

(8)

= − km Arm−1 ,

2-79

i.e. the vector d m−1 is the vector containing the coefficients of the predictor arranged in reverse order and scaled by the term k m .

( m − 1) -th

• The second equation we can derive from (4) is

φ[ m] = ( Vmr −1 ) Am−1 + ( Vmr −1 ) dm −1 + km φ[0] t

t

= ( Vmr −1 ) A m−1 − ( Vmr −1 ) kmA mr −1 + kmφ [0] t

t

or

φ [m ] − ( Vmr −1 ) Am−1 t

km =

φ[0] − ( V

r m −1

)A t

r m −1

φ [m ] − ( Vmr −1 ) Am−1 t

=

ε m−1

(9)

where

ε m −1 = φ[0] − ( Vmr −1 ) Amr −1 t

= φ[0] − (Vm −1 ) A m −1 t

(10)

• In summary, 1. At the end of the (m-1)-th iteration, the available information to the LPC encoder are ε m−1 and the (m-1)-th order predictor Am−1 . 2. In the m-th iteration, the encoder computes km according to (9) and set the highest order term in the m-th order predictor to (see Eqn. (1)) am ,m = k m 2-80

3. The other predictor coefficients are computed according to (1) and (8) as

am, k = am −1, k + d m−1,k = am−1,k − km a m−1,m− k ;

k = 1,2,..., m − 1 ,

(11)

where d m−1, k is the k-th component of the vector d m−1 in (8) 4. Update the term ε m and increase m by 1.

• Exercise: Show that the term ε m can also be calculated recursively.

• Equations (9)-(11) each has a computational complexity of m −1 multiplication. Summing over all m from 1 to N yields a complexity of N

3∑ ( m − 1) = 3N ( N − 1) / 2 ∝ N 2 m =1

• Note that LPC-based speech codec can produce communication quality speech at a rate as low as 2.4 kbps (though 4.8-9.6 kbps are more typical). This is much lower than the 64 kbps required in PCM based codecs. The speech codec used in your cell-phone is a LPC-based codec.

2-81