Asymptotic Performance of Multiple Description Transform ... - CiteSeerX

Asymptotic Performance of Multiple Description Transform Codes J.-C. Batllo and V.A. Vaishampayan Department of Electrical Engineering Texas A&M University College Station, TX 77843

Abstract The multiple description transform coder is introduced for sources with memory and an asymptotic analysis is presented for the squared error distortion. For stationary Gaussian sources, the optimal transform and the optimal bit allocation for the multiple description coder are identical to those for the single description coder. Index terms|Source coding, quantization, multiple descriptions, transform codes.



This work was supported in part by grants NSF NCR-9104566 and NSF NCR-9314221

1

1 Introduction We consider the design of block quantizers for sources with memory when the source is connected to the destination by two parallel channels, either of which may be broken. It is assumed that the channel state information is available to the decoder but not the encoder. This channel model is surprisingly useful for applications such as communicating over a Rayleigh fading channel and for communicating over a lossy packet network. Source codes designed for this channel model are called multiple description source codes. For memoryless sources, rate distortion theoretic results have been presented in [1] and [2], multiple description quantizers have been designed in [3] and [4] and an asymptotic analysis has been presented in [5]. For sources with memory, subsampling approaches have been considered in [6], [7]. Here we introduce the multiple description transform coder (MDTC) for sources with memory and present an asymptotic analysis for the squared error distortion measure. Relevant work on single-channel transform codes is presented in [8] and [9]. This correspondence is organized as follows. In Section 2, we introduce notation and brie y summarize relevant technical details from previous work. In Section 3, we describe the proposed multiple description transform coder and derive its asymptotic performance in Section 4. Next, we present in Section 5 the theoretical performance of this coder when applied to a Gaussian source with memory and to a rst-order Gauss-Markov source.

2 Preliminaries The block diagram of the multiple description scalar quantizer (MDSQ) is shown in Figure 1.a. Contrary to the single-channel case where each sample is mapped to a single index, in the multiple description case, each data sample is mapped to a pair of indices i and j which are then transmitted over two separate channels. Speci cally, the source sample x is rst mapped by q(
) to a cell index l 2 f1; 2; : : : ; N g. Index l is then mapped to (i; j ) by the index assignment a(
) where (i; j ) 2 J . We assume that i; j 2 I = f1; 2; : : :; mg and that J is a subset of I  I containing N pairs. Let R be the rate used on each channel. Thus, in the case of xed length codes, i and j are encoded to R-bit codewords, and in the case of variable length codes, i and j are separately encoded using variable length codes having average codeword length no larger than R bits/sample. The combination of q(
), a(
) and the xed/variable length code is referred to as the MDSQ encoder and is denoted by E. If one channel is down, the receiver obtains only one index, i or j , and thus uses a side decoder to map i, or j , to a reconstruction level ys, s = 1 if i is the only received index, s = 2 if only j is received. If both channels are working, the receiver uses the central decoder to map the indices (i,j ) to a reconstruction level y0. The two side decoders and the central 2

decoder together constitute the MDSQ decoder, D. We de ne the average central distortion by d0 = E [(X ? Y0)2] and the average side distortions by ds = E [(X ? Ys )2], s = 1; 2. We will consider only balanced descriptions, i.e., d1 = d2. It can be shown, based on results in [1] and [2], that for any a 2 (0; 1), there exist coders for which at high rates, d0 and d1 decrease exponentially as 2?2R(1+a) and 2?2R(1?a), respectively, and that these rates of decay are optimum. For a better understanding, the process can be seen in the following manner: the sample x is mapped to a cell of a square box having M = m2 cells, with m rows and m columns. Then, i is the row index and j the column index. The way in which the source samples are mapped to the cells of the box is then the index assignment. The index assignments presented in [3] achieve the above exponential rates of decay, and we choose the one where the cells of the box are scanned in a `zig-zag' manner referred to as the Modi ed Linear Assignment in [3]. Let 2k + 1 be the number of diagonals in the box. An example of such an index assignment is shown on Figure 1.b, with R = 3, m = 8 and k = 2. In this gure, the 21st cell of a 34-level quantizer is mapped to the index (4,6) (the coordinates of the location at which 21 lies). The value of a in the exponent of the expression for d0 and d1 is determined by the rate at which k grows with 2R. Speci cally, in order to have a = 1=n, n > 1, it suces to set k = 2R=n : (1) High rate approximations to the performance of multiple description scalar quantizers have been obtained in [5] for rth power distortion measures. Our derivation in Section 4 is based on the closed-form expressions for r = 2, corresponding to the squared-error distortion measure. For any general source density, the average central and side distortions are given by and

d0 = C2 2?2R(1+1=n)

(2)

d1 = S2 2?2R(1?1=n); (3) where 2 is the source variance and where C and S are determined by the source pdf and by the encoding method (i.e., xed length or variable length coding).

3 Multiple Description Transform Coders

The MDTC is illustrated in Figure 2. We consider a vector of source samples x such that x = (x(1); x(2); : : : ; x(L))tt. The linear transformation A yields the vector u = Ax where u = (u(1); u(2); : : : ; u(L)) . Each component u(l) is encoded by the lth MDSQ encoder 3

El, resulting in a pair of indices which are then transmitted over two separate channels. It is assumed that the MDSQ for the lth transform coecient operates at a rate of Rl bits/sample/channel. The rate vector associated with the MDTC is R = (R1; R2; : : :; RL). At the receiver, if one index is lost, a side decoder of Dl yields a reconstruction level u^s(l), s = 1 or 2. If no index is lost, the central decoder of Dl gives the reconstructed value u^0(l). The inverse transformation A?1 is then applied to the vectors of quantized values, providing the vector y0 = A?1 u^ 0 if both channels are working, y1 = A?1 u^ 1 if only channel 1 works or y2 = A?1 u^ 2 if only channel 2 works. For the single channel case, Huang and Schultheiss developed transform coders for levelconstrained quantization in [8] and proved that the optimal choice for A for stationary Gaussian sources was the transpose of the orthogonal matrix which diagonalizes the autocorrelation matrix of the source. In other words, A must be the matrix of the Karhunen-Loeve Transform (KLT). In Section 5, we will show that for the two channels case, the KLT is also optimal for stationary Gaussian sources. In the following derivations, we only assume that A is orthogonal so that the distortions are equal in both the original and the transform domains. Indeed, we have u = Ax, and u^ s = Ay s, s = 0; 1 or 2. From here it follows that ds(R) =
L1 E [(x ? ys )t(x ? ys )] = L1 E (x ? y s)tAtA(x ? ys )] = L1 E [(Ax ? Ays)t(Ax ? Ay s)] s = 0; 1; 2; (4) = L1 E [(u ? u^ s)t(u ? u^ s)];

where E is the expectation with respect to the source probability distribution. Thus, minimizing the distortion in the transform domain is equivalent to minimizing the distortion in the original domain.

4 Asymptotic Performance of the MDTC We consider the case of the quantization of L samples provided by an orthogonal transformation applied to the original samples and we present an asymptotic analysis. We de ne d0(l)(R) as the average central distortion of the lth quantizer at R bits/sample/channel, and d1(l)(R) as the average side distortion of the lth quantizer at R bits/sample/channel. The average central distortion and the average side distortion of the MDTC are obtained from 4

(4) and are given by

L X d0(R) = L1 d0(l)(Rl);

(5)

l=1

and

L X 1 d1(R) = L d1(l)(Rl); (6) l=1 respectively. We rst derive a bitPallocation formula, which allows the minimization of d0(R) subject to d1(R)  d1 and L1 Ll=1 Rl  R, in establishing two necessary and sucient conditions on the parameters of each quantizer, namely Rl and kl. From (1), it follows that nl = Rl= log2(kl). First, we x the Rl's and we minimize d0(R) with respect to 1=nl, l = 1; : : : ; L, under the constraint d1(R)  d1. For the lth quantizer, from (2) and (3), we have d0(l)(Rl) = Cll22?2Rl(1+1=nl) (7) and d1(l)(Rl ) = Sll22?2Rl(1?1=nl); (8) 2 where l is the variance of the lth transform coecient and Cl and Sl depend upon the pdf of the lth transform coecient. The Lagrangian J is

!

L L X X 1 1 1 ? 2 Ri (1+ n1 ) 2  i J = L Cii 2 (9) ?  d1 ? L Sii22?2Ri(1? ni ) : i=1 i=1 On setting the rst partial derivative of J with respect to 1=nl to zero and solving for , we obtain Rl  = CS l 2?4 nl ; l = 1; 2; : : : ; L; (10) l which yields   1 = ? 1 log Sl l = 1; 2; : : : ; L: (11) n 4R 2 C l

l

l

Having obtained the 1=nl 's as a function of , we now utilize the constraint d1(R) = d1 to eliminate , and so, plugging (11) in (8), we come up with L X 1  = Ld (CiSi)1=2i22?2Ri 1 i=1

5

!2

:

(12)

Therefore, we nally obtain ! 1 =2  Cl Ld1 1 = 1 log ; l = 1; 2; : : : ; L: (13) P 2 1 = 2 L 1=2 2 ?2Ri nl 2Rl Sl i=1 (Ci Si ) i 2 1 1 P P Thus, for Rl's xed, since both functions L1 Li=1 Cii22?2Ri(1+ ni ) and L1 Li=1 Sii22?2Ri(1? ni ) are real-valued convex mappings of the vector (1=n1 ; : : :; 1=nL ), we know from the Lagrange Duality theorem ([10, p. 224]) that nl's given by (13) minimize d0(R) and that d1(R) = d1. More precisely, (13) gives

!

2 L X 1 1 =2 2 ?2Rl (ClSl) l 2 : (14) d0(R) = dL2 1 l=1 And so, since we know that the constraint on d1(R) is satis ed with our choice of nl's which obey (13), we now minimize d0(R) under the constraint L 1X (15) L l=1 Rl = R:

q For an easier computation, we choose to minimize d0(R), and the new Lagrangian is ! L L X X (16) J~ = L(d1)1=2 (ClSl)1=2l22?2Rl ?  R ? L1 Ri : 1 i=1 l=1 On setting the rst partial derivative of J~ with respect to Rl to zero, we obtain  C S 1=2 2 ln(2)2 ! 1 l l l ; l = 1; 2; : : : ; L: (17) Rl = 2 log2  d1  P At this point, we utilize the constraint on the average rate L1 Ll=1 Rl = R to eliminate , which is then given by

!1=L

L Y 2 ln(2) (ClSl)1=2l2 2?2R :  = (d)1=2 1 l=1 We plug this last result in (17), and we obtain ! 1=2 2 ( C 1 l Sl ) l Rl = R + 2 log2 QL ; l = 1; 2; : : : ; L: ( i=1 (CiSi)1=2i2)1=L

6

(18) (19)

q

P

Similarly, since d0(R) and L1 Li=1 Ri are real-valued convex functionals of R, the Lagrange Duality theorem implies that Rl's satisfying (19) minimize the Lagrangian function J~ and the constraint on the average bit rate is satis ed. Finally,n we can derive the number of diagonals kl for the lth quantizer. Indeed, since from (1), kl l = 2Rl , it follows that

v u u kl = t

Cl1=2Ld1 ; l = 1; 2; : : : ; L: (20) P Sl1=2 Li=1(CiSi)1=2i22?2Ri Thus, equations (19) and (20) are necessary and sucient for an optimum operating point. From (14) and (19), we can now derive the trade-o between the overall average central distortion and the overall average side distortion ! L2 L Y d0(R) = (CiSi)1=2i2 2?4R 1 : (21) d ( R ) 1 i=1

5 Gaussian sources We now give the speci c results for the cases of level-constrained (LC) and entropy-constrained (EC) MDTC applied to a zero-mean stationary Gaussian source having variance 2. For such a source, C and S in (2) and (3) are given by C = (33=2)=24 and S = (33=2)=6 in the LC case, and by C = e=24 and S = e=6 in the EC case. If we apply a linear decorrelating transformation, then the dierent components in the transform domain are uncorrelated and Gaussian, and so are independent. Thus, for both the LC and EC cases, the Cl's and Sl's are equal for all l and given as above. Moreover, ! 2 1  (22) Rl = R + 2 log2 QL l 2 1=L ; l = 1; 2; : : : ; L; ( i=1 i ) which is the same result as for the single channel case, given in [11, p. 528]. In this case, we know that the KLT is optimum since the geometric mean of the variances of the transform coecients is minimized. Thus, the Karhunen-Loeve Transform is still an optimum transform for the multiple description case. Equation (20) becomes, for the LC case,

kl =

s

6Ld1 ; P 33=2 Li=1 i22?2Ri 7

(23)

and for the EC case,

s

6Ld1 : (24) P e Li=1 i22?2Ri It can be observed that the value of kl is independent of l, i.e., all the multiple description

kl =

quantizers have the same operating point, thus yielding the same decay rate for each MDSQ. At this point, we can derive from (19) and (23), and from (19) and (24) the expressions for the trade-o between the overall average central distortion d0 and overall average side distortion d1 for a Gaussian source with memory, given by L Y

! L2

2?4R 1 ; (25) d1 i=1 where = 32=16 for the LC case, and = 2e2=144 for the EC case. Moreover, the Toeplitz Distribution Theorem (or Szego theorem [12],[13]) states that for any continuous function f (
) satisfying some minor technical conditions, Z L X 1 1 2 f (i ) = 2 f [(!)] d!; (26) lim L!1 L ?  i=1 where (!) is the power spectral density function of the stationary source. Therefore, we nally obtain the formula valid for both cases   Z ?4R 2 1 d0 = (27) exp  ln (!) d! ; d1 ? with de ned as above. Next, we consider the case of a stationary rst-order Gauss-Markov source, described by the relation xn = xn?1 + "n, where f"ng is an iid random process with zero mean QLGaussian 2 2 2 and variance (1 ?  )x. We know the relation i=1 i = (1 ? 2)L?1, and so, we obtain the trade-o formula for a rst-order Gauss-Markov source, valid for the LC and EC cases, (28) d0 = (1 ? 2)2(1? L1 )x4 2?4R 1 ; d1 with de ned as above.

d0 =

i2

6 Summary In this paper, we have considered a multiple description transform coder and we have derived its asymptotic performance for general source densities with memory. It is shown that the 8

Karhunen-Loeve Transform is optimal and that the bit allocation is identical to the single channel case. Further, performance results have been given for stationary Gaussian sources with memory and for stationary rst-order Gauss-Markov sources.

References [1] A.A. El-Gamal and T.M. Cover, "Achievable rates for multiple descriptions," IEEE Trans. Inform. Th., vol. 28, pp. 851-857, November 1982. [2] L. Ozarow, "On a source coding problem with two channels and three receivers," Bell Syst. Tech. J., vol. 59, pp. 1909-1921, December 1980. [3] V.A. Vaishampayan,"Design of multiple description scalar quantizers," IEEE Trans. Inform. Th., vol. 39, pp. 821-834, May 1993. [4] V.A. Vaishampayan and J. Domaszewicz, "Design of entropy-constrained multiple description scalar quantizers," IEEE Trans. Inform. Th., vol. 40, pp.245-250, January 1994. [5] V.A. Vaishampayan and J.-C. Batllo, "Multiple description transform codes with an application to packetized speech," in Proc. Int. Symp. on Information Theory (ISIT94), (Trondheim, Norway, July 1994), p. 458. [6] A. Ingle and V. Vaishampayan, "DPCM System Design for Diversity Systems with Applications to Packetized Speech," IEEE Trans. Speech and Audio Proc., vol. 3, pp. 48{57, Jan. 1995. [7] N. S. Jayant, "Subsampling of a DPCM speech channel to provide two \self-contained" half-rate channels," Bell Syst. Tech. J., vol. 60, pp. 501{509, Apr. 1981. [8] J.J.Y. Huang and P.M. Schultheiss, "Block quantization of correlated Gaussian random variables," IEEE Trans. Commun. Syst., vol. CS-11, pp. 289-296, September 1963. [9] N. Farvardin and F.Y. Lin, "Performance of entropy-constrained block transform quantizers," IEEE Trans. Inform. Th., vol. 37, pp. 1433-1439, September 1991. [10] D. G. Luenberger, "Optimization by vector space methods," John Wiley & Sons, Inc., Stanford, CA, 1969. [11] N.S. Jayant and P. Noll, "Digital coding of waveforms," Prentice-Hall, Englewood Clis, NJ, 1984. 9

[12] U. Grenander and G. Szego, "Toeplitz forms and their applications," University of California Press, Berkeley, CA, 1958. [13] T. Berger, "Rate distortion theory; a mathematical basis for data compression," Prentice-Hall, Englewood Clis, NJ, 1971.

10

i -

Fixed/ Variable Length code

- @@-

-

Fix/Var Length Decoder

Side Decoder 1

-

x - q(
) - a(
) j -

Fixed/ Variable Length code

- @@-

-

Fix/Var Length Decoder

E

Central Decoder

Side Decoder 2

- y1 - y0 - y2

D

a. Block diagram of the MDSQ. ( ): quantizer encoder, ( ): index assignment. q


1 3 4

a


2 6 5 7 11 8 10 12 9 13 15 14 18 19

16 17 20 23 24

21 22 25 28 29

26 27 31 30 32 33 34

b. Example of index assignment, 2 + 1 = 5. k

Figure 1: Multiple Description Scalar Quantizer.

11

- E1 - E2

x - A

. . .

- EL

- D1

-- Ch 1

- D2 . . .

-- Ch 2

- DL

-?1 - y 1 -A -?1 - y 0 -A -?1 - y 2 -A

The internal structure of Dl is shown below:









- side decoder 1 -- central decoder

-




- side decoder 2

-




-




Figure 2: Multiple Description Transform Coder. E1 , E2 , , EL are MDSQ encoders for the transform coecients, D1, D2 , , DL are MDSQ decoders. :::

:::

12

List of Figures 1 2

Multiple Description Scalar Quantizer. . . . . . . . . . . . . . . . . . . . . . 15 Multiple Description Transform Coder. E1, E2,: : :, EL are MDSQ encoders for the transform coecients, D1, D2,: : : , DL are MDSQ decoders. . . . . . . 16

13