Proc. of Int. Scient. Conf. DSP-MCOM 2001, Digital Signal Processing and Multimedia Communications, Slovakia, 2001, pp.31-33
Trellis Coding For Vector Quantization Ing.Iveta GLADIŠOVÁ, PhD.- Prof.Ing. Ján MIHALÍK, PhD.** FEI TU Košice, Park Komenského13, 041 20 Košice, SR Tel.: +0421-55-6022940; +0421-55-6022854 Abstract - A new source coding technique based on idea of trellis coded modulation is trellis coded quantization.
We
present
a
trellis
coded
quantization and its vector generalization, then experimental results showing its performance on the identically independent distribution Gaussian and Laplacian sources. Trellis coding for vector quantization (TCVQ) yields a lower distortion then optimum vector quantization (OVQ) for a given bit rate at the cost of an increase in implementation complexity. Index terms – trellis, vector quantization
1 Introduction Trellis coded quantization (TCQ) [1] , [4] is a fast effective method of quantizing random variables. Trellis coding exploits correlations between variables and it can use non-rectangular quantizer cells that give it quantization efficiencies not attainable by scalar quantizers. The central ideas of TCQ grew out of the ground – breaking work of Ungerboek [5] in trellis coded modulation. TCQ has proved to be an attractive source coding technique with good rate-distortion performance and yet low implementation complexity. While TCQ’s performance is in many case close to the theoretical ratedistortion bound, an improvement is always possible by generalizing its structure to the vector case. This comes at the cost of an increase in complexity due to vector codebook searching. TCQ of a given rate assigns the output levels [7] of a scalar quantizer with twice **
E-mail:
[email protected] –
[email protected]
that rate to the branches of a trellis according to the “mapping by set partitioning” method of trellis coded modulation. This is done in such a way that the minimum Euclidean distance among the output levels assigned to the same branch is maximized. While this is rather straightforward in the scalar case, it is not a simple task in the vector case and multidimensional space. An algorithm of TCVQ [6] is described in another part of the paper which partitions an optimum vector quantizer codebook into a given number of subsets such that the desired property of maximizing the minimum intrasubset Euclidean distances may be realized here as well. The codebook used in TCVQ plays an important role in the coder’s performance. We present experimental results showing the performance of TCVQ’s of different bit rate and dimensions on the identically independent distribution Gaussian and Laplacian sources. Comparisons are made with OVQ [2]. 2 Trellis coding for vector quantization The basic idea behind TCVQ is the following: Assume that we want to vector quantize a stacionary, memoryless uniform source at the rate of n bits per vector. Performing quantization directly on this uniform source would require an OVQ with
2 vn
reproduction vectors (codevectors), where v is a dimension. The OVQ may represent any source vector with any of available codevectors. The idea behind TCVQ is to first quantize the source more vn + k
codevectors. Of course finely, with 2 this would exceed the allocated rate, so we
Proc. of Int. Scient. Conf. DSP-MCOM 2001, Digital Signal Processing and Multimedia Communications, Slovakia, 2001, pp.31-33 cannot have a free choice of symbols at all the trellis, while the remaining n-1 bits times. In our case we take k=1 and the the specify a codevector out of the active subset. vn +1 TCVQ has 2 codevectors (i.e. twice that We have more codevectors than a nonoptimal bitrate - n quantizer, because there is vn of the OVQ), but only a subset of size 2 of some freedom of choosing from codevectors these codevectors may be used to represent a of either S0 or S1. Of course this choice is not source vector at any instance of time. completly free: the decision made at each vn +1 The vector codebook of 2 codevectors is input vector is linked to decisions made at vn −1 partitioned into subset of 2 codevectors, past and future input vectors, through the each generating four sets. For example n= 2 permissible paths of the trellis. But it is this bits, see Fig.1. The subsets are designed such additional flexibility that leads to the that each of them represents reproduction improved performance. vectors of a coarser bit rate (n-1) quantizer. The four subsets are designated D0, D1, D2 and D3. Also, define S0 = D0 ∪ D2 and S1 = D1 ∪ D3, where S0 and S1 are known as supersets.
Fig.1 Codebook of TCVQ and its supersets
Fig.2. Eight state trellis. Obviously, the rate constraint prohibits the specification of an arbitrary codevector out of The function of the TCVQ encoder is to find 2 vn +1 codevectors. However, it is possible to the best possible reproduction for any given exactly specify, with n bits, one element out sequence of source vectors. Specifically, for a of either S0 or S1. At each input vector, search depth of L, the encoder looks for the assuming we know which one of the superset sequence of codevectors on the L-state trellis to use, one bit can be used to determine the that is at minimum Euclidean distance from active subset, and n-1 bits to specify a the given sequence of L source vectors. Each codevector from the subset. The choice of sequence of codevectors on the trellis is superset is determined by the state of finite specified by a path (a connected sequence of state machine, described by a suitable trellis. L branches) and the choice of codevectors An example of such a trellis, with eight from the subcodebooks associated with the states, is given in Fig.2. The subsets { D0, D1, branches. For a given sequence of source D2, D3} are also used to label the branches of vectors, the encoder uses the well-known the trellis, so the same bit that specifies the Viterbi algorithm to find the best sequence of subset (at a given state) also determines the codevectors. Then it releases all the Lnv bits specifying this sequence at the end of the Lnext state of the trellis. stage trellis, which the decoder uses to Encoding is achieved by spending one bit per sample on specifying the path trough
Proc. of Int. Scient. Conf. DSP-MCOM 2001, Digital Signal Processing and Multimedia Communications, Slovakia, 2001, pp.31-33 generate the reproduction. The process is then 4 Experimental results and conclusions repeated for the next L source vectors. The designed algorithm of TCVQ was simulated on a personal computer for the 3 Design algorithm for TCVQ sequence of 360 000 source samples, from The main goal of the partitioning step is to which we could make input vectors of provide subsets of codevectors with maximal dimension v = 2,4,8,12,16, while the minimum distance within each subset. The following may be selected: bit rate n’ = 1 ÷ 8 overall set of codevectors is partitioned in bit/vector or n bit/sample, number of trellis several stages such that a binary tree of state S = 4, 8, 16, 32, 64 and different search subsets of codevectors is generated. The two depth L . The input training sequence of descendent nodes of any internal node of the samples was generated by identically tree form a partition of the subset at that independent distributed Gaussian (Gpdf) and node. The subsets at the leaves are assigned Laplacian (Lpdf) random variables with zeroto the trellis branches according to the “ mean value and unit-variance. The initial VQ mapping by set partitioning” method [6]. codebooks was used codebooks of OVQ, Given an initial OVQ codebook C of size which were partitioned according this vn +1 N= 2 , the distances between all possible algorithm. The algorithm performance is pairs of codevectors are calculated and listed characterized by the signal noise ratio (SNR) in a nondecreasing order along with the in decibe [dB]. corresponding pairs. This gives a table with The simulation results of TCVQ are Nx(N-1)/2 entries, where the ith entry presented in Tab.1 for different v and n. The corresponds to codevectors bi and b´i that are obtained results were compared with those of at distance d i = b i − b' i . First b0 and b´0 OVQ. It is evident from the tables that at a given dimension v the SNR of TCVQ are placed in the subsets S0 and S1 , increases with the bit rate n. Also these respectively, and the first entry is removed results proof increase of SNR along with from the table. Then the following basic step dimension of space at the same bit rate. In is repeated as many times as necessary until Tab.2 are presented SNR results for different the size of one of the subsets, S0 or S1, parameters S and L for one of selected reaches N/2, at which point the remaining dimension and probability density function. unassigned codevectors (if any) are added to From this table we can see , that ANR the other subset. The basic step involves increase with number of trellis state S and looking for the entry with the smallest index i also with search depth L, but then it changes with one(but not both) of the two codevectors only slightly. It is caused by limitation of this already assigned to either S0 or S1, followed search depth. by adding the unassigned code vector to the This paper deals with the design of an other subset. All entries with both algorithm of trellis coding for vector codevectors assigned are removed from the quantization. The main advantage of this table. method is that few computational costs ensue To partition the overall OVQ codebook into compared with classical vector quantizer subsets, the algorithm is first applied to methods. The princip of TCVQ was overall codebook to partition it into two explained and the obtained results of subsets, followed by applying it to these two simulation designed algorithm were finally subsets to generate four subsets and so on. evaluated. This algorithm uses and provides The performance of TCVQ can be improved good quantization results at low bit rate. if its decoder is modified. The design It may be applied as a good coding method algorithm for TCVQ coder is locally optimal. for purpose of increasing its performance and
Proc. of Int. Scient. Conf. DSP-MCOM 2001, Digital Signal Processing and Multimedia Communications, Slovakia, 2001, pp.31-33 Tab. 1 so we get efficient coding system which may v =2 SNR[dB] be important part of digital processing and 1 2 3 4 5 6 7 8 n’ [bit/vector] communication systems. 0,5
1
1,5
2
2,5
3
3,5
4
OVQ
1,640
4,435
7,018
9,514
12,380
15,327
18,370
21,523
TCVQ
2,397
4,942
7,589
10,403
13.269
16,168
19,235
22,314
n [bit/sample] Gpdf
OVQ
1,256
3,697
6,358
9,057
11,781
14,825
17,838
21,033
TCVQ
1,825
4,641
7,292
9,993
12.779
15,587
18,339
21,247
n’ [bit/vector]
1
2
3
4
5
6
7
8
n [bit/sample]
0,25
0,5
0,75
1
1,25
1,5
1,75
2
Lpdf v =4
SNR[dB]
OVQ
0,746
1,9
3,365
4,664
6,054
7,490
9,010
10,650
TCVQ
0,830
2,318
3,723
5,098
6,524
7,995
9,538
11,228
Gpdf OVQ
0,529
1,7
3,435
4,704
6,040
7,439
8,958
10,576
TCVQ
0,753
2,366
3,781
5,102
6,458
7,776
9,099
10,466
Lpdf v =8
SNR[dB]
n’[bit/vector]
1
2
3
4
5
6
7
8
n [bit/sample]
0,125
0,25
0,375
0,5
0,625
0,75
0,875
1
OVQ
0,371
0,872
1,514
2,209
2,932
3,695
4,501
5,389
TCVQ
0,391
1,096
1,757
2,482
3,223
3,982
4,824
5,720
OVQ
0,348
0,805
1,537
2,711
3,301
3,945
4,753
5,319
TCVQ
0,396
1,054
2,021
2,748
3,394
4,028
4,678
5,330
Gpdf
Lpdf v = 12
SNR[dB] 1
n’ [bit/vector]
n [bit/sample] 0,0833 Gpdf
2
3
4
5
6
7
8
0,167
0,25
0,333
0,4167
0,5
0,5833
0,6667
OVQ
0,239
0,568
0,971
1,430
1,920
2,450
3,027
3,673
TCVQ
0.270
0,714
1,162
1,631
2,128
2,680
3,262
3,927
OVQ
0,196
0,483
0,938
1,651
2,384
2,799
3,320
3,943
TCVQ
0,211
0,665
1,254
1,979
2,377
2,771
3,174
3,577
1
2
3
4
5
6
7
8
0,125
0,1875
0,25
0,3125
0,375
0,4375
0,5
Lpdf v = 16
SNR[dB]
n’[bit/vector]
n [bit/sample] 0,0625 OVQ
0,179
0,414
0,708
1,062
1,430
1,846
2,292
2,810
TCVQ
0,186
0,517
0,856
1,212
1,603
2,018
2,483
2,996
Gpdf
Lpdf
OVQ
0,143
0,366
0,697
1,187
1,914
2,248
2,629
3,100
TCVQ
0,164
0,491
0,895
1,501
1,878
2,170
2,445
2,738
Tab.2
v=2, Lpdf,
n=1 [bit/sample]
SNR [dB] S L 1 5 10 25 50 100
The work was supported by the Grant Agency of Ministry of Education of Slovak republic under GAV - 1/7387/20
4
8
16
32
3,876 4,389 4,512 4,559 4,560 4,562
3,953 4,450 4,591 4,651 4,663 4,663
3,824 4,351 4,527 4,613 4,617 4,631
3,882 4,344 4,512 4,643 4,671 4,676
References [1] Marcellin,M.W.-Fischer,T.R.: “Trellis coded quantization of memoryless and Gauss-Markov sources”. IEEE Trans. Commun., Vol.38, 1990, pp.82-93. [2] Chmúrny, J.- Mihalík,J.: Algorithms for Optimization of Vector Quantizers. Slaboproudý obzor, 47, No.9, 1986, pp.435-440. (In Slovak) [3] Mihalík,J.: Neural Network Clustering Vector Quantizer Design. Neural Network World, Vol.2, No.5, 1993, pp.197-207. [4] Fischer, T. R. - Marcellin, M. W. – Wang, M.: “Trellis – coded vector quantization.” IEEE Trans. Inform. Theory, Vol. 37, 1991, pp. 1551 – 1556. [5] Ungerboeck,G.: Trellis-coded modulation with redundant signal sets, Part I and II. IEEE Commun. Mag., Vol.25, 1987, pp.5-21. [6] Wang,H.S.-Moayeri,N.: Trellis coded vector quantization. IEEE Trans. on commun., Vol.40, No.8, 1992, pp.12731276. [7] Freeman, G.H. – Blake, I. F. – Mark, J. W.: “Trellis source code design as an optimization problem.” IEEE Trans. Inform. Theory, 1988, Special Issue on Coding : Theory and Techniques, pp. 1– 48.
