JOINT CHANNEL-SOURCE CODING FOR ROBUST VECTOR QUANTIZATION : APPLICATION TO SPEECH CODING BOUZID Merouane(1), DJERADI Amar (2) (1)
Architecture of systems laboratory, Research Center of advanced technology development, CDTA, Algiers , Algeria, email:
[email protected] (2)
Electrical Engineering Faculty, Electronic Institute, USTHB University, Algiers , Algeria Abstract. Distortion of parameters of a vector quantization system can be alleviated by using an index assignment channel-source coding technique. To carry out this approach, we have developed an iteratif algorithm based on the principle of the simulated Annealing. The goal of this algorithm is to find globally optimal channel-codes, destined for the implicit protection of a 4.8 KBPS CELP speech coder codebook-indices. Such Non-redundant joint channel-source codes can be obtained by minimizing an appropriate distorsion criterion. Also, an improvement has been introduced in the simulated Annealing algorithm to palliate to a drawback of this last. Thus, we can obtain the best optimal result during the total progress of the improved version of the simulated Annealing algorithm. Keywords. Joint Channel-Source Coding, Index assignment, Simulated Annealing, CELP coder. 1. INTRODUCTION One of the serious problems in digital communication system is the occurrence of errors in the data transmitted over a noisy channel. In this paper, we present a channel coding method using source information. We have adopted, successfuly, an index assignment (IA) technique so as to implement this channel-source coding. This technique has been used without addition of redundance. The distorsion due to channel noise can be alleviated significantly by judicious assignment of code-words (binary indexes) to code-vectors of vector quantization (VQ) system. Such source-dependent channel code-words can be obtained by minimizing an appropriate error criterion. To carry out this minimization, we have developed a randomized algorithm based on the principle of the simulated Annealing (SA). This algorithm has for objective to find globally optimal channel-codes (or close to optimal) for a given source encoder and a given error criterion. The SA algorithm is very known for its efficiency to solve combinatorial optimization problems. However, it can present the disadvantage to not guarantee a globally optimal solution. This is caused by one of its properties that can induce to a loss of the global optimum during the total progress of the algorithm. To remedy to this problem, we have introduced an improvement in our SA algorithm, that consists in retaining the globally optimal result fathered during the total progress of the algorithm. Using this approach, the algorithm is named the improved simulated annealing (ISA) algorithm. Code-words, obtained by application of the ISA algorithm, are used to protect the transmission of the codebooks indices of a 4.8 KBPS CELP speech coder [1,2,3]. 2. INDEX ASSIGNMENT CODING A vector quantizer Q is a mapping of k-dimesional Euclidean space Rk into a finite subset Y of Rk given by :
Q : Rk → Y x → yi
(1)
where Y = {y0 , y1 ,..., yL -1} is known as a codebook of size L= 2n code-vectors, x source-vector maped onto code-vector yi ∈Rk and n represents the number of bits by binary index i. In the case of VQ systems conceived for noiseless transmission, the Non-redundant combined channel-source coding by IA technique can provide robustness against channel errors, when these systems are called to operate in a disturbed environment [4,5,6,7,3]. The problem of finding the best code-vectors rearrangement of a given codebook involves searching every possible IA for the one that yields the best possible performance. The IA vector (or configuration) assigns to each code-vector index i ∈{0,1}n (notation
{0,1}n : set of L binary vector indexes [4,5,8,9] ) a unique binary code-word, belonging to the same set. One symbolizes this configuration by b (b ≡ (b(0), b(1),..., b(L-1)), where b(i) is code-word attributed to the index i, (i.e., to the code-vector yi ). The block-diagram of a vector quantizer on a noisy channel is shown in Figure 1; where η is a random variable taking on values from the set {0,1}n and ⊕ is a bitwise addition operation [mod 2] (logical exclusiveOR).
x
i
C
b b(i) IA
Coder
η
Decoder
Noisy Channel
b(j)
j
D
yj
⊕
b -1 Inverse-IA
Fig. 1. Block-diagram of VQ system for noisy channel The distortion caused by representing a source-vector x of Rk by a code-vector y, is given by a nonnegative distortion measure d(x,y). The overall distortion caused by the vector quantizer and the channel noise is given by [8] : εT =
1 L L ∑ ∑ p(b( j ) b(i )) ∫ p( x ). d ( x, y j ). dx k i =1 j =1 Ri
(2)
where p(x) is the probability density function of the source, and p(j/i) is the conditional probability of decoding yj when transmitting yi . If the distortion measure is assumed to be the widely used squared distance and if the codebook Y is assumed to satisfy the noiseless channel centroid rule, then the minimization problem of the overall distortion can be reduced to the simple minimization of the channel distortion [8,5] : εC =
L 1 L p(i ) ∑ p(b( j ) b(i ) ) ⋅ d ( yi , y j ) ∑ k i =1 j =1
(3)
where p(i) denotes the a priori probability of the code-vector yi. Suppose that the channel used for the code-words transmission is a binary symmetric channel (BSC) with crossover probability ε. To simplify the channel coding problem, we suppose that the channel transition probabilities is sufficiently small (ε«1) so that we can consider only single bit error patterns in a code-word. Thus, an appropriate error criterion for the code-words optimization is given by [3]: εC =
L −1
n
i =0
m=1
∑ p(i ) ∑ d ( yi , yb −1( F (b(i ),m)) )
(4)
where F(i, m) is the index of the code-vector obtained by inverting the mth bit of the index i. 3. SIMULATED ANNEALING FOR INDEX ASSIGNMENT Finding the IA vector b(.) which minimises the error criterion (4) is a combinatorial optimization problem. It is often impratical to evaluate the performance for all possible configurations. Thus, a suboptimal technique must be employed to resolve this IA problem. Particularly, the SA is a powerful optimization algorithm, witch finds good solutions to a variety of combinatorial optimization problems [10,11]. The SA method is based on an analogy between the research of a minimal energy state (stable state), for a physical system and the minimization of an error criterion, for a combinatorial
optimization problem. Each configuration b(.) simulates a state of a hypothetical physical system with the error criterion εC, witch represents the system energy. Our SA algorithm works as follows. Step 1: Initialisation • Define an sbstract "temperature" parameter T and set it to a high value: T = Ti, • Choose, according to an appropriate method, the initial state b. Step 2: Metropolis procedure • Randomly choose a state b' : a perturbation of the state b and calculate the variation of energy: ∆ε = ε C (b' ) − ε C (b) . Metropolis acceptance Rule: a) If ∆ε < 0, replace b by b', go to step 3. b) If ∆ε ≥ 0, replace b by b' with probability exp(-∆ε / T), go to step 3. Step 3: Test of local thermodynamic equilibrium • If in step 2 the number of energy drops exceeds a prescribed number or if too many unsuccessful perturbations (perturbations not resulting in energie drops) occur, goto step 4. • Otherwise, return to the step 2. Step 4: Annealing Step • Decrease slowly the temperature T . Step 5: Test of end • If T drops below some prescribed freezing temperature Tc or if it appears that a stable state is obtained, stop, with b describing the stable state of the system. • Otherwise, return to the step 2 . The algorithm given above, describes the basics of the operation of the algorithm; comments are given in order. Initially, one has to choose a sufficiently high temperature to avoid that the process of the SA is not trapped, at the beginning , in a local minimum. An algorithm based on the iterative improvement method has been developed for the initial configuration (i.e, initial positions of code-vectors) research [3]. In step 2, perturbations of the IA vector b are generated by interchanging randomly the assignments of two code-words only. The change in the configuration is accepted or rejected according to the Metroplis acceptance rule. In step 2a), ∆ε
