MEDIAN-BASED POST-PROCESSING FOR VQ IMAGE TRANSMISSION OVER NOISY CHANNELS R. Iordache, I. T˘abus¸
A. Beghdadi
Signal Processing Laboratory Tampere University of Technology P.0. Box 553, FIN-33101 Tampere, Finland iordache,
[email protected]
L2TI-Institute Galil´ee Universit´e Paris 13 FR-93430 Villetaneuse, France
[email protected]
ABSTRACT For image transmission over noisy channels, the VQ paradigm is a strong competitor. We propose a post-processing scheme able to improve the perceptual quality of the VQ reconstructed image, for various types of noisy channels, when it is combined with an appropriate index assignment. For binary symmetric channels, we use the index assignment resulting from the LBG design of the codebook, which ensures a low channel distortion. For the more challenging case of memory channels we consider index assignments specifically designed. We show that a significant perceptual improvement can be achieved with our post-processing scheme both for memoryless and memory channels. 1. INTRODUCTION When transmitting compressed images over noisy channels the effect of only few errors may be disastrous. With the advent of the progressive image coding techniques, the flexibility of reaching a wide range of rate-distortion operating points shadowed somewhat the VQ image coding. However, the error resilience of progressive schemes is much worse than that of VQ schemes, since for the latter the effect of one error is limited to the VQ block whose index is affected, while for the former not only the quality of the image is degraded, but also the synchronism in the bit stream may be lost, making completely useless the reconstructed image. Several improvements of the error resilience of progressive image coding have been recently reported [1], but they still rely heavily on the use of error correcting schemes. Consequently, for noisy channels applications the VQ image coding is still a strong candidate, which we consider in this paper in connection with its main strength, the channel error resilience. Several fruitful lines of research have been considered for VQ coded image transmission over noisy channels [2, 3, 4]. One of the most appealing techniques is index assignment (IA), which achieves the performance of the best
possible VQ scheme in the absence of errors, while offering a substantial improvement when noise is present. However, the IA design is done under certain hypotheses, the most critical being the following two: (1) the extent of the channel memory (number of symbols affected by one error), and (2) the a priori probability distribution of codevectors. When using the VQ codebook in environments failing to satisfy the design hypothesis, the IA is not effective anymore. In order to increase the robustness of the communication scheme (the range of conditions under which the reconstruction is acceptable),we propose the use of an IA suited for memoryless or memory channels, in tandem with a post-processing scheme. To this end, we present several median-based filtering variants, which are able to improve the perceptual quality of the reconstructed VQ image for various types of noisy channels, when they are combined with an appropriate index assignment. 2. IA FOR NOISY CHANNELS An image VQ is specified by the NV Q d-dimensional codevectors, each codevector being the reconstruction of a L L pixel block (d = L2 ). We consider that the VQ codebook is designed to minimize the source distortion (assuming no errors over the channel). Intuitively, the index assignment permutation, which maps each codevector to a channel code, should be chosen so that indices that are likely to be interchanged at the decoder due to the channel noise correspond to close enough codevectors (in the Euclidian sense). The index assignment is designed for a given a priori probability distribution of the codevectors, which is collected from the training set of images used to construct the codebook. If we send an image for which the a priori probabilities of the codevectors is significantly different from the values used in the IA design, the IA offers little protection against the channel noise. One remedy may be the use of error correcting codes at the expense of increasing the bandwidth, but in the case of heavily corrupted
channels, errors may still be left uncorrected. Disregarding the means used to protect the transmitted data, some postprocessing may be extremely useful to reduce the effects of the channel noise. The effect of the channel noise in the VQ reconstructed image is a block-wise impulsive noise, which appears when the received index is different from the transmitted one. Of course, more different are the blocks corresponding to the received and transmitted indices, more disturbing the induced artifact is. Even though the IA was supposed to arrange the indices such that it is unlikely to mistake a codevector for a distant one due to channel errors, this type of artifact is still present and is the most annoying one, as seen in Fig. 2(b). Coping with impulsive-like noise is the excellence domain of median filters, which constitute the kernel of our post-processing procedure. The post-processing scheme is selective in that it takes different measures according to the expected contrast and activity level in the (corrupted) reconstructed image. 3. POST-PROCESSING TECHNIQUES We consider an image ffmn g of dimensions N N , where fmn is the gray-level associated with the pixel (m; n). A VQ is optimally designed for a rate of r bits/pixel with the codevectors representing L L blocks of pixels (we use interchangeably codevector and codeblock). Let ff^mn g be the noiseless VQ reconstructed image, where each L L block in ffmn g is replaced by the nearest neighbor (in the Euclidean sense) codeblock. In our post-processing scheme an important role is played by the low resolution version of ff^mng, the image fbij g of dimensions M M , M = N=L, where bij represents the mean gray level of the block (i; j ) in the image ff^mn g:
b
ij
PL
=
1
k;`=0
f^ + L2 iL
k;jL+`
:
(1)
Median filtering The simplest post-processing method for reducing the effect of the channel noise is the median filtering of the noisy VQ reconstructed image ff mn g, which in general is different from ff^mn g due to channel errors. To be effective with the block substitution errors (block-like impulses) the window size, Lw Lw , of the median filter should be at least two times lager than the number of pixels in one block, i.e.:
L2 > 2L2: w
(2)
This requirement leads to quite large median windows (e.g. for L = 4 we must set Lw 6). Such median filters are known to destroy most of details in the image, which is an unacceptable behavior (see Fig. 1(c)).
Contrast-driven median filtering The principle of contrast-driven median filtering is to median filter with a window of size Lw Lw only the pixels in those blocks that are most probably mistaken because of the channel noise and are highly visible, and to use a 3 3 median filter for the other pixels. In this way we filter out the most annoying noisy blocks and preserve to a larger extent the details of the image. Of course, the 3 3 median filter has a smoothing effect, and removes some fine details, but at the same time it reduces the jagging due to the VQ coding and the visibility of the other artifacts induced by the channel noise. The information regarding the block visibility is computed at the encoder and is sent, highly protected, to the decoder. As a measure of visibility, we use a block-level contrast, defined as follows. Let bij be the average of the eight neighbors of bij :
b
ij
=
1 X
1 8
k;l=
b( + i
6
k)(j +l)
:
(3)
1:(k;l)=(0;0)
b g, as the ratio We define the block level contrast image, fCij between the block intensity and the average intensity of its neighbors:
C
b ij
=
b : b ij
(4)
ij
b be the lowest integer value so that Cij < Cmax () for a fraction of the pixels in fbij g. We transmit Cmax () as side information, and assume that it is received correctly by the decoder, which also knows the value of the fraction (the additional bit rate to transmit this side information is negligible in the case of large images). Let fgmn g be the noise reconstructed image at the decoder side. We construct the image of block mean gray levels in fgmn g, fdij g, and the associated block contrast image, fCijd g, in b g. the same manner as the coder builds fbij g and fCij The possibly affected blocks in the noisy VQ reconstructed image are estimated as those blocks (i; j ) where Cijd Cmax (). At these locations only, we filter each pixel in the block with a median filter with window Lw Lw . All other pixels in the image fgmng are filtered with a median filter with window 3 3.
Let
C
)
max (
Contrast and activity-driven median filtering In the previously described variant, one cannot make the difference between a high contrast block initially existing in the noiseless VQ reconstructed image and erroneous high contrast blocks due to the channel noise corruption of channel indices. As the high contrast regions in the image have a great visual importance, it is desirable to preserve the corresponding image blocks in the post-processed image.
(a) LBG (PSNR = 27.33 dB)
(b) Received (PSNR = 24.33 dB)
(c) Med. filt. 7 7 of (b) (PSNR = 25.90 dB)
(d) Contrast-driven med. filt. of (b) (PSNR = 26.70 dB)
(e) Contrast and activity-driven med. filt. of (b) (PSNR = 26.71 dB)
(f) Contrast and gradient-driven med. filt. of (b) (PSNR = 26.68 dB)
Fig. 1. A 256 256 region of the noisy reconstructed image and median filtered variants in the case of a BSC with " = 0:01. The codebook has 64 entries, of size 4 4 (bit rate 0.375 bpp), = 0:98; = 0:98; = 260.
To accomplish this task, we transmit to the receiver another scalar value, a threshold on the local activity indicator, besides the contrast threshold. Let fxbij g be the local activity image corresponding to fbij g, where xbij is the variance of the 8 neighbors of bij :
x
b ij
=
1 X
1 8
k;l=
6
b( + i
k)(j +l)
2
b
ij
:
d decoder side, the blocks (i; j ) where Cij Cmax () and d xij < xmax ( ) are labeled as noisy, and we filter each pixel in these blocks with a Lw Lw median filter. All other pixels in the image fgmn g are filtered with a 3 3 median filter.
(5)
1:(k;l)=(0;0)
Let xmax ( ) be the lowest integer value so that x < xmax ( ) for a fraction of the pixels of fbij g. The local activity measure is not critically affected by the channel noise, so we would expect that the high activity blocks, with xbij xmax ( ), are the same in the VQ image and the VQ reconstruction at the decoder. We transmit Cmax () and xmax ( ) as side information, and assume that they are received correctly by the decoder. Let fxdij g be the local activity image corresponding to fdij g, defined in the same way as fxbij g for fbij g (5). At
Contrast and gradient-driven median filtering
b ij
As the noise induced by the channel has an impulsive nature, we can identify the noisy blocks using a local gradient measure on fdij g, namely Sij , defined as the sum of the absolute values of the vertical and horizontal Sobel filtered images. At the decoder side, the affected blocks are estimated d Cmax () and Sijd < , as those blocks (i; j ) with Cij and the pixels belonging to these blocks are filtered with a Lw Lw median filter, while the other pixels are filtered with a 3 3 median filter.
(a) LBG IA (PSNR = 25.01 dB)
(b) HSA IA (PSNR = 26.03 dB)
(c) Contrast-driven med. filt. of (b) (PSNR = 27.31 dB)
Fig. 2. A 256 256 region of the noisy VQ reconstruction for two IAs and contrast-driven median filtered image, in the case of a finite-memory contagion channel with " = 0:01; M = 1; Æ = 10.
4. EXPERIMENTS
5. REFERENCES
The experiments presented in this paper are done using standard gray-level images of size 512 512. The codebook contains 64 4 4 blocks, and it is trained on three images (man, couple, crowd), using LBG algorithm [5].
[1] D. Mukherjee and S. K. Mitra, “A vector set partitioning noisy channel image coder with unequal error protection,” IEEE Journal on Selected Areas in Communications, vol. 18, pp. 829–840, 2000.
Memoryless channel, LBG IA The indices are transmitted over a binary symmetric channel with bit error rate " = 0:01. The effect of the discussed variants of median post-filtering on the noisy VQreconstruction of man image (Fig. 1(a)) can be seen in Fig. 1. It is obvious that selective filtering limits the oversmoothing, while the erroneous blocks are well corrected. From this single example it is difficult to say which of the three selective filtering variants is better, as they provide extremely similar results. Channel with memory, trained IA As a memory channel model, we used the finite-memory contagion channel [6], which has three parameters: the bit error rate ", the memory of the binary noise process M , and a constant Æ , determining the amount of correlation in the noise sequence . In this case the index assignment induced by LBG does not provide protection against channel errors, so we design a robust index assignment for the memory channel, using the Hadamard switch algorithm (HSA) [7]. In Fig. 2(a)(b) we show the types of artifacts induced in the reconstructed image with and without trained IA, and Fig. 2(c) show the contrast-driven median filtered image, when the trained IA is used.
[2] N. Farvardin and V. Vaishampayan, “On the performance and complexity of channel-optimizzed vector quantizers,” IEEE Transactions on Information Theory, vol. 37, pp. 155–160, Jan. 1991. [3] K. L. Hung, C. C. Chang, and T. S. Chen, “Reconstruction of lost blocks using codeword estimation,” IEEE Transactions on Consumer Electronics, vol. 45, pp. 1190–1199, 1999. [4] K. Zeger and A. Gersho, “Pseudo-gray coding,” IEEE Transactions on Communications, vol. 38, pp. 2147– 2158, Dec. 1990. [5] Y. Linde, A. Buzo, and R. M. Gray, “An algorithm for vector quantization design,” IEEE Transactions on Communications, vol. 28, pp. 84–95, January 1980. [6] F. Alajaji and T. Fuja, “A communication channel modeled on contagion,” IEEE Transactions on Information Theory, vol. 40, pp. 2035–2041, 1994. [7] R. Iordache and I. T˘abus¸, “Robust index assignment for finite-memory contagion channels using Hadamard transform,” in Proc. IEEE Int. Symp. on Information Theory and Its Applications ISITA’2000, Honolulu, Hawaii, U.S.A., Nov. 2000, pp. 382–385.
