Abstractâ We propose a simple yet efficient method which reduces the blocking artifact in block-coded images by using a wavelet transform. An image is ...
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 8, NO. 3, JUNE 1998
253
Reduction of Blocking Artifact in Block-Coded Images Using Wavelet Transform Nam Chul Kim, Ick Hoon Jang, Dae Ho Kim, and Won Hak Hong
Abstract— We propose a simple yet efficient method which reduces the blocking artifact in block-coded images by using a wavelet transform. An image is considered a set of onedimensional signals, and so all processings including the wavelet transform are one-dimensionally executed. The artifact reduction operation is applied to only the neighborhood of each block boundary in the wavelet transform at the first and second scales. The key idea behind the method is to remove the blocking component which reveals stepwise discontinuities at block boundaries. Each block boundary is classified into one of shade region, smooth edge region, and step edge region. Threshold values for the classification are selected adaptively according to each coded image. The performance is evaluated for 512 2 512 images JPEG coded with 30 : 1 and 40 : 1 compression ratios. Experimental results show that the proposed method yields not only a PSNR improvement of about 0.69–1.06 dB, but also subjective quality nearly free of the blocking artifact and edge blur. Index Terms— Blocking artifact, JPEG, multiscale analysis, wavelet transform.
I. INTRODUCTION
I
N block-based image or video coding techniques [1], an image or each frame of an image sequence is partitioned into blocks of 4 4, 8 8, or 16 16 pixels, and then each block is coded independently. It is common in the coding of each block that, subject to a bit-allocation strategy for the minimization of distortion under a given bit quota, more bits are allocated to the block mean (DC component) and the rough variations (lower frequency components) of higher energy, and fewer bits to the detailed variations (higher frequency components) of lower energy. In the extreme case of a very low bit rate, most bits are allocated to the block mean, and only a few of bits to the block variations. As the bit rate becomes low, the continuities between adjacent blocks in coded images are severely broken. The discontinuity at the block boundaries annoying human eyes is commonly called the “blocking artifact.” There are two typical methods for the reduction of a blocking artifact in block-coded images: the spatial lowpass filtering method [2], and the iterative method based on projections onto convex sets (POCS’s) [3]. Reeve and Lim [2] tried to remove the blocking artifact by using a 3 3 Gaussian filter at block boundaries. The algorithm is somewhat simple Manuscript received July 3, 1996; revised May 29, 1997. This paper was recommended by Associate Editor R. Lancini. The authors are with the Laboratory for Visual Communications, Department of Electronic Engineering, Kyungpook National University, Taegu 702-701, Korea. Publisher Item Identifier S 1051-8215(98)03981-0.
to implement but might blur real edges at block boundaries. In the POCS-based method, closed convex constraint sets are first defined that represent all of the available data on the original uncoded image. Then alternating projections onto these convex sets are iteratively computed to recover the original image from the coded image. In general, the techniques of the POCS-based method require nearly ten or more iterations of alternating projections over an entire image. In order to efficiently remove the component causing the blocking artifact (we call it the “blocking component” hereafter) while preserving real edges, it is very important that one characterizes the local variations of a coded signal, and discriminates the blocking component in the neighborhood of block boundaries from the characterized local variations. Recently, active studies using a wavelet transform [4] have been done in various areas of signal processing, computer vision, and image coding to analyze, characterize, and compress local variations or frequency components efficiently. The wavelet transform is a tool for decomposing a signal into a set of wavelet basis functions, which are bandpassed signals whose energy is locally concentrated. In this paper, we propose an efficient method using a wavelet transform for reducing the blocking artifact in block coded images. This work evolved from our initial work in [5]. In the proposed method, an image is considered as a set of one-dimensional signals, and the blocking components for the signals are removed, which reveal stepwise discontinuities in the spatial domain so that they appear as impulses at each block boundary in the first scale of the wavelet transform. For the multiscale analysis, we utilize the wavelet transform used for the characterization of edges and singularities in multiscales by Mallat and Zhong [6]. Each of the discontinuities then is classified into three classes, and the removal of the discontinuities in multiscales is applied to each class in a different way.
II. REDUCTION OF BLOCKING ARTIFACT USING WAVELET TRANSFORM A. Wavelet Transform Using Digital Filters In this subsection, we briefly summarize the multiscale analysis and synthesis technique using the discrete wavelet transform introduced in [6]. The wavelet transform of an original signal can be implemented by using digital filters. The wavelet transform in the th scale is the into the detail signal decomposition of
1051–8215/98$10.00 1998 IEEE
254
and the coarse signal
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 8, NO. 3, JUNE 1998
as follows:
(1) where
denotes the digital filter obtained by putting zeros between the coefficients of the filter . The original signal can be reconstructed by the inverse wavelet transform from the set of multiscale detail signals and coarse signal which are decomposed by implementing the wavelet th scale as follows: transform of (1) up to the
(2) denotes the complex conjugate of . The filter where coefficients of digital filters and chosen in [6] are and , respectively. We can see that the filters and have differentiation and smoothing properties, respectively. The discrete wavelet transform described here holds the size of the transformed signal in each scale to be the same as that of the original signal because it is implemented without downsampling and upsampling. This scheme makes the positions of the block boundaries in each scale consistent with those in the spatial domain, and so facilitates the removal of the blocking component at the positions.
Fig. 1. Example of blocking component computed from (3) and (4) when the coded signal contains stepwise discontinuity at a block boundary with the remainder of the quantization noise near zero.
is very close to the original signal, the blocking component reveals an abrupt change at the block boundary. be any horizontal or Let vertical line of an coded image with blocks. to be at We define each block boundary of . Each position of is not defined as a block boundary because the one-sided of the definition is well matched to the wavelet filter has even taps mentioned above. Since the wavelet filter a differentiation property, the stepwise discontinuity shown in Fig. 1 appears as an impulse at the block boundary in the first scale. We assume that such an impulse is caused by the blocking component . So, the wavelet transform of the blocking component in the first scale can be expressed in the neighborhood of the th block boundary by (5)
B. Analysis of Blocking Artifact can be represented as the sum of the A coded signal original signal and quantization noise . Since the blocking component also comes from the quantization noise, the coded signal can be formulated as
denotes the strength of the impulse at the th where block boundary. The desired signal therefore can be which induces reconstructed if the blocking component the impulse is removed. From (3) and (4), the wavelet transform of up to the second scale has the following relations:
(3) and denote the blocking component and where the remainder of the quantization noise, respectively. At this from point, we hope to remove the blocking component the coded signal . Since it is very difficult to eliminate even the remainder of the quantization noise , we do not try to remove here. Consequently, the problem is to expressed as estimate the following desired signal (4) . Fig. 1 shows an example of from the coded signal the blocking component computed from (3) and (4) when the coded signal contains stepwise discontinuity at a block boundary with the remainder of the quantization noise near zero, i.e., . In this case, since the desired signal
(6) appears as an impulse at each block boundary, appears as a dispersed impulse because that satisfies is a derivative of the blocking component smoothed by the smoothing filter . It can be easily shown that if the filter mentioned above is used, is expressed in the neighborhood of the th block boundary by
While
(7)
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 8, NO. 3, JUNE
where denotes the blocking component which appears as a dispersed impulse in the second scale in the case where that satisfies the strength of the impulse in (5) is one. has little influence on the whole because it is consecutively smoothed. So we ignore it on the assumption that it is close to zero. Fig. 2(a) shows an example of the coded signal which is synthesized under the assumption that the blocking as shown in Fig. 1 lies in the shade region, component smooth edge region, and step edge region. Fig. 2(b) and (c) and . In the shade shows its detail signals region near abscissa 150 of Fig. 2(b), there is only an impulse which comes from the blocking component. In the smooth edge region near abscissa 310, such an impulse lies on the hill which comes from the smooth edge component. In the step edge region near abscissa 50, there also is only an impulse which, however, comes from both the blocking component and the step edge component. We note from Fig. 2(b) that the impulses by the blocking component may appear as variations in the first scale, being mixed with local portions of the original signal. Prior to removing the impulse by the blocking component, it thus is necessary to determine where each block boundary is among the three classes of shade region, smooth edge region, and step edge region. Then the removal process has to be performed in a different way for each class. In this paper, the classification procedure for each block boundary is used as
255
(a)
(b)
Th,
IF
Th1, and THEN SHADE REGION IF
Th1
Th, and
or THEN SMOOTH EDGE REGION
Th1, Th1)
(c)
IF Th THEN STEP EDGE REGION where Th and Th1 denote threshold values for the magnitude and the ratio of the absolute values of the wavelet transform coefficients, respectively. C. Blocking Artifact Reduction Once a block boundary is classified as above, the removal operation is performed in a different way for each class. For the shade region, the blocking component is removed from the wavelet transform in the first and second scales. In the first scale, a 1 3 median filter first is used at each block boundary to estimate the wavelet transform of the desired signal as median where
(8)
denotes the estimated value of . In the second scale, as the wavelet transform of the blocking component in the neighborhood of the block
Fig. 2. Example of a coded signal fb (n) with shade, smooth edge, and step edge regions, and its detail signals W1d fb (n) and W2d fb (n): (a) fb (n); (b) d d W fb (n); and (c) W fb (n). 1 2
boundary can be represented as (7), the wavelet transform of the desired signal can be estimated as
(9) Based on the relations of (5), (6), and (8), the impulse strength can be estimated by (10) For the smooth edge region, the blocking component is removed from the wavelet transform only in the first scale. For the step edge region, no removal operation is performed. The reconstructed signal is obtained by the inverse wavelet transform of the signals processed as above. Fig. 3 is the block
256
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 8, NO. 3, JUNE 1998
Fig. 3. Block diagram of the artifact reduction operation using the wavelet transform.
diagram of the proposed artifact reduction operation using the wavelet transform. The threshold values used for the classification of each block boundary ought to be selected adaptively according to the severeness of the blocking artifact that depends on which image is coded and what the compression ratio is. As a measure of the severeness of the blocking artifact, the variance of the wavelet transform of the blocking component can be used. Under some at the block boundary assumptions, the variance can be obtained as [7]
TABLE I PSNR PERFORMANCE OF JPEG, REEVE AND LIM’S ALGORITHM, AND OUR METHOD FOR THE LENA AND PEPPER IMAGES JPEG CODED WITH 30 : 1 AND 40 : 1 COMPRESSION RATIOS, RESPECTIVELY
(11) This equation means that the variance of the wavelet transform of the blocking component at the block boundary can be obtained by the difference of the variance of the wavelet transform of the coded signal at the block boundary and the variance of that at the block center. We then select the threshold value Th to be proportional to . For simplicity, we also select Th1 as half the threshold value Th. That is (12) where
denotes the proportional constant. III. EXPERIMENTAL RESULTS
To evaluate the performance of our artifact reduction method, a computer simulation was performed for 512 512 Lena and Pepper images of 256 levels. The coded images for the experiment were obtained by applying JPEG to the test images. As an objective measure of image quality, PSNR is used. Shown in Table I are the PSNR performances of JPEG, Reeve and Lim’s algorithm, and our method for the Lena and Pepper images JPEG coded with 30 : 1 and 40 : 1 compression ratios, respectively. The constant in (12) for the threshold values used in obtaining these results was chosen
as . This choice was based on the simulation results that the best performance according to each test image and compression ratio was obtained when was in the range of 4–7, while not being so sensitive to . It is found in Table I that the processed images by our method reveal a PSNR improvement of 0.69–1.06 dB over the JPEG coded images, and so our method is 0.06–0.29 dB superior to Reeve and Lim’s algorithm. Fig. 4 shows the Lena image coded by JPEG of a 40 : 1 compression ratio, and the processed images by Reeve and Lim’s algorithm and our method, respectively. In Fig. 4(a), the blocking artifact is remarkably noticeable. In Fig. 4(b), we can see that the blocking artifact is somewhat smoothed by the Gaussian filter, but it is still noticeable. In Fig. 4(c), we, however, can see that the blocking artifact is almost removed and there is no blurring. It can also be seen that the processed image by our method is much better in subjective image quality than the processed image by Reeve and Lim’s algorithm. For the test images JPEG coded at low compression ratios of 15 : 1 to 25 : 1, it is also found that our method shows not only a PSNR improvement but also subjective quality much better over Reeve and Lim’s algorithm and nearly free
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 8, NO. 3, JUNE
(a)
257
(b)
(c) Fig. 4. Lena image coded by JPEG of 40 : 1 compression ratio and the processed images. (a) Coded image. (b) Processed image by Reeve and Lim’s algorithm. (c) Processed image by the proposed method.
of the blocking artifact and edge blur. In addition, computer simulation for other JPEG-coded images and coded images by fractal or VQ-based coding shows a similar tendency. Furthermore, the proposed method can be implemented fast in the spatial domain without wavelet analysis and synthesis [7]. IV. CONCLUSIONS A method of reducing the blocking artifact in block-coded images by using a wavelet transform has been presented. The performance has been evaluated for 512 512 images JPEG coded with 30 : 1 and 40 : 1 compression ratios. Experimental results have demonstrated that the proposed method yields not only a PSNR improvement of about 0.69–1.06 dB, but also subjective quality nearly free of the blocking artifact and edge blur.
REFERENCES [1] K. Sayood, Introduction to Data Compression. San Francisco, CA: Morgan Kaufmann, 1996. [2] H. C. Reeve and J. S. Lim, “Reduction of the blocking effects in image coding,” Opt. Eng., vol. 23, pp. 34–37, Jan./Feb. 1984. [3] Y. Yang, N. P. Galatsanos, and A. K. Katsaggelos, “Projection-based spatially adaptive reconstruction of block-transform compressed images,” IEEE Trans. Image Processing, vol. 4, pp. 896–908, July 1995. [4] IEEE Trans. Signal Processing (Special Issue on Wavelets and Signal Processing), vol. 41, Dec. 1993. [5] N. C. Kim, D. H. Kim, and W. H. Hong, “Reduction of blocking effect in block coded images using wavelet transform,” in Proc. PCS’94, Sacramento, CA, Sept. 1994, pp. 456–458. [6] S. Mallat and S. Zhong, “Characterization of signals from multiscale edges,” IEEE Trans. Pattern Anal. Machine Intell., vol. 14, pp. 710–732, July 1992. [7] I. H. Jang, N. C. Kim, and S. M. Lee, “A fast algorithm with adaptive thresholding for wavelet transform based blocking artifact reduction,” in Proc. IASTED ICSPC’98, Canary Islands, Feb. 1998, pp. 345–351.
