BITRATE AND BLOCKING ARTIFACT REDUCTION BY ITERATIVE PRE-DISTORTION Yiu-Hung Fok *, Oscar C. Au ** and Corina Chang Department of Electrical and Electronic Engineering The Hong Kong University of Science and Technology Clear Water Bay, Hong Kong Email:
[email protected]* and
[email protected]** ABSTRACT Block transform coding is widely used in image and video compression methods. Due to quantization error, there are blocking artifacts in the decoded image. This paper presents a pre-processing method, called predistortion, to reduce both the bitrate and blocking artifacts. The proposed algorithm is an iterative algorithm which reduces the blocking artifacts of the decoded image by introducing distortion intentionally in the encoding process. Simulation shows that, with pre-distortion, additional compression can be achieved with slight degradation in visual quality. Moreover, the blocking artifacts of the decoded image are reduced. 1. INTRODUCTION Block transform coding is widely used in different image and video compression standards, such as JPEG, MPEG-1[1] and MPEG-2[2]. Usually, an image frame is divided into non-overlapping rectangular blocks. Then Discrete Cosine Transform (DCT) is applied to each block separately followed by uniform quantization and variable length coding. Error is introduced during quantization resulting in blocking artifacts in the decoded image. Many researchers have proposed ways to reduce the blocking artifacts so that the subjective visual quality of the decoded image would be better[35]. In general, blocking artifact reduction is a postprocessing procedure. In this paper, we present a novel pre-processing method to reduce both the bitrate and the blocking artifacts of the decoded image in very-low-bitrate (VLBR) applications. We call this pre-distortion (PD) because we are introducing distortion intentionally when encoding the image. This method can be applied to different predictive transform coding standard, such as H.261 and MPEG based standard. 2. PRE-DISTORTION As stated before, the source of blocking artifacts in the reconstructed image is mainly the error introduced during quantization. Since the original natural image is
usually smooth, the error image, which is the difference image between the original and reconstructed images, should have blocking artifacts also. The blocking artifacts are caused by the lack of correlation between the errors at the common edge of adjacent blocks. Although the errors are usually quite correlated within a block, they are uncorrelated among blocks and particular across block boundaries because all the blocks are transformed and quantized independently. While the error at the edge of a block may be positive, the error at the adjacent edge pixels in the adjacent block may be negative. As a result, the errors at the block boundaries have considerably larger contrast, resulting in visually disturbing blocking artifacts. Pre-distortion is an iterative algorithm which reduces the blocking artifacts by alterring the DCT coefficients so as to increase the correlation between the edge pixels of adjacent blocks. Each nonzero DCT coefficient of a block is examined and possibly changed in such a way to re-distribute the error and reduce the blocking artifacts associated with the block. Because the un-alterred quantized DCT coefficients minimizes the mean square error (MSE) assuming uniform distribution of each coefficient, this pre-distortion procedure would generally decrease the blocking artifact at the expense of increasing the overall MSE. If the original blocking artifact is significant, if not dominating, as in the case of VLBR coding, the decrease in blocking artifact will outweight the increase in MSE such that the overall visual quality is improved. In order to avoid the propagation of error from one DCT block to another sequentially, we adopt a two-pass procedure. We divide the image into 8×8 blocks in a checker board pattern as shown in Figure 1. We examine the DCT coefficients of the shaded blocks in the first pass by assuming the white blocks are fixed. In the second pass, we repeat the same procedure on the white blocks with the shaded blocks fixed, and so on. In the algorithm, we use the following blocking artifacts measure [5]:
MSE edge ( x, y ) = 1 ------4N
MSE E =
N–1
∑
{ [ f ( x + m, y ) – f ( x + m, y – 1 ) ]
2
+
m=0
+ [ f ( x + m, y + N – 1 ) – f ( x + m, y + N ) ] + [ f ( x, y + m ) – f ( x – 1, y + m ) ]
x +E
∫x
2
(1)
2 2
+ [ f ( x + N – 1, y + m ) – f ( x + N, y + m ) ] }
x
∫x
2
( x – x ) dx 2
+E
( x – x ) dx
(2)
3 1 3 = --- E + ( 1 – E ) 3 Therefore, the maximum increase in MSE is equal to MSEE - 1/12.
3. SIMULATION RESULTS where f(x,y) is the pixel value at (x,y) and the block size is N×N. MSEedge is the sum of the MSE between each of the four block boundaries with the corresponding adjacent block. Here is the algorithm. Step 1 Divide an image into 8×8 blocks in a checker board pattern as shown in Figure 1. Perform DCT and any scaling before the uniform quantization. Step 2 Consider the shaded blocks. For each scaled DCT coefficient x in a block, such that round ( x ) ≠ 0 , replace x by and x and calculate the x corresponding MSEedge defined in (1). The one that yields a smaller MSEedge will replace x. If round(x) = 0, then x is replaced by 0. However, if the absolute difference between x and round(x) is smaller than a threshold E, x will be replaced by round(x). Step 3 Repeat step 2 using the white blocks. Step 4 Repeat step 2 and 3 until the incremental decrease in the mean MSEedge of all blocks within the frame is less than a threshold T. Note that if x is less than unity, the pre-distortion procedure may create a zero "quantized" DCT coefficient, which will translate into higher compression in general. This is effectively a quantizer with selective deadzone. The threshold E in step 2 is used to control the amount of error added by pre-distortion. Assuming the error is uniform distributed from x to x , the total MSE of using threshold E is given by:
We used 90 frames of Miss America sequence, with size 176×144 (QCIF), as the input of the simulation. The sequence is motion compensated transform coded using H.261 framework. There are 14 Predictive-coded pictures (P-Pictures) between 2 Intra-coded pictures (IPictures). We apply pre-distortion to all the blocks except those at the boundaries of the frame. For the Ipicture, pre-distortion is applied to the image. For the Ppictures, pre-distortion is applied to the residue images. For the proposed algorithm, the P-pictures are predicted from the previous reconstructed picture with predistortion. Table 1 shows the average PSNR, average blocking artifact measure and the average number of bit used to code the sequence using the proposed method. In addition, Figures 2 and 3 show the profiles of MSEedge and PSNR respectively. Figures 4 to 6 show the reconstructed 32th frames without and with pre-distortion. Figures 7 to 9 show the reconstructed 61th frames without and with predistortion. The 32th and the 61th frames are P-picture and I-picture respectively. From Table 1, we find that the proposed algorithm can indeed reduce the bitrate and MSEedge at the expense of considerable reduction in PSNR due to the intentionally added distortion. With 1 iteration, a bitrate saving of 7% and a MSEedge reduction of 40% is achieved with a corresponding 0.54dB drop in PSNR. Comparing Figures 5 and 8 with Figures 4 and 7, the resulting visual quality using 1 iteration of pre-distortion is quite acceptable. This is particularly useful for VLBR applications in which bitrate must be kept very low while visual quality needs only to be acceptable. More bitrate and MSEedge reduction can be achieved with more iterations at the expense of further drop of PSNR resulting in more visual degradation 4. CONCLUSION A novel pre-processing algorithm, pre-distortion, for reducing both the bitrate and the blocking artifacts is
presented. With pre-distortion, additional compression can be achieved with certain degradation in visual quality. Moreover, the blocking artifacts as measured by MSEedge of the decoded image are significantly reduced. This provides a trade-off between the reduction of blocking artifacts and overall MSE. Lastly, this is only some preliminary results, further research work will be carry on in this area. 5. ACKNOWLEDGMENTS This project is supported in part by the HKTIIT grant #HKTIIT 93/94.EG02. Figure 1: Checker Board Pattern 6. REFERENCES
[3]
[4]
[5]
PSNR
MSEedge
Bit
No pre-distortion
34.32
334.5
3202
1 iteration
33.78
201.8
2986
2 iterations
33.23
162.3
2890
3 iterations
32.88
155.3
2852
Table 1: Performance using pre-distortion
missA 700
600 no pre−distortion PD, 3 iteration PD, 2 iteration PD, 1 iteration
500
MSE_edge
[2]
International Standard ISO/IEC 11172, "Coding of moving pictures and associated audio for digital storage media up to about 1.5Mbits/s," Nov. 1992. International Standard ISO/IEC 13818, "Information Technology - generic coding of moving pictures and associated audio," Mar. 1994. S. A. Karunasekera and N. G. Kingsbury, "A distortion measure for blocking artifacts in images based on human visual sensitivity," IEEE Trans. on Image Processing, vol.4, no.6, June 1995, pp. 713-724. Y. Q. Zhang, R. L. Pickholtz, M. H. Loew, "A new approach to reduce the ’blocking effect’ of transform coding (image coding)," IEEE Trans. on Communications, vol.41, no. 2, Feb 1993, pp. 299-302. B. Jeon, J. Jeong and J. M. Jo, "Blocking Artifacts Reduction in Image Coding Based on Minimum Block Boundary Discontinuity," Proc. of SPIE on Visual Comm. and Image Processing 95, vol. 2501, pt. 1, Taiwan, 24-26 May 1995, pp. 198-209.
400
300
200
100
0 0
10
20
30 Frame
40
50
60
Figure 2: MSEedge using pre-distortion missA 37
36
35
PSNR
[1]
34
33
32
31 0
no pre−distortion PD, 3 iteration PD, 2 iteration PD, 1 iteration 10
20
30 Frame
40
50
Figure 3: PSNR using pre-distortion
60
Figure 4: 32th frame without pre-distortion
Figure 7: 61th frame without pre-distortion
Figure 5: 32th frame with pre-distortion, 1 iteration
Figure 8: 61th frame with pre-distortion, 1 iteration
Figure 6: 32th frame with pre-distortion, 2 iterations
Figure 9: 61th frame with pre-distortion, 2 iterations
