INTERPOLATION OF THE DC COMPONENT OF CODED IMAGES USING A RATIONAL FILTER Giovanni Ramponi and Sergio Carrato D.E.E.I., University of Trieste via A. Valerio, 10, 34127 Trieste, Italy e-mail: framponi,
[email protected] ABSTRACT
In this paper, the problem of the interpolation of the DC components of JPEG or MPEG-coded images is addressed, and the use of nonlinear interpolators is proposed which are able to accurately reconstruct the images yielding both sharp details and low blocking artifacts.
1. INTRODUCTION In the recent years there has been an increased interest in image and video compression techniques, due to the large use of this kind of data in multimedia applications. Generally, both single images and image sequences are coded using techniques which are based on the DCT, which is applied blockwise, and a di erent treatment is given to the DC and the AC components of the transformed blocks. When the channel has limited bandwidth or is very busy, it may be useful for the receiver to be able to provide a rough reconstruction of the images when only part of the data has been received; this may be important, e.g., when browsing a database, or during data pre-selection or temporal segmentation of a sequence. This feature can be obtained using approaches based on Progressive Transmission [1] and using speci c coding techniques (e.g. Pyramidal Coding [2]), which however may be less ef cient or less standardised than conventional ones as JPEG [3] or MPEG [4]. A better solution can be given by hierarchical or layered transmission schemes, where standard DCT-based coding techniques are used, but the DC and AC coecients are transmitted with different priorities and levels of error protection [5]. In this paper, we propose a nonlinear interpolator which uses only the DC components of each block to reconstruct the original data, yielding relatively sharp images. Following this approach, it is possible to have images of good quality at low bit rates using standard coding techniques; of course, the AC components may
be transmitted later, possibly on demand, if the actual image is eventually needed. The operator we propose is based on a Rational Filter [6], which is shown to outperform conventional linear interpolators. It is well known, in fact, that a rational function (the ratio of two polynomials) has several properties (i.e., it is a universal approximator and a good extrapolator, can be trained using a linear algorithm, and requires lower degree terms than Volterra expansions) which can make it very e ective in many signal processing tasks.
2. INTERPOLATING PEG/DC IMAGES Let us call PEG/dc image a compressed image obtained by considering only the DC terms of the DCT-ed 8 8 blocks of a JPEG image or an MPEG I-frame, or by approximating the DC terms of an MPEG P- or Bframe [7]. The image can be recovered at its original size interpolating for three times the data by a factor of two. Conventional linear techniques (e.g., [8]) generally yield blurred results or introduce ringing e ects; moreover they tend to smooth the outliers. However, in PEG/dc images the outliers are always due to signi cant details rather than to noise; it is convenient, consequently, to use nonlinear interpolators which are able to preserve steep edges and small details. A further problem is that, due to the large magni cation factor (8 in each direction), any artifact which may be introduced by the interpolator is generally ampli ed and becomes quite visible in the nal image. In particular, separable operators often create jagged diagonal edges. The latter e ect can be evidenced looking at Fig. 1, where a 45 deg. edge between two uniform areas is sketched. Circled values represent original data, while the other numbers are values interpolated using a separable (bilinear) operator. It is easy to observe that only the main diagonal of the edge is uniform (its value is 6); the two adjacent diagonals are formed by alternating values (8,10,8,10,... and 2,4,2,4,... re-
10
10
10
10
10
8
6
4
10
10
10
10
10
6
2
2
10
10
10
8
6
4
2
2
10
10
10
6
2
2
2
2
10
8
6
4
2
2
2
2
10
6
2
2
2
2
2
2
(a)
Figure 2: Samples considered for (a) one- and (b) twodimensional interpolation. Data in the white boxes are available, the others have to be interpolated.
Figure 1: Interpolation of a diagonal edge using a separable bilinear operator. spectively) which give the overall visual impression of an undulating edge, indicated by the dashed curve superimposed on the data. In this case the interpolation factor is 2; the phenomenon becomes even stronger for larger magni cation. For the interpolation of PEG/dc images it is then advisable to use non-separable interpolators. Finally, it should be observed that the available data in PEG/dc images are not exact image luminances of pixels in some positions in the image, as is usually assumed when performing interpolation, but averaged values. This observation hints the possibility of using some enhancement techniques. In the following, it will be shown how a nonlinear operator based on the preceding observations can be fruitfully utilized.
2.1. The one-dimensional case
With reference to Fig. 2, let us rst consider a onedimensional interpolator which operates on four consecutive samples of decimated data, , , , and , in order to reconstruct , the central one. A nonlinear interpolator has already been proposed in [9], where a rational function is used in order to weight the contributions to of its two adjacent samples ( and ) according to the di erences ( ? ) and ( ? ); the resulting operator is able to reconstruct sharp edges more accurately than a linear lter, because it is able to roughly estimate their position with sub-pixel accuracy. However, this approach is not satisfactory with small details (e.g., one pixel wide), which tend to maintain the same size (in term of pixels) after each 2 interpolation step, so that they actually shrink with respect to larger objects. a
b
c
d
x
x
b
a
b
c
(b)
c
d
For the application we are considering, a better reconstruction of small details is required, since small objects can correspond to rather signi cant parts of the original image. An operator of the form c = b + (1) + b c with 2 b = 1+ ( ? ) 2 c = 1+ ( ? ) is able to correctly reconstruct an image border formed by a symmetric or an asymmetric luminance transition, as indicated in Fig. 3i, 3ii, and 3iii. The rst two cases are trivial: since the edge is symmetric, ? = ? and = ( + ) 2. In the third case, the asymmetrical position of the edge (to the left with respect to the center of the window) implies ( ? )2 ( ? )2, which yields a larger value for the weight c relative to the sample , so that the reconstructed edge is actually shifted to the left. The above operator, moreover, gives a strong emphasis to isolated details (Fig. 3iv) while avoiding the blurring typical of linear operators. In the latter case, in fact, the isolated sample close to , the value of which is quite di erent from those of the other samples, is weighted with a larger value ( b = 1+ ( ? )2 c ) with respect to the other sample ( ). It may be seen that this operator actually tends to enlarge small details by a factor larger than 2. A good choice can then be to suitably combine the two nonlinear operators in [9] and in Eq. (1), thus obtaining an algorithm of the form + c (2) = b + b c with 2 2 b = 1 + [( ? ) + ( ? ) ] 2 2 c = 1 + [( ? ) + ( ? ) ] w b
x
x
b
w c
w
w
w
k b
d
w
k c
a
b
d
d
b
k b
d
c
c =
c
a
>
w
c
b
x
w
c
w b
x
w
w
w c
w
k
k
b
c
w
d
a
;
c
b
d
a
;
> w
a
a
d
c
b
c
w
(ii) c
b
d
b
c
a
(iii)
d
(iv)
Figure 3: Behaviour of the detail-enhancing interpolator (Eq. (1)). Dashed lines and circles indicate the (unknown) signal to be recovered and the samples ( , , , and ) available in the 4-sample window, respectively. The crosses represent the values obtained by the nonlinear interpolator for the element in the center of the window. (i, ii) symmetric edge: ' ( + ) 2 ; (iii) asymmetric edge: ' ; (iv) detail: ' . a
b
c
d
x
x
c
b
x
c =
b
which accurately preserves the relative sizes of the objects after the various interpolation steps. With the parameter , the user can control the amount of nonlinearity in the proposed scheme; = 0 yields the bilinear interpolator. k
k
2.2. The two-dimensional case
For what concerns the problem of two-dimensional interpolation, the above mentioned interpolator can be used row- or column-wise to evaluate the pixels (as , , , and in Fig. 2b) which lie close to two already known pixels. It is easily seen that this solution is quite accurate for horizontal and vertical edges. The interpolation of the pixels in the \central" position, as in the gure, is more complicated. Both standard linear operators and the operator described above can be directly used, along the vertical or the horizontal position, using already interpolated pixels. However, with this approach (applied for three times) oblique edges tend to become jagged. Better results may be obtained using a non-separable method. In particular, we compute a weighted average of couples of pixels which are adjacent to along four directions. Couples of similar pixels are privileged. Hence: = ac( + ) +2( bd ( ++ ) ++ eg ( ++ ) +) fh ( + ) ac bd eg fh where 2 ac = 1 (1 + ( ? ) ) 2 bd = 1 (1 + ( ? ) ) e
f
g
h
i
i
i
w
a
c
w
b
d
w
w
w
e
w
g
w
w
=
k a
c
w
=
k b
d
= 1 (1 + ( ? )2 ) 2 fh = 1 (1 + ( ? ) ) It may be noted that, even though the algorithm is non-separable, there is no di erence in performing the interpolation rst along the rows and then along the columns, or vice-versa, and that the operator is isotropic with respect to the horizontal and the vertical directions. An analytical study of the proposed technique has not yet been performed. However, an idea on the behaviour of the lter on diagonal edges can be obtained by testing it on a synthetic image. In Fig. 4, a uniform circle (diameter 160 pixels) has been considered. The image obtained with our algorithm starting from the DC components of 8 8 blocks is shown. It may be noted that edges in all directions are accurately reconstructed, and no artifacts are introduced. As a comparison, the corresponding image obtained with a standard interpolation algorithm (i.e., the Keys interpolator [8]) is signi cantly more blurred and has a higher mse (146.4 instead of 54.8). weg
b
a
(i) a
d
w
f
h
=
=
k e
k f
g
h
:
3. EXPERIMENTAL RESULTS An example of the possible performance of this nonlinear operator for the interpolation of PEG/dc images is shown in Fig. 5, in comparison with a standard linear technique [8]. It may be noticed how the proposed technique accurately reconstructs edges of any orientation, and areas rich in details as well. Also a numerical comparison between these two techniques is in favour of our operator, which yields a signi cantly lower mse (267.6 instead of 501.2).
4. ACKNOWLEDGEMENT This research was supported in part by the European Esprit Project 20229{Noblesse.
5. REFERENCES [1] K.-H.-Tzou. Progressive image transmission: a review and comparison of techniques. Optical Engineering, 26:581{589, July 1987. [2] Peter J. Burt and Edward H. Adelson. The Laplacian pyramid as a compact image code. IEEE Trans. Commun., COM-31(4):532{540, April 1983. [3] W. Pennebaker. JPEG technical speci cation, revision 8. Working Document No. JTC1/SC2/WG10/JPEG-8-R8, August 1990.
Figure 4: Interpolation of a 32 32 PEG/dc synthetic image (top) to its original 256 256 size with the nonlinear interpolator, = 0 01 (middle), and using a classical interpolation algorithm [8] (bottom). k
:
[4] ISO/IEC. Coding of moving pictures and associated audio. ISO/IEC JTC 1/SC 29 WG 11, MPEG 92/No 245, July 1992. [5] Yao Wang, Qin-Fan Zhu, and Leonard Shaw. Maximally smooth image recovery in transform coding. IEEE Trans. Commun., 41(10):1544{1551, October 1993. [6] Giovanni Ramponi. The rational lter for image smoothing. IEEE Signal Processing Letters, 3(3), March 1996.
Figure 5: Interpolation of a 64 64 PEG/dc image to its original 512 512 size: (top) with the nonlinear interpolator ( = 0 01); (bottom) with a standard linear interpolator [8]. k
:
[7] Boon-Lock Yeo and Bede Liu. Rapid scene analysis on compressed video. IEEE Trans. on Circuits and Systems for Video Technology, 5(6):533{544, December 1995. [8] R. G. Keys. Cubic convolution interpolation for digital image processing. IEEE Trans. Acoust., Speech, Signal Processing, ASSP-29(6):1153{1160, 1981. [9] S. Carrato, G. Ramponi, and S. Marsi. A simple edge-sensitive image interpolation lter. In Proc. ICIP-96, pages III{711{III{714, Lausanne (Switzerland), September 1996.
