Color image coding by using the technique of surface fitting - Pattern

Color Image Coding by Using The Technique of Surface Fitting * Y . S. Chen

H. T. Yen

Institute of Electrical Engineering National Tsing-Hua University Hsinchu, Taiwan 300, ROC

Department of Electrical Engineering Yuan-Ze Institute of Technology Taoyuan, Taiwan 320, ROC Abstract

color components RGB or YIQ are assumed) [3]. If these surfaces can be approximated with fewer data than original, the purpose of image data compression is achieved. Surface fitting allows spare inforrnation to be fused into a continuous visual surface [4]. If the surface is smoothly distributed, the surface can be well fitted by fewer sampling data. On the other hand, if the surface is actively distributed, more data will be required to generate satisfied fitting. To achieve efficient coding performance, this spatial characteristic should be exploited thoroughly. In this paper, a hierarchical surface fitting system is proposed for this purpose. The system performs a progressive approximation process for each component surface independently. The coding procedures of each component are quite similar.

A n adaptive color image coding system via the technique of surface fitting is proposed in this paper. The coding strategy is directly applied to the YIQ signals. I n the coding procedure, each component is regarded as a curved surface in the spatial domain and preprocessed with a separated filter to filter out the high frequency parts. The surface fitting technique is applied to describe these surfaces. A progressive approxcimation approach f o r the description of each color componeni is also designed to process the different activity regions in a color image. Three different complexity of images with are tested in our experiment.

1

Introduction

In general, a lot of redundant information exist in the source data of images. The purpose of image compression is to reduce these redundancy efficiently. In general, the adjacent pixels are highly correlated, e.g. in a monochrome picture the adjacent pixels are similar in gray values; while in a color picture the adjacent pixels are similar in luminance and chrominance values. Such information is often utilized for intra-frame image coding. During the past decades, numerous intraframe image coding techniques for compression such as differential pulse code modulation (DPCM), vector quantization (VQ), transform coding (e.g. DCT, DFT), hybrid coding and adaptive versions of these techniques have been developed [l].These techniques were originally developed for monochrome image compression. Meanwhile, most of them can be extended to color ima e coding which also achieve acceptable performance 721. In this paper, we propose a new coding method for the use of color image compression. The principal ideal is stated as follows: Since the variations of each color component in spatial domain can be thought as a surface by using the brightness values for the third coordinate, the expression of a color image is reduced to the description of three curved surfaces (when three

2

The Proposed Method

The whole procedure of our coding system is described as follows. The source data entering the system are the digital RGB signals of an color picture. Because our coding system is designed for YIQ components, it is necessary to convert the digital RGB signals to YIQ signals before coding. And a t the end of the receiver, the YIQ signals need again to be converted into RGB signals for the color monitor to display. Before the surface fitting procedure, the YIQ signals are preprocessed with separated filter independently. After the process of separated filter, the high frequency parts of each component will be filtered out. Then, the adaptive cornponent coding procedure is performed on each component. The component encoder is a three-step progressive approximation system which includes the initial surface fitting and two-pass improvements. The initial surface fitting process is performed to get the low level vision of the component. After the initial fitting, the activity condition of each block will be judged by the MAE (mean absolute error) between the initial fitting and original component. Thus the adaptive improvement process can be performed according to the activity conditions. Two passes of adaptive improvement process are required to get satisfied results. We use condition 0, 1 and 2 to indicate the activity condition of each block. In the decoder, the reconstruction of each component is a progressive approximating

*The ideal of this article is proposed by Dr. Chen, and a further new related research concerning the gradation process of color images has being proceeded by a project. thstitute of Information Science, Acadenlia Sinica, Taipei, Taiwan 1 1 5 , ROC.

37 0-81862920-7192$3.00 0 1992 IEEE

W. H. Hsut

whose MAEs are less than an error tolerance TL1 are called condition 0 block, other patches are called condition 1 blocks. The error tolerance TL1 is a heuristic value. The adaptive, say 4 x 4, subsampling procedure is performed according to the activity conditions of all blocks. In condition 0 blocks, the values in all 4 x 4 vertices are replaced with 0; while in condition 1 blocks, the values in all 4 x 4 are sampled from the error surface. Then the basic surface fitting algorithm is reapplied to these 4 x 4 network of points and the fitting results are added to the initial surface for quality improvernent. And in step 3, the MAEs between the original surface and improved surface are estimated again for all patches. Those patches with MAEs larger than TL2 are denoted as condition 2. The adaptive improvement process is performed in 2 x 2 manner to condition 2 patches. The difference between step 2 and step 3 is that, the improvement procedure in step 3 is in local manner and only the condition 2 patches required to be locally improved. It is worth discussion that the surface improvement strategy in step 2 is in global manner, which for step 3 is in local manner. Because that a local improvement process in step 2 will cause value jump near the edge of patches. This effect will affect the fitting quality. Since the continuity characteristics are almost extract out in steps 1 and 2, the adoption of local improvement in step 3 will not cause surface jump. In addition, the adaptive local surface fitting process is faster than the entire surface fitting. The decoding procedure is similar to the encoding procedure. The received data in the decoder are the activity status bits and the quantized component data in each pass sampling. Thus the decoding procedure is a progressive reconstruction process which appropriately utilizes these data to interpolate the desired surface. Before the procedure of reconstruction, the activity conditions for all patch as are required to be recognized according to the activity status bits. It is also a three steps approximation procedure according to the component data and activity conditions.

progress according to the activity conditions of each block. The more details of separated filter, adaptive component coding, and coding strategy are described below.

Separated Filter The purpose of separated filter is to filter out those parts with different spatial characteristics in an image. Processed with this kind of filter, the high frequency parts will be filtered out and the low frequency parts will be left. The low frequency parts of an image are those with high spatial correlation. Coding strategy applied to such kind of image can often achieve higher performance. There are several kinds of separated filter, such as the well known 3 x 3 average mask. This filter takes an average of a pixel with its 8 neighbors. The filter will cause an image blurred everywhere. Kaneko el al proposed a separated filter which can filter out the edge-like and pulsing pixels in both the horizontal and vertical directions of an image 51. In our method, we adopt the Kaneko e2 ai’s algorit m but change the median operation by the mean operation, since the mean operation is easier to implement and the performance would not be affected. After the preprocessing with this separated filter, we will avoid sampling the noisy or sharp edge pixels which will severely influence the surface fitting results of its neighbor regions.

6

Adaptive Component Coding The uniform bicubic B-spline surface fitting is a uniform interpolation technique. The choice of patch size will decide the trade off between image quality and compression ratio. Thus we design a simple experiment for YIQ components separately. The basic bicubic B-spline surface fitting algorithm is applied to each component at different patch size. Considering the adaptivity and the quality of image, we adopt an adaptive progressive approximation method for component coding. The approximation procedure for each component is from large patch size to small patch size. That for Y component is in a manner from 8 x 8 + 4 x 4 -+ 2 x 2, and that for I and Q components are in a manner from 16 x 16 + 8 x 8 -+ 4 x 4. Since the coding procedure of each component is quite similar, we take Y component for example to illustrate the coding procedure. The encoding procedure can be divided into the following three steps. In step 1, the surface is divided into many 8 x 8 blocks to generate the initial surface fitting. The initial fitting will give us a low level impression of the Y component. In the initial fitting, the smooth parts of the component are well fitted and the active parts are not. Thus the adaptive improvement process can be performed to improve the approximation results of these regions. In step 2, the first pass improvement is performed to generate a surface that interpolates the error value between the initial fitting and the source component. The fitting process is called error surface fitting. The adaptive error surface fitting procedure is performed by MAE measurement and adaptive error surface sampling and fitting. The MAEs between the original surface and fitting surface are estimated for each surface patch. Those patches

Coding Strategy The data required to reconstruct the approximation component surface is the sampling data in three steps. In addition to these signals, the identified code are required to distinguish the activity conditions for every block. The transmission sequence of data is as follows. (1) The identified code of every block. The initial sampling data a t component surface. The first pass sampling data a t error surface. (4 The second pass sampling data a t error surface. At t e receiver, a progressive reconstruction will be performed. In our system, the component blocks are divided into three groups. They are expressed as condition 0, 1, and 2 to denote their activity status, respectively. Activity condition 0: such kind of blocks are well tted after the initial surface fitting. The code 0 is used as their identified code. The transmitted data for condition 0 blocks are just the initial sampling data on component surface. (2) Activity condition 1: such kind of blocks are well approximated after first pass improvement. The code 01 is used as their identified

1,

p

38

average MSE and average SNR to measure the overall distortion between the reconstructed and original color image. From the coding strategy described previously, the bit rate can be defined as follows.

code. The transmitted data for condition 1 blocks are the initial sampling data on component surface and the first sampling on error surface. (3) Activity condition 2: such kind of blocks are the most active blocks which require the second pass improvement to achieve satisfied approximation. The code 11 is used as their identified code. The transmitted data for condition 2 blocks include the initial sampling data on component surface, the first pass and the second pass sampling data on error surface. The dynamic range of RGB components is 0 255. The YIQ components are a linear transformation of RGB components. The dynamic range for Y component is 0 255, while it is -128 +128 for both I and Q components. Although most natural image will not extend the full range, we will regard these as their full range. Let the quantization step size be 4 for each color component. Take Y component for example. (1) Initial sampling data on component: The step size is 4 levels and 6 bits are assigned to each sampling point, then the maximum quantization error is only 2 levels. (2) First pass sampling data on error surface: The step size is 4 levels and 4 bits are assigned to each sampling point on error surface. The representation range is -32 +32. Through the analysis of a few statistics data, the error after initial surface fitting is within this range for almost all natural image. If some points are out of this range, they are often unusual points. These points will cause bad effect in its neighbor point. The use of just our full range to represent it will reduce such effect. (3) Second pass sampling data on error surface: The step size is still 4 levels and 4 bits are assigned to each sampling point on error surface. The strategy and representation range are the same with those in (2).

BPP = C ( N V

-

-

N V = (number of initial sampling vertices) x 6 bits. EPl = (number of condition 1 patches) x 4 x 4 bits. EP2 = p m b e r of condition 2 patches x (16 + 4 ) x 4 bits. A = (number of activity status its) = (NPO x 1 + NPl x 2 + NP2 x 2) bits. N P i = (number of condition i patches), i = 0, 1, 2. TP = Total number of pixels (512 x 480). B P P = Average number of bits

-

L

per pixel. The bit-rate of a color image is the bit-rates sum of YIQ components. Since the original picture required 24 bits for each pixel, the compression ratio of the coding system is defined as

Compression ratio = 24 / B P P . Based upon our three test irnages, the cloud image is the most simple one which is quite smooth in the every component. The lenna image is an image with general complexity. The mandrill image is the most complex one. In our experiments, the error tolerances TL1 and TL2 for each component are set by T L l = 9, TL2 = 14 for Y component, and T L l = 7, TL2 = 12 for I and Q components, respectively. The compression ratio is about 122.5 for the cloud image, 76.9 for the lenna image, and 35.0 for the mandrill image, respectively. The original and reconstructed color images are shown in Figs. 1-3. And the detail quantitative data of distortion and bit-rate are shown in Tables 1-3.

-

3

+ E P i + EP2 + A S ) / T P .

YIQ

Experiments

The original test images are of size 512 x 480 pixels with each R, G I and B components values uniformly quantized to 8 bitslpizel. Three images of different complexity are named (A) cloud, ( B ) lenna, and ( C ) mandrill. The mandrill and lenna test images are acquired from bitnet at Princeton (BITFTP@PUCC). The cloud test image is acquired from video tape by our image grabber system. The analysis of experiments consists of two parts, one is image quality measurement and the other is bitrate evaluation. There are two kinds of measurement for image quality. The first is judged by the subjective fidelity for the reconstructed image in comparison to the original image. The CIE defined a color-difference formula which measure the color difference between two pixels by the Euclidean distance on L*u*w* color space. Frei and Baxter defined a distortion measurement by applying the MSE (mean square error) to the G-space proposed by themself [6]. The author claimed that the measurement is quite agree with subjective criteria. The MSE and SNR (signal-to-noise) criterion are adopted here as image quality measurements for YIQ components separately. And we also define the

4

Concluding remarks

Surface fitting is an important technique in surface description. Its excellent performance in surface representation has been proved. In this paper, the YIQ components of a color image are regarded as three curved surface and the adaptive surface fitting system has been proposed to describe each one of them. The experimental results have confirmed that the system is robust for pictures of different complexity. (1) Steady image quality is achievable through the proper choice of MAE thresholds TL1 and TL2. (2) If steady bitrate is required, the modified block numbers in either pass will be controlled to satisfied this purpose. Our experiments have shown that our system is robust for the two applications.

Acknowledgements This work was supported by National Science Council of Republic of China, Research Contract No. NSC 81-0408-E007-530. The authors also want to thank Miss W.P. Chang and Mr. C.Y. Kang for their help in preparing the related materials of this paper.

39

Fig. 2. The lenna image. (above) original, (below) reconstructed image at 0.312 bitslpixel.

Fig. 1. The cloud image. (above) original, (below) reconstructed image at 0.196 bitslpixel.

References

Table 1. Performance evaluation of the cloud image. ~

~

~

~~~~

~~~

Cloud

%MSB

SNR

Bit-Rate

Condition 0

Condition I

Condition 2

Y I Q

0.0422 0.0056 0.0056

33.75 42.49 42.49

0,138 0.029 0.029

95.9% 100% 100%

2.6%

I .5%

0.0%

0.0% 0.0%

A S M E 0.0178%

ASNR: 37.50db

0.0%

[l] A . K . Kain, “Image data compression: a review,” Proceedzng of the IEEE, Vol. 69, pp. 349-389, Mar. 1981.

[2] H. H. Chen, Y. S. Chen, and W. H. Hsu, “Low-rate sequence image coding via vector quantization,” Szgnal Processzng, Vol. 26, No. 2, 1992.

Tom1 Bit-Rate: 0.196 bap

M S E 40 SNR: db Bit-Rae: billpixel (bpp)

Table 2. Performance evaluation of the lenna image. lenna

%MSE

SNR

0,1269 0.0580 0.0314

28.96 32.37 35.03

~~

~~

Y I Q

Bit-Rate

Condition 0

Condition I

Condition 2

0.239 0.041 0.032

71.1% 82.6% 95.3%

23.9% 17.3% 4.7%

5.0% 0.1% 0.0%

[3] T. Pavlidis, Algorithms for Graphacs and I m a g e Processzng, New Jersey: Computer Science Press, 1982.

~

[4] W. E. L. Grimson, “An implementation of a computational theory of visual surface interpolation,” CVGIP, Vol. 22, pp. 39-69, 1983.

~

ASME 0073%

ASNR 31 37db

TotalBit-Rate 0 3 1 2 bpp

_____

[5] M. Kaneko, Y. Hatori, and A. Koike, “Improvements of transform coding algorithm for motion-compensated interframe prediction errors DCT/SQ coding,” IEEE Trans. SAC, Vol. 7, Aug. 1987.

Table 3. Performance evaluation of the mandrill image. mandrill

%MSE

SNR

Bit-Rate

Condition 0

Condition 1

Condition 2

Y

0.6240 0.2592 0.0770

22.05 25.86 31.13

0.566 0.085 0.034

29.5% 45.8% 93.0%

43.8% 46.3% 6.9%

26.7% 7.9% 0.1%

I Q

Fig. 3. The mandrill image. (above) original, (below) reconstructed image at 0.685 bitlpixel.

A S M E 0.3201%

ASNR: 24.95db

[6] W. Frei and B . Baxter, “Rate-distortion coding simulation for color images,” I E E E Trans. CO~JLmun., Vol. 25, No. 11, pp. 1385-1392, Nov. 1977.

Total Bit-Rate: 0.685 bpp

40