Region-of-interest coding based on set partitioning in ...

REGION-OF-INTEREST CODING BASED ON SET PARTITIONING IN HIERARCHICAL TREES Keun-hyeong Park', Chul So0 Lee and HyunWook Park Dept. of Electrical Engineering, KAIST

373- 1 Guseong-dong, Yuseong-gu, Daejeon 305-701, Korea hwp ark @ athena. kaist .ac.kr *Hanaro Telecom, Inc. Seocho-dong, Seocho-Ku, Seoul 137-728, Korea 2. ROI CODING

ABSTRACT In many image coding applications such as web browsing, image database, and telemedicine, it is useful to reconstruct only a region of interest (ROI) before the rest of the image is reconstructed. In this paper, an ROI coding functionality is incorporated with the set partitioning in hierarchical trees (SPMT) algorithm [ 13. By emphasizing the ROI coefficients, they are coded with higher fidelity than the rest of the image in early stages of progressive reconstruction. The main thrust of this research is how to identify necessary coefficients for the decoder to reconstruct the desired region. The proposed method provides better performance than the previous method. 1. INTRODUCTION

It is essential for the coder to provide a good ratedistortion performance. In addition, other requirements become important in image compression. An example of such requirements is the capability to code a region of interest (ROI) with high priority. It is desirable to incorporate the feature into image coding systems without incurring heavy cost such as increased computational complexity or reduced rate-distortion performance. Atsumi and Farvardin proposed a method to incorporate the ROI coding functionality [2]. However, its performance is not satisfied. In this work, the ROI coding functionality is incorporated into the SPIHT algorithm without degradation of PSNR and computation complexity. Necessary data for the decoder to reconstruct the desired region can be identified with the proposed ROI coding method. This paper is organized as follows: Section 2 presents a new ROI coding method. Section 3 shows experimental results comparing the proposed method with the previous work and the original SPMT. Finally, conclusions are given in Section 4.

0-7803-6725-1/01/$10.000200 1 IEEE

804

2.1. Parent of ROI

In the previous ROI coding algorithm [2], each node is examined whether it is necessary for the decoder to reconstruct the ROI, i.e., whether it is an ROI coefficient, before testing whether it is significant (node test). If the node is not an ROI coefficient, the node test is skipped and taken later. However, each node is tested whether its descendants are significant (descendant test) disregarding whether its descendants are ROI Coefficients. If any descendants of the node are not ROI coefficients, the values of its descendants are unnecessary for the decoder to reconstruct the desired region. We define a PROI coefficient, one of whose descendants is an ROI coefficient. Thus, if we know that a node within the spatial orientation trees is not a PROI coefficient, the coder need not take its descendant test. However, we cannot know whether a node is the PROI coefficient [3]-[5].Thus, we need a new mask of PROI, which indicates the nodes that have ROI coefficients as their descendants. The parent of ROI (PROI) mask is a bit plane indicating which nodes are the PROI Coefficients. The generation of the PROI mask is depicted in Fig. 1. Except the lowest pyramid level, the relation between the ROI mask and the PROI mask is defined as follows,

where PROI(.) and ROI(.) represent the coefficients of the PROI mask and the ROI mask, respectively. Desck(i,j) represents the k-th descendant of the node (iJ), and v denotes the logical OR operator. 2.2. Divisions of the list of insignificant pixels and the list of insignificant sets

Descendant Test

( Threshold T= ,

2". N = ~ o g Z ( m a ~ ' , , { l c , , l } ) l )

ROI Mask

13 FRO1 Mask

Fig. 1. Generation of PROI mask from ROI mask

In the proposed ROI coding algorithm, the node test is skipped if the node is not an ROI coefficient, and the descendant test is also skipped if the node is not an PROI coefficient. After finishing the encoding process of the ROI, the list of insignificant pixels (LIP) and the list of insignificant sets (LIS) contain the nodes that entered the lists at various threshold values during the ROI coding. However, if we use only one LIE' and one LIS, we cannot distinguish the nodes in the lists. This problem can be settled by dividing the LIE' and the LIS into several LIPSand LISs. We divide the LIP and the LIS as follows. At the beginning of tests, if the N-th bitplane is the most significant bit-plane (MSB), the nodes in the highest tree level are added to L P N , and only those with descendants are added to LISN as type A in Fig. 2 [ 13. If a node becomes significant during the descendant test, its four offsprings enter an LE',, or an LIS, according to the threshold (bit plane number n). In case of being added to the LIS,,, these four offsprings are tied together (type B [ 11) to improve the coding efficiency and represented by their parent node. The offsprings are separated (type A) if they become significant during the descendant test as shown in Fig. 2. Node Po is an entry of the LISN at the beginning of the descendant test. If Po becomes significant at n = N, its four offsprings (PI, P2, P3, and P4) are added to LIPN.And Porepresenting its four offsprings is added to LISN. Actually, the type of Po is just changed into B, because Pohas been in LISN.If Po (type B) becomes sig-

805

4

L1pN-2

.

Fig. 2. Divisions of the LIP and the LIS: Each offspring node is added to an LIP,, and an LIS, according to the bitplane at which it becomes the root of the subtree. nificant at n = N-1, the four offsprings of Po are separated and added to LISN as type A. Because PI, Pz, P3, and P4 become the roots of the subtrees at n = N, these nodes are added to LISN. In this way, the LIP and the LIS are divided during the ROI coding.

2.3. Proposed ROI coding algorithm We use a function

to indicate the significance of a set of coordinates T and use the following sets of coordinates as in the original SPIHT [ l ] . O(i,j):set of all offsprings of node (i,j); D(i,j): set of all descendants of node ( i , j ) ; L(i,j ) = D(i,j) - O(i,j). In the proposed system, once the ROI is identified, the coder modifies the transmission order to emphasize the

~

ROI coefficients. That is, to refine the ROI earlier than the rest of the image, the coder just reorders the embedded bitstream. The proposed ROI encoding algorithm works as follows: 1) Start encoding the wavelet transform (WT) coefficients as in [ 11 (start at n = N). 2)As soon as the shape of the ROI is available to the encoder (the encoder can get to know the shape of the ROI at the beginning of the process or the decoder may inform the encoder of the shape of the ROI), interrupt Step 1) and perform the followings. 2.1) Create the ROI mask and the PROI mask. 2.2) Specify the relative importance (R) of the ROI compared to the rest of the image. The smaller the value R is, (the more significant the ROI is,) the higher the quality of the ROI is while the lower the quality of the other area is. 2.3) Start the ROI encoding as shown in Fig. 3 from the beginning of the (Q+1)-th stage (suppose that after completing the Q-th stage of the encoding algorithm and transmitting the corresponding bit stream, the encoder or the decoder identifies the ROI), i.e., n = NQ, to the end of the stage when n = R. 3)Repeat the Step 2.3) for the coefficients that are excluded from the ROI coding with the threshold bit plane from n = N-Q to n = R. 4) Resume encoding the overall WT coefficients as in [ 11 (start at n = R-1). In Fig. 3, the steps starting with # are modified by the previous encoding algorithm [2] and the steps starting with $ are the modification of the proposed encoding algorithm from the previous encoding algorithm. The rest steps and the notations are the same as the original SPMT [ 11. The decoding process of the proposed algorithm can be deduced from the encoding process. 3. EXPERIMENTAL RESULTS

The following results are obtained from monochrome Lena image with 256x256 and 8 bit/pixel (bpp) for fixed rate coding (i.e., without arithmetic coding). Five levels of discrete wavelet decomposition with the 9/7-tap biorthogonal wavelet filters are employed [6]. The ROI is specified as the 5 1x5 1 square shaped region containing an eye of the girl as marked in Fig. 6. The switching bit rate the bit rate when the ROI coding mode starts - is 0.058 bpp. Because the SPIHT shows better visual quality than P E G at very high compression, a bit rate of 0.058 bpp provides an “understandable” image for a user to choose the desired ROI. Fig. 4 shows the PSNR values of the entire image and the ROI at R = 4. This shows that the proposed method yields much better quality than the previous ROI coding method.

806

~~

) Modified Sorting Pass:

a. 1) for each entry (ij) in the LIPS do: # if (U) belongs to the ROI coeficients do: a. 1.1) output S,(i,j); a. 1.2) if S,(i,j)=l then move (ij) to the LSP and output the sign of c ~ , ~ ; a.2) for each entry (ij) in the LISs do: a.2.1) if the entry is of type A then $ i f ( i j ) belongs to the PROI coefficients do: - output S,(D(i,j)); - if S,(D(i,j))=l then - for each (k,l) belongs to O(ij) do: # if(k,l) belongs to the ROI coeficients do: - output Sn(k,l); - if S,(k,l)= 1 then add (k,l) to the LSP ana output the sing of ck,,; - $ if S,(k,l)=O then add (k,l) to the ena of the LIP,; $ if (k,l) doesn’t belongs to the ROI coefficients, move (k,l) to the end of the LIP,; - $ if L(ij) != 0 then move ( i j ) to the end OJ the LIS, as an entry of type B, and goto Step a.2.2); otherwise, remove entry (ij; from the LISs; a.2.2) if the entry is of type B then $ if ( i j ) belongs to the PROI coefficients do: - output S,(L(i,j)); - if S,(L(i,j))=l then - $ add each (k,l) that belongs to O(ij) to the end of the LIS,,,as an entry of type A (S,(D(ij))has become 1 at n = m); - remove (ij) from the LISs; Refinement Pass: for each (ij) in the LSP, except those included in the last sorting pass, # if(i,j) belongs to the ROI coefficients do: - output the n-th most significant bit of Iqjl; ) Quantization-Step Upgrade: decrement n by 1 and goto Step a) if n >= R

1)

Jig. 3. The proposed encoding algorithm in ROI coding mode This means that the PROI mask and the multiple LISs and LIPS reduce the redundant node tests, which are not necessary for the decoder to reconstruct the desired region. Figs. 5 and 6 show the reconstructed images obtained at 0.1 bpp and 0.5 bpp with the SPIHT and the proposed method (R = 4), respectively. The regions containing eyes in Fig. 6 are reconstructed with higher visual quality than those in Fig. 5. Thus, the proposed ROI coding technique can allow users to quickly view a small portion of the image with higher quality without receiving the entire image.

4. CONCLUSIONS

f

22 Z

An ROI coding functionality ,is incorporated into the SPIHT. Because it only modifies the transmission order of the SPMT to emphasize the ROI, it is achieved without degradation of PSNR and computation complexity. The proposed scheme is so flexible that users can request an ROI or several ROIs at any moment. In the proposed algorithm, the PRO1 mask indicates the coefficients that have at least, one ROI descendant within the spatial orientation trees. In addition, this algorithm divides the LIP and the LIS into multiple LIPS and LISs to identify necessary datai.when the decoder reconstructs the image. The ROI coding is valuable in interactive ClienUserver applications linked through narrowband networks. The greatest advantage of the proposed ROI coding is to incorporate a new functionality while its performance is competitive with the original SPIHT algorithm.

O

0

f

’ 01

‘

’

02

03

04

I

”

0 5

06

”

07

OB

09

1

Blt rate ( b p ~ 0

5. REFERENCES

I

O

01

01

03

04

05

06

07

Eit rate (LIDKI)

Fig. 4. PSNR comparison (a) PSNR for the entire image. (b) PSNR for the ROI ( R 4 ) .

(a) (b) Fig. 5. Images obtained from SPIHT. (a) 0.1 bpp (b) 0.5 bPP

(4

(b)

Fig. 6. Images obtained from the proposed method ( R 4 ) . (a) 0.1 bpp (b) 0.5 bpp

807

[ l ] Said and Pearlman, “A new, fast, and efficient image codec based on set partitioning in herarchical trees,” IEEE Truns. Circuits and Systems for Video Technology, Vol. 6, pp. 243-250, June, 1996. [ 2 ]E. Atsumi and N. Farvardin, “Lossyflossless region-ofinterest image coding based on set partitioning in hierarchical trees,” IEEE International Conference on Image Processing (ICIP-98), October 4-7, 1998 Chicago, Illinois, USA. [3] C. Christopoulos, J. Askelof, and M. Larsson, “Efficient region of interest coding techniques in the upcoming JPEG2000 still image coding standard,” IEEE International Conference on Image Processing (ICIP-2000), September 10-13, 2000 Vancouver, Canada. C. Christopoulos, “JPEG2000 Verification Model 7.0 (Technical description),” ISO/IEC JTC l/SC 29/WG 1 WGlN1684, April 25,2000. D. S. Crus, T. Ebrahimi, M. Larsson, J. Askelof, and C. Christopoulos, “Region of interest coding in JPEG2000 for interactive clienUsever applications,” 1999 IEEE 3rd Workshop on Multimedia Signal Processing, pp. 389-394, 1999. [6] M. Antonini, M. Barlaud, P. Mathieu, and I. Daubechies, “Image coding using wavelet transform,” IEEE Trans. Image Processing, Vol. 1 , pp. 205-220, April, 1992.