context based lossless coder based on rls predictor ... - IEEE Xplore

CONTEXT BASED LOSSLESS CODER BASED ON RLS PREDICTOR ADAPTION SCHEME Grzegorz Ulacha, Ryszard Stasinski West Pomeranian University of Technology, and Poznan University of Technology, Poland ABSTRACT

2. LOSSLESS CODING BASICS

In the paper highly efficient context image lossless coder of moderate complexity is presented. Three main plus few auxiliary contexts are described. Predictors are adaptive, enhanced RLS coefficient update formula is implemented. A stage of NLMS prediction is added. Prediction error bias is removed using a robust multi-source approach. An advanced adaptive context arithmetic coder is applied. Experimental results show that indeed, the new coder is both more effective and faster than other state-of-the-art algorithms.

Classical entropy coding schemes, like Huffman, Golomb, arithmetic, are designed with coding of memoryless data sources in mind [7]. That is why all practical lossless coding techniques start with a data modeling stage, which role is to reduce pixel dependencies. A good measure for the stage efficiency is the decrease in image zero-order entropy:

Index Terms— Lossless coding, RLS prediction, NLMS prediction, adaptive context arithmetic coder.



emax

¦p

i

˜ log 2 p i

(1)

i emin

where pi is the probability of i-th source symbol, the range of i is from emin to emax. Zero-order entropy forms the lower bound on the bitrate when coding a memoryless source [7]. 2.1. Predictors

1. INTRODUCTION The end of previous century was particularly important for lossless image coding methods. In 1996 the CALIC algorithm was presented [1], the first widely recognized context coding method. Another important context coding technique of that time has been LOCO-I, used in JPEG-LS standard [2]. Being better than LOCO-I, CALIC algorithm was too computationally complex to implement in a standard. Today the situation is different, the place of CALIC has been taken by even better but much more complex techniques like TMWLEGO [3], WAVE-WLS [4] and the new version of MRP 0.5 – VBS & new-cost [5]. Except for MRP the algorithms are based on a highly promising predictor blending approach. Wavelet transforms have been applied to lossless coding [6], too, however, this approach doesn’t appear to be particularly efficient so far. In the paper a context lossless coding method having medium computational complexity but very high coding efficiency is presented (few seconds to few tens of seconds image processing time). It is based on improved version of RLS method for predictor coefficient adaptation, section 3.1, hence, it is proposed to name it RLS+, additionally, the output is processed by NLMS algorithm, section 3.2. Three principal and some auxiliary contexts are implemented, section 2.2. A sophisticated prediction error bias removal method has been adopted. High algorithm performance would not be possible without sophisticated context arithmetic coder, section 4. Experiments show that indeed, the new algorithm is the best in its class, section 5.

978-1-4244-5654-3/09/$26.00 ©2009 IEEE

H

1917

A linear predictor provides expectation value of coded pixel: r

¦ b ˜P ( j )

xˆ n

j

(2)

j 1

where P(j) are pixels from the coded pixel xn neighborhood, bj are predictor coefficients, and r is predictor rank. Of course, pixel precedence is an ambiguous term, e.g. for line after line scan only the pixels above and to the left can be taken into consideration, see Fig.1. The (rounded) pixel estimate is used for computing prediction error en: en

x n  xˆ n

(3)

The farthest pixels from the coded one provide the worst contribution to its estimate. It is then reasonable to use Euclidean distance when ordering neighboring pixels, Fig.1. shows pixel indices used in (2) for the first thirty j values. 26 24 27 29 20 16 14 17 21 30 19 11 8 6 9 12 22 25 15 7 3 2 4 10 18 28 23 13 5 1 xn Fig. 1. Numbering of neighborhood pixels.

ICIP 2009

2.2. Contexts In context coding techniques several predictors are defined, each linked with some context. The context value is a parameter computed from neighborhood measurements. It is expected that the greater the number of contexts, the better coder is obtained, on the other hand, lots of contexts means high computational complexity. Taking into account a compromise between these factors a method using 3 main and few auxiliary contexts is proposed. The main contexts depend on neighborhood pixel weighted variance, weights are neighboring pixel distances d j from the coded one:

dj

('x )

 ( 'y j ) 2

2

j





1 2

(4)

The weighted mean is: ~ p

1

G

n

¦d

j

˜ P( j )

(5)

j 1

where: n

¦d

G

j

(6)

j 1

Finally, the weighted variance is calculated form the formula:

V~ 2

1

n

¦d G

j

˜ ( P( j )  ~ p) 2

(7)

j 1

The variance thresholds are 0.15 ˜ V 2 and 1.3 ˜ V 2 ( V 2 is arithmetic mean of weighted variances V~ 2 ). Neighborhood size is set to m = 10, the dashed area in Fig.1. Auxiliary contexts are based on edge detection, namely, gradients dh and dv are computed [1]:

dh

P(1)  P(5)  P(2)  P(3)  P(2)  P(4)

dv

P(1)  P(3)  P(2)  P(6)  P(4)  P(9)

(8)

adaptive methods for error bias cancellation are usually implemented. A bias cancellation method is used in both CALIC and JPEG-LS algorithms. For every context the cumulated error sum is determined and prediction error corrected [1], [2]. Differences between methods are minor, the JPEG-LS version is more suitable for hardware realization, as it has neither multiplications nor divisions. In this paper both methods have been implemented in 4 variants each, then, all 8 components summed up (Cmix value below), the detailed description of the approach can be found in [8]. The pixel estimate is then: x

if (dh > 1.6˜dv) context = 4; else if (dv > 1.5˜dh) context = 5;

If the coded image is larger than 256x256 pixels, similar comparison is done for context 2: if (dh > 1.5˜dv) context = 6; else if (dv > 1.5˜dh) context = 7;

(9)

3. PREDICTOR ADAPTATION ALGORITHMS

The reason of using an adaptive predictor comes from its main virtue – it is a self-optimizing device, which means that no side information about coefficients is needed. Moreover, its ability to follow local image properties may result in superior performance, better than for MMSE optimized predictors. The main drawback might be relatively high computational cost, both on the coder and decoder sides. They are two general types of coefficient update formulas: gradient (e.g. LMS), and least-squares (LS). Gradient formulas are simple, but imply relatively slow convergence, LS predictors are faster, but more computationally complex. Here a combination of both methods has been applied. 3.1. RLS+ method

The recursive least squares (RLS) method of updating prediction coefficients is constructed around updates of matrix K of size rur, r is the predictor rank. It is usually initialized as unity matrix multiplied by a small constant. In our approach denoted RLS+ we are using separate matrices K for each context, each is initialized as follows: 0.00048 ˜ I ˜ D ,

K

When neighborhood variance is high (context 3), difference between horizontal dh and vertical gradient dv is tested:

xˆ  Cmix .

where D

(10)

[d1 , d 2 , ... , d r ]T , and d j are defined in (4).

Starting values for predictor coefficients are the same for all contexts, and equal to B = [0.62, 0.625, –0.125, 0.125, – 0.125, –0.125, 0, ... , 0], B is coefficient vector. The RLS update procedure is as in [9], firstly, vector U is calculated: U

K (n) ˜ X ,

(11)

where signal vector X is X [ P(1), P(2), ... ,P(r )]T . Then, learning coefficient is computed:

2.3. Predictor error bias correction

P

One of problems in context coding is slowly changing predictor error component, the prediction error bias. Then,

1918

1 1.25  XT ˜ U

needed in Kalman vector update formula:

(12)

K (n  1) 1.0005 ˜ (K (n)  P ˜ U ˜ UT ) .

(13)

Finally, vector of predictor coefficients is updated:

B(n)  e ˜ K (n  1) ˜ X ,

B(n  1)

As in section 2.2, neighborhood absolute error values e(i)

(14)

where e is computed as follows, the bound M =14:

e

sgn(e) ˜ min{ e ,M} .

4.2. Contexts for probability distributions

(15)

are used for context calculation. Our approach is based on papers of Deng [10], and [5]. The value Z1 is evaluated: Z1

9 max{2 e(1) , 2 e( 2) , ( e(3)  e( 4) ), e(5)  e(10) , e(6)  e(7) , 8 13 3 7 11 e( 4) , e(3) , ( e(8)  e(9) ), ( e(1)  e(2) )} 2 8 8 8

Then we follow the idea from [5], Z2 is computed, (6):

3.2. Normalized LMS algorithm

Normalized LMS method (NLMS) is an improved version of the LMS algorithm, which coefficient update formula is: b j ( n)  P ˜ e ˜ P ( j ) ,

b j ( n  1)

(16)

where e is defined in (15). In NLMS approach the learning coefficient μ is variable, the following formula is proposed:G

P

(20)G

1 rNLMS § · 2 ˜ ¨¨10  ¦ e 2 ( j ) ¸¸ j 1 © ¹

,

n

¦ d ˜ e( j ) , G j1 j

(21)

n = 28. The greater parameter is chosen:

Z3

max{2 ˜ Z1 , 10 ˜ Z 2 } .

(22)

Finally, Z3 is blended with Z4:

Z

(17)

7

1

Z2

Z3  0.48 ˜ Z 4 ,

(23)

where:

where e(j) is the j-th signal sample, here – the prediction error from the preceding stage of algorithm. We are proposing to implement NLMS in an auxiliary signal modeling stage: firstly, RLS+ algorithm is applied, secondly, NLMS prediction is computed, finally, prediction error bias is removed. The additional computational time due to NLMS for Lennagray image is only 2 sec, which extends the methods computational time to 14.3 sec. 4. ARITHMETIC CODER 4.1. Adaptation of probability distribution

When coding absolute values of numbers probability distributions similar to one-sided Laplace one can be expected. Its initial approximation is as follows:

Z4

max{ P(1)  P(3) , P(2)  P(3) , P(1)  P( 2) , 1.1 ˜ P( 2)  P(4) ,

1.1 ˜ P (2)  P ( 4) , 0.8 ˜ P (1)  P (4) , 0.9 ˜ P (3)  P ( 4) }

(24)

The Z value is quantized using 15 thresholds: Th = {3, 8, 14, 20, 27, 34, 43, 55, 66, 80, 100, 120, 150, 180, 240}. 4.3. Quantization of prediction errors

For increasing histogram collection rate its slots number can be decreased by quantization. Namely, e values are divided into 2 parts, the quantized one and the residue, coded separately. Quantization and coding are done as follows. Find new number for slot k from T(k ) d e  T(k  1) for

(18)

thresholds T = {0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 12, 14, 16, 20, 24, 32, 64, 128}. Result is context coded, section 4.2. Residue eq e  T(k ) is treated as q(k)-bit number, q = {0,

where i is the number of histogram slot, a reasonable choice for A is A = 10. Occurrences of absolute prediction error values are collected in the histogram. A forgetting mechanism is introduced, if the number of coded e

0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 3, 5, 6, 7}. If q(k) > 0, eq is coded by one of 7 universal adaptive arithmetic coders. Decoding consists in reading table T(k), and if q(k) > 0, eq is restored, otherwise eq = 0. Then, e T(k )  eq .

ne (i )

¬A ˜ 0.8 ¼  1 , i

exceeds 2s, histogram values are divided by 2: ne (i )

« ne (i ) » « 2 »  1, for all i from 0 to emax. ¼ ¬

4.4. Prediction error sign coding

(19)

The occurrence counter is set to the sum of histogram slots. Good results are obtained for s t 10, we use s = 13, and s = 10 for sign and quantization error coding.

1919

The prediction error sign is coded using 16-context arithmetic coder. Initially two-slot context histograms are set to value 5. Two first bits of context number are error signs for left and upper neighbors (1 and 2 in Fig.1). The remaining two bits are obtained by quantizing the main context number from (23), thresholds are {8, 20, 180}.

[4] H. Ye, “A study on lossless compression of greyscale images,” PhD th., Dept. Electron. Eng., La Trobe University, Oct. 2002.

5. EXPERIMENTS

Results of experiments are summarized in Table 1. The GLICBAWLS [11] algorithm presents the current pinnacle in realistic lossless algorithms performance, better methods process images two orders of magnitude longer. As can be seen, both new versions of RLS method: RLS+, and its cascade with NLMS outperform GLICBAWLS. The new algorithms has given the best results for predictor rank r = 46, hence, its appearance in the table. Computational complexity of RLS/RLS+ algorithms is O(r2), nevertheless, with a higher constant than for static prediction approach [12] (3-4 times longer execution times). As mentioned before, execution time for RLS+-NLMS combination for Lennagrey image was 14.3 seconds, in contrast to 21.2 seconds for GLICBAWLS. Coding times for other 512x512-point images differ by few seconds, only. Programs were written in ANSI C, have not been optimized, and were tested on Pentium 4 with 2.8 GHz clock. REFERENCES [1] X. Wu, N. D. Memon „CALIC – A Context Based Adaptive Lossless Image Coding Scheme”, IEEE Trans. on Communications, May 1996, vol. 45, pp. 437-444. [2] M. J. Weinberger, G. Seroussi, G. Sapiro „LOCO-I: Lossless Image Compression Algorithm: Principles and Standardization into JPEG-LS”, IEEE Trans. on Image Proces., vol. 9, No. 8, 2000, pp. 1309-1324. [3] B. Meyer, P. Tischer, “TMWLego - An Object Oriented Image Modeling Framework”, Proc. Data Compression Conf. 2001, pp. 504.

[5] I. Matsuda, N. Ozaki, Y. Umezu, S. Itoh, “Lossless Coding Using Variable Blok-Size Adaptive Prediction Optimized for Each Image”, Proc. EUSIPCO-05 (CD), 2005. [6] M. Marcellin, M. Gormish, A. Bilgin, M. Boliek, “An Overview of JPEG-2000,” Proc. Data Compression Conf. 2000, pp. 523-541. [7] K. Sayood, “Introduction to Data Compression,” 2nd edition, Morgan Kaufmann Publ., 2002. [8] G. Ulacha, R. Stasinski, “A Time Effective Lossless Coder Based on Hierarchical Contexts and Adaptive Predictors,” Proc. MELECON’08, 2008, pp. 829-834. [9] H. Bekkouche, M. Barret, “Adaptive multiresolution decomposition: applications to loss-less image compression,” Proc. ICASSP’02, 2002, vol. 4, pp. IV_3533-3536. [10] G. Deng, H. Ye, “Lossless image compression using adaptive predictor combination, symbol mapping and context filtering,” Proc. IEEE ICIP-99, 1999, vol. 4, pp. 63-67. [11] B. Meyer, P. Tischer, “GLICBAWLS - Grey Level Image Compression by Adaptive Weighted Least Squares,” Proc. Data Compression Conf. 2001, pp. 503. [12] G. Ulacha, R. StasiĔski, “A New Simple Context Lossless Image Coding Algorithm Based on Adaptive Context Arithmetic Coder,” Proc. IWSSIP 2008, 2008, pp. 45-48. [13] S. Marusic, G. Deng, “Adaptive prediction for lossless image compression,” Signal Proces.: Image Communications, 2002, vol. 17, pp. 363-372.

Table 1: Bit rate values for lossless coding methods compared in the paper

Images

JPEG-LS [2]

Balloon Barb Barb2 Board Boats Girl Gold Hotel Zelda Mean

2.889 4.690 4.684 3.674 3.930 3.922 4.475 4.378 3.884 4.058

LATRLMS [13] 2.75 4.15 4.45 3.48 3.74 3.68 4.34 4.21 3.61 3.823

ALCM+ [8]

Static [12]

GLICBAWLS [11]

RLS r = 46

RLS+ r = 46

2.735 4.175 4.372 3.418 3.684 3.640 4.296 4.199 3.599 3.791

2.695 4.166 4.383 3.400 3.659 3.596 4.304 4.153 3.587 3.771

2.640 3.916 4.318 3.392 3.628 3.565 4.276 4.177 3.537 3.717

2.722 4.110 4.391 3.428 3.668 3.588 4.274 4.166 3.601 3.772

2.662 4.015 4.307 3.349 3.609 3.542 4.252 4.089 3.560 3.709

1920

RLSNLMS r = 46 2.634 4.002 4.300 3.341 3.602 3.529 4.251 4.080 3.555 3.699