FCE: A Fast Content Expression for Server-based Computing Qiao Li
Fei Li
Mentor Graphics Corporation 1001 Ridder Park Drive San Jose, CA 95131, U.S.A. Email: qiao
[email protected]
Department of Computer Science Columbia University New York, NY 10027, U.S.A. Email:
[email protected]
Abstract—Server-based computing (SBC) is an approach delivering computational services across the network with advantages of reduced administrative costs and better resource utilization. In SBC, all application processing is done on servers and only screen updates are sent to clients. We introduce a fast content expression (FCE) for screen updates coding. Given a square region of pixel values, FCE constructs a table of unique pixel values in the region and converts each value in the original region into an index into the table. We have implemented our algorithm and compared it with other popular coding methods, including JPEG-LS, JPEG, gif, gzip, VNC hextile, and various combinations. Our results show that our approach provides low coding complexity with reasonable compression.
I. I NTRODUCTION In recent years, there is a growing trend away from the distributed model of desktop computing toward a more centralized model of server-based computing (SBC). In SBC, all application processing is carried out by a set of shared server machines. Clients connect to the servers for all their computing needs. Since SBC servers maintain the full persistent state of user sessions, the only functionality needed on the client is to be able to send keyboard and mouse input to the server and receive graphical display updates from the server. SBC offers the potential of reducing total cost of computational services through reduced system management cost and better utilization of shared hardware resources. The key enabling technology underlying the SBC approach is the remote display protocol, which enables graphical displays to be served across a network to a client device while applications and even window systems are executed at the server side. A number of SBC encoding techniques have been developed, ranging from higher-level graphics primitives to lowerlevel pixel-based compression techniques. However, existing SBC encoding techniques have been shown to not be effective in supporting the display demands of multimedia applications [4]. In this paper, we present fast content expression (FCE) to encode graphical display content. FCE benefits from the property of temporal and spatial similarity that neighboring pixels are correlated and therefore contain redundant information. Given a square region of pixel values, FCE constructs a table of unique pixel values in the region and represents each value in the original region by an index into the table. As any image can be decomposed into square regions, FCEs of those square regions can represent arbitrary screen update. We have implemented FCE and compared it with other popular image coding methods, including JPEG-LS, JPEG, PNG, GIF, and gzip. Our re-
sults show that our approach provides low coding complexity with reasonable compression. This paper is organized as follows. Section II discussed related work in image compression. Section III introduces the FCE expression for representing image content in a square region. Section IV describes how we apply FCE in developing a new image compression algorithm. Section V presents experimental results comparing FCE compression algorithm with other compression methods. Finally, we conclude with Section VI. II. R ELATED W ORK Previous approaches in encoding screen updates can be classified into two categories, graphics-based and pixel-based. Graphics-based approaches employ a variety of higher-level graphics primitives to represent screen updates in terms of fonts, lines, bitmaps, etc. These approaches are used in systems such as X, Windows Terminal Services [2], Citrix MetaFrame [1], and Tarantella [8]. Despite the range of available encoding primitives, the screen updates associated with multimedia applications such as images and video are typically encoded as raw pixel bitmaps. Little compression is achieved on these screen updates for multimedia applications. Pixel-based approaches are simpler and treat a screen update just as a region of pixels. They are used in systems such as Sun Ray [9] and Virtual Network Computing (VNC) [5]. As these approaches employ similar bitmap encoding primitives for multimedia display workloads, they limit the compression achievable for screen updates. More recently, some SBC pixel-based encoding techniques have been developed that take advantage of the common image characteristics of screen updates. These methods achieve improved compression ratios, but at the cost of higher encoding and decoding complexity. These coding costs limit the utility of these techniques for supporting multimedia applications in SBC environments. Based on image properties, different compression algorithms have different advantages. Existing algorithms with better compression ratio have higher complexity, and thus have limited applications. Hence, a simple, and yet higher compression ratio encoding method is necessary. III. FCE: FAST C ONTENT E XPRESSION FCE is a general extension of the hextile compression method in VNC [5] and takes advantages of temporal and spatial correlations for screen updates by introducing a local table of content expressions.
FCE is an expression with varying length to describe screen update content in a square region. Suppose the update square region has s along each edge, and among the s2 pixel values, there are A totally different values (A ≤ s2 ). If we use a queue to hold all the A different values, then log2 (A) bits are enough to express the offset i into the queue for any specific value in the update region. Namely each pixel value and its offset i has an 1 − 1 mapping relation in the square region. Thus FCE can use the offsets for the values in the queue instead of full pixel values. The format of FCE is shown in Fig. 1. Field 1
Field 2
Field 3
Fig. 1. FCE format to represent image content in a square region.
Each FCE consists of 3 fields. 1. Field 1 (number of different pixel values): This field contains the value A, which indicates how many different pixel values are in this square region. It helps traversing field 2 to extract all pixel values and implies the boundary of field 2 and field 3 in FCE. The value of A depends on the spatial redundancy of the square region in the image. 2. Field 2 (queue of different pixel values): This field is the queue holding all the A different pixel values. The length of this field depends on the value in the first field (A) and number of bits used to store one full pixel value in the machine. 3. Field 3 (offset for all pixels in scan-line order): For each pixel, it uses the offset of its value in the queue (field 2) to denote its pixel value. As there are A different values, each pixel will use up to log2 (A) bits for the offset. Field 3 contains all the offsets for all the pixels in the square region, and has a length of s2 × log2 (A). FCE records these offsets in a scan-line order. An example of writing a FCE expression for a 4 × 4 image is shown in Fig. 2. In this figure, the sample image is divided FCE expressions
2
0
1
0
1
1
1
1
0
0
0
If we use m bits to represent the A (log2 (A) ≤ m) different pixel values, n bits to represent each different full pixel value, the length of FCE, L, represented in bits, for this square region with edge size of s will be, L = m + A × n + s2 × log2 (A)
(1)
For ease of extracting the first field in FCE, we make m as a constant that is not less than log2 (s2 ) as A ≤ s2 . As FCE has a header composed of a pixel value queue and A, its length may be longer than using full pixel values for each pixel presentation. Whether FCE representation is shorter depends on the size of the square region s and the number of different pixel values A, which reflects the temporal redundancies for the images. Hence, if we want to make sure that L is shorter than that used by full pixel value presentation, we need 2 × log2 (s) + A × n + s2 × log2 (A) ≤ s2 × n
If we let n = 8 for the inequality, as used in most machines, the figure shown in Fig. 3 will reflect the relation between A and s. Only those values under the curve should be chosen for A to satisfy Equ. 2. And at the same time, if we know the pixel value distribution of the image, we will be able to select square edge size s. That is, if we denote β as the compression ratio, we have, β(A, s) =
s2 × n 2 × log2 (s) + A × n + s2 × log2 (A)
0
1
1
1
0
’relation.dat’
0
140
2
1
1
2
0
1
0
1
0
1
1
0
0
send to gzip for further compression
0
1
1
0
0
compressed file
number of different pixel values
120
Compressed file:
100
80
60
40
Fig. 2. Sample of using FCE for image compression. 20
into 4 squares whose edge is of 2-pixel long. There are 4 FCE expressions corresponding to these 4 squares. Each FCE uses its own pixel value queue to represent the pixel values in the 2 × 2 square.
(3)
From Equ. 3, the compression ratio β depends on the square size s and A. The optimal size s for the squares is where the highest compression ratio is attained. Suppose the density (the ratio of the number of different pixel values and the total number of pixels) of an image is 0.1, which is typical for a smoothtoned image, this relation between ratio gain and s is shown in Fig. 4. We can only choose those values under the curve for compression ratio β and s. In order to satisfy Equ. 2 and gain as much compression ratio β as possible, we need to find the crossing point for these two curves in Fig. 3 and Fig. 4. We can see that 16 is one of the best choices for image in FCE format with shortest length. That means Equ. 2 and Equ. 3 sets up the theoretical basis for us to choose the square edge size. Relation between s and A
2
(2)
0 0
50
100 square edge size s
150
200
Fig. 3. Relation between square edge and number of different pixel value
Relation between s and ratio ’relation2.dat’ 10
10 times of ratio
8
6
4
2
0 0
50
100 square edge size s
150
200
Fig. 4. Relation between square edge and ratio
IV. C OMPRESSION A LGORITHM From the discussions on FCE format above, we know that, if the a region can be approximated with fewer different pixel values, we can use the FCE expression to compress it. The basis for this approach is the assumption that neighboring pixel values have much similarity. A. Compression Algorithm From discussion from Section III, we choose 16 for s. Assume the give image has 3 planes, called R, G, and B. The outline of the compression algorithm is as follows, 1. For each RGB plane, do the following: 1.1 Decompose image on the plane into squares of size 16 × 16; 1.2 Allocate a matrix Mr,g,b to store offsets for all pixel values; 1.3 For each square i, allocate a queue Qi to store different pixel values in this square. 2. For each square i in each plane, do the following, 2.1 Put all different pixel values in square i into queue Qi ; 2.2 Record offsets for all pixel values in square i, into Mr,g,b . 3. Record Mr,g,b and all Qs for each plane. 4. Use gzip to further compress M and Q. An pseudocode of the algorithm that describe this procedure is given below. Let the number of pixels be S, we allocate matrix R, G, B of size S to contain pixel R, G, and B values, we also allocate offset matrix Mr , Mg , and Mb of size S to hold pixel value offsets in the corresponding square queues; For each square, we allocate a pixel value queue Q. And, for each queue Q, a corresponding array A, which contains number of different pixel values, is initialized to all 0’s. for each R, G, and B planes do for the ith and jth square do if RGB value in this square in Qi,j then record offset in M ; else insert value in Qi,j ; Ai,j + +; record offset in M ; end if end for
for each square in each planes do write Ai,j into compressed file; write Qi,j into compressed file; write M into compressed file; end for Using gzip for each compressed file in each plane. end for An example showing how the algorithm works on previous sample is shown in Fig. 2. It shows the case for one plane. We put A value for each square first in the compressed file, then, we put all different pixel values into the file, and the offset for each pixel value follows. There are three issues for us to notice in our algorithm, which effectively make the file containing FCE expressions favor gzip compression. 1. In order to make the file containing FCE expressions better for gzip compression, we put FCE expressions for each plane one by one. And for each plane, instead of putting FCE for each square one by one, we put the pixel values for all the squares together, then all the offsets. This approach under gzip attains better compression. It depends on the property that neighboring square may have similar number of total different pixel values and offsets. 2. We output offsets as bits instead of as bytes in order to save space. When neighboring pixels are similar to each other, FCE encoding gains a lot of compression because each pixel just needs log2 (A) bits. If A is 1, the whole square region just needs one unit of pixel value in FCE expression. 3. For ease of decoding pixel values, we fix the size of A to max(Ai ) for all the squares. In our experiments, we employ log2 (s2 ) bits, which is s bytes, for A. From the discussion above, we know that the advantages of using FCE is based on the assumption that the density for different pixel values in the image is not large. Given one image with its content of RGB values, the question is whether it is possible to convert to a representation with even less density. If the conversion is good enough, we can achieve a near-lossless solution with even better compression ratio. B. Color Space Plane Conversion The formula is based on Julien’s work [6]. The YUV planes give more similarity among the values for each square. That is, Y = 0.299 × R + 0.587 × G + 0.114 × B
(4)
U = (B − Y ) × 0.565
(5)
V = (R − Y ) × 0.713
(6)
with its reciprocal versions: R = Y + 1.403 × V
(7)
G = Y − 0.344 × U − 0.714 × V
(8)
B = Y + 1.770 × U
(9)
Consider the color space plane conversions, we have some experimental data to show that conversion from RGB planes to
A (R plane) var (R plane) A (G plane) var (G plane) A (B plane) var (B plane) A (Y plane) var (Y plane) A (U plane) var (U plane) A (V plane) var (V plane)
airplane 49 34.23 50 36.95 38 27.07 49 34.99 18 13.61 13 9.36
baboon 84 29.42 93 20.29 94 29.01 86 29.39 43 12.05 40 13.73
fruits 41 25.19 46 25.38 55 24.11 45 24.49 21 10.55 15 9.81
lena 48 29.94 56 29.36 51 24.61 52 29.86 22 9.66 22 9.44
peppers 53 25.71 57 31.93 54 28.88 54 28.88 28 11.34 32 28.88
TABLE I C OLOR PLANE CONVERSION STATISTICS
Y U V planes can make neighboring pixel values more similar to each other in the following table. The table shows how many different pixel values A for each plane and what the variance var is for each square of 16 × 16. From the table, we can see that Y plane has similar A value and variance, compared with R plane. However, U and V planes have very smaller A and variances, compared with G and B planes for the same images. This assures us that this conversion is good for FCE expressions in lossy compression. Also, for the conversion from RGB to Y U V in the formula, encoding and decoding result in a loss, however the error is bounded to be less than 5 for all 256 possible values each color can assume. As this error hardly has any visual effect, the conversion is near-lossless, and the converted data is more amenable to compression. V. E XPERIMENTAL R ESULTS To evaluate the performance of using FCE, we compared its compression performance and coding complexity against several other popular compression methods for toned-images. For desktop-like screen updates, FCE gains much over compression algorithms favoring toned-images. We only consider multimedia application coding only. These methods include JPEG-LS, lossless JPEG using Huffman coding, PNG, gzip, lossy JPEG from IJG with quality equal to 100, and GIF, which is lossy due to the conversion from 24-bit to 8-bit color. We show results for using FCE in both RGB and YUV color space. We used 5 different images for our measurements, which are from a standard collection of test images [7]. The measurements were performed on an IBM NetVista PC with a 1 GHz AMD Athlon CPU and 256 MB RAM, running RedHat Linux 7.1. Table II shows the compression results in terms of total image size after compression for each compression algorithm on each of the 5 images. Table III shows the corresponding compression ratio for each algorithm. Tables IV and V show the coding complexity results in terms of encoding and decoding time, respectively for each compression algorithm on each of the 5 images.
Tables II and III show that FCE consistently achieves much faster encoding and modest decoding with similar compression ratio. Compared with OLI [3], FCE gains better compression ratio and faster coding for multimedia applications. We also measured the performance of FCE-RGB and FCE-YUV with other tile sizes. Our measurements confirmed our earlier analysis that showed a 16×16 size square region for FCE expressions provided the best FCE compression performance. VI. C ONCLUSIONS A ND F UTURE W ORK In this paper, we first introduce a fast content expression to describe screen updates content. Then, we apply it to improve lossless and near-lossless compression for multimedia applications. The new lossless image compression algorithm integrates well with the popular gzip compression utility. We have implemented our algorithm and compared it with other popular coding methods, including JPEG-LS, JPEG, gif, gzip, and various combinations. Our results show that our approach provides superior coding complexity with modest compression, which gains its suitability in server-based computing. R EFERENCES [1] Citrix Systems, Citrix MetaFrame 1.8 Backgrounder Citrix While Paper, June 1998. [2] B. C. Cumberland and G. Carius, Microsoft Windows NT Server 4.0, Terminal Server Edition: Technical Reference Microsoft Press, Redmond, WA, August 1999. [3] F. Li and J. Nieh, Optimal Linear Interpolation for Server-based Computing, in Proceedings of IEEE International Conference on Communications, New York, NY, USA, April 2002, pp. 2542-2546. [4] J. Nieh and S. J. Yang, Measuring the Multimedia Performance of ServerBased Computing Proceedings of the Tenth International Workshop on Network and Operating System Support for Digital Audio and Video, Chapel Hill, NC, June 2000. [5] T. Richardson, Q. Stafford-Fraser, K. R. Wood and A. Hopper, Virtual Network Computing IEEE Internet Computing, Vol. 2, No. 1, January/February 1998. [6] YUV conversion, http://www.webartz.com/fourcc/fccyvrgb.htm [7] Standard test images, http://www.geocities.com/SiliconValley/Lakes/6686/test-images/ [8] The Santa Cruz Operation, Tarantella Web-Enabling Software: The Adaptive Internet Protocol A SCO Technical While Paper, December 1998. [9] Sun Ray 1 Enterprise Appliance, Sun Microsystems http://www.sun.com/products/sunray1
no compression JPEG-LS JPEG Huffman PNG gzip JPEG (IJG quality = 100) GIF FCE-RGB FCE-YUV
airplane 786,432 387,964 450,303 475,634 577,975 213,803 298,066 475,122 391,579
baboon 786,432 606,701 540,577 648,605 786,492 342,881 298,066 727,870 582,061
fruits 786,432 406,508 509,636 491,670 572,525 224,464 298,066 487,718 401,440
lena 786,432 446,342 560,269 525,763 733,192 229,952 298,066 521,897 452,040
peppers 786,432 467,448 550,798 556,653 731,298 248,633 298,066 535,592 496946
peppers 1.68 1.42 1.41 1.07 3.15 2.63 1.47 1.58
AVERAGE 1.89 1.52 1.63 1.38 3.65 3.17 1.49 1.70
TABLE II I MAGE SIZE IN BYTES AFTER COMPRESSION
JPEG-LS JPEG Huffman PNG gzip JPEG (IJG quality = 100) GIF FCE-RGB FCE-YUV
airplane 2.03 1.74 1.65 1.35 3.67 2.63 1.48 1.79
baboon 1.29 1.45 1.21 1.00 2.29 2.63 1.55 1.67
fruits 1.93 1.54 1.59 1.37 3.50 2.63 1.50 1.83
lena 1.76 1.40 1.49 1.07 3.41 2.63 1.47 1.69
TABLE III C OMPRESSION RATIOS
JPEG-LS JPEG Huffman PNG gzip JPEG (IJG quality = 100) GIF FCE-RGB FCE-YUV
airplane 120 320 160 20 270 210 45 98
baboon 210 290 150 25 310 154 46 100
fruits 260 320 190 32 280 189 46 100
lena 190 320 220 24 350 190 45 100
peppers 210 360 210 28 300 190 45 99
lena 160 360 155 50 200 169 156 158
peppers 150 410 167 42 190 150 110 323
TABLE IV E NCODING TIME COMPLEXITY IN MILLISECONDS
JPEG-LS JPEG Huffman PNG gunzip JPEG (IJG quality = 100) GIF FCE-RGB FCE-YUV
airplane 110 390 110 30 180 104 107 157
baboon 140 370 130 40 190 115 324 365
fruits 180 330 160 36 210 130 109 158
TABLE V D ECODING TIME COMPLEXITY IN MILLISECONDS .
