CALIBRATION OF A COMPRESSION ALGORITHM WITH QUALITY ADJUSTED LOCALLY Lea Chanthapanya(1), Xavier Delaunay(1), Roberto Camarero(2), Carole Thiebaut(2), Mathieu Albinet(2), Christophe Latry(2) (1)
THALES Services Parc Technologique du Canal – 3, avenue de l’Europe – Campus 2, Bâtiment C 31400 Toulouse – France Email:
[email protected] (2)
CNES 18, avenue Edouard Belin 31401 Toulouse Cedex 9 – France Email:
[email protected] ABSTRACT Most algorithms designed for on board compression impose a fixed bit-rate. Yet, for Earth observation missions like SPOT5 or PLEIADES, the bit-rate is calibrated so as to obtain the required image quality on the most complex scenes such as dense urban areas. This leads to relatively low compression ratios even on the less complex scenes. In order to optimize the bit-rate on every part of the image, the CNES (Centre Natiaonal d’Etudes Spatiales) is on the final stages of the definition and validation of a compression algorithm with quality adjusted locally. This algorithm derives from the wavelet transform coder of the CCSDS 122 standard [1]. It can adapt the compression rate to local features in the image. This algorithm is being tuned in order to obtain the visual quality levels required by the future Earth observation missions. The compression algorithm takes into account local image features such as the complexity but can also adapt the quality and the compression rate to the level of interest of the region. On-board satellite image processing such as equalization, deconvolution and cloud detection are then very helpful to optimize the compression. Particularly, the compression rate can be highly decreased on regions detected as clouds if they are not valuable for the mission. In order to calibrate the compression algorithm with quality adjusted locally, we have used a method of visual quality comparison driven by local distortion metrics. Three different quality levels have been targeted: the nominal visual quality level corresponding to the CCSDS 122 compression at 3bpp; the higher visual quality level corresponding to the CCSDS 122 compression at 4bpp; the near lossless quality level for which compression noise is smaller than the instrumental noise in the image and for which no compression artefact is visible. Depending on the quality level, the bitrates obtained with the compression algorithm with quality adjusted locally are between 20% and 30% lower than the bit-rates obtained with CCSDS 122 standard. INTRODUCTION On-board CNES PLEIADES satellites, the wavelet transform coder imposes a fixed compression bit-rate. A fixed bit rate algorithm implies that the bit-stream size of each acquisition is entirely predictable, which makes the payload programming task and mass memory management easier. However, this fixed bit-rate is fitted to the worst cases to achieve the required quality on all the scenes, even on the most complex scenes such as dense urban area or rough seas. But it is oversized for the less complex scenes and the compression ratio is not as high as it could be if the bit-rate could vary. Several solutions have previously been considered to loosen the fixed compression bit-rate constraint and to adapt the compression bit-rate locally. The first solution was motivated by the fact that few small areas in the image require more bit-rate than the rest of the image to achieve the same visual quality. This led to the development of an exception processing [2] which increases the bit-rate at these locations using a ROI like compression technique. In the CCSDS 122 standard [1], segments are coded independently of each other. Typically, one segment represents a block of 8 lines in the image. But the segments can be shorter or larger. The minimum size for a segment defined in the
CCSDS 122 standard is 16 blocks of 8x8 wavelets coefficients. This represents a region of 8 lines and 128 columns in the image. Thus, another solution to adapt the compression bit-rate locally was to adjust the bit-rate of the compression segments based on the evaluation of the complexity of the segment These solutions were the premises of a compression algorithm of variable bit-rate with quality adjusted locally. However, concerning those two techniques, the exception processing revealed not flexible enough (it is triggered only on very few areas and drastically increases the bit-rate on these areas) and the compression algorithm with quality adjusted by segment revealed not enough precise to adjust the quality on areas that do not have the shape of the rectangular segments. In parallel to the development of the compression algorithm, work began on the optimization of the full on-board image processing chain. This chain was intended to perform the first image restoration processing which would then completed by the image processing ground segment. These processing steps are the equalization and a first stage of deconvolution. In order to take full advantage of the compression algorithm with quality adjusted locally, a new processing is introduced on-board: the region detection. This last aims at creating regions masks based on which the encoder will adjust the quality and the bit-rate of the compression. In this paper, the processing unit intended for the next CNES mission is presented. It includes a set of algorithms defined by the CNES that are required to offer a high quality enhancement of the images produced as well as a higher compression ratio. A brief overview of the new CNES compression algorithm is also given to understand the challenges of the quality adjusted locally. The compression algorithm can adapt the compression rate to local features in the image in a way more flexible than the exception processing and on more precise areas than the compression algorithm with quality adjusted by segment. The first section of this paper is the current introduction. The second section introduces the on-board image processing chain. The third section provides a brief overview of the compression algorithm with quality adjusted locally. The fourth section discusses the process implemented to calibrate and to assess the performances of this compression algorithm. Last section provides the conclusions and outlooks for the on-board implementation of the processing units. ON-BOARD SATELLITE IMAGE PROCESSING CHAIN The compression with quality adjusted locally is particularly well suited for Earth observations applications in which the different regions of a scene do not have the same degree of interest. A typical example is the case of cloudy satellite acquisitions in the Earth observation images. The study of clouds can be one of the goals of the mission of the satellite, particularly in the case of meteorological satellites such as MTG. In this case, it is essential to keep the maximum of quality on these regions. On the contrary, in Earth observation images acquired by satellites such as PLEIADES which are not dedicated to the study of the atmosphere, clouds are not valuable. Hence, it is possible to decrease the quality of clouds to favour the quality on regions that are more valuable. To reduce the encoding bit-rate on cloud regions, it is first necessary to be able to detect clouds on-board. The on-board cloud detection process is part of a larger on-board image processing chain which is intended to be realized on-board. This on-board image processing chain is presented in Fig. 1. Equalization On the left side of this figure, the multispectral (visible and near-infrared) noted XS and PAN (panchromatic) channels are the outputs of the image sensors. The first processing step is the equalization. It allows correcting the radiometry of the pixels that have been acquired by different sensors. Particularly, the equalization corrects the non linear response of the sensors taking into accounts the dark currents of each sensor cell. Deconvolution The deconvolution filter may be applied on-board to the PAN channel to amplify the high frequencies that have been decreased by the MTF (Modulation Transfer Function) of the optical system. This remains an option since up to now, it is implemented within the ground segment.
Fig. 1: On-board image processing chain Registration Then, in order to perform the cloud detection, the channels have to be registered. Indeed, the staggered arrays focal plane layout means that a given ground pixel is not acquired at the same time by the different spectral bands, the PANXS lag being significantly larger than the lag ranging between two XS bands. Wavelet transform The DWT (Discrete Wavelet Transform) is then performed before cloud detection and encoding. On-board, the cloud detection is based on the DC wavelet coefficients only. DC coefficients are the ones in the low-resolution LL subband. This allows saving computational complexity since the DC coefficients form a low resolution version of the images and since the DWT need to be performed before the encoding stage. Cloud detection Cloud detection is already realized within the PLEIADES ground segment in order to compute a cloud mask and a cloud coverage rate. Cloud information is used as a request filtering parameter by the potential customer, and by the commercial operator to tell if a new acquisition is to be scheduled. Very efficient cloud detections algorithms exist but the difficulty is to adapt one to the on-board complexity constraints. The cloud detection algorithm that has been selected and which is currently being studied for an implementation onboard is based on a linear SVM (Support Vector Machine) classification implemented in the PLEIADES ground segment [4]. It has been designed on-ground and then modified for the on-board cloud detection by CNES. The cloud detection is reduced to a classification of the pixels based on a linear combination of the reflectance of the channels XS0 and XS3 (respectively the blue and near infrared channels). However, at the input of the cloud detection stage, the DC coefficients are raw numerical counts. They are first converted to reflectance values taking into account the absolute calibration factor, the solar elevation and assuming a transparent atmosphere (no radiative transfer code is applied on-board). Then, the SVM weights, learned on-ground on a large and representative set of images, are applied to weight the reflectance of channel XS0 and XS3. The classification is finally performed by comparing the linear combination of the reflectance of the DC coefficients of channels XS0 and XS3 to a threshold learned on-ground. This threshold may be tuned in order to modify the false alarm/good detection SVM performances. The computational
complexity of this algorithm is very low and so compatible with an on-board implementation. It should be noticed that the false alarm probability must be low enough to prevent non cloudy areas to be completely blurred if considered as cloudy. A result of cloud detection with the algorithm described above is provided in Fig. 2. In this figure, the colours indicate different levels of confidence in the detection obtained. They correspond to different distances to the SVM detection threshold. It can be observed that the cloud mask is made of blocks of 8x8 pixels, each corresponding to a DC coefficient in the wavelet transform of the image. This processing of cloud detection could be adapted to other region such as seas, forest, deserts …
Fig. 2. Cloud detection based on the DC coefficients of the wavelet transform Encoding At the output of the cloud detection stage, a cloud mask is obtained and fed at the input of the encoder. Indeed, the compression algorithm with quality adjusted locally has also been designed with the goal of highly reducing the encoding bit-rate on cloud regions. A very efficient way of reducing drastically the compression rate and the quality on such regions is to encode only the DC coefficients in those regions. This is realized by encoding only the quantized version of the DC coefficients (the low-resolution subband) and by skipping the encoding of the high-resolution subbands in the cloud regions. Other regions such as the seas or the urban areas could be encoded with different quality levels and bit-rate depending of the degree of interest of theses regions as a function of the client needs. The compression algorithm with quality adjusted locally is designed to respond to this function. COMPRESSION WITH QUALITY ADJUSTED LOCALLY In order to optimize the bit-rate on every parts of the image, compression algorithm with quality adjusted locally has been included in the CNES on-board image processing chain. This compression algorithm derives from the wavelet transform coder of the CCSDS 122 standard for the on-board image compression [1] with the particularity that it can adapt the compression rate to local features in the image. The compression scheme is represented on Fig. 3. It is similar to the CCSDS 122 compression scheme but the encoder takes as inputs the quality levels, as well as different regions maps which allow adapting the quality to different regions in the image. In the CCSDS 122, the encoding process is performed segment by segment. In the segments, the wavelet coefficients corresponding to the low-frequency subband (LL subband) are denoted by DC. The coefficients corresponding to the high frequencies subbands (HL, LH and HH subbands) are denoted by AC. A quantized version of the DC coefficients is encoded at the beginning of the code stream. Then, the AC coefficients and the refinements bits of the DC coefficients are encoded bit-plane by bit-plane.
Fig. 3. Scheme of the compression with quality adjusted locally
In the compression with quality adjusted locally, the local bit-rate is adjusted by modifying the number of bit-planes that are effectively encoded. This operation mode is already available in the CCSDS 122 standard (BitPlaneStop, StageStop), but the quality levels can only be applied to the entire segment. In this new coder, those parameters can be modified locally which means that the more the local content is complex or important, the more bit-planes are encoded, and hence the higher is the local bit-rate. This allows obtaining an adapted target quality level for every area in the image, even on the most complex. The compression scheme can also take as input those different regions maps mentioned before which allow allocating different quality levels depending on the types of regions in the image. Doing so, it is possible to increase or decrease the mean compression ratio on some particular regions. For example, it is possible to set a minimum quality required on regions of interest (e.g. urban regions) or to drastically reduce the quality on regions of less interest (e.g. clouds or sea regions). In both cases, the number of bit-planes that need to be encoded to achieve this level of quality is computed. The quality is then adjusted by modifying the number of bit-planes encoded However, it is necessary to transmit the number of bit-planes that have been encoded in the different regions to the decoder. This requires a modification of the encoding and decoding algorithm. Hence, the compression algorithm with quality adjusted locally is not fully compliant with the CCSDS 122 standard. CALIBRATION AND PERFORMANCE ASSESSMENT In order to calibrate and assess the performance of the compression algorithm with quality adjusted locally, a method of visual quality comparison driven by local distortion metrics has been used. The local distortion metrics allow focusing on areas on which the quality is not as high as the required quality. However, a full visual analysis of the decompressed image allows detecting compression artefacts that are not detected by the local distortion metrics. Three different quality levels have been targeted: 1. The nominal visual quality level that corresponds to the CCSDS 122 compression at 3bpp. 2. The higher visual quality level that corresponds to the CCSDS 122 compression at 4bpp. 3. The near lossless quality level at which the compression noise is smaller than the instrumental noise in the image and at which no compression artefact is visible. Calibration In order to find the parameters sets associated to each quality level, a learning set of image composed of a few simulated PLEIADES images (bit-depth: 12, resolution: 70cm) and simulated post-PLEIADES images have been prepared by CNES. These last images should be representative of future CNES post-PLEIADES high resolution satellite. The parameters have been tuned based on visual and numerical comparisons of the decompressed images. The number of bit-planes that are encoded by the compression algorithm with quality adjusted locally is tuned depending on the targeted quality. For the nominal visual quality level, it is based on the quality obtained on the urban images compressed with the CCSDS 122 at 3bpp. Indeed, urban images are the images on which the compression is the less efficient.
The approach used to tune the algorithm is the following: The different areas composing the images are first classified using local complexity criteria. For each class, the quality level obtained with the CCSDS 122 compression at 3bpp is computed. An adapted and improved version of this quality level is then targeted in each class by modifying the number of bit-planes that are encoded. The same method is employed to find the compression parameters sets for the PLEIADES and simulated postPLEIADES images, for the nominal image quality (quality equivalent to the CCSDS 122 compression at 3bpp) and for the higher image quality (quality equivalent to the CCSDS 122 compression at 4bpp). For the near-lossless visual quality level, the targeted error is established to be equivalent to the instrumental noise in the image. As the compression algorithm adjusts the bit-rate by modifying the number of bit-planes that are encoded, it introduces some distortion in the image. These distortions are flatten or blurred areas that are mostly visible in the noisy areas as illustrated in Fig. 4. Nevertheless, this is only a “cosmetic problem” since the distortion is lower than the instrumental noise level of the image.
(a) (b) Fig. 4. Sea image. (a) Reference not compressed. (b) Image compressed with the compression algorithm with quality adjusted locally. Other parameters have been introduced to control the minimum and maximum quality levels that should be targeted on the different areas of an image. The maximum quality levels have been tuned based on the visual analysis of the most complex images of the training set, i.e. images of dense urban areas. Fig. 5 compares (a) the reference image that is not compressed to (b), (c) and (d) the images compressed with different values for the maximum quality parameter. Note that in these figures the contrast has been adjusted in order to see differences that are only visible in the darker areas. In Fig. 5 (b), compressed with a low value for the maximum quality parameter, light flatten areas can be distinguished in the darker zones at the edge of brighter zones. In Fig. 5 (c), compressed with the default value for the maximum quality parameter, this flatten area is not visible. In Fig. 5 (d), compressed with a higher value for the maximum quality parameter, the visual quality is not much higher. At least the default value for the maximum quality parameter is thus required to have a good image quality. Nevertheless, the difference of bit-rate is very low when using the default or higher value (about 0.1bpp). This result obtained based on PLEIADES images has also been verified on simulated post-PLEIADES images. The minimum quality level can also be controlled using the quality parameter. This parameter allows keeping the local image quality to a minimum acceptable value. This parameter has been tuned based on the visual analysis of the less complex images of the training set. Fig. 6 compares two values of the minimum quality parameter. Again, the contrast has been adjusted in order to see the differences. The blocks effects are due to the zoom on a small portion of the image. With a low value of the minimum quality parameter, the compressed image has many blurred areas in which the level of the compression noise is approximately equal to the level of the image variance of the area. With the default minimum quality parameter, the compressed image does not exhibit these blurred areas. The default minimum quality parameter is thus considered to be the minimum acceptable visual quality level.
(a)
(b)
(c)
(d)
Fig. 5. Zoom on an extract of a PLEIADES image in a dense urban area. (a) Reference image (not compressed). (b) Image compressed with a low value for the maximum quality parameter. (c) Image compressed with the default value of the maximum quality parameter. (d) Image compressed with a higher value of the maximum quality parameter.
Low value of the minimum quality parameter
Default value of the minimum quality parameter
Fig. 6. Image compressed with different minimum quality parameters levels Performance Assessment At the nominal visual image quality level, the following compression artefacts have been found in the different types of PLEIADES images that have been studied. They are more visible in the simulated post-PLEIADES images but have all been found using contrast adjustment: Dense urban image: Flatten areas have been found in the darkest zones. Field image: No visible artefact found. Desert image: A blurred area has been found in a dark zone. Forest image: Some flatten areas have been found. Industrial image: An artefact has been found in a dark window in the image. Sea image: Flatten areas around objects in the sea and in noisy areas. Rural image: No visible artefact found. Semi-rural: Very few artefacts on the roads in the darker zones.
In the higher image quality level, even less artefacts are visible. They can be found at the same places as in the nominal image quality level i.e. mainly in the shadows in the roads, around the objects in the sea or near strong contrast transition. But they are less perceptible than in the nominal image quality level. In the near lossless image quality level, no artefact is visible. The parameters sets associated to each quality level have then been used for the compression of a test image set composed of several large PLEIADES images. Table 1 reports the mean and median bit-rates (in bpp) obtained the images of this set. Note that no specific region processing has been applied here. The compression algorithm only adapts the bit-rate to the local content complexity in the image. Table 1: Mean and median bit-rates (bpp) obtained on large PLEIADES images. Nominal image quality Higher image quality (equivalent to the quality (equivalent to the quality Near lossless quality at 3bpp with the CCSDS 122) at 4bpp with the CCSDS 122) Mean bit-rate
2.361
2.669
2.838
Median bit-rate
2.299
2.630
2.860
One drawback of this compression algorithm is the variability of the bit-rates which makes the payload programming task more difficult than on PLEIADES for which the maximum bit-rate is limited to 4bpp [3]. The mean results obtained on this set of PLEIADES images allow drawing the following conclusions: 1.
2.
3.
At the nominal visual quality level, the compression algorithm with quality adjusted locally allows saving about 20% of the bit-rate. With a mean bit-rate of 2.36bpp, it offers the same visual image quality level as the CCSDS 122 compression at 3bbp. At the higher visual quality level, the compression algorithm with quality adjusted locally allows saving about 30% of the bit-rate. With a mean bit-rate of 2.67bpp, it offers the same visual image quality level as the CCSDS 122 compression at 4bpp. The compression algorithm with quality adjusted locally offers a near lossless image quality at a mean bit-rate of 2.84bpp.
Depending on the quality level, the bit-rates obtained with the compression algorithm with quality adjusted locally are between 20% and 30% lower than the bit-rates obtained with CCSDS 122 standard. CONCLUSIONS AND OUTLOOKS The compression with quality adjusted locally has already shown to be very efficient for the compression of Earth observation images under visual quality constraints. Depending on the quality level, the bit-rates obtained with the compression algorithm with quality adjusted locally are between 20% and 30% lower than the bit-rates obtained with CCSDS 122 standard. Software modules are currently being implemented to assess the performance of the full on-board image processing chain working with the compression with quality adjusted locally. These modules will be able to produce regions masks such as cloud masks, sea mask or even urban area masks that will then be fed to the encoder in order to adapt the quality level to the degree of interest of the region that has been acquired by the satellite. REFERENCES [1] CCSDS: Image Data Compression Recommended Standard – CCSDS 122.0-B-1 Blue Book (2005). [2] P. Lier, G. Moury, C. Latry and F. Cabot, “Selection of the SPOT5 image compression algorithm”, Proc. SPIE 3439, Earth Observing Systems III, 541, October 1998 [3] C. Thiebaut, C.Latry, R. Camarero, “PLEIADES-HR Commissioning Period: Validation of the on-Board Compression”, OBPDC, Venice, 2014 [4] C. Latry, C. Panem, and P. Dejean. “Cloud detection with SVM technique”, IEEE Geoscience and Remote Sensing Symposium, IGARSS, Barcelona, 2007
