Resourcesat-2 Image Restoration for Differential Pulse Code

Proceedings of the 2013 IEEE Second International Conference on Image Information Processing (ICIIP-2013)

Resourcesat-2 Image Restoration for Differential Pulse Code Modulation Compression Artifacts Anurag Pushpakar, Nitant Dube, Debajyoti Dhar, R. Ramakrishnan Space Application Centre ISRO Ahmedabad, India

The 10 bit data from LISS-3 and LISS-4 is encoded to seven bits on-board using Differential Pulse Code Modulation (DPCM) and is decoded back to 10 bits on ground. Similarly 12 bit data from AWiFS is encoded to 10 bits onboard using Multi-linear Gain (MLG) approach and decoded back to 12 bits on ground.

Abstract— Remote sensing satellites use onboard compression techniques to overcome the limited bandwidth and increasing data volume requirements of images. On-board compression using Differential Pulse Code Modulation (DPCM) is implemented for Resourcesat-2, LISS-3 and LISS-4 sensors. Implemented DPCM is a lossy compression and hence renders artifacts, when images are decompressed on ground. In this paper a technique for restoration of DPCM artifacts is proposed and its performance is evaluated using Mean Square Error (MSE) and Peak Signal to Noise Ratio (PSNR). Proposed technique is used as part of operational data products generation software and sample results are shown. Keywords—DPCM; IRS; LISS-3; Artifact; RAW; RAD; MSE; PSNR

I.

LISS-4;

DPCM is an efficient data compression technique used for reducing transmission rate of digital picture information [5]. In DPCM, difference value between samples and reference pixel is coded using forward look-up-table (LUT) and these coded values are packed onboard along with reference pixels for transmission. Later on at ground station, pixels are decoded back from the coded values and reference pixel using reverse LUT. Fig. 1 depicts the implementation procedure of DPCM in LISS-4 and LISS-3 of RS-2.

Compression;

INTRODUCTION

Improvement in pixel resolution, swath and radiometric resolution of sensors results in increased data volume to be transmitted from satellite to ground stations. Limited bandwidth available for transfer of data forces onboard data handling systems to use data compression. Different onboard data compression techniques have been attempted and implemented on various missions to tackle this issue [1], [2], [3].

Raw Data 10 bit

Onboard Processing

Radiometric Corrected Data( 10 bit)

Selection of onboard image compression technique is significantly different as compared to ground based applications. Onboard compression requires a trade-off between implementation complexity and amount of compression which can be achieved. Onboard encoders are specifically designed using the sensor characteristics and statistical properties of images to be acquired. Some of the finer details in image can be sacrificed for the sake of saving a little more bandwidth or storage space. Approximation of original image is enough for most applications, as long as the error between the original and compressed image is tolerable [4].

Apply LUT

Computation of Pixel Values(10 bit)

Apply Reverse LUT Ground Processing

Fig. 1. DPCM in RS-2

DPCM LUTs are designed to retain information at places, where difference from reference pixel is small (homogeneous regions). However in order to accommodate the information in required number of bits, a trade off is made where more uncertainty is expected in high contrast regions (Cloud and Snow edges). DPCM used in RS-2 is a lossy compression and loss occurs at all pixels, except reference pixel. There is a drastic increase in loss when the coded value reaches ±31 (refer Table III). This error in the DPCM reconstructed image is called as DPCM artifact.

Indian Remote Sensing Satellite, Resourcesat-2 (RS-2) was launched by PSLV-C16 on 20th April, 2011. RS-2 has three sensors namely LISS-3 (Spatial Resolution: 23 meters), LISS-4 (Spatial Resolution: 5.8 meters) and AWiFS (Spatial Resolution: 56 meters). RS-2 is a continuation mission of Resourcesat-1 (RS-1) with improved radiometric resolution. Radiometric resolution has improved from seven bits (RS-1) to 10 bits (RS-2) for LISS-3 and LISS-4 and from 10 bits (RS-1) to 12 bits (RS-2) for AWiFS.

978-1-4673-6101-9/13/$31.00 ©2013 IEEE

Quantize using Forward LUT

Computation of Pixel Difference

Pixels which have high uncertainty cannot be restored and hence the value of these pixels is interpolated using neighboring pixels. Different interpolation approaches have been described in the literature. Among them, an interesting class of filters relies on convolution of image with scaled kernels [6]. Researchers believe that the more accurate the kernel is, clearer the recovered image will be [7].

545

Proceedings of the 2013 IEEE Second International Conference on Image Information Processing (ICIIP-2013)

This paper discusses the approach used to restore RS-2 LISS-3 and LISS-4 images affected by DPCM encoding and decoding errors. DPCM configuration for RS-2 is discussed in section-2, DPCM artifacts are discussed in section-3, image restoration approach is brought out in section-4, results of proposed technique is shown in section-5 and conclusion is provided in section-6. II.

TABLE II. Forward & Reverse DPCM Look up table Difference 0 ±(1-2) ±(3-4) ±(5-6) ±(7-8) ±(9-10) ±(11-12) ±(13-14) ±(15-16) ±(17-18) ±(19-20) ±(21-22) ±(23-24) ±(25-26) ±(27-28) ±(29-30) ±(31-34) ±(35-38) ±(39-42) ±(43-46) ±(47-54) ±(55-62) ±(63-70) ±(71-78) ±(79-94) ±(95-110) ±(111-126) ±(127-142) ±(143-174) ±(175-206) ±(207-238) ±(239-1023)

DPCM CONFIGURATION IN RS-2

LISS-3 and LISS-4 sensors of RS-2 have two ports (odd & even). DPCM algorithm works separately on odd and even pixels of a port. DPCM operation works on four pixels taken alternatively from a block of eight pixels in the across track direction (four from odd and four from even). One pixel is taken as reference and is kept intact while other pixels are replaced with indices corresponding to their difference with the adjacent pixel based on the pre-decided forward DPCM LUT. Table I gives an overview of the DPCM algorithm. In case of odd port, P3 is the reference pixel and determines the value of P1 & P5 while decoding and subsequently P5 is used to determine the value of P7. Similarly for the even port, P6 is the reference pixel and determines the value of P4 & P8 and subsequently P4 is used to determine the value of P2. Whenever P5 and P4 are marked as erroneous then the error is propagated automatically to P7 and P2 respectively. Table II shows the forward (Encoding) and reverse (Decoding) LUT along with uncertainty present for each of the decoded values. From Table II it is observed that during encoding, as the difference value increases uncertainty in decoded value also increases. A wide range of differences (>=±239, last row) are mapped on to the same index. During decoding they are mapped to a value which is closer to the lower limit (±254.5) of the window leading to a high amount of uncertainty in recovered value.

Encoded LUT Value 0 ±1 ±2 ±3 ±4 ±5 ±6 ±7 ±8 ±9 ±10 ±11 ±12 ±13 ±14 ±15 ±16 ±17 ±18 ±19 ±20 ±21 ±22 ±23 ±24 ±25 ±26 ±27 ±28 ±29 ±30 ±31

III.

Decoded LUT Value 0 ±1.5 ±3.5 ±5.5 ±7.5 ±9.5 ±11.5 ±13.5 ±15.5 ±17.5 ±19.5 ±21.5 ±23.5 ±25.5 ±27.5 ±29.5 ±32.5 ±36.5 ±40.5 ±44.5 ±50.5 ±58.5 ±66.5 ±74.5 ±86.5 ±102.5 ±118.5 ±134.5 ±158.5 ±190.5 ±222.5 ±254.5

Uncertainty 0 ±0.5 ±0.5 ±0.5 ±0.5 ±0.5 ±0.5 ±0.5 ±0.5 ±0.5 ±0.5 ±0.5 ±0.5 ±0.5 ±0.5 ±0.5 ±1.5 ±1.5 ±1.5 ±1.5 ±3.5 ±3.5 ±3.5 ±3.5 ±7.5 ±7.5 ±7.5 ±7.5 ±15.5 ±15.5 ±15.5 ±392

DPCM ARTIFACTS

DPCM is implemented in across track direction and hence artifacts are also seen in same direction. They are mainly encountered in high contrast regions. Artifacts can extend in an image from one to six lines depending on the size of object. Artifacts in an image may occur for specific bands based on the difference value. DPCM artifacts are visible in regions like high reflecting roof tops of buildings, snow and clouds. DPCM artifacts as observed in images are shown in Fig. 3.

Using the reverse LUT, uncertainty in the decoded pixels is shown in Table II. From the uncertainty values, it is apparent that uncertainty increases as the coded value increases. As the coded value reaches ±31, there is a drastic increase in uncertainty of counts (±392). This is the main reason for artifacts in DPCM decoded images. TABLE I. DPCM operation for odd pixels Odd Pixels Depth DPCM operation Output Bits after LUT mapping & adding sign bit After MSB repacking

P1 10

P3 10

P5 10

P7 10

P1,3’=P1P3

Ref Pixel

P3,5’=P3P5

P5,7’=P5-P7

6

6

Bit of Ref. pixel & 6 bits=P5’(7 bits)

Bit of Ref. pixel & 6 bits=P7’ (7 bits)

6

Bit of Ref. pixel & 6 bits=P1’ (7 bits)

Reference pixel is sent in original form- 10 bits LSB 7 bits of Ref. pixel=P3’ (7 bits)

In a similar way, DPCM is applied on the even pixels by taking P6 as the reference pixel.

Fig. 3.(a) LISS 4 Band 2 image showing artifacts in cloudy area. (b) LISS 4 Band 2 image showing artifact on multiple roof tops.

546

Proceedings of the 2013 IEEE Second International Conference on Image Information Processing (ICIIP-2013)

IV.

counts. Pixels with uncertainty of ±392 counts are interpolated using neighboring pixel, excluding those having high uncertainty. Values of all the pixels with uncertainty of the order of ±15.5, ±7.5, ±3.5 and ±1.5 counts are restored by adding half of the uncertainty based on neighborhood analysis.

IMAGE RESTORATION TECHNIQUE

Satellite images as received on ground are called as RAW image. Anomalies like detector assembly stagger and non uniformity in detector response (Radiometric distortions) are present. These images undergo radiometric correction and the processed images are called as RAD images, which are free from radiometric distortions [8].

Neighborhood analysis for pixels with low uncertainty (±15.5, ±7.5, ±3.5 and ±1.5 counts) is carried out by estimating mean within a specific neighborhood. If the mean is greater than the decoded pixel value, then half the uncertainty is added otherwise it is subtracted.

Two types of data sets are used for evaluation of image restoration technique i.e. • •

LISS 4 images. Simulated DPCM images

Simulated images were restored using the above discussed three approaches. Restored images were compared with original non-noisy 10 bit images. Mean Square Error (MSE) and Peak Signal to Noise Ratio (PSNR) were used to quantify the performance of restoration approaches. MSE of an estimator is the difference between values implied by an estimator and the true values of the quantity being estimated. It measures the averaged of squares of errors. MSE is found out as in (1)

For quantification of the restoration technique, 12 bits satellite data was used which was then converted to 10 bits to match with the LISS-4 radiometric resolution. RS-2 DPCM coding and decoding was applied on the 10 bits data. Three different approaches for restoring images were attempted. • • •

MSE=1/n ∑(Yi – Y^ )2

(1)

where Y^ is a vector of n predictors and Yi is the vector of true values. PSNR is ratio of maximum possible power of a signal and power of corrupting noise that affects the signal. PSNR can be found out as in (2)

Approach-1: Restoration on RAW images. Approach-2: Restoration on RAD images. Approach-3: Restoration on RAD images using DPCM artifact model generated using RAW Data.

PSNR=10.log10 (MAXi2 / MSE)

(2)

where MAXi is maximum possible pixel value of the image.

In Approach-1 artifact identification and restoration is done on RAW data prior to application of detector normalization LUT. In this approach all artifacts are accurately identified. Restoration uses information from neighboring pixels which have different responses and hence pixels are not properly restored.

Table III quantifies the performance of three restoration approaches mentioned previous section. TABLE III. Results of restoration approaches

Approach-2 identifies and restores artifact affected pixels at RAD level. In this approach as normalization LUT changes the count values (during Radiometric Correction) and hence all artifact pixels are not properly identified. However all identified pixels are properly restored as all neighboring pixels are normalized.

QUANTIFICATION PARAMETERS (FULL SCENE) MSE

APPROACH 1

APPROACH 2

APPROACH 3

377.559

565.966

345.732

PSNR

34.427

32.669

34.810

Among the three restoration approaches attempted, third approach provides the best result with least MSE and highest PSNR. A flowchart of restoration procedure is shown in Fig. 5. Quantitatively Approach-3 provides an improvement of 8.42% and 38.91% in terms of MSE when compared to Approach-1 and Approach-2 respectively.

Both the above discussed approaches have inherent limitations. As approach-1 corrects the artifacts using neighboring pixels, which have different responses and hence restored pixel has some uncertainty. Similarly as Approach-2 is unable to identify artifact affected pixels correctly and hence during restoration it may use neighboring artifact affected pixel and result in improper restored pixel. Therefore, another approach was formulated as a combination of above two approaches. This uses artifact identification of artifact affected pixels as done in approach-1 (at RAW level) and performs restoration as done in approach-2 (at RAD level).

Pixels with high uncertainty (±392) are interpolated using average, weighted average, and adaptive kernels. Table IV displays the results of different averaging kernels used for interpolation.

Approach-3 is a combination of approach-1 and approach2, artifact identification is done at RAW level and all artifacts are corrected at RAD level after application of normalization LUT. Uncertainty increases significantly for pixels coded to ±31 and otherwise is of the order of ±15.5, ±7.5, ±3.5 and ±1.5

547

Proceedings of the 2013 IEEE Second International Conference on Image Information Processing (ICIIP-2013)

S.No.

Original Images

DPCM Simulated Images

DPCM Restored Images

Set-1

Set-2

Fig. 4. Original, DPCM simulated and DPCM restored images. TABLE IV. Results of various averaging kernels Approach Average of adjacent pixels

MSE 376.536

Weighted average adjacent pixels Weighted average of all neighboring pixels Interpolation using nearby pixels except those with uncertainty of ± 392

TABLE V. Quantification parameters

PSNR 34.493

Location wise parameters

419.405 to 589.367

33.971 to 32.493

(a)Original & (b)DPCM applied

474.717 to 689.651

33.433 to 31.811

(b)Original & (c)Restored

345.732

34.810

MSE

PSNR

MSE

Set1

PSNR Set2

28802

15.603

8810

20.747

6564

22.025

786

31.243

V.

RESULTS

Restoration approach was applied on LISS-4 images and visual checks are made to ensure that artifacts are removed. Fig. 6, 7, 8 and 9 show Geo-corrected color composites of LISS-4 original noisy images and images restored using the proposed algorithm.

Results from Table IV shows that interpolation should be performed using neighboring pixels excluding those having an uncertainty of ± 392 counts. Quantification of devised restoration approach was carried out by comparing MSE and PSNR of two datasets as shown in Fig. 4. Results in Table V shows that the original non-noisy images and DPCM simulated images have a MSE of 28802 and 8810, which after restoration was reduced to 6564 and 786 respectively. Performance of technique was also supported by increase in PSNR from 15.603 and 20.747 to 22.025 and 31.243 respectively as shown in Table V. RAW image

Radiometric Correction

RAD image

Modeling for Artifacts

Model for image restoration

Restored RAD image

Restoration using neighborhood analysis and interpolation

Fig. 6. (a) Original LISS-4 image (b) Corrected LISS-4 image

Fig. 5. Descriptive flowchart of Restoration Approach

548

Proceedings of the 2013 IEEE Second International Conference on Image Information Processing (ICIIP-2013)

Performance evaluation using the proposed three approaches shows that image restoration should be carried out on RAD images using modeling information about uncertainty from RAW data. Interpolation of the artifact pixels should use all neighboring pixels except those with a high uncertainty. The proposed restoration technique for Resourcesat-2 LISS-3 and LISS-4 substantially reduces the artifacts in the final processed images. For the two sets of DPCM simulated data, MSE of restored images have improved from 28802 and 8810 to 6564 and 786 respectively. Performance of technique was also supported by increase in PSNR from 15.603 and 20.747 to 22.025 and 31.243 respectively.

Fig. 7. (a) Original LISS-4 image (b) Corrected LISS-4 image

ACKNOWLEDGMENT We gratefully acknowledge the guidance provided by, Sh. A.S. Kiran Kumar, Director (SAC) and Sh. Santanu Chowdhury, Deputy Director, SIPA (SAC). REFERENCES [1]

Fig. 8. (a) Original LISS-4 image (b) Corrected LISS-4 image

[2]

[3]

[4]

[5]

[6]

Fig. 9. (a) Original LISS-4 image (b) Corrected LISS-4 image

VI.

CONCLUSION

[7]

Three approaches for LISS-4 artifacts correction were developed and tested. Approach I identify and correct artifacts at RAW level while Approach II does the same at RAD level. Approach III was identified as combination of both the approaches as it models for artifact identification at RAW level and corrects them at RAD level. Artifacts are correctly identified in approach-1 as they are the RAW decoded counts as received. While restoring, approach-1 uses neighboring pixels with different responses and hence restored pixel value is having uncertainty. Correction is performed at RAD level (free of Radiometric imbalances) in approach-2, hence restoration is proper in approach-2. Approach-3 which is a combination of approaches 1 and 2 results to perform better.

[8]

549

G. Yu, T. Vladimirova and M. Sweeting, "A New Automatic On-Board Multispectral Image Compression System for Leo Earth Observation Satellites," in Digital Signal Processing, 2007 15th International Conference on, 2007. C. Thiebaut, D. Lebedeff, C. Latry and Y. Bobichon, "On-board compression algorithm for satellite multispectral images," in Data Compression Conference, 2006. DCC 2006. Proceedings, 2006. J. Zhang, H. Li and C. W. Chen, "Distributed coding techniques for onboard lossless compression of multispectral images," in Multimedia and Expo, 2009. ICME 2009. IEEE International Conference on, 2009. R. Tyagi, and D. K. Sharma, "Digital Image Compression Comparisons using DPCM and DPCM with LMS Algorithm," in International Journal of Computer Applications & Information Technology on, 2012. vol.1, issue II. J. H. Choi, D. H. Kang, K. D. Lee and Y.-H. Lee, "DPCM with median predictors," in Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, pp. 199-209, 1992. X. Artaechevarria, A. Munoz-Barrutia and C. Ortiz-de-Solorzano, "Restoration of Biomedical Images using Locally Adaptive B-Spline Smoothing," in Image Processing, 2007. ICIP 2007. IEEE International Conference on, 2007. J. Wang, K. Lu, Q. Wang and J. Jia, “Kernel Optimization for Blind Motion Deblurring with Image Edge Prior,” in Mathematical Problems in Engineering, 2012, p. 10. G. Joseph, “Fundamentals of Remote Sensing”, Universities Press,India, 2005.