Nonlinear Elastic Model for Flexible Prediction of Remotely Sensed

IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, VOL. 11, NO. 5, MAY 2014

1005

Nonlinear Elastic Model for Flexible Prediction of Remotely Sensed Multitemporal Images M. Mamun, X. Jia, Senior Member, IEEE, and M. J. Ryan, Senior Member, IEEE

Abstract—While an increasing number of satellite images are collected over a regular period in order to provide regular spatiotemporal information on land-use and land-cover changes, there are very few compression schemes in remotely sensed imagery that use historical data as a reference. Just as individual images can be compressed for separate transmission by taking into account their inherent spatial and spectral redundancies, the temporal redundancy between images of the same scene can also be exploited for sequential transmission. In this letter, we propose a nonlinear elastic method based on the general relationship to predict adaptively the current image from a previous reference image without any loss of information. The main feature of the developed method is to find the best prediction for each pixel brightness value individually using its own conditional probabilities to the previous image, instead of applying a single linear or nonlinear model. A codebook is generated to record the nonlinear point-to-point relationship. This temporal lossless compression is incorporated with spatial- and spectral-domain predictions, and the performances are compared with those of the JPEG2000 standard. The experimental results show an improved performance by more than 5%. Index Terms—Multispectral imagery, mutual information (MI), nonlinear model, temporal compression.

I. I NTRODUCTION

T

HE transmission of remotely sensed images across communication channels is challenging due to recent advances in satellite technologies that generate terabytes of data every day. The use of compression techniques is therefore very critical in reducing the volumes of data to be transmitted. Current data reduction methods are achieved mainly via the compression of the individual data set in the spatial and spectral domains. Spatial predictor defined on a predefined neighborhood can be used to predict every pixel, and the difference between the predicted and the actual values is encoded [1]. The predictive coding is followed by entropy coding of the residual image. Spatial prediction of the residual image further improves the compression gain [2]. In spectral prediction, the

Manuscript received December 9, 2012; revised April 2, 2013 and August 17, 2013; accepted September 12, 2013. Date of publication October 28, 2013; date of current version December 11, 2013. M. Mamun is with the Department of Computer Science and Engineering, Rajshahi University of Engineering and Technology, Rajshahi 6204, Bangladesh (e-mail: [email protected]). X. Jia and M. J. Ryan are with the School of Information Technology and Electrical Engineering, The University of New South Wales at the Australian Defence Force Academy, Canberra, A.C.T. 2600, Australia (e-mail: X.Jia@ adfa.edu.au; [email protected]). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/LGRS.2013.2284358

prediction is achieved by considering the colocalized pixels in the neighboring bands. One of the most promising nonlinear predictors, CALIC, is applied to the 3-D spectral volumetric data in which the bands are decorrelated by examining their spectral and spatial gradients, where the prediction is performed using two corresponding pixels in the previous two bands [3]. As remotely sensed images are now commonly gathered in regular short intervals, it presents an opportunity to use historical images for predicting the current image. In this letter, we introduce temporal domain compression by exploiting the general dependence between successive images. Rather than transmitting a complete image to end users every time, only the differences between the current and previous image data, after prediction models are applied, are transmitted. The temporal correlation between frames has been widely exploited in video compression between the frames [4], but there are very few studies in multitemporal remote sensing image compression. In [5], the reference image used for temporal prediction is assumed to be linearly related to the current image of the same scene unless there are significant land-cover changes. In that letter, lossy temporal domain compression is performed on change-removed data (changed data are transmitted separately without compression). Lossy compression is preferable for applications in which a certain amount of distortion is not an issue. The disadvantage of this approach is that the users cannot retrieve the original data, so it is not suitable for sequential compression where the next image is predicted from the current one. We propose a lossless temporal compression method in this study. The selection of a prediction model is a critical step in data compression. A fuzzy logic rule was introduced to create a set of predictors [6]. The concept of clustered differential pulse code modulation, in which the spectra of the images are clustered and an optimal predictor is computed for each cluster to decorrelate the image, was introduced by Mielikainen and Toivanen [7]. The optimal predictor is based upon minimizing the sum of squared errors (SSE) within each cluster. Vector quantization is a similar process [8]. The geometrically registered sequential images will be highly correlated when the imaged area contains few landcover changes, with most of the differences being primarily due to variations in the sensing process and atmosphere. In that case, the linear prediction is effective and reasonable. However, predictions are likely to be seriously in error if a linear model or a fixed nonlinear model is fitted to data which contain significant land-cover changes. In this letter, an elastic (discrete) nonlinear prediction strategy is proposed for coping with general relationship between two-date data sets.

1545-598X © 2013 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

1006

IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, VOL. 11, NO. 5, MAY 2014

II. M ETHOD A. Polynomial Prediction Model Given two image data sets, Z = {X = xi , Y = yi |i = 1, . . . , n}, where X and Y are taken at time t and t + 1, respectively, and n is the total number of observations, regression analysis can solve the following minimization problem: ψ opt = arg min ψ∈λ

n 

d (ψ(xi ), yi )

(1)

i=1

where ψ(·) is a regression function and d(ψ(xi ), yi ) = (ψ(xi ) − yi )2 is a residual or error function. λ is a set of candidate regression functions, for example, a general polynomial function defined as λ = α0 + α1 x 1 + α2 x 2 + · · · · · · · · · · · · · · · + αp x p

(3)

α is associated with the correlation coefficient between the two data sets, which measures the strength of the linear relationship. The correlation coefficient between two image data sets (X and Y ) is defined as n  (xi − x ¯)(yi − y¯) (4) rXY =  i=1 n n   2 2 (xi − x ¯) (yi − y¯) i=1

i=1

where x and y are the sample means of X and Y , respectively. If there are significant real changes in the imaged area in the new data set, higher ordered polynomials may be selected. However, a fixed nonlinear model has a limited capacity to accommodate real irregular relationship. An elastic model is introduced in the next section. B. Elastic Nonlinear Model An elastic nonlinear model is proposed based on the concepts of mutual information (MI) and conditional probability. While correlation measures linear dependence between two images, MI measures general dependence, including both linear and nonlinear. It is defined as  p(y, x) . (5) MI(X : Y ) = MI(Y : X) = p(x, y) log2 p(y)p(x) y∈Y x∈X

Different temporal relationships between two artificial images. TABLE I D IFFERENT C ASES OF T EMPORAL R ELATIONSHIP

(2)

where p is the degree of the polynomial. In the case of successive temporal images, it is not realistic to expect that the nonchanged areas have identical brightness values, even though the images may have been acquired only short period apart, as certain differences are often induced due to different environmental and imaging conditions. While these factors are complicated, a linear model is generally acceptable for describing the temporal relationship. Therefore, a first-order polynomial is adopted as the prediction function for images which contain either no or only a small portion of real changes. Taking α0 = β and α1 = α for the first-order polynomial, the candidate regression function becomes ψ(xi ) = αxi + β.

Fig. 1.

MI(X : Y ) is bounded by H(X), the entropy of X, if H(X) < H(Y ) [9]. Normalized MI makes the output in the range of [0, 1] by finding the ratio to the (possible) maximum MI [10] MIn (X : Y ) = 

MI(X : Y ) = . (6) MI(X : X)MI(Y : Y ) H(X)H(Y ) MI(X : Y )

A normalized MI value of close to one indicates high general dependence, i.e., there is a strong point-to-point relationship, while a value close to zero indicates low dependence between two images. Fig. 1(a) and (b) gives two small examples. They are the scatter plots of two artificial images, from which we can examine the brightness relationship between them. Table I lists the entropy values, correlation coefficients, and MI results. We can see that, in case (a), the two images are identical. The correlation coefficient and the normalized MI are both ones. On the other hand, the second example illustrates a relationship that does not form a line (low correlation), but the point-to-point dependence is strong (perfect MI). Real changed pixels may exhibit this kind of a sudden discontinuity in the relationship between images. This is very difficult to predict from the previous sequence of images using a linear model or a fixed nonlinear function. In such cases, we propose an elastic nonlinear model by considering their mutual relationship. Unfortunately, the relationship is not often one to one between the real multitemporal remote sensing data sets, i.e., their normalized MI is not 100%. The proposed solution is to estimate the one-to-one relationship using the mean of the conditional density, as detailed in the following. Considering a given value in the previous image X, xi ∈ {X}, the corresponding values in the current image Y are listed (i) as yj ∈ {Y }, j = 1, . . . , m. m is the number of gray-level values on the current image which are associated with a single value on the previous image. For N -bit multispectral images, the complete gray-level set is χ ∈ {0, 1, 2, . . . , 2N − 1}. xi and

MAMUN et al.: ELASTIC MODEL FOR FLEXIBLE PREDICTION OF REMOTELY SENSED MULTITEMPORAL IMAGES

1007

Fig. 4. Two-dimensional conditional distribution of the pixel values of the current image for the given pixel value of 80.

C. Compression and Transmission Using the Nonlinear Elastic Model Fig. 2. Images captured in years (a) 2000 and (b) 2001 used for experiments (band 3).

Fig. 3.

Step 1) A 3-D histogram (similar to Fig. 3) is calculated. Step 2) The mean of the conditional probability for each given reference gray-level value is calculated from the 3-D histogram table and used as the predicted value for the current image. Step 3) The predicted values from all previous image graylevel values are tabulated to form a codebook. Step 4) A predicted image is generated from the previous image and the codebook.

Three-dimensional histogram for images in Fig. 2.

(i) yj

both belong to χ. The predicted value for the current image is calculated as y¯i =

m 

(i)

wj yj

(7)

j=1 (i)

where wj is the conditional density of yj ; then, for each value of xi in the previous image wj =

p(yj ∩ xi ) . p(xi )

The sequential image transmission is based on the assumption that the previous image is held at both the transmitter and receiver ends. When new image data of the same area arrive, they are predicted and transmitted by following the proposed steps listed hereinafter.

(8) (i)

Equation (7) gives the mean of the conditional density of yj given xi . Figs. 2–4 show an example of how to estimate the mutual relationship. The two images shown in Fig. 2 contain some real changes. The 3-D histogram of that image is given in Fig. 3. We can see that, for each gray-level value of the previous image, it associates with a group of gray-level values with various frequencies in the current image. The value of 80 is chosen as an example, and its associated data are plotted in Fig. 4, i.e., conditional distribution of yi for xi = 80. While this is not a one-to-one relationship, the approximate corresponding value is worked out using (7), and the result is 45. Based upon a conditional distribution analysis on every gray level presented in the previous image, a complete predictor set is formed, which serves as a discrete nonlinear elastic model for temporal data predication.

∀xi ∈ χ;

if(X == xi ) ⇒ (Y == y i )

Step 5) The residual image R is obtained by finding the difference between the new data and the predicted data R = Y − Y . This residual image can then be transmitted directly or after further compression. The codebook will be transmitted too so that the data can be recovered at the receiver end. The basic way to transmit the codebook is to transmit all data pairs (xi , y i ) i = 1, . . . , m. We propose a better way by transmitting only y i , i = 1, . . . , 2N . xi will not be transmitted, but all possible values are considered in ascending order. If there are any gray-level values that are not used, the predicted values are listed as “ Y ” in the codebook and transmitted arbitrarily at the lowest cost, for example, as the most frequent number in the codebook to minimize the entropy of the codebook. The transmission load for the codebook is insignificant for a large image. For example, for 8- and 12-b images of size 512 by 512 pixels, the total entropy Et values, including the residual data and the overhead codebook (worse case), will be For 8-b image, For 12-b image,

Ep × 512 × 512 + 2048 1 = Ep + 512 × 512 128 Ep × 512 × 512 + 49152 3 = Ep + Et = 512 × 512 16

Et =

1008

IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, VOL. 11, NO. 5, MAY 2014

TABLE II C OMPARISON OF SSE AND E NTROPY FOR S INGLE L INEAR M ODEL AND N ONLINEAR M ODEL W ITH A RTIFICIAL DATA S ET

Fig. 5. Two-dimensional conditional distribution of the pixel values of the current image for the given pixel value of 80.

Fig. 6. (a) Three-dimensional histogram table and (b) the corresponding conditional probability table for the artificial images.

where Ep is the entropy of the residual data. The total entropy is increased by a small factor and is negligible if images of very large sizes are transmitted. If there are many arbitrary codebook values ( Y ) for the unused gray-level values or the same predictor is repeated several times, then the codebook entropy becomes lower than this worse case. This procedure of lossless coding using the past reference images enables the compression opportunity in temporal domain of satellite images. Further using of the existing spatial and spectral domains will improve the overall gain [11], [12].

Fig. 7. Comparison of nonlinear and single linear predictions for images in Fig. 2.

III. E XPERIMENTS A. Experiments With Artificial Temporal Images To illustrate the proposed method, two small artificial images I1 and I2 were generated as the previous and current images, respectively (see Fig. 5). Suppose that each image has a gray-level range of 0–9; then, the 3-D histogram and the conditional probability table derived are shown in Fig. 6(a) and (b), respectively. All predictors were calculated, and the resulting codebook is T = {Y , 7, 4, 5, 7, 6, Y , Y , Y }. The unused gray-level values in the previous image ( Y ) can be transmitted arbitrarily. The comparisons of SSE and entropy of the residuals resulting from the application of nonlinear prediction and single linear prediction to these artificial images are given in Table II. It can be seen that the nonlinear model performs better than the single linear model. The reason can be explained from the following statistics. The entropies of the previous and current images are 2.1972 and 2.4194, respectively. The normalized MI value is 0.8202.

Fig. 8. Comparison of residual entropy resulting from nonlinear prediction and actual entropies of current and difference images for images in Fig. 2.

The correlation coefficient is 0.1228. The linear prediction is unsuccessful due to the low correlation, and the high MI case is predicted better using the proposed nonlinear method. B. Experiments With the Landsat ETM+ Images The images given in Fig. 2 are two subsets of Landsat Enhanced Thematic Mapper Plus (ETM+) data recorded over Canberra, Australia, in 2000 and 2001, respectively. A comparison of the residual entropies from the single linear and nonlinear models is given in Fig. 7. In Fig. 8, the residual entropy resulting from nonlinear prediction is compared with the actual entropies of the current and difference images, with the latter sometimes being higher than the former due to the large deviation in the dynamic range between the previous reference and current images. In all cases, the proposed method performs better and results in lower image entropies than the original image entropy.

MAMUN et al.: ELASTIC MODEL FOR FLEXIBLE PREDICTION OF REMOTELY SENSED MULTITEMPORAL IMAGES

TABLE III C OMPARISON OF E NTROPIES FOR THE C OMPRESSED DATA S TREAM

IV. C OMPARISON W ITH JPEG2000 Temporal compression can be combined with existing spatial- and spectral-domain compression algorithms to compete with modern state-of-the-art lossless approaches. Dual-stage decorrelation is needed in which the nonlinear decorrelation strategy developed is followed by residual decorrelation [2], [13]. Spatial coding of the residual image is expected to reduce further the number of bits required to be transmitted after the first-stage temporal prediction. The residuals derived from the temporal prediction are passed to the second stage of decorrelation that considerably increases performance by significantly reducing the residual entropy. Thus, the dual decorrelation approach successfully achieves the complete decorrelation to improve the prediction accuracy and indicates considerable savings in terms of computational cost as well. Thus, the nonlinear adaptive model is updated to compare with the state-of-the-art lossless technique JPEG2000. The comparison with entropy of the JPEG-encoded difference image is shown in Table III. V. C ONCLUSION The proposed elastic nonlinear prediction method is a novel adaptive approach, based on conditional distribution, which enhances local prediction in the radiometric domain. It can accommodate better the individual brightness relationship between two date image data sets due to the changes of ground cover types during the imaging interval. An approximate oneto-one relationship is best estimated and used to generate an effective codebook. The compression is lossless, and it is suitable for data transmission between the ground stations. A good improvement has been observed using the proposed method compared with that of the single linear approach if applied to the images with high MI but low correlation. The compression ratio will be higher for larger images when the overhead becomes negligible. In the study, images of 8 b/pixel have been used. For images of 10–12 b/pixel, the conditional probability becomes highly scattered and is low in each bin, which increases the computational load and reduces the reliability in the estimate of probability density. The solution is to reduce the number of bits by

1009

quantization, as adopted in other similar studies [15]. In terms of compression ratio, the proposed nonlinear method is relatively insensitive to the number of bits per pixel. While the residual image’s entropy is higher with the greater number of bits per pixel, the original data’s entropy is high as well. The main difference is the increased overhead for high-resolution images, which becomes insignificant when the size of the transmitted image is large. The temporal compression can be integrated with spatialand spectral-domain compressions. Overall, multispectral data transmission will benefit from this 4-D compression scheme. R EFERENCES [1] Z. Wang, H. Xu, Y. Tian, J. Tian, and J. Liu, “Integer Haar wavelet for remote sensing image compression,” in Proc. 6th Int. Conf. Signal Process., 2002, vol. 1, pp. 715–718. [2] C. Lin and Y. Hwang, “An efficient lossless compression scheme for hyperspectral images using two-stage prediction,” IEEE Geosci. Remote Sens. Lett., vol. 7, no. 3, pp. 558–562, Jul. 2010. [3] E. Magli, G. Olmo, and E. Quacchio, “Optimized onboard lossless and near-lossless compression of hyperspectral data using CALIC,” IEEE Geosci. Remote Sens. Lett., vol. 1, no. 1, pp. 21–25, Jan. 2004. [4] W. Zhu, X. Tian, F. Zhou, and Y. Chen, “Fast disparity estimation using spatio-temporal correlation of disparity field for multiview video coding,” IEEE Trans. Consum. Electron., vol. 56, no. 2, pp. 957–964, May 2010. [5] Z. Wei, D. Qian, and J. E. Fowler, “Multitemporal hyperspectral image compression,” IEEE Geosci. Remote Sens. Lett., vol. 8, no. 3, pp. 416– 420, May 2011. [6] B. Aiazzi, P. Alba, L. Alparone, and S. Baronti, “Lossless compression of multi/hyper-spectral imagery based on a 3-D fuzzy prediction,” IEEE Trans. Geosci. Remote Sens., vol. 37, no. 5, pp. 2287–2294, Sep. 1999. [7] J. Mielikainen and P. Toivanen, “Lossless compression of hyperspectral images using a quantized index to lookup tables,” IEEE Geosci. Remote Sens. Lett., vol. 5, no. 3, pp. 474–478, Jul. 2008. [8] M. J. Ryan and J. F. Arnold, “The lossless compression of AVIRIS images by vector quantization,” IEEE Trans. Geosci. Remote Sens., vol. 35, no. 3, pp. 546–550, May 1997. [9] T. M. Cover and J. A. Thomas, Elements of Information Theory, 2nd ed. Hoboken, NJ, USA: Wiley, 2006. [10] M. Fauvel, J. Chanussot, and J. A. Benediktsson, “Kernel principal component analysis for the classification of hyperspectral remote sensing data over urban areas,” EURASIP J. Adv. Signal Process., vol. 2009, no. 1, p. 783194, Mar. 2009. [11] E. Magli, “Multiband lossless compression of hyperspectral images,” IEEE Trans. Geosci. Remote Sens., vol. 47, no. 4, pp. 1168–1178, Apr. 2009. [12] Z. Jing and L. Guizhong, “An efficient reordering prediction-based lossless compression algorithm for hyperspectral images,” IEEE Geosci. Remote Sens. Lett., vol. 4, no. 2, pp. 283–287, Apr. 2007. [13] G. Carvajal, B. Penna, and E. Magli, “Unified lossy and near-lossless hyperspectral image compression based on JPEG 2000,” IEEE Geosci. Remote Sens. Lett., vol. 5, no. 4, pp. 593–597, Oct. 2008. [14] J. A. Richards and X. Jia, Remote Sensing and Digital Analysis., 4th ed. New York, NY, USA: Springer-Verlag, 2006. [15] M. Hasan, M. R. Pickering, and X. Jia, “Robust automatic registration of multi-modal satellite images using CCRE with partial volume interpolation,” IEEE Trans. Geosci. Remote Sens., vol. 50, no. 10, pp. 4050–4061, Oct. 2012.