Efficiency Determination of Scanner Data Fusion Methods of Space

International Young Scientists Forum on Applied Physics

YSF-2015

Efficiency Determination of Scanner Data Fusion Methods of Space Multispectral Images V.V. Hnatushenko

O.O. Kavats

Department automated information processing systems O.Gonchar Dnepropetrovsk national university, DNU Dnepropetrovsk, Ukraine e-mail: [email protected]

Department information technologies and systems National metallurgical academy of Ukraine, NMetAU Dnepropetrovsk, Ukraine e-mail: [email protected]

I.O. Kibukevych Department information technologies and systems National metallurgical academy of Ukraine, NMetAU Dnepropetrovsk, Ukraine e-mail: [email protected] Abstract — This paper explores the major remote sensing data fusion techniques at pixel level and reviews the concept, principals, limitations and advantages for each technique. This paper focused on traditional techniques like intensity huesaturation- (IHS), Brovey, principal component analysis (PCA) and Wavelet. The results show that in comparison with classical fusion methods synergistic processing of scanner multispectral data using the proposed information technology based on ICAand wavelet transformation gives more qualitative result. The synthesized image has high information content without color distortion. For each of the indices used to assess spatial and spectral quality of the merged images, the best results are obtained with the proposed method. Keywords — panchromatic; multispectral image; fusion methods; ICA; wavelet.

I.

INTRODUCTION

Images of modern remote-sensing systems allow us to solve various problems such as carrying out operational monitoring of land resources, city building, monitoring state of the environment and influence of anthropogenic factors, detecting contaminated territories, unauthorized buildings, estimating the state of forest plantation and other. Classically in solving these problems are used fusion methods of images with different spectral channels for increasing information content of initial data [1-3]. In satellite images, the lower spatial resolution multispectral images are fused with higher spatial resolution panchromatic images. To receive a synthesized image with the best value of information content it is necessary to choose the most effective fusion method because it affects the thematic problem solving. II.

STATEMENT OF THE PROBLEM AND ANALYSIS OF RELATED WORK

Many fusion techniques have been proposed for the fusion of multispectral and panchromatic images [3-10]. Many survey papers have been published recently, providing

September 29 – October 2, 2015 | Dnipropetrovsk, Ukraine

overviews of the history, developments, and the current state of the art of remote sensing data processing in the image based application fields. The selection of an appropriate image fusion technique has a great influence on the resulting fused product, which again limits its applicability. Spatial resolution of multispectral images increases, as a rule, in consequence of implementation following basic steps: 1. Converting a multispectral image with low spatial resolution from the red-green-blue (RGB) basis to any three coordinate basis, in which one of the coordinate is equivalent to brightness distribution. 2. Increasing the sample rate of converted image up to sample rate of panchromatic image and further interpolation (linear, bicubic, nearest neighbor rule, etc.). 3. Replacement of the brightness component of the converted image with a panchromatic image of high spatial resolution. 4. Inverse transformation in the RGB basis. The quality of the fusion method is determined by the increase of spatial resolution and the existence (absence) of color distortion. Numerous works related to preprocessing of multichannel digital images focus on improving their visual quality excluding physical mechanisms of fixing information, especially, correlation between channels, making it impossible to determine the image information content from the standpoint of analysis and interpretation (method “Brovey”). Other works based on calculation of statistic parameters of digital images (“principal components method”, (PCA)), determination of which is difficult due to large dimension of primary data [3]. Some works are based on the transition to color-difference metrics of computer graphics ("color-metric methods"), where the question about decorellation of primary data is solved [6]. The modification to the IHS method, called the IHS-SA, proposed the incorporation of weighted coefficients on the green and blue bands so as to reduce the difference between the intensity and the panchromatic bands. However, these methods allow us to take into account only spectral components of primary grayscale image. One of the

978-1-4673-6977-0/15/$31.00 ©2015 IEEE

1

International Young Scientists Forum on Applied Physics most modern methods is the variations of the discrete wavelet transformation (DWT) [5, 8, 9]. But all the classic methods, including Gram-Schmidt, Brovey, PCA, independent component analysis (ICA) and IHS-algorithm, do not include constructing characteristics of modern scanning devices, appropriate structures and data formats of scanning systems [3-6, 10]. Separate use of these approaches leads to distortion of primary color images. The reasons for distortions are algorithms which were designed for mergering images from satellite SPOT. In contrast to the corresponding characteristics of the space device, the panchrom-wave length of modern satellites (IKONOS, Worldview-2, Worldview-3 etc.) has been extended from the visible to the near infrared. In addition, modern scanner systems are more advanced and have more than four channels (such as WorldView-3 - eight channels). Therefore there is a necessity in new technology, which will increase spatial resolution of satellite images considering physical mechanisms of fixing species information and conducting further research of fusion efficiency with obtaining quantitative estimates (criteria) of information content (quality) of synthesized images. III.

MAIN PART

In [11] a new information technology based on ICA- and wavelet transformations is proposed, which can significantly increase the information content of the primary data and does not lead to color distortion. The algorithm scheme is shown in Fig. 1. After ICA converting it is proposed to replace the first component of multispectral image with panchromatic image. Next step after replacing is the inverse ICA converting and converting received image into the HSV color model (HSVMOD). Another step of the algorithm is transformation original multispectral images in HSV color space (HSVMUL). Next step of formation a new image with high information content is replacement brightness component V of HSVMUL image with brightness component of HSVMOD and converting result from HSV color model to RGB color model. The final step of the algorithm is fusion of the MulICA image with PAN using wavelets and receive output image. One of the most important results is determination of basic wavelets and the level of decomposition. It was defined that the best results of panchromatic and multispectral image fusion gives Daubechies wavelet by 20th decomposition level (Mul ICA-Wavelet Daubechies). The independent component analysis is considered as an extension of the principal components analysis to the problem of blind separation of independent sources from their linear mixtures. The independent component analysis is a statistical method for transforming an observed multidimensional random vector into components that are statistically as independent from each other as possible. In addition, the axis need not be orthogonal. The model used in the analysis of independent components can be represented as: y = H x,

(1)

where, y is a m-dimensional random vector, x is a n-dimensional random vector with independent components, H is some unknown transformation Rn → Rm, m ≥ n. One of the most advanced and powerful mathematical

2

YSF-2015

tools for pansharpening aerospace images is wavelet transform [1, 11]. But the separate application of the wavelet transform often leads to artifacts in the synthesized image. Initially, the images are subject to decomposition of the first level, as a result of which, for each of the images matrixes of approximating and detailing coefficients are obtained. The detailing coefficients obtained for both images form linear combinations with taking into account the numerical values of the coefficients of the linear forms of combination (a, b), that is: ⎧ AppiN = ciN , ⎪ N ⎨ l ,1 l ,1 l ,2 l ,2 l ,3 l ,3 ⎪ Deti = ∑ {a ⋅ d P + b ⋅ di } ,{a ⋅ d P + b ⋅ d i } ,{a ⋅ d P + b ⋅ d i } l =1 ⎩

(

)

, (2)

where, AppiN and Deti are approximating and detailing wavelet components of the new multispectral image. The subscript indicates the appropriate multispectral channel ( i = { R, G, B} ) or panchromatic image ( P ). Then the first level of the wavelet reconstruction is performed. The input data for the implementation of wavelet reconstruction is the approximating coefficients obtained after wavelet decomposition of the synthesized color images, which are taken without changes, and, as the detailing coefficients, the obtained linear combinations are used. Then follows the search of arguments, which are the coefficients of the linear combination of forms, in which the entropy for the new synthesized image reaches maximum values [12]. Thus, as a result of wavelet reconstruction, we obtained multichannel images with high information content, synthesized with the use of wavelets, by maximizing the entropy function.

Figure 1. Algorithm scheme

The evaluation of the employed fusion technique is also an important step in the fusion process. Various quality metrics have been used in the literature to study, compare and assess implementation of the fusion technique. In order to determine the capabilities of each method as for the quality of multispectral

September 29 – October 2, 2015 | Dnipropetrovsk, Ukraine

International Young Scientists Forum on Applied Physics images, the quantitative estimates of the information content of initial and synthesized multispectral images were obtained for all fusion methods and for the newly developed technology based on ICA- and wavelet transformations, such as information and signal entropy, SSIM, PSNR, etc. Decorrelation methods of spatial brightness distribution are based on the calculation of statistical parameters of digital images, determination of which is difficult for large primary data. Also, these methods take into account only the contribution of spectral information contained in the original multispectral images [13]. Entropy can directly reflect the average information content of an image. If the entropy of a fused image is higher than that of a parent image, this is an indication that the fused image contains more information. The correlation coefficient is a similarity metric wherein a value closer to one represents good similarity between the compared images. SSIM should be near to +1 for good quality fusion [14]. PSNR value should be as high as possible.

Table 2 shows the PSNR values for each fusion methods for three channels.

Research of influence of fusion methods was performed on the primary scanner images by WorldView-2 satellite. The figure 2 (a, b) shows fragments of panchromatic and multispectral images. After image fusion by methods discussed above, images that even visually are more precise than primary multispectral image were obtained, but with significant color distortion (Fig. 2 c-k). The synthesized image of the new information technology based on ICA- and wavelet-transformation is shown in Figure 2 (l). The table 1 shows the values of information and signal entropies obtained for multispectral and panchromatic primary images and also for the synthesized images by all specified methods (image size 4604 * 4600 pixels). From table 1 it is clear that entropies of the image merged by proposed technology is higher than that of the original image. This clearly indicates information content of fuse image is greater than that of original image. TABLE I. THE VALUES OF INFORMATION AND SIGNAL ENTROPY Image Panchromatic image Primary multispectral image Test multispectral image PCA ICA HSV Brovey Gram-Schmidt Linear IHS Nonlinear IHSmodel Wavelet Haar Wavelet Daubechies Proposed technology

Marking

Value of Information Entropy

Value of Signal Entropy

PAN

7.103

7.541

MUL

7.007

7.195

TEST

7.246

7.526

PCA ICA HSV CN BROVEY GRAM-SCHMIDT IHS LINEAR

7.077 7.004 6.897 6.057 7.053 7.145

7.119 7.195 6.909 6.685 7.067 7.561

IHS NONLINEAR

7.249

7.795

HAAR

7.151

7.484

DAUBECHIES

7.177

7.558

PROPOSED

7.314

7.381

September 29 – October 2, 2015 | Dnipropetrovsk, Ukraine

YSF-2015

a)

b)

c)

d)

e)

f)

g)

h)

i)

k)

l)

j)

Figure 2. Fragments of the panchromatic and the multispectral images: a) primary panchromatic, b) primary multispectral, c) synthesized by the principal components method, d) by the independent component method, e) by IHS linear model, f) by nonlinear model for HIS, g) by HSV method, h) by the Brovey method, i) by the method of Gram – Schmidt, j) using Haar wavelet, k) by Daubechies wavelets, l) synthesized the proposed technology. TABLE II. PSNR VALUES Image /Channel

R

G

B

MUL PCA ICA HSV CN BROVEY GRAM-SCHMIDT IHS LINEAR IHS NONLINEAR HAAR DAUBECHIES

32.276 33.223 33.223 33.275 33.011 33.186 33.177 32.969 33.918 33.911

32.275 33.616 33.616 33.254 33.375 33.350 33.604 33.017 34.374 34.380

32.275 33.692 33.691 33.290 33.442 33.442 33.703 33.343 34.425 34.452

PROPOSED

34.313

34.965

34.280

Tables 3, 4 show the CORR and SSIM values for each fusion methods for three channels. Wavelets fare better than the standard methods in that the spectral information distortion is minimized.

3

International Young Scientists Forum on Applied Physics TABLE III. CORR VALUES Image /Channel

R

G

B

MUL PCA ICA HSV CN BROVEY GRAM-SCHMIDT IHS LINEAR IHS NONLINEAR HAAR DAUBECHIES PROPOSED

0.859 0.955 0.955 0.959 0.946 0.958 0.939 0.932 0.967 0.967 0.972

0.859 0.961 0.961 0.958 0.954 0.960 0.951 0.935 0.972 0.972 0.975

0.859 0.958 0.958 0.955 0.949 0.958 0.948 0.935 0.970 0.971 0.976

TABLE IV. Image /Channel MUL PCA ICA HSV CN BROVEY GRAM-SCHMIDT IHS LINEAR IHS NONLINEAR HAAR DAUBECHIES PROPOSED

SSIM VALUES R 0.555 0.652 0.732 0.755 0.778 0.762 0.759 0.766 0.797 0.884 0.892

G 0.555 0.683 0.711 0.776 0.707 0.700 0.799 0.842 0.626 0.813 0.901

Our future research will focus on the improvement of the proposed technology in the processing of multichannel digital images involving information obtained in the infrared range. References [1]

[2]

[3]

[4] B 0.555 0.663 0.750 0.768 0.794 0.789 0.781 0.858 0.604 0.795 0.893

[5]

[6]

[7]

[8]

IV.  CONCLUSIONS Different methods are studied and compared with the help of assessment parameters. We provided an overview of the main methods and approaches available in the literature, including the standard (and widely used) methods and the most recent developments. Most of the fusion techniques that have been proposed are based on the compromise between the desired spatial enhancement and the spectral consistency. In this paper, an information technology based on ICA- and wavelet transformation has been adopted for multichannel image fusion. The qualitative experimental results show that the processing of multispectral images using the new information technology based on ICA- and wavelettransformations in comparison with classical fusion methods, gives a result of a better quality. The synthesized image has higher information content in comparison with the primary images without color distortion. This is evidenced by obtained characteristics of SSIM, PSNR, CORR and others. The research has shown that the value of information entropy exceeds the corresponding values obtained for the results of using other fusion methods.

4

YSF-2015

[9]

[10]

[11]

[12]

[13]

[14]

G. Vivone, L. Alparone, J. Chanussot, M. D. Mura, Garzelli, and G. Licciardi, “A critical comparison of pansharpening algorithms,” in Proc. IEEE Int. Conf. Geoscience Remote Sensing, pp. 191–194, July 2014. V. V. Gnatushenko, “The use of geometrical methods in multispectral image processing”, Journal of Automation and Information Sciences, vol. 35, Issue 12, pp. 2-9, 2003. C. Pohl and J. L. Genderen Van, “Multisensor image fusion in remote sensing: concepts, methods and applications”, International journal of remote sensing, vol. 19, no. 5, pp. 823-854, 1998. Y. Sumantyo and Hiroaki Kuze “Performance Analyzing of High Resolution Pan-sharpening Techniques: Increasing Image Quality for Classification using Supervised Kernel Support Vector Machine”, Research Journal of Information Technology, vol. 3, pp.12-23, 2011. Chen F. et al, “Fusion of remote sensing images using improved ICA mergers based on wavelet decomposition”, Procedia Eng. 29, pp. 29382943, 2012. Chu Heng and Zhu Weile, “Fusion of IKONOS Satellite Imagery Using IHS Transform and Local Variation”, IEEE Transactions on Geoscience and remote sensing, vol. 5, no. 4, pp. 653 – 657, October 2008. R. Akula, R. Gupta, M. R. Devi, and Vimalda, “An efficient PAN sharpening technique by merging two hybrid approaches”, Procedia Eng. 30, pp. 535–541, 2012. D. E. Nirmala, B. S. Paul, and V. Vaidehi, “A novel multimodal image fusion method using shift invariant discrete wavelet transform and support vector machines”, Proc. of the International Conference on Recent Trends in Information Technology, IEEE Xplore Press, Chennai, Tamil Nadu, pp. 932-937, Jun. 3-5 2011. G. P. Nason and B. W. Silverman, “The stationary wavelet transform and some statistical applications,” in Wavelets and Statistics, vol. 103, A. Antoniadis and G. Oppenheim, Eds. New York: Springer-Verlag, pp. 281–299, 1995. C. Thomas, T. Ranchin, L. Wald, and J. Chanussot, “Synthesis of multispectral images to high spatial resolution: A critical review of fusion methods based on remote sensing physics,” IEEE Trans. Geoscience Remote Sensing, vol. 46, no. 5, pp. 1301–1312, May 2008. V. Hnatushenko and A. Kavats “Information technology increase spatial resolution of digital satellite images based on wavelet transformation and IСA”, Proceedings of the National University "Lviv Polytechnic", series "Computer Science and Information Technology", Lviv, pp. 2832, 2013. V. V. Hnatushenko, Vik. V. Hnatushenko, O. O. Kavats, and V. Yu Shevchenko, “Pansharpening technology of high resolution multispectral and panchromatic satellite images”, Scientific Bulletin of National Mining University, Issue 4, pp. 91-98, 2015. Z. J. Wang, D. Ziou, and C. A Armenakis, “Comparative analysis of image fusion methods”, IEEE Trans. On Geoscience and Remote Sensing, no. 43(6), pp. 1391–1402, 2005. Z. Wang and A.C. Bovik, “A universal image quality index”, IEEE Signal Process. vol. 9, no. 3, pp. 81–84, March 2002.

September 29 – October 2, 2015 | Dnipropetrovsk, Ukraine