A Maximum Likelihood Approach to Joint Image Registration and Fusion

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 20, NO. 5, MAY 2011

1363

A Maximum Likelihood Approach to Joint Image Registration and Fusion Siyue Chen, Member, IEEE, Qing Guo, Member, IEEE, Henry Leung, Member, IEEE, and Éloi Bossé

Abstract—Both image registration and fusion can be formulated as estimation problems. Instead of estimating the registration parameters and the true scene separately as in the conventional way, we propose a maximum likelihood approach for joint image registration and fusion in this paper. More precisely, the fusion performance is used as the criteria to evaluate the registration accuracy. Hence, the registration parameters can be automatically tuned so that both fusion and registration can be optimized simultaneously. The expectation maximization algorithm is employed to solve this joint optimization problem. The Cramer-Rao bound (CRB) is then derived. Our experiments use several types of sensory images for performance evaluation, such as visual images, IR thermal images, and hyperspectral images. It is shown that the mean square error of estimating the registration parameters using the proposed method is close to the CRBs. At the mean time, an improved fusion performance can be achieved in terms of the edge preservation measure AB=F , compared to the Laplacian pyramid fusion approach.

Q

Index Terms—Affine transformation, expectation maximization, image fusion, image registration, multisensor images.

I. INTRODUCTION

I

MAGE fusion is a process of combining relevant information from two or more input images into a single image with higher quality or more complete information. It has a wide variety of applications in remote sensing, medical diagnosis, computer vision, etc. Since the multiple input images might be taken at different time or from different viewpoints, registration, which aligns images into a common coordinate system, is a crucial step leading to successful fusion. Current registration methods can be broadly classified into feature-based, Fourier-based and intensity-based methods [1]. The feature-based approaches usually extract geometric Manuscript received December 01, 2008; revised April 20, 2009; accepted October 11, 2010. Date of publication November 01, 2010; date of current version April 15, 2011. This work was supported in part by the Natural Sciences and Engineering Research Council (NSERC) of Canada and in part by the State Scholarship of China Scholarship Council under Grant [2007]3020. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Margaret Cheney. S. Chen is with Complex System, Inc., Calgary, Canada, T2L 2K7. She is also with Department of Electrical and Computer Engineering, University of Calgary, Calgary, AB, Canada T2N 1N4 (e-mail: [email protected]). Q. Guo is with Center for Earth Observation and Digital Earth, Chinese Academy of Science, Beijing, China 100190 (e-mail: [email protected]). H. Leung is with the Department of Electrical and Computer Engineering, University of Calgary, Calgary, AB, Canada T2N 1N4 (e-mail: [email protected]). É. Bossé is with the Defense R&D Canada-Valcartier, Valcartier, QC, Canada G3J 1X5 (e-mail: [email protected]). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TIP.2010.2090530

features (also known as control points, such as intersections and landmarks), and use a least square (LS) criteria to estimate the registration parameters [2], [3]. The Fourier-based methods work with images in the frequency domain [4], [5]. They compute the registration parameters by utilizing the properties of translation and rotation in Fourier transform. As for the intensity-based methods, the registration parameters are estimated by maximizing the similarity measures between pixel values of the input images. Normalized cross correlation [6], LS [7], and maximum likelihood (ML) [8] are some popular criteria used. Similarly, the methods of image fusion can also be categorized into the feature-based, the transform-based and the pixelbased methods. The feature-based methods first extract features from the input sources by segmentation [9], [10], and then perform fusion to enable the detection of useful features with higher confidence. The transform-based methods convert the input images into a common transform domain, such as wavelet domains [11], [12] and pyramid domains [13]. Fusion is then applied by combining their transform coefficients. As for the pixel-based methods [14], [15], they usually generate a fused image in which each pixel is determined from a set of pixels from the input sources. Although many algorithms have been proposed for image registration and image fusion, respectively, they are traditionally viewed as two independent processes. More precisely, image registration is performed, followed by image fusion, which usually assumes that perfect registration has been achieved [14], [16]. However, in real practice, the registration process cannot guarantee error-free. And the registration errors can deteriorate the fusion performance or even result in an fused image poorer than the input images. Hence, fusion performance can actually be used as a measure to evaluate whether the input images are aligned properly. Based upon this idea, we propose to treat registration and fusion as a joint process, in which the fusion performance and the registration accuracy are evaluated simultaneously. Under the assumption that only affine transform exists between the input images, registration becomes an estimation problem of the translation and rotation parameters [17]. Meanwhile, the fusion is to obtain the estimate of the true scene [18], [19]. These two estimation problems can be combined into a single ML formulation and solved by the expectation maximization (EM) algorithm. Developed by Dempster et al. [20], the EM algorithm is a general method of finding the ML estimate of the parameters of an underlying distribution from the given observed data set when the data is incomplete or has missing values. It is particularly useful in situations where optimizing the likelihood function is analytically intractable, but the likelihood function can

1057-7149/$26.00 © 2010 IEEE

1364

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 20, NO. 5, MAY 2011

be simplified by assuming the existence of missing data [21]. In our proposed method, the true scene is modeled as a mixture of Gaussians, and the sensor images are described as a linear transform of the true scene plus white Gaussian noise. The EM algorithm is, thus, employed to estimate the true scene along with the model parameters. In the E-step, an approximation to the expectation of the log-likelihood function with the complete data is computed based upon the current parameter estimates. In the M-step, a new parameter estimate is computed by finding the value that maximizes the function found in the E-step. The paper is organized as follows. In Section II, the problem of joint image registration and fusion is formulated. Section III presents the EM algorithm. Performance analysis, which includes the effect of registration error on fusion and the Cramer-Rao bound (CRB), is given in Section IV. The proposed method is then applied to several types of multisensor image pairs for performance evaluation. The experimental results are reported in Sections V and VI concludes the paper.

is the zero mean Gaussian noise the sensor offset, and with a variance . Since the image formation parameters , , and can vary from coefficient to coefficient, the model in (4) can express local polarity reversals, complementary features, spatial variation of sensor gain and noise. In particular, a constraint as proposed in [25] is set so that the value can be taken from only. This constraint acof knowledges that a sensor may be able to “see” certain objects , may fail to “see” other objects , or may “see” certain objects with a polarity-reversed representa. tion Given the previously mentioned models, the complete data set of the joint image registration and fusion problem can be de. Correspondingly, fined as the incomplete data is , and the unknown pawith . rameters are Since each sensor image is an independent observation on the true scene, the log likelihood function of the complete data can be written as

II. PROBLEM FORMULATION Assume that sensor images are generated from a true scene, which is an image obtained under conditions of uniform lighting, unlimited visibility and perfect sensors [23], [24]. , in which is Let’s denote the true scene as the pixel coordinate. It should be noted that could be the intensity value or the transform coefficient. If is the transform coefficient, such as the multiscale transform (MST) coefficient, is referred to as the pixel coordinate at the th level of the pyramid by default. Registration is to align one image to another. Hence, without loss of generality, we assume two sensor images, denoted as and , with , which indicates that the sensor is well aligned with the true scene . As for the image other image , we assume that an affine transform exists between and . That is [22] (1) where and is

(5) Assuming tion, we have

follows the

-term mixture Gaussian distribu-

(6) It

is

difficult

to

optimize

as it contains the log of the sum. A hidden random variable is, thus, introduced to identify which term in the mixture probability density function (PDF) (6) produces . By this way, the likelihood function is simplified as

represents the - and -axis translations,

(2) The four elements of cover any combination of rotation, skewing, shearing, and scaling. Substituting into (1), we have

(3) Given the correspondence between and , now let’s explore the relationship between and . Based upon the formation model of sensory images [23], [24], the mapping from to can be written as (4) where denotes the sensor gain (which includes the effects of local polarity reversals and complementarity), is

(7) where

the

still and

complete data becomes , the incomplete data is , and the unknown parameters are with .

III. JOINT IMAGE REGISTRATION AND FUSION USING EM In this study, the EM algorithm is employed to estimate the model parameters and to produce the fused image. Each iteration of the EM algorithm consists of two main steps

(8) (9)

CHEN et al.: A MAXIMUM LIKELIHOOD APPROACH TO JOINT IMAGE REGISTRATION AND FUSION

where represents to the th iteration. The EM algorithm is apand each pixel location to obplied to each sensor image and . In performing these estimates, tain the estimates of the computation will employ the pixels within an window around . Hence, we assume that a rough registration has been performed so that the registration error is within the window. In addition, we also assume that the image formation parameters vary slowly with location [23]. In particular, we assume they hold the same value within the window. For this reason, the indices on these parameters can be dropped. For ex. ample, we have

1365

independent of . The fused image is computed by maximizing with respect to . That is, by setting

(14) we have

A. E-Step The computation of the conditional expectation in the E-step requires the following PDFs. The joint marginal PDF of and given the complete data is

(15) (10)

In addition, the conditional PDF

B. M-Step In the M-step, the EM algorithm would update the parameter in (13). estimates to the new values that maximize However, the maximization of in (13) is difficult to perform analytically due to the nonlinear coupling between the and . This problem can be solved by the parameters , in (13) is SAGE algorithm [26]. More precisely, first maximized by setting the differential of with respect to and to zero with fixed at . That is

is

(11) for

, and

can be written as (16) leads to

(12) based upon (4). The E-step performs an average over the unavailable parts of the complete data conditioned on the incom. That is, see plete data and current parameter estimates equation (13) at the bottom of the page, where is a term that is

(17)

(13)

1366

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 20, NO. 5, MAY 2011

and

where

(18) leads to (19) The estimated is quantized into 1, is maximized with respect to fixed at and , i.e.,

, or 0. Then, with and

(26)

(20)

and , a cubic convolution To compute interpolation [27] with the following kernel is used

Similarly, we can obtain the other parameters as follows: (21)

(27) .

(22)

Assume that is the pixel coordinate corresponding to with the given registration parameters. If is within a rectangular subdivision , where rounds to the nearest integers towards minus infinity, we have

(23)

As for updating the registration parameters, we reorganize them into , i.e., (24)

(28) . Therefore, see where equation (29) at the bottom of the page, where (30)

and obtain the estimates by solving can be computed in a similar way. IV. PERFORMANCE ANALYSIS

(25)

In this section, we perform theoretical analysis on the effect of registration error on the fusion performance. We then show that the proposed joint registration and fusion method is indeed

(29)

CHEN et al.: A MAXIMUM LIKELIHOOD APPROACH TO JOINT IMAGE REGISTRATION AND FUSION

converged. The CRB of the joint registration and fusion estimation problem is also derived to assess the efficiency of the proposed method. A. Effect of Registration Error on Image Fusion Assume that there is no registration error. The estimation error of can be written as

1367

B. CRB of Joint Image Registration and Fusion CRB is a performance bound that measures the accuracy of an estimate. The CRB of EM is defined by (36) where

is the Fisher Information Matrix, given by (37)

The elements of the previously mentioned as

can be derived

(31) Substituting (4) into (31), we have

(32) where . , is white Gaussian noises with zero Considering , mean, the expectation of the estimation error on the true scene . In other words, the estimate of is equal to zero, i.e., by EM is unbiased if there is no registration error. , where is the registration error. This Let leads to the deviation from to . In this case, the estimation can be written as See equation (33) at the bottom error on , (33), shown of the page. If we define at the bottom of the page, can be further given by (34), shown at the bottom of the page. And the expectation of the estimation error becomes

(38) (39)

(40)

(41) From the block matrix inverse theory, we have

(35) Equation (35) indicates that the bias of the estimation on depends upon the differences between the pixel values at and . As the registration error increases, the disand becomes larger, tance between which results in an increased and .

(42) and

(43)

(33)

(34)

1368

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 20, NO. 5, MAY 2011

TABLE I INPUT IMAGE PAIR DETAILS

Fig. 1. Illustrations of the 15 pairs of testing images.

Denoting the elements of

and can be expressed as

,

to decompose the image pairs into five levels, and the EM algorithm is applied to perform joint registration and fusion. The number of Gaussians in (6) is set to 2, and the local analysis window size is 5 5. A simple average of the sensor image is used as the initial guess of the true scene. The initial values of and , are 0.76 and 0.24, respecthe mixing parameters, is initialized as tively. The variance of noise, i.e., (44) while

is initialized as (45)

where defined as

. The stopping criteria is

(46)

In addition, based upon (26) and (29), the expectations of and can be computed. V. EXPERIMENTAL RESULTS In this section, 15 pairs of multisensor images, obtained from the database of Manchester University [28], are used to evaluate the performance of the proposed joint method. These images were captured by various types of sensors, such as Daedalus scanner, CASI hyperspectral scanner, 30 channel DRA hyperspectral scanner, visual light camera, and infra-red (IR) thermal camera, as listed in Table I. In addition, they include a wide variety of industrial, urban and natural scenes, as shown in Fig. 1. It should be noted that the images in this data set are already registered. To simulate the misalignment between multisensor images, the images generated by Sensor 1 are manually rotated and translated. Laplacian pyramid transform [29] is then used

where . It is observed that the algorithm in our experiments generally converged in 17 to 42 iterations. The results of joint registration and fusion are shown in Fig. 2. As seen, the proposed method can combine the complementary information from multiple sensors and improve the perceptual quality. In order to have an objective evaluation of the joint approach from the perspective of estimation, MSE results of registration are obtained from experiments and compared to the derived CRBs. In the experiments, the images shown in Fig. 1 are used as the clean sensor images. Gaussian noise is, thus, added to the . For a fixed clean sensor images to generate noise variance, 80 Monte-Carlo runs are carried out on each image pairs. In each Monte-Carlo simulation, the registration parameters are randomly generated. The MSE is then computed over all the testingimages and all the Monte-Carlo runs, i.e., (47) where

denotes the estimate at the th trial, and . Since the true scene is not available, we can not

CHEN et al.: A MAXIMUM LIKELIHOOD APPROACH TO JOINT IMAGE REGISTRATION AND FUSION

1369

Fig. 4. Comparison of the CRB and MSE of the estimation of the translation parameter at y -axis.

Fig. 2. Fused image by using the proposed joint approach.

Fig. 5. Comparison of the CRB and MSE of the estimation of the rotation parameter.

Fig. 3. Comparison of the CRB and MSE of the estimation of the translation parameter at x-axis.

compare the MSE between the fused image and the true scene with the corresponding CRB. Figs. 3–5 plot the MSE results of estimation of registration parameters versus the signal-to-noise ratios (SNRs), which varies from 0 dB to 40 dB. It is seen that the estimation of translation parameters is very close to the CRB at a high SNR value, i.e., when SNR is higher than 20 dB. As for the estimation of rotation parameters, it is slightly deviated

from the CRB even at a high SNR value. A common observation of both translation and rotation parameter estimation is that the gap between the estimation by EM and the CRB becomes larger as the SNR becomes smaller. We then compare the proposed joint approach with two standard image registration approaches, i.e., the control-point based least square (CP-LS) [30] method and the mutual information (MI) based method [31]. In the CP-LS method, a number of control points is first selected uniformly within the images, and the registration parameters are then estimated by means of the least square fit. For the MI based method, we use an exhaustive search to find the optimal registration parameters which maximize the MI. The comparison results are illustrated in Figs. 6–8. It is found that the joint approach consistently outperforms the CP-LS method. However, it is not as good as the MI based method when the SNR is lower than 10 dB. When the SNR is further decreased, it is slightly better than the MI based method, which shows its robustness to noise compared to the MI based method. It should be noted that the superiority of the MI based

1370

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 20, NO. 5, MAY 2011

Fig. 6. Performance results of estimating the translation parameter at x-axis by the joint approach, the CP-LS and the MI based method.

Fig. 8. Performance results of estimating the rotation parameter by the joint approach, the CP-LS and the MI based method.

Fig. 7. Performance results of estimating the translation parameter at y -axis by the joint approach, the CP-LS and the MI based method.

Fig. 9. Fusion performances of the joint approach and the separate processes . of registration and fusion in terms of the value of Q

method is partially due to the exhaustive search strategy to maximize the MI criteria, which guarantee a global optimization while at the same time is very resource-consuming. It is found that using the same computer, the MI-based time requires almost 19 times of computation time over the joint approach. Furthermore, the fusion performances of the proposed joint approach and the separate processes of registration and fusion are compared. For the separate processes, the two registration methods, i.e., CP-LS and the MI-based method, are still employed. After registration, the Laplacian pyramid approach as in [33] is used to produce fused images. The Laplacian pyramid approach has the same pyramid decomposition process as the joint approach. However, after decomposition, the Laplacian approach performs fusion by averaging all those decompositions, while the joint approach performs fusion by selecting weighting coefficients to optimize the likelihood function. The [32], which is based upon the idea of measuring metric the preservation of input edge information in the fused image, is used to evaluate the fusion performance objectively. This

TABLE II PERFORMANCE RESULTS OF FUSION WHEN SNR = 25 dB

metric has been shown to be the most consistent one with subjective evaluation among several popular metrics used for fuof all sion evaluation [28]. Table II lists the values of the testingimages when SNR is at 25 dB. It is seen that the joint

CHEN et al.: A MAXIMUM LIKELIHOOD APPROACH TO JOINT IMAGE REGISTRATION AND FUSION

approach consistently outperforms the separate scheme with the CP-LS method for registration. Meanwhile, in most cases, the joint approach has a better fusion performance than the separate scheme with the MI-based method for registration. However, for the images of No. 2, 5, 6, 9, and 11, the separate scheme with the MI-based registration method is slightly better than the joint approach. This is not only due to the higher registration error of the joint approach, but also due to its nonsignificant improvement of fusion performance compared to the simple coefficient averaging as in the separate scheme. Fig. 9 plots the average over 15 images when SNR varies from 0 dB to 40 dB. It is seen that at high SNR, e.g., 35 and 40 dB, the joint approach is not as good as the separate scheme with the MI-based registration method. As SNR decreases, the superiority of the joint approach becomes more and more obvious. In addition, the joint approach has a much better fusion performance than the separate scheme with the CP-LS registration method. VI. CONCLUSION This paper proposes to use the EM algorithm to solve the joint image registration and fusion problem. This method differs from most methods in the literature in the sense that registration and fusion are performed simultaneously. The registration parameters are estimated in the M-step of the EM by maximizing the conditional expectation function, and fusion is performed in the E-step by using the current estimate of the model parameters. The major advantage of the proposed joint approach is to automatically tune the registration parameters so that an optimal fusion performance can be achieved. The CRB is derived to evaluate the performance of the proposed joint method. Applying the proposed method to several types of multisensor images, we show that the MSEs between the estimated and the true registration parameters are consistent with their CRBs at high SNRs. Furthermore, the joint approach is found to have a better fusion performance due to its ability of mitigating the effects of registration errors and noise corruptions. VII. ACKNOWLEDGEMENT The authors would like to thank Manchester University and Dr. V. Petrovic for providing the multisensor images. REFERENCES [1] B. Zitová and J. Flusser, “Image registration methods: A survey,” Image Vis. Comput., vol. 21, pp. 977–1000, 2003. [2] K. S. Arun, T. S. Huang, and S. D. Blostein, “Least-squares fitting of two 3-D point sets,” Trans. Pattern Anal. Mach. Intell., vol. PAMI-9, no. 5, pp. 698–700, Sep. 1987. [3] S. Umeyama, “Least-squares estimation of transformation parameters between two point patterns,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 13, no. 4, pp. 376–380, Apr. 1991. [4] B. S. Reddy and B. N. Chatterji, “An FFT-based technique for translation, rotation, and scale-invariant image registration,” IEEE Trans. Image Process., vol. 5, no. 8, pp. 1266–1271, Aug. 1996. [5] H. S. Stone, B. Tao, and M. McGuirre, “Analysis of image registration noise due to rotationally dependent aliasing,” J. Vis. Commun. Image Represent., vol. 14, pp. 114–135, 2003. [6] P. A. Van den Elsen, E. D. Pol, T. S. Sumanaweera, P. F. Her, S. Napel, and J. R. Adler, “Grey value correlation techniques used for for automatic matching of CT and MR brain and spine images,” in Proc. SPIE Visualizat. Biomed. Comput., 1994, vol. 2357, pp. 227–237.

1371

[7] P. Thevenaz, U. E. Ruttimann, and M. Unser, “A pyramid approach to subpixel registration based on intensity,” IEEE Trans. Image Process., vol. 7, no. 1, pp. 27–41, Jan. 1998. [8] W. L. S. Costa, D. R. Haynor, R. M. Haralick, T. K. Lwellen, and M. M. Graham, “A maximum-likelihood approach to PET emission/ attenuation image registration,” in Proc. IEEE Nucl. Sci. Symp. Med. Imag. Conf., 1993, pp. 1139–1143. [9] D. A. Clausi and H. W. Deng, “Feature fusion for image texture segmentation,” in Proc. IEEE 17th Int. Conf. Pattern Recognit., Aug. 2004, vol. 1, pp. 580–583. [10] X. Shi and R. Manduchi, “A study on Bayes feature fusion for image classification,” in Proc. IEEE Workshop Statist. Alg. Comput. Vis., Madison, WI, 2003, p. 95. [11] M. Gonzalez-Audicana, J. L. Saleta, R. G. Catalan, and R. Garcia, “Fusion of multispectral and panchromatic images using improved IHS and PCA mergers based on wavelet decomposition,” IEEE Trans. Geosci. Remote Sens., vol. 42, no. 6, pp. 1291–1299, Jun. 2004. [12] K. Amolins, Y. Zhang, and P. Dare, “Applications of wavelet transforms in image fusion,” in Proc. Urban Remote Sens. Joint Event, 2007, pp. 1–7. [13] Z. Liu, K. Tsukada, K. Hanasaki, Y. K. Ho, and Y. P. Dai, “Image fusion by using steerable pyramid,” Pattern Recognit. Lett., vol. 22, no. 9, pp. 929–939, Jul. 2001. [14] V. S. Petrovic and C. S. Xydeas, “Gradient-based multiresolution image fusion,” IEEE Trans. Image Process., vol. 13, no. 2, pp. 228–237, Feb. 2004. [15] Y. Xia and M. S. Kamel, “Novel cooperative neural fusion algorithms for image restoration and image fusion,” IEEE Trans. Image Process., vol. 16, no. 2, pp. 367–381, Feb. 2007. [16] S. Zheng, W. Z. Shi, J. Liu, G. X. Zhu, and J. W. Tian, “Multisource image fusion method using support value transform,” IEEE Trans. Image Process., vol. 16, no. 7, pp. 1831–1839, Jul. 2007. [17] Y. M. Zhu and S. M. Cochoff, “Likelihood maximization approach to image registration,” IEEE Trans. Image Process., vol. 11, no. 12, pp. 1417–1426, Dec. 2002. [18] X. Shi and R. Manduchi, “On the bayes fusion of visual features,” Image Vis. Comput., vol. 25, no. 11, pp. 1748–1758, Nov. 2007. [19] Y. Yang, C. Z. Han, X. Kang, and D. Q. Han, “An overview on pixellevel image fusion in remote sensing,” in Proc. IEEE Int. Conf. Autom. Logistics, Aug. 2007, pp. 2339–2344. [20] P. Dempster, N. Laird, and D. Rubin, “Maximum likelihood from incomplete data via the EM algorithm,” J. Roy. Stat., vol. 39, pp. 1–38, 1977. [21] H. Hong and D. Schonfeld, “Maximum-entropy expectation-maximization algorithm for image reconstruction and sensor field estimation,” IEEE Trans. Image Process., vol. 17, no. 6, pp. 897–907, Jun. 2008. [22] A. A. Goshtasby, 2-D and 3-D Image Registration for Medical, Remote Sensing, and Industrial Applications. Hoboken, NJ: Wiley, 2005. [23] R. K. Sharma, T. K. Leen, and M. Pavel, “Probabilistic image sensor fusion,” in Advances in Neural Information Processing Systems 11. Cambridge, MA: MIT Press, 1999. [24] , R. S. Blum and Z. Liu, Eds., Multi-Sensor Image Fusion and Its Applications. Boca Raton, FL: CRC Press, 2006. [25] J. Yang and R. S. Blum, “A statistical signal processing approach to image fusion for conceled weapon detection,” in Proc. IEEE Int. Conf. Image Process., 2002, vol. I, pp. 513–516. [26] J. A. Fessler and A. O. Hero, “Space-alternating generalized EM algorithm,” IEEE Trans. Signal Process., vol. 42, no. 10, pp. 2664–2677, Oct. 1994. [27] W. Li and H. Leung, “A maximum likelihood approach for image registration using control point and intensity,” IEEE Trans. Image Process., vol. 13, no. 8, pp. 1115–1127, Aug. 2004. [28] V. Petrovic, “Subjective tests for image fusion evaluation and objective metric validation,” Inf. Fus., vol. 8, pp. 208–216, 2007. [29] P. J. Burt and E. Adelson, “The Laplacian pyramid as a compact image code,” IEEE Trans. Commun., vol. COM-31, no. 4, pp. 532–540, Apr. 1983. [30] B. Zitova and J. Flusser, “Image registration methods: A survey,” Image Vis. Comput., vol. 21, pp. 977–1000, 2003. [31] H. Chen and P. K. Varshney, “Mutual information based CT-MR brain image registration using generalized partial volume joint histogram estimation,” IEEE Trans. Med. Imag., vol. 22, no. 9, pp. 1111–1119, Sep. 2003. [32] C. Xydeas and V. Petrovic, “Objective image fusion performance with visible differences,” Electron. Lett., vol. 36, pp. 308–309, 2000. [33] Z. Zhang and R. S. Blum, “A categorization and study of multiscaledecomposition-based image fusion schemes,” Proc. IEEE, vol. 87, no. 8, pp. 1315–1328, Aug. 1999.

1372

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 20, NO. 5, MAY 2011

Siyue Chen (M’00) received the B.S. degree from the School of Electronic Information, Wuhan University, Wuhan, China, in 1999, and the M.Sc. and Ph.D. degrees in electrical and computer engineering from the University of Calgary, AB, Canada, in 2001 and 2005, respectively. Since 2006, she has been working as a Software Engineer at Complex System, Inc., Calgary, Canada. She is also a Postdoctoral Research Fellow at the University of Calgary. Her research interests include data fusion, signal processing, multimedia, speech enhancement, and sensor network.

Qing Guo (M’09) received the Ph.D. degree from the Department of Physics, Harbin Institute of Technology, Harbin, China, in 2010. She was an exchange student in the Department of Electrical and Computer Engineering, University of Calgary, AB, Canada, from September 2007 to August 2009. She is currently with the Center for Earth Observation and Digital Earth, Chinese Academy of Sciences, China. Her current interests include remote sensing image processing (registration and fusion), optical information processing and information security.

Henry Leung (M’90) received the Ph.D. degree in electrical and computer engineering from McMaster University, Hamilton, ON, Canada. He is currently a Professor in the Department of Electrical and Computer Engineering, University of Calgary, AB, Canada. Before that he was with the Defense Research Establishment Ottawa, ON, Canada, where he was involved in the design of automated systems for air and maritime multisensor surveillance. His research interests include chaos, computational intelligence, data mining, information fusion, nonlinear signal processing, multimedia, sensor networks, and wireless communications.

Éloi Bossé received the B.A.Sc., M.A.Sc., and Ph.D. degrees in electrical engineering from Laval University, Quebec, Canada. He is currently the head of Decision Support System Section, Defense Research and Development Canada-Valcartier, Quebec, Canada. In 1981, he joined the Communications Research Center, Ottawa, Canada, where he worked on radar signal processing, high resolution spectral analysis, and radar tracking in multipath. In 1988, he was transferred to the Defense Research Establishment Ottawa (DREO) and in (1992) to the Defense Research and Development Canada, DRDC, Valcartier, QC, Canada, where he worked on data and information fusion, and now heads the Decision Support Systems Section. His current interests include data and information fusion, reasoning under uncertainty, neural networks, evidential, and possibilistic theories. He has published over 80 papers in journals and conference proceedings.