frequency domain which reduces the computational overhead significantly. ... Image Registration by Maximization of Mutual Information,â IEEE transactions on.
Multimodal and Multiband Image Registration using Mutual Information Christoph Strecha1 and Rik Fransens1 Luc Van Gool1 Visics KU Leuven, Belgium {Christoph.Strecha, Rik.Fransens, Luc.VanGool}@esat.kuleuven.ac.be
1
Introduction
In this paper, we present a novel histogram based method for estimating and maximising mutual information (MI) between two multimodal and possibly multibanded signals. Histogram based estimation methods are a common means for estimating the MI between two signals and the derivative of MI with respect to these signals. However, these approaches do not scale well towards higher dimensions of the signals involved. This is due to the exponential explosion of the number of bins needed to accurately estimate the marginal and joint densities. We introduce a new estimation method which relies on the combination of non-uniform quantisation of the signal spaces and kernel density estimation to deal with this problem. Furthermore, we show how existing 1D-1D methods can be improved by using a combination of weighted histogram updates and kernel convolutions. These convolutions can be computed efficiently in the frequency domain which reduces the computational overhead significantly. The weighting scheme, on the other hand, enables us to compute analytical derivatives of MI with respect to either of both signals, which is important for further optimisation purposes. We illustrate our approach with several applications in parametric and non-parametric multimodal image registration. Our case study is the registration of multiband aerial images. More particularly, we demonstrate how optimisation of MI can succesfully allign intensity, infra-red, natural color and pseudo-color images in the cse of parametric (affine) and non-parametric (optical flow) transformations. To define the MI between two images I1 and I2 , we regard them as random variables X and Y and their intensity values at a certain coordinate in the images as the joint outcome of a random experiment. The MI between X and Y , denoted as I(X; Y ), is a measure for the statistical dependency between both variables. It attains its maximal value if there exists a bijective relationship (i.e. a reversible 1:1 mapping) between the values of X and Y . On the other extreme, I(X; Y ) drops to zero if X and Y are statistically independent. This paper mainly focusses on the derivation of a robust, derivable estimator for MI. The two principal methods for estimating MI are histogram based methods ([1], [2]) and kernel density estimation (KDE) approaches ([3], [4]). First, we show how these can be combined, to obtain a fast convolution based estimator, both for MI and its derivatives. Although kernel convolutions are sometimes
used to smoothen marginal and joint histograms, this often happens in a rather ad hoc manner. The link with KDE establishes a sound theoretical foundation for estimating kernel properties such as size and orientation, and shows how these properties can be dynamically updated in the course of the registration process. Next, we show how the approach can be extended to higher dimensional (’multi-banded’) signals. Though, in principle, it is possible to extend the previous approach towards higher dimensions, this would be very impractical. The main reason is the large number of bins needed to accurately estimate the marginal and joint densities. We employ a combination of non-uniform vector quantisation of the signal spaces and KDE to tackle this problem. In the procedure, more bins are assigned to regions of high probability, while less bins are assigned to regions of low probability. This results in an accurate density estimation and simultaneously keeps the CPU and memory load under control. We apply our algorithms to the parameteric and non-parametric registration of aerial images, where multimodality arises from the different spectral sensitivities of the sensors. With several examples, we will demonstrate how maximisation of MI can accurately register near infra-red (NIR), pseudo color, natural color and intensity images of the same scene. In the parametric case, we have restricted ourselves to affine transformations, but the approach can be easily extended to more advanced transformation models, which e.g. take camera calibrations and/or geodesics into account.
2
Experiments
The first experiment is an affine registration of a natural color image (I1 ) with a pseudo-color near-infrared (NIR) image (I2 )(fig.1). The second experiment is an affine registration of a pseudo-color NIR image (I1 ) and a natural intensity image (I2 )(fig.2). The third experiment is a non-parametric registration of a single-layer NIR image (I1 ) and a natural intensity image (I2 ), depicting an airport scene. The size of the images is 469 x 730, and there are small displacements between both images which cannot be modelled by an affine transformation. The results of the registration experiment are shown in figure 3.
References 1. F. Maes, A. Collignon, D. Vandermeulen, G. Marchal, P. Suetens, “Multimodality Image Registration by Maximization of Mutual Information,“ IEEE transactions on Medical Imaging, Vol. 16(2), pp. 187-198, 1997. 2. R. Moddemeijer, “On Estimation of Entropy and Mutual Information of Continuous Distributions,” Signal Processing, Vol. 16, pp. 233-248, 1989. 3. P. A. Viola, “Alignment by Maximization of Mutual Information,” Phd thesis, M.I.T. A.I. Technical Report No. 1548, 1995. 4. P. A. Viola, W. M. Wells, “Alignment by Maximization of Mutual Information,” International Journal of Computer Vision, Vol. 24(2), pp. 137-154, 1997.
Fig. 1. Affine registration of a natural-color image I1 and a pseudo-color NIR image I2 . The first row shows both images and a checkerboard overlay of their initial positions. The disparity is mainly due to rotation and translation. The second row shows an overlay of I1 and I2 ◦F after registration, and 2 detail views of this registration.
Fig. 2. Affine registration of a pseudo-color NIR image (I1 ) and a visible intensity image (I2 ). The first row shows both images and a checkerboard overlay of their initial positions. The second row shows an overlay of I1 and I2 ◦F after registration, and 2 detail views of this registration.
Fig. 3. Non-parametric or ’fluid’ registration of a NIR image (I1 ) and a visible intensity image (I2 ). The first row shows both images and an overlay of their initial positions. The second row shows detail views of the overlay of I1 and I2 ◦ F before and after registration. The first 2 images of the second row show the allignment of a detail of the stadion. The 3th and 4th image show the allignment of a detail of the airstrip.
