Image Registration Guided by Particle Filter

Image Registration Guided by Particle Filter Edgar R. Arce-Santana ⋆ , Daniel U. Campos-Delgado, and Alfonso Alba

⋆⋆

Facultad de Ciencias, Diagonal Sur S/N, Zona Universitaria, C.P. 78290, San Luis Potosi, S. L. P., Mexico.

Abstract. Image Registration is a central task to different applications, such as medical image analysis, stereo computer vision, and optical flow estimation. One way to solve this problem consists in using Bayesian Estimation theory. Under this approach, this work introduces a new alternative, based on Particle Filters, which have been previously used to estimate the states of dynamic systems. For this work, we have adapted the Particle Filter to carry out the registration of unimodal and multimodal images, and performed a series of preliminary tests, where the proposed method has proved to be efficient, robust, and easy to implement.

1

Introduction

The goal of Image Registration is to find an optimal geometric transformation between corresponding image data [1],[2], where the criterion for optimality depends on a specific application. This task is very important to many applications involving image processing or analysis such as medical-evaluation, biomedical systems, image guidance, depth estimation, and optical flow. In the past 10 years, many methods have been published; an extensive and comprehensive survey can be found in [3],[4]. Image Registration Methods can be classified as global or local. In the global approaches one searches a model, often a parametric one such as rigid, affine, projective, or curved transformation, explaining the similarities between images. Local methods or dense registration seek individual correspondences for each pixel in both images. A special kind of registration is called Multimodal Image Registration, in which two or more images coming from different sources are aligned; this process is very useful, for example, in computer aided visualization in the medical field, since it allows one to find correspondences between functional and anatomical images. In the literature, the more common global methods are those based on intensity changes [5],[6],[7]. Within these methods, there is a very popular one based on Mutual Information (MI), proposed independently by Viola and Wells ⋆

⋆⋆

The author was supported by Grant PROMEP/103.5/04/1387 and Fac. de Ciencias, UASLP, Mexico. The author was supported by Grant PROMEP/103.5/09/2416 and Fac. de Ciencias, UASLP, Mexico.

2

Edgar R. Arce-Santana, Daniel U. Campos-Delgado, and Alfonso Alba

[8],[9] and Collignon and Maes [10],[11]. Even though it is theoretically robust, it requires to find the maximum mutual information, for example, using optimization techniques such as hill-climbing, making the methods very sensitive to its initial parameters and the derivative calculations. Other related work is found in [12], where an affine transformation between images is modeled locally using a linear approximation (first order Taylor’s expansion), but due to this approximation, the algorithm must use a differential multiscale framework in order to approximate large geometric displacements with a series of shorter ones. Other global approaches that have demonstrated to be very efficient to estimate affine transformation parameters are those based on Bayesian Estimation. Generally, in these methods, Image Registration is solved by estimating a statistic, for example the mean or mode, from a cost function, given a posterior probability [13],[14],[15]. A commonly used estimator is the maximum a posteriori (MAP), which can be found by minimizing an energy function; unfortunately, these functions are highly nonlinear with respect to the geometrical parameters, requiring complex optimization methods and demanding considerable computing time. Under the same Bayesian approach, in the present work, we propose a new algorithm based on a method that has been used to estimate states of nonlinear dynamics systems, known as Particle Filter [16],[17]. This method has been adapted to solve the problem of image registration, showing that the methodology behind Particle Filter can be used in a very efficient way to solve these kinds of image processing tasks. The paper is organized as follows: Section 2 describes the basis of the Particle Filter; in Section 3, we describe how the filter is adapted to the image registration problem; Section 4 shows some experiments and results; and finally, in Section 5, some conclusions are presented.

2

Particle Filter Basis

Particle Filter is a method based on Bayesian Estimation that uses a Monte Carlo algorithm to estimate probability density statistics [16],[17]. The key idea is to obtain a posterior density from a set of random samples with associated weights, which allows one to estimate state variables defining a dynamic system. In order to establish the problem, consider the evolution of a state sequence, given by sk = fk (sk−1, vk−1 ), (1) where fk (.) is a function, possibly nonlinear, of the system state vector sk−1 , at time k-1, and vk−1 is independent and identically distributed (iid) noise. Thus, the goal is recursively to estimate the state vectors sk from the measurements zk = hk (sk , nk ),

(2)

where hk (.) is a possibly nonlinear function, and nk are iid noise samples. In particular, we want to estimate the distribution p(sk |z1:k ) from a set of measurements z1:k = {zi , i = 1, ..., k}. Suppose that we have access to the pdf

Lecture Notes in Computer Science

3

p(sk−1 |z1:k−1 ) at time k-1. The method consists in a prediction stage to approximate the pdf of the state system at time k, via the Chapman-Kolmogorov equation Z p(sk |z1:k−1 ) =

p(sk |sk−1 )p(sk−1 |z1:k−1 )dsk−1 .

(3)

Notice that, we have used the fact that p(sk |sk−1 , z1:k ) = p(sk |sk−1 ), which describes a Markovian process. At time k, a measurement zk is available, and using the Bayes’ theorem, we can predict (update stage) the posterior density p(sk |z1:k ) =

p(zk |sk )p(sk |z1:k−1 ) , p(zk |z1:k−1 )

(4)

where the normalizing constant is p(zk |z1:k−1 ) =

Z

p(zk |sk )p(sk |z1:k−1 )dsk .

(5)

The equations (3) and (4) provide a form to estimate the posterior probability p(sk |z1:k ) in a recursive way. This structure, in many cases, cannot be analytically established, particularly when fk (.) and/or hk (.) are nonlinear. One way to solve the problem is applying Particle Filter. In this method, the distribution p(sk |z1:k ) is represented by samples (particles) {(sik , wki ) : i = 1, ..., Ns }, where i sik are Pthei particles’ values, at time k, and wk are the associated weights such that i wk = 1. The Particle Filter algorithm is by nature iterative, and is composed of two stages: A) Prediction stage, in which each of the state variables (particles) is modified according to the equation (1); and B) Update stage, where the particles’ weights are recalculated using the measurements information described by equation (2). It is convenient to mention that it is common to take p(sk |s1:k−1 ) = p(sk |sk−1 ), which means that the actual state variables depend on their previous values (1st order Markov process). Also the likelihood function p(zk |s1:k ) = p(zk |sk ) depends only on the latest measurement. In the prediction stage, particles at time k-1 are propagated to generate new particles at time k. A common drawback in this process is a phenomenon known as the Degeneracy Problem [18], where after a few iterations, all but one particle will have negligible weights. One way to resolve this problem is by resampling, in order to eliminate particles having small weights and concentrate in particles having large contribution: the particles that represent p(sk−1 |zk−1 ) at time k-1 are used to obtain new particles (resampling); next, the state of each of these particles are modified according to (1)(prediction stage); finally, their weights wki are updated using the likelihood function p(zk |sk ) in order to obtain representative samples of p(sk |zk ).

3

Image Registration Guided by Particle Filter

In the present work, we adapted the Particle Filter algorithm to solve the image registration problem between two images I1 and I2 , which are related by the

4


observation model I1 (x, y) = F (I2 (T (x, y))), where F (.) is an intensity transfer function, depending on the gray values of I2 , and T (.) is the affine transformation:   λx cos θ − sin θ 0 T ([x, y]) = [x, y, 1]  sin θ λy cos θ 0  . (6) dx dy 1

In this equation, θ represents the rotation angle of one image with respect to the other one, λx is the scale factor on the x-axis, λy on the y-axis, dx is the translation on x, and dy on y-axis. To carry out the registration, we assume as state variables the vector (sik ) = i T θk , λx ik , λy ik , dx ik , dy ik , the geometric transformation parameters in (6); and define the state equation, which in this case is a simple random walk:  i   i    θk−1 θk vθ  λx ik   λx ik−1   vλx   i  i     λy  =  λy    (7)  ik   ik−1  +  vλy  .  dx   dx   vdx  k k−1 vdy dy ik dy ik−1

Notice that the prediction equation is very simple: it is the preceding value T disturbed by the noise vector vθ , vλx , vλy , vdx , vdy . Here we assumed Gaussian and independent noise for each parameter with zero mean and standard deviations σθ , σλ , σd , respectively. In the update stage, we want to know how well each new particle value sik fits the observation model F (I2 (T (x, y))) = I1 (x, y) + γ(x, y),

(8)

where γ(x, y) is iid noise, with zero mean and standard deviation σγ . In order to accelerate the process, we only consider a set of m uniformly distributed pixels P = {(xi , yi ); i = 1...m}, in the images I1 , and I2 , so that the likelihood function may be expressed as Y p(zk |sik ) = p(I1 (x, y), F (I2 (T (x, y)))|sik ). (9) (x,y)∈P

Notice that the affine transformation is applied to the coordinates of one of the images, meaning that the observation model is highly nonlinear with respect to the transformation parameters. Next, we detail the image registration algorithm guided by the Particle Filter. 3.1

Algorithm: Image Registration guided by Particle Filter

i Given a set of particles at time k-1, {(sik−1 , wk−1 ) : i = 1, ..., Ns }, where the state values are given by the transformation parameters h iT i sik−1 = θk−1 , λx ik−1 , λy ik−1 , dx ik−1 , dy ik−1 (10)


5

1. For each particle, compute the cumulative probability as c0k−1 = 0 cik−1

=

(11)

ci−1 k−1

+

i , for wk−1

i = 1, ..., Ns .

(12)

2. For each particle sik−1 , do the resampling as follows: – Generate a uniform random value u ∈ [0, 1]. – Find the smallest j such that cjk−1 ≥ u. – Select the state sbik−1 = sjk−1 . 3. Obtain the new set of samples at time k (Prediction stage), using the equation: sik = sbik−1 + vk . 4. For each new state sik , compute the corresponding weights wki = p(zk |sik ); that is the likelihood function (Update stage): Y (13) p(I1 (x, y), I2 (T (x, y))|sik ). p(zk |sik ) = (x,y)∈P

P 5. Normalize the weight, such that i wki = 1. 6. Once the particles’ weights have been computed, we may evaluate the mean for each affine transformation parameter to compute the estimations: PNs i i PNs i i w λ , w θ , E[λxk |zk ] = i=1 E[θk |zk ] = i=1 PNs k i k i PNs k i k i wk dxk , E[λy k |zk ] = i=1 wk λk , E[dx k |zk ] = i=1 PNs i i E[dy k |zk ] = i=1 wk dyk .

4

(14)

Experiments and Results

In this section, we present two different kinds of results. The first one corresponds to an observation model which tries to match similar gray values in order to carry out the image registration. The model used in the experiments was I2 (T (x, y)) = I1 (x, y) + γ(x, y).

(15)

In this case, the function F (.) was the identity. In order to avoid numerical instability, due to the likelihood p(zk |sik ) being computed as the product of the individual measurement of pixels in the set P , it is appropriated to choose a robust γ function [19], having the maximum at the mean, and tails quickly reaching nonzero values. A function that satisfies both requirements is the following one: γ(x, y) =

1 1+

|I1 (x,y)−I2 (T (x,y))| σγ2

(16)

where σγ2 is a parameter that depends on the standard deviation of the noise. Using (15) and (16), the following experiment consisted on registering 256 × 256 pixel images. For the experiment, we used as initial parameter values the

6


identity (θ = 0, λx = 1, λy = 1, dx = 0, dy = 0), 100 particles, and 256 pixels for computing the likelihood. The image to register was obtained by applying an artificial affine transformation with θ = −45, λx = 0.6, λy = 0.6, dx = 10, and dy = 15. The estimated parameters, after 5 seconds, were θ = −44.9276, λx = 0.59821, λy = 0.59822, dx = 9.937, and dy = 15.015; Figure 1 shows the obtained registered image.

Fig. 1. Left: reference image; middle: image to register; right: registered image I2 .

Next, we tested the robustness of the algorithm with respect to noise. We added normal random values, with standard deviations from 0 to 20, to the image I2 in Fig. 1, and used the same initial values that the previous experiment. The Box-Plot of the True Relative Mean Error (TRME) of the affine parameters are shown in panel A) of Fig. 2; the mean and standard deviation are shown in panels B) and C). This error function has the advantage of independently taking into account the unit scales of the quantities to evaluate, and it is computed as follows: P5 s∗ (k)−s(k) | k=1 | s∗ (k) T RM E = , (17) 5 where s∗ (k) corresponds to the k-th true parameter value, and s(k) to the estimated one by the algorithm. Notice that, no matter the noise level, the median and mean are between 4% and 7%, and the standard deviation is between 1% and 4%. In the second set of experiments, we used as measure model the Mutual Information (MI) defined as I(I1 , I2 ) = H(I1 ) + H(I2 ) − H(I1 , I2 ),

(18)

where H(I1 ) is the entropy of the gray value distribution of the image I1 , and H(I1 , I2 ) is the joint entropy of the gray value distribution of the images I1


7

Fig. 2. A)TRME-Box-Plot; B)TRME-Mean; C) TRME-SD, for noise standard deviation from (0,...,20).

and I2 . This is a more general measure, since it allows one to register Multimodal images, coming from different sensor types. An example of such images is in panels A) and B) of Fig. 3, where the first one is a CT-Image and the other one is a RM-Image. This measure is more general in the sense that if I2 is the result of a 1-1 intensity function mapping F (I1 ), then I(I1 , I2 ) = I(I1 , F (I1 )) = I(I1 , I1 ) = H(I1 ). Also, we know that this measure is bounded: 0 ≤ I(I1 , I2 ) ≤ min(I(I1 , I1 ), I(I2 , I2 )). Hence, we may define as likelihood function 1 (H(I1 ) − I(I1 , T (I2 )))2 γ(H(I1 ), I(I1 , T (I2 )); σ) = √ exp − , (19) 2σ 2 2πσ for a given σ, and an affine transformation T (.), since we have defined as reference image I1 . Note that (19) has a gaussian shape and reaches its maximum when I1 (x, y) = F (I2 (T (x, y))). In Figure 3.C), we used a checker-board in order to appreciate the registration details, in which each square corresponds, alternatively, to the image in A) and to the registered image in B). Multi-modal image registration is very important in the medical field, since, for example, it allows one to find correspondences between functional and anatomical images, or align a set of MR-slices to reconstruct a 3D brain volume. In this work, we present an example of 3D volume reconstruction in which we took 27 axial MR-slices, each one of 128 × 128 pixels. For each one of these slices, we applied an artificial relative geometrical transformation with respect to the previous one. For the translation parameters, random values were generated be-

8


Fig. 3. Left: CT-image; center: MR-image; right: CT and MR registration.

tween -10 to 10 pixels, and for the rotation between -10 to 10 degrees; panel A) in Fig. 4 shows the volume obtained from these transformed slices. In order to realign the images, the affine transformation between two consecutive images Ii and Ij was found, and it was propagated from Ij until the last image; the process was repeated from the slice I2 to I27 . Panel B) in Fig. 4 shows the 3D reconstructed volume. To verify numerically how well the displacement and ro-

Fig. 4. A) Non-aligned volume; B) Aligned volume.

tation transformation were estimated, we compute the squared error for each parameter at each pair of adjacent slices. Figure 5 shows the histograms of these estimations. Notice that the maximum translation error on the x-axis is about 1.6 pixels, that corresponds only to one slice; on the y-axis the maximum error is less than 0.5 pixels; and only one slice had a rotation error of 4.5 degrees, the


9

rest had less than 2 degrees. We found that these maximum errors correspond to the last slices, from bottom to top in Fig. 4, this is because to the whole volume is composed only by 27 slices from which the last ones are the less similar (top of head), and thus are more difficult to align. Finally, Table 1 shows the mean and

Fig. 5. Error histograms: A) displacement on x-axis; B) displacement on y-axis; C) rotation.

standard deviation of the MSE of the estimated geometrical transformations. Table 1. Mean and SD of the MSE Parameter dx dy θ

5

Mean SD 0.1979 0.3923 0.0700 0.0901 0.0105 0.0162

Conclusions and Future Work

In the present work, we described a new algorithm for Image Registration based on Particle Filter method used to estimate states variables of dynamic systems. This method was adapted to carry out affine image registration. Although Particle Filter is by nature stochastic, we presented experiments where the algorithm showed to be very efficient and accurate to estimate geometric transformation parameters. Finally, we showed that the algorithm is easy to implement and robust with respect to noise, and it is possible to include complex likelihood expressions, in contrast to any optimization method which complexity depends on this term. Some immediate perspectives for future research include: a generalization of the proposed methodology for the registration of anatomic and functional 3D brain images, and implementing the algorithm in a parallel computer architecture.

10


References 1. Jan Modersitzki, Numerical Methods for Image Registration, Oxford University Press, 2004. 2. J.V. Hajnal, D.L.G. Hill, D.J. Hawkes, Eds., Medical Image Registration, Boca Raton, FL: CRC Press, 2001. 3. B. Zitova, Jan Flusser, Image Registration Methods: a survey, Image and Vision Computing, Vol. 21, pp. 977-1000, 2003. 4. L.G. Brown, A survey of image registration techniques, ACM Computing Survey, Vol. 24, pp. 326-376, 1992. 5. D.L.G. Hill, D.J. Hawkes, Across-modality registration using intensity-based cost functions , in Handbook of Medical Imaging: Processing and Analysis, I. Bankman, Ed. New York: Academic, pp. 537-553, 2000. 6. E. Ding, T. Kularatna, M. Satter, Volumetric image registration by template matching , Medical Imaging 2000 (Bellinham, WA: SPIE), pp. 1235-1246, 2000. 7. R. Shekhar, V. Zagrodsky, Mutual Information-based rigid and non-rigid registration of ultrasound volumes , IEEE Transactions on Medical Imaging, Vol. 21, pp. 9-22, 2002. 8. P.A. Viola, W.M. Wells III, Alignment by maximization of mutual information, in Proc. 5th Int. Conf. Computer Visiom, Cambridge, MA, pp. 16-23, 1995. 9. W.M. Wells III, P.A. Viola, H. Atsumi, S. Nakajima, R Kikinis, Multi-modal Volumen Registration by Maximization of Mutual Infromation, Medical Image Analysis, Vol. 1, pp. 35-51, 1996. 10. A. Collignon, F. Maes, D. Delaere, D. Vadermeulen, P. Suetens, G. Marchal, Automated multimodality medical image registration using information theory, in Proc. 14th Int. Conf. Process Med. Imag., Y. Bizais, C. Barillot, and R. Di Paola, Eds., Ile de Berder, France, pp. 263-274, 1995. 11. F. Maes, A. Collignon, D. Vadermeulen, G. Marchal, P. Suetens, Multimodality image registration by maximization of mutual information, IEEE-Trans. Med. Image., Vol. 16, no. 2, pp. 187-198, 1997. 12. S. Periaswamya, H. Farid, Medical image registration with partial data, Medical Image Analisys, Vol. 10, no. 3, pp. 452-464, 2006. 13. E.R. Arce-Santana, A. Alba, Image registration using Markov random coefficient and geometric transformation fields, Pattern Recognition, Elsevier, Vol. 42, pp. 1660-1671, 2009. 14. E.R. Arce-Santana, A. Alba, Image Registration Using Markov Random Coefficient Fields, In: V.E. Brimkov, R.P. Barneva, H.A. Hauptman (Eds.), Combinatorial Image Analysis, LNCS Vol. 4958, Springer, Berlin, 2008, 306-317 15. J.L. Marroquin, Edgar Arce, S. Botello, Hidden Markov measure field models for image segmentation, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 25, pp. 1380-1387, 2003. 16. M. Sanjeev Arulampalam, Simon Maskell, Neil Gordon, and Tim Clapp, A Tutorial on Particle Filters for Online Nonlinear/Non-Gaussian Bayesian Tracking, IEEE Transaction on Signal Processing, Vol. 50, pp. 174-188 , 2002. 17. B. Ristic, S. Arulampalam, N. Gordon, Beyond the Kalman Filter: Particle Filter for Tracking Applications, Artech House, 2004. 18. A. Doucet, On sequential Monte Carlo methods for Bayesian filtering, Dept. Eng., Univ. Cambridge, UK, Tech. Rep., 1998. 19. Gerhard Winkler, Image Analysis, Random Fields and Markov Chain Monte Carlo Methods: A Mathematical Introduction, 2nd ed., Springer, New York, 2003. 20. 3D Slicer home page http://www.slicer.org.