A Fast B-Spline Pseudo-inversion Algorithm for Consistent Image ...

A Fast B-Spline Pseudo-inversion Algorithm for Consistent Image Registration Antonio Trist´an and Juan Ignacio Arribas Laboratorio de Procesado de Imagen Universidad de Valladolid. Spain [email protected],[email protected] http://www.lpi.tel.uva.es

Abstract. Recently, the concept of consistent image registration has been introduced to refer to a set of algorithms that estimate both the direct and inverse deformation together, that is, they exchange the roles of the target and the scene images alternatively; it has been demonstrated that this technique improves the registration accuracy, and that the biological significance of the obtained deformations is also improved. When dealing with free form deformations, the inversion of the transformations obtained becomes computationally intensive. In this paper, we suggest the parametrization of such deformations by means of a cubic B-spline, and its approximated inversion using a highly efficient algorithm. The results show that the consistency constraint notably improves the registration accuracy, especially in cases of a heavy initial misregistration, with very little computational overload. Keywords: Free form image registration, consistent registration, B-splines, inverse transformation.

1

Introduction

Elastic image registration is an important and challenging issue in medical imaging, and it has been successfully applied to a number of problems of great interest over the recent years [1]. The anatomy being imaged involves quite often deformable tissues, such as the breast [2,3], the brain [4], the kidney, and so on [1], and there lies the importance of fully deformable models for image registration. The idea of consistent image registration has been recently introduced in [5], and further explored in [6], although the convenience of this technique has been pointed out by an increasing number of authors [4,7,8,9,10]. The idea is that when one has to register a pair of images, that is, to deform a model to match a given scene, it is useful to estimate as well the inverse transformation, that is, the one that matches the scene to the model, constraining both transformations to be the inverse of each other. This helps to reduce the presence of local minima, usually yielding more biologically meaningful results [5,6,7], and even more accurate and robust estimates of the deformation [9,10]. This is especially well suited for elastic registration, where local minima may become especially problematic. Consistent W.G. Kropatsch, M. Kampel, and A. Hanbury (Eds.): CAIP 2007, LNCS 4673, pp. 768–775, 2007. c Springer-Verlag Berlin Heidelberg 2007 

A Fast B-Spline Pseudo-inversion Algorithm

769

image registration is then an interesting issue in medical imaging, where elastic registration becomes more useful each day. Although it is a relatively recent idea, a number of algorithms have been developed considering this technique. The common way to impose the consistent constraint has consisted in adding a penalising term to the cost function [4,6,10]. In [9], authors propose a physical deformation model in which the forces that deform the image are symmetrised following the action and reaction principle. The method here proposed is different in the sense that it requires the effective inversion of the geometric transformation obtained; similar approaches can be found in [5,7], where authors iteratively invert the deformation at each point, or in [8], where direct and inverse optical flow are rectified in each iteration with a linear estimate of the inverse transformation. The use of a penalising term is well suited for variational approaches [6,10], but the effective inversion of the transformation may be a better choice in non-iterative algorithms. However, this inversion is computationally intensive with elastic registration, as is the case in [5], where a fixed point technique must be performed at each pixel. The main contribution of our paper is the development of a technique that provides a very efficient way to impose the consistent constraint using a parametric elastic transformation (a cubic B-spline, BS); to the best of our knowledge, it is the first attempt to apply the direct inversion methodology with a parametric deformation. Although our algorithm does not result in an exact inversion of the deformation, but in a pseudo-inversion instead, experiments demonstrate that it substantially improves the registration accuracy, with a fair computational overload. In section 2 we briefly introduce the use of B-splines as interpolators. Section 3 is devoted to the pseudo-inversion algorithm. In section 4 we show some results that demonstrate the usefulness of the algorithm, and finally, in section 5, we give some final considerations and conclude.

2

B-Spline Representation of the Deformation

The use of BS in image registration has been widely validated and widespread [2,3], being their main advantage their very little computational load. Compared to the classical Thin Plate Splines (TPS, [11]), they show the drawback of its worse regularity properties; however, the hierarchical interpolation technique presented in [12] overcomes in practice this difficulty. Here, we are going to center our attention in the two-dimensional case, although the extension to 3-D is immediate. Let J be an image with a compact support Ω ≡ [0, X]×[0, Y ] ⊂ R2 ; the deformation at a point (x, y) ∈ Ω is then given by the tensor product:  3 3    x + k=0 l=0 Bk (s) Bl (t) Φx(i+k)(j+l) x 3 3 , (1) = y y + k=0 l=0 Bk (s) Bl (t) Φy(i+k)(j+l) where i = M x/X − 1, j = N y/Y  − 1, s = M x/X − M x/X, t = N y/Y − N y/Y 1 , and Φx,y ij are the interpolation parameters to compute, one for each 1

Here x represents the greatest integer minor than x.

770

A. Trist´ an and J.I. Arribas

and every of the 2 × (M + 3) × (N + 3) grid nodes (note that there is a grid for each dimension), with M + 3 and N + 3 the number of nodes in each dimension, being verified that −1 ≤ i ≤ M + 1, −1 ≤ j ≤ N + 1, and 0 ≤ s, t < 1. Bi (·), i = 0, 1, 2, 3 are the BS kernels, which are third degree polynomials [12], in such a way that BS interpolators are piecewise polynomials, with the nice property that they are second order differentiable. Besides, they have another interesting property, their locality: a change in one of the spline coefficients Φx,y ij affects only to a vicinity of the corresponding node (i, j). In practice, this means that to interpolate a new control point (x, y) ∈ Ω, only a few (16, in the 2D case) grid nodes have to be changed. This property is used in [12] to hierarchically interpolate the control points of the deformation in a least square sense. This is the same approach followed in [2], and in our own work as well.

3

Pseudo-inversion Algorithm

A BS does not need to be invertible, and in case it is, its inverse transformation is not necessarily another BS. To impose the consistent constraint, we define the pseudo-inverse BS Ψ of a given BS Φ as a BS with the same support and grid nodes as Φ; we use these grid nodes as control points (the values to interpolate) and then we force the displacements at the grid points of Ψ to equal exactly the inverse deformation of that given by Φ at these points. This means that the inversion is exact at the grid nodes, and only approximated elsewhere. To obtain the inverse transformation at the grid nodes, we follow an approach similar to [5], which in fact is a variant of the fixed point method. From eq. (1), the deformation at a point x ≡ (x, y) ∈ Ω can be written in a compact form: x = g (x) = x + f (x) .

(2)

For a given point x0 ∈ Ω, finding the value of the inverse transform g−1 (x0 ) implies to find the point xt ∈ Ω  such that g(xt ) = xt + f (xt ) = x0 , and then it will be verified that g−1 (x0 ) = x0 + f −1 (x0 ) = xt , where in this case f −1 is not the inverse of f but simply a convenient and comfortable notation. Thus, for each grid node x0 , we are interested in finding the point xt such that: x0 − f (xt ) = xt ⇔ hx0 (xt ) = xt ,

(3)

what reduces to find a fixed point in function hx0 , defined for each grid node. To do so, starting from the initial approximation x0t , the updating rule is: n

= hx0 (xnt ) = x0 − f (xnt ) ⇔ f −1 (x0 ) = −f (xnt ) . xn+1 t ∞

(4)

If series {xnt }n=0 converges toward a non infinite value, this value is xt . Unlike in [5], (4) is only used at the grid nodes; at the same time, the successive evaluations are exact here due to the continuum nature of BS, while in [5] interpolation is needed, thus convergence here will be faster.

A Fast B-Spline Pseudo-inversion Algorithm

771

Once the deformation is known at the grid nodes, it only remains to compute the parameters Φ of the BS. At grid nodes s = t = 0, [12], and we can write:    2 2   x x −x k=0 l=0 αk αl Φ(i+k)(j+l) , (5) = 2 2 y y − y k=0 l=0 αk αl Φ(i+k)(j+l) where α2 = α0 = 16 and α1 = 23 are the result of evaluating the BS kernels at 0. Thus, (5) defines 2 linear systems of (M + 3) × (N + 3) equations with (M + 3) × (N + 3) unknowns. Nevertheless, each value of the displacement f −1 (xi , yj ) affects only to nine Φx coefficients and another nine Φy , so the coefficient matrix is highly sparse. The generalisation to three or more dimensions is immediate; in fact, the tensorial character of the interpolation via BS permits that the systems of equations may be written by blocks, as shown in eq. (6): ⎞ ⎞ ⎛ x,y,... ⎞ ⎛ −1 fx,y,... (x−1 ) Φ−1 α1 Λ α2 Λ 0 0 ··· 0 0 ⎜ x,y,... ⎟ ⎜ −1 (x0 ) ⎟ ⎜ α0 Λ α1 Λ α2 Λ 0 · · · 0 0 ⎟ ⎜ ⎟ ⎟ ⎜ Φ0x,y,... ⎟ ⎜ fx,y,... −1 ⎜ 0 α0 Λ α1 Λ α2 Λ · · · 0 ⎟ ⎜ fx,y,... ⎟ 0 ⎟⎜ (x1 ) ⎟ ⎜ Φ1 ⎜ ⎟=⎜ ⎟, ⎜ .. ⎟ . .. ⎟ ⎜ .. ⎟ ⎜ .. .. .. .. .. ⎝ . ⎠ . .. . . . ⎠⎝ . ⎠ ⎝ . . −1 Φx,y,... 0 0 0 0 · · · α0 Λ α1 Λ fx,y,... (xM+1 ) M+1 ⎛

(6)

where the vector unknowns Φ and the values of displacements f −1 (xi ) are the result of fixing the first of the indices of the grid, i, and the d-dimensional matrices have been arranged like column vectors. By the own nature of the BS [12], the extreme nodes in each dimension (i = −1, i = M + 1, for x coordinate) correspond to points not included in Ω, so its inversion is not done following (4). Instead, we make f −1 (x−1 ) = f −1 (x0 ) and f −1 (xM+1 ) = f −1 (xM ). Besides, matrix Λ must present the same block-tridiagonal structure, implying that the value of the parameter at y is only affected by the value of displacement at the same y and at the next adjoining two, so that the system can be solved in a recursive way. Starting with the first dimension we may write, for both two systems: α0 ΛΦn−1 + α1 ΛΦn + α2 ΛΦn+1 = fn−1 , −1 < n < M + 1,

(7)

where we have adopted the notation fn−1 = f −1 (xn ) and extended the n subindex to the boundary values Φ−2 and ΦM+2 . We aim to find a solution in the form: Φn+1 = Σn Φn + θn , −2 ≤ n ≤ M,

(8)

and substituting (8) into (7), one obtains: α0 ΛΦn−1 + α1 ΛΦn + α2 Λ (Σn Φn + θn ) = fn−1 ⇒ α0 α1 1 −1 −1 Φn−1 + Φn + Σ n Φn + θ n = Λ fn ⇒ α2 α2 α2     1 −1 −1 α1 ID + Σn Φn = −Φn−1 + Λ fn − θn . α2 α2

(9)

772

A. Trist´ an and J.I. Arribas

Identifying terms in (9), the following recursion is obtained:  Σn−1 = −  θn−1 =

α1 ID + Σn α2 α1 ID + Σn α2

−1 , −1 < n ≤ M + 1 −1 

 1 −1 −1 Λ fn − θn , −1 < n ≤ M + 1, (10) α2

At the same time, if ΣM+1 can be written like γ · Id , it is easily proved by induction that Σn = γn · Id , and thus the system solution follows: Φn = γn−1 Φn−1 + θn−1 , −1 ≤ n ≤ M + 1 1 γn−1 = − α1 , −1 ≤ n ≤ M + 1 α2 + γn   1 −1 −1 Λ fn − θn , −1 ≤ n ≤ M + 1. θn−1 = −γn−1 α2

(11)

Matrix Λ is again block-tridiagonal, hence the calculus of Λ−1 fn−1 may be done by applying the algorithm in a recurrent fashion to the successive dimensions until reaching the last one, in which Λ will simply be 1. Regarding the first term in the recursion, we have experimentally proved that the best choice is to force ΦM+2 = ΦM+1 , and thus we set γM+1 = 1 and θM+1 = 0.

4

Results

To validate the proposed technique, we have chosen a simple and easy to implement registration algorithm, based on the work by Xiao et. al. [3]. It is a classical block-matching algorithm, where a grid (note that this grid is different to that of the BS interpolator) is superimposed to the images to register; then a block of pixels is placed at each node and moved to the surrounding positions to find the corresponding block in the other image that best match it. The quality of the match is measured in [3] as the linear correlation between the blocks, but we use Mutual Information (MI, [13]) instead, since this allows us to perform multimodal registration. As in [3] a Bayesian framework is used to smooth the estimated displacements at each node. The last step in [3] is the interpolation of the displacements by a gradient-descent based optimization of the BS coefficients. We use the technique suggested in [12] instead, since it is more efficient and it has been successfully tested [2]. Finally, we swap the roles of each image in the block-matching step, and so we can estimate the inverse BS transformation; then we use the pseudo-inversion algorithm and average the obtained BS coefficients with the coefficients estimated for the direct BS transformation. This procedure is enclosed in a multirresolution framework, which is known to result in more robust and accurate estimates [1]2 . 2

We have used 5 resolution levels; for the block matching, block-size was 11 × 11 pixels, and a block was centered at every six pixels. BS grid had size 64 × 64 nodes.

A Fast B-Spline Pseudo-inversion Algorithm

773

Fig. 1. Example of the test deformations. From left to right: original PD slice (with the random displacements). T1 slice deformed with a 3.6 bending energy TPS. Result of matching the deformed T1 slice to the original PD slice. All images have a size of 181 × 217 pixels.

Data used to test the algorithm is that available at the BrainWeb3 [14]. It consist of an MRI phantom volume of the brain, comprising PD, T1, and T2 modalities. The use of synthetic images allows us to use a gold standard: over the starting images (slices taken from the MRI volume), we establish 12 uniformly distributed landmarks, and for each of them a random vector is generated that represents the deformation at that point. The deformation is then interpolated with a TPS [11], so the results are not biased by the type of interpolation. This process may generate deformations of very diverse nature, so we have parametrised the extent of the deformation depending on its bending energy [11]. Besides, the use of known deformations allows the computation of the exact error at each image point as the Euclidean distance between the displacement vector estimated and its real value. An example of the kind of deformations generated, and the registration result, is given in Fig. 1. Results shown comprise first the successful registration rate percentage, where we took the criterion of considering a successful registration whenever the mean registration error is bellow the unit (i. e. the minimum achievable displacement); we have used all possible combinations of MRI images (PD, T1, and T2) and for each successful registration we have computed the mean error and the 90% confidence interval. Results ares shown in Table 1, for a total of 100 experiments with 3.6 bending energy (as in Fig. 1), with and without the consistent constraint. By comparing each pair of cases, it remains evident that the consistent constraint notably improves not only the robustness (the percentage of successful registrations) but the accuracy (mean error) as well. As a final result, we show in Fig. 2 the behaviour of the mean registration error as the deformation becomes larger (in terms of its bending energy), for the PD/T2 case; we have chosen it as the most adverse case, as shown in Table 1, and even so the advantage of the consistent constraint is evident for reasonably large deformations. 3

http://www.bic.mni.mcgill.ca/brainweb

774

A. Trist´ an and J.I. Arribas

Table 1. Registration performance in terms of the successful registration rate, and the mean value and 90% confidence interval of the error of the estimated displacement in pixels, both without and with the consistent constraint Scene Model Without consistent constraint With consistent constraint PD PD 98%(0.55 ± 1.13) 100%(0.51 ± 1.04) PD T1 83%(0.78 ± 1.68) 91%(0.63 ± 1.50) PD T2 86%(0.77 ± 1.63) 95%(0.78 ± 1.57) T1 PD 80%(0.82 ± 1.81) 90%(0.75 ± 1.65) T1 T1 97%(0.61 ± 1.31) 99%(0.56 ± 1.18) T1 T2 84%(0.82 ± 1.82) 90%(0.75 ± 1.67) T2 PD 83%(0.79 ± 1.66) 95%(0.70 ± 1.52) T2 T1 84%(0.79 ± 1.71) 92%(0.73 ± 1.61) T2 T2 99%(0.55 ± 1.14) 99%(0.51 ± 1.05)

Mean registration error vs. bending energy of the deformation (PD/T2) 2 Without the consistent constraint

Mean registration error in pixels

With the consistent constraint 1.75

1.5

Bending energy = 3.6 1.25

1

0.75

0.5 0

1

2

3

4

5

6

7

8

9

10

11

12

Bending energy

Fig. 2. Mean registration error versus bending energy of the deformation to recover, with and without the consistent constraint. Curves are a third degree polynomial fit of 400 randomly generated samples of the PD/T2 case.

5

Conclusion

BS are able to represent very complex and large deformations, with little computational load and nice smoothness properties when combined with hierarchical interpolation [12]; all these facts make them an interesting choice in elastic image registration [2,3]. We have presented an efficient algorithm to find a BS transformation that approximates the inverse transformation of another given BS, which allows us to introduce the concept of consistent image registration and all its benefits in the algorithms that make use of BS interpolation. Compared to [5], we have to invert exactly the transformation at only the 2 × M × N BS grid nodes, instead of at all image pixels, drastically reducing the

A Fast B-Spline Pseudo-inversion Algorithm

775

computational effort. Once the transformation is inverted at these points, the algorithm presented in section 3 is able to very efficiently compute the BS parameters. The main drawback of this technique is that the inversion is exact for the grid nodes, but only approximated elsewhere. However, the results demonstrate that it is accurate enough for our purposes, and they confirm the hypothesis that the consistent constraint produces more accurate and robust estimates of the deformation, as it has been previously suggested [9,10]. Acknowledgments. Authors want to thank Dr. R. C´ ardenes and Dr. J. CidSueiro for their comments. This work was supported by grant numbers TEC0406647-C03-01, from the Comisi´ on Interministerial de Ciencia y Tecnolog´ıa, Spain, and FP6-507609 SIMILAR Network of Excellence from the European Union.

References 1. Zitova, B., Flusser, J.: Image registration methods: a survey. Image and Vision Computing 21(11), 977–1000 (2003) 2. Rueckert, D., Sonoda, L., Hayes, C., Hill, D., Leach, M., Hawkes, D.: Nonrigid registration using free-form deformations: application to breast MR images. IEEE Trans. on Medical Imaging 18(8), 712–721 (1999) 3. Xiao, G., Brady, J., Noble, J., Burcher, M., English, R.: Nonrigid registration of 3-D free-hand ultrasound images of the breast. IEEE Trans. on Medical Imaging 21(4), 405–412 (2002) 4. Shen, D., Davatzikos, C.: HAMMER: hierarchical attribute matching mechanism for elastic registration. IEEE Trans. on Medical Imaging 21(11), 1421–1439 (2002) 5. Christensen, G., Johnson, H.: Consistent image registration. IEEE Trans. on Medical Imaging 20(7), 568–582 (2001) 6. Johnson, H., Christensen, G.: Consistent landmark and intensity-based image registration. IEEE Trans. on Medical Imaging 21(5), 450–461 (2002) 7. Cachier, P., Rey, D.: Symmetrization of the non-rigid registration problem using inversion-invariant energies: application to multiple sclerosis. In: Medical Image Computing and Computer-Assisted Intervention - MICCAI’2000, pp. 472–481 (2000) 8. Thirion, J.P.: Image matching as a diffusion process: an analogy with Maxwell’s demons. Medical Image Analysis 2(3), 243–260 (1998) 9. Rogelj, P., Kovacic, S.: Symmetric image registration. Medical Image Analysis 10(3), 484–493 (2006) 10. Zhang, Z., Jiang, Y., Tsui, H.: Consistent multi-modal non-rigid registration based on a variational approach. Pattern Recognition Letters 27(7), 715–725 (2006) 11. Bookstein, F.: Principal warps: thin-plate splines and the decomposition of deformations. IEEE Trans. on Pattern Anal. and Machine Intell. 11(6), 567–585 (1989) 12. Lee, S., Wolberg, G., Shin, S.: Scattered data interpolation with multilevel Bsplines. IEEE Trans. on Visualization and Computer Graphics 3(3), 228–244 (1997) 13. Viola, P., Wells, W.M., I.: Alignment by maximization of mutual information. In: Proc. of Fifth International Conference on Computer Vision, 1995, pp. 16–23 (1995) 14. Kwan, R.S., Evans, A., Pike, G.: MRI simulation-based evaluation of imageprocessing and classification methods. IEEE Trans. on Medical Imaging 18(11), 1085–1097 (1999)