improving image registration by correspondence ... - CiteSeerX

IMPROVING IMAGE REGISTRATION BY CORRESPONDENCE INTERPOLATION ´ Hildur Olafsd´ ottir† , Henrik Pedersen‡§ , Michael Sass Hansen , Henrik Larsson‡ , Rasmus Larsen 

Technical University of Denmark, DTU Informatics, Lyngby, Denmark University of Iceland, Faculty of Electrical and Computer Engineering, Reykjavik, Iceland ‡ Glostrup Hospital, Functional Imaging Unit, Copenhagen, Denmark § King’s College London, Division of Imaging Sciences, London, UK  Harvard Medical School, Childrens Hospital, Computational Radiology Laboratory, Boston, MA, USA †

ABSTRACT This paper presents how using a correspondence-based interpolation scheme for 3D image registration improves the registration accuracy. The interpolator takes into account correspondences across slices, which is an advantage, particularly when the volume has thick slices, and where anatomies lie non-parallel to the slice direction. We use our previously presented approach for correspondence-based interpolation and demonstrate results on two different datasets, brain and cardiac MRI. The results are evaluated (i) qualitatively by examination of gradient images and cardiac pig atlases and (ii) quantitatively by registering downsampled brain data using two different interpolators and subsequently applying the deformation fields to the original data. The results show that the interpolator provides better gradient images and a more sharp cardiac atlas. Moreover, it provides better deformation fields on downsampled data, increasing the registration accuracy of original data to 5.8% on average with respect to a standard interpolator. Index Terms— Image interpolation, image registration, image gradient, atlas, magnetic resonance imaging 1. INTRODUCTION In general, segmentation and registration algorithms are highly dependent on the quality and smoothness of image gradients. When dealing with 3D magnetic resonance images (MRI), where slice resolution is typically lower than the in-plane resolution, a good interpolation becomes essential to obtain a high-quality image gradient. The standard linear or spline interpolation assumes that anatomies correspond in a regular grid. When this assumption is violated, the image gradient along the slice direction (z), is poorly estimated, in particular highly non-smooth. Fig. 1 shows this effect. Registration-based (correspondence-based) interpolation aims at improving the correspondence across slices. It is based on 2D registrations between adjacent slices. Hence, rather than assuming smoothness in intensity, the assumption is that the anatomy is consistent across slices. There has

978-1-4244-4128-0/11/$25.00 ©2011 IEEE

1524

(a)

(b)

(c)

Fig. 1. Four-chamber plane of a cardiac MR volume. (a) Nearest-neighbour interpolation, (b) Linear interpolation (c) Absolute z-gradient using linear interpolation. been recent work on correspondence-based interpolation, e.g. [1, 2]. Inspired by those, we demonstrated the importance of taking into account correspondence both ways, providing more robustness in the interpolation where correspondence between slices is poor [3]. The basis for our approach is the set of 2D registrations between each pair of slices, both ways. The intensity of a new slice is then weighted by (i) the deformation functions and (ii) the intensities in the warped images. The effect of interpolation scheme on registration has been investigated in a few studies, e.g. when dealing with information theoretic similarity measures [4, 5]. However, the effect of using correspondence interpolation for registration has not been studied. The present paper demonstrates on two different datasets how applying correspondence interpolation scheme improves the final 3D registration result compared to standard intensity-based interpolation. 2. METHODS The basis for our correspondence-based interpolation approach is the set of 2D registrations between each pair of slices, both ways. Now, given a new slice coordinate, zI the corresponding new 2D image (slice), I is given by I = (1−α)IA (ϕBA (X, αwBA )) +αIB (ϕAB (X, (1 − α)wAB ))

(1)

ISBI 2011

where IA and IB are the slices adjacent to the new slice. ϕAB and ϕBA are the deformations from IA to IB and IB to IA respectively, with parameters wAB and wBA and X is the 2D sampling grid. This means that IA (ϕBA (X, wBA )) ∼ IB (X) IB (ϕAB (X, wAB )) ∼ IA (X). α is the weight determined by the inter-slice distances, α = (zI − zB )/(zA − zB ). Note that both the intensity and the amount of deformation is determined by α. The parts IA (ϕBA (X, αwBA )) and IB (ϕAB (X, (1 − α)wAB )) are given by bilinear interpolation in the z-plane. Any registration algorithm may be used as input to this interpolation scheme. For the experiments in this paper we have used the free-form deformation model (FFD) [6] for all registrations. As a similarity measure we have used sum of squared differences. For comparing interpolation techniques in terms of 3D registration accuracy, we have also used FFDs as the transformation model and sum of squared differences as a similarity measure in all registrations. For the following demonstration of gradients and registration accuracy, we now generally derive a gradient for a given similarity measure, D(T(ϕ(w)), R), where T is the 3D target (moving) image, R is the 3D reference image and ϕ is the transformation function with parameters w. The gradient with respect to the parameters is defined by ∇D(w) =

∂ϕ ∂T ∂D · · ∂w ∂ϕ ∂T

unit under standardized protocols. The voxel resolution is 1 × 1 × 1mm. 4. EXPERIMENTAL RESULTS The aim of the study is to test if 3D registration accuracy improves by using correspondence-based interpolation compared to using one of the best standard intensity interpolation method, namely cubic B-splines. It is not a simple task to design experiments that without doubt confirm our hypothesis since most validation measures will be dependent on the actual image intensities. A part of the experiments will therefore consist of qualitative examinations, while using downsampled versions of the high-resolution brain data allows us to carry out a quantitative experiment. The details and the results of the experiments are given in the following. 4.1. Inspection of image gradients As derived in Equation 2, one of the essential factors guiding the registration is the gradient of the target image evaluated at the deformed reference coordinates. Here, we qualitatively examine this gradient with respect to (a) linear intensitybased interpolation, (b) spline intensity-based interpolation, (c) correspondence-based interpolation. Figure 2 demonstrates the z-gradients of the deformed target (final deformation) for the three types of interpolators for both datasets.

(2)

From this we see that there are three important factors that guide the registration, namely the derivative with respect to the target image (∂D/∂T), which in case of a sum of squared differences measure is simply the difference image; the gradient of the transformation function (∂ϕ/∂w); and the gradient of the target image evaluated at the deformed reference grid (∂T/∂ϕ). Later in the paper, the quality of this gradient image will be investigated for different interpolation techniques.

(a)

(b)

(c)

(d)

(e)

(f)

3. DATA MATERIAL 32 cardiac pig MR images were acquired on a 1.5 T MR scanner. Each dataset consisted of 10-12 short-axis slices with 30 frames per cardiac cycle. In this paper, we only considered the first frame. Breath-hold position was accurately controlled from a respirator, thus minimizing slice misalignments. The in-plane resolution is 2 × 2mm, the slice thickness is 5 mm, and the inter-slice gap 5-6mm. Secondly, ten brain images from healthy volunteers were downloaded from the MIDAS Data Server (see acknowledgement). The images are T1 MR images, acquired on a 3T

1525

Fig. 2. Gradients of deformed target image, along slice direction (z-component of ∂T/∂ϕ) for (a,d) linear, (b,e) splinebased (c,f) correspondence-based interpolation for (a-c) the brain application and (d-f) the pig cardiac application (absolute values for better visualization). For each case, sampling density is the same for all three interpolators.

4.2. Cardiac pig atlas The registration accuracy is highly important when building an atlas with detailed anatomies and a high level of sharpness. For the cardiac data, we have slices of 5mm thickness, with a 5mm gap, increasing the need for a good interpolator. Here we qualitatively examine an atlas, which is constructed by averaging the set of images, pre-registered to a common reference, using (a) spline intensity-based interpolation, (b) correspondence-based interpolation. A zoom of the cardiac pig atlas resulting from these experiments is shown in Figure 3




(a)

(b)

Fig. 3. A cross-section of the cardiac atlas zoomed in at the left ventricle. The atlas is constructed by averaging registered images, using (a) spline intensity-based, (b) correspondencebased interpolation. The regions where anatomies lie nonparallel to the slice direction (z-axis) are highlighted.

4.3. Registration of downsampled brain images Here, we downsample the brain images to a slice resolution of 4mm by averaging every four slices. We select a reference in the original resolution (1mm) and carry out registration of the downsampled target images using (a) spline intensity-based interpolation, (b) correspondence-based interpolation. To get an idea of the registration accuracy, with respect to interpolation, we calculated MSD for the experiments in (a) and (b). This is shown in a box plot in Figure 4. Summarizing the results gives an 8.8% decrease in MSD on average, when using correspondence-based interpolation, compared to spline intensity-based.

MSD

7

6.5

6

Intensity−based

Registration−based

Fig. 4. Comparison of mean squared intensity differences (MSD) for the downsampled brain dataset with respect to the type of interpolation.

1526

4.4. Applying deformations from downsampled registrations to original resolution data Here, we adopt the results from the downsampled experiment and apply to the original brain data, of isotropic resolution, using (a) deformations from experiment (a), Section 4.3, (b) deformations from experiment (b), Section 4.3. This means that the images in (a) and (b) are the same, while the deformations are different, allowing us to quantitatively measure the registration accuracy in terms of by Mean Squared Differences (MSD) without effects from the actual interpolated image intensities. The results of this experiment in terms of percentage decrease in MSD, are shown in Table 1. A paired t-test of the MSD results revealed that correspondence-based interpolation gave significantly lower MSD than intensitybased (p-value 10−5 ). Table 1. Percentage improvement in mean squared intensity difference (MSD) when applying deformation fields from registration of downsampled brain images (using correspondence-based interpolation compared to splinebased interpolation) to original brain images. case % improvement in MSD 1 6.89 2 6.40 3 5.58 4 5.63 5 5.21 6 5.47 7 4.62 8 7.61 9 4.61 mean 5.78

5. DISCUSSION AND CONCLUSION As pointed out in Equation 2, the target image gradient is an important factor for the registration algorithm. Figure 2 shows clearly that the image gradient of the deformed target is much better estimated when using correspondence interpolation than when using standard intensity-based interpolation techniques. Obviously, for the pig application, the slice resolution is very poor, which makes it difficult to perfectly estimate the true underlying image and its gradient. Despite of this, we see a big improvement using the correspondencebased approach. For both data sets, it is noted that the interpolation is particularly important in regions where an edge does not lie parallel to the slice direction. We suggest that this improved gradient image is the main reason that the registration accuracy improves when using correspondence-based interpolation. For the pig MRI study, using correspondence-based interpolation for image registrations in atlas building, results

in more accurate registrations and hence, anatomies become more sharp, compared to when using intensity-based registration. This is seen in Figure 3. The sharpness gives a more precise location of the anatomies, which may increase the accuracy in labelling the atlas. The main conclusion from this is that we may build a high-resolution (2mm slices) sharp and a precise atlas, using low-resolution data (10mm slices). The final output of a registration of cardiac MRI is typically segmentation of the most important anatomies. Although the atlas results look promising, we cannot rule out the fact that a part of the improved results is due to the interpolation itself and not the registration accuracy. A further confirmation of the increased registration accuracy based on segmentations would be desirable. The down-sampled brain registration results are promising with respect to the MSD shown in 4. However, we note that even though the same reference was used for both sets of registrations, this result may only show the difference in interpolated intensities rather than measuring the actual registration accuracy. This factor is eliminated in Subsection 4.4, where the accuracy is measured on the same images, using different deformations, depending on the interpolator. The percentage improvement is slightly lower than the previously obtained, which might give a clue of how big the factor of the interpolated intensities is with respect to measuring the actual registration accuracy. In general, we have shown that enhancing correspondence in interpolation improves the image quality, its gradient and in particular 3D registration results. The effect of correspondence becomes more important as the slices are thicker, since this infers that corresponding points may lie further away, i.e. the adjacent slices are more different. The technique could also be used for time-series of images, where similar to the slice thickness, the determining factor for the importance of correspondence would be the time-frame interval. There are a few issues that need to be considered when establishing the correspondences (2D registrations). The first is to ensure that no folding is present since this would lead to incorrect interpolation. Recently, many techniques that guarantee that the deformations do not fold have been introduced. Hence, this should be easy to avoid. Secondly, the registration might fail to provide correct correspondences. This might be either when the changes between slices were too large to cover by the transformation model, providing too little deformation, or when non-corresponding structures were matched, possibly providing too large deformation. The first case would not give worse interpolation than the standard intensity-based. The latter, could however give problems. This is more likely to happen when slices are very different (thick) and is obviously hard to solve entirely. Despite of these concerns, even though we dealt with slice resolution of up to 10mm we did not experience problems with this. In conclusion, we have demonstrated on two different datasets, how using a correspondence-based interpolation

1527

scheme improves registration results, in particular for data where resolution in one dimension is poor. 6. ACKNOWLEDGEMENTS We would like to acknowledge Aarhus University Hospital, Skejby, for providing the pig MRIs and CASILab at The University of North Carolina at Chapel Hill for making brain MRIs available on the MIDAS Data Server at Kitware, Inc. [7]. For all image registrations, we have used a MATLAB framework by Martin Vester Christensen [8]. Finally, H. ´ Olafsd´ ottir, H. Pedersen and M.S. Hansen thank the Danish Agency for Independent Research – Technology and Production Sciences for funding. 7. REFERENCES [1] G.P. Penney, J.A. Schnabel, D. Rueckert, M.A. Viergever, and W.J. Niessen, “Registration-based interpolation,” IEEE Transactions on Medical Imaging, vol. 23, no. 7, pp. 922–926, 2004. [2] D.H. Frakes, L.P. Dasi, K. Pekkan, H.D. Kitajima, K. Sundareswaran, A.P. Yoganathan, and M.J.T. Smith, “A new method for registration-based medical image interpolation,” IEEE Transactions on Medical Imaging, vol. 27, no. 3, pp. 370–377, 2008. ´ [3] H. Olafsd´ ottir, H. Pedersen, M.S. Hansen, M. Lyksborg, M.F. Hansen, S. Darkner, and R. Larsen, “Registrationbased interpolation applied to cardiac MRI,” in Proceedings of SPIE, 2010, vol. 7623, p. 762336. [4] M. Seppa, “Continuous sampling in mutual-information registration,” IEEE Transactions on Image Processing, vol. 17, no. 5, pp. 823–826, May 2008. [5] F. Maes, A. Collignon, D. Vandermeulen, G. Marchal, and P. Suetens, “Multimodality image registration by maximization of mutual information,” IEEE Transactions on Medical Imaging, vol. 16, no. 2, pp. 187–198, 2002. [6] D. Rueckert, L.I. Sonoda, C. Hayes, D.L.G. Hill, M.O. Leach, and D.J. Hawkes, “Nonrigid registration using free-form deformations: application to breast MR images,” IEEE Trans. on Medical Imaging, vol. 18, no. 8, pp. 712–721, 1999. [7] E. Bullitt, D. Zeng, G. Gerig, S. Aylward, S. Joshi, J.K. Smith, W. Lin, and M.G. Ewend, “Vessel tortuosity and brain tumor malignancy: a blinded study,” Academic radiology, vol. 12, no. 10, pp. 1232, 2005. [8] M Vester Christensen, Image Registration and Optimization in The Virtual Slaughterhouse, Ph.D. thesis, Technical University of Denmark, 2008.