A symmetric block-matching framework for global registration

A symmetric block-matching framework for global registration Marc Modata,b , David M. Casha,b , Pankaj Dagaa , Gawin P. Winstonc , John S. Duncanc and S´ebastien Ourselina,b a Centre

for Medical Image Computing, University College London, UK. Research Centre, Institute of Neurology, WC1N 3BG, University College London, UK. c Department of Clinical and Experimental Epilepsy, WC1N 3BG, University College London, UK. b Dementia

ABSTRACT Most registration algorithms suffer from a directionality bias that has been shown to largely impact on subsequent analyses. Several approaches have been proposed in the literature to address this bias in the context of non-linear registration but little work has been done in the context of global registration. We propose a symmetric approach based on a block-matching technique and least trimmed square regression. The proposed method is suitable for multi-modal registration and is robust to outliers in the input images. The symmetric framework is compared to the original asymmetric block-matching technique, outperforming it in terms accuracy and robustness.

1. INTRODUCTION Medical image registration is core to many image analysis pipelines. It consists of bringing two or more images into spatial alignment, often mapping one image into the space of another. However, recent studies have highlighted that this directionality in the registration can create a bias in analyses.1–3 Several approaches have been proposed to address this issue in the context of non-linear registration. Christensen and Johnson4 jointly optimise forward and backward transformations while minimising an inverse-consistency error criterion. Avants et al.,5 Vercauteren et al.6 and others optimise the transformation parameters in a mid-point image or a set of intermediate images. Little has however been proposed to remove directionality bias in the case of global registration. Reuter et al.7 developed a global registration technique based on robust statistic optimisation performed in a mid-point space. This approach minimises the residual differences between input images while rejecting outliers. A linear intensity scaling is used in order to increase the robustness of the algorithm and deal with different ranges of intensity within the same modality or pulse-sequence. The proposed approach for robust and symmetric registration differs from Reuter et al. in two main aspects. First, we used a block-matching approach8 to establish the spatial correspondences, where the normalised cross-correlation is used as a measure of similarity. Due to the small dimension of the blocks under consideration, the proposed approach is suitable for multimodal registration cases and is robust in the presence of outliers.9 The second main difference is that joint forward and backward transformation parameters are simultaneously calculated rather than using a mid-point. This removes the need to discretise the transformed input images into an average space, which can be problematic when input images have different fields of view and resolution, often the case in multimodal registration. We evaluated the accuracy and robustness of our symmetric block-matching based approach using several databases of synthetic MR images, MR longitudinal studies, multimodal studies of MRI, PET and CT scans, and pairs of pre-operative and intraoperative MR.

Medical Imaging 2014: Image Processing, edited by Sebastien Ourselin, Martin A. Styner, Proc. of SPIE Vol. 9034, 90341D · © 2014 SPIE CCC code: 1605-7422/14/$18 · doi: 10.1117/12.2043652 Proc. of SPIE Vol. 9034 90341D-1 Downloaded From: http://proceedings.spiedigitallibrary.org/ on 03/19/2015 Terms of Use: http://spiedl.org/terms

2. METHOD 2.1 Block-matching for global registration: the classical approach In the original block-matching method, the registration is performed using a two-step approach: the first step is to establish point correspondences between the current transformed floating image and the reference image using the block-matching strategy. This is done by dividing the reference and transformed floating images into blocks of uniform size. Each block in the reference image is compared with image blocks in the corresponding neighbourhood of the transformed floating image. The matching block is the one with the maximum normalised cross-correlation (NCC) with the reference block. The second step is to compute the transformation parameters from these point correspondences using a least trimmed square (LTS) regression method, which ensures robust outlier rejection. This optimisation technique is also computationally efficient as each LTS iteration has an analytical solution. These two steps are performed iteratively, where the optimisation starts with a large block search neighbourhood (corresponding to gross displacements) and then moves towards a smaller search neighbourhood (corresponding to finer displacements) in a coarse to fine strategy.

2.2 Directionality bias in image registration The classic block-matching image registration maps one image into the space of another, producing a transformation TI→J that maps image I to image J. If the two images were reversed so that J is mapped to I, a new transformation TJ→I is produced. The directionality that is prevalent in many image registration −1 algorithms would result in TI→J 6= TJ→I . If an algorithm is symmetric, then these two transformations would be the inverse of each other, and the registration result would not depend on the order of images.

2.3 Symmetric extension to block-matching global registration Since the original block-matching based registration introduces a directionality, the results would yield transformations that are not the inverse of each other when the order of images are switched. The inverse-consistent extension to the block-matching algorithm ensures that the registration is symmetric and no bias is coming from the registration direction. Similar to the classical approach, the optimum transformation is estimated in an iterative fashion where the current estimated transformation is updated at each iteration. Each iteration follows a two-step scheme where the first step is to establish point correspondences between the two images. Using block matching, ~i ~j two sets of correspondences are estimated: C I→J mapping points from image I to image J and CJ→I to map points from image J to image I where i and j denote the block indexes in images I and J respectively. The second step is to update the transformation parameters estimated through LTS regression. In the original block-matching-based approach, the updated parameters are incremental and composed with the current estimate of the transformation. However, since matrix multiplication is not commutative, this approach would break symmetry. If the current forward (Ft ) and backward (Bt ) transformations are symmetric, i.e. Ft = Bt−1 are inverse of each other and their updates ft and bt are calculated, then the updated transformations Ft+1 and Bt+1 need to computed as Ft+1 = ft ◦ Ft and Bt+1 = Bt ◦ bt in order to maintain −1 symmetry, Ft+1 = Bt+1 . However if the images are reversed to generate new transformations Ft0 = Bt and 0 Bt = Ft , then the updates would be multiplied in the reverse order as they were in the original run. In order to address this issue, the block-matching correspondences are always established using the original image positions in both I and J. To ensure inverse-consistency between the transformation parameters, the transformation matrices obtained through the LTS fitting are averaged10 and updated: −1 ~ ~ TI→J = Expm Logm LTS CI→J + Logm LTS CJ→I /2 −1 ~ ~ TJ→I = Expm Logm LTS CJ→I + Logm LTS CI→J /2 ,

Proc. of SPIE Vol. 9034 90341D-2 Downloaded From: http://proceedings.spiedigitallibrary.org/ on 03/19/2015 Terms of Use: http://spiedl.org/terms

where Expm and Logm are the exponential and logarithm matrix operator. Using such an approach, the result transformations, forward and backward, are inverse-consistent and the directionality of the registration does not affect the recovered transformation parameters.

3. VALIDATION The proposed block-matching method has been implemented as part of the NiftyReg open-source software∗ . All registrations were run using the same parameters to allow for a fair comparison of the results. NCC was used as a local measure of similarity in the block-matching procedure. The block dimensions were chosen to be 4 × 4 × 4 voxels. The blocks were sorted according to decreasing measure of variance and the top 50% of the blocks in the reference images were considered for the LTS optimisation step. During the LTS optimisation, 50% of the blocks with the largest squared Euclidean residuals (according to the current estimated transformation) were considered as outliers. For the registrations, a multi-scale pyramidal approach with three resolution levels was used to avoid local minima and increase the capture range. The registrations produced a full affine transformation (12 degrees of freedom). For the RIRE database, 4 resolution levels and a rigid transformation (6 degrees of freedom) were used.

3.1 Validation on synthetic data In order to assess the capture range and robustness of the proposed symmetric approach, two synthetic images† were used: a T1-weighted and a T2-weighted MRI. Both images were simulated with 9% noise level and 20% intensity non-uniformity. The images were a priori in alignment and known affine transformations were applied to one image. Registrations were then performed using the symmetric approach and the asymmetric approach in both forward and backward directions. The registration error was computed at the 8 corners and the average Euclidean distance between the ground truth and the recovered positions for these corner points are reported. The known transformations were generated by applying rotations from -45°to 45°at 15°increment step, translations from -45 to 45 voxels with 15 voxels increment, scaling from 50% to 150% with a 10% increment and finally shearing from -0.2 to 0.2 with a 0.05 increment. This resulted in 4851 transformations (varying each of the 12 components independently) and a total of 14553 individual registrations (symmetric, asymmetric forward, asymmetric backward). Any registrations results with an average error larger than 2 voxels were classified as a failure. The two images were also registered using the FLIRT algorithm,11 part of the FSL software package‡ . Normalised mutual information was used as a similarity measure, with initial transformations set to identity. Due to the high level of noise and intensity non-uniformity FLIRT lead to an error of 1.30 and 1.16 voxels for the forward and backward approaches respectively. The success rates for the symmetric, forward and backward implementations were 91.12%, 78.96% and 86.02% respectively. For 1742 transformations, the symmetric approach successfully recovered the transformation when one asymmetric scheme failed. In 410 cases, the symmetric recovered the known transformation when both he forward and backward scheme failed. For 19 occurrences the symmetric approach failed whereas the backward method successfully recovered the transformation and in 1 case the symmetric registration failed while both asymmetric approaches recovered the transformation.

3.2 Validation on the RIRE database The Retrospective Image Registration Evaluation (RIRE) project, based on the Vanderbilt database,12 consists of CT, PET and MRI scans with associated marker-based gold standard transformation§ . We assessed ∗

http://sourceforge.net/projects/niftyreg http://brainweb.bic.mni.mcgill.ca/brainweb ‡ http://fsl.fmrib.ox.ac.uk/fsl/fslwiki/FLIRT § http://www.insight-journal.org/rire †


the accuracy of the proposed symmetric approach using this database and compared its result with the asymmetric approach and FLIRT. For all registrations, the CT and PET images were used as references and the landmark errors were assessed in their space. The voxel dimensions of the CT and PET images were 0.65 × 0.65 × 4.00 and 2.59 × 2.59 × 8.00 millimetres respectively. The proton density (PD), T1-weighted (T1w) and T2-weighted (T2w) MRI were used as floating images. Figure 1 shows coronal views of multiple modality scans from a subject.

CT

NMINIP--.1....'

.71,--,PrT,

MFM.1,-'.-7--7--TI`"/",,

PD MRI

T1w MRI

T2w MRI

PET

Figure 1. Corresponding coronal slices of the same subject in the multiple modalities available as part of the RIRE database.

Intra-subject registration was performed on 9 subjects and the registration error, defined as the Euclidean distance in millimeters, between the ground truth and the recovered transformed positions, was computed at 774 landmark positions. Table 3.2 shows, for all registration techniques, the mean and maximum errors per modality and the overall error. Figure 2 shows the cumulative registration error over all landmarks when the CT or PET images are used as reference. Table 1. Registration errors from multi-modal intra-subject registrations using the RIRE database. Sym. BM: symmetric block matching registration, Asym. BM: original, asymmetric block matching, FLIRT: global registration algorithm in FSL. The columns show the mean and maximum errors of the various multimodal combinations registered, with the overall error in the last common.

Reference image Floating image Sym. BM (mean error) Sym. BM (max error) Asym. BM (mean error) Asym. BM (max error) FLIRT (mean error) FLIRT (max error)

CT PD 1.71 3.07 2.13 12.45 1.57 3.81

CT T1w 1.60 3.62 1.95 4.85 1.25 3.24

CT T2w 1.64 3.33 3.39 25.24 1.85 4.24

PET PD 3.11 9.28 4.39 21.48 32.36 206.33

PET T1w 3.54 9.12 9.94 68.98 40.21 266.24

PET T2w 2.49 7.55 2.68 7.38 30.70 218.87

ALL 2.33 9.28 3.94 68.98 17.25 266.24

3.3 Validation on the MIRIAD database We used the MIRIAD database¶ ,13 to assess the transitivity properties of the symmetric and asymmetric approaches. The MIRIAD database consist of brain MRI scans from 46 subjects diagnosed with Alzheimer’s disease and 23 age-matched healthy controls. For all subjects, we used a baseline, 6-month and 12-months followup scans. The 6-months and the 12-months follow-up scans were registered to their corresponding baseline (T6−BL and T12−BL respectively) and the 12-months scans were registered to the 6-months scans (T12−6 ). Transitivity error was defined as the average Euclidean distance at the transformed corners of the baseline image using the direct correspondences (T12−BL ) compared to the composed transformation (T12−6 ◦ T6−BL ). The mean transitivity error over the 69 subjects was 0.54mm (0.27) and 0.66mm (0.34) for the symmetric and asymmetric approaches. The symmetric approach leads to significantly lower transitivity error than the asymmetric approach (two-tailed, paired t-test, p < 0.005). Using FLIRT, with the default parameters, we obtained a mean error of 1.00 (0.71) mm. The minimum and maximum average errors were [0.15,1.35], [0.20,1.60] and [0.27,4.74] for the symmetric block-matching, the asymmetric block-matching and FLIRT respectively. ¶

http://www.ucl.ac.uk/drc/research/miriad


Commulative error in millimeters over 404 landmarks (CT images only)

3

2

10

1

10

0

10

Commulative error in millimeters over 370 landmarks (PET images only)

10

FLIRT with NMI (FSL) Asymmetric block−matching Symmetric block−matching Number of landmarks

Number of landmarks

10

3

FLIRT with NMI (FSL) Asymmetric block−matching Symmetric block−matching

2

10

1

10

0

0

20

40 60 80 Cummulative error in millimetres

100

10

0

20

40 60 80 Cummulative error in millimetres

100

Figure 2. Cumulative error in millimetres over multiple landmarks using the symmetric and asymmetric blockmatching approaches and FLIRT . The left- and right-hand side plots show the errors assessed when the images used as reference are CT and the PET respectively.

3.4 Intra-operative images A dataset comprising 15 pairs of pre- and intra-operative T1-weighted MR images, was used to assess the robustness of the proposed algorithm to missing tissues. The pre-operative MRI were acquired on a 3T GE Signa Excite HD (General Electric, Waukesha, Milwaukee, WI, USA) with a spatial resolution of 0.9 × 0.9 × 1.1 mm. The intra-operative scans, acquired on a 1.5T Siemens Espree, have a spatial resolution of 1.1 × 1.1 × 1.3 mm and were all acquired after some tissue resection was performed. Note that the intra-operative acquisitions do not have the brain centre in the field of view. Figure 3 shows the pre- and intra-operative scans from one subject. Pre-operative image

Affinely registered intra-operative image

Affinely registered pre-operative image

Intra-operative image

Figure 3. Pre- and intra-operative example images.

We performed the 15 intra-subject registrations using the proposed symmetric block-matching approach and twice with the asymmetric block-matching approach (one where pre-operative image served as reference


and one where the inraoperative was reference). It is difficult to localise anatomical landmarks in the intraoperative MRI images due to severe non-linear geometric distortions and degraded image quality, hence we only performed a qualitative analysis of the registration results. Two reviewers were asked to qualify the registration results as successful, acceptable or failure and the results are reported in Table 2. The symmetric was able to successfully register all 15 datasets, while multiple cases using the asymmetric registrations were classified as failures. Table 2. Qualitative evaluation of pre- and intra-operative registration performance.

Symmetric Forward Backward

Successful 15 12 5

Acceptable 0 1 3

Failure 0 2 7

4. DISCUSSION We presented a symmetric extension of the block-matching algorithm for global registration. The algorithm has been compared to the original asymmetric framework in both mono- or multi-modal registration. Using simulated transformations, the symmetric formulation led to a increased capture range. This method lead to a lower transitivity error over multiple time-points and multiple subjects, a small but acknowledged source of bias in subsequent analyses. Using the RIRE database for evaluation of inter modality registration, we found that the symmetric approach outperformed its asymmetric counterpart and an established technique (FLIRT) both in terms of accuracy and in term of robustness as shown by lower cumulative errors over multiple landmarks. Finally, using pre- and intra-operative MRI data, the proposed algorithm was shown to be robust to missing tissue. Note that all validations were performed without using any masking or initial alignment of the brain centre of mass, as the overall aim was to compare several methods in the same context. Future work will investigate automatic detection of the percentage of outliers by merging information derived from the forward and backward transformations.

Acknowledgements Marc Modat is supported by the UCL Leonard Wolfson Experimental Neurology Centre. The Dementia Research Centre is supported by Alzheimer’s Research UK, Brain Research Trust, and The Wolfson Foundation. Pankaj Daga is supported by EPSRC (EP/J020990/01). Sebastien Ourselin receives funding from the EPSRC (EP/H046410/1, EP/J020990/1, EP/K005278), the MRC (MR/J01107X/1), the EU-FP7 project VPH-DARE@IT (FP7-ICT-2011-9-601055), the NIHR Biomedical Research Unit (Dementia) at UCL and the National Institute for Health Research University College London Hospitals Biomedical Research Centre (NIHR BRC UCLH/UCL High Impact Initiative).

REFERENCES [1] Yushkevich, P. A., Avants, B. B., Das, S. R., Pluta, J., Altinay, M., Craige, C., and Initiative, A. D. N., “Bias in estimation of hippocampal atrophy using deformation-based morphometry arises from asymmetric global normalization: an illustration in ADNI 3T MRI data,” NeuroImage 50, 434–445 (Apr 2010). [2] Thompson, W. K., Holland, D., and Initiative, A. D. N., “Bias in tensor based morphometry stat-ROI measures may result in unrealistic power estimates,” NeuroImage 57, 1–4; discussion 5–14 (Jul 2011). [3] Fox, N. C., Ridgway, G. R., and Schott, J. M., “Algorithms, atrophy and Alzheimer’s disease: cautionary tales for clinical trials,” NeuroImage 57, 15–18 (Jul 2011). [4] Christensen, G. E. and Johnson, H. J., “Consistent image registration,” IEEE Transactions on Medical Imaging 20, 568–582 (Jul 2001).


[5] Avants, B. B., Epstein, C. L., Grossman, M., and Gee, J. C., “Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain,” Medical Image Analysis 12, 26–41 (Feb 2008). [6] Vercauteren, T., Pennec, X., Perchant, A., and Ayache, N., “Symmetric log-domain diffeomorphic registration: a demons-based approach,” in [International Conference on Medical Image Computing and Computer-Assisted Intervention ], 754–761 (2008). [7] Reuter, M., Rosas, H. D., and Fischl, B., “Highly accurate inverse consistent registration: a robust approach,” NeuroImage 53, 1181–1196 (Dec 2010). [8] Ourselin, S., Roche, A., Subsol, G., Pennec, X., and Ayache, N., “Reconstructing a 3D structure from serial histological sections,” Image and Vision Computing 19, 25–31 (Jan 2001). [9] Commowick, O., Wiest-Daessle, N., and Prima, S., “Block-matching strategies for rigid registration of multimodal medical images,” in [Biomedical Imaging (ISBI), 2012 9th IEEE International Symposium on ], 700–703 (2012). [10] Alexa, M., “Linear Combination of Transformations,” ACM Transactions on Graphics 21, 380–387 (2002). [11] Jenkinson, M. and Smith, S. M., “A global optimisation method for robust affine registration of brain images,” Medical Image Analysis 5, 143–156 (Jun 2001). [12] West, J., Fitzpatrick, J., Wang, M. Y., Dawant, B. M., Maurer, C. R., Kessler, R. M., Maciunas, R. J., Barillot, C., Lemoine, D., Collignon, A., Maes, F., Suetens, P., Vandermeulen, D., van den Elsen, P. A., Napel, S., Sumanaweera, T. S., Harkness, B., Hemler, P. F., Hill, D. L., Hawkes, D. J., Studholme, C., Maintz, J. B., Viergever, M. A., Malandain, G., and Woods, R. P., “Comparison and evaluation of retrospective intermodality brain image registration techniques,” Journal of computer assisted tomography 21, 554–566 (Jul 1997). [13] Malone, I. B., Cash, D., Ridgway, G. R., Macmanus, D. G., Ourselin, S., Fox, N. C., and Schott, J. M., “MIRIAD-public release of a multiple time point Alzheimer’s MR imaging dataset,” NeuroImage 70, 33–36 (Apr 2013).
