Atlas to Patient Registration with Brain Tumor Based on a Mesh-free Method Idanis Diaz1 and Pierre Boulanger1 Abstract— Brain atlas to patient registration in the presence of tumors is a challenging task because its presence cause brain structure deformations and introduce large intensity variation between the affected areas. This large dissimilarity affects the results of traditional registration methods based on intensity or shape similarities. In order to overcome these problems, we propose a novel method that brings closer the atlas and the patient’s image by simulating the mechanical behavior of brain deformation under a tumor pressure. The proposed method use a mesh-free total Lagrangian Explicit Dynamic algorithm for the simulation of atlas deformation and a data driven model of the tumor using multi-modal MRI segmentation. Experimental results look structurally very similar to the patient’s image and outperform two of the top ranking algorithms.
I. I NTRODUCTION The importance of atlas to patient registration relies on the diverse applications. One of the most important application is image segmentation, which can be performed by projecting the labeled structures of an atlas to the patient image. Another important application is statistical image analysis of diseases based on population studies. Registering population data to a common reference space allows to compare neuro-anatomical features between groups classified by: age, sex, and disease evolution grade. Also, registered atlases to patients can be used for surgical and treatment planning. The accuracy strongly depend on how good the atlas to patient registration is performed which in the presence of a tumor becomes a challenging task because tumors cause brain structure deformations and introduce large intensity variations between the affected areas in the patient’s image and the healthy tissue in the atlas image. In order to address these problems some approaches have been introduced where bio-mechanical models of brain tissues are used to simulate the mass-effect deformation produced by growing a tumor seed in the atlas [1], [2]. The objective is to obtain a more realistic warping of the atlas by deforming the brain structures according to the mechanical pressure caused by the tumor growth. Although, approaches that use bio-mechanics models to simulate the brain deformation yielded better results than direct registration over the images, these approaches also have some limitations, one of them is the assumption that a tumor models have regular shapes which is not the case for real tumors. Brain tumors, specially at an advanced stage, such as glioblastomas, show irregular shapes causing irregular deformations. Another issue of these approaches is the use of I. Diaz and P. Boulanger are with Department of Computing Science, University of Alberta, AB, Canada
[email protected]
[email protected]
Finite Element Methods (FEM) to compute bio-mechanical deformation. Regular meshes are not suitable to simulate large deformations because large distortions of the FEM mesh require the ability to deal with non-linear energy function. Gooya et al. [2] introduced an Eulerian formulation to overcome FEM’s limitation. However the use of a regular grid to fit an irregular geometry during the simulation is a limitation of these methods. In this paper, we introduce a new approach to atlas to patient registration that uses mesh-free Total Lagrangian Explicit Dynamic method to deal with the tumor mass-effect simulation, and a new tumor growth model for the simulation which uses the shape of the segmented tumor from multimodal MRI data instead of assuming an unrealistic regular shape. Section II describes in details our approach. Section III presents some experiments results to evaluate the accuracy and Section IV concludes discussing the pos and cons of the method. II. ARMSTG A LGORITHM We will refer to our method as ARMSTG, which is the acronym for Atlas Registration Based on Mesh-free Simulation of Tumor Growth. ARMSTG consists of three main steps. The first step involves tumor segmentation and registration of the patient to the atlas space. This step is performed with the purpose of bringing the tumor from the patient’s image space to the atlas space. The second step is the core of ARMSTG and consists of four sub-steps that will be explained with some details below. The third step is a deformable registration that refines the results and deformation of the atlas image. This last step will be also explained in Section II-B. A. Tumor Growth and Mass-effect Simulation in Atlas Space The tumor growth simulation has two main steps. In the first step ARMSTG shrinks the segmented tumor boundary (extracted from the segmentation of multi-modal MRI using an algorithm described in [3] and [4]) to a unique seed point using a level set method. During this shrinking process, we inwardly propagates the boundary of the segmented tumor with a constant velocity in the inverse normal direction until it becomes a singular point (tumor center). At each iteration of the level set ARMSTG saves in a list the points of the evolving interface and a link to their previous positions, creating a displacement map. In the second step, ARMSTG regrows the tumor in the atlas image from the tumor center in the outward direction using the displacement map of the level set saved during the shrinking process. The point coordinates
computation entails tracking the displacements ∆x for each point x produced by the level set procedure. ARMSTG uses a Lagrangian formulation for the boundary evolution equation in order to estimate the displacements: ∆x ∆t = v(x). The second step of the tumor growth and the masseffect simulation are performed over a sample of points representing the atlas. Those points are scattered in image space representing the brain tissues, and serve as the force field nodes and integration points for a mesh-free simulation that computes the deformations created by the tumor. The simulation starts from a seed position in the atlas image. 1) Sampling and Seeding the Bio-mechanical Model to Simulate Tumor Growth and Mass-effect: ARMSTG implements an adaptive sampling method proposed in [5]. This sampling method place nodes based on a feature map that is computed according to the distribution of the gradient in such a way that areas with the most image detail, e.g. boundary around ventricles, contain the most node density. In our algorithm, the seed location coincides with the center of the inner contour generated by the level set method during the shrinking steps. All the seed points initially share the same location in the atlas image, later they are distributed around the tumor expanding boundary as the simulation runs. The seed points are added to the list containing the generated nodes. 2) Mesh-free Method for the Mass-Effect Simulation: ARMSTG employs a mesh-free method to solve the equation system that describes the mechanical behaviour of brain tissues under to the pressure of the growing tumor. The tumor growth simulation starts by growing the tumor seed using the displacement map built during the shrinking steps. At each iteration, the points belonging to the tumor boundary moves outward until the original tumor boundary is reached. As the tumor boundary grow, new points falling inside of the tumor become part of the boundary. As the simulated tumor grows the displacements propagate to the rest of the field nodes. In this way, the brain structures are deformed according to the tumor progression. ARMSTG estimates the displacement u at every point x = (x, y, z) by interpolating this field variable from neighboring points falling inside a local support domain [6] around the point of interest. Figure 1 summarizes the mesh-free procedure implemented. Our method estimates the derivatives D using Thin Plate Spline Radial Basis Functions (RBF) [6], and the mass m and volume v following the approach in [7], based on a spherical interpolation. Following Miller et al. [8], the deformation gradient t0 X is computed from the shape derivatives matrix and the nodal displacements u i.e. t0 X = D′t u + I3 , where I3 is a 3 × 3 identity matrix, and the strain-displacement matrix is defined as: (1) (2) (n) t = [t0 BL , t0 BL , . . . , t0 BL ], 0 BL (1) (i) t = D′t0 XT . 0 BL In the notation, the left superscript represents the current time or iteration and left subscript represents the time of
Fig. 1: ARMSTG Algorithm.
the reference configuration which in this case is 0 (initial configuration). The forces in the next step in Figure 1 are defined as: ∫ t 0F
= 0V
t Tt 0 0 BL 0 Sd V
(2)
where t0 BL is the full strain-displacement matrix and t0 S is the second Piola-Kirchoff stress vector. Equation 2 is solved by numerical integration applied over the set of nodes. The second Piola-Kirchoff stress vector is:
t 0S
1 = µJ−2/3 (I3 − It0 C−1 ) + λ(J − 1)Jt0 C−1 3
(3)
where I is the first invariant of the Deviatoric Right Cauchy Green deformation tensor C, where J is the determinant of the deformation gradient, and µ and λ are the shear and bulk modulus of the material respectively. This equation is a neo-Hookean model describing a hyper-elastic mechanical behaviour of the brain tissues. The values for Young’s modulus (E) and Poisson’s ratios (v) were different for each type of tissue, i.e., parenchyma and tumor 3.0 and 0.49 respectively, and CSF 10 and 0.1. These values were taken from Miller et al. works in [8]. As in TLED, ARMSTG uses the central difference method algorithm derived from Newton’s second law to calculate the global nodal displacement defined as: t+∆t u 0
=
−∆t2 t F + 2t0 u − t−∆t u. 0 M
(4)
At each iteration ARMSTG updates the coordinates of each point in the problem domain by: t+∆t x = t x+ t+∆t u. How0 ever, the algorithm updates some points according to a prepestablished displacement. These points are those corresponding to the tumor boundary which create the displacement map given in Section II-A.2, and the points belonging to the boundary of the brain tissue whose displacements are set to 0 during the simulation. ARMSTG executes the main loop of the algorithm nl times, where nl is the number of contours generated by the level set method.
B. Deformable Registration In this step ARMSTG applies a diffeomorphic demons algorithm over binary images representing the brain tissues of the patient and the atlas with tumor, instead of the patient and deformed atlas directly. The registration of binary instead of intensity images gives better results since the algorithm does not have to deal with the irreconcilable intensity differences between the source and target images. The deformation field obtained from registering the binary images is then applied to the deformed atlas in intensity scale. III. E XPERIMENTAL R ESULTS We evaluate the ARMSTG’s performance against two popular algorithms used in MRI registration of the human brain [9]. The experiments are conducted over twelve MRIs taken from the BRATS challenge training data set [10]. The experiments yielded several registered images whose similarities to the respective original were measured using a metric based on distance fields. A. The Similarity Metric
(a) Patient
by extracting the largest connected component to a manually selected seed in the respective binary images. Figure 2 shows some examples of the segmented ventricles. 1) ARMSTG vs other brain MRI registration methods: We compared the performance of ARMSTG with other registration methods, such as the Automatic Registration Toolbox (ART) [12], and the symmetric diffeomorphic image registration with cross-correlation (SyN), described in [13]. Both algorithms yielded the best registration results in the evaluation carried out by Klein et al. [9]. Both methods were used with default parameters described in their respective manuals. In order to perform these comparisons, we applied Kruskal-Wallis test over a table containing the similarity metric values yielded by the three methods, ARMSTG, SyN, and ART, over the set of twelve cases. We obtained a p-value of 1.36 × 10−5 , meaning there is a significant difference between the two groups. Figure 4 is a box plot of the registration error summarizing the results produced by the three methods. The first box corresponds to ARMSTG and shows that our method outperformed SyN and ART represented by the other two boxes in the figure respectively. Figure 3 shows some registration results obtained with the three methods and the respective tumors. This figure corroborates that ARMSTG outperformed the other two methods, since the registered images look structurally more similar than the registered images obtained with the other two methods.
(b) Ventricles
(c) Boundary
Fig. 2: Ventricle segmentation from a patient. In order to measure the similarity between registered atlas images to a patient case, we compute the distance between the ventricle boundaries of the registered atlas and the patient images. The distance metric applied on this work is based on a distance field [11]. The metric used in this work to measure the similarity between a registered atlas image Ia and the patient’s image Ip is: ∑ dist(Ia , Ip ) = min ∥xa − xp ∥ (5) xa ∈Ia
xp ∈Ip
where ∥xa − xp ∥ represents the Euclidian distance between a pixel xa and a pixel xb from the registered atlas image A and the patient image B respectively. Equation 5 is computed from the distance field of the patient ventricle boundaries. The ventricle segmentation is carried out semi automatically
Fig. 3: Various cases of Atlas deformation by a tumor: (from left to right) (1) original image, (2) ARMSTG, (3) SyN, (4) ART. The colors in the tumor images represent: yellow/edema, red/enhancing tumor, magenta/non-enhancing tumor, cyan/necrosis. In average ARMSTG registered atlas to patient images in 85 seconds which is less than the time in average required by
and mass-effect simulation. In the future we would like to extend ARMSTG to deal with other human organ registration such as lung and bladder. V. ACKNOWLEDGMENT Authors wish to thank CISCO Systems for the grant which supported this work. R EFERENCES
Fig. 4: Box plot of the registration error comparing the ARMSTG algorithm vs. SyN and ART: (1) ARMSTG, (2) SyN, and (3) ART.
the other two methods and a CPU version of our approach. IV. C ONCLUSION This paper introduced a novel atlas to patient registration method with tumor. ARMSTG deforms the brain structures of the atlas by simulating the tumor growth and mass-effect using a bio-mechanical model of the brain tissue deformation. The experimental results proved that the strategy of introducing the tumor in the atlas simulating the tumor growth and mass-effect lessen the difference between the target and the source improving the registration results of a diffeomorphic demon algorithm. We demonstrate that our approach outperformed two nonlinear registration methods for brain atlas to patients image ranked as the top best algorithms SyN and ART [9]. One advantage or our method in comparison with the other two is its ability to deal with tumors and the large deformation caused by the mass-effect. Although other authors introduced this strategy [14], [1], [2], ARMSTG extends significantly the idea by using a total lagrangian mesh-less method for the simulation, and a guided tumor growth model using the actual shape of the segmented tumor from the original image’s patient. The use of a meshless method provided some advantages such as the ability of handling large deformations without re-meshing, and making possible the use of a tumor growth model guided by the actual shape of the tumor instead of a spherical or regular shape. Our tumor growth model does not have to deal with the problem of seed initialization as in [14], [1], [2]. The ARMSTG algorithm does not require to search the initial seed position with an optimization procedure and is not sensitive to the initial seed size. ARMSTG was evaluated with different sampling methods and demonstrated to be robust to parameter variation such as the parameters involve in the level set procedure and the sample size. Another advantage of our method is the use of parallel processing using GPU which considerably helped to reduce the computational time required for the tumor growth
[1] E. Zacharaki, C. Hogea, D. Shen, G. Biros, and C. Davatzikos, “Nondiffeomorphic registration of brain tumor images by simulating tissue loss and tumor growth,” NeuroImage, Feb. 2009. [2] A. Gooya, G. Biros, and C. Davatzikos, “Deformable registration of glioma images using EM algorithm and diffusion reaction modeling,” IEEE Trans. Med. Imaging., vol. 30, no. 2, pp. 375–390, Feb. 2011. [3] I. Diaz, P. Boulanger, R. Greiner, B. Hoehn, L. Rowe, and A. Murtha, “An automatic brain tumor segmentation tool.” Conf Proc IEEE Eng Med Biol Soc, vol. 2013, pp. 3339–3342, 2013. [4] M. Ben Salah, I. Diaz, R. Greiner, P. Boulanger, B. Hoehn, and A. Murtha, “Fully automated brain tumor segmentation using two mri modalities,” in Advances in Visual Computing, ser. Lecture Notes in Computer Science. Springer Berlin Heidelberg, 2013, vol. 8033, pp. 30–39. [5] Y. Yang, M. N. Wernick, and J. G. Brankov, “A fast approach for accurate content-adaptive mesh generation,” IEEE Transactions on Image Processing, vol. 12, no. 8, pp. 866–881, 2003. [6] G. R. Liu and Y. Gu, An Introduction to Meshfree Methods and Their Programming. Springer, 2005. [7] M. M¨uller, R. Keiser, A. Nealen, M. Pauly, M. Gross, and M. Alexa, “Point based animation of elastic, plastic and melting objects,” in Proceedings of the 2004 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, ser. SCA ’04. Eurographics Association, 2004, pp. 141–151. [8] K. Miller, A. Wittek, and G. Joldes, “Biomechanical modeling of the brain for computer-assisted neurosurgery,” in Biomechanics of the Brain, ser. Biological and Medical Physics, Biomedical Engineering, K. Miller, Ed. Springer New York, 2011, pp. 111–136. [9] A. Klein and et al., “Evaluation of 14 nonlinear deformation algorithms applied to human brain mri registration,” NeuroImage, vol. 46, no. 3, pp. 786 – 802, 2009. [10] B. Menze and et al., “The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS),” IEEE Transactions on Medical Imaging, p. 33, 2014. [Online]. Available: https://hal.inria.fr/hal-00935640 [11] M. Jones, J. Baerentzen, and M. Sramek, “3d distance fields: a survey of techniques and applications,” Visualization and Computer Graphics, IEEE Transactions on, vol. 12, no. 4, pp. 581–599, July 2006. [12] T. N. I. Tools and R. C. (NITRC). (2011) Automatic registration toolbox. [Online]. Available: http://www.nitrc.org/projects/art/ [13] B. B. Avants, N. J. Tustison, G. Song, P. A. Cook, A. Klein, and J. C. Gee, “A reproducible evaluation of ants similarity metric performance in brain image registration.” NeuroImage, vol. 54, no. 3, pp. 2033– 2044, 2011. [14] A. Mohamed, E. I. Zacharaki, D. Shen, and C. Davatzikos, “Deformable registration of brain tumor images via a statistical model of tumor-induced deformation.” Med. Image Anal., vol. 10, no. 5, pp. 752–63, 2006.
