Incorporating low-level constraints for the retrieval of ...

Incorporating low-level constraints for the retrieval of personalised heart models from dynamic MRI Christopher Casta1 , Patrick Clarysse1 , J´erˆome Pousin2 , Jo¨el Schaerer1 , Pierre Croisille1 and Yue-Min Zhu1 1

Universit´e de Lyon, CREATIS; CNRS UMR5220; INSERM U630; INSA-Lyon; 2 Universit´e de Lyon, ICJ; CNRS UMR5208; INSA-Lyon; Universit´e Lyon 1; 69621 Villeurbanne, France.

Abstract. We have recently presented the dynamic deformable elastic template (DET) model for the retrieval of personalised anatomical and functional models of the heart from dynamic cardiac image sequences. The dynamic DET model is a finite element deformable model, for which the minimum of the energy must satisfy a simplified equation of Dynamics. It yielded fairly accurate results during our valuation process on a 45 patients cine MRI database. However, it experienced difficulties when dealing with very large thickening throughout the cardiac cycle, or on highly pathological cases. In this paper, we introduce prescribed displacements as low level constraints to locally drive the model. Non prescribed contour nodes are displaced according to a combination of forces extracted from prescribed points and image gradient. Prescribing a few points in a whole sequence allows us to retrieve much better segmentations on rather difficult cases.

1

Introduction

Retrieving personalized anatomical and functional models from clinical cardiac images remains a challenge. Magnetic resonance imaging (MRI) is a versatile imaging modality, able to provide the required data to reconstruct patient specific models. A few papers have targeted the spatio-temporal analysis of the heart function from dynamic image sequences. Montagnat proposed a dynamic framework based on simplex meshes to analyze 4D SPECT data [1], treating the temporal dimension geometrically. Sermesant proposed a bio-inspired electromechanical model of the heart designed both for the simulation of its electrical and mechanical activity, as well as for the segmentation of time series of medical images [2]. Billet extended this approach to cardiac motion recovery using the adjustment of the previous electromechanical model of the heart to cine MR images [3]. Recently, Lynch proposed a parametric motion model, using a priori knowledge about the temporal deformation of the myocardium that is embedded in a level-set scheme [4]. In a previous paper [5], we introduced the dynamic deformable elastic template model (Dynamic DET) for the automatic segmentation

II

and tracking of the heart in dynamic cine MRI sequences. This spatio-temporal approach imposes temporal smoothness and periodicity constraints to improve the regularity and continuity of the extracted contours throughout the cardiac cycle. The performance of the Dynamic DET was assessed in the context of the MICCAI 2009 LV Segmentation Challenge (http://smial.sri.utoronto.ca/ LV_Challenge/Home.html). In the present study, we propose to improve the robustness and accuracy of the obtained models by introducing prescribed displacements to some contour nodes. Attraction of the model by wrong borders can be avoided by introducing a limited number of imposed ”passage” points (which are manually designated by the user). Moreover, such point prescription could be provided by a pre-processing step, e.g. extraction of a displacement field from tagged MRI. First, the principle of the Dynamic DET model is recalled. Then, the proposed methodologies to impose prescribed displacements into Dynamic DET are introduced. In the last section, preliminary results on real pathological human MRI sequences are presented.

2

Dynamic DET model

In this section, we briefly expose the dynamic DET model which is fully detailed in [5]. 2.1

Model main equations

We assume the data is available as sequences of N -2D or -3D images, sampling the cardiac cycle. To simplify the mathematical treatment of the problem, we assume that the cardiac cycle is parameterized by t ∈ [0, 1[. The DET model is a deformable volumetric model submitted to external constraints imposed by the image. The equilibrium of the model is obtained through the minimization of an energy E which is the sum of an elastic deformation energy EElastic and energy Edata due to the action of external image forces f: Z Z 1 tr(σT ) dΩ, Edata (u) = − f (u) dγ Eelastic = 2 Ω ∂Ω where σ and  are the 3D strain and deformation tensors and Ω is the a priori model domain at rest. The a priori left ventricular (LV) model is an annulus in 2D (a half-ellipsoid in 3D)(Figure 1). The material is considered to be isotropic, homogenous and completely defined by its Young modulus and its Poisson coefficient. ∂Ω is the border of the object domain Ω. These energy terms can be approximated by discretizing the displacement u and the force f , using the finite element method (FEM). The displacement is approximated by linear functions on these elements, while the forces are sampled

III

Fig. 1. Sample ring mesh used as initial template for 2D short-axis cardiac MR segmentation tracking. at nodal points. Under this approximation, the minimum of the energy must satisfy the following equation: KU = F(U)

(1)

where K is the stiffness matrix corresponding to the response of the elastic material, and U and F are respectively the displacement and force vectors on mesh nodes. The model resulting from equation 1 is purely static. Extending this method to Dynamics [5], the heart dynamics is controlled by the simplified Dynamics equation (where acceleration is neglected): ˙ + KU = F(U, t) DU

(2)

We consider the matrix D to be a multiple of identity. It can thus be replaced by a single scalar α. 2.2

Function basis

Solutions to equation (2) do not neceassarily satisfy the periodicity and smoothness constraints. Hence, we look for solutions in a finite-dimensional subspace F generated by a set of Fourier harmonics. The frames represent a collection of uniformly-spaced noisy samples of a smooth and periodic phenomenon. Thus, we can assume the force fields Fn derived from the images to be samples of an element of F. One and only one elen ment of F satisfies F( N ) = Fn , ∀n, 0 ≤ n < N . The discrete Fourier Transform n of the F samples is defined as [6] (with N even): N/2

F(t) =

X

fl e2πilt

l=−N/2

 ∀l, 0 ≤ l < N2  dft[l], l with f = dft[l + N ], ∀l, − N2 < l < 0 and dft[l] = 1 N ∀l, l = ± N2 2 dft[ 2 ],

1 N

PN −1 n=0

Fn e

−2πiln N

IV

In order to reduce the noise impact and to enforce smoothness of the motion, either data or the computed solution are filtered. In the Fourier basis, filtering the high frequencies can be achieved by setting fl = f−l = 0 for l > a, where a is the highest admissible frequency. 2.3

Algorithm implementation

Solution to equation (2) is achieved through a pseudo-instationary process [7]. Roughly speaking, it consists in introducing a parameter τ , and considering a pseudo-instationary problem with respect to τ derived from the original problem. d + K and consider, Let’s define the operator A = α dτ 

dU dτ

= F(U) − AU U(0) = 0.

(3)

If U converges when τ → +∞, then it tends towards a limit which is a solution of the nonlinear time dependent problem. Discretizing the previous equation with finite differences to solve the temporal equation leads to (see [5] for details): 1 τ −1 α τ 1 α + + + K)Uτn = F(Uτn−1 ) + U U ∆τ ∆n ∆τ n ∆n n−1 which is a linear system and thus straightforward to solve. (

3

(4)

Displacement constraints applied to the model

In the previous section, the model was driven only by image based forces, which are generated with the following sequence of operations: – Preprocess image with a median filter (5x5 kernel). – Extract the contours with a Sobel edge detector. – Compute a Gaussian smoothed gradient of the contours (σ = 0.5) to obtain a vector field. – Ensure that forces are null on the contours by applying a Geman-McClure function to the vector norms. While this approach allows to obtain fairly accurate results in most cases, the method experiences difficulties when dealing with very large thickening throughout the cardiac cycle, or highly pathological cases (ellipsoidal shape on short axis images). Therefore, to obtain better segmentations, we introduce prescribed displacements to locally drive the model with several methods. Prescribed displacements have been used notably in parameter-free elastic deformation for 2D and 3D registrations [8]. With such a method, non prescribed contour nodes are displaced according to a combination of forces extracted from prescribed points and image gradient. In the next subsections, two methods to integrate prescribed displacements as low level constraints to the DET model are described and discussed.

V

3.1

Peckar’s Method

We consider the static case, for which the theory is simpler to apprehend. Let’s consider equation 1. The incorporation of prescribed displacements requires a modification of the previous system of equations. Let’s suppose that a known ˜ of the solution is prescribed on a fixed number of domain points (the subset U contours of the heart in our case). In the abscence of external image-based forces, the vector in the original system is set to zero. We will obtain our solution from the modified system : ˜ =F ˜ KU

(5)

˜ is a modified stiffness matrix, and F ˜ contains contributions from where K ˜ of the solution. the known subset U For example, to incorporate the prescribed displacement value u ˜i to the system, we would put u ˜i to the i th position in F, and leave the other elements of F to zero. Then, we transform the matrix K by filling its i th row with 0 and setting the element Kii to 1. Note that if all values of the solution are prescribed, ˜ = I and F ˜ = U, ˜ as we would expect. we obtain K This process is adapted to the dynamic case, and we can prescribe the value of any subset of the solution, at any given time point. Mainly, it consists in prescribing displacement of some points during initialization. These points are then considered as static points, for which the speed u˙i is equal to zero. 3.2

Payne’s Method

Here again, we consider only the static case for the theoretical explanation. The complete system of equations for equation 1 can be written :   K11 u1 + K12 u2 + ... + K1n un = Fn K21 u1 + K22 u2 + ... + K2n un = Fn  etc. If we want to fix the displacement of a node, for example ui = u ˜i , we can add a large value βI to the coefficient Kii and replace the second member of the equation by β u ˜i . If β is much larger than the other coefficients of the stiffness matrix, this modification is equivalent to replacing the initial equation by : βui = β u ˜i Thus, we have correctly fixed the displacement of the node with the required value. Moreover, the global system is still symmetric, and very few modifications to the program are necessary (see [9] for more details). This process can be extended to the dynamical scheme, and we can fix the displacement of any node at any given time point of the sequence. Mainly, the stiffness matrix K being a constant and identical at each time of the sequence, adding a large value βI to the coefficient Kii and replacing the second member of the equation (in equation 2) by our prescribed value at the given time will ensure a correct displacement of the considered node.

VI

3.3

Hybrid Methods

In the above subsection, we presented two methods to prescribe the displacement of points at selected locations. As can be seen, both methods don’t take into account external image-based forces. Therefore, the model evolution is only driven by the prescribed point displacements. This is not what we want, since the non prescribed nodes of the model must follow the contours of the myocardium. To this aim, non prescribed nodes are displaced using a combination of forces extracted from both prescribed points and image based forces. With both methods, we operate as follow : – Compute prescribed displacements, for end-diastolic (ED) and end-systolic (ES) phases, by selecting a few points. – Modify the stiffness matrix. – Interpolate the displacement of the nodes which were prescribed at ED or ES phases to the other phases, by averaging. – Compute image-based forces. With Peckar’s method, we then substitute image-based force by prescribedbased force for points which were prescribed. With Payne’s method, for points which were prescribed, we then substitute image-based force by a linear combination of image-based force and prescribedbased force. The way both methods act for dynamic segmentation of the heart will be discussed in Section 6.

4

Method parameters and User Interaction

The parameters of the algorithms can be divided into the following categories : – Initial Template: In 2D spatial dimension, the initial template is an annulus representing both endocardium and epicardium. It is defined with three geometrical parameters: the center of the annulus, the radius and the thickness. These parameters are manually defined by the user on the ED frame. The template is meshed by dividing the ring into triangles from a regular partition of the LV into sectors and concentric rings (see Figure 1). – Mechanical parameters: The Young modulus represents the rigidity of the model, while the Poisson coefficient characterizes the ability of the material to be compressed. – Algorithm parameters: The stopping criterion defines when the algorithm has converged. We can also choose the number of resolution levels with which the sequence images are processed. The number of harmonics used in Fourier filtering is usually fixed to 5. A contraction parameter (CP) has also been introduced, to initialize the template correctly at the end-systolic (ES) phase, by making the ring thicker. CP is set automatically during the initialization, based on a priori learning to

VII

automatically identify endocardial points and fit a circle at ED as well as ES phase. Finally, at the launching of the algorithm, an interactive window allows the selection of points in the images which will be prescribed using either Peckar’s or Payne’s method. It consists in choosing some outter and inner contour points of the myocardium, at ED and ES phases, in order to drive the model closer to the solution. When using Payne’s method, an additional parameter β allows to balance image-based and prescribed-based forces (see section 6).

5

Results

The experiments were conducted on 17 2D Cine MRI sequences of MICCAI’09 Segmentation Challenge [10] to study the impact on the segmentation results of prescribed displacements with both methods. The Young modulus was set to 0.15 and the Poisson coefficient to 0.2 (this is to cope with the adaptation of the initial template to the data and the myocardial area variation during the cardiac cycle, in 2D). The center of the annulus and its radius were found using a semi-automatic initialization, and CP was automatically set depending on the initialization on both ED and ES phases. The thickness was set to 8 pixels. The annulus was generated with 3 rings and divided into 20 sectors. The number of resolution levels was set to 3, and the stopping criterion to 10−5 .

Method Avg Dist I (mm) Avg Dist O (mm) Avg DM I Avg DM O No prescription 3.51 3.11 0.84 0.92 Peckar 3.41 2.67 0.84 0.93 Payne 3.10 2.07 0.86 0.95

Table 1. Mean Average perpendicular distance for the inner (Avg Dist I) and outer (Avg Dist O) contours. Mean Dice metric for the inner (Avg DM I) and outer (Avg DM O) contours, for HF-NI-15, HF-I-06, HYP-37 and N-05 datasets.

Figure 2 shows the results of a LV segmentation on a patient affected by heart failure with no ischemia (4th slice level of the HF-NI-15 dataset), at ED, ES and mid-diastolic phases, for segmentation with no prescription, and with prescribed displacements with Peckar’s and Payne’s methods. Prescription was done using 2 to 5 points for inner and outter contours, at ES and ED phases. Table 1 references Dice metric and average perpendicular distance between estimated and reference contours, for each method, averaged over 4 complicated cases belonging to different classes (heart failure with or without ischemia, hypertrophy, normal). Note that these quantitative results have been evaluated based on ED and ES phases only.

VIII

(a)

(b)

(c)

(d)

(e)

(f)

(g)

(h)

(i)

Fig. 2. DET model superimposed onto the images of HF-NI-15 sequence at a median slice of the heart, at ED (left), ES (middle) and mid-diastolic (right) phase. The pink mesh is the model. It is translucent so that correspondance between model and image contours can be appreciated. Figures (a), (b) and (c) correspond to a segmentation with no prescription. Figures (d), (e) and (f) correspond to a segmentation with Peckar’s method. Figures (g), (h) and (i) correspond to a segmentation with Payne’s method (β = 50).

IX

6

Discussion

The main advantage of our method is that it allows for the simultaneous segmentation of both endocardium and epicardium over a whole dynamic sequence of images, after the initialization of the templace at the ED phase, and optionally, the prescription of a few points. The results shown in figure 2 demonstrate the advantages of using prescribed displacements. The method experiences difficulties on highly pathological cases, as seen on subfigures (a), (b) and (c). While the segmentation is good at the ED phase, the model cannot correctly follow the epicardium at ES phase, yielding inaccurate results. Subfigures (d), (e) and (f) show the results obtained using Peckar’s method. This method ensures perfect placement of the prescribed points, but at the detriment of regularity (see the lower left part of each images). Subfigures (g), (h) and (i) give the results obtained using Payne’s method. This method leads to the best results among the three. It generates accurate results without sacrificing contour regularity. The value β presented in section 3.2 represents the balance between image-based and prescribed-based forces. If set too low, only the image-based forces will be taken into account. If set too high, prescribed-based forces will be too strong compared to image-based ones, and we will lose regularity. Using an intermediate value allows the model to be attracted correctly towards the prescribed points, while still allowing some slight displacements of prescribed nodes due to image-based forces, which results in better regularity and overall better segmentation. With values of β ranging from 0 to 500, an intermediate value of 50 appeared as a good compromise between image-based and prescribed-based forces. Using an interactive prescription method, we are not sure that the selected points are perfectly placed onto the contours of the myocardium, hence it is necessary to allow prescribed nodes to be slightly displaced. For this reason, Payne’s method appears to be the most suited in the case of segmentation of Cine MRI image sequences.

7

Conclusion

We have presented an improved dynamic DET, for the segmentation and tracking of the heart from MRI sequences, and more generally, for the analysis of soft deformable structures in periodic motion. Experiments on rather difficult cases show a very good overall ability of the model to track the heart borders using a few prescribed points to improve accuracy and robustness. Payne’s method is the best one to extract accurate contours. Based on these results, one could use motion information from MRI tagging to drive the deformation. Finally, extending DET model to track heart borders and motion in 3D, from both Cine and tagged MR image sequences, is very appealing. It should be noted that such extension does not require any further theoretical developments, since the equations remain valid for the 3D case. However, 3D MR image processing poses a number of purely technical problems, such as inter-slice alignement, 3D interpolation and data visualization. This is the purpose of our future work.

X

Acknowledgements This work has been supported by the R´egion Rhˆones Alpes through the Simed project of cluster ISLE, as well as the CNRS-GDR STIC-Sant´e through the support of the MedIEval action (Medical Image Evaluation). It is also part of the French ANR (http://www.agence-nationale-recherche.fr/) projects GWENDIA and VIP.

References 1. J. MONTAGNAT and H. DELINGETTE, 4D deformable models with temporal constraints: application to 4D cardiac image segmentation. Medical Image Analysis, vol. 9, no. 1, p. 87-100, 2005. 2. M. SERMESANT, H. DELINGETTE and N.AYACHE, An Electromechanical Model of the Heart for Image Analysis and Simulation. IEEE Transactions in Medical Imaging, vol. 25, no. 5, p. 612-625, 2006. 3. F. BILLET, M. SERMESANT, H. DELINGETTE and N. AYACHE, Cardiac Motion Recovery and Boundary Conditions Estimation by Coupling an Electromechanical Model and Cine-MRI Data. Functional Imaging and Modeling of the Heart, vol. 5528 of LNCS, p. 376-385, 2009. 4. M. LYNCH, O. GHITA and P.F. WHELAN, Segmentation of the Left Ventricle of the Heart in 3D+t MRI Data Using an Optimised Non-Rigid Temporal Model. IEEE Transactions in Medical Imaging, vol. 27, no. 2, p. 195-203, 2008. 5. J. SCHAERER, C. CASTA, P. CLARYSSE, Y. ROUCHDY and J. POUSIN, “A Dynamic Elastic Model for Segmentation and Tracking of the Heart in MR Image Sequences”, Medical Image Analysis, in press. 6. A. V. OPPENHEIM, R. W. SCHAFER and J. R. BUCK, Discrete-time signal processing (2nd ed.). Upper Saddle River, NJ, USA : Prentice-Hall, Inc., 1999. 7. P. G. CIARLET and J.-L. LIONS, Handbook of Numerical Analysis, Volume 1, Finite difference methods, Solution of equations in Zn .N orth − Holland, 1990. 8. W. PECKAR, C. SCHNORR, K. ROHR and H.S. STIEHL. ”Parameter-Free Elastic Deformation Approach for 2D and 3D Registration Using Prescribed Displacements.”, Journal of Mathematical Imaging and Vision, vol. 10, no. 2, p. 143-162, March 1999. 9. O. C. ZIENKIEWICZ, R. L. TAYLOR and J. Z. ZHU, The Finite Element Method : Its Basis and Fundamentals. Butterworth-Heinemann, 2005. 10. P. RADAU, Y. LU, K. CONNELLY, G. PAUL, A.J. DICK, G.A. WRIGHT. “Evaluation Framework for Algorithms Segmenting Short Axis Cardiac MRI.”, The MIDAS Journal - Cardiac MR Left Ventricle Segmentation Challenge, http: //hdl.handle.net/10380/3070