Link¨oping Studies in Science and Technology. Dissertations No. 917
Automated Feature Detection in Multidimensional Images
Einar Heiberg
Departments of Biomedical Engineering & Medicine and Care Center for Medical Image Science and Visualization
Link¨oping December 2004
Cover: The cover image shows a vortex ring in green behind the mitral valve. It is detected by the pattern matching approach presented in Paper II in the thesis. The flow is depicted by particle trace technique and shows flow through the left ventricle. The blue wire-frame structure shows a segmentation of the left ventricle derived with the technique described in Paper V. See Figure 31 for more details.
Printed by: UniTryck, Link¨oping, Sweden ISBN 91-85297-10-0 ISSN 0345-7524 Distributed by: Link¨oping university Department of Biomedical Engineering SE-581 85, Sweden c 2004 Einar Heiberg
No part of this publication may be reproduced, stored in a retrieval system, or be transmitted, in any form or by any means, electronic, mechanic, photocopying, recordning, or otherwise, without prior permission of the author.
Abstract Manual identification of structures and features in multidimensional images is at best time consuming and operator dependent. Feature identification need to be accurate, repeatable and quantitative. This thesis presents novel methods for automated feature detection in multidimensional images that are independent on imaging modality. Feature detection is described at two abstraction levels. At the first low level the image is regionally processed to find local or regional features. In the second medium level results are taken from the low level feature detection and grouped into objects or parts that can be quantified. A key to quantification of cardiac function is delineation of the cardiac walls which is a difficult task. Two different methods are described and evaluated for delineation of the left ventricular wall from anatomical images. The results show that semi-automatic delineation is a huge time saver compared to manual delineation. To obtain a robust results as much a priori and image information as possible should be used in the delineation process. Regional cardiac wall function is further studied by deriving and analyzing strain-rate tensors from velocity encoded images. For flow encoded images novel methods to find regional flow structures such as vortex cores, flow based delineation, and flow quantification are proposed. These methods are applied to study blood flow in the human heart, but the techniques outlined are general and can be applied to a wide array of flow conditions.
iii
List of Papers This thesis is based on the following papers, which will be referred to by their Roman numerals: I.
E. Brandt, L. Wigstr¨om, B. Wranne, Segmentation of Echocardiographic Image Sequences Sequences Using Spatio-temporal Information, In proceedings of MICCAI 99, vol. 1679 of Lecture Notes in Computer Science, (Cambridge, UK), pp. 410-419, Springer Verlag, ISBN 3-540-66503-X, 1999.
II.
E. Heiberg, T. Ebbers, L. Wigstr¨om, and M. Karlsson, Three Dimensional Flow Characterization using Vector Pattern Matching, IEEE Transactions on Visualization and Computer Graphics 9(3) pp. 313319, 2003.
III.
P. Selskog, E. Heiberg, T. Ebbers, L. Wigstr¨om, and M. Karlsson, Kinematics of the heart: strain-rate imaging from time-resolved three-dimensional phase contrast MRI, IEEE Transactions on Medical Imaging, vol. 21(9), pp. 1105-1109, 2002.
IV.
E. Heiberg, L. Wigstr¨om, M. Carlsson, A.F. Bolger, M. Karlsson, Time Resolved Three-dimensional Segmentation of the Left Ventricle in Multimodality Cardiac Imaging, Submitted.
V.
E. Heiberg, A.F. Bolger, M. Karlsson, Flow Quantification from Time Resolved Three Dimensional Phase Contrast MRI, Submitted.
v
Preface This thesis is accompanied with a CD-ROM containing the thesis as a searchable pdf-file and the figures in color. It also contains software developed in the work with this thesis. Furthermore it contains movies and rotatable 3D-images to better explain the time-resolved and three dimensional structures of the results. Extended figures are indicated with the symbols (three dimensional rotatable image) and (movies) in the thesis. Enjoy! This work was supported by the following grants: • Swedish Medical Research Council (MFR), Pressure and flow in heart and vessels, B. Wranne, (K1998-14X-009481-08B, K1999-71X-009481-09C, K2001-1X-09481-11B). • Swedish Heart and Lung foundation, Base grant for Department of Medicine and Care, B. Wranne, (1998-41604, 2000-41688, and 2000-41454) • Swedish Technical Research Council (TFR), Non-invasive strain mapping of the human heart using time-resolved 3D phase contrast magnetic resonance imaging, M. Karlsson, (299-99-477). • Center for Industrial Information Technology (CENIIT, Link¨opings universitet), Efficient methods for parameter estimation from velocity data, M. Karlsson, (99.11) • Swedish Research Council (VR). Automated segmentation and flow characterization in cardiovascular MRI. T. Ebbers, E. Brandt. (2001-3493). • Swedish Heart and Lung foundation. Noninvasiv m¨atning av tryckskillnader i hj¨arta och k¨arl. J. Engvall. (200041688). • Swedish Heart and Lung foundation. P˚averkas remodellering i hj¨arta och k¨arl av tryckf¨orh˚allandena. J. Engvall (200141745). • Swedish Heart and Lung foundation. Research months (3+3). E. Heiberg. (20040206, 20041197). • Stina och Birjer Johanssons forskningsstiftelse. Automatiserad segmentering av v¨anster kammare fr˚an MR-bilder. E. Heiberg. ¨ • Landstinget i Osterg¨ otland. Regional funktionsbed¨omning av v¨ansterkammare fr˚an MR bilder.
vii
Contents Abstract
iii
List of Papers
v
Preface
vii
Contents
ix
1
Multidimensional Images
1
2
Rationale of the Thesis
3
3
Aim of the Thesis
5
4
Scaling Temporal Dimensions
7
5
Low Level Image Features 5.1 Requirements of local image descriptors . . . . . . . . . . . . . .
9 9
6
Local Tensor Description 6.1 Interpretation of the tensor representation . . . . . . . . . . . . . 6.2 Generalization to tensor fields . . . . . . . . . . . . . . . . . . . 6.3 Invariants and tensor mappings . . . . . . . . . . . . . . . . . . .
11 13 17 19
7
Low Level Feature Detection 7.1 Low level feature detection in solids . . . . . . . . 7.1.1 Extracting objects . . . . . . . . . . . . . . 7.1.2 Analyzing material properties . . . . . . . 7.2 Low level feature detection in fluids . . . . . . . . 7.2.1 Classical local descriptors for vector fields 7.2.2 Vector filters for pattern matching . . . . .
23 23 24 26 27 27 28
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
ix
CONTENTS 8
9
Medium Level Feature Detection 8.1 Medium feature detection in solids 8.1.1 Segmentation approaches . 8.1.2 Boundary tracing . . . . . 8.1.3 Deformable models . . . . 8.2 Medium feature detection in fluids
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
31 31 31 33 34 35
Results 9.1 Feature detection in solids . . . . . . . 9.1.1 Low level feature detection . . . 9.1.2 Medium level feature detection . 9.2 Feature detection in fluids . . . . . . . . 9.2.1 Low level feature detection . . . 9.2.2 Medium level feature detection .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
37 37 37 39 49 49 54
. . . . .
. . . . .
10 Discussion and Future Work
59
Bibliography
61
Appended Papers
71
Paper I
71
Paper II
83
Paper III
93
Paper IV
101
Paper V
123
List of Figures
139
List of Tables
143
List of Equations
145
Notation and Abbreviations
147
Acknowledgments
151
x
Chapter 1 Multidimensional Images Multidimensional image volumes can be of different kinds. Examples of a three dimensional image are a MR (Magnetic Resonance) images of the vertebrae (3D) or a stack of time resolved two dimensional images of the heart (2D+T). Four dimensional images could be time resolved images (3D+T), for instance of the human heart. One could even perceive five dimensional images consisting of 3 spatial and two temporal dimensions, where the two temporal dimensions are cardiac phase and respiratory phase respectively [1]. Traditionally an image has been a set of image elements, where each image element was a scalar. Mathematically such an image can be viewed as a scalar field or a tensor field of zero order tensors. With the advances of imaging modalities this is changing. Using MRI or ultrasound it is possible to encode the images by velocity using phase contrast imaging [2] or employing the Doppler effect [3], respectively. With ultrasound Doppler technology, only one velocity component can be acquired (along the propagation direction of the sound beam) [4], [5], while using MRI all three velocity components in a three dimensional volume can be acquired [6], [7]. Three dimensional temporally resolved flow measurements are becoming increasingly important for understanding the complex intraventricular flow and examples of works using particle trace technique for visualization and understanding are [8, 9, 10, 11, 12, 13]. Mathematically, velocity encoded images can be viewed as a vector field or a tensor field of first order tensors.
1
CHAPTER 1. MULTIDIMENSIONAL IMAGES Using MRI it is also possible to directly measure the diffusion of water in the human body [14]. Diffusion can be represented as second order tensors. Other examples where one want to study tensor fields are tensor fields that are derived from velocity data. Examples of that is given in Paper III, where strain-rate is calculated from phase contrast velocity measurements. Another type of derived tensors is strain [15].
2
Chapter 2 Rationale of the Thesis Manual identification of structures and/or features in multidimensional images is at best time consuming and operator dependent. The human vision system is a masterpiece in terms of pattern recognition in noisy and otherwise complex environments. For multidimensional images, however, feature detection might be very difficult since humans have obvious difficulties of looking at patterns in multidimensional spaces. This is partly due to limitations of visualization devices and difficulties to construct user interfaces. Feature identification and subsequent quantification need to be accurate, repeatable and quantitative in a meaningful way. There is a rapid development in imaging techniques in terms of spatial and temporal resolution. This have two effects in the clinical routine. Firstly, it is not possible to manually interpret all these images to find interesting structures. Secondly, it may also be so that one do not want to see the whole image volume, only certain structures or subsets of the complete volume. Feature detection in multidimensional images is a process that needs to be applied to different abstraction levels to be successful. First a low level feature detection scheme is applied that finds local/regional structures. The result from the low level feature detection is taken as an input to a medium level feature detection. In this step the local information is gathered to distinct objects or parts that can be labeled, described, and quantified. Finally one can perceive a high level step where the results from the quantification is taken as input and compared with normal values and put into a context so that the high level can work as decision support or even as an autonomous system. When imaging solid materials (such as muscles or soft tissue) the most important medical application is detection and delineation of anatomical structures. Examples are finding the cardiac walls which enables quantification of several clini3
CHAPTER 2. RATIONALE OF THE THESIS cally important quantities such as end-systolic and end-diastolic volume, ejection fraction [16], [17], [18], and myocardial mass [19], [18]. Successful delineation enables direct quantification of regional asynchronicity and regional delay [20] given sufficient temporal resolution which has been shown to be difficult to perceive visually [21]. A delineation of the myocardium also enables the possibility to automatically calculate intra-cardiac pressure fields [22], and it is a necessary preprocessing step for calculation of strain in the human heart [23], [24], [25]. Detecting and delineating anatomical structures is not limited to the cardiovascular examples given above, it arise in all parts of the human body. For velocity encoded images such as velocity encoded images for instance, the situation is more difficult since it is harder to manually find features and it is even more time-consuming, mainly due to the complexity of adequate visualization. Medical applications arise typically in the cardiovascular system where one would like to quantify flow conditions, and detect flow features such as vortices. Compared to delineation of anatomical structures where the basic anatomy is also known, far less is known about the expected features for flow feature detection. Furthermore the temporal behaviors of the flow show more topological variation, than the temporal variations of spatial structures such as the heart chambers. The number of possible structures or topologies are also higher in vector fields. One of medical tensor images is diffusion tensor from the brain. The diffusion tensor shows in what direction the water can diffuse with least resistance, and in the brain this is along the fiber tracts [26]. Tensor images may also be derived from vector images such as for instance with strain rate as discussed in Paper III, or variants of velocity gradient tensors, see Section 7.2.1. For tensor images the visualization is even harder than for vector images. Typical tensor visualization is based on glyphs [27], variants of line integral methods [28] or combinations thereof [29]. Visual occlusion is one of the main problem, and therefore it is particularly important to develop feature detection algorithms for tensor fields, since it is very difficult to manually find structures guided by current visualization techniques.
4
Chapter 3 Aim of the Thesis Much is still left to learn about how the human heart works, and especially the relation between wall motion and flow. To study this, new quantitative methods to study both wall function and motion as well as cardiac flow are needed. This thesis aims to develop and improve methods for automated feature detection for multidimensional scalar, vector and tensor images. The goal has been to create methods and tools that are applicable to the clinical problems of today since the data sets and the clinical need is already here. More specifically the aim of the thesis was to: • Develop robust segmentation algorithms for the left ventricle that are applicable to a wide range of image data. • Develop tools to quantify global and regional left ventricular function. • Develop methods to locate and identify flow structures in measured velocity fields. • Develop methods to quantify blood flow in the human heart. The data sets keeps getting bigger with better spatial, and temporal resolution. It is a big challenge to keep up with this pace for the image processing community.
5
Chapter 4 Scaling Temporal Dimensions A scaling problem arise in temporally resolved images, such as 2D+T or 3D+T. Note that 2D+T images are different from spatially true 3D images. The scale that links a temporal dimension to a spatial dimension needs to be established: — ’How many meters is one second?’ We need a reference velocity, in [m/s] that can link a temporal a dimension in [s] to a spatial dimension [m]. To link space-time to a spatially 4D space, Einstein used the speed of light as a reference velocity. In medical images the linking velocity needs to be established in experiments, by experience or by use of common sense. One approach to get an initial guess is to establish a reference length and a reference time. Thus, the reference velocity is then given by the ratio between the reference length and the reference time. Finding reference length or reference time might be easier when the temporal is cyclical (such as the heart cycle). One method using the fact that we have a cyclic temporal dimension for establishing the linking velocity is presented in Paper I, where the reference length was chosen as the total excursion of a point in the image, and as reference time the total length of the heart beat. For time resolved vector spaces the situation is even more complicated, since if we map the temporal dimension to a spatial dimension, then we have a four dimensional space with three dimensional vectors. Mathematical difficulties may arise, for instance that the velocity Jacobian is not a square matrix and cannot be interpreted in terms of its eigenvalues. Therefore in this thesis time resolved vector fields are treated as a series of three dimensional ’snap-shots’. A completely unresolved question is how to approach this when there are two1 temporal dimensions as proposed by Sigfridsson, et al. [1].
1
One could argue that this is not a true 3D+T+T data set, it is merely a 3D+T where the temporal dimension have two different information propagation velocities. Nevertheless, to analyse this data satisfactory it is necessary to break it down to a 5D data set.
7
Chapter 5 Low Level Image Features An image is very rich on information, and low level feature detection is the process of extracting locally interesting features. Consider a small neighborhood in an image volume, say 5x5x5 voxels. This small region lies on a 625 dimensional manifold, and is theoretically very rich on information, and it gets even worse with vector or tensor worth images. If each voxel would be binary, then the number of possible patterns in that neighborhood would 2625 ≈ 10188 which is an incredibly large number, compare with the estimated number of particles in the universe is around 1080 . Fortunately, most of the possible patterns or parts of the manifold is neither interesting nor common. Designing low level feature detection is about deciding on a suitable representation to approximate and describe the interesting local neighborhoods.
5.1
Requirements of local image descriptors
There are three properties that is important for local image descriptors [30]. Local representations should ideally be unique, equivariant, and invariant (at least to some degree). Examples and interpretation of these three properties is illustrated in Table 1.
9
CHAPTER 5. LOW LEVEL IMAGE FEATURES
Uniqueness Representing orientation with normal vectors does not work. Invariance Change in scale should not be reflected in the representation. Equivariance Change in orientation have to be reflected proportionally in the representation.
6=
=
∝
Table 1: Uniqueness, invariance and equivariance of local orientation representations
If the representation does not meet the uniqueness requirement, then the representation can not be smoothed, which is a big disadvantage. Equivariance is important for quantification, if the changes is not proportional, then subsequent quantification is impossible. Invariance is application dependent, but in most applications the feature detection should not be too sensitive to the local scale.
10
Chapter 6 Local Tensor Description This section describes a local neighborhood description that can be used for both scalar, vector, and tensor worth images. The local orientation tensor description was first presented by Knutsson [31]. The neighborhood description features both local topology, and local orientation. I will start the description with scalar fields, and the show how it can be generalized and extended for vector, and tensor fields. The assumption with this neighborhood representation is that the neighborhood is simple [30]. This local tensor orientation fulfills all three requirements on local image descriptors (the invariance to a certain degree). The local orientation tensor estimate is found by using quadrature filters. Quadrature filters are filters with a zero transfer function in one half plane of the frequency domain [30]. The normal vector of that plane is said to be the orientation of the quadrature filter. The quadrature filter is a complex filter and its real part corresponds to a ’line’ detector and its imaginary part to an ’edge’ detector. A 2D-quadrature filter is illustrated in Figure 1. The quadrature filters are orientation sensitive so that the magnitude of the filter response q is given by:
|q| ∝ (ˆ nsˆ)2
(1)
where q is the quadrature filter response, n ˆ is the direction of the quadrature filter, and sˆ is the direction/orientation of the local neighborhood. Directly from the quadrature filter responses, the tensor description proposed by Knutsson, et al. [31] can be constructed [30]: Te (¯ r) =
X
|qk (¯ r )| αˆ nk n ˆ Tk − βI
(2)
k
11
CHAPTER 6. LOCAL TENSOR DESCRIPTION
a)
b)
c)
Figure 1: Quadrature filters in 2D. a) Quadrature filter in the Fourier domain. b) Real part of quadrature filter, corresponding to a ’line’ detector. c) Imaginary part of quadrature filter corresponding to an ’edge’ detector.
where T is the local orientation tensor, r¯ show the spatial dependency, qk is the output from quadrature filter k with the direction given by n ˆ k , N is the number of filters required, I is the identity tensor. The parameters N , α, and β depends on the dimensionality, and is given below. The tensor representation can be seen as an ’interpolation’ of the filter responses to a representation that keeps both magnitude information (i.e., how similar the neighborhood is to the filter) and orientation information. The ’interpolation’ is done over a quadratic surface produced by summation of outer products of the filter responses. Details on how this local orientation tensor can be interpreted is given in Section 6.1. The filters in Equation 2 needs to be distributed evenly over a half-sphere. The minimum number of required filters are three in 2D, six in 3D space and twelve in 4D space [30]. Non-even filter distribution can be compensated for, using a process called tensor whitening [32]. In two dimensions the filter directions should point to the vertices of a hexagon, illustrated in the left panel of Figure 2. The coefficients should be N = 3, α = 34 , and β = 13 . One possible filter distribution in the two dimensional case is: n ˆ 1 = (1, 0) √ n ˆ 2 = ( 12 , √ 3/2) n ˆ 3 = (− 21 , 3/2)
(3)
In three dimensions an even filter distribution is achieved if the filters are oriented in a direction between origo and a vertex on a hemi-icosahedron (one of the platonic bodies), illustrated in the right panel of Figure 2. The coefficients should be N = 6, α = 45 , β = 41 . One possible realization of the filter distribution for the 12
6.1. INTERPRETATION OF THE TENSOR REPRESENTATION three dimensional case is: n ˆ 1 = c(a, 0, b) n ˆ 2 = c(−a, 0, b) n ˆ 3 = c(b, a, 0) n ˆ 4 = c(b, −a, 0) n ˆ 5 = c(0, b, a) n ˆ 6 = c(0, b, −a)
a=2 √ b = (1 + 5) √ c = (10 + 2 5)−1/2
2D
(4)
3D
Figure 2: Filter distributions in 2D and 3D, the direction of the filters are given as vectors between origo and vertices on the platonic bodies. Left:hexagon which gives the filter directions in 2D. Right: The icosahedron, which gives the direction of the filters in 3D. The four dimensional case is given by Granlund, et al. [30].
6.1
Interpretation of the tensor representation
This section describes one possible interpretation on the local orientation tensor representation. An analytical derivation of the expression for the local orientation tensor for a set of more or less idealized neighborhoods for the three dimensional cases, is given to describe how the local orientation tensor representation can be interpreted. Consider three different idealized local topology cases shown in Figure 3. The first case Figure 3a) is a neighborhood with one unique orientation sˆ. Topologically, this would correspond to a ’planar’ neighborhood, with only one dominating 13
CHAPTER 6. LOCAL TENSOR DESCRIPTION
a)
b)
c)
Figure 3: Different cases of local topology in a three dimensional scalar field. a) Planar neighborhood. b) Linear neighborhood. c) Isotropic neighborhood.
orientation (across the plane), see Figure 3a. The filter response will in this idealized case be given by:
qk (¯ r ) = (ˆ nk · sˆ s)2
(5)
where s is a scalar that tells how well the local frequency in the neighborhood matches the filter (similarity). For simplification of the analytical expressions this will be introduced in the orientation vector as s = sˆ s = (x, y, z)T , and sˆ will denote the normalized orientation vector, where |ˆ s| = 1. Now the tensor can analytically be derived by combining Equation 2, 4, and 5: 14
T11 = 1/2c2 (5b4 c2 − a2 − b2 + 5a4 c2 )x2 + ... 1/2c2 (−b2 − a2 + 5b2 c2 a2 )y 2 + ... 1/2c2 (−b2 − a2 + 5b2 c2 a2 )z 2 = x2 T12 = 5c4 a2 b2 xy = xy T13 = 5c4 a2 b2 xz = xz T21 = 5c4 a2 b2 xy = xy T22 = 1/2c2 (−b2 − a2 + 5b2 c2 a2 )x2 + ... 1/2c2 (−b2 + 5b4 c2 − a2 + 5a4 c2 )y 2 + ... 1/2c2 (−b2 − a2 + 5b2 c2 a2 )z 2 = y 2 T23 = 5c4 b2 a2 yz = yz T31 = 5c4 a2 b2 xz = xz T32 = 5c4 b2 a2 yz = yz T33 = 1/2c2 (−a2 + 5a2 c2 b2 − b2 )x2 + ... 1/2c2 (−a2 + 5a2 c2 b2 − b2 )y 2 + ... 1/2c2 (−b2 + 5b4 c2 − a2 + 5a4 c2 )z 2 = z 2
(6)
6.1. INTERPRETATION OF THE TENSOR REPRESENTATION where a, b, c are given in Equation written as : 2 x T = xy xz
4. The tensor Tij can in matrix notation be xy xz y 2 yz = sT · s yz z 2
(7)
where s = (x, y, z) = sˆ s is the orientation of the structure multiplied with a scalar s that describes how similar the filters are to the local neighborhood. The properties of the tensor T can be investigated in terms of its eigenvalues and eigenvectors. The characteristic equation of the tensor in Equation 7 is: λ3 − s 2 λ2 = 0
(8)
where s2 = x2 +y 2 +z 2 . From this we can directly see that the tensor has only one non-zero eigenvalue, λ1 = s2 , and its corresponding eigenvector eˆ1 is sˆ, since: 3 x x2 xy xz x + xy 2 + xz 2 Tˆ s = 1s Ts = 1s xy y 2 yz y = 1s yx2 + y 3 + yz 2 = z zx2 + y 2 z + z 3 xz yz z 2
(9) 2 (x + y 2 + z 2 )x = 1s (x2 + y 2 + z 2 )y = (x2 + y 2 + z 2 )z
x (x2 +y 2 +z 2 ) y = s z
s2 sˆ s= s
λ1 sˆ
In this analytical case with one unique orientation in the neighborhood, the tensor has one non-zero eigenvalue and the corresponding eigenvector is the orientation sˆ. Remember that if v ˆ is an eigenvector to a second order tensor, then −ˆ v is also an eigenvector. We can only expect Equation 1 to hold when the input data is an exact (but rotated) copy of the filter. What happens if the neighborhood is not that simple, but instead consists of a linear combination of two orientations. Topologically this corresponds to a linear neighborhood, see Figure 3b. For this case the input is: qk (¯ r ) = (ˆ nk · s1 )2 + (ˆ nk · s2 )2
(10)
x21 + x22 y1 x1 + y2 x2 z1 x1 + z2 x2 y12 + y22 z1 y1 + z2 y2 T = x1 y1 + x2 y2 x1 z2 + x2 z2 y1 z1 + y2 z2 z12 + z22
(11)
The tensor T will be:
15
CHAPTER 6. LOCAL TENSOR DESCRIPTION where (x1 , y1 , z1 ) = s1 sˆ1 = s1 , and (x2 , y2 , z2 ) = s2 sˆ2 = s2 . The tensor has the characteristic equation: λ3 − λ2 (s21 + s22 ) + λ |s1 × s2 |2 = 0
(12)
Observing that λ3 = 0, and |s1 × s2 | = |s1 | |s2 | |sin(θ)|, where θ is the angle between sˆ1 and sˆ2 we obtain: s 2 s21 + s22 s21 + s22 λ1,2 = (13) − |s1 |2 |s2 |2 sin2 (θ) ± 2 2 If the two orientations sˆ1 , and sˆ2 are orthogonal, then the eigenvalues can be simplified as: λ1 = s21 ,
λ2 = s22
(14)
and the corresponding eigenvectors will be eˆ1 = sˆ1 , and eˆ2 = sˆ2 respectively. Note that if the eigenvalues are equal and the two structures are orthogonal, then any vector in the plane spanned by s1 , and s2 is an eigenvector to the tensor, and therefore eigenvector decomposition is not necessarily unique. For nonorthogonal structures the eigenvectors will be an orthogonalized system, of the two orientations, since the tensor T is real and symmetric. The eigenvalues of the tensor are given by Equation 13. For the special case when the eigenvalues are equal, then the corresponding eigenvectors can be written as: eˆ1 =
sˆ1 + sˆ2 , |ˆ s1 + sˆ2 |
eˆ2 =
sˆ1 − sˆ2 |ˆ s1 − sˆ2 |
(15)
The third case is an isotropical neighborhood, illustrated in Figure 3c). The tensor will in this case be:
x21 + x22 + x23 y1 x1 + y2 x2 + y3 x3 z1 x1 + z2 x2 + z3 x3 y12 + y22 + y32 z1 y1 + z2 y2 + z3 y3 T = x1 y1 + x2 y2 + x3 y3 x1 z2 + x2 z2 + x3 z3 y1 z1 + y2 z2 + y3 z3 z12 + z22 + z32
(16)
where (x1 , y1 , z1 ) = s1 sˆ1 = s1 , (x2 , y2 , z2 ) = s2 sˆ2 = s2 , and (x3 , y3 , z3 ) = s3 sˆ3 = s3 . The tensor has three non-zero eigenvalues. To summarize the interpretation of the tensor representation, the beauty of the representation, is the ability to hold several hypothesis of neighborhood orientation, eˆ1 , eˆ2 , and eˆ3 , with the certainty given by λ1 , λ2 , and λ3 , respectively. The ability to hold several hypothesis enables us to also classify the topology of 16
6.2. GENERALIZATION TO TENSOR FIELDS neighborhood. The eigenvalues and their corresponding eigenvectors of the local orientation tensor contain information about the type of neighborhood. The three possible topologies in the three dimensional case is illustrated in Figure 3. In three dimensions (3D) the neighborhood is considered to be ’planar’ if we have only one large eigenvalue (λ1 λ2 , λ3 ), ’linear’ when there are two larger eigenvalues (λ1 ≈ λ2 λ3 ), and finally when all three are approximately equal (λ1 ≈ λ2 ≈ λ3 ), an ’isotropic’ neighborhood. This is further discussed in Section 6.3. Another interesting analytical case is to study if the filter response is biased with a DC term, i.e., if the filter responses are: qk (¯ r ) = (ˆ nk · s1 )2 + DC Then the tensor T will be: 2 x + DC xy xz y 2 + DC yz T = xy 2 xz yz z + DC
(17)
(18)
which has the eigenvalues λ1 = (x2 + y 2 + z 2 ) + DC, λ2 = DC, λ3 = DC with the same eigenvector as in Equation 9. It is therefore of great importance to design filters such that the DC components are avoided.
6.2
Generalization to tensor fields
The same local orientation tensor representation that was used to represent local orientation for the scalar fields can be generalized to vector and tensor fields. In scalar fields the term ’local orientation’ is intuitively interpreted as variations in space of the field (frequency and phase). This is closely related to the common approximation that a field locally can be described as a simple function. In vector fields there are two possible interpretations of the term ’local orientation’. In this thesis the term ’local orientation’ in vector fields will be used to describe linear combination of energy in each vector or tensor component, i.e, the orientation of the quantity of each point of the field. This interpretation of orientation is not the one that is common in image processing literature, but it is more convenient when using vector fields describing flow velocities, or other physical quantities. The meaning of the term local orientation (as used in this thesis) in the 17
CHAPTER 6. LOCAL TENSOR DESCRIPTION two cases, scalar fields and vector fields respectively is illustrated in Figure 4.
Figure 4: Difference of meaning of the term ’local orientation’ in vector and scalar fields. Left: Scalar field with a vertical orientation (the signal is constant on the vertical axis). Right: Vector field with a horizontal orientation, however the energy of the field is spatially variating (although, constant on the vertical axis).
The tensor representation for local orientation can be extended for neighborhood representation for general tensor fields. Instead of constructing the tensor representation from orientation sensitive quadrature filters, other feature sensitive filters can be constructed. The feature sensitivity can be sensitivity to edges or borders or sensitivity for specific vector structures, such as feature detection in flow fields. This idea that was originally presented in Paper II. The idea of using convolution with vector patterns combined with Clifford algebra has actually give rise to a whole new opening in the research field for analyzing vector fields [33],[34]. The tensor representation can be constructed as a direct summation of a linear combination of outer products of each filter direction: T(¯ r) =
N X
f (hk (¯ r )) αˆ nk n ˆ Tk − βI
(19)
k=1
where T is now a similarity tensor (rather than a local orientation tensor), r¯ show the spatial dependency, N is the number of filters required, f is a function of the filter responses hk (¯ r ), I is the identity tensor, n ˆ k is the direction of the filters. The filters need to be orientation sensitive so that the function of the filter response at the center of a structure is given by: f (hk (¯ r )) = (ˆ nk · sˆ)2
(20)
where n ˆ k is the direction of the filter k and sˆ is symmetry axis of the structure. This should be interpreted as the angular sensitivity for desired structures should 18
6.3. INVARIANTS AND TENSOR MAPPINGS be proportional to the cosine squared between the structure and the filter. Since the scalar product used in vector worth convolution is linear then it is not possible to construct vector worth filters that has the directional sensitivity as the square of the dot product between the orientation of the filter and the vector field, as Equation 20. We can, however use the tensor representation in Equation 2 if we define the function f as the square, which yields: T(¯ r) =
N X
h2k (¯ r ) αˆ nk n ˆ Tk − βI
(21)
k=1
Using the other definition where orientation in vector fields is defined as local changes of energy in the vector components then the local feature tensors should be constructed as [30]: T(¯ r) =
!1/2
N X
X
k=1
mn...o
2 qkmn...o (¯ r)
αˆ nk n ˆ Tk − βI
(22)
where qkmn...o denotes the filter response by filtering the components of vector or tensor field fmn...o with quadrature filters.
6.3
Invariants and tensor mappings
Another approach to analyzing tensor fields directly is to find means to simplify the tensor, and find mappings of the tensor that describes tensor in a simpler manner. What type of mapping that is necessary is of course very application dependent. Note that in general decomposing the tensor or calculating invariants is not a linear process, and very often the uniqueness property that was described in Section 5.1 is lost. A tensor invariant is a mapping of the tensor that is independent of change of coordinate system. Examples of such invariants are the eigenvalues λ (not the eigenvectors of course). Since λ1 + λ2 does not change at a change of coordinate system it is therefore also an invariant, in fact any non-zero function of one or more invariants is also an invariant. There are therefore an infinite number of possible invariants. A number of common invariants and their practical application is given in Table 2. Some of the invariants have names and notation in the literature. For instance the invariants In are called the ’principal invariants’. One does not need to be so radical to map the tensor to a scalar, one could instead decompose the tensor into parts, where each part is easier to interpret. A nice 19
CHAPTER 6. LOCAL TENSOR DESCRIPTION overview of tensor decompositions is given by Boring [35]. A tensor decomposition decompose the tensor into two or more tensors whose sum or product is the original tensor, but where each part have specific properties. An overview of possible decompositions is given in Table 3. Finally another possible general interpretation of a tensor T is to analyse it’s geometric properties. Like the planar geometry of vectors, 2nd-order tensors have a natural geometry in the form of quadratic surfaces derived from: sˆTˆ sT = 1
(23)
where sˆ = [x y z], and the tensor T. Possible quadratic surfaces are real or imaginary ellipsoids, real or imaginary elliptical cylinders, one or two sheet hyperboloids, hyperbolic cylinders, and finally, real or imaginary parallel planes. Elliptic cones, elliptic paraboloids and hyperbolic paraboloids do not occur since there are no strictly linear terms of x, y, z. The type of geometry that the quadratic form exhibit is given by their eigenvalues.
20
6.3. INVARIANTS AND TENSOR MAPPINGS Definition
Interpretation
λ1
Largest eigenvalue (without sign). For strainrate tensor it represents instantaneous expansion/compression in the direction where the most deformation takes place.
I1 = λ1 + λ2 + λ3 = tr(T)
Sum of all eigenvalues, and also the trace of the tensor. For strain-rate tensor the trace should be zero if the mass is conserved.
I2 = λ1 λ2 + λ2 λ3 + λ3 λ1
Sum of the cross sectional areas of three orthogonal slices defined by any two of the principle axes.
I3 = λ1 λ2 λ3 = det(T)
Product of all eigenvalues, and also the determinant of the tensor. For strain tensor this is equivalent to the mass of a the element.
λ1 −λ2 λ1
Linear measure, see Section 7.1.1, and [36].
λ2 −λ3 λ1
Planar measure, see Section 7.1.1, and [36].
λ3 λ1
Spherical measure, see Section 7.1.1, and [36].
λ21 + λ22 + λ23
Sum of squares of the eigenvalues. Example of possible use is strain-rate gives total amount of strain-rate.
kTk =
√
√
p
λ21 + λ22 + λ23
3kT− 31 tr(T)Ik √ 2kTk
3kT− 31 tr(T)Ik √ 2kTk
Tensor norm (Frobenius norm), can also be calculated as the square root of squared elements of the tensor. Relative anisotropy, as proposed in [37]. Proposed for diffusion tensors. Fractional anisotropy, as proposed in [37]. Proposed for diffusion tensors.
Table 2: Tensor invariants and their practical application. The eigenvalues are sorted without the sign so that |λ1 | > |λ2 | > |λ3 | 21
CHAPTER 6. LOCAL TENSOR DESCRIPTION
Decomposition
Interpretation
SymmetricAntisymmetric T=S+A S = 12 (T + TT ) A = 12 (T − TT )
The antisymmetric portion A corresponds to rigid body rotation, while the symmetric portion S contains the stretch and shear components of the tensor.
Isotropic-Deviator T=D+U D = T − tr(T) 31 I U = tr(T) 31 I
A tensor can be decomposed into the sum of isotropic U and deviator tensors D. This assists by removing isotropic details which can overwhelm the non-isotropic information present in the deviator portion.
Eigenvalue/Eigenvector A tensor can be decomposed into the products of its Eigenvectors and Eigenvalues. For symmetric tensors, the eigenvectors form an orthogonal basis allowing diagonalization of the tensor. SVD
The tensor can be decomposed into two orthogonal sets of vectors and a set of singular values. This is related to diagonalization, but a possible advantage over Eigenvector- Eigenvalue Decomposition exists since it is always possible to diagonalize any tensor using SVD, though at the expensive of having to deal with two orthogonal basis.
T = Q1 ΣQT2
Polar Decomposition T = QS
where Q is orthogonal and S is symmetric positive semidefinite. If T is invertible then S is positive definite and the decomposition is unique. Q gives isometric rotation and S is a stretch deformation. One advantage of Polar Decomposition is that the stretch tensor S contains more amplitude information as compared to the symmetric portion of the Symmetric-Antisymmetric Decomposition.
Table 3: Common tensor decompositions.
22
Chapter 7 Low Level Feature Detection The type of local image features that are interesting are very application dependent. A typical application in scalar fields/images is to find objects embedded in the image volume. Thus on a low-level scale one should look for edges that separate objects from each other or the background. This means that one should look for edges/borders in 2D and planes in 3D. When the image volume is time resolved then one should look for hyper-planes in 3D+T. The local neighborhood might also be a combination of all these features, and in some applications it is interesting to find whether the neighborhood is indeed has one topological structure or if it is a combination of different cases. When detecting edges, borders, and planes in many applications not only detecting might be enough one is often interested in their orientation, or velocity in time resolved image volumes. Other possible local feature that one would like to detect could be local image intensity maxima/minima. For vector fields there is a mechanical distinction whether the analyzed material is a fluid or whether it is a solid material. Fluids are characterized by that the physical quantity shear-rate can exist, and that they can not sustain a load. In fluids one are particularly interested in local features such as vortex cores, separation lines, or finding regions of parallel flow.
7.1
Low level feature detection in solids
When analyzing solid materials regionally a common approach is to analyse the motion and deformation of the solid material. This is typically done by analysis of strain-rate, strain tensors. One are typically interested in finding areas with 23
CHAPTER 7. LOW LEVEL FEATURE DETECTION anisotropy [36], or regions where the tensor field shows a consistent temporal and spatial behavior. Other possible interesting features are local geometric measures such as how linear, planar, or isotropic is the tensor. A medical example could be to analyse the myocardium, and analyse contractile events to determine weather the myocardium contracts or not. If it does not contract the myocardium is then either infarcted, stunned or hibernating.
7.1.1
Extracting objects
When preprocessing the image for object extraction, then the most important case of feature detection is edge detection. Therefore edge detection will be described here in detail, and readers interested in other kind of feature detection such as for instance corners are recommended to read [30] or [38]. If the image volume is time resolved then one would like to incorporate the temporal dimension to include as much image information as possible. The idea of using temporal information in segmentation of image sequences is not new; it is proposed and discussed e.g in [39], [40], and [41]. To simultaneously include both spatial and temporal information we need to look in the image volume for border-like or planar-like structures continuous in space and time. We are searching for planar likelihood or similarity to a planar (hyper-planar in 3D+T) neighborhood. Using the local tensor description we can use the relations of the eigenvalues. Consider the three idealized cases in Figure 3, e.g. the eigenvalues λ = (λ1 , λ2 , λ3 ) = (1, 0, 0) for a planar neighborhood assuming that the eigenvalues are sorted in a descending order. For a linear neighbor1 1 1 1 1 hood, λ = √2 , √2 , 0 , and λ = √3 , √3 , √3 for an isotropic neighborhood. Likelihoods li for the different topological cases (planar, linear, and isotropic, respectively) can be formed as a linear mapping of the eigenvalues to a topological likelihood vector as: l1 α11 α21 α31 λ1 l2 = α12 α22 α32 λ2 (24) α13 α23 α33 l3 λ3 The linear mapping A can be found by solving the following linear equation system [42]: √1 √1 1 α11 α21 α31 1 0 0 2 3 1 1 α12 α22 α32 (25) 0 √2 √3 = 0 1 0 1 α13 α23 α33 0 0 1 0 0 √3 24
7.1. LOW LEVEL FEATURE DETECTION IN SOLIDS which yields the linear mapping: 1 p −1 0 l1 λ1 p l2 = 0 (2) −p (2) λ2 l3 λ3 0 0 (3)
(26)
This can easily be expanded to the four-dimensional case [43]. The same mapping have been used also for diffusion tensors [36] even though it was differently derived. A similar mapping as Equation 26 from eigenvalues to topological likelihood has been proposed [30]: l1 =
λ2 − λ3 λ3 λ1 − λ 2 , l2 = , l3 = λ1 λ1 λ1
(27)
The difference is only in the normalization of the likelihood estimate. Note that the estimate proposed here is not normalized with the energy response of the filters. If one has a priori information on for instance desired direction of the planar neighborhood one can take the scalar product between eˆ1 , and the desired direction n ˆ , and multiplied with the corresponding eigenvalue or either planar likelihood l1 , as proposed in Paper I, and [42]. In many applications one also have information on the local phase on the border. A common example is for instance in echocardiography where on know that the transition between blood and myocardium should be a black/white transition. To find this information one needs to look at the phases of the quadrature filters directly or remove the absolute value in Equation 2 and look at the phase of the eigenvalues as commented on in Paper I. The case where one has a priori information on direction is further explored in Paper IV where the term concordant and discordant edges are introduced. An example of these two edge types is shown in Figure 5. In a concordant edge, both the direction of the local image gradient and the surface normal point in approximately the same direction; the directions are discordant in some places in the case of discordant edges. The edge type depends on the image modality being used and the imaged structure. In the case of concordant edge the deformable model is attracted to a black-white transition. In the case of discordant edge the deformable model is attracted to any border/edge regardless of the type of transition (blackwhite, or white-black). A faster alternative than to use the tensor representation to find edges when one have a priori information on the orientation is presented in Paper IV. Instead 25
CHAPTER 7. LOW LEVEL FEATURE DETECTION
0 0
Concordant edge
Discordant edge
Figure 5: The arrows denote image local gradient at the segmented surface. Black and white arrows indicate regions where the model normal and the image gradient have similar or opposite directions, respectively. Zeros denote homogenous areas with a small local image gradient. The black and white lines define the endocardial and epicardial surfaces respectively.
of first estimating the local orientation in the image the image is filtered with a set of directional sensitive edge detectors, and a border probability can be estimated. Temporal information can be incorporated at a very low cost by smoothing the edge forces in the subsequent analysis steps rather than performing temporal smoothing in the edge detection. The filters used in the paper is implemented using separable filters [44].
7.1.2
Analyzing material properties
Deforming solid materials can be analyzed by looking at the physical quantities strain-rate, strain, and stress. One important medical example is the analysis of myocardium. Strain-rate, and strain can be derived from velocity measurements as described in, [45], and [46, 47] respectively. The derivation of the strain-rate tensor is described in Paper III. Normalized averaging was used to smooth the velocity data prior to derivation. In order to be able to do the calculations necessary for computing strain-rate tensor a tensor array toolbox for Matlab was implemented [48]. This tensor array toolbox was also used in Paper II for calculation of the pattern similarity tensor, and is included in the CD-ROM supplement to this thesis. The use of normalized convolution instead of normalized averaging 26
7.2. LOW LEVEL FEATURE DETECTION IN FLUIDS followed by convolution have later been proposed and used successfully implemented [49]. It is also possible is to derive strain is by a technique called tagging where the magnetization is destroyed in a uniform grid. The grid then deforms as the material deform. By following the intersections of the grid, a Lagrangian deformation field is obtained. From this field a strain tensor easily be calculated. A disadvantage with tagging is that the inhomogeneous magnetization is spoiled after some time, and therefore the whole cardiac cycle cannot be analyzed with this method. Another possible physical quantity that can be analyzed is diffusion tensor, typically applied to the brain and used to analyse fiber tracts. It is also possible to derive diffusion tensors in the heart showing fiber orientation [14]. The ultimate goal when analyzing deformation of solid materials is to calculate stress. This is however difficult, since it requires information of fiber direction (for anisotropic materials), and a constitutive equation for that material.
7.2
Low level feature detection in fluids
There are few appropriate algorithms for automated feature detection in measured fluid data sets. The reason for that could be that until recent years it has been difficult, or even impossible to acquire such data. There are several algorithms available for flow characterization of computational fluid dynamics (CFD) simulations. Such simulations of, for instance, air flow around an aircraft have been available for a longer period of time and consequently there have been more development for feature detection algorithms specialized for such applications.
7.2.1
Classical local descriptors for vector fields
In this section a summary of existing feature detection approaches for flow velocity vector fields will be given. For a further review of available approaches to flow characterization, see Paper II. Existing algorithms for three dimensional flow characterization can be divided into two major categories: curvature or velocity gradient based. Both categories, however, have important limitations. Detection of general structures in vector fields can be accomplished by using the concept of critical points. Critical points are points at which the magnitude of 27
CHAPTER 7. LOW LEVEL FEATURE DETECTION the vector field vanishes and these can be characterized using the eigenvalues of the Jacobian of the vector field [50]. Critical points can simply be seen as the corresponding operation as feature detection (of local extremes) in scalar fields by looking at the gradient. Critical points in three dimensions can be clustered to attracting/repelling nodes, saddle points, and spirals/saddles respectively [51]. Velocity gradient based methods can also be tailored to detect of vortices or vortical structures. The simplest example is calculation of vorticity. Intuitively the magnitude of the vorticity vector ω could be suitable for detection of vortices. This classification might fail, however, since |ω| can not distinguish between shear and pure rotation [52]. Other examples using velocity gradients to detect vortices were proposed by Chong, et al. [53]. They defined a vortex core as a region where the eigenvalues of ∇u are complex, where u is the vector field. Jeong, et al. proposed a definition of a vortex in incompressible flow in terms of eigenvalues of the tensor S 2 + Ω2 where S and Ω are the symmetric and anti-symmetric part of ∇u respectively [54]. In general, algorithms based on velocity gradients have the disadvantage that they are very noise sensitive (since they use local estimates of the derivative). Curvature based methods estimate curvature in the vector field. This means that they are tailored to detect vortices in the vector field. The first to introduce the concept of using curvature estimates to detect vortex cores in simulated vector fields was Sadarjoen, et al. [55]. The algorithm evaluates the curvature at each point in the input data set and tries to estimate the distance and direction to a vortex center. The field of scattered detected centers is then integrated to form a curvature center distribution, (CCD). The CCD can be seen as vortex core likelihood. The main disadvantage is the inability to handle vortices curved in 3D space adequately. The main disadvantage with the described methods is that the characterization is local and does not reflect the fact that a vortex is a regional flow structure. The locality also makes the algorithms noise-sensitive, as well as resolution dependent. Other examples of more general local fluid descriptors are strain-rate, viscous stress, stress, reversible momentum flux density, and moment flux density tensor [56].
7.2.2
Vector filters for pattern matching
Instead of trying to design an analytical classification scheme for a flow structure, we can instead search for similarities between the measured velocity field and a set of patterns, one for each flow topology that we want to be able to characterize. This approach also has the advantages of noise insensitivity and ability to handle low resolution data sets, since we are looking at a similarity in a region (depends 28
7.2. LOW LEVEL FEATURE DETECTION IN FLUIDS on pattern sizes) instead of looking locally in one point. Instead of using quadrature filters when building the tensor, as in feature detection in scalar fields, vector filters are used. These filters are sensitive for different flow structures, and examples are shown in Figure 6.
Vortex
Converging
Diverging
Parallel flow
Figure 6: Example of some basic patterns used for feature detection.
From these basic patterns new patterns can be constructed as linear combinations. One example is swirl which is a linear combination of parallel flow and a vortex, illustrated in Figure 7. Even the two patterns diverging, and converging flow, in Figure 6 are actually linear combinations of a line source and parallel flow.
Figure 7: Swirl constructed as a linear combination of parallel flow and a vortex.
The first step in the proposed feature detection algorithm is to normalize the input velocity field as we want to look for topological similarities instead of similarity of velocity distributions. This can be seen as pattern matching between streamlines since streamlines are unaffected of normalization. Using the unified approach in Chapter 6.2, and Equation 19 a local similarity tensor can be constructed as: T (¯ r) =
6 X k=1
h2k (¯ r)
5 T 1 n ˆ n ˆk − I 4 k 4
(28) 29
CHAPTER 7. LOW LEVEL FEATURE DETECTION Note the difference between the tensor representation in scalar fields where the filter response here is squared. To calculate likelihood or similarity to the defined patterns a normalization of the filters is necessary. The filters are normalized such as the normalization scalar γ is determined by the following equation: ZZZ ¯ (ξ) ¯ · γG(ξ)f ¯ (ξ) ¯ dξ¯ = 1 γG(ξ)f (29) Ω
where G is a function describing the desired pattern, and f is a radial weight function to make the pattern spatially localized. The tensor is interpreted in terms of it’s eigenvalues. The largest eigenvalue gives the similarity to the pattern (normalized between zero and one, where one is an exact match). The corresponding eigenvector gives the orientation. Note that the ’sign’ of the orientation is lost. For instance, for parallel flow we only know the axis of the parallel flow, not the direction vector. The direction vector can, however be found by interpolating the filter response in the direction of the eigenvector.
30
Chapter 8 Medium Level Feature Detection Medium level feature detection in multidimensional images deals with the problem of detecting and representing shapes or regions of interest. This process is usually referred to as ’segmentation’ or ’delineation’ in the literature. Traditionally segmentation is used mainly for scalar images, and therefore I have in this thesis used the term ’feature detection’ to point out that this is in fact a general problem that is applicable to vector and tensor images.
8.1
Medium feature detection in solids
Segmentation of medical images is difficult due to the complexity and variability of the shapes of interest [57]. Another difficulty is the lack of a ’gold standard’ method except for manual delineation that is very time consuming and operator dependent. It is not even obvious how the results of two algorithms should be compared [58].
8.1.1
Segmentation approaches
A fundamental aspect of segmentation algorithms are the chosen object representation. An excellent review article on different object models for cardiac images is presented by Frangi, et al. [59]. The first attempts to automatic segmentation of medical images was simple techniques such as global or local thresholding [60]. The major advantage with this representation is that it is very easy to implement. It is also easy to calculate object quantities such as volume, it is simply to count the voxels. Disadvantages are that the representation is not capable of subvoxel resolution, and direct visualization with volume or surface rendering give poor results due to the difficulty 31
CHAPTER 8. MEDIUM LEVEL FEATURE DETECTION to calculate surface normals. Furthermore, a main concern of this representation is that the local methods can generate infeasible object boundaries due to spuriously detected edges. Seed growing algorithms was early introduced [61] and has been combined with multi-spectral approaches where a set of images acquired using different imaging parameters was used forming a feature vector in each voxel [62], [63]. Edge based or boundary tracing algorithms have been developed that, from a user selected point traced a boundary, often using dynamical programming [39]. Boundary tracing approaches were also combined with algorithms based on regional information [64] or knowledge based algorithms [65]. Many of the algorithms was combined with various types of methods to include a priori information, such as mathematical morphology, [66], [67], Bayesian reasoning [68] or self learning approaches such as neural networks [69], probabilistic atlas [70]. One of the most popular method to include a priori information has been by using deformable curves, or models. The deformable model approach that has attracted the most attention is the popularly termed ’snakes’ [71], [72]. Snakes are inherently a 2D approach, the concept of deformable models, can however be extended to multidimensional images, 2D+T [73], 3D [74], [75] or 3D+T [76]. The main difficulty for deformable models in 3D is the object representation. For numerical stability the object representation may need to be refined, and this is a non-trivial task in 3D. Commonly used representations are triangles [17], or tetraeders [77]. A recent segmentation approach in medical images is level set algorithms. The novelty with this approach lies in the object representation with the introduction of a level set function φ(x, y, ...). The object boundaries are found where: φ(x, y, ...) = 0
(30)
where φ is the level set function, and x, y, ... are spatial coordinates. An analogy of the level set function is the isobar curves on the weather forecasts. The function φ is usually simply sampled over a Cartesian grid. The level set function is calculated as a solution to a differential equation. Starting from a boundary condition φ0 which can be seen as equivalent to placing seed points in seed growing approaches. From this boundary condition a solution φn is found by solving a partial differential equation where zero-level set has propagated with a speed determined by a speed image, and locally estimated parameters such as curvature of the level set surface. The process is repeated until some sort of convergence or stopping criteria is met. The calculation of the level set is rather computationally expensive and therefore several approximations and fast schemes have been proposed [78]. An interesting approach to further speed the process up is to do the 32
8.1. MEDIUM FEATURE DETECTION IN SOLIDS calculations on the modern hardware accelerated graphics card [79]. An overview of level set methods and their applications is given by Sethian [78]. The advantage with level sets is the ability to handle complex geometries, and multiple objects. The disadvantage is that it is difficult to include a priori information, even though there have been attempts [80, 81]. Another recent segmentation algorithm is active appearance models. In active appearance models the behavior, and basic shape is learned from a manually delineated training set. A three dimensional (3D/2D+T) implementation for the left ventricle is given by [82]. An excellent overview and review of active appearance models is given by Stegmann [83, 84]. One disadvantage with active appearance models and some probabilistic models is that they do require a learning database. To be able to cope with pathology this image data base needs to be relatively large. Furthermore this database is image modality dependent and therefore it is cumbersome to apply the algorithm to new type of image data. In Paper I a boundary tracing algorithm was used, and in Paper IV a deformable model was used. These two methods will be described here in more detail.
8.1.2
Boundary tracing
With boundary tracing the general concept is to trace the boundary around the complete object. For the left ventricle a common approach is to do a circular resampling of the cavity such as proposed by among others [63, 85]. This circular resampling gives an image where the task is to draw a line from left to right in the resampled image with minimal energy in some measure. An example of the resampling process is shown in Figure 8. Finding a minimal cost path on the image can be done by using Dynamical Programming, and particularly Dijkstra’s algorithm. In [86] and Paper I a novel formulation of this problem was introduced taking to account also a curvature like measure when calculating the optimal trace. The advantage of dynamical programming methods is that they ensure that the global optimum is found. It has recently been shown that a more numerically correct method to solve this problem is to use so called fast marching methods since they are able to use the L2 norm when calculating distance, whereas Dijkstra’s algorithm are forced to use L1 approximations [87, 88]. A major disadvantage with the method is that the circular resampling limits the possible topology of the segmented object significantly and this is particularly cumbersome close to the papillary muscles.
33
CHAPTER 8. MEDIUM LEVEL FEATURE DETECTION
Resampled image
Figure 8: Left: Original echocardiographic image with resampled area overlayed. Right: The resulting circular resampling.
In some applications it is not necessary to resample along the edge and in these circumstances boundary tracing using dynamical programming is an excellent technique. Such examples are segmentation of a m-mode line [89].
8.1.3
Deformable models
The basic idea of the deformable model concept is starting with a ruff estimate of the desired shape and then deform this initial model by minimizing a functional E determined by the shape of the model and the image volume [57]. In general the functional E of the model represented by the representation V is: E(V ) = S(V ) + P(V )
(31)
where the functional E can be viewed as the energy of the deformable model, S is the internal deformation energy, and P is a scalar potential field that couples the model to the image. In order to minimize E numerically it is necessary to discretize the representation V . The energy of the functional is minimized by setting the gradient to zero which yields an Euler-Lagrange equation describing a force equilibrium requirements on the nodes [57], [15]: F ext + F user − F int = 0
(32)
This Euler-Lagrange equation is slightly modified and solved using an iterative scheme in Paper IV by: on+1 = on + γ (F ext + F user − F int ) n ˆ 34
(33)
8.2. MEDIUM FEATURE DETECTION IN FLUIDS where γ is a scaling term, and o is the discretized object representation.
8.2
Medium feature detection in fluids
For fluids the idea of delineate the flow into different regions of interest for quantification purposes is to my knowledge novel. In the literature there have been attempts to track regions of flow for visualization purposes [90, 91, 92], but not for quantification to my knowledge. A vital part of segmentation approaches is the used representation. It is difficult to construct a suitable representation to handle volumes of flow. The chosen representation of the flow is based on individual particle traces. A particle trace is the path that an imaginary particle would take through a velocity field. Based on the , the particle trace equation is written as: definition of velocity u = dx dt dx(t) = u(x(t), t) (34) dt where u is the measured velocity field, x(t) is the position of the imaginary particle. This equation is usually solved by Rung-Kutta technique [93]. The main problem in studying a specific flow with particle trace technique is the need for careful selection of the position where integration should begin x(t0 ). The point of emission is hereafter denoted emission plane, since one almost always are releasing more than one trace simultaneously, and on a structured grid. The relevant flow origin may not be a simple shape, and as result pathline visualization is highly operator dependent. Instead of attempting to target the shifting size, shape and position of an appropriate emission site an approach whose first step is creation of pathlines from an oversized and offset rectangular emission plane is proposed. Particle traces are required to pass one or more circular region of interest planes to be sorted into different groups. In Paper V the approach is applied to diastolic inflow into the left ventricle, and the flow through the left ventricle is analyzed in Paper V. Particle traces are emitted in a plane above the mitral valve in the left atria. The first plane that the particle traces needs to pass is a plane at the mitral vena contracta. This plane effectively removes ’junk’ traces that does not belong to the mitral inflow. A second plane is placed in the left ventricular outflow tract, and by using this plane the traces that passed through the left ventricle during one heart beat can be analyzed.
35
Chapter 9 Results This chapter describes some of the results achieved by applying the methods described and proposed in this thesis.
9.1
Feature detection in solids
The aim of the two papers Paper I and Paper IV are to do medium level feature detection of the inner (endocardial) and outer (epicardial, only Paper IV) confines of the left ventricular wall. The two papers uses two different approaches for low level feature detection of edges/borders. The common denominator of the two approaches are however the intellectual concept to use as much available image data as possible.
9.1.1
Low level feature detection
Performance of border likelihood detection using the local tensor representation and quadrature filters was evaluated in Paper I. It was concluded that using local orientation for border likelihood weak and incomplete borders could be detected where a more traditional gradient approach failed. In Figure 9, the result of the border likelihood calculation on a phantom image sequence is shown. Another example of the importance of including temporal information is given in Figure 10. In this example the temporal edge detection is done by temporal smoothing of the edge force as proposed in Paper IV in a deformable model instead of doing the temporal edge detection directly. In three out of nine MRI gradient echo image sequences a noticeable different could be found between when the temporal smoothing was included. Temporal smoothing had the effect that the segmentation result was more prone to stay at the true endocardial surface and not get 37
CHAPTER 9. RESULTS caught up by the papillary muscles. Frame #1
Frame #2
Frame #3
Frame #4
Frame #5
Frame #6
Frame #7
Frame #8
Frame #9
Figure 9: Left: Test image sequence with a slowly moving border with a fair amount of noise. Right: Border detected in frame #5 of the image sequence. The ’straight’ line is the true border and the ’wigged’ line is the border detected by the algorithm.
a)
b)
Figure 10: Example of effect of temporal edge detection. a) with temporal edge detection, b) without temporal edge detection. From a segmentation of the left ventricle regional wall motion parameters can be derived, enabling studies of regional differences in myocardial contractility and motion. Paper III aims instead to derive parameters that describes the motion and deformation of the myocardial wall directly using low level image feature detection on velocity encoded images. The paper describes and discuss the derivation, interpretation, and visualization of strain-rate tensors from three dimensional phase contrast data. A major problem when analyzing strain-rate data is the visualization of the result. Visualizing second order tensor fields is very difficult. The 38
9.1. FEATURE DETECTION IN SOLIDS strain-rate tensor is an indefinite tensor (i.e eigenvalue can have be both negative and positive) which adds another level of difficulty. An example of the suggested visualization is shown in Figure 11. In this example glyphs are used to show the ration between the different eigenvalues, and color is used to show the magnitude and sign of the the larger eigenvalue. Other methods to visualize this tensor have been proposed such as hybrid schemes that display both glyphs and volume rendering of textures describing the tensor field [29].
Figure 11: Strain-rate invariant It = λ21 + λ22 + λ23 on the left ventricular wall (a), (b). Lower panels the strain-rate tensors in four different time frames (cf) shown as ellipsoids colored by λ1 displaying the main strain-rate direction, strain-rate magnitude in the main direction, and the ratio between the three strainrate principle values. Corresponding time frames are early diastole, end diastole, systole, and end systole.
9.1.2
Medium level feature detection
Both Paper I and Paper IV deals with automated delineation of the left ventricle. From delineations of the left ventricle from cardiac images a wide range of global left ventricular parameters can be derived such as left ventricular mass, end 39
CHAPTER 9. RESULTS systolic/diastolic volumes, stroke volume, cardiac output, ejection fraction, peak ejection rate, peak filling rate, volume curve and volume loops. Examples of regional parameters are radial contraction/expansion velocity which can be used to detect asynchronicity, regional wall thickness, regional wall thickening, regional fractional wall thickening. The two papers uses different segmentation methods. Paper I uses a boundary tracing algorithm to find the final shape, whereas Paper IV uses a deformable model approach. The boundary tracing algorithm have also been used to track m-model lines in echocardiographic images and was used for placement of ROI’s to study the cyclic variation of integrated backscatter from the ultrasound beam [94]. Another application where the boundary tracing algorithm also have successfully been used are ice-radar profiles on glaciers. An example is shown in Figure 12.
Figure 12: Boundary tracing algorithm described in Paper I applied to a part of an ice radar profile on a Svalbard glacier. The detected edge (white) shows the rock under the glacier. The y-axis shows the depth and the x-axis is the distance along the glacier profile. Data courtesy of Ola Brandt, Norwegian Polar Institute.
A fast level set algorithm was tried for the left ventricle, but failed completely in cases where the image contrast was too poor, especially in MRI gradient echo images in patients. Segmentation of the left ventricle using level set algorithms have however been used with success in other studies, but then on images with rather good image contrast [81]. The used algorithm was an implementation of the algorithm described by Nilsson, et al. [95]. It could be so that since this algorithm uses a rather crude approximation the curvature calculation a true level set algorithm may perform better. It can however be used on for instance on Cardiac 40
9.1. FEATURE DETECTION IN SOLIDS CT data, and two examples is given in Figure 13. The fast level set algorithm should only be seen as a ruff initialization step. In order to be useful for calculation purposes it is necessary to apply a full or banded level set algorithm, to achieve sub-voxel accuracy.
Figure 13: Smoothed results from a segmentation using a fast level set algorithm. Top: An example of segmentation of the aortic arch. Bottom: An example of segmentation of the renal arteries. The surface triangulations are done by making an isosurface of the smoothed result.
41
CHAPTER 9. RESULTS Paper IV describes a multimodality segmentation approach based on deformable models. It is implemented into a powerful research tool, that is available on the CD-ROM supplemental to this thesis. It is intended as a multimodality research tool for cardiac image analysis. The graphical user interface is shown in Figure 14.
Figure 14: Example of the user interface for the cardiac image analysis software that is included in the CD-ROM supplement to this thesis.
The approach in Paper IV is to implement one algorithm that is generally applicable to different image modalities. The segmentation algorithm was tested for feasibility on different image types, and examples of the results are shown in Figure 15. An example of 3D visualization of a fully automatic segmentation of an MRI SSFP image sequence is shown in Figure 16.
42
9.1. FEATURE DETECTION IN SOLIDS
a)
b)
c)
d)
e)
f)
g)
h)
i)
Figure 15: Examples segmentation results on different image modalities. a) Surface rendering of result on Cardiac CT data (endo and epi contour). b) Echocardiographic short axis image sequence. c) Gated blood pool SPECT image. d) MRI SSFP image sequence. e) MRI Gradient echo image sequence. f) Delayed enhancement image, the small circles are user placed pins to guide the epicardial surface. g) MRI TOF image of the aorta (animal data). h) MRI PDW image of the aorta (animal data). i) MRI STIR image short axis slice in a patient.
43
CHAPTER 9. RESULTS
Figure 16: Three dimensional visualization of the segmentation result on a MRI-SSFP dataset.
The segmentation algorithm was validated for Cardiac CT, MRI gradient echo, and MRI SSFP in Paper IV where further details on the validation is given. Correlation plots between manual and automatic segmentation results are shown in Figure 17 and corresponding Bland-Altman plots are shown in Figure 18. The largest errors in the segmentation were in the two most basal slices where the volume differed with 37% compared to 3% for the whole left ventricle, so the major part of the eventual manually corrections needs only to be applied to the two most basal slices. The segmentation results are not good enough for direct clinical use without manual interaction. It is however a large step ahead to greatly speed up the process. A manual delineation takes around 25 minutes (two time frames) and the automated segmentation takes approximately 45 seconds on an ordinary desktop computer. Thereafter manual corrections takes about 2-3 minutes depending on the data quality. In total a semi-automatic approach starting with an automatic segmentation followed by small manual corrections takes around five times less time than complete manual segmentation. Note that this proposed semi-automatic approach results in a segmentation for all time frames which enables studying inflow/outflow patterns that gives insight of constructive or obstructive dysfunction of the heart. Furthermore it is also possible to quantify peak filling/ejection rate.
44
9.1. FEATURE DETECTION IN SOLIDS
2 400 y=0.92x+0.46 R =0.99 P 0). Right: results from vortex core detection using the proposed algorithm.
b. c. d. e. f.
4
Convolve with each filter, (1). Create tensor field, (8). Calculate eigenvalues of each tensor in the tensorfield. Take largest eigenvalues as similarity output, Fig. 2. Take the corresponding eigenvector as symmetry axis of the structure.
MATERIALS
The described algorithm was implemented using Matlab (MathWorks, Inc. Natick, Massachusetts). The results were visualized using the software package Ensight (CEI Inc., Apex, North Carolina). Three types of data sets were used in the evaluation of the proposed method: synthetically constructed vector fields (with or without noise), a CFD simulation, and measured blood flow in the human heart. To compare the proposed method with an existing method, the 2 criterium by by Jeong and Hussein [14] was used.
88
where U is the input velocity field (before the noise was added), N is normal distributed noise with a zero mean, and is the region of the vortex. In the last vector field, a parallel flow is linearly superimposed (instead of noise). The velocity of the parallel flow has the same velocity as the largest velocity in the vortex. To further investigate the noise sensitivity, a second set of synthetically constructed vector fields was created, consisting of a vortex with random orientation and random position in a 25 25 25 box. Five different noise levels were used and 200 experiments were performed at each noise level to determine the accuracy of the position and orientation estimates. Examples from two application areas were used. The first example is a CFD data set that contains the velocity distribution over a supersonic vessel calculated at a transonic speed (Mach 0.8) with an 8 degree angle of attack and a 6 degree sideslip (data courtesy CEI, Inc). Before application of the proposed algorithm, the data was resampled to a Cartesian grid. The second application example is a medical imaging example and contains measurements of blood flow in a healthy volunteer, comprising the human heart with a resolution of 1 4 4 mm. The velocity data set was acquired using a time-resolved 3D phase contrast pulse sequence on a GE Signa Horizon EchoSpeed scanner (General Electric Medical Systems, Milwaukee, Wisconsin) [24], [25]. The data set was resampled to 2 2 2 mm voxels using tricubic interpolation. This data is more complex in the sense that the measurements contain noise and that the flow is strictly wall bounded and spatially complicated.
5
RESULTS
The results of the two methods (2 as described by Jeong et al. [14] and pattern matching as described in this paper) applied to the four synthetic datasets are shown in Fig. 2. The left column shows one slice of the input volume and the middle column shows the negative part of 2 (the second largest eigenvalue of S2 þ 2 , where S and are the symmetrical and nonsymmetrical part, respectively, of the velocity Jacobian). Nonnegative 2 are shown as black pixels. The third column is the result of the pattern matching and shows the largest eigenvalue of the tensorfield. The parameters were radius R ¼ 2 and ¼ 1. A difference in noise insensitivity is easily observed, with the proposed method being much more insensitive to noise. Note also the discrepancy between the location of the vortex in the fourth row. Note that the pattern used is a smaller
PAPER II HEIBERG ET AL.: THREE-DIMENSIONAL FLOW CHARACTERIZATION USING VECTOR PATTERN MATCHING
317
TABLE 1 Noise Sensitivity
vortex than in the input data and, thus, only the vortex core is detected. The results of the second set of synthetically constructed dataset are illustrated in Table 1 (mean SD). The angle difference is measured as the space angle between the estimated orientation and the true orientation modulo 180 degrees. The center was defined as the voxel with the highest similarity. Note that, despite a very large amount of noise, both position estimates and orientation estimates are rather accurate. The noise levels can be compared with the illustrated vector fields in Fig. 2. The results of a flow characterization on the supersonic vessel traveling at subsonic speed with side slip are illustrated in Fig. 3. A vortex core detection did not produce any results, whereas the swirl pattern detects two strong swirls at each wing-tip, shown in Fig. 3a. Isosurfaces of left and righthand swirl similarity were created and streamtubes were released from those isosurfaces. The same level was chosen for both the isosurfaces. The left vortex in the figure is a lefthanded swirl and vice versa. The pitch angle was approximately 60 degrees (a relatively high pitch). Note the asymmetrical placement of the swirl due to the side slip. Such asymmetric placement of vortices on the leeward side of the body may cause control and maneuvrability problems that occur for aircraft at high angles of attack or side slip [3]. In Fig. 3b, a cut plane is color-coded by likelihood of converging/diverging flow. Areas with diverging flow are coded white and converging flow dark gray. Streamtubes are shown to depict the general flow field. Note that the angle of attack and side slip explains the nonsymmetry of the converging and diverging regions. Fig. 3c shows a side view of the vessel color-coded with the likelihood of parallel flow. White regions are regions where the flow is similar to straight parallel flow (i.e., free flow). Fig. 4 shows the results of a flow characterization of the blood flow inside the human body. Fig. 4a shows a timeframe when the left atrium is contracting and assists in the filling of the left ventricle. An isosurface of the calculated vortex core similarity is shown in white. Behind the mitral valve, located between the left atrium and the ventricle, a fully developed vortex ring can be seen. From the isosurface, streamlines are emitted. A few streamlines are also released in the valve orifice to show the direction of the blood flow. It has been proposed that vortices behind the valves are of importance for closing the valves [26]. Fig. 4b demonstrates swirl detection in the human aorta. The anatomical outline of the aorta is made transparent. Regions with swirling flow are shown by isosurface with high similarity to left and righthanded swirl, respectively. The darkest isosurface shows lefthanded swirl. Development of
Fig. 3. Supersonic vessel traveling at subsonic speed with six degrees side-slip and eight degrees angle of attack. (a) Isosurfaces are created from the swirl similarity (left and righthanded, respectively) and, from them, streamtubes are released showing two distinct swirling regions. (b) A color-coded cut plane showing regions of converging flow (white) and diverging flow (dark gray). Note that angle of attack and the side slip explains the nonsymmetry of the converging and diverging regions. (c) Likelihood of parallel flow is demonstrated in a clip plane. Black regions are regions where the flow differs from straight parallel flow (free flow).
a righthanded and a lefthanded swirl pair has earlier been predicted and described in the human aorta [27].
6
DISCUSSION
We have presented a method for feature detection in vector fields that is based on pattern matching between a vector field and patterns of flow structures of interest. One advantage of a pattern matching approach compared to other methods is illustrated in Fig. 2, bottom row, where parallel flow is superimposed on a vortex. The 2 method, as all other velocity Jacobian based methods, is not affected by the parallel flow since the Jacobian removes the constant field. The proposed method generates a result that corresponds much better to what the human eye would identify as a center and the vortex definition by Lugt: “a multitude of particles rotating around a common center” [9]. Methods based on helicity or normalized helicity do
89
PAPER II 318
IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS,
VOL. 9,
NO. 3,
JULY-SEPTEMBER 2003
used. For cases where multiple patterns are used (for instance, righthand and lefthand swirl or vortex cores and parallel flow), one isosurface can be generated per similarity measure. The isosufaces can be coded in different shades of gray, as in Fig. 4b. For overlapping structures, transparent isosurfaces can be used. Preliminary experiments with the method have shown that a good visual perception of the overall flow conditions can be acheived if streamlines are released from isosurfaces with a high similarity to the desired flow structure, as shown in Fig. 3a. This approach, creating short streamlines from detected regions with important flow structures, such as vortices or swirls or even regions where the flow is different from the free flow, might be a promising method for creating an overview of large flow fields. The main difficulty for the visualization is to choose a good threshold level of the similarity. In the practical experiments with vortex core detection, this has not been troublesome since the distribution of the similarity values is quite distinct. For vortex core detection, a fixed threshold has been used at 0.5. More elaborate methods for finding good threshold values should be a subject for future research. Techniques such as histogram analysis need to be investigated.
Fig. 4. Results of flow characterization of blood flow in the human heart and the aorta. (a) A complete vortex ring can be seen below the mitral valve (the valve between the left atrium and the left ventricle) in a healthy normal heart. The vortex core is shown as a white isosurface and streamlines are released from the isosurface. The time in the heart cycle is the beginning of the atrial contraction, when blood flows from the left atrium to the left ventricle. Streamlines were also released in the valve orifice. (b) Swirling flow in the human aorta. The dark isosurface shows lefthanded swirl and the brighter isosurface righthanded swirl. Time in the heart cycle is in the middle of the contraction phase of the left ventricle when the blood is ejected into the aorta.
consider the constant field [28], but would still only reflect local and not regional properties of the flow. A disadvantage of the method presented in this paper is that the patterns have to be axisymmetric. More precise is the requirement that the filter response should be proportional to the square of the cosine of the angle between the pattern and the structure. This does not mean that the algorithm does not work on vector fields containing non-axisymmetric structures. It will then result in a least-square solution to the axisymmetric case. Further, the similarity measure is not very sensitive to axisymmetricality, for instance, vortex cores are correctly identified also for vortices with eliptical cross sections.
6.1 Visualization Since the pattern matching approach transforms a vector field to a scalar field with similarity to selected features/ structures, traditional visualization approaches for scalar fields, such as isosurfaces and volume rendering, can be
90
6.2 Computational Considerations Computationally, the proposed method can be made very efficient. The convolution of the filter responses can be significantly reduced by the observation that several filters are linear combinations of each other (for example, a swirl is a linear combination of parallel flow and a vortex). For the patterns described in Fig. 1, only three different types of filters are required. There are several fast algorithms for multidimensional filtering available, using optimized sequential filter networks [29] or filtering in the Fourier domain, where the Fourier transform of the filters can be precomputed. In conclusion, there are several advantages to the method presented here. The method is robust in terms of noise sensitivity due to the fact that it is regional and does not look only at local gradients. The method is applicable to a broad range of flow structures of interest. As shown with the supersonic vessel example, it can detect swirling flow with a very high pitch. Swirling flow with a high pitch is a case where other available vortex detecting methods (eigenvalue and curvature-based methods) are unsuccessful. Automated feature detection with pattern matching combined with emitted streamlines and streamtubes is a new promising tool in automatic visualization of very large sets of computed as well as measured data.
ACKNOWLEDGMENTS The authors would like to thank Dr. Ann Bolger for providing valuable comments and CEI Inc. for providing data for the supersonic vessel. The work was supported by the Swedish Research Council, Swedish Heart and Lung Foundation, and Center of Industrial Information Technology, Linko¨ping University.
PAPER II HEIBERG ET AL.: THREE-DIMENSIONAL FLOW CHARACTERIZATION USING VECTOR PATTERN MATCHING
REFERENCES [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11]
[12] [13]
[14] [15]
[16] [17] [18] [19] [20] [21] [22] [23] [24]
D.C. Banks and B.A. Singer, “A Predictor-Corrector Technique for Visualizing Unsteady Flow,” IEEE Trans. Visualization and Computer Graphics, vol. 1, no. 2, pp. 151-163, June 1995. D.N. Kenwright and R. Haimes, “Vortex Identification—Applications in Aerodynamics: A Case Study,” Proc. IEEE Visualization 1997, pp. 413-416, 1997. P. D. Orkwis, R. Sengupta, and S.M. Davis, “Flow Field Saddles and Their Relation to Vortex Asymmetry,” Computers and Fluids, vol. 26, no. 5, pp. 505-524, 1997. D.N. Kenwright, “Automatic Detection of Open and Closed Separation and Attachment Lines,” Proc. IEEE Visualization 1998, pp. 151-158, 1998. D.N. Kenwright and R. Haimes, “Automatic Vortex Core Detection,” IEEE Computer Graphics and Applications, vol. 18, no. 4, pp. 70-74, July/Aug. 1998. I.A. Sadarjoen, F.H. Post, B. Ma, D. Banks, and H.G. Pagendarm, “Selective Visualization of Vortices in Hydrodynamic Flows,” Proc. IEEE Visualization 1998, pp. 419-422, 1998. P. J. Kilner, G.-Z. Yang, A.J. Wilkes, R.H. Mohiaddin, D.N. Firmin, and M.H. Yacoub, “Asymmetric Redirection of Flow through the Heart,” Nature, vol. 404, no. 13, Apr. 2000. A. Fyrenius, L. Wigstro¨m, T. Ebbers, M. Karlsson, J. Engvall, and A.F. Bolger, “Three Dimensional Flow in the Human Left Atrium,” Heart, vol. 86, no. 4, pp. 448-455, 2001. H.J. Lugt, “The Dilemma of Defining a Vortex,” Recent Developments in Theoretical and Experimental Fluid Mechanics, pp. 309-321, 1979. J. Helman and L. Hesselink, “Representation and Display of Vector Field Topology in Fluid Flow Data Sets,” Computer, vol. 22, no. 8, pp. 27-36, Aug. 1989. P. Bakker, M. de Winkel, “On the Topology of Three-Dimensional Separated Flow Structures and Local Solutions of the NavierStokes Equations,” Topological Fluid Mechanics, Proc. IUTAM Symp., pp. 384-394, 1989. A. Perry and M. Chong, “Topology of Flow Patterns in Vortex Motions and Turbulence,” Applied Scientific Research, vol. 53, nos. 34, pp. 357-374, 1994. R. Cucitore, M. Quadrio, and A. Baron, “On the Effectiveness and Limitations of Local Criteria for the Identification of a Vortex,” European J. Mechanics and Biomedical Fluids, vol. 18, no. 2, pp. 261282, 1999. J. Jeong and F. Hussain, “On the Identification of a Vortex,” J. Fluid Mechanics, vol. 285, pp. 69-94, 1995. C. Schram and M.L. Riethmuller, “Vortex Ring Evolution in an Impulsively Started Jet Using Digital Particle Image Velocimetry and Continuous Wavelet Analysis,” Measurement Science and Technology, vol. 12, no. 9, pp. 1413-1421, 2001. M. Roth and R. Peikert, “A Higher-Order Method for Finding Vortex Core Lines,” Proc. IEEE Visualization 1998, pp. 143-150, 1998. M. Chong, A.E. Perry, and B.J. Cantwell, “A General Classification of Three-Dimensional Flow Field,” Physics of Fluids, vol. A, no. 2, p. 765, 1990. R. Peikert and M. Roth, “The ’Parallel Vectors’ Operator—A Vector Field Visualization Primitive,” Proc. IEEE Visualization 1998, pp. 263-270, 1999. H.-G. Pagendarm, B. Henne, and M. Ru¨tten, “Detecting Vortical Phenomena in Vector Data by Medium-Scale Correlation,” Proc. IEEE Visualization 1999, pp. 409-412, 1999. Y. Lavin, R. Batra, and L. Hesselink, “Feature Comparison of Vector Fields Using Earth Mover’s Distance,” Proc. IEEE Visualization 1998, pp. 103-110, 1998. T. Wischgoll and G. Scheuermann, “Detection and Visualization of Closed Streamlines in Planar Flows,” IEEE Trans. Visualizaton and Computer Graphics, vol. 7, no. 2, pp. 165-172, Apr.-June 2001. H. Knutsson, “Representing Local Structure Using Tensors,” Proc. Sixth Scandinavian Conf. Image Analysis, pp. 244-251, 1989. G. Granlund and H. Knutsson, Signal Processing for Computer Vision. Linko¨ping: Kluwer Academic, 1995. L. Wigstro¨m, L. Sjo¨qvist, and B. Wranne, “Temporally Resolved 3D Phase-Contrast Imaging,” Magnetic Resonance in Medicine, vol. 36, pp. 800-803, 1996.
319
[25] L. Wigstro¨m, T. Ebbers, A. Fyrenius, M. Karlsson, J. Engvall, B. Wranne, and A.F. Bolger, “Particle Trace Visualization of Intracardiac Flow Using Time-Resolved 3D Phase Contrast MRI,” Magnetic Resonance in Medicine, vol. 41, no. 4, pp. 793-799, 1999. [26] B.J. Bellhouse and F.H. Bellhouse, “Fluid Mechanics of the Mitral Valve,” Nature, vol. 224, no. 219, pp. 615-616, 1969. [27] P.J. Kilner, G. Z, R.H. Mohiaddin, D.N. Firmin, and D.B. Longmore, “Helical and Retrograde Secondary Flow Patterns in the Aortic Arch Studied by Three-Directional Magnetic Resonance Velocity Mapping,” Circulation, vol. 88, no. 5, Part 1, pp. 22352247, 1993. [28] Y. Levy, D. Degani, and A. Seginer, “Graphical Visualization of Vortical Flows by Means of Helicity,” AIAA J., vol. 28, no. 8, 1990. [29] H. Knutsson, M. Andersson, and J. Wiklund, “Advanced Filter Design,” Proc. Scandinavian Conf. Image Analysis, 1999. Einar Heiberg (born Brandt) received the MSc degree in electrical engineering and computer science from Linko¨pings Universitet 1998 and is currently a PhD student in the Department of Biomedical Engineering. He is a research engineer in the Department of Medicine and Care, Linko¨pings Universitet, Sweden. His current research interest is automated feature detection, image segmentation, flow visualization, and cardiovascular dynamics.
Tino Ebbers performed his undergraduate studies in electrical engineering at the University of Twente, The Netherlands, and received the MSc degree in 1996. Thereafter, he commenced his research in 3D cardiovascular fluid dynamics based on MRI data in the Department of Biomedical Engineering, Linko¨pings Universitet, Sweden. For this research, he received the PhD degree in biomedical engineering in 2000. After his PhD studies, he proceeded with research in this area during a postdoctoral year in the Department of Medicine and ¨ Care, Linkopings Universitet, Sweden. Since September 2002, he has worked for Philips Medical Systems in The Netherlands. Lars Wigstro¨m received the MSc degree in electrical engineering from Linko¨pings Universitet in 1992 and is currently a PhD student in the Department of Biomedical Engineering and also one of the coordinators for the Center of Medical Image Science and Visualization (CMIV). He works as a research engineer in the Department of Medicine and Care, Linko¨pings Universitet, Sweden. His research interest is related to new methods for acquisition and visualization of multidimensional MRI data. Matts Karlsson performed his undergraduate studies in mechanical engineering at Linko¨pings Universitet and received the MSc degree in 1989. In 1995, he received the PhD degree in applied thermodynamics and fluid mechanics, also from Linko¨pings Universitet. The title of the thesis was: “On Modelling of Constriced Blood Flow—Mitral Regurgitation and Aortic Coarctation.” In 1995-1996, he was a research fellow at the Falk Cardiovascular Research Center, Stanford University and a visiting investigator at the Research Institute/Palo Alto Medical Foundation. Currently, he is a professor of biomedical engineering in the Department of Biomedical Engineering, Linko¨pings Universitet.
. For more information on this or any computing topic, please visit our Digital Library at http://computer.org/publications/dlib.
91
Paper III
Paper III Kinematics of the heart: strain-rate imaging from time-resolved three-dimensional phase contrast MRI P. Selskog, E. Heiberg, T. Ebbers, L. Wigstr¨om, and M. Karlsson
IEEE Transactions on Medical Imaging, vol. 21(9), pp. c IEEE. Reprinted with permission. 1105-1109, 2002.
93
PAPER III IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 21, NO. 9, SEPTEMBER 2002
1105
Kinematics of the Heart: Strain-Rate Imaging From Time-Resolved Three-Dimensional Phase Contrast MRI Pernilla Selskog*, Einar Heiberg, Tino Ebbers, Lars Wigström, and Matts Karlsson
Abstract—A four-dimensional mapping (three spatial dimensions time) of myocardial strain-rate would help to describe the mechanical properties of the myocardium, which affect important physiological factors such as the pumping performance of the ventricles. Strain-rate represents the local instantaneous deformation of the myocardium and can be calculated from the spatial gradients of the velocity field. Strain-rate has previously been calculated using one-dimensional (ultrasound) or two-dimensional (2-D) magnetic resonance imaging) techniques. However, this assumes that myocardial motion only occurs in one direction or in one plane, respectively. This paper presents a method for calculation of the time-resolved three-dimensional (3-D) strain-rate tensor using velocity vector information in a 3-D spatial grid during the whole cardiac cycle. The strain-rate tensor provides full information of both magnitude and direction of the instantaneous deformation of the myocardium. A method for visualization of the full 3-D tensor is also suggested. The tensors are visualized using ellipsoids, which display the principal directions of strain-rate and the ratio between strain-rate magnitude in each direction. The presented method reveals the principal strain-rate directions without a priori knowledge of myocardial motion directions.
+
Index Terms—Cardiovascular system, kinematics, magnetic resonance imaging.
I. INTRODUCTION
T
HE heart muscle, which is a complex three-dimensional (3-D) structure, has anisotropic, nonlinear, and time-dependent mechanical properties. During the cardiac cycle, the myocardium undergoes large elastic deformations as a consequence of the active muscle contraction along the muscle fibers and their relaxation, respectively [1]. Physiological factors such as the pumping performance of the ventricles, the distribution of coronary blood flow, and the regional vulnerability to ischemia Manuscript received December 1, 2001; revised July 6, 2002. This work was supported in part by the Swedish Technical Research Council, in part by the Swedish Medical Research Council, in part by the Swedish Heart and Lung Foundation, the Cente/TMI for Industrial Information Technology at Linköping University, and in part by the Swedish Foundation for Strategic Research (CORTECH). Asterisk indicates corresponding author. *P. Selskog is with the Department of Biomedical Engineering, Linköping University, 581 85 Linköping, Sweden (e-mail:
[email protected]). E. Heiberg and L. Wigström are with the Department of Biomedical Engineering and the Department of Medicine and Care, Linköping University, 581 85 Linköping, Sweden. T. Ebbers is with the Department of Medicine and Care, Linköping University, 581 85 Linköping, Sweden. M. Karlsson is with the Department of Biomedical Engineering, Linköping University, 581 85 Linköping, Sweden. Digital Object Identifier 10.1109/TMI.2002.804431
and infarction are affected by the mechanical properties of the myocardium [2]. The local instantaneous deformation of the myocardium is represented by the strain-rate (rate of deformation) tensor. A four-dimensional (4-D) description (three spatial dimensions time) of myocardial strain-rate may help to describe the mechanical properties of the myocardium, which is of interest in the assessment of myocardial function. Strain-rate has previously been calculated from two-dimensional (2-D) magnetic resonance imaging (MRI) [3], and strain-rate imaging using echo-Doppler has been suggested as a clinical tool [4], [5]. However, these techniques assume that myocardial motion only occurs in one plane (2-D MRI) or in one direction (ultrasound). A technique where the 3-D strain-rate is calculated from 2-D MRI data has also been suggested [6], where the results are based on approximation of the values in the third direction. All these techniques lack information of through plane motion. Time-resolved 3-D velocity data, enabling calculation of stain rate, can be obtained from 3-D phase contrast MRI. This paper presents a method for calculation of the 3-D strain-rate tensor field from time-resolved 3-D phase contrast MRI velocity data and suggests a method for visualization of the full strain-rate tensor. II. METHODS A. Theory To calculate strain-rate from a velocity field, the 3 3 veis calculated according to locity gradient tensor (Jacobian) (1) are the three velocity components in the where , direction, [7]. , which Strain-rate is represented by the strain-rate tensor [7] is the symmetric part of (2) The eigenvalues and eigenvectors of the strain-rate tensor are the principal values and the principal directions of strain-rate in the myocardium. The sign of the eigenvalue distinguishes between positive and negative material stretching in the direction of the corresponding eigenvector.
0278-0062/02$17.00 © 2002 IEEE
95
PAPER III 1106
IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 21, NO. 9, SEPTEMBER 2002
Fig. 2. (a) Strain-rate calculations were performed on a mathematical model representing a cylinder with altering radius and height. (b) The radius and height vary according to a sinusoidal function through 16 time frames.
Fig. 1. Tensor visualization using ellipsoids where the three axes of the ellipsoid represent the main directions of strain-rate and the length of the axis represents the strain-rate magnitude in each direction. The ratio between the magnitude of the eigenvalues thus determines the shape of the ellipsoids.
ocardium magnitude image data were used as a certainty measure of the obtained velocity data. The gradients were estimated by weighting of a forward and a backward difference according to
B. Visualization
(4)
and eigenvectors of the tensor were calThe eigenvalues culated and used for visualization. The strain-rate tensor in each point can be visualized as an ellipsoid [8], as shown in Fig. 1. The directions of the three axes of the ellipsoid are the eigenvectors of the tensor, and the length of the ellipsoid in each direction is proportional to the magnitude of the corresponding eigenvalue, respectively. The ratio between the magnitude of the eigenvalues thus determines the shape of the ellipsoid. The longer and more slender an ellipse is, the more line-like is the deformation in that neighborhood. To suppress large differences in magnitude and enable visualization of deformation directions, the eigenvalues have been normalized so that the magnitude of the largest eigenvalue is one. This makes all ellipsoids visible while preserving the ratio between the eigenvalues. The magnitude of the largest eigenvalue can be represented by the color of the ellipsoid. To display stretch separated from shortening, the eigenvalues must be sorted, separating the large positive strain-rate values representing stretch from the negative values representing major shortening. An invariant representing the total amount of strain-rate in each voxel was calculated according to (3) The visualization method and the need for 3-D measurements can be demonstrated using a mathematical model representing a cylinder with varying radius and height. The radius and height vary according to a sinusoidal function, as shown in Fig. 2. C. Implementation When calculating velocity gradients in the in vivo data, velocity data from the blood pool must be excluded from gradient estimates in the myocardium to avoid blood pool velocities’ affecting the gradients. The distance between the beginning and the end of a particle trace [9] in combination with magnitude and velocity information along the particle trace can be used to estimate the probability of a voxel belonging to the myocardium [10]. A binary myocardium mask was obtained by applying a threshold to the myocardium probability data. Masked my-
96
is the resolution in the direction of the gradient and where and are the velocity and the certainty in the voxel . The velocity field was smoothed prior to gradient calculations to prevent noise from affecting velocity gradient calculations. To avoid border artifacts, smoothing weighted by the certainty of the obtained velocity data was performed using normalized averaging with a Gaussian applicability function [11] (5) , is the vewhere is a 3-D Gaussian function with locity component in the direction , denotes convolution, and voxelwise multiplication. All calculations were performed in MATLAB (MathWorks, Inc., Natick, MA). The eigenvalues and eigenvectors were transferred to the EnSight visualization program (CEI, Inc., Apex, NC), which provides the possibility of studying arbitrary slices and surfaces interactively in the data set. D. In Vivo Data Calculations were performed using velocity data for a healthy 24-year-old volunteer with a heart rate of 66 bpm. The velocity measurements were performed using a 1.5 T Signa Horizon Echospeed scanner (General Electric Medical Systems, Milwaukee, WI) with a 3-D cine phase contrast pulse sequence, allowing offline retrospective gating based on the signal from a pulse oximeter [9], [12]. Velocity encoded data were acquired in all three directions at 12 times for each phase-encoding step. A complete set of 3-D -space data was interpolated in the time domain to 32 timeframes using an interpolation method based on a normalized convolution algorithm with a Gaussian interpolation function [9]. Using peripheral gating, i.e., gating based on the signal from a pulse oximeter on the finger, the first time frame corresponds to approximately 200 ms after onset of electrical systole. Only a cylindrical region of -space was collected in order to reduce the acquisition time by not acquiring the corners of the plane [13].
PAPER III SELSKOG et al.: STRAIN-RATE IMAGING FROM TIME-RESOLVED 3-D PHASE CONTRAST MRI
All three velocity components were measured in a 30.0 30.0 11.2 cm axial volume, suitable for myocardial motion studies, with a spatial resolution of 1.2 4.0 4.0 mm and a ms, TE ms, temporal resolution of 108 ms (TR m/s, flip angle , one NEX, acquisition VENC time 39:14 min). The technique includes automated unwrapping of aliased velocities [9]. The MRI data were resampled using zero-filling in the Fourier domain to a voxel size of 1.5 1.5 1.5 mm. Saturation bands superior and inferior to the imaging volume were used to reduce the signal from the blood [14]. After the 3-D inverse Fourier transform, the phase contribution from concomitant field (Maxwell) effects was calculated and subtracted from the velocity data [15]. A linear 3-D least square fit to the data from regions containing stationary tissue was subtracted for correction of eddy current effects [16]. Conventional 2-D time-resolved gradient-echo images were acquired in short- and long-axis cardiac views for better depiction of anatomical landmarks.
1107
Fig. 3. Strain-rate at two different time frames, 3 (left) and 14 (right), in a slice from the mathematical model shown in Fig. 2. The main direction is circumferential and strain-rate magnitude in the main direction varies with the radius. Strain-rate in the main direction is positive in time frame 3, i.e., there is stretch in the main direction. In time frame 14, the model is returning to its original shape and the main strain-rate is negative, corresponding to shortening in the main direction.
III. RESULTS Strain-rate tensors in a slice of the cylinder model, from time frames 3 and 14, are shown in Fig. 3. The early time frame, when the model represents increasing radius and thickness as well as decreasing height, is shown to the left. The ellipsoids are color-coded according to the magnitude of the largest eigenvalue. Strain-rate in the main direction is positive, i.e., there is stretch in the main direction. From the orientation of the ellipsoids, it can be seen that the main direction is circumferential and the color code displays that the circumferential strain-rate magnitude varies with the radius of the cylinder. The ellipsoids become more circular toward the outer radius, which reveals a varying ratio between strain-rate in the three principal directions. In time frame 14, shown in Fig. 3 (right), the model is returning to its original shape. In this case, the main strain-rate is negative, corresponding to shortening in the main direction. Note that the apparent motion is in the longitudinal and radial directions but that the main direction of strain-rate is circumferential. The global motion or velocity of the model cylinder may not reflect what takes place on the local level. For example, as the radius increases, the major strain-rate component is in the circumferential direction. Fig. 4 shows a long-axis slice through the myocardium in four different time frames: early diastole, end diastole, systole, and end systole. The overlaid color represents the invariant , showing a larger amount of strain-rate in the systolic and early diastolic time frames compared to the other two. This is consistent with a time plot of , shown in Fig. 5, where peaks are found in early diastole and in systole. In Fig. 6, the most positive and the most negative eigenvalues for each point are shown separately to display stretch (positive eigenvalues) and shortening (negative eigenvalues) in early diastole and in systole. Since the strain-rate tensors are known in the complete data volume, strain-rate can be visualized in any part of the myocardium. Fig. 7 shows a surface in the left ventricular myocardium and the strain-rate tensor in a region of interest on this surface in an early diastolic time frame. The strain-rate tensor in the region of interest is shown for four dif-
Fig. 4. A color map of the invariant (I ) for a long-axis slice through the myocardium in (a) early diastole, (b) end diastole, (c) systole, and (d) end systole. A time plot of I at a point in the lateral left ventricular wall is shown in Fig. 5.
ferent phases of the cardiac cycle. The orientation of the ellipsoids represents the main direction of instantaneous deformation, which changes throughout the cardiac cycle. The surface color represents the invariant , and the ellipsoids are colored according to the largest eigenvalue (sorted by magnitude). IV. DISCUSSION Measurement of strain-rate in the myocardium would be advantageous for studying cardiac kinematics. Rigid body translation and rotation, where there is no deformation, should not contribute to a proper measure of deformation rate. The presented method calculates the full 3-D strain-rate tensor, which meets this condition and is a complete representation of strain-rate.
97
PAPER III 1108
IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 21, NO. 9, SEPTEMBER 2002
= + +
Fig. 5. Strain-rate invariant I at a point in the lateral left ventricular wall (highlighted in the left figure) throughout the cardiac cycle (right). Peaks are found in early diastole and in systole. (a)–(d) refer to the time frames displayed in Fig. 4.
=
+
+
Fig. 7. Strain-rate invariant I on a myocardial surface in the left ventricular wall (a) and (b). The strain-rate tensors in four different time frames (c)-(f) are represented by ellipsoids colored by displaying main strain-rate direction, strain-rate magnitude in the main direction, and the ratio between the three strain-rate principle values (early diastole, end diastole, systole, end systole).
Fig. 6. (a), (b) Stretch and (c), (d) shortening in a long-axis slice of the myocardium are shown by separately displaying the values of the most positive and most negative eigenvalues for each point. (a) and (c) display an early diastolic time frame; (b) and (d) display a systolic time frame.
Calculations from one-dimensional or 2-D velocity data will not provide information of strain-rate in all three directions. Since not all dimensions are included, the information of main directions will be incomplete. By calculating strain-rate from 3-D velocity data, there is no need for a priori knowledge of the main directions of deformation. The 3-D strain-rate tensor includes the principal directions and will therefore provide information of the main directions of strain-rate. This eliminates the errors that would occur due to beam angle or slice selection when using ultrasound or 2-D MRI techniques, respectively. The results show that the main direction of strain-rate is nonplanar and varies with time and throughout the myocardium. Furthermore, the principal directions of strain-rate might not coincide with the main motion directions. This is, for instance,
98
demonstrated by the cylinder model, where strain-rate measurements in the main motion direction would not reveal that the main direction of strain-rate is circumferential. This, again, indicates that 3-D data and calculations are necessary to study strain-rate in the myocardium. The invariant represents the amount of strain-rate regardless of stretch or shortening. The results shown in Figs. 4 and 5 indicate that the strain-rate in end systole and end diastole is small compared to strain-rate in early diastole and systole. Invariants or color maps of the eigenvalues only provide information of strain-rate magnitude. However, the strain-rate tensor contains full information about the instantaneous deformation, including both magnitude and direction. The ellipsoid method makes it possible to visualize the complete tensor in one plot. From Fig. 7 can be seen that the main strain-rate direction, represented by the orientation of the ellipsoids and the strain-rate magnitude in this direction, represented by the color of the ellipsoids, vary with time. The colored surface in the figure mainly serves as orientation but also provides information of the total amount of strain-rate.
PAPER III SELSKOG et al.: STRAIN-RATE IMAGING FROM TIME-RESOLVED 3-D PHASE CONTRAST MRI
The limitations of the technique include some blurring of the velocity data due to respiratory motion. Respiratory compensation or gating was not used in this study, to avoid prolonging of the scan time. Future improvements of the pulse sequence should aim to include respiratory compensation even if individual images from the 3-D data set do not show any prominent respiratory artifacts. The limited temporal resolution may cause underestimation of peak velocities and thereby possible underestimation of the velocity gradient and the strain-rate values. A higher temporal resolution could be achieved by further optimization of the pulse sequence, resulting in a shorter repetition time. To calculate strain-rate only inside the myocardium, a mask was used to segment the myocardium velocity data by applying a threshold to the myocardium probability. A single threshold may not be optimal in all slices and time frames; occasionally there will be remaining voxels from the blood pool and surrounding tissue. Voxels containing blood velocity data may therefore still be included in the strain-rate calculations at the borders of the myocardium. This does not, however, affect the results in other parts of the wall, and the effect would be reduced by increased spatial resolution. When comparing strain-rate tensors in different phases of the cardiac cycle, it should be noted that it is generally not the same part of the myocardium that is displayed in all time frames, as the heart moves. For example, a part of the myocardium found in the middle of the ventricular wall in a diastolic time frame may no longer be in the mid-wall in other parts of the cardiac cycle. In conclusion, myocardial strain-rate is 3-D, and therefore the full strain-rate tensor must be treated and displayed for visualization of myocardial mechanics. The presented method displays the full strain-rate tensor, revealing the main direction of strain-rate without any assumptions of myocardial motion directions in the calculations. REFERENCES [1] A. D. McCulloch, L. K. Waldman, J. Rogers, and J. M. Guccione, “Large-scale finite element analysis of the beating heart,” Crit. Rev. Biomed. Eng., vol. 20, no. 5,6, pp. 427–449, 1992.
1109
[2] J. M. Guccione, A. D. McCulloch, and L. K. Waldman, “Passive material properties of intact ventricular myocardium determined from a cylindrical model,” J Biomech. Eng., vol. 113, pp. 42–55, 1991. [3] V. J. Wedeen, “Magnetic resonance imaging of myocardial kinematics. Technique to detect, localize and quantify the strain rates of the active human myocardium,” Magn. Reson. Med., vol. 27, no. 1, pp. 52–67, 1992. [4] A. Stoylen, A. Heimdal, K. Bjornstad, H. G. Torp, and T. Skjaerpe, “Strain rate imaging by ultrasound in the diagnosis of regional dysfunction of the left ventricle,” Echocardiography, vol. 16, no. 4, pp. 321–329, 1999. [5] M. Kowalski, T. Kukulski, F. Jamal, J. D’hooge, F. Weidemann, F. Rademakers, B. Bijnens, L. Hatle, and G. R. Sutherland, “Can natural strain and strain rate quantify regional myocardial deformation? A study in healthy subjects,” Ultrasound Med. Biol., vol. 27, no. 8, pp. 1087–1097, 2001. [6] M. D. Robson and R. T. Constable, “Three-dimensional strain-rate imaging,” Magn. Reson. Med., vol. 36, no. 4, pp. 537–546, 1996. [7] A. J. M. Spencer, Continuum Mechanics. London, U.K.: Longman Scientific Technical, 1980. [8] C.-F. Westin, “A tensor framework for multidimensional signal processing,” Dissertation 348, Linköping Studies in Science and Technology, Linköpings Universitet, Linköping, Sweden, 1994. [9] L. Wigström, T. Ebbers, A. Fyrenius, M. Karlsson, J. Engvall, B. Wranne, and A. F. Bolger, “Particle trace visualization of intracardiac flow using time-resolved 3D phase contrast MRI,” Magn. Reson. Med., vol. 41, no. 4, pp. 793–799, 1999. [10] T. Ebbers, “Cardiovascular fluid dynamics. Methods for flow and pressure field analysis from magnetic resonance imaging,” Dissertation 690, Linköping Studies in Science and Technology, Linköpings Universitet, Linköping, Sweden, 2001. [11] H. Knutsson and C. F. Westin, “Normalized and differential convolution: Methods for interpolation and filtering of incomplete and uncertain data,” in Proc. IEEE Computer Society Conf. Computer Vision and Pattern Recognition, 1993, pp. 515–523. [12] L. Wigström, L. Sjöqvist, and B. Wranne, “Temporally resolved 3D phase contrast imaging,” Magn. Reson. Med., vol. 36, pp. 800–803, 1996. [13] P. Irarrazabal and D. G. Nishimura, “Fast three dimensional magnetic resonance imaging,” Magn. Reson. Med., vol. 33, no. 5, pp. 656–662, 1995. [14] M. Drangova, Y. Zhu, and N. J. Pelc, “Effect of artifacts due to flowing blood on the reproducibility of phase-contrast measurements of myocardial motion,” J. Magn. Reson. Imag., vol. 7, no. 4, pp. 664–668, 1997. [15] M. A. Bernstein, X. J. Zhou, J. A. Polzin, K. F. King, A. Ganin, N. J. Pelc, and G. H. Glover, “Concomitant gradient terms in phase contrast MR: Analysis and correction,” Magn. Reson. Med., vol. 39, pp. 300–308, 1998. [16] N. J. Pelc, F. G. Sommer, K. C. Li, T. J. Brosnan, R. J. Herfkens, and D. R. Enzmann, “Quantitative magnetic resonance flow imaging,” Magn. Reson. Q., vol. 10, no. 3, pp. 125–147, 1994.
99
Paper IV
Paper IV Time Resolved Three-dimensional Segmentation of the Left Ventricle in Multimodality Cardiac Imaging E. Heiberg, L. Wigstr¨om, M. Carlsson, A.F. Bolger, M. Karlsson
Submitted.
101
PAPER IV
Time Resolved Three-dimensional Segmentation of the Left Ventricle in Multimodality Cardiac Imaging
Einar Heiberg ∗,a,e , Lars Wigstr¨om a,e Marcus Carlsson b Ann F. Bolger d Matts Karlsson c,e a Department
of Medicine and Care, Link¨ oping University, Sweden
b Department c Department d Department e Center
of Clinical Physiology, Lund University, Sweden
of Biomedical Engineering, Link¨ oping University, Sweden
of Medicine/Cardiology, University of California, San Francisco, USA
for Medical Image Science and Visualization, Link¨ oping University, Sweden
Abstract We propose a robust approach for multimodality segmentation of the cardiac left ventricle. The method is based on the concept of deformable models, but extended with an enhanced and fast edge detection scheme that includes temporal information, and anatomical a priori information. The algorithm is implemented with a fast numeric scheme for solving energy minimization, and efficient filter nets for fast edge detection. This allows clinically applicable time for a whole time resolved 3D cardiac data set to be acheived on a standard desktop computer. The algorithm is validated on images acquired using MRI Gradient echo, MRI (SSFP) images, and Cardiac CT, and tested for feasibility with three other imaging modalities, including gated blood pool SPECT, echocardiography and late enhancement MRI.
Key words: Image Segmentation, Deformable models, Left Ventricle, 3D, time resolved, edge detection
∗ Corresponding author. Email address:
[email protected] (Einar Heiberg).
Preprint submitted to Medical Image Analysis
103
22nd November 2004
PAPER IV
1
Introduction
Segmentation of the left ventricle (LV) is of great clinical interest since it allows to directly measure important paramaters such as end-diastolic volume, ejection fraction, and myocardial mass. If the segmentation of both endocardial (inner) and epicardial (outer) contours is done over time it furthermore allows measurement of both global (normal/constrictive/restrictive filling patterns) and regional LV function. The left ventricle is difficult to segment, however, since the image quality may suffer from the technical difficulties of imaging in the setting of cardiac and respiratory motion. Further, the internal and external LV contours are variable between individuals and undergo complex deformations and translations with each heart beat. A broad spectrum of LV segmentation techniques for different imaging modalities have been proposed, ranging from simple techniques such as thresholding or region growing, to boundary tracing [1], probabilistic or statistical models [2,3] level-sets [4], deformable models/active contours [5,6], and 3D active appearence models [7]. Unfortunately, active appearance models have problems of coping with shape variability outside the learning set, and it is computationally expensive to have a learning set that includes all phases of the cardiac cycle. The level set methods have a problem with including a priori information, and are rather computationally intensive for 3D+T cardiac data sets. One of the main contributions of this paper is the multimodality approach. The presented method can easily be adopted to specific imaging modalities. We have tried to put the focus on the unifying cardiac analysis rather than focus on the different characteristics of different imaging modalities. This is important since the rapid development of new imaging techniques makes it cumbersome to develop segmentation algorithms for each image modality. It is not only new imaging modalities that are being developed equally important is the development of new contrast agents and imaging parameters that completely changes the characteristics of the images.
2
Method
The basic idea behind the concept of deformable models is that a geometrical representation of an object is allowed to deform under an internal deformation energy (controlling the allowed shapes of the deforming object), and an external potential energy field (from the input images or user interaction). The deformable object is allowed to deform such as it fits the input data. The energy functional of the deformable model o can be written as E:
104
2
PAPER IV
E(o) = D(o) + P(o)
(1)
where D is the internal deformation energy, and P is the external potential field. Minimizing the energy of the deformable object o, leads to an EulerLagrange equation which basically states or express that the internal and external forces must balance at equilibrium [8,9]. In this paper deforming the object is based solely on solving a modified Euler-Lagrange equation that states that the forces need only to balance along the normal vector n ˆ at the model surface.
n ˆ F ext + n ˆ F user − n ˆ F int = 0
(2)
The reason for modifying the standard Euler-Lagrange equation is that we do not want the parametrization of the deformable model o to be influenced by the force, merely the shape of the object. The shape of the object is found by an iterative process given by:
on+1 = on + γ (ˆ nF ext + n ˆ F user − n ˆ F int ) n ˆ
(3)
where γ is a scaling term. This term is calculated as the product between the mesh’s smallest point-to-point distance and a constant, and divided by the largest force on the entire model. The external forces F ext are the sum of a balloon force, and an edge force. The internal forces F int are constituted of a curvature force, a slice force, a temporal acceleration force, and a temporal damping force. The deformable model representation o in this paper has less topological freedom than most proposed deformable models, but has the advantage of being fast, robust and applicable to segmentation of the left ventricle. The method was designed to incorporate as much a priori information as possible in order to minimize the impact of image artifacts, and to be applicable to image data from multiple modalities. The adaption to different imaging modalities is done by a few carefully selected parameters. There are four parameters govering the forces on the deformable model, and four parameters that controll how the image intensity is treated. With respect to implementation, the ease of user interaction was given a high priority. A poor user interface may prevent the algorithm from being used to its full potential in the clinical setting. 3
105
PAPER IV
2.1
Model Representation
The left ventricular model used in this approach is a time-resolved mesh representation of the LV as an open ’cone’, sliced along the cone’s long axis with an equal number of points in each slice. The cone’s geometry is quite similar to the 3D active contour model proposed by Kucera et al.[9] except that that model was 2D+T, and our model being extension to 3D+T. The number of points in each slice is 80. The model o is stored in a 3D array of node-points as: o(l, t, z) = [1..80] × [1..T ] × [1..Z]
(4)
where T are the number of timeframes, and Z are the number of slices. Each node-point in o points out a location in space and time (x,y,z,t) where z,t coordinates are implicitly stored. Image data are assumed to be approximately aligned with a short axis view, and data not conforming to that need to be resampled. The model is initalized as a small cone which the user positions with a single mouse click in the approximate center of the LV. The image intensity in that region is taken as an estimate of the blood pool signal intensity. The segmentation is then performed first on the endocardial surface. That surface is subsequently used to initialize the segmentation of the epicardial surface.
2.2
Detecting concordant and discordant edges
A majority of the existing deformable model segmentation approaches use traditional edge detection techniques such as Sobel filters or Monga-Deriche operators to produce an edge image [10]. Some rely on a more local edge detector, Canny-edge operator, to create an attraction field [11]. The ideal edge detector is fast, and only detects relevant edges and is insensitive to noise. To approximate this ideal edge detector it is necessary to use as much image information as possible [12], that is 3D+T or 2D+T. The basic idea is that, by knowing the general shape, we can readily approximate a priori the direction in which to anticipate the edge. This allows us to use a direction-sensitive edge filter to find the edges in only that particular direction, thereby avoiding spurious edges in other directions. In Section 2.4 we show how to include temporal information in the edge detection at a very small computational cost to improve on this. The idea of including temporal information is not new. It has been previously been in [1,12,13,14]. We further separate the proposed edge detection scheme in two variants, depending on the type of edge being expected. These edge types are denoted as
106
4
PAPER IV
0 0
Concordant edge
Discordant edge
Figure 1. The arrows represent local gradients of the at the segmented surface. Black and white arrows indicate regions where the model normal and the image gradient have similar or opposite directions, respectively. Zeros denote homogenous areas with a small local image gradient. The black and white lines define the endocardial and epicardial surfaces respectively.
’concordant edge’ and ’discordant edge’. An example of these two edge types is shown in Fig. 1. For a concordant edge, both the direction of the local image gradient and the surface normal point is in approximately the same direction. When directions of local image gradient are discordant in some places of the edge we denote the edge as discordant. The edge type depends on the image modality being used, and whether the target surface is endocardial or epicardial. In the case of concordant edge the deformable model is attracted to a black-white transition. In the case of discorant edge the deformable model is attracted to any border/edge regardless of the type of transition (black-white, or white-black). The input image is processed with an edge detection scheme in four different directions φ = 0◦ , 45◦ , 90◦ , 135◦ (evenly distributed within half of the unit circle) producing four ’edge images’ Eφ , can be written as:
concordant edge: Eφ = I ∗ fφ ∗ saφ ∗ sb⊥φ
discordant edge: Eφ = I ∗ fφ ∗ saφ ∗ sb⊥φ ∗ dφ
(5)
where I is the input image, ∗ denotes image convolution, fφ is a small edgedetecting filter (second order differentiation), dφ is a derivate filter, and saφ is a small directional smoothing filter applied recursively a times. All filters are directionally sensitive, denoted by the index φ , and ⊥φ denotes a direction perpendicular to the direction φ. The kernels are f = [−2 1 2], d = [0 − 1 1], , s = [1 2 1]. The directional sensitivity is implemented by rotating the kernels. Due to the grid structure the bandwidth is not isotropic, but that does not pose any practical problems since this effect is smoothed out by the smoothing filters s. More advanced and optimized filters could be used instead of these, 5
107
PAPER IV
but the filters used here have the advantage of being very fast, and easy to change with the two intuitive parameters, a and b. Note that different filters are applied to the two edge types, it is not merely the absolute operator that differs. Furthermore, it is also possible to use more than four filters, if the filters were more angle sensitive. Too much angle sensitivity may not be advantageous, however, as the model also must be able to deform around sharp edges where the edge direction is less well defined.
2.3
Force Calculation
Minimizing the energy in Equation 1 gives an Euler-Lagrange equation, with three different types of forces. • image-dependent forces (balloon force, edge force) • internal forces (curvature, slice force, acceleration, damping) • user interaction forces (pin forces) The forces are calculated on points/nodes on the model o(l, t, z) surface. Each node-point in o store x, y coordinates, and z, t are given implicitly. The derivatives are calculated using the kernels h1 [−1, 0, 1], and h1 [1, −2, 1] for first order and second order differentiation respectively.
2.3.1
Image dependent forces
There are two image dependent forces, an inflating ’balloon force’, and one edge force. The ’balloon force’ is not the standard balloon force, but a position-dependent function of the image. Thus each node in the geometric representation is influenced by a different inflating/deflating force. By making the balloon force image-dependent, we are able to incorporate both regional image intensity information and edge information in the model. This idea builds on the ideas proposed by Chakraborty et al.[15]. How the image intensity is mapped to a force differs between the two types of edges, concordant and discordant, respectively:
concordant edge: F B (l, t, z) = n ˆ (α0 I(x, y, z, t) + λα1 )
−
discordant: F B (l, t, z) = n ˆ e
108
6
(I(x,y,z,t)−λ)2 α0
(6)
!
+ α1
(7)
PAPER IV
where I is the input image, λ is an estimated image intensity for the object, α0 , α1 are two image modality-dependent constants, and (x,y,z,t) is the spatiotemporal coordinate of the node-point o(l, t, z) How the coefficients α0 , and α1 are estimated is discussed in Section 2.6. The other image dependant force comes from the edges in the image. The edge detection scheme in Section 2.2 results in four precalculated edge images Ei . These edge images are mapped to an edge force by: |ˆ neˆl | El (x, y, z, t) + |ˆ neˆs | Es (x, y, z, t) |ˆ neˆl | + |ˆ neˆs |
(8)
(ˆ neˆl )El (x, y, z, t) + (ˆ neˆs )Es (x, y, z, t) |ˆ neˆl | + |ˆ neˆs |
(9)
concordant: F E (l, t, z) = n ˆ
discordant: F E (l, t, z) = n ˆ
where (x, y, z, t) is the spatio-temporal coordinate for the node-point o(l, t, z), and El is the precalculated edge image for the filter where |ˆ neˆl | is largest, and respectively Es is the filter response for which |ˆ neˆs | is second largest. Note that since we are using the model normal n ˆ to interpolate differently from the edge images we are only looking for edges in a priori estimated directions, and can disregard spurious edges.
2.3.2
Internal forces
There are four internal forces in the proposed algorithm, curvature, slice, acceleration, and damping forces. The purpose of the curvature force is to ensure that the model surface (in the short axis plane) is smooth. It is based on calculating the curvature (curvature along the parametric line following the contour in each slice) on the model surface.
F C (l, t, z) = n ˆ κ(l, t, z)
(10)
where κ is the curvature along a parametric curve, parametrized over l:
κ(l, t, z) =
x ˆ ∂o ∂l
2
2
yˆ ∂∂l2o − x ˆ ∂∂l2o
x ˆ ∂o ∂l
2
+ yˆ ∂o ∂l
yˆ ∂o ∂l
2 32
(11)
The slice force is a force that ensures that the model is smooth in the long-axis plane, however, note that the slice force only acts in the x-y plane, and not in the z-direction. It is based on the second derivative along the long-axis, and 7
109
PAPER IV
act to damp ’acceleration’ along model surface in the long axis direction, and is simply calculated as:
F Z (l, t, z) = n ˆ
∂ 2 o(l, t, z) ∂z 2
(12)
A big advantage between more geometrically complex models is that the equations for ensuring the smoothness of the model become much simpler with the chosen ’cone’ geometric model. The acceleration, and damping force are introduced to ensure temporal smoothness of the model. The acceleration force is simply the force according to Newton’s second law to accererate a particle, and the damping force is an imaginary force damping the velocity (and thus damping oscillations). They are calculated as: ∂o(l, t, z) ∂t
(13)
∂ 2 o(l, t, z) ∂t2
(14)
F D (l, t, z) = −ˆ n
F M (l, t, z) = n ˆ
2.3.3
User interaction force
In order to allow user interaction, so-called pin forces were included in the model formulation. Pin forces were introduced by McInerney et al.[5]. When a pin is placed, the closest point on the surface is located within the same slice and time frame. The force from that point is calculated, and the point and its neighbors are pulled towards the pin. The pin forces are the basis for the three basic types of user interaction with the segmentation program: (1) The user can place pins at specific locations and times, and the model can then be refined with a relatively small number of iterations. (2) The user can interactively drag the contour with the mouse. This is implemented with an invisible pin that the user moves while the optimization runs in real time. (3) The user can manually draw a section of the wall with a pen-tool. To subsequently include this change temporally, a set of pins are placed along the line drawn by the user. To increase the computational speed only the five (three for epicardium) closest nodes to a pin are attracted to the pin by the resulting force. The pin also attracts the contour in other timesframes than the point was placed, this is set to be 30 % of the timeframes. The user interaction force for the affected points is given by:
110
8
PAPER IV
F P (l, t, z) = wi,j n ˆ (p − o(l, t, z))
(15)
where p is the coordinate of the pin, and wi,j is a weight for the different nodes, and is given by:
1 1 + |i|
wi,j =
!
1 1 + |j|
!
(16)
where i are the number of node-points away from the closest point such as i = 0 for the closest point, and j are the number of timeframes away from the point, i.e j = pt − t.
2.4
Temporal edge detection
Temporal information in the edge detection is included by temporally smoothing the edge force at one node-point F E (l, t, z) over several timeframes.
1 F E¯ (l, tn , z) = P wi
wi =
X N
wi F E (l, tn+i , z)
(17)
i=−N
1 1 + |i|
(18)
where N is an image modality dependent constant controlling the amount of temporal smoothing, and F E¯ is the temporally smoothed edge force. Note that we smooth the edge force over spatially different positions o(l, tn+i , z) This means that if we have found the correct edge, then the spatial extent of the temporal smoothing filter is in fact optimal. The sum in (17) can be calculated by convolution, at a completly negligable cost in processing time compared to the model deformation.
2.5 Deforming the Model
The different kinds of forces are calculated for each node-point and summed to form the deforming forces as: 9
111
PAPER IV
F ext = αB F B + αE F E
(19)
F int = αC F C + αZ F Z + αM F M + αD F D F user = αP F P
where αB ...αP are image modality dependent coefficients, and F B is the balloon force, F E is the edge force, F C is the curvature force, F Z is the slice force, F M is the mass force (or temporal acceleration), F D is the damping force, and F P is the user interaction force (pin-force). These coefficients are hidden from the user, and they should be changed only when a different image modality is used. The coefficients used for the shown image modalities is given in Table 1. Section 2.6 describes how these parameters are tuned. Note that the forces Concordant edges: Image modality
αE
αC
αM
αZ
α0
α1
a
b
MR Gradient Echo endo
20
1
0.125
0.05
1
-0.7
1
2
MR SSFP endo
15
1
0.5
0.05
1
-0.5
1
2
CT Cardiac endo
10
0.5
-
0.01
1
-0.9
5
3
NM Blood pool SPECT endo
0.5
0.5
0.5
0.05
1
-0.8
0
0
US Echocardiography endo
-40
2
-
0
-1
1.5
10
10
Image modality
αE
αC
αM
αZ
α0
α1
a
b
MR Gradient Echo epicard
40
2
5
0.01
100
-0.5
2
2
MR SSFP epicard
30
5
2
0.02
10
-0.8
1
2
MR Late enhancement endo.
10
4
-
0.05
80
-0.5
2
4
MR Late enhancement epi.
200
10
-
0.5
40
-0.5
4
2
CT Cardiac epicard
200
40
-
0.05
1000
-0.5
5
5
Disconcordant edges:
Table 1 Coefficients used for different types of images. αE denotes edge force coefficient, αC curvature coefficient, αM mass force coefficient, αZ slice force coefficient, α0 ,α1 are mapping parameters for the image intensity to balloon force, a number of times a smoothing filter are applied (cross edge), b number of times the smoothing filter are applied (along edge). - denotes not applicable (image data not time resolved).
112
10
PAPER IV
are 2D vectors in the xy plane, and therefore the model cannot change size in in the ’z-direction’ (slice direction). Left ventricular long axis motion is incorporated by using an ad hoc model, described in the Section 2.9. The model is deformed according to:
on+1 = on + γ (ˆ nF ext + n ˆ F user − n ˆ F int ) n ˆ
(20)
The number of points in the model starts at 25 per slice/time frame, and after 150 iterations it is increased to 40, and after 30 more iterations it is increased to the final value of 80. The constant γ is set at 0.75 times the minimal distance between two node points divided by the largest force on the model. It is decreased to a minimum value of 0.05 times the normalization. In order to achieve numerical stability, after each iteration the points defining the contour of each slice in every time frame are redistributed to maintain equidistance between points. Care is also taken to avoid rotation between slices by maintaining the correspondance between individual reference points for each slice. This is done by in each slice tracking the point that has the smallest angle between the x-axis, and center of gravity of the model.
2.6
Tuning the image modality coefficients
The easiest method of tuning the parameter set when introducing a new image modality is by taking the parameters from the image modality that are most similar, and then adjusting the parameters. The most critical parameters are α0 , and α1 , they describe how the image intensity is mapped to an inflating deflating force. However, tests have shown that they are not more critical than the same coefficents can be used on images for different MRI manufacturers. Note that α0 is either 1 or -1 for the concordant edge case. These parameters are tuned by manual inspection of a balloon image in one slice. The edge image coefficient αE is gradually increased until the surface is stopped at the desired edges. The curvature parameters should initially not be set too low since that might lead to numerical instability. The curvature coeffiecient αC is then adjusted to the desired edge smoothness. Generally the slice coefficient αZ is not critical, and its value depend mainly on the slice resolution. The integer valued parameters a and b regulating smoothing in the edge detection are easily adjusted based on the image resolution, and by visual inspection of the edge images. Finally the amount of temporal smoothing in the edge detection is fixed to 15% of the heart cycle. 11
113
PAPER IV
2.7
Papillary muscles
It is not trivial how to treat papillary muscles. The current recommendation in clinical practice is to include papillaries that are attached to the wall, and exclude the papillaries in slices where they have detached from the wall. [16,17]. When doing wall motion analysis the clinical practice is to exclude them completly and in order to be able to follow the cardiac wall and not let the papillaries interfere with wall thickness and wall motion calculation. To be able to have the best of two worlds, i.e facilitate wall motion analysis and correct LV volume calculatio a hybrid scheme was designed. The segmentation is performed while excluding the papillary muscles using an ad-hoc algorithm. After a course segmentation of the endocardium (fewer iterations), a shape was fitted based on its L0.5 norm in each slice and timeframe. The shape was given on polar form by the equation:
r(ω) = c0 + c1 sin(2ω) + c2 cos(2ω)
(21)
where r is the radius, and ω is the angle and c0 , c1 , c2 are coefficients for each slice. The reason for taking the L0.5 norm is to avoid weigthing the papilary muscles to the curve, i.e we want a curve that fits the original curve without doing an average solution including the papillary muscles. The fitting process is intialized with the L2 solution which can be calculated by the least square solution of an over determined equation system. The parameters are then sucessively adjusted for 20 iterations using a gradient search. Thereafter the shape is refined using the deformable model approach given in this paper by another 20 iterations. This process gives a segmentation that enables good wall motion analysis, and an example is given in Fig. 5. Thereafter the volume of papillary muscle and trabeculation within the ’blood pool’ is estimated by using a threshold of the pixels within the blood pool. The threshold is calculated as the mean intensity in the myocardium plus 30 % of the difference of the signal intensity in the blood pool and the intensity in the myocardium.
2.8
Coupling endocardium and epicardium
To get the clinically important parameter myocardial mass, it is necessary to deliniate both the endocardium and the epicardium. Ideally one would like to couple the epicardial surface to the endocardial surface. This is not possible with the current approach that ’inflates’ the endocardial surface, until it fits to the hopefully true endocardial border. Furthermore the parameter λ is estimated as the mean image intensity at a small distance outside of the endocardium. Therefore an ad-hoc solution has been applied to restrict the
114
12
PAPER IV
epicardial surface within close vicinity of the endocardium. The epicardial surface is initiated at the position of the endocardial surface, and at the position of the endocardium a strong inflating force (stronger than the maxiumum expanding/deflating force) is applied to avoid a situation where the endocardium lies outside of the epicardium. If the distance between the endocardium and epicardium is larger than a constant set at 2.5 cm, then a local deflating force linearly increases pulling the epicardium towards the endocardium. The imporantance of this restriction is that it reduces the amount of manual correction necessary if the epicardial edge detection fails.
2.9
Long Axis Motion
The principal deformation of the left ventricle can be separated in two major modes. The LV walls move inward and thicken, creating a deformation in the short axis. At the same time, the atrioventriclar plane moves towards the cardiac apex, creating a long axis deformation. Long axis motion is a very important component of LV relaxation and contraction, but is often neglected when estimating global left ventricular functional parameters such as ejection fraction. The ideal solution would be to track the atrioventricular valve plane from the images, but with the current through plane resolutions (with MRI typical around 8mm) this is not possible. Therefore an alternative scheme to include long-axis motion was designed. The long-axis motion is measured from a separatly acquired acquisition. If the measured long-axis motion was for instance 1.6 cm and the slice thickness is 1cm, then in end-systole the most basal slice is not included at all, and the second most basal slice is accounted for with 40%. For other phases in the heart cycle the above correction is applied with a fractional value based on the left ventricular volume (at max volume, end-diastole no correction is made). If no separate long-axis image is acquired then, the long-axis motion can still be estimated based on the assumption that the volume of the myocardium itself remains constant over the complete heart cycle. There are indications that the LV mass varies by a small amount from end-diastole to end-systole [16], but the difference is small. The fluctuation in the curve relating myocardial volume to time reflects errors related to not taking the long axis motion into account. The variability in the myocardial volume over time can be diminished by progressive exclusion of the most basal slices (closest to the atrium) in order to acheive a constant myocardial volume. A myocardial volume curve with, and without correction for long axis motion is shown in Fig. 2. In this example the ejection fraction increased from 53% to 73% with an automated detected long axis motion of 13mm (measured in a long axis image to 12.6mm). When estimating the left ventricular volume the volume from the papillary muscle estimation were included. 13
115
PAPER IV
Volume [ml]
160 140 120 100 80 60 40 0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Timeframe Figure 2. Myocardial volume over the cardiac cycle with (squares) and without (triangles) correction for long axis motion. The corresponding LV blood volume (excluding papillary muscles) is also shown (circles).
3
Materials
The SSFP MRI datasets were acquired on a Philips Intera scanner, the gradient echo MRI and the Late Enhancement datasets on a Siemens Magnetom Vision scanner, the echocardiographic image sequence on a GE VingMed System FiVe scanner, and finally 3D Cardiac CT images were acquired using a Siemens Sensation 16 scanner. For quantitative purposes nine gradient echo images (healthy volonteers), and 14 SSFP images from a broad patient population, and 7 Cardiac CT data sets were used. A graphical user interface was implemented in Matlab (Mathworks, Natwick, USA) and compiled to a standalone application. The more time-intensive parts of the algorithm such as filtering and model deformation were implemented in C. A fully functional version of the software including installation instructions and documentation can be downloaded from the software homepage (http://www.cmiv.liu.se/software).
4
Results
The segmentation algorithm was tested for feasibility on six different image types, MRI Gradient Echo, MRI SSFP, MRI Delayed enhancement, Cardiac CT, echocardiography image sequence, and gated blood pool SPECT. Examples of the results are shown in Figure 3. To validate the accuracy of the method nine gradient echo (healthy volonteers), 14 MRI-SSFP data sets (from a mixed patient population) and 7 Cardiac CT data sets (mixed patient population) were analysed both manually and automatically with no user interaction. Papillary muscles was set to be excluded to be able to perform wall motion analysis. The volume of the papillary muc-
116
14
PAPER IV
sles were included in the volume calculation as described in Section 2.7. Left ventricular mass, endsystolic and diastolic volumes were compared, and the results are shown using Bland-Altman plots [18] in Fig. 4. The results of the epicardium detection was not satisfactory without manual corrections for gradient echo images, furthermore not all the Cardiac CT data sets were time resolved due to considerations of radiation exposure. The segmentation results is not clinically applicable without manual corrections. Time to do the manual corrections (time resolved) is about 2-3 minutes (depending on the data quality). The largest difference between manual and automatic segmentation was in the two most basal slices. A percentage error between automatic and manual end-diastolic volume was defined as the difference in volume divided by the manually measured volume. The standard deviation of the error in the most basal slice was 37%, whereas for all slices the error was 3.7%. The total computational time for segmentation of the whole left ventricle in a typical MRI dataset with 12 slices and 30 timeframes was approximately 40 seconds on a 1.4GHz Intel Pentium 4 PC. The time for the edge detec-
a)
b)
c)
d)
e)
f)
Figure 3. Examples segmentation results on short axis images on different patients and different image modalities. a) Surface rendering of result on Cardiac CT data (endo and epi contour). b) Echocardiographic image sequence. c) Gated blood pool SPECT image. d) MRI SSFP image sequence. e) MRI Gradient echo image sequence. f) Delayed enhancement image, the small circles are user placed pins to guide the epicardial surface.
15
117
PAPER IV
30 EDV Auto-Manual [ml]
EDV [ml] Auto
2 400 y=0.97x+2.2 R =0.99 P |λ2 | Eigenvectors corresponding to λi Domain of convolution
NOTATION AND ABBREVIATIONS γ ˆ U Pˆ T I F n ˆ
Normalization factor Normalized velocity field Pattern (vector filter) Local orientation or similarity tensor Identity tensor Force on deformable model Surface normal on deformable model
149
Acknowledgments This thesis is dedicated to my wife Lisa – thanks for all love and support! Lisa, Anton and Agnes thanks for being so lovely and for reminding me that most important aspects of life is not cardiac imaging. I want to thank my supervisor Matts Karlsson for being a constant source of inspiration and for all good and crazy ideas and having an open mind on possible future applications. Thanks to my past and current research colleagues for making it so fun and exciting to work, Lars, Tino, Andreas, Calle, John-Peder, Per, Pernilla, Bengt, Ann, and many more. I would also like to thank all my new friends at Department of Clinical Physiology in Lund for the warm welcome and friendly atmosphere and hope for an exciting research period with you.
H¨ogan¨as, December 2004
Einar
