Combining Snakes and Active Shape Models for

IEEE Computers in Cardiology 2000 Vol 27

Combining Snakes and Active Shape Models for Segmenting the Human Left Ventricle in Echocardiographic Images G Hamarneh, T Gustavsson Chalmers University of Technology, Göteborg, Sweden Abstract We propose a method for segmenting the human left ventricle (LV) in ultrasonic images, which is based on principles from both Active Shape Models (ASM) and Active Contour Models (ACM). Principal Component Analysis (PCA) is applied to a frequency-based shape representation of the LV thus eliminating the need for the difficult determination of corresponding landmarks. The average ventricular shape and the set of most significant shape variation modes are obtained from a training set. The LV boundaries in new images are found by placing an initial ACM (Snake) and allowing it to deform only according to the examined shape variations. A training set of 105 expert-segmented echocardiographic images was used. Improvements in the segmentation results were obtained especially in cases where the ventricular boundary was partly occluded by noise.

1.

Introduction

Ultrasound echocardiography is a non-invasive nonexpensive tool for clinical diagnosis and analysis of heart function. A vital step towards this analysis is the segmentation of endocardial boundaries of the left ventricle (LV) [5,7,9,10,11,12]. Ultrasonographic heart images are often characterized by weak echoes, echo dropouts and high levels of speckle noise causing erroneous detection of LV boundaries. Snakes or Active Contour Models (ACM) [6] and its variants [1,2,4,8] overcome part of these limitations by considering the boundary as a single inherently connected smooth structure, and by supporting intuitive interactive P re p a re a tra in in g set o f L V im ag es

E x p e rt d e lin e a tio n w ith o u t p o in t c o rre s p o n d e n c e

mechanisms for guiding the segmentation. In our application, human guidance is often needed to guarantee acceptable results. A helpful strategy is to present the Snake with a priori information about the shape of the LV. Statistical knowledge about shape variation can be obtained using Point Distribution Models (PDM), which are central to Active Shape Models (ASM) [3]. PDMs, which are obtained by performing Principal Component Analysis (PCA) on landmark coordinates labeled on many example images, have been applied to the analysis of echocardiograms [11]. This is problematic since it requires defining and labeling corresponding stable landmarks. We adopt a similar approach for capturing the main modes of ventricular shape variation, however, we represent shapes by descriptors that eliminate the need for point correspondence, namely the Discrete Cosine Transform (DCT) coefficients. We then use ACM for segmentation but with its deformations constrained according to the prior knowledge of ventricular shape.

2.

Methods

A general overview of the method is depicted in Figure 1. To arm the Snake model with a priori information about typical shape variations of the LV, a training set of images combined with medical experts’ delineations were obtained. This set of manually traced contours is used to obtain a model of the typical ventricular shape variations by first applying a reparameterization of the contours, which gives a set of DCT coefficients replacing the spatial coordinates, and then applying PCA to find the strongest modes of shape variation.

D C T of c o o rd in a te s

PC A of D C T c o e ffic ie n ts

D e fin e a llo w a b le s p a c e

D C T of S n a k e c o o rd in a te s

P ro je c t D C T c o e ffic ie n ts o n a llo w a b le s p a c e

T ra in in g N ew L V im ag e to s e g m e n t

In itia liz e S n a k e

D e fo rm S n a k e w ith tra d itio n a l fo rc e s yes

D one!

no C o n v e rg e ?

N e w c o n s tra in e d sn ak e

ID C T o f p ro je c te d c o e ffic ie n ts

A p p lic a tio n

Figure 1. Flowchart illustrating our method for segmenting the human LV.


This results in an average ventricular shape represented by a set of average DCT coefficients, and the principal components along with the fraction of variation each component explains. To segment a new LV image, we initialize a Snake and, unlike classical Snakes, it is not allowed to deform freely, but instead its deformations are constrained in such a way that the resulting contour is similar to the training set. To attain the constrained deformations we first obtain the DCT coefficients for the active contour coordinates, then project them onto an allowable Snake space defined by the main principal components obtained previously. This is followed by an Inverse DCT (IDCT) that converts the frequency-domain shape parameters back to spatial coordinates. This is repeated until convergence, which is defined when the majority of Snake nodes do not change their locations significantly.

2.1.

Active Contour Model Formulation

To fit the Snake model to the image data while maintaining contour smoothness, different forces are exerted on the nodes deforming the Snake so as to minimize its total energy. This energy depends on both the shape of the Snake’s contour and its location within the image data ) X Y and is reflected via internal and external energy terms, respectively [6]. The equation we used here to update the position of the Snake nodes is %T VI T %T VI T B&ITENSILE T H (1)

ÅÅÅÅÅÅÅÅÅC &IFLEXURAL T &IEXTERNAL T &IINFLATION T where VI T X I T YI T are the Snake nodes, I K . , H is a damping coefficient, B and C are weighting factors, %T is a finite time step. &ITENSILE T VI T VI T VI T

(2) is a tensile force (resisting stretching) acting on node I at time T . TENSILE &IFLEXURAL T &ITENSILE T &ITENSILE T &I T (3) is a flexural force (resisting bending) . &IEXTERNAL T 0 X I T YI T

(4) is an external image-derived force that moves the snake towards regions of higher intensity gradient in the image . 0 X Y r ) S X Y and ) S X Y is the intensity of the pixel X Y in a smoothed version of the image.

&IINFLATION T & ) S X I YI NI T

(5) is an inflation force where NI T is the unit vector in the direction normal to the contour at node I , £¦ IF Å) X Y p 4 & ) X Y ¦¤ (6) ¦ OTHERWISE ¦¥ is a binary function that links the inflation force to the image data, and 4 is an image intensity threshold.

2.2.

Shape Re-parameterization

The one-dimensional discrete cosine transform (DCT) of the sequence of X I contour coordinates (and similarly for the YI coordinates) is defined as

.

8 K W K X I COS

I

Q I K

.

(7)

and the inverse DCT is give as XI

.

W K 8 K COS

K

Q I K

.

(8)

where K K . and I K . and £¦ . Å K ¦ W K ¤ ¦¦ . Å b K b . ¦¥ The DCT was favored as the frequency domain shape parameterization because of its excellent energy compaction properties, its real coefficients, and the correspondence between the coefficients since they capture specific spatial contour frequencies.

2.3.

Principal Component Analysis

To identify the main modes of shape variation in the training contours, we perform PCA on their DCT coefficients. The same number, say - , of DCT coefficients is obtained for the set of X and Y coordinates representing each shape either by interpolating the spatial coordinates or truncating the DCT coefficients. PCA then yields the principal components (PCs) or main variation modes, A J , and the associated variance explained by each mode, MJ . Only the first T PCs are used, i.e. J K T , since they are sufficient to explain most of the variance. The average of the coefficient vectors, 8 , is also calculated.

2.4.

Constraining Contour Deformations

Presented with a new image, a Snake contour is first initialized and then allowed to deform according to equation (1). In order to guarantee a Snake contour resembling an acceptable shape, we constrain the resulting deformed contour, \ VI T I K . ^ , by obtaining the - DCT coefficients for the contour, 8 , and projecting the coefficient vector onto the subspace of principal components (allowable shape space) as follows 8 PROJ 8 !B (9) where ! ¡ A A K AT ¯° is a matrix containing the ¢ ± main variation modes and B is a vector of scalar weighing factors and is calculated as B !4 ! !4 8 8

(10) Before performing the IDCT, we restrict the projected


coefficients ( 8 PROJ ) to lie within o MJ since in most applications the population typically lies within three standard deviations from the mean.

3.

Results

We applied our method on real echocardiographic data that included 105 images of the human LV, which were manually traced by medical experts. There was no point correspondence between the frames. The number of traced points varied between 28 and 312. The DCT of the manual tracings was calculated followed by PCA on the truncated coefficients where 5 variation modes of 56 possible were enough to explain 95% of the total variation, 12 were enough for 99%, and 24 for 99.9%. Figure 2 depicts the first two shape variation modes. To illustrate the method, we first used test examples where Gaussian noise was added to the manual tracings, DCT was calculated, truncated then projected on the allowable shape space, and then IDCT was performed. It was visually obvious how the constrained contour resembles a much more plausible boundary of the LV than the noisy one, see Figure 3. Next we tested our method on real

-1 0 0

echocardiographic data by initializing the Snake on an image that wasn’t included in the training set (i.e. cross validation) and allowing it to deform. This was followed by a DCT-Truncation-Projection-IDCT procedure. The result of the Snake segmentation alone, due to noise and echo dropouts in the image, often gave unreasonable and unacceptable shape of the LV. Conversely, employing constrained deformations resulted in acceptable LV boundaries. See the different example results in Figure 4 and Figure 5.

4.

Conclusions

We present a method for segmenting the human LV in echocardiographic images based on constraining the deformations of a traditional Snake so that only allowable (similar to training examples) segmentation results are obtained. The method utilizes the strength of ACM in producing smooth and connected boundaries along with the strength of ASM in producing shapes similar to those in a training-set. Better results were obtained when using the new method for segmenting the LV compared to those obtained with classical snakes. 150

150

100

100

50

50

m ea n

-5 0

P C 1±1 std

0 50

0

50

100 (a)

150

0 50

150

150

100

100

50

50

100 (b)

150

100 (d)

150

P C 2±1std 100

150 -1 5 0

-1 0 0

-5 0

0

50

100

150

0 50

Figure 2. Mean contour and the first and second variation modes (weighted by ±1 std).

Acknowledgements Ghassan Hamarneh is funded by the Visual Information Technology (VISIT) program supported by the Swedish Foundation for Strategic Research (SSF).

Amini A, Weymouth T, Jain R. Using dynamic programming for solving variational problems in vision. IEEE Trans. on PAMI 1990;12(9):855-67.

150

0 50

Figure 3. (a) Manual tracing. (b) Noise added to (a). (c) IDCT of truncated DCT coefficients of (b). (d) The projection of (c) on the allowable shape space (note the similarity to (a)). [2] [3]

[4]

References [1]

100 (c)

[5]

Cohen L. On active contour models and balloons. CVGIP: Image understanding 1991;53(2):211-8. Cootes T, Taylor C, Cooper D, Graham J. Active Shape Models-Their Training and Application. Computer Vision and Image Understanding 1995;61(1):38-59. Grzeszczuk R, Levin D. “Brownian strings”: Segmenting images with stochastically deformable contours. IEEE Trans. on PAMI 1997;19(10):1100-14. Hunter I, Soraghan J, Christie J, Durrani T. Detection of echocardiographic left ventricle boundaries using neural networks. Computers in Cardiology 1993:201-4.


[6]

Kass M, Witkin A, Terzopoulos D. Snakes: Active Contour Models. International Journal on Computer Vision 1988;1(4):321-31. [7] Malassiotis S, Strintzis M. Tracking the left ventricle in echocardiographic images by learning heart dynamics. IEEE Trans. on Medical Imaging 1999;18(3):282-90. [8] McInerney T, Terzopoulos D. T-Snakes: Topology adaptive snakes. Medical Image Analysis 2000;4:73-91. [9] Mikic I, Krucinski S, Thomas J. Segmentation and tracking in echocardiographic sequences: active contours guided by optical flow estimates. IEEE Trans. on Medical Imaging 1998;17(2):274-84. [10] Papadopoulos I, Strintzis M. Bayesian contour estimation of the left ventricle in ultrasound images of the heart. IEEE Conference on Engineering in Medicine and Biology Society 1995;1:591-2.

[11] Parker A, Hill A, Taylor C, Cootes C, Jin X, Gibson D. Application of point distribution models to the automated analysis of echocardiograms. Computers in Cardiology 1994:25-8. [12] Taine M, Herment A, Diebold B, Peronneau P. Segmentation of cardiac and vascular ultrasound images with extension to border kinetics. IEEE Proceedings of the Ultrasonics Symposium 1994;3:1773-6. Address for correspondence. Ghassan Hamarneh, Department of Signals and Systems, Chalmers University of Technology, Göteborg, SE-412 96, Sweden. Email: [email protected]

(a)

(b)

(c)

(d)

Figure 4. The progress (a to d) of a Snake overlaid on an ultrasound image of the LV (dashed) and the result of the DCT-Truncation-Projection-IDCT (continuous).

Figure 5. Snake contours (dashed) and the constrained contours (continuous) with increasing number of iterations (left to right, top to bottom).