Using Prior Shapes in Geometric Active Contours in a ... - CiteSeerX

International Journal of Computer Vision 50(3), 315–328, 2002 c 2002 Kluwer Academic Publishers. Manufactured in The Netherlands.

Using Prior Shapes in Geometric Active Contours in a Variational Framework YUNMEI CHEN Department of Mathematics, University of Florida, Gainesville, FL 32611, USA [email protected]

HEMANT D. TAGARE Department of Diagnostic Radiology and Department of Electrical Engineering, Yale University, New Haven, CT 06520, USA [email protected]

SHESHADRI THIRUVENKADAM, FENG HUANG AND DAVID WILSON Department of Mathematics, University of Florida, Gainesville, FL 32611, USA [email protected] [email protected] [email protected]

KAUNDINYA S. GOPINATH AND RICHARD W. BRIGGS Department of Radiology, University of Florida, Gainesville, FL 32611, USA [email protected] [email protected]

EDWARD A. GEISER Division of Cardiology, Department of Medicine, University of Florida, Gainesville, FL 32610, USA [email protected]

Received December 18, 2001; Revised July 18, 2002; Accepted July 18, 2002

Abstract. In this paper, we report an active contour algorithm that is capable of using prior shapes. The energy functional of the contour is modified so that the energy depends on the image gradient as well as the prior shape. The model provides the segmentation and the transformation that maps the segmented contour to the prior shape. The active contour is able to find boundaries that are similar in shape to the prior, even when the entire boundary is not visible in the image (i.e., when the boundary has gaps). A level set formulation of the active contour is presented. The existence of the solution to the energy minimization is also established. We also report experimental results of the use of this contour on 2d synthetic images, ultrasound images and fMRI images. Classical active contours cannot be used in many of these images. Keywords: segmentation, registration, shape prior, active contour, variational method

316

1.

Chen et al.

Introduction

Geometric active contours are widely used for segmentation, that is, for finding object boundaries in images. Geometric active contours are user initialized curves which dynamically evolve in the image to minimize their energy (or length). Their energy is designed in such a way that its minimum occurs when the trace of the curve is over points of high gradient in the image. Because object boundaries are often defined by such points, the active contour becomes stationary at the boundary. There is a serious practical problem with active contours—they do not becomes stationary at a boundary if the boundary has segments which have low gradient. A geometric active contour will often “leak” through such “gaps” in the boundary. The problem is that standard active contours do not have any information about how the gaps are to be bridged. One solution to the problem is to incorporate into the active contour some prior information about the expected overall shape of the boundary. Then the active contour can compare its shape with the expected shape and bridge the gaps in a meaningful way. In this paper, we propose a variational framework for incorporating prior shape knowledge in the active contour. We modify the energy function of the contour so that it depends on the image gradient as well as the prior shape. The modified energy function gives a satisfactory segmentation in the presence of gaps, even when the gaps are a substantial fraction of the overall boundary. We specify the entire prior shape, because a priori, we do not know where the low contrast segments will occur in the boundary. The key idea in our algorithm is the creation of a shape term in the energy of the active contour. The notion of shape of n points in the plane (or higher dimensions) is now well established (Mardia, 1998; Small, 1996). The shape of a data set is usually defined as: When all the information in a data set about its location, scale, and orientation is removed, the information that remains is called the shape of the data set. (Small, 1996) The shape term in our energy function is consistent with this notion—it is independent of the translation, rotation, and scale of active contour and truly represents shape.

Active contours with prior shapes are especially useful in segmenting medical images. In certain medical imaging modalities, object boundaries have poor contrast and resolution, in other modalities anatomical objects are only partially imaged. In functional MRI, for example, images have a low resolution and anatomy appears at low contrast. Also, ultrasound images (esp. cardiac ultrasound images) exhibit signal drop out so that some segments of a boundary are not be visible in the image. Furthermore, ultrasound images often have a limited imaging range and are not able to image an entire organ. A standard active contour cannot successfully segment these images. On the other hand, an active contour with a shape prior can segment them (as we show in Section 6). In addition to using the prior shape to modify the active contour, our algorithm also provides accurate estimates of translation, rotation and scale between the active contour and the prior shape. These estimates are useful in their own right. They can be used to align images that have homologous curves. We discuss this in one of the experiments in Section 6. There is a simple level set form of the active contour which we propose. Further, we report experimental results on synthetic images, ultrasound images, and functional MRI images. We show that with appropriate prior shapes, our technique is capable of finding boundaries in these images. The application to functional MRI is somewhat unusual. The aim is not only to segment the images, but to use the segmentation for aligning the images. We use the estimates of translation, rotation and scale to do this. We show that the resulting alignment significantly reduces the number of false activation areas. Finally, we mention a key aspect of our variational approach. It turns out that the existence of the minimizer of our energy function can be, and is, established theoretically. This is an important benefit of the variational framework. 1.1.

Organization of the Paper

This paper is organized as follows. Below, in Section 2, we briefly review geometric active contours and relate our technique to other reported techniques. In Section 3, we propose our model for incorporating prior shape in active contours. The level set form is in Section 4. Section 5 contains the numerical method. Experimental results are reported in Sections 6 and 7 concludes the paper. The proof of the existence of

Using Prior Shapes in Geometric Active Contours

the minimizer to the energy function is contained in Appendix A. 2.

Background and Literature Review

The literature on active contours is vast and we do not review all of it here. Our aim is to briefly discuss the formulation of active contours and then review work that directly deals with the incorporation of shape in active contours. 2.1.

Active Contours: Edge- and Region-Based

Let ⊂ R 2 be a closed and bounded region of the plane (say a rectangle), and I : → R be an image defined on . Let C( p) = (x( p), y( p)) ( p ∈ [0, 1]) be a differentiable parameterized curve in . The classical snake model proposed by Kass et al. (1988) is formulated by minimizing the energy function:

1

|C ( p)|2 dp − λ

0

1

|∇ I (C( p))| dp.

(1)

0

The first term in Eq. (1) is called the internal energy and is used to control the smoothness of the curve. The second term is the external energy and is used to attract the contour towards the boundary of the object. In Caselles (1993), Malladi (1995), Caselles et al. (1997), and Malladi et al. (1995) proposed the following geometric deformable contour model based on the level set interpretation of the Euclidean curve: u t = g(|∇ I |)|∇u|(div(∇u/|∇u|) + ν),

(2)

where g(|∇ I |) can be chosen as g(|∇ I |) =

1 , 1 + β|∇G σ ∗ I |2

(3) |x|

where β > 0 is a parameter, and G σ (x) = σ1 e− 4σ 2 , so that g(|∇ I |) is strictly positive in homogeneous regions and near zero on the edges (Caselles, 1997; Cootes IVC, 1999; Yezzi, 1997). In this formulation, the evolving contour is the zero level set (u(x, y, t) = 0) of the solution of Eq. (2). The level set formulation is an improvement on the original formulation because it can handle multiple contours and topological changes. A further modification called geometric active contour was proposed in Caselles (1997) and Kichen (1995). The main modification was to propose an image

317

gradient dependent Riemannian arc length function for the contour. The arc length was taken to be the “energy” E(C) of the contour and was designed in such a way that minimizing it leads to active contour becoming stationary at pixels of high gradient:

1

E(C) =

g(|∇ I |)(C( p))|C ( p)| dp,

(4)

0

where g(|∇ I |) is the function defined in Eq. (3). In many images, object boundaries cannot be detected by seeking pixels of high image gradient. Region-based methods that make use of homogeneity on the statistics of the localized features and properties have been developed for such images. In Zhu (1996) the region competition algorithm was presented. This algorithm combines the attractive geometrical features of snake/balloon models and statistical technique of region growing, and was derived by minimizing a generalized Bayes/MDL criterion using the variational principle. cancel: One of the The region-based methods proposed in Cremers (2001), Chan (2001), Chan and Vese (2001), and Tsai (2001) are an active contour based on minimizing the Mumford-Shah functional (Mumford, 1989). Recently, the Geodesic Active Region models (Paragios, 1999, 2000, 2001) were proposed. These models combine boundary (Caselles, 1997; Kichen, 1995; Kass, 1988; Cohen, 1991) and region-based (Samson, 2000; Zhu, 1996) segmentation models. The evolution of the active contour is influenced by a region force that optimizes the segmentation according to the expected intensity of different regions, and a boundary force that contains information regarding the boundaries between the different regions. 2.2.

Using Shape in Active Contours

In Cootes IPMI (1999) and Wang (1998), a statistical model of shape variation was constructed from a set of corresponding points across the training images. This information was used in a Bayesian formulation to find the object boundary. In Cootes (1995) a Gaussian model was fit to a training set of corresponding feature points. In Cootes IVC (1999) mixed models were used to fit to the data for specific applications where the distributions are non-Gaussian. In an alternate approach in Staib (1992), Staib and Duncan specified the shape of the curve by creating statistical priors on the Fourier coefficients of the contour.

318

Chen et al.

This prior was incorporated into segmentation processing in a Bayesian framework. Szekely et al. (1996) also developed Fourier parameterized shape models. In their model, an elastic fit of of the shape model in the subspace of eigenmodes restricts the deformations. More deformable models in medical image segmentation can be found in McInerney (1996). Another approach using two-dimensional shape templates was presented in Tagare (1997), where a deterministic energy function was created whose minimum corresponded to a boundary in the image that is similar shape to the template. In Yuille (1992) a method of deformable templates was proposed for feature extraction from faces. The features of the interest were described by a parameterized template which interacts dynamically with the image to minimize the energy function. Recently, statistical shape knowledge has been incorporated in the edge based or region based active contours. In Leventon (2000), Leventon, Grimson and Faugeras extended geometric active contours by incorporating shape information into the evolution process. They used principal component analysis and signed distance functions of the curves to form a statistical shape model from a training set. The segmentation process embeds an initial curve as a level set of a higher dimensional surface, and evolves the surface based on image gradients and curvature towards a maximum a posterior estimate of shape and pose. In Cremers (2001) Cremers, Schn orr and Weickert incorporated statistical shape knowledge into the Mumford-Shah segmentation scheme (Mumford, 1989) by minimizing a functional that includes the shape energy and the Mumford-Shah energy. Prior statistical shape information is also used to control a total deformation of the active contour formed by global transformation (e.g. rotation, scaling, and traslation) plus a local deformation (see Paragios, 2002; Rousson, 2002; Satto, 2002). 3.

Active Contour with a Shape Prior

We now present our variational approach for image segmentation using prior shape information. The main idea behind our active contour is to propagate it by a velocity that depends on the image gradients and the shape prior. The velocity is designed such that the propagation stops when the active contour arrives at high gradients and forms a shape similar to the shape prior. The shape prior is obtained by a modified procrustes

method (Chen et al., 2001b), and the evolution of the active contour is governed by minimizing an energy functional depending on image gradient and the shape prior. Our model differs from the model in Leventon (2000) because we use a variational approach instead of a probability approach. That makes it possible to discuss the existence of the model solution in BV (bounded variation) space. To begin the mathematical description, we first specify when two curves have the same shape. Two curves C1 and C2 have the same shape, if there exist a scale µ, a rotation matrix R (rotation by an angle θ ), and a translation vector T such that C1 coincides with C2new = µRC2 + T . As in Section 2, let C( p) = (x( p), y( p)) ( p ∈ [0, 1]) be a differentiable parameterized curve in an image I with a regular parameter p. Let C ∗ be a curve, called the shape prior, representing the shape we expect of the boundary, and let g(|∇ I |) be the function defined in Eq. (3). To get a smooth curve C that captures higher gradients we would like to minimize the arc-length of C in the conformal metric ds = g(|∇ I |)(C( p))|C ( p)|dp. To capture the shape prior C ∗ , we would like to find a curve C and the transformation µ, R, T , such that the curve C new = µRC + T and C ∗ are closely aligned. Therefore, to measure the amount of high gradients under the trace of C and the closeness of C to C ∗ we introduce the energy function E(C, µ, R, T ), E(C, µ, R, T ) =

1

g(|∇ I |(C( p)))

0

λ 2 + d (µRC( p) + T ) |C ( p)| dp, 2 (5) where, λ > 0 is a parameter, and d(x, y) = d(C ∗ , (x, y)) is the distance of the point (x, y) from C ∗ , In the numerical computation, this distance function is obtained by the fast marching method proposed by Sethian. The first term on the right hand side of Eq. (5) measures the amount of high gradient under the trace of the curve and the second term measures the closeness to the shape prior. The curve C and the transformation parameters µ, R and T evolve to minimize E(C, µ, R, T ). The evolution is done by gradient descent with respect to µ, R and T and first variation descent with respect to C. At the stationary point of the descent, we expect C to lie over points of high gradient in the image, and

Using Prior Shapes in Geometric Active Contours µ, R and T determine the transformation that maps C to C new . The gradient/variation descent for the energy of Eq. (5) is performed by the following system: ∂C = −vn, C(0, p) = C0 ( p), (6) ∂t ∂µ = −λ d∇d · RC|C ( p)| dp, µ(0) = µ0 , (7) ∂t ∂θ dR = −λµ d∇d · C |C ( p)| dp, θ (0) = θ0 , ∂t dθ (8) ∂T (9) = −λ d∇d|C ( p)| dp, T (0) = T0 , ∂t where n is the outward unit normal to C, v = ∇g · n + gk + λµ(d∇d) · (Rn) + λd 2 k, and k is the curvature of the curve C. In Eqs. (7)–(9), the function d is evaluated at µRC( p) + T , while the functions g, k, and n are evaluated at C( p). Finally, it remains to specify how the prior curve C ∗ is obtained. Our approach to finding C ∗ is to use a set of training images which contain example curves with similar shape, and to extract an average of these curves. Suppose that the training set contains n given curve C1 , . . . , Cn with similar shape, but different size, orientation and translation. Let A1 and A2 denote the interior regions of the curves C1 and C2 as subsets of R2 . Define a(C1 , C2 ) = area of (A1 ∪ A2 − A1 ∩ A2 ).

(2.6)

We use a(C1 , C2 ) to measure the similarity of the shapes of C1 and C2 . To compare the shapes of the n contours, we fix C1 , realign C j ( j = 2, . . . , n) to C1 by finding a scale µ j , a rotation matrix R j and a translation vector T j such that the area a(C1 , C new j ) is = µ R C + T . After the reminimized, where C new j j j j j alignment of these curves, we find the average shape C ∗ by C ∗ = (C1 + n2 C new j )/n. In case the sample curves have large shape variations, we use a clustering process to create the prior. The procedure is as follows. Fix a curve from the training set (say C1 ), and align the rest of the curves to this curve by the method metioned above. Define n σ = i=2 a(C 1 , C i )/(n − 1). Then the first cluster consists of all the curves C j , such that a(C1 , C j ) < σ/3.

319

The remining curves then are grouped into several clusters by repeating the process. The details are available in Chen et al. (2001). 4.

The Level Set Formulation

Level set methods are extensively used in active contour evolution because they allow the curve to develop cusps, corners, and topological changes. We now give a variational level set approach for our active contour along the lines proposed in Zhao (1996) and Chan (2001). Let the contour C be the zero level set of a Lipschitz function u such that {x | u(x) > 0} is the set inside C. Let H (z) be the Heaviside function, that is H (z) = 1 if z ≥ 0, and H (z) = 0 if z < 0, and δ = H (z) (in the sense of distribution) be the Dirac measure. Then, the length of the zero level set of u in the conformal metric ds = g|C ( p)| dp can be computed by g|∇ H (u)| = δ(u)g|∇u|, where is the image domain. The similarity of the shapes between ∗ the zero 2level set of u and C can be evaluated by δ(u)d (µRx + T ))dx, where the distance function d is the same as that in Eq. (5). Therefore, the level set formulation of our variational approach is λ min δ(u) g(|∇ I |) + d 2 (µRx + T ) |∇u|. u,µ,R,T 2 (10) The evolution equations related to the Euler-Lagrange equations for Eq. (10) are ∂u λ 2 ∇u = δ(u) div g + d , ∂t 2 |∇u| x ∈ , t > 0, (11) ∂u = 0, x ∈ ∂, t > 0, u(x, 0) = u 0 (x), ∂n x ∈ , (12) ∂µ = −λ δ(u)d∇d · (Rx)|∇u| dx, ∂t t > 0, µ(0) = µ0 , (13) ∂θ dR = −λ δ(u)µd∇d · x |∇u| dx, ∂t dθ t > 0, θ(0) = θ0 , (14) ∂T = −λ δ(u)d∇d|∇u| dx, ∂t t > 0, T (0) = T0 , (15) where R is the rotation matrix in terms of the angle θ , and the function d is evaluated at µRx + T .

320

Chen et al.

5.

Numerical Technique

Denote

We now explain how we solve the Eqs. (11)–(15) numerically. Following the approach in Chan (2001), we replace H and δ in (11)–(15) by the slightly regularized versions of them, denoted

x Pi,n j = 0, 1 2

if |z| >

1 + cos

πz ∈

u t (i, j, tn ) =

n u i,n+1 j − u i, j

t

if |z| ≤

,

where t at (i, j, tn ) is approximated using the central difference scheme: ∇u (i, j, tn ) div |∇u|

2 2 [(u y )i,n j ] (u xx )i,n j − 2(u x )nij (u y )i,n j (u x y )i,n j + [(u x )i,n j ] (u yy )i,n j , n 2 n 2 3/2 ([(u x )i, j ] + [(u y )i, j ] )

where

(u y )i,n j = (u xx )i,n j = (u yy )i,n j = (u xy )i,n j =

n n u i+1, j − u i−1, j

2h u i,n j+1 − u i,n j−1 n u i+1, j

,

, 2h n n − 2u i, j + u i−1, j

h2 n u i, j+1 − 2u i,n j + u i,n j−1

,

, h2 n n n n u i+1, j+1 − u i−1, j+1 − u i+1, j−1 + u i−1, j−1 4h 2

2h

, y Pi,n j =

Pi,n j+1 − Pi,n j−1 2h

.

|∇u|inj

2 + n 2 n = max − + min x u i, j , 0 x u i, j , 0 2 2 12 n n + max − + min + , y u i, j , 0 y u i, j , 0 (16) where n − x u i, j =

∇u is the time step. The diffusion term div ( |∇u| )

(u x )i,n j =

n n Pi+1, j − Pi−1, j

The term |∇u| at (xi , y j , tn ) is computed by

To discretize the Eqs. (11)–(15), we use a finite difference scheme. Let u i j denote the value of u at the pixel xi = i h, y j = j h, where h is the pixel size. Denote u(xi , y j , tn ) by u inj . The time derivative u t at (i, j, tn ) is approximated by the forward difference scheme:

=

We use the forward or backward finite difference schemes adaptively to approximate ∇ P · ∇u. That is

where

and, δ (z) = H (z) =

λ 2 n n d (µ R (xi , y j )T + T n ). 2

(∇ P · ∇u)i,n j − n n n = max x Pi,n j , 0 + x u i, j + min x Pi, j , 0 x u i, j − n n n + max y Pi,n j , 0 + y u i, j + min y Pi, j , 0 y u i, j ,

   1, if x > H (z) = 0, if z ≤ −   1 1 + z + π1 sin πz if |z| ≤ , 2

Pi,n j = g(|∇ I |)(xi , y j ) +

.

n − y u i, j =

n u i,n j − u i−1, j

u i,n j

h − u i,n j−1 h

n , + x u i, j = n , + y u i, j =

n n u i+1, j − u i, j

h u i, j+1 − u i,n j h

Then, Eq. (11) is descretized by n n n n n u i,n+1 j = u i, j + tL u i, j , µ , R , T

, .

(17)

with u i,0 j = u 0 (xi , y j ), which is the distance function for the initial contour C0 . In Eq. (17) L u i,n j , µn , R n , T n n (∇ P · ∇u)i,n j ∇u n = δ u i, j + Pij div (i, j, tn ) . |∇u|nij |∇u| To speed up the calculation for u a narrow band idea introduced in Chopp (1991) (also see Malladi, 1995; Adalst, 1995) was used in our computation. We modified their method at each iteration by only computing those u such that δ (u) > 0. Moreover, to ensure the level set function stays well behaved, i.e. to have the signed distance function near the front, the technique for up dating u at each iteration, developed in used in Chan (2001), Sussman (1994), and Zhao (1996) is also used in our computation. This procedure uses a new function v(x), which is the steady state solution to the


with µ0 = µ0 , θ 0 = θ0 , and T 0 = T0 , where

equation ∂v = sign(v)(1 − |∇v|), v(·, 0) = u(·, t), ∂s as u(., t) for the next iteration t + t. In this equation, the sign function controls the flow of information to “straighten out” the level set on either side of the zero level set, and produce a function v with |∇v| = 1 corresponding to the signed distance function. The equation for v is descretized as n n n vi,n+1 j = vi, j + t sign vi, j |∇v|i, j − 1 , where |∇v|i,n j is computed by the same scheme as that in Eq. (16). The Eqs. (13)–(15) are discretized as x µn+1 = µn − , F u i,n j , µn , R n , T n · R n i yj ij θ n+1 = θ n − F u i,n j , µn , R n , T n µn ij

T n+1

321

dRn xi , · dθ y j = Tn − F u i,n j , µn , R n , T n , ij

F u i,n j , µn , R n , T n = h 2 λt|∇u|inj δ u i,n j dinj ∇d(i, j, tn ).

6.

Experimental Results and Applications

We have tested our model on synthetic images, functional MR brain images and ultrasound images.

6.1.

Synthetic Image

The aim of the first experiment was to verify that the active contour with the prior shape can fill in the “gaps” in a boundary. We used the parameterized form of contour evolution (6)–(9) as well the level set form (Eqs. (11)–(15)) for this experiment. The results were similar. Figure 1(a) shows a curve that we used as the prior shape for this experiment. We generates images using

Figure 1. (a) The prior shape C ∗ , (b) The image with the initialized active contour (the ellipse), (c) The final stationary contour, (d) The contour C ∗ (solid curve), and the contour µRC + T (dotted curve).

322

Chen et al.

this curve in the following way: We took the region inside the curve to be a white object. This object was partially occluded and placed on a black background in an image with an arbitrary translation, rotation, and scale, as shown in Fig. 1(b) (in this figure, only the circular arc part of the original curve is available). The task was to see whether the active contour with the prior shape information can utilize this partial boundary while filling in the rest. The active contour was initialized as the ellipse as shown in Fig. 1(b). Evolving the active contour according to the level set formulation of Eqs. (6)–(9) with the parameters β = 1, σ = 0.5 (in g(x)), λ = 1, dt = 0.02, µ0 = 1.4, θ0 = 0, T0 = (0, 0), we get the stationary contour C in Fig. 1(c). The transformation parameters are µ = 1.25, θ = 0.50, and T = (−2.14, −2.13)(pixels). In Fig. 1(d) the prior shape C ∗ , and the contour µRC( p) + T are shown as the white and the dotted curves. From Fig. 1(c) and 1(d) we can see that the contour C captures the high gradient in the image I and the prior shape C ∗ , even though complete gradient information is not available.

6.2.

Cardiac Ultrasound Images

The aim of the second experiment was to segment the epicardium (the outer boundary of the myocardium surrounding the left ventricle) in a four-chamber image of the heart (see Fig. 2(b) for a typical image). The epicardium is not completely imaged in the image, and our task is to find and complete the boundary using a prior shape. To create the prior shape, epicardial boundaries were outlined in 112 patients by an expert echocardiographer. The boundaries were grouped as explained in Section 3 so that translation, orientation and scale were factored out. We partitioned the set into such clusters using a statistical technique as mentioned in Section 3. Fig. 2(a) shows one of the clusters and its average contour. Our goal was to segment the epicardium in Fig. 2(b), using the average contour as the prior shape. The contour shown in Fig. 2(b) was used as an initial contour. This contour was evolved according to the parameterized evolution of Eqs. (6)–(9), and it finally stopped at the location of the solid contour

Figure 2. (a) A cluster of 79 curves and their average shape C ∗ . (b) The initial contour in the ultrasound image. (c) The active contour at its stationary point (solid curve) and the expert’s epicardium (dotted curve), (d) The contour µRC + T , where C, µ, R, and T are the transform parameters.


in 2(c). To validate this result we had the expert manually segment the epicardium. The dotted contour in 2(c) is the expert’s epicardium. We can see that our segmentation is close to the expert’s. The parameters we used in this experiment are λ = 2, β = 1, σ = 0.5, dt = 0.05, µ0 = 1, θ0 = 0, T0 = (0, 0). The solutions we obtained are the contour C in Fig. 2(c), and the transformation parameters µ = 0.66 θ = 0.10 and T = (−0.88, −0.10) (pixels). In Fig. 2(d) we present the contour µRC + T for inspection. It is similar to the average shape C ∗ in Fig. 2(a).

6.3.

Functional MRI

The aim of the last experiment was to show that our active contour not only provides an appropriate segmentation, but also provides accurate estimates of the transformation parameters. These estimates can be used to align functional MR images. Functional MR images are time series images, that permit visualization of local changes in cortical blood volume, flow, and oxygenation. These changes are correlated with mental activity. Typically a subject is exposed to baseline and activation conditions while his/her brain is being scanned in the MR machine. A spin sequence is used that enables the visualization of blood volume and oxygenation. The images from baseline are compared with images from activation to obtain those pixels which are statistically different. These pixels are assumed to indicate locations of brain activity. Due to the long scanning time, some head movement from the subject is unavoidable. Head movement introduces a mis-registration between the images which in turn causes spurious signal changes. Typical T2∗ weighted brain images show that the spurious signal changes due to motion are the same order as the signal changes from the task. Therefore, small motions impact the fMRI time series adversely. Proper image registration (realignment of the images) is needed to minimize the effect of motion on the fMRI signal. In this study, we examined langauge production in which subjects spoke aloud during the MR acquisition. This added stimulus-correlated motion artifacts in addition to random global head motion artifacts. The motion artifacts produce false positives and mask real activation, thus confounding interpretation and reducing the usefulness of the data. Accurate and reliable motion correction and image registration algorithms can minimize problems due to these artifacts.

323

It has been pointed out in Biswal (1997) that intensity based image registration methods do not provide good results for aligning time series images. This is because of the complex nature of intensity variations, such as different distribution of signal intensity during a task activation and a rest scan, and nonuniform intensity variation in an imaging slice due to motion and temporal variation of the magnetic field homogeneity. Feature-based methods are not adversely affected by such changes. Feature based methods first find the important features in both images, and then align the images by aligning the features. The shape prior based active contour proposed in this paper offers reliable feature extraction (because prior information is used) as well as reliable estimates of translation, rotation and scale which can be used to align the images. To apply our method to fMRI, we used a segmented corpus callosum in a high resolution MR image as the shape prior C ∗ , and used our active contour to find the boundary of the poorly determined corpus callosum in each of the time series images. The spatial transform that mapped the segmented contour to the prior C ∗ was used to register the time series images. Figure 3(a) shows the high resolution image and the outlined corpus callosum that was used as the prior shape. Figure 3(b) and (c) two consecutive images in the fMRI time series. They have a much lower resolution than the image in Fig. 3(a). The active contour was initialized in both images as an ellipse as shown in Fig. 3(b) and (c). It was evolved with parameters λ = 1, β = 1, σ = 0.5, dt = 0.02, µ0 = 2, θ0 = 0, T0 = (0, 0). The active contour became stationary at curves C1 and C2 which are shown in Fig. 3(b) and (c) respectively. We also obtained the transformation parameters µ1 = 1.37, θ1 = 0.08, T1 = (38.58, 43.18) (pixels) that mapped C1 to the prior curve C ∗ , and µ2 = 1.37, θ2 = 0.08, T2 = (38.14, 43.18) (pixels) that mapped C2 to C ∗ . To validate that the segmentation was correct, we plotted µ1 R1 C1 + T1 , and µ2 R2 C2 + T2 back on the high resolution image from which the prior was constructed. This is shown in Fig. 3(d) and (e). By comparing 3(d) and (e) with (a) we can see that the contours Ci (i = 1, 2) do segment the corpus callosum and remain faithful to the prior shape. Finally, the trans−1 −1 −1 formation C2 = µ1 µ−1 2 R1 R2 C 1 + µ2 R2 (T1 − T2 ) provides the alignment of these two time series images. Figure 4 shows the functional activation maps for an overt word generation task obtained by (a) using AFNI’s intensity-based registration method and (b) using the feature-based method. In this experiment the

324

Chen et al.

Figure 3. (a) The corpus callosum from a high resolution image. This is C ∗ , (b) fMRI images with the initialized active contour (the ellipse) and the final active contour, (c) The final contours overlayed on the high resolution images.

Figure 4. Functional activation maps for an overt word generation task obtained by (a) AFNI’s intensity-based registration method, (b) the proposed feature-based method.


task was single word generation to a series of randomly spaced category cues. Four functional runs were acquired with 150 images in each run, and 1.5 sec. per image.The dark dots in the figures represent high signal. Figure 4(a) and (b) show that the feature-based registration method produces a significant decrease in the motion-related noise compared to the AFNI technique. Regional activation in medial cortex at and near the crosshair results from local blood flow and oxygenation changes which affect the MR signal intensity subsequent to neuronal activity. These correspond to areas also observed in covert language generation studies. Compared to AFNI, the feature-based registration method reduces motion artifacts, which are predominant in areas of high signal contrast. 7.

Conclusion

We proposed the addition of a shape prior to an active contour in a variational framework. The key idea was to introduce an energy term which measured the closeness of the active contour to a prior contour in a way that is independent of the location, orientation, and scale of the contour. We also proposed a level set formulation and a numerical algorithm for the active contour. The existence of the minimum of the energy function is also established. In experiments with real-world images, the active contour could reliably segment images in which the complete boundary was either missing (the ultrasound images) or was low resolution and low contrast (fMRI). Besides the segmentation, the algorithm also provides estimates of translation, rotation, scale that map the active contour to the prior shape. These estimates are useful in aligning images. The theoretical analysis and experimental results in this paper were carried out for 2-d images. The extension to 3-d images can be done (the shapes are aligned using 3-d translation, rotation, and scaling) and will be the subject of future reports. Appendix: Existence of the Minimum In this appendix we discuss the existence of the solution to the minimization in Eq. (10). The same argument can also be applied to Eq. (5). As suggested in Chan (2001) we may rewrite (10) in the form λ 2 min g(|∇ I |(x) + d (µRx + T ) |Dχ E |, χ E ,µ,θ,T 2 (A.1)

325

where is a bounded open set in R n with Lipschitz boundary (say = (0, 256)2 ), R is the rotation matrix in terms of an angle θ, and χ E is the characteristic function of E = {x ∈ |u(x) ≥ 0}. This minimization is over all the characteristic functions of E in BV () (note that the set E varies as u evolves), µ ∈ (0, 256], θ ∈ (−π, π ], and T ∈ [−256, 256] × [−256, 256]. The minimization problem (A.1) involves a weighted total variation norm for the functions with finite perimeters. It also minimizes more than one argument. A special case, where the weight function is one, and χ E is the only minimizing argument, has been studied in Chan (2001). To study the existence for the problem (A.1), it is necessary to introduce the concept of weighted total variation norms for functions of bounded variation. Let us begin with recalling the definitions of functions with bounded variation (Giusti, 1985). Definition A.1. Let ⊂ R n be an open set and let f ∈ L 1 (). Define

|∇ f | =: sup

φ∈

f (x)divφ(x)d x ,

where =: φ ∈ C01 (, R n ) | |φ(x)| ≤ 1, on . Definition A.2. A function f ∈ L 1 () is said to have bounded variation in , if |∇ f | < ∞. We define BV() as the space of all functions in L 1 () with bounded variation. For a function f ∈ BV(), we define f BV () = f L 1 () + |∇ f |.

Definition A.3. A measurable subset E of R n has finite perimeter in , if the characteristic function χ E ∈ BV(). It has been known that if f ∈ BV(), then for any t ∈ R, the level set E t = {x ∈ | f (x) > t} has finite perimeter, i.e. χ Et ∈ BV(). Next we define the weighted total variation norm with the weight function α(x). Definition A.4. Let ⊂ R n be an open set. Let also f ∈ L 1 () and α(x) be positive valued continuous and bounded functions on . The weighted total variation

326

Chen et al.

norm of f with the weight function α(x), denoted by α(x)|∇ f |, is defined by

α(x)|∇ f | =: sup

φ∈α

f (x)divφ(x)dx ,

Moreover, from the boundedness of µm , Rm , and Tm , there exist a constant µ ∈ (0, 256], a constant θ ∈ (−π, π ], and a constant vector T ∈ [0, 256] × [0, 256], such that µm → µ, θm → θ, Tm → T.

where α =: φ ∈ C01 (, R n ) | |φ(x)| ≤ α(x),

on . (A.2)

Lemma A.5 (Giusti, 1985). Let ⊂ R n be an open set with a Lipschitz boundary. Suppose that f n is a bounded sequence in BV (). Then, there is a subsequence converging in L p () to a function f ∈ BV () for any 1 ≤ p < n/(n − 1). Main Theorem. Let = (0, 256)2 and let C ∗ be a differentiable contour, and I ∈ L ∞ (). The minimization problem (A.1) has a solution χ E ∈ BV (), µ ∈ (0, 256], θ ∈ (−π, π], and T ∈ [−256, 256] × [−256, 256].

Noticing that µ, R, and T are involved in the continuous function P through a distance function, we have Pm converges to P uniformly on . Therefore, for any 0 < < p0 there exists a constant M = M() > 0 such that if m > M P − ≤ Pm ≤ P + . Then by the definition of α (ref. (A.2)) we have P− ⊂ P . Therefore, for any fixed φ ∈ P− , if m > M, χ Em divφ dx ≤ sup χ Em divφ dx φ∈ Pm

=

Proof: Let E m , µm , Rm , and Tm be minimizing sequences of (A.1). Denote

and

χ E divφ d x = lim

m→∞

χ E ∈BV,µ,R,T

P|Dχ E |.

(A.3)

Since g(|∇ I |)(x) ≥ 1/(1 + CI L ∞ () ) ≤, for a suitable constant C = C(β, σ ) > 0, the function P is bounded below by a constants p0 > 0. Therefore, χ Em is a bounded sequence in BV (). By Lemma A.5, there exist subsequences of χ Em , µm , θm , and Tm such that, without changing notation, χ Em converges to a function f in L 1 () and a.e. on . Since χ Em is either 0 or 1, f is either 0 or 1 a.e.. We may view f as the characteristic function χ E of a set E. Therefore, we have χ Em → χ E ,

in L 1 ().

χ Em divφ

(A.5)

The combination of (A.4) and (A.5) shows for any φ ∈ P− ,

Pm (x) = P(x, µm , Rm , Tm ).

inf

(A.4)

Furthermore, since χ Em converges in L 1 () to χ E , for any φ ∈ P− ,

P(x) = P(x, µ, R, T ) λ = g(|∇ I |)(x) + d 2 (µRx + T ). 2

Then, as m → ∞ Pm |Dχ Em | →

Pm ∇χ Em .

χ E divφ dx ≤ lim

Pm ∇χ Em .

m→∞

Therefore, sup

φ∈ P−

χ E divφ dx ≤ lim

m→∞

Pm ∇χ Em .

By letting → 0, and then using the Definition A.4 and (A.1), we get

P|Dχ E | = sup

φ∈ P

≤ lim

m→∞

→

χ E divφ dx Pm |∇χ Em |

inf

χ E ∈BV,µ,R,T

P|Dχ E |.

Using Prior Shapes in Geometric Active Contours This shows that solution χ E ∈ BV (), and χ E µ, θ, and T are the solutions to the minimization problem (A.1). Acknowledgments Yunmei Chen is partially supported by the NSF grant DMS-9972662 and NIH P50-DC03888. Hemant D. Tagare is supported by the grant R01-LM06911 from the National Library of Medicine. Richard W. Briggs and Kaundinya S. Gopinath are partially supported by the NIH grant P50-DC03888 and the VA RR&D Brain Rehabilitation Center. References Adalsteinsson, D. and Sethian, J.A. 1995. A fast level set method for propagating interfaces. J. Comput. Phys., 118(2):269–277. Biswal, B.B. and Hyde, J.S. 1997. Contour-based registration technique to differentiate between task-activation and head motion induced signal variations in fMRI. Magn. Reson. Med., 38:470– 476. Caselles, V., Catté, F., Coll, T., and Dibos, F. 1993. A geometric model for active contours in image processing. Numerische Mathematik, 66:1–31. Caselles, V., Kimmel, R., and Sapiro, G. 1997. Geodesic active contours. International Journal of Computer Vision, 22(1):61–79. Chan, T.F. and Vese, L.A. 2001. Active contours without edges. IEEE Trans. Image Processing, 10(2):266–277. Chan, T.F. and Vese, L.A. 2001. A level set algorithm for minimizing the Mumford-Shah functional in image processing. In Proceedings 1st IEEE Workshop on Variational and Level Set Methods in Computer Vision, Vancouver, B.C., Canada, pp. 161–168. Chen, Y., Thiruvenkadam, S.,Tagare, H.D., Huang, F., Wilson, D., and Geiser, E.A. 2001a. On the Incorporation of shape priors into geometric active contours. In Proceedings 1st IEEE Workshop on Variational and Level Set Methods in Computer Vision, Vancouver, B.C., Canada, pp. 145–152. Chen, Y., Wilson D., and Huang, F. 2001b. A new procrustes methods for generating geometric models. In Proceedings of World Multiconference on Systems, Cybernetics and Informatics, Orlando, pp. 227–232. Cohen, L.D. 1991. Pn active contour models and ballons. Image Understanding, 53:211–218. Cootes, T., Taylor, C., Cooper, D., and Graham, J. 1995. Active shape model—their training and application. Computer Vision and Image Understanding, 61:38–59. Cootes, T. and Taylor, C. 1999. Mixture model for representing shape variation, Image and Vision Computing, 17(8):567–574. Cootes, T., Beeston, C., Edwards, G., and Taylor, C. 1999. Unified framework for atlas matching using active appearance models. In Int’l Conf. Inf. Proc. in Med. Imaging, Springer-Verlag, pp. 322– 333. Chopp, D. 1991. Computing minimal surface via level set curvature flows. Lawrence Berkeley Laboratory Technical Report, University of Berkeley, CA, 1991.

327

Cremers, D., Schn orr, C., and Weickert, J. 2001. Diffusion-snakes: Combining statistical shape knowledge and image information in a variational framework. In Proceedings of 1st IEEE Workshop on Variational and Level Set Methods in Computer Vision, Vancouver, B.C., Canada, pp. 137–144. Dryden, I.L. and Mardia, K.V. 1988. Statistical Shape Analysis. John Wiley & Sons: New york. Giusti, E. 1985. Minimal Surfaces and Functions of Bounded Variation. Birkhauser. Kass, M., Witkin, A., and Terzopoulos, D. 1988. Snakes: Active contour models. Inter. Journal of Computer Vision, 1:321–331. Kichenassamy, S., Kumar, A., Olver, P., Tannenbaum, A., and Yezzi, A.J. 1995. Gradient flows and geometric active contour models. In Proc. ICCV’95, IEEE Computer Soc. Press, Cambridge, Mass., pp. 810–815. Kichenassamy, S., Kumar, A., Olver, P., Tannenbaum, A., and Yezzi, A.J. 1996. Conformal curvature flows: From phase transitions to active vision. Arch. Rat. Mech. and Anal., 134:275–301. Leventon, M.E., Grimson, W.E.L., and Faugeras, O. 2000. Statistical shape influence in geodesic active contours. Proc. IEEE Conf. Comp. Vision and Patt. Recog., pp. 316–323. Malladi, R., Sethian, J., and Vemuri, B. 1995. Shape modeling with front propagation: A level set approach. IEEE Trans. Pattern Anal. Machine Intell., 17:158–175. McInerney, T. and Terzopoulous, D. 1996. Deformable models in medical image analysis: A survey. Medical Image Analysis, 1(2):91–108. Mumford, D. and Shah, J. 1989. Optimal approximation by piecewise smooth functions and associated variational problems. Comm. Pure Appl. Math., 42:557–685. Osher, S. and Sethian, J.A. 1988. Fronts propagating with curvaturedependent speed: Algorithms based on Hamilton-Jacobi formulation. Journal of Computational Physics, 79:12–49. Paragios, N., Rousson, M., and Ramesh, V. 2002. Maching distance functions: A shape-to-area variational approach for global-to-local registration. In the Proceedings of Computer Vision-ECCV2002, the 7th European Conference on Computer Vision, Copenhagen, Denmark, pp. 775–789. Paragios, N. 2000. Geodesic active regions and level set methods: Contributions and applications in artificial vision. Ph.D. thesis, School of Computer Engineering, University of Nice/Sophia Antipolis. Paragios, N. 2001. A variational approach for the segmentation of the left ventricle in MR cardiac images. In Proceedings 1st IEEE Workshop on Variational and Level Set methods in Computer Vision, 13 July 2001, Vancouver, B.C., Canada, pp. 153– 160. Paragios, N. and Deriche, R. 1999. Geodesic active regions for supervised texture segmentation. In IEEE ICCV Cofu, Greece, pp. 926– 932. Rousson, M. and Paragios, N. 2002. Shape priors for level set representations. In the Proceedings of Computer Vision-ECCV2002, the 7th European Conference on Comuter Vision, Copenhagen, Demark, pp. 78–92. Samson, C., Blance-Féraud, L., Aubert, G., and Zérubia, J. 2000. A level set model for image classification. Int. J. Comp. Vis., 40(3):187–197. Satto, S. and Yezzi, A.J. 2002. Deformation: Deforming motion, shape average and the joint registration and segmentation of images. In the Proceedings of Computer Vision-ECCV2002, the

328

Chen et al.

7th European Conference on Computer Vision, Copenhagen, Denmark, pp. 32–47. Small, C.G. 1996. The Statistical Theory of Shape. Springer Verlag: Berlin. Staib, L. and Duncan, J. 1992. Boundary finding with parametrically deformable contour methods. IEEE Trans. Patt. Analysis and Mach. Intell., 14(11):1061–1075. Sussman, M., Smereka, P., and Osher, S. 1994. A level set approach for computing solutions to incompressible two phase flow. J. Comput. Phys., 119:146–159. Szekely, G., Kelemen, A., Brechbuhler, C., and Gerig, G. 1996. It Segmentation of 2D and 3D objects from MRI volume data using constrained elastic deformations of flexible Fourier contour and surface models. Medical Image Analysis, 1(1):199–234. Tagare, H.D. 1997. Deformable 2-D template matching using orthogonal curves. I.E.E.E. Trans. on Medical Imaging, 16(1):108–117. Tsai, A., Yezzi, Jr., A., and Willsky, A.S. 2001. Curve evolution implementation of the Mumford-Shah functional for image segmen-

tation, denoising, interpolation, and magnification. IEEE Transactions on Image Processing, 10(8):1169–1186. Wang, Y. and Staih, L. 1998. Boundary finding with correspondence using statistical shape models. In Proc. IEEE Conf. Comp. Vision and Patt. Recog., pp. 338–345. Yezzi, A., Kichenassamy, S., Kumar, A., Olver, P.J., and Tannenbaum, A. 1997. A geometric snake model for segmentation of medical imagery. IEEE Trans. Medical Imaging, 16:199– 209. Yuille, A., Hallinan, P.W., and Cohen, D.S. 1992. Feature extration from faces using deformable templates. Int. J. Computer Vision, 8:99–111. Zhao, H.K., Chan, T., Merriman, B., and Osher, S. 1996. A variational level set approach to multiphase motion. J. Commput. Phys., 127:179–195. Zhu, S.C. and Yuille, A. 1996. Region Competition: Unifying snakes, region growing, and Bayes/MDL for multiband image segmentation. IEEE PAMI, 18:884–900.