Fast Variational Segmentation using Partial Extremal Initialization

Fast Variational Segmentation using Partial Extremal Initialization Jan Erik Solem, Niels Chr. Overgaard, Markus Persson and Anders Heyden Applied Mathematics Group, School of Technology and Society, Malm¨o University, Sweden [email protected]

Abstract In this paper we consider region-based variational segmentation of two- and three-dimensional images by the minimization of functionals whose fidelity term is the quotient of two integrals. Users often refrain from quotient functionals, even when they seem to be the most natural choice, probably because the corresponding gradient descent PDEs are nonlocal and hence require the computation of global properties. Here it is shown how this problem may be overcome by employing the structure of the Euler-Lagrange equation of the fidelity term to construct a good initialization for the gradient descent PDE, which will then converge rapidly to the desired (local) minimum. The initializer is found by making a one-dimensional search among the level sets of a function related to the fidelity term, picking the level set which minimizes the segmentation functional. This partial extremal initialization is tested on a medical segmentation problem with velocity- and intensity data from MR images. In this particular application, the partial extremal initialization speeds up the segmentation by two orders of magnitude compared to straight forward gradient descent.

1. Introduction Segmentation is the process of partitioning given data (e.g. images) into regions with different properties. Variational segmentation is a technique where the segmentation is defined as a (local) minimizer of an energy functional tailored to the problem at hand. Such segmentation functionals can contain both region-based and boundary-based terms depending on the application. This paper deals with variational segmentation using both region and boundary information. This is a frequently occurring example, e.g. when parts of the data contain strong edges or when region based segmentation is combined with regularization [2]. The literature on variational segmentation is vast. Historically, boundary based formulations dominated in earlier work whereas today region-based formulations are very common. Traditional active contour methods, such as

snakes [6], geodesic active contours [1], and similar models [4, 15], all achieve segmentation by evolving a simple, closed parametrized curve (the active contour) under the influence of a driving force, provided by an edge map constructed from the data. The evolution is continued until an equilibrium state, defining the segmentation, is reached. The formulation of segmentation problems as regionbased variational problems has proved to be useful in a wide variety of applications where reliable edge-maps are either difficult or impossible to construct. Region-based segmentation models are often inspired by the classical models of Mumford-Shah [8] and Chan-Vese [2]. The segmentation is determined by the partitioning (into subsets or subregions) that minimizes the deviation of the data from a given a priori model for each subregion. In [2], for instance, the a priori image model stipulates that the gray-levels are (nearly) uniform in each subregion. The model [11], which was developed independently from [2], is a refined version of the latter, in which image statistics is incorporated. In this paper we will use the level set method, cf. [10], to represent our boundaries. Level sets are a popular choice since they are suited to handle changes in topology, a property which is crucial in many medical applications. The paper contains the following contributions. We propose to use region-based quotient functionals for certain types of segmentation problems. Quotient functionals may sometimes be superior to traditional functionals constructed to solve the same problem. For instance, the quotient functional may contain fewer weight parameters to be preset by the user. We introduce a technique, partial extremal search, to find good initializations for gradient descent PDEs corresponding to quotient functionals. Experiments have been carried out on real MR data, showing that the performance of the proposed method, in terms of computation time, can be increased several orders of magnitudes compared to the traditional gradient descent method applied to the standard region-based segmentation functional for this problem.

Let f (x) : R3 → R be a given potential function, and define a functional as the volume integral

2. Background To introduce notation and as a courtesy to the reader we briefly describe the necessary background on the level set method and gradient descent flows. The level set method for evolving implicit surfaces was introduced independently by [5] and [9]. The time dependent surface Γ(t) is represented implicitly as the zero level set of a function φ(x, t) : R3 × R → R as Γ(t) = {x ; φ(x, t) = 0} .

(1)

For curves the representation is the same with φ(x, t) : R2 × R → R instead. The sets Ω = {x ; φ(x, t) < 0} and Ωc = {x ; φ(x, t) > 0} are called the interior and the exterior of Γ, respectively. Using this definition the outward unit normal n and the mean curvature κ are given as n=

∇φ |∇φ|

and κ = ∇ ·

∇φ . |∇φ|

(2)

One important, frequently used example is the signed distance function, where the additional requirement |∇φ| = 1 is imposed. To evolve the surface according to some derived velocity, a PDE of the form ∂φ + v|∇φ| = 0 , ∂t

(3)

is solved, where v is the velocity normal to the surface. For a more thorough treatment of the level set method and implicit surface representations, cf. [10]. In the variational level set method, functionals are used to derive gradient descent motion PDEs of the form (3). This is done through the use of the differential (Gˆateaux derivative). The Gˆateaux derivative is related to the gradient ∇E of a functional E as
∇E v dσ , (4) dE(Γ)v =


f (x) dx .

E(Γ) =

(6)

Ω

This is a general form of a region-based functional where f can be e.g. the deviation from an image model [2, 11]. There are some issues when considering region-based problems. We will use an illustrative example to highlight some of the difficulties. Let us consider a simple example of minimizing
E(Γ) = E• (Γ) + E◦ (Γ) =


f (x) dx +

Ω

dσ ,

(7)

Γ

for some form of measurement f (x) and regularization1 . The first integral is sometimes denoted the fidelity term, or data term. The gradient descent flow of the functional (7) is ∂φ = (f + κ)|∇φ| . ∂t

(8)

One of the issues with this type of problems is that there is a need for a “balancing force”. The reason for this is that E• depends on the size of the region. Depending on the sign of f (x) this will lead to undesirable “blow up” or “shrinking” effects. To counter this effect, additional balancing terms are usually added to stop the evolution at the desired solution. Common examples are region terms for the complement Ωc [2, 11] and edge data [13]. Another option is to divide by the size of the region and use the average potential instead. The functional representing the average potential is
f (x) dx Eavg (Γ) =

Ω


dx

Γ

1 = |Ω|


f (x) dx ,

(9)

Ω

Ω

where v is the normal velocity of a perturbation of Γ. For details cf. e.g. [14, 3, 16]. The gradient descent flow is then obtained by solving the following motion equation ∂φ = ∇E |∇φ| , ∂t

(5)

from an initial surface.

where we use |Ω| to denote the volume of Ω. For quotient functionals we have the following general result: Proposition 1: If Γ is an extremal of the functional
f (x) dx E(Γ) =
Ω

,

(10)

g(x) dx

3. Region-based variational problems For many applications one is interested in finding regions in measured data, such as images and MRI data. One can easily formulate region-based functionals where Γ is the (unknown) boundary of some region to be found.

Ω

then Γ is a level set of the function f (x)/g(x). 1 In practice these terms are weighted depending on the amount of regularization needed. For simplicity, we will ignore this and consider the weights to be included in f (x).

Proof: By the quotient rule, the Gˆateaux derivative of E is     g dx Γ f v dσ − Ω f dx Γ g v dσ Ω dE(Γ)v =  2 g dx Ω   f v dσ − E(Γ) Γ g v dσ Γ  = g dx Ω  (f − E(Γ)g)v dσ Γ  = . (11) g dx Ω The derivative dE(Γ) must vanish at an extremal of E(Γ), so Γ (f − E(Γ)g)v dσ = 0, for all normal velocities v, hence the extremal must satisfy the Euler-Lagrange equation f − E(Γ)g = 0. Since E(Γ) is scalar, it follows from this relation that f (x)/g(x) = E(Γ), i.e. Γ is a level set.
From this it follows that the extremals of (9) are level sets of the function f (x). The gradient descent flow corresponding to (9) is is then given by the gradient ∇Eavg =

f − Eavg (Γ) . |Ω|

(12)

By modifying the region part from (6) to (9) one obtains a slightly more complicated gradient descent flow,   f − Eavg (Γ) ∂φ = + κ |∇φ| , (13) ∂t |Ω| which involves the computation of |Ω| and E(Γ). This makes the procedure global, as opposed to the gradient descent of (6) which only depend on the value of f (x) locally. One of the benefits of the modification is an evolution which does not suffer from the “blow up” or “shrinking” mentioned above. Another gain is the interesting form of the extremals which will be used in the next sections.

4. Initialization by partial extremal search In this section we will propose a procedure for finding good initial values or solutions for the gradient descent PDEs associated with variational segmentation models involving region-based fidelity terms in quotient form. Assume that the model requires the minimization of a functional of the form
f (x) dx
Ω
+ h(x) dσ , E(Γ) = E• (Γ) + E◦ (Γ) = Γ g(x) dx Ω

where image data enters the model through the functions f and g in the fidelity term E• and (possibly) through h in the regularity term E◦ . From Proposition 1 we know that an extremal of E• is necessarily a level set of the function ψ(x) = f (x)/g(x).

This suggests that we begin our search for the minimum of E(Γ) among the surfaces which are level sets of ψ(x). That is, if we define L to be the set of surfaces which has the form Γ = {x : ψ(x) = c} for some real number c, then we first find a surface Γ∗ which solves the following problem: Γ∗ ∈ L

E(Γ∗ ) = min E(Γ) Γ∈L

(14)

This partial extremal search is essentially a onedimensional search with respect to the parameter c, a problem which can be solved fast. Our idea is to use the level surface Γ∗ as initial data for the gradient descent PDE (5) for E. Note that the proposed method can be interpreted as a thresholding of ψ(x) with an automatic choice of threshold value. Also note that only a certain region-of-interest needs to be searched. If the optimal level set is found in a region, the same level set can be used globally. We illustrate the method below with a practical example.

5. Experiments In this section we will apply the procedure to experiments on phase contrast MRI data. First we review some previous work on this type of data and then we show the effectiveness of the proposed initialization procedure.

5.1. Phase contrast MRI Phase contrast MRI is based on the property that a uniform motion of tissue in a magnetic field gradient produces a change in the MR signal phase, Φ. This change is proportional to the velocity of the tissue, v. The MR signal from a volume element accumulates the phase [12], Φ(r, T ) = γB0 T + γv · G during time T , where B0 is a static magnetic field, γ the gyro-magnetic ratio and G(r, t) the first moment of the magnetic field gradient. If the field gradient is altered between two consecutive recordings, then by subtracting the resulting phases Φ1 − Φ2 = γv · (G1 − G2 ) ,

(15)

the velocity in the (G1 − G2 )-direction is implicitly given. In this way a desired velocity component can be calculated for every volume element simultaneously.

5.2. Gradient descent approach As is common in the variational level set framework one can look for a surface that partitions the data using a gradient flow of a functional. With phase contrast MRI one has the option of combining cues from both velocity and intensity, i.e. the MR magnitude. The surface should enclose as much of the flow as possible and the flow should be continuous inside the enclosing

surface. In [13] this lead to the formulation as minimizing the following functional

χ(v)dx + R(x)dx EV (Γ) = c

 Ω   Ω 

flow outside Γ discontinuities inside Γ
= [R(x) − χ(v)] dx + const. , (16) Ω

where R(x) : R3 → [0, 1] is a vector field discontinuity measure which gives high values where the velocity is discontinuous and χ(v) is a characteristic function, cf. [13]. The surface should also be aligned to the blood vessel border, i.e. the surface outward normal should be parallel (or anti-parallel) to the image gradient at every point and positioned so that the magnitude of the image gradient |∇I| is large. This results in the minimization of following alignment energy functional
− |∇I · n| dσ , (17) EI (Γ) =

 Γ  normal component of gradient at Γ

which has been analyzed in e.g. [7]. Combining the information from both velocity and intensity gives the functional ETot (Γ) = EV + EI which (under some assumptions on the intensity, cf. [13]) has gradient descent flow as    ∂φ ∇φ = −χ + R − sign ∇I · ∆I |∇φ| . (18) ∂t |∇φ|

5.3. Extremal search For the gradient descent procedure above, good startvalues are crucial for the procedure to finish in reasonable time. The execution time for the data described below (in a non-optimized Matlab implementation) was close to four hours. In order to find good initial values we introduce a modification of the functional in Section 5.2. Let us define a new functional as

1 E− (Γ) = [R(x)−χ(v)] dx− |∇I ·n| dσ , (19) |Ω| Ω Γ where the region-based terms are modified to take into account the size of the region Ω. We know from above that the extremals of the region-based part of (19) are level sets of the function R(x) − χ(v). One alternative to using (19) is to look for extremals of
R(x) dx
E/ (Γ) =
Ω − |∇I · n| dσ , (20) Γ χ(v) dx Ω

instead. This formulation has the benefit of avoiding taking the difference between the two region integrals. Another benefit is that it is no longer necessary for the two functions R(x) and χ(v) to be of similar magnitude. There is also no need to have parameters controlling the weight of the different region terms which is a great benefit. We know that the extremals of the region-based part of (20) are level sets of the function R(x)/χ(v). The proposed segmentation procedure is then as follows: 1. Compute ψ(x) as ψ(x) = R(x) − χ(v) or ψ(x) = R(x)/χ(v) depending on if E(Γ) = E− (Γ) or E(Γ) = E/ (Γ) is used. 2. Do a one-dimensional search over the level sets of ψ to find c∗ = argmin E(Γc ) , c

where Γc = {x; ψ(x) = c}. 3. Choose the level set Γ∗ = {x; ψ(x) = c∗ } with lowest value as initial surface for the segmentation. 4. If necessary, run gradient descent from this surface until the first local minimum of E(Γ) is attained. Note that the entire functional is used in the search for c∗ , not just the region-based part. We are using the extremals of the region-based terms as our starting candidates and using the entire functional with both velocity and intensity information to help find the best initial surface. This ensures that the information from the intensity edges is also used to choose the right starting surface. Figure 1 shows the result of the proposed search procedure for a phantom-example. The data is a flow phantom consisting of a rubber hose containing flowing ion-enriched water. The hose is submerged in stationary water of the same kind. Two full 3D volumes (110 × 153 × 59) of MRI intensity data and phase contrast velocity data2 of the flow phantom respectively with a resolution of 1.00 × 1.00 × 2.00mm3 were used. The far-left graphs show the value of the functionals for different level sets where the top row is using E− (Γ) and the bottom row using E/ (Γ). The middle column shows the segmentation surface obtained as the best level set of f (x) for each of the two examples. To the far-right, cross-sections of the velocity magnitude is shown with cross-sections of the initial surface shown in white. The results are comparable in quality with the quotient functional E/ (Γ) performing slightly better if judged visually. The initialization was obtained using a crude Matlab implementation where a (61 × 61 × 60) subset of the data was searched at even intervals through the range of all level sets. For a 20 step search the execution time (on a 3GHz Pentium 2 The actual size of the measurement was (256 × 256 × 60) but only the most relevant part was used to speed up computations.

5

4

x 10

2

0

−2

−4

−6

−8 −1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

1

2

3

4

5

6

7

8

9

10

5

6

x 10

4 2 0 −2 −4 −6 −8 −10 −12 −14 0

Figure 1. Initial surfaces determined as level sets, i.e. extremals of E− (Γ) (top row) and E/ (Γ) (bottom row) respectively. (left) Energy (19) and (20) as function of the level set value for f (x) = R(x) − χ(v) and f (x) = R(x)/χ(v) determined from a subset of the volume. (center) Initial surface as the level set of f (x) corresponding to the minimum of the function on the left. (right) Part of a cross-section with initial surface. The grey level values are velocity magnitude and cross-sections of the initial surface is shown in white.

Figure 2. Final segmentation of the hose data. Three views of the final surface computed from the initial value as shown in Figure 1.

Method Gradient descent Proposed method

Initialization 7s

Gradient descent 13480 s 470 s

Table 1. Comparison of execution times for the hose data. The standard gradient descent method as described in Section 5.2 is compared to the proposed method in Section 5.3

formance of variational region-based segmentation by several orders of magnitude. By reformulating the fidelity term in terms of quotients we took advantage of the structure of the fidelity-extremals and searched them for the best partial extremal and used that as a good initial value. We applied this procedure to segmentation of phase contrast MRI data and showed the superiority over standard gradient descent. Limitations & future work • The extremal search is only partial. Regularization or surface terms are currently not explicitly incorporated in the framework. This has the drawback that the initialization may give undesirable results for highly irregular data sets, e.g. data with high noise.

(a)

(b)

(c)

Figure 3. The aorta data set. (a) One intensity slice of the total volume with the region of interest marked in white. (b) Velocity magnitude for the region of interest. (c) Intensity for the region of interest.

4) was 7 seconds and for a 40 step search 15 seconds. The crude search with 20 levels gave a satisfactory initial segmentation (see Figure 1). The execution time depends only on the size of the search-region and the accuracy of the onedimensional search. The execution times give us reason to believe that the initial segmentation could run well below one second if implemented correctly in e.g. C++/C. See Table 1 for a comparison between the proposed method and gradient descent using standard initialization. Figure 3 shows parts of the aorta data set which consists of a (256 × 256 × 20) measured volume with a region of interest of size (51 × 51 × 20). The initial segmentation with the partial extremal search takes 1.5 seconds and is of very good quality. The resulting segmentation is shown in Figure 4 with a plot of the functional values used to determine the best level set together with images of cross-sections and the full surface. As a comparison, the computation time is 21 seconds with the gradient descent method. The gradient descent method requires manual initialization and tuning of parameters controlling the weighting of the different terms in the functional ETot (Γ). The gradient descent procedure is somewhat sensitive since the wrong choice of weights give undesirable results.

6. Conclusions In this paper we introduced a method, partial extremal initialization, which has the potential of increasing the per-

• Future work will also include the investigation of extending the results to a larger class of functionals, such as where the integrand depends on the segmentation.

References [1] V. Caselles, R. Kimmel, and G. Sapiro. Geodesic active contours. Int. Journal of Computer Vision, 1997. [2] T. Chan and L. Vese. Active contours without edges. IEEE Trans. Image Processing, 10(2):266–277, 2001. [3] G. Charpiat, R. Keriven, J.-P. Pons, and O. Faugeras. Designing spatially coherent minimizing flows for variational problems based on active contours. In International Conference on Computer Vision, ICCV 2005, Beijing, China, pages 1403–1408. IEEE Computer Society, 2005. [4] L. D. Cohen. On active contour models and balloons. CVGIP: Image Understanding, 53(2):211–218, 1991. [5] A. Dervieux and F. Thomasset. A finite element method for the simulation of Rayleigh–Taylor instability. In R. Rautman, editor, Approximation Methods for Navier–Stokes Problems, volume 771 of Lecture Notes in Mathematics, pages 145–158. Springer, Berlin, 1979. [6] M. Kass, A. Witkin, and D. Terzopoulos. Snakes: Active contour models. Int. J. Computer Vision, 1(4):321–331, 1987. [7] R. Kimmel and A. Bruckstein. Regularized Laplacian zero crossings as optimal edge integrators. Int. Journal of Computer Vision, 53(3):225–243, 2003. [8] D. Mumford and J. Shah. Optimal approximation by piecewise smooth functions and associated variational problems. Comm. Pure Appl. Math., 42:577–685, 1989. [9] S. Osher and J. A. Sethian. Fronts propagating with curvature-dependent speed: Algorithms based on HamiltonJacobi formulations. Journal of Computational Physics, 79:12–49, 1988. [10] S. J. Osher and R. P. Fedkiw. Level Set Methods and Dynamic Implicit Surfaces. Springer Verlag, 2002. [11] N. Paragios and R. Deriche. Geodesic active regions: A new paradigm to deal with frame partition problems in computer vision. International Journal of Visual Communication and Image Representation, 2000.

4

16

x 10

14 12 10 8 6 4 2 0 −2

0

1

2

3

4

5

Figure 4. The result of the initial segmentation using the proposed partial extremal search. (top row) Plot of the functional (20) used to find the best initial level set, cross-section showing velocity magnitude with a cross-section of the initial surface shown in white and cross-section showing intensity with a cross-section of the initial surface shown in white. (bottom row) Three views of the full surface.

[12] N. J. Pelc, R. J. Herfkens, A. Shimakawa, and D. Enzmann. Phase contrast cine magnetic resonance imaging. Magnetic Resonance Quarterly, 4(7):229–254, 1991. [13] M. Persson, J. E. Solem, K. Markenroth, J. Svensson, and A. Heyden. Phase constrast mri segmentation using velocity and intensity. In The 5th International Conference on Scale Space and PDE methods in Computer Vision, Scale Space 2005, Hofgeismar, Germany, pages 119–130. Springer, 2005. [14] J. E. Solem and N. Overgaard. A geometric formulation of gradient descent for variational problems with moving surfaces. In The 5th International Conference on Scale Space and PDE methods in Computer Vision, Scale Space 2005, Hofgeismar, Germany, pages 419–430. Springer, 2005. [15] C. Xu and J. L. Prince. Snakes, shapes and gradient vector flow. IEEE Trans. on Image Processing, 7(3):359–369, 1998. [16] A. Yezzi and A. Mennucci. Conformal metrics and true gradient flows for curves. In International Conference on Computer Vision, ICCV 2005, Beijing, China, pages 913–919. IEEE Computer Society, 2005.