Graph Cut Based Deformable Model with Statistical ... - CiteSeerX

Graph Cut Based Deformable Model with Statistical Shape Priors Noha El-Zehiry and Adel Elmaghraby Computer Engineering and Computer Science Department - University of Louisville

fnyelze01,adelg

Abstract This paper presents a novel graph cut based segmentation approach with shape priors. The model incorporates statistical shape prior information with the active contour without edges model [6]. Our model also relaxes the homogeneity constraint that assumes that the image is modeled by a piecewise constant approximation.

The major contri-

bution of this paper is to present a graph cut optimization for the energy function. Hence, the resultant approach is a fully automatic shape based segmentation approach that is insensitive to initialization and does not require any user interaction. Due to the polynomial time complexity of graph cut optimization approaches, our segmentation technique is much faster than the state of the art deformable models segmentation approaches.

1. Introduction

Graph cuts and deformable models are two of the most important frameworks that have been used in the last decade to solve the image segmentation problem. Recent studies such as [7, 8, 13] are in the favor of combining deformable model approaches and graph cuts optimization to bene t from the advantages of deformable models in describing a wide variety of segmentation techniques and meanwhile take advantage of the fast global optimization associated with the use of graph cuts. In this paper, we extend this trend by presenting a shape based segmentation approach; a new geometric active contour model with statistical shape priors. The energy formulation of the model relaxes the global piecewise constant constraint in [6] and [8]. The energy will be optimized using graph cuts. We will point out the most relevant papers to our approach and we refer the reader to [5] and [12] for surveys of graph cut methods and deformable models, respectively. User interaction and sensitivity to initialization seem to be two common obstacles towards the design of an automatic graph cut segmentation approach that 978-1-4244-2175-6/08/$25.00 ©2008 IEEE

@ louisville.edu

incorporates prior knowledge [1, 2, 11, 13]. El-Zehiry et al. [8] introduced a discrete version of the ChanVese active contour model in [6] and performed the optimization using graph cuts. Their approach is insensitive to initialization and robust to ill-de ned edges. However, it assumes piecewise constant image model so it can not handle images with inhomogeneities. On the other hand, the algorithm in [10] presented an active contour model that accounts for local inhomogeneities and takes statistical shape priors into consideration but their model is formulated in a level set framework which makes the numerical implementation very computationally expensive. The contribution of our paper is two fold; First, from the graph cuts perspective, the algorithm will incorporate shape priors into a graph cut segmentation model. The model will be fully automatic and will not require any user interaction. Second, from the deformable models perspective, the energy function will be minimized using graph cuts rather than level sets which makes the model much less computationally intensive. 2. Related Graph Theory Background

This section reviews the graph theoretic description of graph representability of functions. Let E be a function of n binary variables x1 , x2 , ..., xn . Let G = fV ; Eg be a graph with n + 2 vertices, i.e., V = f v1 , v2 , ..., vn , S , T g, each variable xi has a corresponding vertex vi plus two terminal vertices S and T . An ST cut C is a set of edges whose removal separates the the source from the target in two different partitions P1Pand P2 , respectively. The cost of the cut Cost(C ) = vp 2P1 ;v2 2P2 wpq where wpq is the weight of the edge connecting the vertices vp and vq . The graph G represents the energy function E if for each con guration of the variables x1 , x2 , ..., xn , xi 2 f0; 1g, the min ST cut (the ST cut that has the least cost) results in partitioning the vertex set V such that vi 2 P1 if xi = 1 and vi 2 P2 if xi =0. In this case, the value of E is equal to the cost of the min ST cut

(up to a constant). And the function can be minimized using max ow-min cut algorithms. A clear formulation of the graph representability has been introduced earlier by Kolmogorov and Zabih [9]. They introduced the F 2 class theorem that states that if E is a function of n binary variables X p X p;q E (xp ) + E (xp ; xq ) (1) E (x1 ; :::; xn ) = p

p 0 is a constant that controls the eect of the regularization term. 3.2

Graph representation framework

This section describes the discrete formulation of the energy function in equation (6), discusses the graph representability of it and the feasibility of optimizing it using max- ow/min-cut algorithms. We introduce a binary variable xp associated with each pixel p = (x; y) 2 such that; 1; p 2 ! ; xp = (7) 0; p 2 n!. The discrete formulation of the length of the contour has been discussed by Boykov et al. [3], ElZehiry et al. [8]. El-Zehiry et al. [8] used the discrete formulation of the length of the contour in a similar framework to optimize the MumfordShah functional. We will use their representation here P to rewrite the third term in equation (6) as  vp vq 2E (G ) wpq (xp + xq 2xp xq ) the edge set of the graph that we will construct to represent the function F1 and wpq is the weight of the edge connecting the vertices vp and vq 1 . Hence the corresponding discrete formulation formulation of the energy function F1 is: F1 (m1 ; m2 ; C ) = P (im(x;y) m1 (x;y))2 +log(1 ) log(P r(!)))xp +

( 212 P (im(x;y) m2 (x;y))2 +log(2 ) log(P r( n!)))(1 xp ) 2

( P 22 + vp vq 2E (G) wpq (xp + xq 2xp xq ) (8) 1 For

more details about the formulation and the choice of

the edge weights, we refer the reader to [8]

( ( )) and E p (0) = (im(x;y)2m22 2 (x;y)) + log (2 ) log (P r ( n! )).  If E p (1) > E p (0) add a t-link S vp with weight E p (1) E p (0), and if E p (0) > E p (1) add a t-link vp T with weight E p (0) E p (1). 6. Construct the neighboring links (n-links) as follows:  For each pixel p 2 ! identify all the vertices vp 2 N (p) in the 8- neighborhood system of the vertex vp described in [3, 8].  If jxp 2 xq j=1 add an n-link vp vq with weight 
wpq = je pq pq j where pq and jepq j are the orientation of the edge epq (between the two vertices vp and vq ) and its length, respectively.

where n is the total number of pixels in the image. The parameters m1 (x; y) and m2 (x; y) are the mean intensity values of the pixels in the sets S1 = W \ ! and S2 = W \ ( n!), respectively. And W is a rectangular window centered at the pixel p. The mean intensities are represented in terms of the binary variable xp as: P z 2W (x;y ) im(z )xz P (9) m1 (x; y ) = z 2W (x;y ) xz P xz ) z 2W (x;y ) im(z )(1 P (10) m2 (x; y ) = (1 x ) 2 ( )

z W x;y

z

The discrete formulation in (8) resembles the energy function in (1) with E p (xp ) is equal to the rst two sums in (8) and E p;q (xp ; xq ) is equal to the third sum. Regularity is guaranteed because E (0; 0) and E (1; 1) =0 but E (1; 0) and E (0; 1)  0 since the weights are always positive and hence E (0; 0) + E (1; 1)  E (0; 1) + E (1; 0). 3.3

The algorithm

Given a training data set of images

S

=

fI1 ; I2 ; :::; IN g, the segmentation algorithm can be de-

scribed as follows: Construction of the shape model 1. For each Ii 2 S apply a rigid transformation to register all the images to the same cartesian space. Obtain the registered set of images e R = fIe1 ; Ie2 ; :::; If N g where Ii = T (Ii ) and T is the registration transformation. Segment the registered images to obtain the segmented set SS = fS1 ; S2 ; :::; SN g. 2. The shapePNprior probability is calculated as: i=1 Si and P r( n! ) = 1 P r(! ) P r (! ) = N

Construction of the graph 3. Initialize the contour C as a circle. Initialization of the variable xp follows, such that, xp = 1 if p is inside the circle and zero, otherwise. 4. Construct a connected graph G = fV ; Eg, V = fv1 ; v2 ; :::; vn ; S ; T g, each vertex vi corresponds to a variable xi and the two terminal vertices S and T will represent the class labels. 5. Construct the terminal links (t-links) of the graph as follows;  Calculate 1 and 2 8p 2 . 2  Calculate E p (1) = (im(x;y)2m12 1 (x;y)) +log(1 )

2

log P r !

Minimizing the energy function 7. Find the min cut of the graph C using the max ow / min-cut described in [4]. 8. The min cut partitions the graph into two partitions P1 that contains the source and P2 that contains the target. The new class labels are obtained as follows; if vi 2 P1 thenxi = 1 and if vi 2 P2 then xi = 0. 9. Use the updated values of the binary variable to update the parameters 1 and 2 and hence the energy function F1 . 10. Repeat the steps from 4-8 until F1k F1k+1 <  4

Experimental Results

The gray scale version of the Caltech leaves data set2 has been used to test the performance of the algorithm. The data set consists of 184 images of leaves of dierent? shapes on dierent backgrounds. The dimensions of the images are 896  592. All the images have been registered to the rst image. We have used 84 images to calculate the shape prior probability distributions and the rest of images were used to test the quality of the segmentation. Figure 1 shows examples of dierent kinds of leaf images that are used to construct the image model. The images were aligned to get the average shape that has been used to calculate the shape prior distribution. Figure 2 shows a sample comparison of performing the segmentation without the shape prior knowledge and without relaxing the global piecewise constant constraint and the segmentation using our approach. Our approach clearly extracts the leaves 2 available

at:

Datasets/Caltech101

www.vision.caltech.edu/Image-

without the surrounding objects that share the same intensity level. The model takes ve or less iterations to converge. The algorithm was implemented in C++ on a 2GHZ CPU with 1GB RAM. It takes 4.38 mSec in average. For quantitative assessment, we compared our segmentation results to the ground truth by calculating the sensitivity, speci city, similarity index and misclassi cation. The average results are: 95 % for sensitivity, 99.94 % for speci city, 97.27 % for similarity index and 0.86 % for misclassi cations.

5. Conclusion and Future Work

The paper presented a graph cut based active contour model with statistical shape priors. The model accounts for local inhomogeneities which makes it more appropriate in practice. The advantage of this model over the graph cuts segmentation approaches is that it does not require any user interaction to initialize seeds in the objects or to enforce hard constraints. From the active contours perspective, the model is optimized using graph cuts which makes it much faster than other active contour models that are implemented in the computationally intensive level set framework. For future work, we plan to extend the model for volumetric image segmentation References

Figure 1. Model Construction. The first three images represent a sample of the segmented leaves used in the model construction. The last image is the constructed model.

Figure 2. The left column shows the segmentation using the segmentation model in [8]. The right column shows the segmentation results of our proposed approach.

[1] A. Blake, C. Rother, M. Brown, P. Peres, and P. Torr. Interactive image segmentation using an adaptive gmmrf model. In ECCV(1), pages 428{ 441, 2004. [2] Y. Boykov and M. Jolly. Interactive organ segmentation using graph cuts. In MICCAI, pages 276{286, 2000. [3] Y. Boykov and V. Kolmogorov. Computing geodesics and minimal surfaces via graph cuts. In ICCV, 2003. [4] Y. Boykov and V. Kolmogorov. An experimental comparison of min-cut/max- ow algorithms for energy minimization in vision. In TPAMI, volume 26, pages 1124{1137, 2004. [5] Y. Boykov and G. Lea. Graph cuts and ecient n-d segmentation. In Int. J. of Comp. Vis., volume 70, pages 109{131, 2006. [6] T. F. Chan and L. A. Vese. Active contours without edges. IEEE Transactions on Image Processing, 10(2):266{277, 2001. [7] J. Darbon. A note on the discrete binary mumford shah model. In MIRAGE, 2007. [8] N. El-Zehiry, P. Sahoo, S. Xu, and A. Almaghraby. Graph cut optimization for the mumford shah model. In VIIP, August 2007. [9] V. Kolmogorov and R. Zabih. What energy functions can be minimizedvia graph cuts? IEEE TPAMI, 26(2):147{159, 2004. [10] J. Liu, D. Chelberg, C. Smith, and H. Chebrolu. A likelihood-based active contour model for medical image segmentation. In BMVC, September 2007. [11] G. Slabaugh and G. Unal. Graph cuts segmentation using an elliptical shape priors. In ICIP, pages 1222{ 1225, 2005. [12] C. Xu, D. Pham, and J. Prince. Image segmentation using deformable models. In IHandbook of Medical Imaging., volume 2, 2000. [13] N. Xu, R. Bansal, and N. Ahuja. Object segmentation using graph cuts based active contours. In cvpr, volume 02, page 46, Los Alamitos, CA, USA, 2003. IEEE Computer Society.