A Multi-label Front Propagation Approach for Object Segmentation Hua LIa,b,
Abderrahim ELMOATAZc
Jalal FADILIa
Su RUANd
a
GREYC-ISMRA, CNRS 6072, 6 Bd. Maréchal Juin, 14050 Caen, France Dept. of Electronics & Information Engineering, Huazhong Univ. of Sci. & Tech., State Education Commission Laboratory for Image Processing and Intelligent Control, P.R.China c LUSAC, Site Universitaire, BP78, 50130, Cherbourg-Octeville, France d Equipe Image, L.A.M.(EA 2075), Dept. GE&II, IUT de Troyes, 10026 Troyes, France {hua.li j.fadili}@greyc.ismra.fr
[email protected] [email protected] b
Abstract For effective image segmentation methods, speed, accuracy and smoothness of the result are essential. In this paper, an iterative object segmentation approach is proposed based on minimal path theory. Each iterative step includes one morphological dilatation and one multi-label front propagation. A narrow band is obtained by dilating the current contour with the known size. A new contour is again formed by multi-label front propagation, which is based on minimal path theory. Its propagation speed is decided by the local image mean values together with the edge function. The final boundary will be obtained automatically through finite iterations. This algorithm is a global optimization method. It is simple and fast with complexity O(N). The initial contour may be chosen freely. The multi-label front propagation guarantees continuity and smooth contours with the capability to handle topology changes. Furthermore, it is easy to extend to the 3D case. Some experimental results are also presented.
1. Introduction Image segmentation is one of the first and most important tasks in image analysis and computer vision. In computer vision literature, various methods dealing with object segmentation and feature extraction are discussed [1]. However, because of the variety and complexity of images, robust and efficient segmentation algorithms are still a very challenging research topic. Among various image segmentation techniques, active contour model [2] has emerged as a powerful tool for semiautomatic object segmentation. The basic idea is to evolve a curve, subject to constraints from a given image, for detecting interesting objects in that image. But this model has several disadvantages, such as sensitivity to initialization, easy to local minimum, and difficulty to handle topology changes. Some approaches have been
proposed to improve the robustness and stability of snakes [3, 4]. However, these active contour models still have problems to find a good global minimum. Spurious edges generated by noise may stop the evolution of the curve, giving an insignificant local minimum of the energy. In order to overcome the problem of local minimization for classical active contour models, Cohen proposed the minimal path theory in [5, 6]. With this approach the image is defined as an oriented graph characterized by its cost function. The boundary segmentation problem becomes an optimal path search problem between two nodes in the graph. This approach is implemented by fast marching evolution[7,8]. Its complexity is O(NlogN). Furthermore, the classical front evolution speeds rely on the edge function decided by the image gradient, to stop the curve evolution. The disadvantage of edge-based stopping function is its inaccuracy since it depends on a Gaussian smoothed image, and in some case where there is a gap in the edge information, the nearest seed will flood the adjacent region. The new kind constraint were proposed in [9,10], in which the stopping term does not depend on the gradient of image, but is instead related to a particular segmentation of the image. Deschamps also proposed an improved fast marching method [11], in which the speed function is inversely proportional to the difference between the gray level of the starting point and other points in the image. The use of this kind of information to differentiate the different front speeds has a bigger potential than the traditional speed function. In [12], Xu proposed a graph cuts-based active contours approach, which combines active contour model and the optimization tool of graph cuts, for object segmentation. Graph cuts theory is similar to the minimal path principle. In his method, the graph with appropriate pixel connectivity and edge weights must be constructed, which is an uneasy and time consuming task. The initial boundary should be near from the object boundary. His method cannot segment
0-7695-2128-2/04 $20.00 (C) 2004 IEEE
multiple objects and cannot treat with topology changes automatically. In this paper, we propose an iterative object segmentation approach for fast, robust and flexible segmentation. Each iteration step includes a morphological dilatation and a multi-label front propagation. The multilabel front propagation method is proposed based on the minimal path theory. It overcomes the problem of local minimum by using fast sweeping front propagation method [13]. In its speed function, we consider the local image mean values to improve the accuracy and smoothness of the result. Its complexity is O(N). Firstly, an initial contour is chosen freely, and a local narrow band is formed from this initial boundary with a morphological dilatation operator. Then, multi-label front propagation is used. The inner boundary and the outer boundary of this narrow band are labeled differently, and propagate in different speed functions towards unlabeled regions; a new contour that better separates the different labeled regions is formed within the narrow band. Through repeating the dilatation and propagation steps, the closest contour that is a global minimum within the narrow band is found. The narrow band is obtained by dilatation of the current contour with the known size, and a new boundary is formed by the multi-label front propagation. The contour is replaced iteratively until the objective segmentation is achieved. In brief, our algorithm is simple and fast with complexity O(N). The initial contour may be chosen freely in position and size. The multi-label front propagation provides continuity and smooth contours free of topology changes. This method is easy to extend to 3D case. This paper is organized as the following. Section 2 is the detailed algorithm. In Section 3, the experimental results on syntactic and real medical images are shown. Finally, conclusions and possible extensions are discussed in Section 4.
2. Our Approach Our algorithm may be summarized in the following steps: 1. Choose the initial contour freely anywhere in the image. 2. Use morphological dilatation operators to expand the initial boundary to form a narrow band. The size of the narrow band can be specified by the user for a given segmentation, image or a class of images according to the characteristics of local minima. 3. Identify all the points on the outer boundary of the narrow band as a single label, and identify all the points on the inner boundary of the narrow band as another single label.
4. Use multi-label front propagation to propagate the labeled boundaries with different speed functions towards unlabeled regions, the evolution will stop at the contact point and form a new boundary automatically. Here, the speed function considers the local image information (local mean values) and does not require the choice of a stopping threshold. 5. Repeat step 2- 4. The algorithm iteratively replaces a contour with a global minimum within the narrow band formed from it until the final result is achieved. In the following subsection, we give some details about multi-label front propagation.
2.1. Multi-label front propagation method In minimal-path theory [6], the surface of minimal action U is defined as the minimal energy integrated along a path between a starting point p0 and any point p, shown in the following Equation (1):
U ( p ) = inf
Ap 0 , p
{∫ P~(C( s))ds}= inf {E (C )} Ω
Ap 0 , p
(1)
where Ap0,p is the set of all paths between p0 and p. C(s) represents a curve on a 2D image. Ω is its domain of definition. E(C) represents the energy along the curve C. ~ P is the integral potential. The minimal path between p0 and any point p in the image can be easily deduced from this action map U by solving the following Eikonal Equation (2): ~ (2) ∇U = P with U ( p0 ) = 0 The definition of the region Rk associated with a point pk is the set of points of the image closer in energy to pk than to any other point. Consider all the point satisfying U pi ( p ) = U pj ( p ) , at these points, the front starting from pi to computer Upi meets the front starting from pj to computer Upj and the propagation stops. In another words, these points form the boundary between the region Ri and the region Rj. Without loss of generality, Let X be a set of continuous points in the image, UX is the minimal action with potential ~ P and starting points {p, p ∈ X } . It is easy to see that
U X = min p∈X U p . Considering all the point satisfying U Xi ( p ) = U Xj ( p ) , these points separate the region RXi to region RXj. This gives us a rule to find the region’s boundary based on finding all the points where different minimal actions U are equal to any others. It is a global way for extracting the global minimum. So, the basic idea of multi-label front propagation is that any number of independent contours propagate with different velocities depended on the label and local image mean values, towards the unlabeled space; the region or
0-7695-2128-2/04 $20.00 (C) 2004 IEEE
contour that first reached the specific pixel is calculated; and the evolution will stop at the contact point and form the new boundary automatically. The fast marching method proposed by Sethian [7, 8] is used as the evolution scheme in minimal path theory to provide efficient evolution. Its complexity is O(NlogN). But in our approach, the front evolution scheme is an extension of fast sweeping method, which is presented by Zhao [13, 14] for computing the numerical solution of Eikonal equations on a rectangular grid. It gives the same result as fast marching method but with lower complexity O(N). Since the low computational cost of fast sweeping method is maintained, the complexity of our method is O(N), where N is the number of grid points. As the discussion in section 1, the classical speed function for front evolution is just based on the gradient information, and the segmentation quality relies on the efficiency of the edge detection applied to the real image. Considering this, in our multi-label front propagation, we propagate the front according to the local mean information, only adding gradient information in some specific situation when needed. As shown in Figure 1, the inner boundary Bin and the outer boundary Bout are labeled differently, and evolved in opposite directions. The propagation is stopped at the contraction of the two fronts. The propagation speed for the labeled points is: F ( x ) = I ( x ) − mean (l ) + f (∇I ( x )) if L( x ) = l (3) where l is the label, mean(l) represents the mean value of all pixels with same label l. f ( ∇I ( x )) is a function of the image gradient.
(a)
(b)
(c)
Figure 1. (a) an original contour; (b) using dilatation to obtain the narrow band; (c) the inner and outer boundary and their propagation. The reason why we use the morphological dilatation operator to obtain the narrow band is that the iteration step size can be controlled easily by adjusting the size of the structure element and the dilatation times. The structure element can be selected based on the size and shape of the object to be segmented and the global characteristic of the processed image. If the segmented object is larger, the step size may be larger. Of course, different structure elements can be used. The experimental results show that
performances are better with 8-connexity template than that with 4-connexity template.
3. Experimental Results We have tested our method on the syntactic image and real medical images in MRI data sets. Several examples for object segmentation are shown. In Figure 2, in the detected gray image, there are several objects. Our method can detect the outer boundary of the entire objects using only one initial contour shown in Figure 2(a). As for the holes in the image, we set an initial seed within the holes and another outside, and then use multilabel front propagation to detect them. The combination of the results from every initial contour is the final segmentation. Here, we also show that initial curve does not necessary surround the objects. Figure 2(b) shows the result on the original noisy image. Figure 2(c) shows the segmentation result on the smoothed image, which is much smoother than the result in Figure 2(b). In Figure 3, we give the segmentation results on the real MRI medical brain slices. By choosing different initialization, our method can detect the ventricles parts and the white matter in the brain image by finite iterations respectively. Figure 3(a) and Figure 3(c) are the original images with initialization. The final results are shown in Figure 3 (b) and (d). In Figure 4, we give the segmentation results on the medical liver slices. Because the gray values outside the liver is very complex, we first used the fast sweeping evolution with the speed function proposed in [15] and an initial seed point inside the liver part to get an initial boundary near the actual boundary, and then we used our method to obtain the final result. Here only one iteration is needed. Figure 4(b) is the final segmentation results. Figure 4(c) is the segmentation result by using the same method on 3D image directly.
4. Conclusions In this paper, we proposed a new iterative object segmentation approach including the multi-label front propagation and morphological dilatation. This method does not require thinking the edge function and choosing the stopping time. The initial contour can be an arbitrary shape located anywhere in the image. It is fast and simple with complexity O(N), and can deal with the problem of topology changes. In the future, our research will focus on how to combine a priori information and statistical characteristics of the desired objects, such as texture, shape, intensity profile, image modality; how to detect line-like shapes and unclosed boundary; how to provide more accurate segmentation result in the general case of complex image.
0-7695-2128-2/04 $20.00 (C) 2004 IEEE
References [1] (a)
(b)
(c)
Figure 2. Segmentation results on gray image with multiple objects, noise and holes.
[2] [3] [4]
[5]
(a)
(b)
[6] [7] [8] [9]
(c)
(d)
Figure 3. Segmentation results on 2D brain slices.
[10]
[11]
[12] (a)
(b) [13] [14]
[15] (c)
Figure 4. Segmentation results on 2D liver slices and 3D liver image.
X.Munoz, and et al., “Strategies for Image Segmentation Combining Region and Boundary Information”, Pattern Recognition Letter, 2003, 24, pp. 375-392. M.Kass, A.Witkin, and D.Terzopoulos, “Snakes: Active Contour Models”, International Journal of Computer Vision, 1988, 1, pp. 321-331. T.McInerney, and D.Terzopoulos, “Deformable Models: in Medical Image Analysis: A Survey”, Medical Imaging Analysis, 1996, 1(2), pp. 91-108. J.S.Suri, K.C.Liu, and et al., “Shape Recovery Algorithms using Level Sets in 2D/3D Medical Imagery: A State of the Art Review”, IEEE Trans. on Info. Tech. in Biomedicine, 2002, 6(1), pp. 8-28. L.D.Cohen, and R.Kimmel, “Global Minimum for Active Contour Models: A Minimal Path Approach”, International Journal of Computer Vision, 1997, pp.57-78. L.D.Cohen, “Multiple Contour Finding and Perceptual Grouping using Minimal Paths”, Journal of Mathematical Imaging and Vision, 2001. J.A.Sethian, “A Fast Marching Level Set Method for Monotonically Advancing Fronts”. Proc. Nat. Acad. Sci., 1996, 93(4), pp. 1591-1595. J.A.Sethian, “Fast Marching Methods”, SIAM Review, 1999, 41. T.F.Chan, and L.A.Vese, “Active Contours without Edges”, IEEE Trans. on Image Processing, 2001, 10(2), pp. 266277. A.Yezzi, A.Tsai, A.Willsky, “A Fully Global Approach to Image Segmentation via Coupled Curve Evolution Equations”, Journal of Visual Communication and Image Representation, 2002, 13, pp.195-216. T.Deschamps, “Curve and Shape Extraction with Minimal Path and Level-Sets Techniques. Applications to 3D Medical Imaging”, Ph.D dissertation, L’UNIVERSITE de PARIS-DAUPHINE, December 2001. N.Xu, R.Bansal, and N.Ahuja, “Object Segmentation using Graph Cuts based Active Contours”, Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition, 2003, 2, pp. 46-53. H.K.Zhao, “Fast Sweeping Method for Eikonal Equation I: Distance Function”, http://www.math.uci.edu/~zhao, 2002. H.K.Zhao, and et al., “Implicit and Non-parametric Shape Reconstruction from Unorganized Points using Variational Level Set Method”, Computer Vision and Image Understanding, 2000, 80(3), pp.295-319. R.Malladi, and J.A.Sethian, “A Unified Approach to Noise Removal, Image Enhancement, and Shape Recovery”, IEEE Trans. on Image Processing, 1996, 5, pp. 1554-1568.
Acknowledgement This research work is financial supported by GRAVIR (Groupe Régional d’Action pour la Valorisation Industrielle de la Recherche) of Basse Normandie, France.
0-7695-2128-2/04 $20.00 (C) 2004 IEEE
