SEGMENTING SIMILAR SHAPES VIA WEIGHTED GROUP ...

SEGMENTING SIMILAR SHAPES VIA WEIGHTED GROUP-SIMILARITY ACTIVE CONTOURS Peng Lv1 , Qingjie Zhao1 , Dongbing Gu2 1

Beijing Key Lab of Intelligent Information Technology School of Computer Science, Beijing Institute of Technology, Beijing 100081, P.R. China 2 School of Computer Science and Electronic Engineering, University of Essex, Colchester, UK 1

ABSTRACT This paper aims to segment similar targets shapes from multiple images by using unsupervised weighted group-similarity active contour model. We first use global contrast based saliency detector to extract the rough regions from the given multiple images group. Then a new algorithm is developed to measure the corresponding weight coefficients according to the similarities between rough regions and their latent common shape. In order to overcome the problem which caused by the trade-off between frame-specific details and group similarity more effectively during the evolution, a novel weighted group-similarity active contour model (WGSAC) is proposed, which reduces the noises generated from saliency detector dynamically and enables the curves to move toward the targets boundaries on different weighted images. Experiments on synthesized and real multiple images demonstrate that our approach is able to yield more stable segmentation results than previous methods. Index Terms— Active contours, group similarity, segmentation, saliency detection, shape similarity 1. INTRODUCTION AND RELATED WORK Target segmentation is a very important task in many applications. In order to meet this goal, a variety of segmentation methods have been proposed during last two decades. Edgebased [1] and region-based [2] active contour models are two classical methods for segmenting the target from an image by optimizing an energy functional. What’s more, for segmenting the target with particular shape from multiple images or video sequences, shape prior is often embedded into the curve evolution, as described in [3]. In addition to active contours, learning based technical [4] is also widely used for detecting or segmenting targets. Nevertheless, these shape prior based methods are supervised and often require a training set or manually initialization before segmenting, which limits their application. This work is supported by the National Natural Science Foundation of China (No. 61175096 and No. 61273273) and Specialized Fund for Joint Building Program of Beijing Municipal Education Commission.

978-1-4799-8339-1/15/$31.00 ©2015 IEEE

4032

In [5], Zhou et. al propose a unsupervised region-based active contour model with group similarity to extract similar shapes from multiple images, which uses low-rank transform matrix to restrict the similarities between curves shapes. However, the method has two limitations: (a) The regionbased active contours may fail to segment the target under sophisticated background, which would cause the method to generate inaccurate results; (b) Furthermore, when some images have severe noises that make their shapes different from the latent common shape significantly, the method could not balance the frame-specific details and global similarity effectively, which would lead to false segmentation on these disunity images. In order to cope with these limitations, in the paper, we propose a novel weighted group-similarity active contour model (WGSAC) based method for segmenting similar targets from multiple images. Our paper mainly has the following three-fold contributions: 1. We proposed a novel unsupervised framework for segmenting similar objects from images by integrating saliency detection and weighted group-similarity active contours. 2. We introduced a new algorithm to measure the weight coefficients of rough targets regions, which provide important cues for our weighted active contour model. 3. We also proposed a novel weighted group-similarity active contour model (WGSAC) to segment similar shapes from rough regions. The corresponding weight coefficients contribute to guiding the curves motion and correcting the shapes during the evolution, which enable our model to balance the trade-off between frame-specific details and group similarity effectively. The rest of the paper is organized as follows. Sect. 2 describes our weight group-similarity active contours based framework for segmenting similar targets from image group. We show the experimental results in Sect. 3. Some conclusion is given in Sect. 4.

ICIP 2015

where v(1) denotes the smallest value of distance set V = {vi |i = 1, · · · , n}. Finally, we use Logistic function to compute the weight coefficient of each shape as follows:

2. PROPOSED METHOD 2.1. Rough Regions Detection Giving a image group with n multiple images, {I1 , · · · , In }, before segmenting the targets, we first extract rough regions from images by using saliency detector, which is widely used for extracting noticeable target region. The global contrast based salient region detector [6] introduces a histogram-based contrast method to define pixel saliency values, and then extract rough regions by comparing histograms of each regions. After several GrabCut [7] iterations, rough target shape S on saliency map is obtained. 2.2. Shape Weight Coefficient Measurement To measure the weight coefficient ωi corresponding to each rough target region, firstly, we calculate the similarities between two shapes obtained by saliency detector. Since that Hu moments are proven to be invariant to image changes, such as rotation and scaling [8], in the model, we use Hu moments to represent the shape context of target region. The similarity between two shapes is defined by: |sign(hit ) · log hit − sign(hjt ) · log hjt | , t=1,··· ,7 |sign(hit ) · log hit | (1) where hit and hjt are Hu moments of Si and Sj , respectively. Here low value of di,j indicates high similarity between two shapes. Moreover, for each shape Si , we use vi to represent the average of distances to other shapes as follows: di,j (Si , Sj ) = max

1∑ di,j (Si , Sj ). n j=1 n

vi =

(2)

Since the shapes which significantly different from the latent common shape are in the minority, then they would tend to have large values of vi . Therefore, it is appropriate to rank saliency maps by the corresponding distance values. To make the distance value vi more distinguishable, for each shape we only consider the top K smallest values of set Di = {di,j |j = 1, · · · , n} and then rewrite vi as: vi =

K 1 ∑ di,(j) (Si , Sj ), K j=1

(3)

where K is a constant and di,(j) denotes the j-th smallest element of set Di . Before calculating the weight coefficients, we first normalize vi to value nvi in [0, 1] as follows: nvi = e(1−vi /v(1) ) ,

(4)

4033

ωi =

1−τ 1+e

−σ(nvi −0.5)

+ τ,

(5)

where both τ and σ are constants. 2.3. Weighted Group-Similarity Active Contours For each shape Si on obtained saliency maps, it could be represented by a series of discrete points (xi,j , yi,j ) on its curve: Ci = [xi,1 , · · · , xi,m , yi,1 , · · · , yi,m ]T . Therefore, two identical shapes which denoted by C1 and C2 would satisfy the following condition: C2 = Φ1 T, [

where Φ1 =

C1x 0 0 C1x

C1y 0 0 C1y

(6) ] 1 0 , 0 1

(7)

and T is the low-rank affine transformation defined by T = [a11 , a21 , a12 , a22 , b1 , b2 ]T , which describes the shape changes, such as scaling, translation, and rotation. Similarly, for a shapes matrix which consists of n identical shapes, C = [C1 , · · · , Cn ], the following condition will be satisfied: [C1 , · · · , Cn ] = Φ1 [T1 , · · · , T2 ].

(8)

Since the rank of transformation matrix [T1 , · · · , T2 ] is less or equal to 6, therefore, the shape matrix C is also a lowrank matrix, which satisfies rank(C) 6 6. It is obvious that rank(C) would increase when some shapes change, moreover, the greater the differences between a changed shape and other shapes are, the harder it would be for the changed shape to be represented by others. Therefore, the difference between different shapes in multiple images group could be restricted by the rank of shape matrix C. In order to segment a series of similar shapes C1 , · · · , Cn from multiple images, Zhou et. al [5] propose an active contour model with group similarity based on the characteristics of low rank shapes matrix. However, when some disunity shapes are significantly different from the latent common shape of images group, which caused by the saliency detector, occlusion, or other possible conditions, the curves are prone to be affected and converge to false boundaries during the evolution, as shown in frame #4, #5, and #8 in the top row of Fig. 1. Although group similar shapes could be segmented from these disunity images by increasing penalty and enforcing the restraint of shapes matrix, in this case, most frame-specific details of the shapes would be removed and inaccurate segmentation results would be obtained, as shown in the middle row of Fig. 1. As a result, model [5] is hard to balance the trade-off between frame-specific details and global similarity.

#1

#2

#3

#4

#5

#6

#7

#8

Fig. 1. Segmentation results on Heart with severe noises. Top: results of Zhou’s model [5] with low penalty constraint. Middle: results of Zhou’s model [5] with high penalty constraint. Bottom: results of the proposed model with weighted constraint. To cope with this problem, basing on the weight coefficients of multiple shapes, we propose a novel weighted group-similarity active contour model which is able to adjust the constraints of curves flexibly on different images and maintain frame-specific details while keeping similarity of the curves. To restrict different shapes separately, in our model, we define a new weighted shape matrix S as follows: S = w · C = [ω1 C1 , · · · , ωn Cn ].

(9)

Moreover, in order to achieve the purpose that segmenting similar shapes from different saliency maps while maintaining their frame-specific details, we propose the following objective energy function: min C

n ∑

ωi fi (Ci ) + λ||w · C||∗ ,

(10)

i=1

where ||w · C||∗ is a convex and continuous nuclear norm [9], which is able to represent the similarities between shapes robustly. What’s more, fi (Ci ) is the energy functional of regionbased active contour model [2]. In our model, large weight coefficients drive the curves to target boundaries for maintaining frame-specific details, while, small weight coefficients could keep the curves similar with the latent common shape on disunity images. Therefore, our proposed model is able to deal with the trade-off between noise removal and signal preserving more effectively than Zhou’s model. To optimize the objective energy function defined in Eq. 10, Proximal Gradient method [10] is applied. Our objective energy function belongs to the following optimization category: min F (w · C) + λR(w · C), (11) C

where R(w · C) = ||w · C||∗ . The differentiable function

4034

F (· ) is given by: F (w · C) =

n ∑

fi (ωi Ci ) ,

i=1

n ∑

ωi fi (Ci ) = w · F (C). (12)

i=1

In order to optimize the energy function defined in Eq. (11), the following iterative procedure is applied: 1 1 w · Ck+1 = arg min ||w · C − [w · Ck − ∇F (w · Ck )]||2F C 2 µ λ + R(w · C), µ (13) where ∇F (w · Ck ) = [ω1 ∇f1 (C1 ), · · · , ωn ∇fn (Cn )]. As shown in [11], the optimization problem in Eq.(13) could be solved by using singular value thresholding algorithm. Therefore, we obtain a new iterative procedure as follows: Ck+1 = w−1 · D λ (w · Ck − µ

1 ∇F (w · Ck )), µ

(14)

where D λ (·) is a singular value thresholding operator introµ

duced by J. Cai et. al in [11], and w−1 = (1/ω1 , · · · , 1/ωn ). After several iterations, more accurate and stable results are obtained, as shown in the bottom row of Fig. 1. 3. EXPIREMENTAL RESULTS 3.1. Expiremental Setup Implementation Details: The proposed method is implemented under Windows 7 platform on a Intel(R) Core(TM)i7 3.4GHz processor with 3GB memory. We set K = ⌈ n2 ⌉ in Eq. (3). In Eq. (5), we set σ = 10 and τ = 0.95, respectively. For the optimization of the weighted energy function, we set λ = 45 and µ ∈ [3, 5] in Eq. (14). Dataset: We test our model and Zhou’s model [5] on both synthesized and real images groups. We use the Heart

#1

#2

#3

#4

#5

0.978

0.083

0.110

0.889

0.112

#6

#7

0.983 0.071

(b)

(a)

(c)

Fig. 2. Segmentation results on Bottle1 sequence. For the top left panel, weight coefficients of the rough targets regions are shown below. (a) Segmentation results of [5] with low penalty constraint; (b) segmentation results of [5] with high penalty constraint; (c) segmentation results of the proposed model with weighted constraint. Table 1. Means and standard deviations of segmentation results on three tested image groups. Three methods are compared: [5] with low penalty, [5] with high penalty, and the proposed method (WGSAC). Methods [5] with low penalty [5] with high penalty the proposed method (WGSAC) Metrics Heart Bottle1 Bottle2

MAD (px) HD (px) Dice(%) MAD (px) HD (px) Dice(%) MAD (px) HD (px) Dice(%) 9.1/4.5 16.1/4.4 93.1/4.9 10.2/4.8 28.4/12.3 86.5/12.8 17.5/14.4 30.5/17.4 79.7/17.7 20.8/17.6 54.9/29.7 76.4/16.8 33.7/19.7 95.6/56.9 68.4/16.5 14.1/7.7 37.6/23.7 89.6/6.0 31.9/22.3 56.8/32.3 74.5/19.1 40.2/28.2 82.4/39.7 69.8/14.1 17.2/9.1 39.2/19.8 83.4/7.8

sequence from [5] as the synthesized dataset and strengthen noises in images for better comparison. Moreover, we also test two sequences in Bottle, which comes from ETHZ shape classes, to demonstrate the improvement of the proposed method on real multiple images.

in [5], are used to compare the implemented methods. Limited by the inflexible adjustment of constraint between the multiple images, method [5] could not yields good results neither with low nor high penalties. With the constraint of weight coefficients, our method performs better than method [5], especially under HD and Dice criterions on tested images groups.

3.2. Qualitative And Quantitative Analysis Without the constraint of weight coefficients, method [5] fails to segment the shapes accurately when the noise is very large, as shown in frames #4, #5, and #8 in the top row of Fig. 1 and frames #2 and #7 in Fig. 2(c). Although Zhou’s model could extract the highly similar shapes by enforcing the penalty of the shapes matrix, in this case, some important frame-specific details would be lost, which results in large segmentation errors, as shown in Fig. 2(b), especially frame #6 and #7. Segmentation results of the proposed method are presented in last panels of Fig. 1 and Fig. 2. It is shown that under the flexible constraints on different shapes, our method is able to cope with the trade-off between frame-specific details and the group similarity of multiple shapes more robustly. As shown in Fig. 2, in our model, hight weight coefficients drive the curves move to the target boundaries in frame #1, #4, and #6, while in frame #2 and #7, small weight coefficients enable the curves to resist severe noises and keeps their shapes similar with latent common shape. Now we quantify our method. As shown in Table 1, three evaluation metrics (MAD, HD, and Dice), which introduced

4035

4. CONCLUSION In this paper, we develop a novel unsupervised weighted group-similarity active contours based method for segmenting similar shapes from multiple images. Firstly, we detect the rough targets regions by using salient region detector, and measure the corresponding weight coefficient vector, which provides flexible weight information for our method. Nevertheless, in Zhou’s model [5], the trade-off between frame-specific details and group similarity is hard to balance. To breakthrough this limitation, we propose a new weighted group-similarity active contour model, which enables the curves to converge to targets boundaries while keeping the similarity between the shapes in multiple images. Future work will aim at developing a more powerful algorithm to measure the similarities between shapes and embedding target appearance cues into active contour model, which might improve the segmentation performance of our method.

5. REFERENCES [1] Michael Kass, Andrew Witkin, and Demetri Terzopoulos, “Snakes: Active contour models,” Int. J. Comput. Vision, vol. 1, no. 4. [2] Tony F Chan and Luminita A Vese, “Active contours without edges,” IEEE Trans. Image Process., vol. 10, no. 2, pp. 266–277, 2001. [3] Daniel Cremers, “Dynamical statistical shape priors for level set-based tracking,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 8, pp. 1262–1273, 2006. [4] Fei Chen, Huimin Yu, Roland Hu, and Xunxun Zeng, “Deep learning shape priors for object segmentation,” in IEEE Conf. Comput. Vision Pattern Recogn. IEEE, 2013, pp. 1870–1877. [5] Xiaowei Zhou, Xiaojie Huang, James S Duncan, and Weichuan Yu, “Active contours with group similarity,” in IEEE Conf. Comput. Vision Pattern Recogn. IEEE, 2013, pp. 2969–2976. [6] Ming-Ming Cheng, Guo-Xin Zhang, Niloy J Mitra, Xiaolei Huang, and Shi-Min Hu, “Global contrast based salient region detection,” in IEEE Conf. Comput. Vision Pattern Recogn. [7] Carsten Rother, Vladimir Kolmogorov, and Andrew Blake, “Grabcut: interactive foreground extraction using iterated graph cuts,” in ACM Trans. Graph., 2004, vol. 23, pp. 309–314. [8] Radhika Sivaramakrishna and NS Shashidhar, “Hu’s moment invariants: how invariant are they under skew and perspective transformations?,” in IEEE Proc. 97: Commun., Power and Comput. IEEE, 1997, pp. 292– 295. [9] Maryam Fazel, Matrix rank minimization with applications, Ph.D. thesis, PhD thesis, Stanford University, 2002. [10] Yurii Nesterov et al., “Gradient methods for minimizing composite objective function,” 2007. [11] Jian-Feng Cai, Emmanuel J Cand`es, and Zuowei Shen, “A singular value thresholding algorithm for matrix completion,” SIAM J. Optim., vol. 20, no. 4, pp. 1956– 1982, 2010.

4036