BSLIC: SLIC Superpixels Based on Boundary Term

SS symmetry Article

BSLIC: SLIC Superpixels Based on Boundary Term Hai Wang 1, *, Xiongyou Peng 1 , Xue Xiao 1 and Yan Liu 1,2 1 2

*

Electro-Mechanical Engineering School, Xidian University, Xi’an 710071, China; [email protected] (X.P.); [email protected] (X.X.); [email protected] (Y.L.) State Key Laboratory for Manufacturing Systems Engineering, Xi’an Jiaotong University, Xi’an 710054, China Correspondence: [email protected]; Tel.: +86-187-1084-7964

Academic Editor: Ka Lok Man Received: 24 November 2016; Accepted: 22 February 2017; Published: 26 February 2017

Abstract: A modified method for better superpixel generation based on simple linear iterative clustering (SLIC) is presented and named BSLIC in this paper. By initializing cluster centers in hexagon distribution and performing k-means clustering in a limited region, the generated superpixels are shaped into regular and compact hexagons. The additional cluster centers are initialized as edge pixels to improve boundary adherence, which is further promoted by incorporating the boundary term into the distance calculation of the k-means clustering. Berkeley Segmentation Dataset BSDS500 is used to qualitatively and quantitatively evaluate the proposed BSLIC method. Experimental results show that BSLIC achieves an excellent compromise between boundary adherence and regularity of size and shape. In comparison with SLIC, the boundary adherence of BSLIC is increased by at most 12.43% for boundary recall and 3.51% for under segmentation error. Keywords: superpixel; SLIC; edge pixel; boundary term

1. Introduction Although superpixels were firstly described as over-segmentations by Ren and Malik in 2003, they have still not been exactly defined [1,2]. Generally, a superpixel can be treated as the set of pixels that are similar in location, color, texture, etc. Superpixels can capture image redundancy and transform pixel-level computing into a region-level operation [3], which can greatly reduce the complexity of subsequent image processing tasks. Superpixel segmentation has become an important pre-processing step in various image processing applications, such as image segmentation [4–9], saliency detection [10–12] and classification [13,14]. Existing superpixel segmentation methods can be classified into three major categories: the spectral-graph-based method, the gradient-ascent-based method and the optimization-theorybased method. The spectral-graph-based method is an optimal procedure to globally minimize the graph-based objective functions [15]. This method treats image pixels as the nodes of graph and sets the edge weights of the graph proportional to the affinities between neighboring pixels [3]. Shi and Malik proposed the Normalized Cut (NCut) criterion for graph segmenting [16]. Although NCut generates superpixels of a similar size and compact shapes [3], its complexity brings a heavy burden into the computation works. Felzenszwalb and Huttenlocher (FH) [17] is another typical spectral-graphbased algorithm, which has a complexity of O( NlogN ) [18] and is faster than NCut. However, the superpixels generated by FH are very irregular in size and shape. The gradient-ascent-based method forms superpixels by updating the clustering of pixels in the direction of gradient ascent until a certain convergence criterion is met [3]. Mean Shift (MS) [19] and Quick Shift (QS) [20]

are two gradient-ascent-based methods. They have complexities of O N 2 and O dN 2 (d is a small constant) respectively, and the superpixels generated by them are still irregular in shape [3]. Turbopixel [21] is another gradient-ascent-based algorithm, which can provide more regular

Symmetry 2017, 9, 31; doi:10.3390/sym9030031

www.mdpi.com/journal/symmetry

Symmetry 2017, 9, 31   

2 of 14 

Symmetry 2017, 9, 31

2 of 14

Turbopixel  [21]  is  another  gradient‐ascent‐based  algorithm,  which  can  provide  more  regular  superpixels  and  shorten  the  processing  time  to    [3].  The  optimization‐theory‐based  method  superpixels and shorten the processing time to O( N ) [3]. The optimization-theory-based method formulates the image segmentation as a minimization problem of energy function, which is derived  formulates theinformation.  image segmentation a minimization oftwo  energy function, which is derived from  image  SuperPB as[22]  and  SEEDS problem [15]  are  classical  methods  based  on  from image information. SuperPB [22] and SEEDS [15] are two classical methods based on optimization optimization theories. SuperPB generates superpixels by minimizing two pseudo‐boolean functions,  theories. SuperPB generates superpixels by minimizing two pseudo-boolean functions, and its and its computing speed is independent from the superpixels number. SEEDS is a two‐stage method,  computing speed is independent from the superpixels number. SEEDS is a two-stage method, which which firstly segments the image into coarse superpixels and then refines the boundaries. The energy  firstly segments the image into coarse superpixels and then refines the boundaries. The energy function function of SEEDS is established based on the color similarity and histogram of the initial superpixels.  of SEEDS is established based on the color similarity and histogram of the initial superpixels. Thus, Thus, SEEDS features an excellent compromise between accuracy and efficiency.  SEEDS features an excellent compromise between accuracy and efficiency. SLIC (Simple Linear Iterative Clustering) is the most widely used superpixel method. Essentially,  SLIC (Simple Linear Iterative Clustering) is the most widely used superpixel method. Essentially, it is a local k‐means clustering method [3], and does not belong to the above three categories. SLIC  it is a local k-means clustering meaningful  method [3],regions  and does not belong to the feature  above three categories. groups  pixels  into  perceptually  in  a  five‐dimensional  space,  which  is  SLIC groups pixels into perceptually meaningful in coordinates.  a five-dimensional feature space, is defined  by  the  CIELAB  color  space  and  the  regions ,   pixel  Compared  with  the which above‐ defined by the CIELAB color space and the x, y pixel coordinates. Compared with the above-mentioned mentioned methods, SLIC has several advantages. Firstly, SLIC has lower complexity. For    image  methods, SLIC has several advantages. Firstly, SLIC has lower complexity. For N image pixels, pixels, the complexity of SLIC is just  . Secondly, SLIC can generate superpixels with compact,  the complexity of SLIC is just O( N ). Secondly, SLIC can generate superpixels with compact, regular regular size and shape. Moreover, SLIC is simple to use and understand [3,23].  size and shape. Moreover, SLIC is simple to use and understand [3,23]. Figure 1 displays the segmentation results of SEEDS, SuperPB, SLIC and Turbopixel. Numbers  Figure 1 displays the segmentation results of SEEDS, SuperPB, SLIC andcannot  Turbopixel. Numbers of  the  generated  superpixels  are  the  same  or  approximate.  The  edges  that  be  extracted  by  of the generated superpixels are the same or approximate. The edges that cannot be extracted by superpixels are labelled with a white dotted line. It can be observed from the first row that SEEDS  superpixels areworst  labelled with a in  white line. ItTurbopixel  can be observed from thethan  first row thatbut  SEEDS possesses  the  regularity  size dotted and  shape.  is  much  better  SEEDS,  still  possesses theSuperPB  worst regularity in size and shape. row  Turbopixel much enlarged  better thanimages  SEEDS,for  butthe  stillregion  worse worse  than  and  SLIC.  The  second  is  the  ispartial  than SuperPB and SLIC. The second row is the partial enlarged images for the region containing image containing image edges. It is observed that SEEDS owns higher boundary adherence, while the other  edges. It is observed that SEEDS owns higher boundary adherence, while the other three methods three methods cluster more pixels of slender rod and sky into one superpixel. Segmentation results  cluster slendera rod and skybetter  into one superpixel.between  Segmentation results suggest that suggest more that  pixels SLIC  of provides  relatively  compromise  boundary  adherence  and  SLIC provides a relatively better compromise between adherence and regularity. However, regularity.  However,  the  boundary  adherence  of  the boundary superpixels  generated  by  it  still  needs  to  be  the boundary adherence of the superpixels generated by it still needs to be improved. improved. 

 

(a) 

 

(b)

(c)

(d) 

Figure 1. Superpixel segmentation results of four algorithms. (a) SEEDS; (b) SuperPB; (c) simple linear  Figure 1. Superpixel segmentation results of four algorithms. (a) SEEDS; (b) SuperPB; (c) simple linear iterative clustering (SLIC); (d) Turbopixels. The second row is the partial enlarged superpixels defined  iterative clustering (SLIC); (d) Turbopixels. The second row is the partial enlarged superpixels defined by the green rectangles.  by the green rectangles.

In this paper, based on SLIC, an improved superpixel method is proposed to achieve an excellent  In this paper, based on SLIC, an improved superpixel method is proposed to achieve an excellent compromise  between  boundary  adherence  and  the  regularity  of  size  and  shape.  Boundaries  and  compromise between boundary adherence and the regularity of size and shape. Boundaries and regions are two affinitive cues for humans to perform segmentation [24]. As the small regions in an  regions are two affinitive cues for humans to perform segmentation [24]. As the small regions in

Symmetry 2017, 9, 31 Symmetry 2017, 9, 31   

3 of 14 3 of 14 

image domain, superpixels rely on region information more than boundary information. In order to  an image domain, superpixels rely on region information more than boundary information. In order to efficiently align the SLIC superpixels to image edges and obtain better segmentation results, a global  efficiently align the SLIC superpixels to image edges and obtain better segmentation results, a global usage of boundary and region information is conducted. In contrast to the previous method, which  usage of boundary and region information is conducted. In contrast to the previous method, which utilizes multi‐scale boundary information to segment a specific image region instead of the whole  utilizes multi-scale boundary information to segment a specific image region instead of the whole scene  [25], [25],  this this  paper paper  adopts adopts  aa  simple simple  edge edge  detecting detecting  algorithm algorithm  to to  obtain obtain  the the  complete complete  boundary boundary  scene information. Herein, the modified SLIC is expressed as BSLIC, where B stands for the boundary term.  information. Herein, the modified SLIC is expressed as BSLIC, where B stands for the boundary term. The advantages of BSLIC are summarized as follows: First, the modified method features competitive  The advantages of BSLIC are summarized as follows: First, the modified method features competitive boundary adherence, adherence,  which  be  validated  by  qualitative  and  quantitative  experiments  on  boundary which cancan  be validated by qualitative and quantitative experiments on databases databases BSDS500 [26]. Second, BSLIC, which only modifies two steps of SLIC, is very simple to use  BSDS500 [26]. Second, BSLIC, which only modifies two steps of SLIC, is very simple to use and and understand. Third, the superpixels generated by BSLIC have good compactness and regularity  understand. Third, the superpixels generated by BSLIC have good compactness and regularity in size in size and shape, which is an effective inheritance from SLIC.  and shape, which is an effective inheritance from SLIC. The rest of the paper is organized as follows: Section 2 provides an overview of the algorithm.  The rest of the paper is organized as follows: Section 2 provides an overview of the algorithm. Section 3 introduces the proposed BSLIC algorithm, including the distribution of cluster centers, the  Section 3 introduces the proposed BSLIC algorithm, including the distribution of cluster centers, initialization of edge centers and the distance measurement. Section 4 shows the experimental results  the initialization of edge centers and the distance measurement. Section 4 shows the experimental and necessary discussions. Finally, Section 5 concludes the paper.  results and necessary discussions. Finally, Section 5 concludes the paper. 2. Overview of BSIC 2. Overview of BSIC  BSLIC The data  data points  points of  of BSLIC  BSLIC are  are vectors BSLIC  is is  essentially essentially  aa  k-means k‐means  clustering clustering  method. method.  The  vectors  that that  describe the features of image pixels. In fact, spectral clustering [27] and subspace clustering [28] describe the features of image pixels. In fact, spectral clustering [27] and subspace clustering [28] are  are also capable of implementing the clustering of image pixels. However, the memory of these also capable of implementing the clustering of image pixels. However, the memory of these two kinds  two kinds of methods are demanding. Both the spectral clustering and the subspace clustering of methods are demanding. Both the spectral clustering and the subspace clustering method have to  method have to compute the affinity matrix of data points, which is an n × n matrix for an  image with compute the affinity matrix of data points, which is an    matrix for an image with  pixels. For  n pixels. For images with moderate resolution, as 321 × 481, the affinity matrix would take up images with moderate resolution, such as  321 such 481, the affinity matrix would take up more than  10 memory cells. As a pre-processing step, superpixels should be fast to compute, more than 2 × 10 2 10   memory cells. As a pre‐processing step, superpixels should be fast to compute, memory‐ memory-efficient and to  simple use. inBSLIC,  our BSLIC, k-means method chosento  toimplement  implement the  the efficient  and  simple  use. toSo,  in So, our  the  the k‐means  method  is  is chosen  clustering of image pixels. clustering of image pixels.  Plane centers

Plane centers initialization

Edge centers

Superpixels

Edge exists Superpixels segments

Orignal image

K-means clustering

Gray image

Gray level tranformation

Edge image

Canny filter

  Figure 2. The flow chart of the proposed BSLIC algorithm.  Figure 2. The flow chart of the proposed BSLIC algorithm.

Figure 2 illustrates the flow chart of BSLIC. In BSLIC, two kinds of cluster centers, “plane center”  Figure 2 illustrates the flow chart of BSLIC. In BSLIC, two kinds of cluster centers, “plane center” and “edge center” are initialized. We first initialize plane centers (red dots) in a hexagon distribution.  and “edge center” are initialized. We first initialize plane centers (red dots) in a hexagon distribution. For each plane center, an edge pixel within its local searching region is initialized as an edge center,  For plane anby  edge pixel within its localoperator,  searching region initialized as edge  an edge center, and each these  are  center, marked  green  dots.  A  canny  the  most iswidely  used  detecting  and these are marked by green dots. A canny operator, the most widely used edge detecting algorithm, algorithm, is utilized to obtain the boundary information, and the boundary term is then incorporated  into  the  local  k‐means  clustering.  For  each  iteration,  the  pixel‐wise  distance  is  dependent  on  the  boundary  term.  If  there  exists  an  edge  pixel  along  the  straight  line  connecting  the  two  pixels,  the 

Symmetry 2017, 9, 31

4 of 14

is utilized to obtain the boundary information, and the boundary term is then incorporated into the local k-means clustering. For each iteration, the pixel-wise distance is dependent on the boundary term. If there exists an edge pixel along the straight line connecting the two pixels, the distance is set as Inf ; otherwise, it is calculated as the weighted sum of color and space Euclidean distance. The iterating process is terminated when the residual error is less than 5%, and the iterating time is often no more than 10 [2]. 3. Algorithm of BSLIC The SLIC algorithm has been described by [3] in detail. In this section, only the differences between BSLIC and SLIC are presented. The BSLIC algorithm is different from SLIC in the following three aspects: initializing cluster centers in hexagon rather than square distribution, additionally choosing some specific edge pixels as cluster centers, incorporating boundary term into the distance measurement during k-means clustering. The technical details are discussed below. 3.1. Distribution of Cluster Centers In SLIC, the initialized cluster centers are squarely distributed as shown in Figure 3a. Centers Ci0 , Ci1 , Ci2 , Ci3 , Ci4 , Ci5 , Ci6 , Ci7 and Ci8 are sampled with an interval of S0 pixels. Rectangles in a black solid line are initialized superpixels. Given the number of image pixels N and the desired superpixels number k, the interval S0 can be obtained as: r N S0 = (1) k Square superpixels display central symmetry, but the regions containing the image edges are generally asymmetric. In order to improve the boundary adherence of superpixels across image edges, the symmetry properties of superpixels are reduced from central symmetry to axial symmetry. Both hexagons and diamonds display axial symmetry. Below, the initializing centers in hexagon and diamond distributions are discussed. Figure 3b displays the geometric sketch of a hexagon distribution. Centers Cj0 , Cj1 , Cj2 , Cj3 , Cj4 , Cj5 and Cj6 are sampled at intervals of Sh pixels in the horizontal direction, and Sv pixels in the vertical direction. Hexagons in the black solid line are superpixels. To make the hexagon superpixel equal to the square superpixel in area, side lengths of the square and hexagon should follow the mathematical relationship: √ x12 3 3 = (2) 2 x22 where x1 and x2 are the side lengths of the square and hexagon, and x1 is obtained by: r x1 =

N k

(3)

For a hexagon distribution, the horizontal and vertical intervals are given as: Sh =

√

Sv =

3x2

(4)

3 x2 2

(5)

Combining Equations (2)–(5), Sh and Sv are obtained as: v u N u Sh = t  √  3 2

·k

(6)

Symmetry 2017, 9, 31

5 of 14

s√

Symmetry 2017, 9, 31   

5 of 14  3 N Sv = · (7) 2 k For diamond distribution, which is displayed in Figure 3c, its horizontal and vertical intervals  For diamond distribution, which is displayed in Figure 3c, its horizontal and vertical intervals are are equal to that of the hexagon distribution.  equal to performs  that of thek‐means  hexagon clustering  distribution. SLIC  within  the  2 2   local  region  of  pixels.  For  a  square  SLIC performs k-means clustering within the 2S × 2S0 local region of pixels.  in Figure 3a  For a square 0 distribution, there are at most nine adjacent superpixels within the local region. Pixel distribution, there are at most nine adjacent superpixels within the local region. Pixel i in Figure 3a depicts this. The  2 2   local searching region of    is marked by the rectangle in the blue dotted  depicts this. The 2S × 2S local searching region of i is marked by the rectangle in the blue dotted line, 0 0 line, there exist nine adjacent superpixels ( ,  ,  ,  ,  ,  ,  ,    and  ) in the scope.  there exist nine adjacent superpixels (Ci0 , Ci1 , Ci2 , Ci3 , Ci4 , Ci5 , Ci6 , Ci7 and Ci8 ) in the scope. This means This means that, at most, nine distance calculations are needed for the assignment of one pixel. The  that, at most, nine distance calculations are needed for the assignment of one pixel. The complexity for complexity for each iteration of the clustering is 9 . Compared with square distribution, the hexagon  each iteration of the clustering is 9N. Compared with square distribution, the hexagon distribution has distribution has a lower complexity of 6   for one iteration of the clustering. As shown in Figure 3b,  a lower complexity of 6N for one iteration in Figure 3b, there are six adjacent there are six adjacent superpixels ( ,  ,  of the ,  clustering. ,    and As shown ) in the local searching scope of pixel .  superpixels (C , C , C , C , C and C ) in the local searching scope of pixel j. The complexity of the j0 j1 j2 j3 j4 j5 The complexity of the diamond distribution is also 6 , which is displayed in Figure 3c.  diamond distribution is also 6N, which is displayed in Figure 3c. : Superpixel

Ci2

Sh Cp1

Ci3

Cj1

Ci5

Ci6

Ci7

Ci8

p Cj5

(a)

Cp2

Cj2

j

x2

2S0

i Ci0

Ci4

: Image pixel Sh

x1

Sv

Ci1

: Superpixel center

Sv

S0

: 2S0 × 2S0 search region

Cp5 Cj0

Cj3

Cj4 (b)

Cp6

Cp0

Cj6 Cp3

Cp4

(c)

 

Figure  3.  3. Distribution  of  Cluster  Centers.  (a)  Square  distribution;  (b)  Hexagon  distribution;  (c)  Figure Distribution of Cluster Centers. (a) Square distribution; (b) Hexagon distribution; Diamond distribution.  (c) Diamond distribution.

Performing k‐means clustering in a limited size, rather than the whole image domain, is the key  Performing k-means clustering in a limited size, rather than the whole image domain, is the key point to speed up the SLIC algorithm because it largely reduces the number of distance calculations.  point to speed up the SLIC algorithm because it largely reduces the number of distance calculations. By  initializing  cluster  centers  in  the  hexagon  or  diamond  distribution,  the  number  of  distance  By initializing cluster centers in the hexagon or diamond distribution, the number of distance calculations  can  decrease  by  3 .  With  the  same  sample  intervals,  cluster  centers  initialized  in  the  calculations can decrease by 3N. With the same sample intervals, cluster centers initialized in the hexagon distribution are the same as that of the diamond distribution. In other words, the superpixel  hexagon distribution are the same as that of the diamond distribution. In other words, the superpixel segmentation results of the hexagon distribution are the same as that of the diamond distribution. In  segmentation results of the hexagon distribution are the same as that of the diamond distribution. our work, cluster centers are initialized in the hexagon distribution.  In our work, cluster centers are initialized in the hexagon distribution. 3.2. Initialization of Edge Centers  3.2. Initialization of Edge Centers The superpixels across the ground truth edges, which are called the edge‐across superpixels, are  The superpixels across the ground truth edges, which are called the edge-across superpixels, the  root  of  under  segmenting.  Edge‐across  superpixels  are  displayed  in  Figure  Their  are thecause  root cause of under segmenting. Edge-across superpixels are displayed in4c,d.  Figure 4c,d. cluster centers are initialized in the hexagon and square distribution respectively. The white and red  Their cluster centers are initialized in the hexagon and square distribution respectively. The white solid lines describe the image edges and the superpixels’ boundaries. Each closed region in a red solid  and red solid lines describe the image edges and the superpixels’ boundaries. Each closed region line  a  superpixel.  edge‐across  superpixels  denoted  as  ,  and  the  in arepresents  red solid line representsThe  a superpixel. The edge-acrossare  superpixels areFdenoted as Faccurate  i , and the superpixels are denoted as  R , which refers to the superpixels that exactly align to the ground truth  accurate superpixels are denoted as Ri , which refers to the superpixels that exactly align to the ground edges. The subscript    plays the role of counting the superpixels. It is found that the edge‐across and  truth edges. The subscript i plays the role of counting the superpixels. It is found that the edge-across accurate superpixels have a similar size. As shown in Figure 4c, the edge‐across superpixels  F   and  and accurate superpixels have a similar size. As shown in Figure 4c, the edge-across superpixels F F   have 67 and 101 pixels, while the accurate superpixels  R ,  R   and  R   have 99, 122 and 126 pixels.  1 and F2 have 67 and 101 pixels, while the accurate superpixels R1 , R2 and R3 have 99, 122 and For images with a resolution of  481 321, regions in  8 8  size (approximately 67 pixels) are hardly  126 pixels. For images with a resolution of 481 × 321, regions in 8 × 8 size (approximately 67 pixels) are different  from  regions  in  11 11  (approximately  126  pixels).  In  order  to  improve  the  boundary  adherence of edge‐across superpixels, the area of edge‐across superpixels should be decreased.  Edge‐across superpixels are the small regions containing image edges, while the edges are the  place where discontinuity in intensity or color happens. For an edge‐across superpixel, pixels located  in two sides of the edge are very dissimilar. To assign these pixels two superpixels, the exact edge 

Symmetry 2017, 9, 31

6 of 14

hardly different from regions in 11 × 11 (approximately 126 pixels). In order to improve the boundary adherence of edge-across superpixels, the area of edge-across superpixels should be decreased. Edge-across superpixels are the small regions containing image edges, while the edges are the place where discontinuity in intensity or color happens. For an edge-across superpixel, pixels located in two sides of the edge are very dissimilar. To assign these pixels two superpixels, the exact edge pixels Symmetry 2017, 9, 31    6 of 14  Symmetry 2017, 9, 31    are chosen as cluster centers. Centers initialized from edge pixels are called “edge center”, and 6 of 14  those that are initialized in a uniform geometric distribution are called “plane center”. The two kinds of pixels are chosen as cluster centers. Centers initialized from edge pixels are called “edge center”, and  pixels are chosen as cluster centers. Centers initialized from edge pixels are called “edge center”, and  cluster centers are illustrated in Figure 5. Edge center is determined by the following rules: those that are initialized in a uniform geometric distribution are called “plane center”. The two kinds  those that are initialized in a uniform geometric distribution are called “plane center”. The two kinds  of cluster centers are illustrated in Figure 5. Edge center is determined by the following rules:  1. If there exist image edges in the S0 × S0 searching region of plane center (like Ci1 ), choose a of cluster centers are illustrated in Figure 5. Edge center is determined by the following rules:  median edge image  pixel asedges  the edge center (like  searching  Ei1 ); 1. If  there  exist  in  the  region  of  plane  center  (like  ),  choose  a  1. If  there  exist  image  edges  in  the    searching  region  of  plane  center  (like  ),  choose  a  2. median edge pixel as the edge center (like  If there is no image edge within the S0 × S0 region (like Ci4 ), no edge center is generated; );  median edge pixel as the edge center (like  );  2. If there is no image edge within the    region (like  ), no edge center is generated;  3. If there is no image edge within the  To avoid any noisy pixel being chosen as cluster center, modify the edge center to the lowest 2.   region (like  ), no edge center is generated;  3. To avoid any noisy pixel being chosen as cluster center, modify the edge center to the lowest  gradient position in the corresponding 3 × 3 local neighborhood; 3. To avoid any noisy pixel being chosen as cluster center, modify the edge center to the lowest  gradient position in the corresponding  3 3  local neighborhood;  4. gradient position in the corresponding  Edge center is introduced to reduce the of edge-across superpixels. It is necessary 3 negativity 3  local neighborhood;  4. Edge center is introduced to reduce the negativity of edge‐across superpixels. It is necessary to  to keep it near to the image edges. During the 10 iterations, edge centers remain unalterable, 4. Edge center is introduced to reduce the negativity of edge‐across superpixels. It is necessary to  keep it near to the image edges. During the 10 iterations, edge centers remain unalterable, and  and only plane centers are updated to the mean value. keep it near to the image edges. During the 10 iterations, edge centers remain unalterable, and  only plane centers are updated to the mean value.  only plane centers are updated to the mean value. 

F1 F1

R1 R1

R3 R3

F2 F2

R1 R1

F1 F1

R2 R2

F2 F2

(c) (c)

(b) (b)

(a) (a)

R2 R2

(d) (d)

   

Figure 4. Edge‐across superpixels. (a) Original image with slender rod; (b) Edge image with canny  Figure 4. Edge-across superpixels. (a) Original image with slender rod; (b) Edge image with canny filter Figure 4. Edge‐across superpixels. (a) Original image with slender rod; (b) Edge image with canny  filter  operator;  (c)  Partial  enlarged  superpixels  with  only  plane  centers  initialized  in  hexagon  operator; (c) Partial enlarged superpixels with onlywith  plane centers initialized hexagon distribution; filter  operator;  (c)  Partial  enlarged  superpixels  only  plane  centers in initialized  in  hexagon  distribution; (d) Partial enlarged superpixels with only plane centers initialized in square distribution.  (d) Partial enlarged superpixels with only plane centers initialized in square distribution. distribution; (d) Partial enlarged superpixels with only plane centers initialized in square distribution.  Ci1 Ei1 Ci1 Ei1

j j

Ci5 Ci5

Ci3 Ci3

Ci2 Ci2

Ci6 Ci6

Ci0 Ci0

: S0 × S0 search region : S0 × S0 search region : Image edge : Image edge : Small local neighborhood : Small local neighborhood : Edge center : Edge center

Ci4 Ci4

   Figure  5.  Plane  center  and  edge  center.  Dots  in  solid  black  ,  ,  ,  ,  ,    and    are  Figure  5. Plane Plane center center and and edge edge  center.  Dots  in  solid  black  ,  C  i6 and    are  Figure 5. center. Dots in solid black C , C , , C , , C ,, C , , C and are “plane “plane center”; Square in solid red    is “edge center”.  i0 i1 i2 i3 i4 i5 “plane center”; Square in solid red  center”; Square in solid red Ei1 is “edge  is “edge center”.  center”.

Superpixel  segmentation  results  with  initialized  edge  centers  are  displayed  in  Figure  6.  Superpixel  segmentation  results  with  initialized  edge  centers  are  displayed  in  Figure  6.  Superpixel results with initialized edge centers displayed in Figure 6. Compared Compared  with segmentation Figure  4c,  the  edge‐across  superpixel  F   is  are successfully  transformed  into  the  Compared  with  Figure  4c,  the  edge‐across  superpixel  F   is  successfully  transformed  into  the  with Figure 4c, the edge-across superpixel F1 is can  successfully into the accurate superpixel accurate  superpixel  R   in  Figure  6a,  which  improve transformed boundary  adherence  to  some  degree.  accurate  superpixel  R   in  Figure  6a,  which  can  improve  boundary  adherence  to  some  degree.  R in Figure 6a, which can improve boundary adherence to some degree. However, when it comes 4 However, when it comes to square distribution, there is no apparent decrease in area for edge‐across  However, when it comes to square distribution, there is no apparent decrease in area for edge‐across  superpixels  F   and  F , as shown in Figure 4d and Figure 6b.  superpixels  F   and  F , as shown in Figure 4d and Figure 6b.  We  owe  the  better  segmentation  results  of  Figure  6a  to  the  asymmetry  of  the  edge‐across  We  owe  the  better  segmentation  results  of  Figure  6a  to  the  asymmetry  of  the  edge‐across  superpixel. For small regions without image details, such as edges and texture, the pixels possess  superpixel. For small regions without image details, such as edges and texture, the pixels possess  high similarity with others. These small regions can be described as possessing local symmetry to  high similarity with others. These small regions can be described as possessing local symmetry to  some degree. However, this is inconsistent for edge‐across superpixels, the pixels of which are located  some degree. However, this is inconsistent for edge‐across superpixels, the pixels of which are located 

Symmetry 2017, 9, 31

7 of 14

to square distribution, there is no apparent decrease in area for edge-across superpixels F1 and F2 , as shown in Figures 4d and 6b. We owe the better segmentation results of Figure 6a to the asymmetry of the edge-across superpixel. For small regions without image details, such as edges and texture, the pixels possess high similarity with others. These small regions can be described as possessing local symmetry to some degree. However, this is inconsistent for edge-across superpixels, the pixels of which are located on two sides of the edge and are very dissimilar. Edge-across superpixels are asymmetrical in the local region. Compared with superpixels initialized in square distribution, the hexagon distribution reduces the central and axial symmetry. Moreover, initializing edge centers largely destructs the horizontal and vertical symmetry of the initialized superpixels. Symmetry 2017, 9, 31    7 of 14 

R1

R4

F2

R2

R1

F1

R3

F2

R2

 

(a) 

(b)

Figure  6.  Partial  enlarged  superpixels superpixels  segmentation segmentation  with  edge  cluster cluster  centers centers initialized. initialized.  (a)  Figure 6. Partial enlarged with edge (a) Plane  Plane centers initialized in a hexagon distribution; (b) Plane centers initialized in a square distribution.  centers initialized in a hexagon distribution; (b) Plane centers initialized in a square distribution.

3.3. Distance Measurement  3.3. Distance Measurement From Figure 4c to Figure 6a, there is no apparent decrease in the area of edge‐across superpixel  From Figure 4c to Figure 6a, there is no apparent decrease in the area of edge-across superpixel F2 . F . This part introduces an available method to reduce the area of  F . In SLIC, the distance between  This part introduces an available method to reduce the area of F2 . In SLIC, the distance between pixels pixels is defined as the weighted sum of color and spatial Euclidean distance. For the pixels that are  is defined as the weighted sum of color and spatial Euclidean distance. For the pixels that are similar similar in spatial locations and color but belong to different objects, such as pixels    and    shown in  in spatial locations and color but belong to different objects, such as pixels i and j shown in Figure 7a, Figure 7a, SLIC tends to assign them to one superpixel. Setting a large value for the distance between  SLIC tends to assign them to one superpixel. Setting a large value for the distance between j and Ci is   and    is an effective way to assign    and    to different superpixels.  an effective way to assign i and j to different superpixels. Herein,  the  given  pair  of  pixels  is  connected  by  straight  line  over  the  image  domain.  The  Herein, the given pair of pixels is connected by straight line over the image domain. The boundary boundary term is incorporated by checking the presence of image edge along the straight line. Figure  term is incorporated by checking the presence of image edge along the straight line. Figure 7 7 illustrates this process. The edge image, which is obtained by a canny operator, is a binary image.  illustrates this process. The edge image, which is obtained by a canny operator, is a binary image. In fact, any existing edge detecting algorithms, such as Sobel, Robert, etc., are suitable for the work.  In fact, any existing edge detecting algorithms, such as Sobel, Robert, etc., are suitable for the work. Our problem is transformed into checking if there exists a nonzero element along the straight line. If  Our problem is transformed into checking if there exists a nonzero element along the straight line. it exists, such as pixel    and center    shown below, the distance would be Inf. Otherwise, like    and  If it exists, such as pixel i and center Ci shown below, the distance would be Inf. Otherwise, like j , the distance would be calculated as the weighted sum of color and spatial Euclidean distance. The  and Ci , the distance would be calculated as the weighted sum of color and spatial Euclidean distance. modified distance equation can be described as follows:  The modified distance equation can be described as follows:

d=

 r  

 2 d2c + m2 ·∙ Sd0s ,, No edge  

(8) (8)

In f , Edge , exits

  and d are   are color and spatial Euclidean distances. Parameter  is the weighting factor. It  where d and where color and spatial Euclidean distances. Parameter m is  the weighting factor. It is c s is suggested that    should be in the range of [1, 40] in CIELAB color space [3].  suggested that m should be in the range of [1, 40] in CIELAB color space [3]. Superpixel  segmentation  results  shown  in  Figure  8  with  boundary  term  incorporated.  Superpixel segmentation results are are  shown in Figure 8 with boundary term incorporated. Compared Compared  with  Figure  6a,  our  distance  measurement  successfully  transforms  the  edge‐across  with Figure 6a, our distance measurement successfully transforms the edge-across superpixel F2 into superpixel  F   into Raccurate  superpixel  R ,  realizing  a  further  progress  in  improving  boundary  accurate superpixel 5 , realizing a further progress in improving boundary adherence. adherence. 

j

where    and    are color and spatial Euclidean distances. Parameter    is the weighting factor. It  is suggested that    should be in the range of [1, 40] in CIELAB color space [3].  Superpixel  segmentation  results  are  shown  in  Figure  8  with  boundary  term  incorporated.  Compared  with  Figure  6a,  our  distance  measurement  successfully  transforms  the  edge‐across  superpixel  into  accurate  superpixel  R ,  realizing  a  further  progress  in  improving  boundary  Symmetry 2017,F9,  31 8 of 14 adherence. 

j j Ci

Ci i

i

 

(a) 

(b)

Figure 7. Boundary term incorporation. (a) Partial enlarged superpixels on original image; (b) Partial  Figure 7. Boundary term incorporation. (a) Partial enlarged superpixels on original image; (b) Partial enlarged superpixels on logical edge image.  Symmetry 2017, 9, 31     8 of 14  Symmetry 2017, 9, 31  8 of 14  enlarged superpixels on logical edge image.

RR11 RR55

RR22

RR44 RR33

   Figure 8. Superpixels segmentation result with modified distance calculation. Figure 8. Superpixels segmentation result with modified distance calculation.  Figure 8. Superpixels segmentation result with modified distance calculation. 

4. Experimental Results 4. Experimental Results  4. Experimental Results  In this section, we first discuss some important issues of BSLIC, including the iterating process In this section, we first discuss some important issues of BSLIC, including the iterating process  In this section, we first discuss some important issues of BSLIC, including the iterating process  and the effects of parameters. Then the evaluation of the BSLIC is presented together with other and the effects of parameters. Then the evaluation of the BSLIC is presented together with other four  and the effects of parameters. Then the evaluation of the BSLIC is presented together with other four  four algorithms. Qualitative quantitative experimental results are alldisplayed,  displayed,and  andnecessary  necessary algorithms.  Qualitative  and  quantitative  experimental  results  are  all  algorithms.  Qualitative  and and quantitative  experimental  results  are  all  displayed,  and  necessary  comparison is carried out to validate the competitive performance of BSLIC. comparison is carried out to validate the competitive performance of BSLIC.  comparison is carried out to validate the competitive performance of BSLIC.  4.1. BSLIC Superpixels 4.1. BSLIC Superpixels  4.1. BSLIC Superpixels  The iterating process of BSLIC is displayed in Figure 9. Figure 9a shows the superpixels with only The iterating process of BSLIC is displayed in Figure 9. Figure 9a shows the superpixels with  The iterating process of BSLIC is displayed in Figure 9. Figure 9a shows the superpixels with  planeplane  centers initialized, in which the superpixels have good axial symmetry in both the only  centers  initialized,  in  the  have  good  axial  in  both  only  plane  centers  initialized,  in which  which  the superpixels  superpixels  have  good  axial symmetry  symmetry  in horizontal both the  the  and vertical directions. Figure 9b is the superpixels with edge centers initialized. It is obvious thatIt  horizontal  is  horizontal and  and vertical  vertical directions.  directions. Figure  Figure 9b  9b is  is the  the superpixels  superpixels with  with edge  edge centers  centers initialized.  initialized.  It the is  axial symmetry of the edge-cross superpixels is destroyed. Figure 9c,d are the first and final iterating obvious that the axial symmetry of the edge‐cross superpixels is destroyed. Figure 9c,d are the first  obvious that the axial symmetry of the edge‐cross superpixels is destroyed. Figure 9c,d are the first  results. Compared to the final results, the first iteration has already aligned to the true boundaries in and final iterating results. Compared to the final results, the first iteration has already aligned to the  and final iterating results. Compared to the final results, the first iteration has already aligned to the  the whole picture. true boundaries in the whole picture.  true boundaries in the whole picture. 

(a)  (a) 

  

(b) (b)

(c) (c)

(d)  (d) 

Figure 9. BSLIC iterating process. (a) Superpixels with only plane centers; (b) Superpixels with both  Figure 9. BSLIC iterating process. (a) Superpixels with only plane centers; (b) Superpixels with both  Figure 9. BSLIC iterating process. (a) Superpixels with only plane centers; (b) Superpixels with both plane and edge centers; (c) First iterating result of BSLIC; (d) Final segmentation result of BSLIC.  plane and edge centers; (c) First iterating result of BSLIC; (d) Final segmentation result of BSLIC.  plane and edge centers; (c) First iterating result of BSLIC; (d) Final segmentation result of BSLIC.

The  The     norm is adopted to compute the residual error between two iterations. In BSLIC, the  norm is adopted to compute the residual error between two iterations. In BSLIC, the  iteration of k‐means clustering is not terminated until the residual error of the cluster centers is less  iteration of k‐means clustering is not terminated until the residual error of the cluster centers is less  than  than 5%.  5%. In  In the  the simulation  simulation experiments,  experiments, we  we found  found that  that small  small changes  changes of  of superpixel  superpixel segmentation  segmentation  results  happened  after five  or  six  iterations.  Figure 10  displays  the  change  curve  of  residual  results  happened  after five  or  six  iterations.  Figure 10  displays  the  change  curve  of  residual error  error  against  iterating  times.  After  10  or  fewer  times,  the  residual  error  becomes  convergent,  which  is 

Symmetry 2017, 9, 31

9 of 14

The L2 norm is adopted to compute the residual error between two iterations. In BSLIC, the iteration of k-means clustering is not terminated until the residual error of the cluster centers is less than 5%. In the simulation experiments, we found that small changes of superpixel segmentation results happened after five or six iterations. Figure 10 displays the change curve of residual error against iterating times. After 10 or fewer times, the residual error becomes convergent, which is consistent with the empirical data in [3]. Figure 11 shows the BSLIC segmentation results with different k and m. It is indicated that larger m achieves better regularity for superpixels, and smaller m achieves better adherence to image boundaries. Our explanation for this is that the spatial distance plays a key role when m is large, like 40, so the superpixels obey the geometric distribution in the image plane. In contrast, when m is small, the superpixels mainly obey the color distribution. To achieve a compromise on superpixels regularity and boundary adherence, m is usually set as 15. For parameter k, it is obvious that larger k aligns to more detailed boundaries. Considering the running time of the algorithm, it is appropriate Symmetry 2017, 9, 31  9 of 14      [100, 2500] for 481 × 321 images. 9 of 14  for k toSymmetry 2017, 9, 31  be in the range

 

 

Figure 10. Change curve of residual error against iterating times.  Figure 10. Change curve of residual error against iterating times. Figure 10. Change curve of residual error against iterating times. 

 

 

 

 

 

 

 

 

 

 

(a) 

(b)

(c) 

    (a)  superpixels  segmentation  results.  (b) (c)  Figure  11.  BSLIC  Superpixels  number  100, 500, 1000 ,  weighting factor  5, 15, 30 . (a) BSLIC segments when  5; Column (b) BSLIC segments when  Figure  11.  BSLIC  superpixels  segmentation  results.  Superpixels  number  100, 500, 1000 ,  Figure 11. 15; Column (c) BSLIC segments when  BSLIC superpixels segmentation results. Superpixels number k = [100, 500, 1000], 30.  weighting factor  5, 15, 30 . (a) BSLIC segments when  5; Column (b) BSLIC segments when  weighting 15; Column (c) BSLIC segments when  factor m = [5, 15, 30]. (a) BSLIC segments when m = 5; Column (b) BSLIC segments 30. 

4.2. Qualitative Comparisons  when m = 15; Column (c) BSLIC segments when m = 30. We compare BSLIC with four other methods: SEEDS, SuperPB, SLIC and Turbopixels. All the  4.2. Qualitative Comparisons 

500  images  of  dataset  BSDS500  are  segmented  by  these  five  methods.  As  the  superpixels  are  We compare BSLIC with four other methods: SEEDS, SuperPB, SLIC and Turbopixels. All the  segmented to give a description of features for a subsequent image processing task, they cannot be  500  images  of  dataset  BSDS500  are  segmented  by  these  five  methods.  As  the  superpixels  are  too small in size. Herein, we maintain the size of a superpixel larger than 80, which is nearly in  9 9  segmented to give a description of features for a subsequent image processing task, they cannot be  size.  For  images  in  481 321  resolution,  when  the  number  of  superpixels    exceeds  2000,  the  too small in size. Herein, we maintain the size of a superpixel larger than 80, which is nearly in  9  9  generated superpixels are approximately an average size of 77. Therefore, during the experiments, 

Symmetry 2017, 9, 31

10 of 14

4.2. Qualitative Comparisons We compare BSLIC with four other methods: SEEDS, SuperPB, SLIC and Turbopixels. All the 500 images of dataset BSDS500 are segmented by these five methods. As the superpixels are segmented to give a description of features for a subsequent image processing task, they cannot be too small in size. Herein, we maintain the size of a superpixel larger than 80, which is nearly in 9 × 9 size. For images in 481 × 321 resolution, when the number of superpixels k exceeds 2000, the generated superpixels are approximately an average size of 77. Therefore, during the experiments, k takes its value at intervals of 100 from 50 to 2500. Weighting factor m is within the range of [5, 15, 30]. Figure 12 displays part of the experimental results. The number of superpixels is the same or approximate in every row. Column (a) gives a qualitative presentation for BSLIC segmentations over the whole image domain. The superpixels are compact and uniform hexagons in shape, which is Symmetry 2017, 9, 31    10 of 14  benefits from initializing centers in the hexagon distribution. The superpixels are also regular in size, benefits from initializing centers in the hexagon distribution. The superpixels are also regular in size,  which is attributed to the locality of k-means clustering. Although SLIC segmentations over the which is attributed to the locality of k‐means clustering. Although SLIC segmentations over the whole  whole image domain are not displayed here, it is necessary to mention that both SLIC and BSLIC image domain are not displayed here, it is necessary to mention that both SLIC and BSLIC superpixels  superpixels are regular and compact in size and shape. Columns (b)–(f) provide the partial enlarged are regular and compact in size and shape. Columns (b)–(f) provide the partial enlarged results of the  results of the solid rectangle tagged regions, which contain image edges. Compared with the other four solid rectangle tagged regions, which contain image edges. Compared with the other four methods,  methods, BSLIC superpixels achieve higher adherence to image edges. For the other four methods, BSLIC superpixels achieve higher adherence to image edges. For the other four methods, we draw  we draw conclusions as follows: (1) SEEDS is highly irregular in size and shape; (2) SuperPB has conclusions  as  follows:  (1)  SEEDS  is  highly  irregular  in  size  and  shape;  (2)  SuperPB  has  high  high regularity and compactness in both size and shape, but it lacks flexibility when it comes to the regularity and compactness in both size and shape, but it lacks flexibility when it comes to the regions  regions containing edges; (3) SLIC Turbopixel an equal regularity in and  size shape,  and shape, but their containing  edges;  (3)  SLIC  and and Turbopixel  have have an  equal  regularity  in  size  but  their  boundary adherences are lower than BSLIC. boundary adherences are lower than BSLIC. 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

  (a) 

 

  (b) 

(c)

(d)

(e) 

(f) 

Figure 12. Part of experimental results. (a) BSLIC segments; (b) Partial enlarged superpixels of BSLIC; 

Figure 12. Part of experimental results. (a) BSLIC segments; (b) Partial enlarged superpixels of BSLIC; (c)  Partial  enlarged  superpixels  of  SEEDS;  (d)  Partial  enlarged  superpixels  of  SuperPB;  (e)  Partial  (c) Partial enlarged superpixels of SEEDS; (d) Partial enlarged superpixels of SuperPB; (e) Partial enlarged superpixels of SLIC; (f) Partial enlarged superpixels of Turbopixles.  enlarged superpixels of SLIC; (f) Partial enlarged superpixels of Turbopixles.

4.3. Quantitative Comparisons  The performance metrics of superpixel segmentation methods are different from that of image  segmentation  and  other  visual  tasks  [29].  Recall  is  a  professional  index  in  information  retrieval.  Boundary recall (BR) was first adopted to measure the accuracy of image segmentation in [1]. Under  segmentation error (UE) is another metric to measure the error rate of image segmentation. BR and 

Symmetry 2017, 9, 31

11 of 14

4.3. Quantitative Comparisons The performance metrics of superpixel segmentation methods are different from that of image segmentation and other visual tasks [29]. Recall is a professional index in information retrieval. Boundary recall (BR) was first adopted to measure the accuracy of image segmentation in [1]. Under segmentation error (UE) is another metric to measure the error rate of image segmentation. BR and UE are widely used to measure the boundary adherence of superpixels, which can be demonstrated by Symmetry 2017, 9, 31    [3,15,21,22]. BR and UE measure the boundary adherence in boundary-wise and 11 of 14  region-wise situations, respectively. BR is the fraction of ground truth edges that are retrieved by superpixel boundaries within a small neighboring region. The radius of the neighboring region is To represent the image more faithfully, high BR and low UE are usually expected. Given the set of  denoted as boundaries  σ, which is equal or 2 in the set  work. is the fraction superpixel ground superpixel  Resto,  1and  the  of UE ground  truth  of edges  Grd regions across , ,…, ,  truth edge. To represent the image more faithfully, high BR and low UE are usually expected. Given the where , ,…,   refer to the    ground truth edge images, BR can be expressed as  set of superpixel boundaries Res, and the set of ground truth edges Grd = { Grd1 , Grd2 , . . . , Grd M }, | ∩ | 1 where Grd1 , Grd2 , . . . , Grd M refer to the M ground truth edge images, BR can be expressed as

BR

(9)   | | 1 | Res ∩ Grdm | BR = (9) ∑ where  the  operator  ||  denotes  the  size  of  element  M man  | Grdm |in  pixels,    is  5  on  average  in  database  =1 BSDS500. UE can be expressed as  where the operator |·| denotes the size of an element in pixels, M is 5 on average in database BSDS500. UE can be expressed as 1 1 UE | |   (10)   M

    : ∩ ∅  1 M 1 Q    − N UE = Res | | i ∑ N ∑ ∑    is the image pixels,  M   is the number of segments for one ground truth image,  m =1 q=1 i:Res ∩ Q 6=∅

(10)

where    is one  q i segment for superpixel segmentation results.  where N is the image pixels, Q is the number of segments for one ground truth image, Resi is one As the rounding operation is done when calculating the horizontal and vertical intervals, and  segment for superpixel segmentation results. the  initialized  centers  are  updated  as  the  lowest  gradient  position  in  a  corresponding  3 3  local  As the rounding operation is done when calculating the horizontal and vertical and 13  neighborhood,  the  numbers  of  actually  generated  superpixels    are  not  equal intervals, to  .  Figure  the initialized centers are updated as the lowest gradient position in a corresponding 3 × 3 local displays  the  UE  curves  against  .  Obviously,  BSLIC  has 0 lower  UE  than  in  the  other  methods.  neighborhood, the numbers of actually generated superpixels k are not equal to k. Figure 13 displays According to the obtained UE, the boundary adherence can be sorted from high to low as: BSLIC >  the UE curves against k0 . Obviously, BSLIC has lower UE than in the other methods. According to the SEEDS  >  SuperPB  >  SLIC  >  Turbopixels.  Figure  14a,b  displays  the  BR  curves  with  radius  of  obtained UE, the boundary adherence can be sorted from high to low as: BSLIC > SEEDS > SuperPB neighborhood  σ 1, 2  respectively. It is observed that BSLIC produces a uniform increase for BR  > SLIC > Turbopixels. Figure 14a,b displays the BR curves with radius of neighborhood σ = 1, 2 when  compared  SuperPB,  SLIC  and  Turbopixels.  BSLIC  is  a  bit  inferior  to  SEEDS  respectively. It iswith  observed that BSLIC produces a uniform increase forlittle  BR when compared with in  boundary  recall.  However,  BSLIC  has  apparent  advantages  over  SEEDS  in  the  regularity  SuperPB, SLIC and Turbopixels. BSLIC is a little bit inferior to SEEDS in boundary recall. However, and  compactness in size and shape.  BSLIC has apparent advantages over SEEDS in the regularity and compactness in size and shape.

  Figure 13. Under segmentation error. 

Figure 13. Under segmentation error.

  Symmetry 2017, 9, 31

12 of 14

Figure 13. Under segmentation error. 

(a) 

(b)

Figure 14. Boundary recall. (a)  Figure 14. Boundary recall. (a) σ σ = 1; 1; (b)  (b) σ =σ 2. 2. 

BSLIC is a modified SLIC method, and a specific comparison between the two methods are conducted. The experimental data are listed in Tables 1 and 2. Table 1 is the data of BR with σ = 1. The generated number of superpixels k0 is set in the range of [500, 1000, 1500, 2000, 2500], and m is [5, 15, 30]. For boundary recall, Table 1 shows that BSLIC can achieve at most a 6.23%, 8.56% and 12.43% increase when m is 5, 15 and 30. For under segmentation error, Table 2 shows that BSLIC can achieve at most a 1.46%, 2.77% and 3.51% decrease when m is 5, 15 and 30. Table 1. Experimental data of boundary recall (BR). k0

500

1000

1500

2000

2500

m=5

SLIC BSLIC Improving Rate

0.5932 0.5849 −0.0083

0.6346 0.6858 0.0512

0.6245 0.6888 0.0623

0.6791 0.6981 0.0190

0.6782 0.7193 0.0411

m = 15

SLIC BSLIC Improving Rate

0.5646 0.6501 0.0856

0.6035 0.6672 0.0638

0.6504 0.7055 0.0552

0.6798 0.7139 0.0340

0.6957 0.7397 0.0440

m = 30

SLIC BSLIC Improving Rate

0.5014 0.6257 0.1243

0.5500 0.6613 0.1113

0.6046 0.6786 0.0740

0.6470 0.7042 0.0572

0.6793 0.7222 0.0430

Table 2. Experimental data of under segmentation error (UE). k0

500

1000

1500

2000

2500

m=5

SLIC BSLIC Improving Rate

0.2146 0.2348 −0.0202

0.1540 0.1395 0.0146

0.1455 0.1334 0.0121

0.1247 0.1204 0.0043

0.1170 0.1113 0.0057

m = 15

SLIC BSLIC Improving Rate

0.1731 0.1415 0.0277

0.1505 0.1369 0.0136

0.1242 0.1172 0.0071

0.1140 0.1078 0.0062

0.1064 0.1012 0.0051

m = 30

SLIC BSLIC Improving Rate

0.1936 0.1585 0.0351

0.1572 0.1346 0.0227

0.1307 0.1176 0.0130

0.1193 0.1067 0.0126

0.1090 0.1036 0.0053

5. Conclusions This paper presents a superpixel method that achieves an excellent compromise between the boundary adherence and the regularity in size and shape. By initializing cluster centers in a hexagon distribution and initializing the “edge center” as an exact edge pixel, the symmetry properties

Symmetry 2017, 9, 31

13 of 14

of superpixels are evidently reduced, which can effectively improve the boundary adherence of edge-across superpixels. With the boundary term being incorporated into the distance calculation of clustering, the boundary adherence of superpixels generated by BSLIC is further improved. Compared with SLIC, the segmentation quality of BSLIC is increased by at most 12.43% for boundary recall and 3.51% for under segmentation error. Based on local k-means clustering, BSLIC superpixels are more regular in size and more compact in shape than the state-of-art methods. One of our future research projects is to apply the BSLIC method to the two-stage semantic image segmentation models. Acknowledgments: This research is supported by the Fundamental Research Funds for the Central Universities of China (No. JB160409) and the open foundation of State Key Laboratory for Manufacturing Systems Engineering (No. sklms201511). Author Contributions: Hai Wang and Xiongyou Peng conceived and designed the experiments; Xiongyou Peng performed the experiments; Hai Wang and Xiongyou Peng analyzed the data; Yan Liu contributed analysis tools; Xue Xiao wrote the paper. Conflicts of Interest: The authors declare no conflict of interest.

References 1. 2.

3. 4.

5.

6.

7. 8.

9. 10. 11. 12. 13. 14.

15.

Ren, X.; Malik, J. Learning a classification model for segmentation. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), Nice, France, 13–16 October 2003. Veksler, O.; Boykov, Y.; Mehrani, P. Superpixels and Supervoxels in an Energy Optimization Framework. In Proceedings of the European Conference on Computer Vision-Part V (ECCV Part V), Crete, Greece, 5–11 September 2010. Achanta, R.; Shaji, A.; Smith, K.; Lucchi, A.; Fua, P.; Susstrunk, S. SLIC Superpixels Compared to State-of-the-Art Superpixel Methods. IEEE Trans. Pattern Anal. 2012, 34, 2274–2282. [CrossRef] [PubMed] Li, Z.; Wu, X.M.; Chang, S.F. Segmentation using superpixels: A bipartite graph partitioning approach. In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), Providence, RI, USA, 16–21 June 2012. Gonzalo-Martín, C.; Lillo-Saavedra, M.; Menasalvas, E.; Fonseca-Luengo, D.; García-Pedrero, A.; Costumero, R. Local optimal scale in a hierarchical segmentation method for satellite images. J. Intell. Inf. Syst. 2016, 46, 517–529. [CrossRef] Schick, A.; Bauml, M.; Stiefelhagen, R. Improving foreground segmentations with probabilistic superpixel Markov random Fields. In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition Workshop (CVPR Workshop), Providence, RI, USA, 16–21 June 2012. O, N.K.; Kim, C. Unsupervised Texture Segmentation of Natural Scene Images Using Region-based Markov Random Field. J. Signal Process. Syst. 2016, 83, 423–436. [CrossRef] Ayvaci, A.; Soatto, S. Motion segmentation with occlusions on the superpixel graph. In Proceedings of the IEEE International Conference on Computer Vision Workshop (ICCV Workshop), Kyoto, Japan, 27 September–4 October 2009. Lee, T.; Fidler, S.; Levinshtein, A.; Sminchisescu, C.; Dickinson, S. A Framework for Symmetric Part Detection in Cluttered Scenes. Symmetry 2015, 7, 1333–1351. [CrossRef] Fu, K.; Gong, C.; Yang, J.; Zhou, Y.; Gu, I.Y.H. Superpixel based color contrast and color distribution driven salient object detection. Signal Process.-Image 2013, 28, 1448–1463. [CrossRef] Xie, Y.; Lu, H.; Yang, M.H. Bayesian Saliency via Low and Mid-Level Cues. IEEE Trans. Image Process. 2013, 22, 1689–1698. [PubMed] Peng, H.; Bing, L.; Ling, H.; Hu, W.; Xiong, W.; Maybank, S.J. Salient object detection via structured matrix decomposition. IEEE Trans. Pattern Anal. 2016, 99, 796–802. [CrossRef] [PubMed] Feng, J.; Cao, Z.; Pi, Y. Polarimetric Contextual Classification of PolSAR Images Using Sparse Representation and Superpixels. Remote Sens. 2014, 6, 7158–7181. [CrossRef] Garcia-Pedrero, A.; Rodríguez-Esparragón, D.; Gonzalo-Martín, C.; Ibarrola, E.; Lillo-Saavedra, M.; Marcello, J. Automatic identification of shrub vegetation of the Teide National Park. In Proceedings of the International Work Conference on Bio-Inspired Intelligence (IWOBI), San Sebastian, Spain, 10–12 June 2015. Bergh, M.V.D.; Boix, X.; Roig, G.; Capitani, B.; Gool, L.V. Seeds: Superpixels extracted via energy-driven sampling. Int. J. Comput. Vis. 2015, 111, 1–17.

Symmetry 2017, 9, 31

16. 17. 18. 19. 20. 21. 22. 23.

24.

25. 26. 27. 28.

29.

14 of 14

Shi, J.; Malik, J. Normalized Cuts and Image Segmentation. IEEE Trans. Pattern Anal. 2000, 22, 888–905. Pedro, F.; Felzenszwalb, P.F.; Daniel, P.H. Efficient Graph-Based Image Segmentation. Int. J. Comput. Vis. 2004, 59, 167–181. Thomas, H.C.; Charles, E.L.; Ronald, L.R.; Clifford, S. Introduction to Algorithms, 3rd ed.; The MIT Press: Cambridge, MA, USA, 2009; pp. 43–45. Comaniciu, D.; Meer, P. Mean Shift: A robust approach toward feature space analysis. IEEE Trans. Pattern Anal. 2002, 24, 603–619. [CrossRef] Vedaldi, A.; Soatto, S. Quick Shift and Kernel Methods for Mode Seeking. In Proceedings of the European Conference on Computer Vision-Part IV (ECCV Part IV), Marseille, France, 12–18 October 2008. Levinshtein, A.; Stere, A.; Kutulakos, K.N.; Fleet, D.J.; Dickinson, S.J.; Siddiqi, K. TurboPixels: Fast Superpixels Using Geometric Flows. IEEE Trans. Pattern Anal. 2009, 31, 2290–2297. [CrossRef] [PubMed] Zhang, Y.; Hartley, R.; Mashford, J.; Burn, S. Superpixels via Pseudo-Boolean Optimization. In Proceedings of the International Conference on Computer Vision (ICCV), Barcelona, Spain, 6–13 November 2011. Liu, Y.J.; Yu, C.C.; Yu, M.J.; He, Y. Manifold SLIC: A Fast Method to Compute Content-Sensitive Superpixels. In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 27–30 June 2016. Fowlkes, C.; Martin, D.; Malik, J. Learning affinity functions for image segmentation: Combining patch-based and gradient-based approaches. In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), Madison, WI, USA, 16–22 June 2003. Garcia-Pedrero, A.; Gonzalo-Martin, C.; Fonseca-Luengo, D.; Lillo-Saavedra, M. A GEOBIA methodology for fragmented agricultural landscapes. Remote Sens. 2015, 7, 767–787. [CrossRef] The Berkeley Segmentation Dataset and Benchmark. Available online: https://www2.eecs.berkeley.edu/ Research/Projects/CS/vision/bsds/ (accessed on 24 November 2016). Luxburg, U. A tutorial on spectral clustering. Stat. Comput. 2007, 17, 395–416. [CrossRef] Peng, X.; Tang, H.; Zhang, L.; Zhang, Y.; Xiao, S. A Unified Framework for Representation-Based Subspace Clustering of Out-of-Sample and Large-Scale Data. IEEE Trans. Neural Netw. 2016, 27, 2499–2512. [CrossRef] [PubMed] Liu, M.Y.; Tuzel, O.; Ramalingam, S.; Chellappa, R. Entropy rate superpixel segmentation. In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), Colorado Springs, CO, USA, 20–25 June 2011. © 2017 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).