Image Segmentation Using Topological Persistence

Image Segmentation Using Topological Persistence David Letscher and Jason Fritts Saint Louis University Department of Mathematics and Computer Science {letscher, jfritts}@slu.edu

Abstract. This paper presents a new hybrid split-and-merge image segmentation method based on computational geometry and topology using persistent homology. The algorithm uses edge-directed topology to initially split the image into a set of regions based on the Delaunay triangulations of the points in the edge map. Persistent homology is used to generate three types of regions: p-persistent regions, p-transient regions, and d-triangles. The p-persistent regions correspond to core objects in the image, while p-transient regions and d-triangles are smaller regions that may be combined in the merge phase, either with p-persistent regions to refine the core or with other p-transient and d-triangles regions to potentially form new core objects. Performing image segmentation based on topology and persistent homology guarantees several nice properties, and initial results demonstrate high quality image segmentation.

1

Introduction

Image segmentation algorithms classically fall into one of two classes: region-based methods or edge-based methods. Region-based methods typically employ clustering, region growing, or split-and-merge methods to construct regions based on feature characteristics of the region(s). Conversely, edge-based methods use edge detection to find edges within the image, and then use some method to link the edges to form boundaries that define regions. Both classes of segmentation methods have their advantages and disadvantages, so research has also delved into hybrid methods that combine region- and edge-based segmentation [6]. This paper presents a new hybrid approach to image segmentation based on computational topology. The method is fundamentally based on the split-andmerge paradigm, using edge-directed topology to initially split the image into a set of regions, and then using region-based merging to combine select regions into the final segmentation. The algorithm uses a three-step approach: 1) Perform edge detection. 2) Split the image into regions using edge-directed computational topology, based on persistent homology. 3) Merge select regions with similar features in order of topological persistence. W.G. Kropatsch, M. Kampel, and A. Hanbury (Eds.): CAIP 2007, LNCS 4673, pp. 587–595, 2007. c Springer-Verlag Berlin Heidelberg 2007

588

D. Letscher and J. Fritts

More specifically, the algorithm first uses edge-detection to produce an edge map, which serves as the guide for topological region splitting. Given this edge map, edges are linked to form regions based on the theory of persistent homology. The result of this topological splitting is an initial region segmentation that contains three types of regions: p-persistent regions, p-transient regions, and dtriangle regions. The p-persistent regions are larger regions corresponding to core objects in the image. The p-transient and d-triangle regions are smaller regions of the image, which correspond to either additional sections of the existing core objects or sub-sections of new core objects in the image. The p-transient and d-triangle regions are merged, in order of increasing persistence, either with ppersistent regions to further refine the existing core objects or with each other to potentially define new core objects in the final segmentation. The primary contribution of this paper is the novel use of persistent homologybased topology in the second step, and the corresponding order it imposes on region merging in the third step. Any edge detection method could be used in step one, and any feature characteristics could be used for region merging in step three. The second step is similar to the algorithm in [11]. However, the third step adds topological guarantees for the completed segmentation and incorporates region information to improve the quality of the final result. In the remainder of the paper, we discuss the proposed method in greater detail, with emphasis on the details of the topological region splitting and merge ordering in steps two and three, respectively. Section 2 presents the pertinent background in computational topology and persistent homology. Section 3 presents the proposed edge-directed topological segmentation algorithm. Section 4 presents preliminary results for this new algorithm. Section 5 presents related work, and Section 6 concludes the paper and discusses future directions.

2

Background in Topology

Since the key aspect of the proposed image segmentation algorithm is our approach to region splitting and merging based on computational topology, we first present the necessary background on topology and persistent homology. We present these topological concepts as they pertain to images and image segmentation, so we assume the existence of an image I with points in 2 . 2.1

Alpha Complexes and Delaunay Triangulations

Suppose we have a set of points X = {x1 , . . . , xn } in I that have been identified by some edge detection algorithm. For any positive radius α, we can consider the set of all disks of radius α with centers in X (see Fig. 1). As α grows, larger regions of the image are covered and disks begin overlapping. The centers of the overlapping disks can be connected to form edges and triangles. The pattern in which these disks intersect defines a triangulation, with the vertices of the triangulation corresponding to the points in X. An edge in the triangulation is formed when two disks meet in a point that is not in the interior of another disk. Similarly, a triangle is formed when three disks meet in a single

Image Segmentation Using Topological Persistence

589

Fig. 1. α-complexes for α = 1.2, 1.6, and 2.0, and the full Delaunay triangulation that is formed at α ≥ 4.6

point that is not in the interior of another disk. The triangulation constructed in this way is called the Delaunay triangulation and has many nice properties [2]. This triangulations can be found in O(n log n) time in two dimensions [2,5,7]. Figure 1 illustrates the alpha complexes for a set of points with α = 1.2, 1.6, 2.0, and 4.6. As visible in the figure, at α = 1.2, the disks are of radius 1.2 and fourteen pairs of disks intersect to form 14 edges. Some of these edges are connected with other edges, forming four connected components. When the radius α grows to 1.6, more edges are formed, and all of the edges are connected into a single connected component. Additionally, three triplets of disks intersect to form three Delaunay triangles. As the radius further grows to α = 2.0, 5 additional edges and 2 additional triangles are formed. Finally, when α grows to α = 4.6, the full Delaunay triangulation over all the vertices is formed. For a given value of α, the set of vertices, edges, and triangles defines the subcomplex, Kα . This complex accurately represents the topology generated by the union of disks with radius α centered at each vertex [3]. An important characteristic of complexes is that, for two radii, α and α , such that α < α , the subcomplex of α is a subset of the subcomplex of α , i.e. Kα ⊂ Kα . Any sequence of complexes with one a subset of the next is referred to as a filtration. At sufficiently large α, the complex becomes a full triangulation, in which all regions in the graph over X are covered by Delaunay triangles, as demonstrated in Fig. 1 for α ≥ 4.6. This complex is a superset of all other subcomplexes in X. The edges and triangles in Kα piece the points in X into a cohesive union. Recalling that we are defining X as the set of edge points generated from performing edge detection on I, one problem that occurs is that small regions may be left in the complement of the subcomplex. These regions would disappear for slightly larger α. However, increasing α to remove these regions may create new small regions. To counter this problem and effectively deal with small regions in the complement of the complex, we turn to persistent homology. 2.2

Persistent Homology

In algebraic topology, homology provides a way to measure the “complexity” of a space. In the study of complexes inside the plane we are concerned with the first homology group, H1 (K), and its corresponding Betti number, β1 . Fortunately, in this application, β1 can be readily defined without in-depth knowledge of algebraic topology. For a connected graph where V and E are the number of vertices and edges in a graph, respectively, the Betti number is defined as β1 = E − V + 1. If

590


we have a triangulated subcomplex in the plane, β1 = C − V + E − F , where C is the number of connected components and F the number of faces (or triangles) in the complex. The full definitions are more complex (see [9] for a full treatment), but in our situation the theory reduces to the formulae above. As mentioned in the previous section, as α increases, the resulting sequence of subcomplexes is known as a filtration. As the triangulations change, the Betti number, β1 (Kα ), for each subcomplex Kα in the filtration will change with α. As α increases, β1 will decrease when a hole is filled in (i.e. a triangle is created), and may increase when a new edge is created (see Fig. 2). In other words, when a hole is filled, F increases by one. Conversely, when an edge forms, the number of edges E increases by one, and the number of connected components C may decrease by one (if the edge connects two previously separate components). Related to this, persistence refers to how long regions in the complement of Kα last until they are contained in Kα+p for some p > 0. The p-persistent Betti number of Kα , β1p (Kα ), is β1 (Kα ) minus the number of regions in the complement of Kα that are contained in Kα+p . In spirit, it is the Betti number of the complex obtained from Kα with “small” holes filled in. For formal definitions in a more general setting, and algorithms for their calculation, see [4]. Consider two subcomplexes in a filtration, Kα and Kα+p , where p > 0. Recall that this means that Kα ⊂ Kα+p , so Kα+p contains the same vertices, edges, and triangles as Kα , but may also contain new edges and triangles. Holes in Kα that are still holes in Kα+p are regions that are persistent in p. We shall refer to these regions as p-persistent regions. Holes in Kα that are filled in between Kα and Kα+p are regions that are not persistent in p. We shall refer to these as p-transient regions. And finally, the individual triangles that were already filled in Kα will be referred to as d-triangles. For subcomplexes of the plane, loops that do not persist bound regions that disappear completely in Kα+p . An equivalent definition to the more general one, is β1p (Kα ) = C − V + E − F − R, where R is the number of regions in the complement of Kα that are contained in Kα+p . Note that this is not necessarily true in higher dimensions.

3

Segmentation Algorithm

The proposed segmentation algorithm is a hybrid split-and-merge segmentation algorithm that employs computational topology and persistent homology for image splitting and region feature characteristics for region merging. The algorithm first employs some edge detection algorithm, such as the Canny algorithm [1] or the wavelet-based detector that we used [8], to determine the set of edge points, X, in the image. Topological splitting is the performed over X to generate an initial segmentation with three types of regions: p-persistent regions, p-transient regions, and d-triangles. The topological splitting is controlled by two parameters, α and p, where α defines the radius of the disks and p indicates the persistence. In the final step, the algorithm merges the p-transient and


591

β1 8 4 0 0

C V E F

= = = =

1

C V E F

4 18 14 0

β1 = 0 β

β

1 p

1 18 22 0

C V E F

β1 = 5

p 1 0

= = = =

2

2

β

1 p

1 18 27 5

C V E F

β1 = 5

p 1 0

= = = =

3

2

β

1 p

1 18 35 10

C V E F

β1 = 8

p 1 0

= = = =

4

2

β

1 p

1 18 37 13

C V E F

β1 = 7

p 1 0

= = = =

5 α

2

β

1 p

1 18 38 18

β1 = 3

p 1 0

= = = =

2

p 1 0

1 p

2

Fig. 2. This figure demonstrates various complexes in a filtration, including their Betti numbers and persistent Betti numbers for each Kα and Kα+p

d-triangle regions with either the p-persistent regions or each other to generate the final segmentation. As discussed in the previous section, the p-persistent regions represent larger regions that remain (are not filled in) in Kα+p . Conversely, p-transient are smaller regions that are filled in between Kα and Kα+p , while d-triangles are the smallest regions, which already existed as triangles in Kα . The d-triangles are akin to “thickened” edges separating regions in the initial segmentation. The p-persistent regions are crucial in that they define the core objects in the initial segmentation. There are two reasons to believe these regions are important to the segmentation: they are completely surrounded by edges, and they all contain a disk of radius α + p in their interiors. These initial regions can then be expanded upon to create a full segmentation. We perform this process in such a way that the persistent homology is respected. One approach would be to use the simplification algorithms presented in [4] to expand these regions and preserve this homological information. However, this method uses only the edge information, essentially expanding the regions so that the region boundaries consist of the shortest possible edges. This approach has the significant drawback that it does not incorporate any region information. An alternative strategy for expanding the persistent regions to a full segmentation is to merge neighboring regions based on their feature characteristics. In our case, we simply used the average color value for each region, but any desired feature vector could be used. Homological persistence dictates the merge order of the regions, with regions being ordered according to the disk radius α for which the triangle was formed, starting from the smallest. In order to preserve the topology of the segmentation, regions will only be merged if the persistent Betti number remains unchanged.

592


Algorithm 1. Algorithm for edge-directed topological image segmentation 1: 2: 3: 4: 5: 6: 7: 8:

4

Identify set of edges, X, in the image. Find the Delaunay triangulation of X. Calculate β1p (Kα ) and identify regions in 2 − Kα as p-persistent, p-transient, or d-triangle regions. Let F be the set of regions ordered by increasing α values. while (F is not empty) do Let σ be the first face in F . And let F = F − {σ}. Merge σ with the most similar adjacent region provided that β1p (merged regions) = β1p (Kα ) end while

Results

Figures 3 and 4 demonstrate the results of the algorithm for a sample image, both over multiple stages of the process, and over multiple separate images, respectively. As evident, each of the resulting segmentations effectively distinguishes the critical objects in the image. Figure 5 demonstrates how the choice of α and p affects the resulting segmentation. For small values of α large regions are not separated from each other in the initial stages of the algorithm. For fixed α larger values of p remove small regions from consideration.

Fig. 3. Major steps in the algorithm: (a) original image (b) complement of the α complex for α = 4, p = 0. (c) the persistent regions at α = 4, p = 3 (d) final segmentation.

Fig. 4. Example segmentations a stop sign, a rose and a baseball player

Image Segmentation Using Topological Persistence p=0 23 Regions

p=2 22 Regions

p=4 13 Regions

47 Regions

34 Regions

21 Regions

43 Regions

28 Regions

22 Regions

593

α=2

α=4

α=6

Fig. 5. Affect on the choice of parameters of the resulting segmentation

The segmentation algorithm has several nice guarantees. In particular, “large” regions in the complement of the edges will not be broken into separate regions. Specifically, any ball of radius p in the complement of the edges will not be divided between multiple segments. The choice of α will affect how well the points identified by the edge detection fill out to complete the boundary of regions. In particular, if the points identified as edges have neighbors with a distance of α/2 in each direction then boundaries of regions will be correctly reconstructed.

5

Related Work

An early image segmentation method using Delaunay triangulations was presented in [6]. Their method is distinct in that they do not use persistent homology, but instead start with a full Delaunay triangulation and generate the core objects in the final segmentation solely through region merging. A more recent image segmentation method based on Delaunay triangulation is presented in [11]. They use similar criteria for doing their initial segmentation. However, whereas we use Delaunay triangulation for defining different types of regions and then perform region merging to generate the final segmentation, they use Delaunay triangulation for defining boundaries, and use thinning to eliminate undesired edges in the final segmentation.

594


Another recent image segmentation method based on Delaunay triangulation is presented in [10]. They generate the full Delaunay triangulation first, followed by region merging. They merge regions by extracting a skeleton of the segmented image from the connected components graph of the triangulations. Each separate connected component in the skeleton defines one region in the final segmentation.

6

Conclusion and Future Directions

We have presented an image segmentation algorithm that uses both edge and region information and uses techniques from computational topology. The segmentations obtained clearly delineate key objects in the images and are good initial indicators of the effectiveness of the method. There are several directions of followup for this algorithm. These include studying different orderings for merging the d-triangles and p-transient regions, and using alternate feature characteristics for region merging (as opposed to just average region color value). Significant improvement could also potentially be achieved in using machine learning methods to help identify appropriate parameterizations of α and p according to the characteristics of the image and edge features. Another improvement is to utilize more information from the edge detection algorithms. The current algorithm only uses the magnitude of the estimated gradient. The direction of the gradient can also be incorporated in finding an anisotropic Delaunay triangulation. These triangulations can better represent the regions divided by the edge information.

References 1. Canny, J.: A Computational Approach To Edge Detection. IEEE Trans. Pattern Analysis and Machine Intelligence 8, 679–714 (1986) 2. Edelsbrunner, H.: Algorithms in Combinatorial Geometry. Springer, New York (1987) 3. Edelsbrunner, H.: The Union of Balls and its Dual Shape. In: Proceedings of the Ninth Annual Symposium on Computational Geometry, pp. 218–231 (1993) 4. Edelsbrunner, H., Letscher, D., Zomorodian, A.: Topological Persistence and Simplification. Discrete and Computational Geometry 28, 511–533 (2002) 5. Fortune, S.: A Sweepline Algorithm for Voronoi Diagrams. Algorithmica 2, 153–174 (1987) 6. Gevers, T., Smeulders, A.W.M.: Combining Region Splitting and Edge Detection through Guided Delaunay Image Subdivision. In: Proc. of the 1997 International Conference on Computer Vision and Pattern Recognition, pp. 1021–1026 (1997) 7. Guibas, L., Knuth, D., Sharir, M.: Randomized Incremental Construction of Delaunay and Voronoi Diagrams. Algorithmica 7, 381–413 (1992) 8. Mallat, S., Zhong, S.: Characterization of signals from Multiscale Edges. IEEE Trans. Patt. Anal. and Mach. Intell. 14, 710–732 (1992) 9. Massey, W.: A Basic Course in Algebraic Topology. Springer, Heidelberg (1991)


595

10. Prasad, L., Skourikhine, A.N.: Vectorized Image Segmentation via Trixel Agglomeration. In: Brun, L., Vento, M. (eds.) GbRPR 2005. LNCS, vol. 3434, pp. 12–22. Springer, Heidelberg (2005) 11. Stelldinger, P., Ullrich, K., Meine, H.: Topologically Correct Image Segmentation Using Alpha Shapes. In: Kuba, A., Ny´ ul, L.G., Pal´ agyi, K. (eds.) DGCI 2006. LNCS, vol. 4245, pp. 542–554. Springer, Heidelberg (2006)