Binary Image Skeleton Representation by Compound ...

Binary Image Skeleton Representation by Compound Bezier Curves Leonid Mestetskiy Department of Computational Mathematics and Cybernetics Moscow State University, Russia

1

Introduction: Skeleton of Binary Image vs Binary Image of Skeleton

Skeleton (or medial axis representation) is a powerful and widely used tool for image shape analysis. Methods based on the computation of object skeletons are commonly used in pattern recognition and images shape classification. Skeleton or middle axis of the object is an important descriptor of the image shape. Skeletons are used to feature generation for determine the similarity measures of various shapes in the construction of classifiers. Originally, the concept of skeleton was denoted for continuous objects: the skeleton of a closed region in Euclidean plane is a locus of centers of maximum empty circles in this region. The circle is considered to be empty if all its internal points are internal points of the region. The concept of the skeleton (the middle set of points) was introduced and investigated by Blum (Blum, 1967). He gave a geometric definition of the skeleton of two-dimensional object, investigated its geometric properties, and demonstrated its usefulness for shape description and shape analysis. However, this concept became the most popular in shape analysis and classification of discrete bitmap digital images. Therefore, the need to generalize the concept of the skeleton for discrete images was raised. In principle, we can formulate two approaches to extend the concept of the skeleton to discrete images. The first approach, which is the most popular because of ease of implementation, will be called discrete. It consists in a morphological transformation of the original image (Figure 1a) and construction new image (Figure 1b), which can be regarded as a skeleton. In this new bitmap medial axis represented by discrete lines one pixel wide. We can say that the resulting image is a digital image of the skeleton. The discrete approach is implemented in different ways: based on distance maps, thinning, Voronoi diagrams of boundary points (Siddiqi & Pizer, 2008; Costa & Cesar, 2001). Another approach, which we call continuous, is based on the approximation of a discrete object by the geometrical figure in terms of a continuous geometry (Figure 1c) and the construction of the skeleton for this figure (Figure 1d). The resulting skeleton is considered as a continuous skeleton of discrete objects. Both approaches have their advantages and disadvantages. The main advantage of the discrete approach is the simplicity of the algorithm and a graphic visualization of the skeleton in the source raster format (Figure 1а,b). A continuous approach has its advantages. The main advantage of the approach is the continuous medial representation of the object's shape (Siddiqi & Pizer, 2008) as a geometrical graph with a radial function, which determines the width of the object. Radial function sets at each point of the geometrical graph radius of the inscribed circle centered at this point. Continuous medial representation allows the use of graph theory and computational geometry algorithms for image shape analysis and recognition.

It is well known that the topological structure of the skeleton is very sensitive to any changes in the boundaries of shape, in particular, to noise in the image. Therefore, all skeletonization algorithms, both discrete and continuous need special post-processing (pruning) to eliminate the effects of noise in the resulting skeleton. Pruning discrete skeleton is based on heuristic criteria. But pruning of a continuous skeleton is based on a rigorous formal criteria and methods. Another advantage of the continuous approach is its higher computational efficiency. Experiments show that the computation of the continuous skeleton is significantly lower than the discrete skeleton, even taking into account the fact that the discrete approach is more suited to parallel computing. Comparative analysis shows the advantages of continuous skeleton compared to discrete. These advantages are mathematical rigor, information content and computational efficiency. But to realize these benefits, we need a continuous model of a binary image, which allows us to apply well-developed ideas of computational geometry to skeletonization. This article focuses on the development and use of this model. We offer essentially the first complete solution of this problem, implemented and tested in practice. This paper describes the full implementation of the continuous approach to the skeleton construction for binary raster images of any complexity. We propose an original method for continuous skeleton representation as a planar graph whose edges are segments of straight lines and quadratic parabolas. In conclusion, the theoretical and experimental estimates of the computational complexity of the developed approach are presented. The main problem of the continuous approach is the complexity of its implementation.

(a)

(b)

(c)

(d)

Figure 1: (a) – the discrete binary image, (b) – the discrete skeleton, (c) – the continuous binary image, (d) – the continuous skeleton and inscribed circles.

This chapter of the book is based on the studies described in papers and books (Mestetskiy, 2006, 2007, 2008, 2009, 2010). The author thanks the Russian Foundation for Basic Research for the support on this study.

2

Concept of Continuous Skeleton of Binary Image

The aim of the concept of continuous skeleton for raster binary image is to use the correct and elegant model of Voronoi diagram of line segments and polygonal figures to obtain the skeleton of a discrete object. Such application should ensure that, firstly, the correct definition of the skeleton for this object, and secondly, to enable highly efficient use of computational geometry algorithms for obtaining the skeleton. The proposed idea consists of three parts (Figure 2): - Approximation of binary images (Figure 2a) by polygonal figures (Figure 2b); - Construction of Voronoi diagram of obtained set of figures by methods of computational geometry (Figure 2c); - Obtaining the skeleton from the Voronoi diagram in a convenient format for further analysis (Figure 2d).

(a)

(b)

(c)

(d)

Figure 2: (a) – source binary image, (b) – polygonal approximation, (c) – medial representation of the polygon, (d) – resultant skeleton after pruning.

The choice of a polygonal figure for the approximation is explained by the fact that the skeleton of a polygonal figure is a fairly simple structure and can be obtained from the Voronoi diagram of this figure. There are well developed methods of computational geometry that can be used for the construction of such a diagram. On the other hand, using a polygonal approximation raises the question of removing "noisy" branches of the skeleton. The implementation of this approach required solving of several complex problems. 1. The correct approximation of the binary bitmap by polygonal figures. By correctness we mean the construction of such figures, whose boundaries are described by simple polygons that do not intersect each other and have no self-intersections. This problem is solved by the algorithm described in Section 3. 2. Obtaining Voronoi diagrams for polygonal figures. It is possible to use known algorithms for this purpose. However, these algorithms work effectively with a simply-connected figures (without holes). But for complex scenes consisting of a large number of multiply connected elements, they require too much time for computation. Our proposed algorithm is effective for just such scenes. It is based on the construction of the dual graph of Voronoi diagram, the so-called Delaunay graph. This algorithm is described in Section 4, paragraphs 4.1-4.3. 3. Regularization of the skeleton based on the pruning. Regularization is necessary because the approximation of the bitmap image by polygonal figures with a given accuracy is not unique. But the skeleton that we want should be invariant to this approximation. Therefore, we must distinguish a normal part of the Voronoi diagram of a polygonal shape, which in some form is present in any variant of approximation. This problem occurs in both discrete and continuous approach. It is solved by cutting of some skeleton branches. A continuous approach allows us to perform this cutting on the basis of a

mathematically rigorous formal criterion. The resulting subgraph of the Voronoi diagram is called the skeleton base. The principles of its construction are described in Section 4.4. 4. The transformation of the skeleton in the format of the compound Bezier curve. The skeleton of a polygonal figure is a geometrical graph whose edges are described by straight line segments and quadratic parabola. The data structure describing a graph is very complicated. The proposed method for representing parabolic segments of the skeleton graph by quadratic Bezier curves allows us to simplify this data structure. As a result, the skeleton is described by means of a planar rectilinear graph. This solution is described in Section 5. The general scheme of the algorithm that implements the approach described is shown in Figure 3. Binary bitmap image

Approximation of the binary bitmap by polygonal figures Search of boundary corridors

Tracing of boundary corridors

Polygonal figure skeletonization

Constructing of the boundary adjacency tree Constructing of the Delaunay graph

Regularization and formatting of the skeleton Pruning of the polygonal figure skeleton Constructing of the Compound Bezier Curve

Constructing of the Voronoi diagram

Constructing of minimal perimeter polygons

Continuous skeleton of binary image

Figure 3: The structure of the algorithm for constructing a continuous skeleton of binary image bitmap.

3

Continuous Representation of Raster Image Boundary

3.1

Discrete Figures in Binary Image

A binary image is a two-colored picture where one or several objects of one color are located against a background, which has another color. Such image is represented in computer as a matrix of points (pixels). Each element of this matrix can have only one of two colors. Without loss of generality, we will consider a binary image as a black-and-white image: object points are black, and background points are white. Let’s define an adjacency structure on a set of pixels as follows. For a black pair of pixels we will define neighborhood as 8-adjacency, and for a white pair and multi-colored pair - as 4-adjacency. A set of one-colored pixels is called connected if for each pair of pixels in it there is a path from one pixel to another, consisting of sequentially neighboring pixels of the same color. Maximal connected set of

pixels of one color is called a connected component. If all pixels of component lay on the same straight line, such component is called degenerated. Let’s define discrete figures as connected black-colored components. There are 5 connected components in the image in Figure 4a, two of them are discrete figures. 3.2

The Continuous Approximation of Discrete Figure

Let's consider pixels as points with integer co-ordinates in Euclidean plane. We will call a pair of 4-adjacent multi-colored points a boundary pair, and a segment connecting these points - a boundary segment. Two components, to which points of a boundary pair belong, are called adjacent, and the boundary pair is called dividing for these components. A set of all dividing boundary pairs for two adjacent components we will call a boundary corridor (Figure 4b). Each discrete figure defines one or more boundary corridors. Let's say, that a closed path lies in a boundary corridor if it crosses once each boundary segment of this corridor. A path crosses a segment if it has a common point with it and lies on different sides from this segment in some neighborhood of cross point. There exists the minimal length path among all paths lying in a boundary corridor. This path is a closed polyline. We will call it a separating minimal perimeter polygon (MPP). If a discrete figure and all its holes are not degenerated then all its MPP are simple polygons (Figure 4c). For degenerated figure or degenerated hole MPP degenerates in a line segment. The set of all MPP of a discrete figure defines a polygonal figure - «a polygon with polygonal holes».

(a)

(b)

(c)

Figure 4: (a) – the binary image with 5 components and 2 discrete regions, (b) – boundary corridors, (c) – minimal perimeter polygons.

Thus, we have defined minimal perimeter polygonal figures that approximate discrete figures of binary image. Note that the set of approximating polygonal figures always exists and is unique for the given binary image. 3.3

Boundary Corridor

The construction of a boundary corridor includes two steps: search of a corridor and its tracing. Corridor search is understood as a problem of finding one boundary pair of points (Figure 5a). Search of such pair can be executed by row scanning of the binary image. After finding the boundary pair, the boundary tracing algorithm will work. This algorithm reveals all other boundary pairs of a corridor.

(a) (b) (c) Figure 5: Corridor tracing: (a) the initial position of tracer pair, (b) the consecutive positions of tracer pair, (c) the sequence of test points (tracing track).

Contour tracing begins with the first boundary pair and finds sequentially other boundary pairs of a corridor. Current found boundary pair we will call a tracer pair. Tracing process consists in consecutive moving of the white end of a tracer pair in a positive direction relative to the black end (Figure 6). The derived point is called a test point. All possible choices of test point for different positions of the tracer pair are presented in Figure 5. Current position of the tracer pair is shown by a solid line, and new possible positions depending on a color of a test point - by a dotted line.

Figure 6: Choice of the next test point (labeled as square) for different positions of tracer pair (solid line) during tracing process of boundary corridor.

A new position of a tracer pair depends on a color of a test point and is determined by the following rule. The test point replaces in a point of the same color in the tracer pair. Consecutive moving of a tracer pair allows allocating all boundary points corresponding to one boundary contour (Figure 5b). Tracing process ends when tracer pair returns to its initial position. After ending of corridor tracing the search of the next corridor will be continued from that location where the first boundary pair of the previous corridor is found. Process ends, when the image row line scanning is completed.

3.4

Minimal Perimeter Polygon

The sequence of test points forms an ordered list called a tracing track (Figure 5c). By sequentially connecting all black points of this and all white points separately, we will have "walls" of a boundary corridor. The left wall consists of black points, and the right one - of white points. The minimal perimeter polygon lies between corridor walls. All vertices of MPP are points of a tracing track. We will call such point a corner. The problem of MPP construction consists in choice of corner points from a tracing track.

(a)

(b)

(c)

(d)

(e)

(f)

Figure 7: (a,b) correction of coverage sector, (c,d) new angular point forming, (e,f) coverage sector doesn’t change.

The first corner point is defined from the initial position of the tracer pair (Figure 5a). It is obvious, that the right point of this pair is always a corner. Note that two consecutive vertices in MPP should be connected by line segment lying between corridor walls completely. It means, that if the next (in particular, the first) corner point is found it is necessary to search for the next corner point as a point laying from it «in line of sight» inside a corridor. Let's define a concept of «coverage sector» for a corner point. At the initial moment (for the current found corner point) it is equal to 360 ° and isn’t limited at all. Then points of a track after this corner point are sequentially considered. The coverage sector is corrected by the following rules (Figure 7): 1. If the test point is located inside the coverage sector then the sector changes (Figure 7a,b). If the test point is black (Figure 7a), it is declared as the left side of sector, if it is white (Figure 7b) – as the right one. 2. If the white point is located outside the coverage sector to the left of its left side (Figure 7c) then the left black point of the sector is declared as the new corner point. Similarly, if the black point is located outside the sector to the right of its right side then the right white point declared as the new corner point (Figure 7d). 3. In all other cases (Figure 7e,f) the coverage sector doesn’t change. For the reason of these rules all corner points will be sequentially found. The set of these corner points is considered as a circular list of points. The process ends, when the initial corner point will be chosen as a corner point (but not as a test point!). The Figure 8 illustrates the process of constructing of MPP for the Figure 5. Figure 8a shows the first corner point (marked by square), and shows part of the tracing track. Figure 8b presents sequential steps of coverage sector correction, from starting position (left image), until the next corner is obtained (right image). And Figure 8c shows final obtained MPP.

4

Polygonal Figure Skeleton

4.1

Voronoi Diagram and Delaunay Graph of Polygonal Figure

Universal analytical description of the skeleton for any class of closed domain Ω in R 2 cannot be obtained. Nevertheless, for some definite classes of domains (in particular, for domains with boundary consisting of finite number of polygons) an analytical description exists.

(a)

(c)

(b) Figure 8: Find the next corner point of the minimal perimeter polygon: (a) the initial corner point, (b) obtain the next corner point by sequential steps of the correction of the coverage sector, (c) the minimal perimeter polygon.

Let P ∈ R 2 be a multiply-connected polygonal domain and S (P ) be a set of all sites of P ( S ( P ) consists of all vertices and sides of P called vertex-sites and segment-sites, respectively). The medial axis M ( P ) is a subset of Voronoi diagram VD ( P ) of the site set of P . This is the feature most algorithms computing medial axis of a polygonal figure rely on. The concept of a Voronoi diagram of line segments is commonly used for skeleton construction of a polygonal figure (Drysdale & Lee, 1978; Kirkpatrick, 1979). The polygonal figure boundary is a union of linear segments and vertices, which are considered as the Voronoi sites. The Voronoi diagram of these sites is generated and the skeleton is extracted as a subset of the diagram. The skeleton of a polygonal figure with n sides can be obtained from the Voronoi diagram taking O ( n) time. By-turn, there are known effective O ( n log n) algorithms to construct the Voronoi diagram for the general set of linear segments (Fortune, 1987; Yap, 1987) as well as for the sides of a simple polygon (Lee, 1982) or multiplyconnected polygonal figures (Mestetskiy & Semenov, 2008). Degenerated disks of zero radiuses centered in the convex vertices of polygonal figure are empty as they have no internal points and, therefore, don’t contain boundary points of a figure. Besides, they are the maximal empty disks as they don’t contain other empty disks. So, points, which coincide with convex vertices of a polygonal figure, belong to a polygonal figure skeleton. A polygonal figure skeleton looks like a planar graph with edges consisting of line segments and parabolas. Skeleton vertices are convex vertices of a polygonal figure (one degree vertices) and also points - centers of inscribed circles, tangent to figure boundary in three or more points (three and more degree vertices). The radial function is defined in each skeleton point as the radius of inscribed circle centered in this point It is especially necessary to note, that the polygonal figure skeleton always exists and is unique. 4.2

Construction of Skeleton based on Delaunay Graph

The main idea of the algorithm is based on constructing the Delaunay graph of figure sites and an adjacency tree of figure boundary contours. Let P be a multiply-connected polygonal domain.

8

8

9

9

0

0 7

12 6

13

13 5

11 1

7

12

5

14

1

4

14

15 10

10

3 2

6

11

4

15 3

2

(a)

(b)

Figure 9: The Voronoi diagram (a) and the Delaunay graph (b) of a polygonal domain site set (circles denote vertex-sites and squares denote segment-sites).

Two sites of P are adjacent if they are adjoining to each other (vertex-site and segment-site of a common side) or there exists a disk inscribed in P and touching both sites (in other words, if their Voronoi cells have non-empty intersection). The Delaunay graph DG ( P ) of P is a graph ( S , E S ) where S is a set of sites of P and E S ⊆ S × S contains all pairs of adjacent sites from S . Similarly, the Delaunay graph of a subset S ′ of S is defined as a graph ( S ′, E S ′ ) where S ′ ⊆ S and E S ′ ⊆ S ′ × S ′ contains all pairs of adjacent sites from S ′ . The Delaunay graph of P is a dual structure for the Voronoi diagram of P . In Figure 9 polygonal figure is presented with its Voronoi diagram and Delaunay graph constructed; the sites are enumerated from 0 up to 15. The duality of DG ( P ) and VD ( P ) is that there exists a bijection between their structure 8

9

8

9 0

0

7

7 6

5 1

1

8

9

0

7 6 1

7 6

5 4

1

(d)

5 4

3

2

(c) 8

9

0

3

2

(b)

5 4

4 3

2

(a)

6

2

3

(e)

Figure 10: Constructing the Delaunay graph of a simple polygon: (a): a source polygon; (b): an initial sequence of elementary chains; (c)–(e): iterative merging Delaunay graphs.

elements: every VD ( P ) vertex with its incident edges and incident faces is assigned to DG ( P ) face with its incident edges and incident vertices. Given DG ( P ) constructed, VD ( P ) can be obtained in O ( n) time. In (Karavelas, 2006) the algorithm to transform Delaunay graph to the corresponding Voronoi diagram is described. Actually, under the non-degeneracy assumption that no point in the plane is equidistant to more than three sites of P , the Delaunay graph is the triangulation of a set of the sites of P (possibly, with non-straight edges). The algorithm to compute the Delaunay graph of a simple polygon involves the following steps: 1) Generating initial elementary chains of sites and constructing their Delaunay graphs (Figure 10b); 2) Iterative pairwise merge of Delaunay graphs of chains. At each iteration the following operations are performed (Figure 10c–e): • Clearing Delaunay graphs in every pair so that the Delaunay condition is satisfied for every edge relative to the union of sites from both graphs in the pair. At this stage some edges can be eliminated in every pair of Delaunay graphs (Figure 11a–c); • “Sewing” Delaunay graphs in every pair so that the Delaunay condition is satisfied for every new inserted edge relative to the union of sites from both graphs in the pair. At this stage every pair of Delaunay graphs is “sewed” and new Delaunay graphs are formed (Figure 11b,d). 8

9

8

9

0

0 7 6

7 6

5

1

4 3

2

4 3

2

(a)

C013

(b)

8

9

8

9

0 7 6 1

0

7 5

6 1 2

7 6

5 4

3

(c)

8

9

0

4 2

5

1

C356

1

4 3

3

(d)

5

2

(e)

Figure 11: Merging two Delaunay graphs: (a) Delaunay graphs; (b) new edges (3,4) and (3,5) are already inserted within “sewing” procedure; (c) eliminating the face (0,1,3) for which the Delaunay condition is broken; (d) the edges (1,5) and (3,6) are candidates to be inserted but for sites (1,3,5) inscribed circle doesn’t exist so (3,6) is inserted; (e) Delaunay condition for sites (0,1,3) is broken (circle C013) and for sites (3,5,6) is satisfied (circle C356).

The computational complexity of the algorithm is O ( n log n) . This algorithm is similar to one proposed in (Lee & Schachter, 1980) for merging Delaunay triangulations. 4.3

Skeleton of Multiply-Connected Polygonal Figure

Fast algorithms for constructing skeleton of simple polygon with n vertices through Voronoi diagram have computational complexity O ( n log n) in worst case (Lee, 1982). Known generalizations to the case of a polygonal figure with holes (Srinivasan et al., 1992) have computational complexity O ( kn + n log n) , where k - quantity of polygonal holes, n - general number of vertices. For some problems such computational complexity leads to too big expenses of time. For example, in the problem of construction of an external skeleton for segmentation of the text document image (Mestetskiy, 2006) values k and n have an order 103 and 105 accordingly. At the same time, efficient algorithms for Voronoi diagram construction of linear segment set (Fortune, 1987; Yap, 1987) don’t use specific features of segment set of polygonal figure boundary owing to their universality. In particular, these algorithms build Voronoi partitioning not only inside, but also outside of a polygonal figure and this is superfluous work. Our solution is based on concept definition for adjacency of polygonal figure boundary contours and on construction of so-called adjacency tree of these contours. 11 10 9 0

8

1 2

6

4

3

5

7

12

(a) 1

(b)

11 10

2

3

9

0 7

4

8

12 5

6

(c)

(d)

Figure 12: Figure boundary adjacency tree construction: (a) the polygonal figure, (b) intercontour circles, (c) the boundary adjacency graph with extracted boundary adjacency tree (solid), (d) transforming of the figure to the polygon.

Two boundary polygons are adjacent if the circle inscribed into a figure and contacted both of these polygons exists. The given relation of contour adjacency defines a graph of contour adjacency. It is obvious, that this graph is connected. Each spanning set of it (the minimal connected spanning subgraph) is a tree. Such tree we call a polygonal figure boundary adjacency tree. In Figure 12a the image with 12 boundary contours is presented. Inscribed circles, concerning pairs of contours, show the adjacency relation (Figure12b). In Figure 12с the polygonal figure boundary adjacency is shown, and one of the boundary adjacency trees is presented by solid lines.

The boundary adjacency tree gives the chance to reduce a problem of a polygonal figure skeletonization to a problem of a simple polygon skeletonization. For this purpose let’s transform chains of side of polygons by «cutting-in» them into one another. As a result the polygonal figure conditionally transforms to "polygon" (Figure 12d). In details this process was described in (Mestetskiy, 2006). O (n log n) sweepline algorithm for finding boundary adjacency tree and figure skeleton construction on its basis is described ibidem. 4.4

Polygonal Figure Skeletal Base

The problem of “noisy” branches exists for continuous and for discrete skeletons. Small irregularities in figure boundary lead to occurrence of skeleton branches, unessential for analysis of image form. A problem of skeleton regularization consists in removing these branches and leaving only fundamental part of a skeleton which characterizes properties of the form. This fundamental part looks like a skeleton subgraph. We will call it a skeletal base. Transformation of a skeleton to a skeletal base consists in removing of unessential vertices and edges, this process is called pruning. Let C be a polygonal figure with the boundary ∂C, the skeleton S and the skeleton radial function ρ (s), s∈S. The skeleton is a planar graph S = ( P, E ) with the set of vertices P and edges E. A skeleton vertex with one incident edge is terminal, and with two or more edges – internal. An edge incident to terminal vertex is also called terminal; an edge incident to two internal vertices is called linking. Linking edges can enter in one or more cycles and in this case they are called cyclic. Pruning is a consecutive removal of some terminal vertices and skeleton edges incidental to them. In process of pruning degree of some vertices changes in particular, internal vertex can become terminal, or its degree can become equal to 2. Pruning guarantees preservation of a skeleton connectivity and also preservation of all cycles in a skeleton as it doesn’t touch cyclic edges. Let’s consider an assessment criterion of “essentiality” of a terminal edge. Essential edges remain in a skeletel base, and unessential edges are cut.

(a)

(b)

(c)

(d)

Figure 13: Skeletal subgraph silhouette construction: (a) the initial image, (b) skeleton, (c) truncated subgraph of skeleton, (d) the silhouette of the subgraph.

Let S ′ = ( P ′, E ′) be some connected subgraph of skeleton S = ( P, E ) , such that P ′ ⊆ P , E ′ ⊆ E and that there are no cyclic edges of skeleton S among edges from the set E \ E ′ (Figure 13). It means, that graph S ′ can be derived from skeleton S by removing (“pruning”) of a part of vertices and edges of skeleton, and such removal doesn’t destroy cycles and doesn’t break connectivity of the graph. Such graph S ′ we will call truncated subgraph of S . We will consider the set of points formed by union of all inscribed circles centered in points of truncated subgraph S ′ , which radiuses are equal ρ (s), s∈ S ′ . This set of points forms the closed region which we will call a silhouette of subgraph S ′ . The important

property of a truncated subgraph silhouette is the topological equivalence to figure C. In particular, a silhouette is a connected set. Let a skeletal base of figure C be the minimal truncated subgraph S ′ of its skeleton S with silhouette VS′ satisfying a condition H (C , VS ′ ) ≤ ε , where ε > 0 is regularizing parameter, and H (C , VS ′ ) Hausdorff distance between figure C and silhouette VS ′ . It is necessary to note, that for each value of parameter ε the skeletal base always exists and is unique.

(a)

(b)

(c)

(d)

(e)

Figure 14: Skeletal base construction: (a) the initial image, (b) the polygonal figure and its skeleton, (c,d,e) the skeleton subgraphs and their silhouettes.

We will call the derived skeletal base as a continuous skeleton of a discrete figure. It is possible to present the process of a skeletal base construction as sequence of the truncated subgraphs of skeleton {S m′ }. Here S 0′ = S , S m′ +1 ⊂ S m′ , m=0,…,M, and for all S m′ the following condition ′ is the required skeletal base. is satisfied: H (C , VS m′ ) ≤ ε . The last element of this sequence S M ′ condition According to our definition of a skeletal base, for each truncated subgraph S ′ ⊂ S M H (C , VS ′ ) > ε takes place or there are no terminal edges in S M′ . The described process is illustrated by an example in Figure 14. Here ε = 2 . Computational complexity of this algorithm depends linearly on a number of skeleton vertices, i.e. it is at worst O (n) , where n - quantity of polygonal figure vertices.

5

Skeleton as Compound Bezier Curve

5.1

Bezier skeleton concept

Geometric construction of a polygonal figure skeleton is simple enough: it is a planar graph with straight-line and parabolic edges (Figure 15). However, such analytical description of skeletons presents some difficulties. Presence of parabolic edges raises certain problems in constructing, storing, processing, and utilizing skeletons in image analysis. The general form for a parabola is described by an implicit equation. This is not handy for calculation of parabolas intersections, for drawing and analysis.

Figure 15: A polygonal figure and its skeleton.

This shortcoming generates the tendency to handle skeletons having no parabolic edges. This idea is implemented in the concept of straight skeleton (Aichholzer & Aurenhammer, 1996). But the straight skeleton suffers from certain shortcomings, videlicet: complexity of mathematical definition, low algorithmic efficiency, regularization complexity if noise effects are available. We propose an original method of describing a skeleton in the form of a planar graph with straight edges. It doesn’t require computing parabolic edges either at the step of the Voronoi diagram computing, or at the steps of skeleton storing, drawing and processing. This can be achieved as follows. 1. The skeleton of a polygonal figure is the union of a set of the first and second order elementary Bezier curves. This union we call the compound Bezier curve. 2. A compound Bezier curve is defined by its control graph, which is obtained from the control polygons of elementary Bezier curves. Every control graph has linear edges. Thus, to describe the skeleton, a straight-line control graph is needed (Figure 16).

Figure 16: The control graph of the skeleton from figure 15. Terminal vertices are black and handle vertices are white.

The set of control graph vertices consists of two subsets. The first subset is formed by vertices of polygonal figure skeleton. And the second one consists of the certain control points called handles of Bezier curves. 5.2

Structure of the skeleton

Assume that M is a polygonal figure on R 2 with the Euclidean distance d ( p, q ), p, q ∈ R 2 . The boundary of the figure ∂M consists of several simple polygons. An empty disk of the figure M of radius r ≥ 0 centered at a point p is the closed point set

{

K r ( p ) = q : q ∈ R 2 , d ( p, q ) ≤ r

}

such that K r ( p ) ⊂ M . A maximal empty disk (or, inscribed disk) K rmax ( p ) of the figure M is the empty disk that is not contained in any other empty disks. The skeleton S of the figure M is the set of all centers of maximal empty disks of the figure S = p : p ∈ M , K rmax ( p ) ≠ ∅ .

{

}

This definition of the skeleton is more accurate as comared to the one given in the introduction since terminal vertices of skeletal graph are determined. According to this definition all convex vertices of the figure are terminal vertices of the skeleton. A non-degenerate maximal empty disk touches the figure boundary at least at two points. Every point of the figure can be considered as a degenerate disk of zero radius. These disks are empty because they do not contain internal points and therefore, the boundary points of the figure. Degenerated disks centered at convex vertices of figure are maximal empty disks because they are not contained in other empty disks. Consequently, convex vertices of polygonal figure are part of the skeleton.

A radial function is determined at every point of skeleton. Radial function is equal to a radius of the inscribed disk centered at this point. The radial function assigns “the width” of figure relative to the points of the skeleton. Let S be the skeleton of the polygonal figure M . The total number of points in the set S is infinite, but it occurs that all these points are located at the finite set of the straight-line and quadratic parabolic segments. Let s ∈ S be a point of a skeleton and g1 , g 2 ∈ ∂M be the two closest boundary points of s ∈ S . The points g1 and g 2 may have different positions on the figure boundary. We will call the boundary point by a corner point if it is the vertex of polygonal figure, and simple point otherwise. Three cases of g1 and g 2 type combinations are possible: g1 and g 2 make a pair of corner points, a pair of simple points or a corner and a simple point. g1

s

g1

g1

g1

s

s g2

(a)

g2

(b)

s

g2

(c)

g2

(d)

Figure 17: Bisector types of polygonal figure.

If both g1 , g 2 are corner points then the point s ∈ S lies on the medial perpendicular of the straight line segment [ g1 , g 2 ] (Figure 17a). If both points g1 , g 2 are simple and lie on different sides of the figure then s is equidistant from these sides. So the point s lies on the bisector of the angle, formed by these sides (Figure 17b). If these sides are parallel then s lies on the straight line equidistant from these sides (Figure 17c). But if one of the points (for example, g1 ) is corner and the other ( g 2 ) is simple then s is equidistant from g1 and from the side of polygon, which contains g 2 . In this case s lies on the parabola having a focus g1 . And the directrix of the parabola is the side of polygon such that g 2 lies on this side (Figure 17d). Thus, we distinguish three types of lines. The first line (straight line) is defined by the pair “vertexvertex”, the second one (bisector) is defined by the pair “side-side” and the third one (parabola) is defined by the pair “vertex-side”. Every point of the skeleton lies on one of these lines. Let us use the following terminology. Vertices and sides of polygonal figure are called sites. The maximal connected subset of the skeleton equidistant from the pair of sites is called middle axis or, bisector. There are vv-bisectors, ss-bisectors and vs-bisectors for the pairs of sites “vertex-vertex”, “sideside” and “vertex-side”, respectively. 5.3

Skeleton Vertices

We aim to propose a method describing the skeletal graph such that calculation of the equations of parabolic bisectors is not needed. The skeleton vertices are equidistant to three or more sites. To find these vertices tangent circles can be constructed for the triplets of sites. Calculation of such circles involves a number of geometric tasks (Figure 18) related to the following combinations:

1) 2) 3) 4)

three vertex-sites (Figure 18a); two vertex-sited and one segment-site (Figure 18 b,c); two segment-sites and one vertex-site (Figure 18 d,e); three segment-sites (Figure 18 f).

(a)

(d)

(b)

(c)

(e)

(f)

Figure 18: Tangent circles for the triplets of sites.

The second and third combinations involve two cases depending on whether the vertex-sites match the terminal points of segments. Assume that the tangent circle exists and the sequence of tangent points is defined. Then the tangent circle is unique. To compute the center t of the circle tangent three sites s1 , s2 , s3 , the following system of equation is to be solved:

d 2 (t , s1 ) = d 2 (t , s2 )  2 2 d (t , s1 ) = d (t , s3 ) In the cases in figures 18a,c,e,f both equations are linear. But in the cases in figures 18b,d one equation is linear, and the other is quadratic. After expressing the Y-coordinate of the point t through the X-coordinate in the linear equation it become possible to reduce the second equation to the usual quadratic equation, which is easily solved. The obtained solution has to satisfy two auxiliary conditions, which are easily checked. The first condition requires the projections of t onto the segment-sites to lie on these segments themselves. The second condition requires the tangent circle to lie inside the figure. This means the center of tangent circle is required to lie to the left of the segment-site. 5.4

Edges as Bezier Curves

Explicit description of the parametric curve V (t ) = ( x (t ), y (t )), t ∈ [0,1] provides handy tools to deal with parabolic edges of skeleton. V (t ) determines the skeleton edge with the vertices V (0) and V (1) . The main idea of our solution is that every parabolic edge of the skeleton can be represented by a quadratic Bezier curve

V (t ) = V0 B02 (t ) + V1B12 (t ) + V2 B22 (t ) , t ∈ [0,1] ,

where B02 (t ) = (1 − t ) 2 , B12 (t ) = 2t(1 − t) , B22 (t ) = t 2 are Bernstein polynomials. This curve is determined by its control triangle {V0 ,V1 ,V2 } . The points V0 and V2 are called the terminal points, and the point V1 is handle point of the Bezier curve. V0 and V2 are vertices of skeleton, but V1 is not a skeleton vertex. Such a way of edge description is compact and easy-to-use since only one handle point together with two terminal ones defines every edge. Also skeleton drawing and handling becomes very simple since various effective algorithms to handle Bezier curves are known. Generalized description is based on representation of linear edges of the skeleton in the form of first order Bezier curves V (t ) = V0 B01 (t ) + V1B11 (t ) , t ∈ [0,1] . Here points V0 , V1 denote terminal points of bisector. And B01 (t ) = 1 − t and B11 (t ) = t are Bernstein polynomials. A parabolic skeleton edge is a vs-bisector. Let A and B be a pair of sites that assign this bisector. Moreover, let A be a vertex and B be a side of polygonal figure connecting vertices B1 and B2 . We will denote the side B itself as well as a line containing it by B1B2 . Without loss of generality, let us assume the polygonal figure be on the left to the side B1 B2 . Let us drop the perpendicular from the point A to the straight line B1 B2 calling the intersection D . Denote the middle of AD by O . The sites A and B designate a rectangular Cartesian coordinate system originated at O and having DA as its Y-axis and a line parallel to B1 B2 as its X-axis (Figure 19).

y

V2 V0 A

V x

O B1 C 0

D

U

С2

B2

Figure 19: Parabolic curve for vs-bisector.

Given the sites A and B , the bisector of A and B consists of the centers of the circles that touched both B1B2 and A . Let V0 and V2 be the terminal points of this bisector. And let С0 and С2 be the projections of V0 and V2 onto the straight line B1B2 (Figure 19). Let us examine a point V = ( x, y ) on the bisector and its orthogonal projection U on the straight line B1B2 . The point A coordinate pair is (0, p ) . Since AV 2 = UV 2 we obtain x 2 + ( y − p) 2 = ( y + p) 2 . Then the bisector parabolic equation is

1 2 x. 4p Given the points V0 = ( x0 , y0 ) and V2 = ( x2 , y2 ) , consider two lines that tangent to parabola at V0 and V2 . As is known, the equation of a tangent line for a curve F ( x, y ) = 0 at point ( xˆ , yˆ ) is Fx′( xˆ, yˆ ) ⋅ ( x − xˆ ) + Fy′ ( xˆ , yˆ ) ⋅ ( y − yˆ ) = 0 . In our case, we have F ( x, y ) = x 2 − 4 py . Then the equations for tangent lines at the points V0 and V2 on the curve are the following: 2 x0 ⋅ ( x − x0 ) − 4 p ⋅ ( y − y0 ) = 0 , (1) 2 x2 ⋅ ( x − x2 ) − 4 p ⋅ ( y − y2 ) = 0 . (2) y=

Since

y0 =

1 2 1 2 x0 , y2 = x2 , 4p 4p

(3)

x1 =

1 ( x0 + x2 ) , 2

(4)

y1 =

1 x0 x 2 . 4p

(5)

the solution of the system (1)-(2) is

Thus, we have obtained the point of tangent lines intersection V1 = ( x1 , y1 ) . Permutation of Bernstein polynomials to the quadratic Bezier curve equation gives the parametric equations for Bezier curve V (t ) :

x(t) = (x0 − 2x1 + x2 )t 2 − 2(x0 − x1 )t + x0

,

(6)

y(t) = ( y0 − 2y1 + y2 )t 2 − 2( y0 − y1 )t + y0

,

(7)

t ∈ [0,1] . Permutation of (4) to (6) presents

x(t ) = x0 + ( x2 − x0 ) ⋅ t .

And permutation of (3) and (5) to (7) presents

y(t) =

[

]

(8)

[

]

1 2 1 (x0 − 2x0 x2 + x22 )t 2 − 2(x02 − x0 x2 )t + x02 = ( x0 − x2 ) 2 ⋅ t 2 − 2 x0 ( x0 − x2 )t + x02 = 4p 4p =

1 [( x0 − x2 )t − x0 ]2 . 4p

From (8) and (9) we have

y (t ) = that is the equation of the parabola of vs-bisector.

1 [x(t )]2 , 4p

(9)

Thus, we have a parabolic bisector described as a quadratic Bezier curve. This curve is assigned by a control triangle {V0 , V1 , V2 }. Two vertices V0 , V2 are terminal points of the bisector, and V1 is the point of intersection of tangents lines. Consequently, in order to obtain bisector as the Bezier curve it is necessary to calculate tangent lines at the terminal points of a bisector and to find their intersection. Let us consider the solution of this problem. 5.5

Control Triangle of Skeleton Edge

Let A = (0, p ) be the focus of a parabola and y = − p be the directrix of the parabola. Assume that the point V = ( xˆ, yˆ ) lies on the parabola and C = ( xˆ ,− p ) is the projection of V onto the directrix (Figure 20). Let us show that a tangent line to a parabola at the point V = ( xˆ, yˆ ) is orthogonal to the vector AC .

y

p

A

V x

-p

С

Figure 20: Orthogonality of tangent and direction from the focus to the point of projection.

The equation of the tangent line to the parabola x 2 − 4 py = 0 at the point V = ( xˆ, yˆ ) is 2 xˆ ⋅ ( x − xˆ ) − 4 p ⋅ ( y − yˆ ) = 0 . (10) We have that the vector ( 2 xˆ ,−4 p ) is a normal vector of the tangent line and is collinear to the vector AC . This property makes it possible to find tangent lines at the terminal points V0 and V2 of a skeleton parabolic edge. This requires the projections С0 and С2 of V0 and V2 , respectively, onto the straight line B1 B2 to be calculated first. Then the vectors AС0 and AС2 are to be calculated. These vectors are orthogonal to the corresponding tangent lines. The source data to identify tangent lines to the bisector at its terminal vertices is the following. Given the pair of the sites A , B and two terminal points V0 = ( x0 , y0 ) , V2 = ( x2 , y2 ) of bisector, let us find the handle vertex V1 of the control triangle {V0 , V1 , V2 } . Without loss of generality, assume that the site A is a vertex, the site B =[ B1 , B2 ] is a side of the polygonal figure and the polygonal figure lies to the left of B . Let us introduce the following notation. Let PQ denote the vector with an initial point P and a terminal point Q . By [ P1Q1 × P2Q2 ] denote the cross product, by ( P1Q1 , P2Q2 ) denote the scalar product, by V + PQ denote a shift of point V by vector PQ , by PQ denote length of the vector. The algorithm to solve the problem is following: Algorithm steps

1. Find the parameter p of the parabola:

p=

[ B1 B2 × B1 A] 2 ⋅ B1 B2

.

2. Find points С0 , С2 which are projections of V0 , V2 , respectively:

С0 = B1 + B1 B2 ⋅

(B B , B V ) ,

С2 = B1 + B1 B2 ⋅

(B B , B V ) .

1

2

1 0

B1 B2 1

2

1 2

B1 B2

3. Find vectors AC0 and AC2 :

AC0 = (С0 .x − A.x, С0 . y − A. y ) = (a, b)

AC 2 = (С2 .x − A.x, С 2 . y − A. y ) = (c, d ) (a, b) and (c, d ) are coordinate pairs of AC0 and AC2 , respectively. 4. Solve the system of equations

 a ⋅ ( x − x0 ) + b ⋅ ( y − y 0 ) = 0  c ⋅ ( x − x2 ) + d ⋅ ( y − y2 ) = 0 5. The solution of the system gives the coordinates of the handle point V1 = ( x1 , y1 ) of the control triangle. 5.6

Compound Bezier Curve

We showed that each parabolic edge of the skeleton (vs-bisector) can be described by its quadratic Bezier curve. For generality we can consider linear edges (vv-bisectors and ss-bisectors) to be linear Bezier curves V (t ) = V0 B01 (t ) + V1 B11 (t ) , t ∈ [0,1] . Here points V0 , V1 denote terminal points of bisector. From B01 (t ) = 1 − t and B11 (t ) = V1 ⋅ t we have V (t ) = V0 ⋅ (1 − t ) + V1 ⋅ t . Thus, the skeleton is a union of Bezier curves of first- and second-order. We call this union the “compound Bezier curve” analogously to the related font design concept, where compound curves describe the closed outlines of font symbols. In this paper, curves describe more complex structure that is a connected planar graph. Planarity of the control graph of the compound Bezier curve is an important property of the control graph. This property can be proved as follows. Let us examine the vertex-site A and the segment-site B connected with the parabolic edge. If points V0 and V2 lie on the same side of the Y-axis, i.e., x0 and x2 are of the same sign, then from (5) it follows that y1 ≥ 0 and the point V1 lies above the segment B . Assume that x0 and x2 have different signs (Figure 21). Since the focus A of the parabola is the concave vertex of polygonal figure then the angle α , formed by incident sides of A , belongs to the interval π < α < 2π . Let us examine the angle ∠V0 AV2 between vectors AV0 and AV2 . It is obvious that

π  ∠V0 AV2 ≤ 2π −  α + 2 ⋅  = π − α < π . 2  Consequently, the point V2 lies below the straight line AV0 passing through the focus A , and point V0 . This straight line intersects parabola at the point V0 and at the point V ∗ with the coordinates ( x ∗ , y ∗ ) , moreover x ∗ > x2 . The equation of the straight line AV0 is y = p + a ⋅ x , where a is the angular coefficient. The points of intersection of this straight line with the parabola can be found from the equation

p + ax =

1 2 x . 4p

y V*

α π/2

π/2 A

V2

V0 x0

x

x*

x2

V1 B

Figure 21: Planarity of the control graph

This quadratic equation has two roots:

(

)

(

)

x0 = 2 p ⋅ a − a 2 + 1 , x∗ = 2 p ⋅ a + a 2 + 1 . Intersection point V1 of the tangent lines has an ordinate y1 . From the equation (5) and the condition x ∗ > x2 we obtain the following estimation 1 1 1 y1 = x0 x 2 > x0 x ∗ = x0 x 2 = 4p 4p 4p

=

[ (

)] [ (

)]

[

]

1 1 2 p ⋅ a − a2 +1 ⋅ 2 p ⋅ a + a2 +1 = 4 p 2 ⋅ a 2 − ( a 2 + 1) = − p . 4p 4p

We obtain y1 > − p and the point V1 lies above the segment B , too. Thus, we have that the control triangle of a parabolic edge does not intersect its own segment-site and lies inside the union of empty circles centered at the points of a parabolic segment. Consequently, the sides of a control triangle do not have intersections with the remaining edges of control graph. But this means that the control graph of skeleton is planar. 5.7

Radial function of skeleton

To each point of a skeleton a radial function assigns a radius of an inscribed empty disk centered at this point. Let us examine representation of the radial function if the skeleton is represented by the compound Bezier curve. Given the terminal points V0 and V1 of a linear ss-bisector together with r0 and r1 , we can find the radius of the empty disk centered at any inner point of the edge V0 V1 ( r0 and r1 are radii of the disks centered at V0 and V1 , respectively). The radius of empty disk centered at the point V (t) = V0 ⋅ (1− t ) + V1 ⋅ t is r(t) = r0 ⋅ (1− t) + r1 ⋅ t . (11) Let us consider the vs-bisector case. In the local coordinate system (Figure 20) we have simple relation between radii of disks and ordinates of the points of bisector r (t ) = y (t ) + p . From the property of Bernstein polynomials B02 (t ) + B12 (t ) + B22 (t ) = 1 we obtain

r (t ) = y0 B02 (t ) + y1 B12 (t ) + y2 B22 (t ) + p =

= ( y0 + p) ⋅ B02 (t) + ( y1 + p) ⋅ B12 (t) + ( y2 + p) ⋅ B22 (t) = = r0 ⋅ B02 (t) + (y1 + p) ⋅ B12 (t) + r2 ⋅ B22 (t). Therefore

r(t) = r0 B02 (t) + r1B12 (t) + r2 B22 (t) .

(12)

Let us consider the disc centered at the handle point V1 . For radius of this disk we have r1 = y1 + p . This disk is called a handle disk. As it follows from geometric analysis (Figure 21), r1 is the distance from the point V1 to the line B1 B2 . We obtain:

r1 =

[ B1 B2 × B1V1 ]

.

B1 B2

Thus, the formulas (11) and (12) look like Bezier splines. Now let us consider the vv-bisector. All empty disks centered at this bisector inner points touch the common vertex of polygonal figure. Therefore the radius of an empty disk centered at the point V (t ) = V0 ⋅ (1 − t ) + V1 ⋅ t is defined as distance from the point V (t ) to A . We see that within the vv-bisector the raidus of an empty disk can not be presented in Bezier spline form. Thus, in order to compute the radial function for any point of vv-bisector, coordintaes of related

(a)

(b)

(c)

(d)

Figure 22: Comparing with the stright skeleton: (a) polygonal figure and its skeleton, (b) control graph of the skeleton, (c) control disks of radial function, (d) the straight skeleton (Aichholzer, Aurenhammer, 1996) for this polygon.

concave vertices of polygonal figure should be stored in the skeleton data structure. At the same time, vsbisector and ss-bisector require coordinates of centers of handle disks as well as radiuses of handle disks to be stored in the skeleton data structure. An example in Figure 22 shows that the resulting skeleton and its control graph gives a more accurate description of the object shape compared to the straight skeleton. The figure and straight skeleton in this example are taken from skeleton (Aichholzer & Aurenhammer, 1996). Skeleton representation based on the compound Bezier curve is a handy tool for visualization, storage and image shape analysis in computer vision. To visualize the skeleton it is enough to utilize standard graphic applications supporting drawing of straight-line segments and Bezier curves as well. Generally, graphic libraries are supplied with the tools to draw cubic Bezier curves. To exploit such programs in order to draw quadratic Bezier curves the known conversion of control polygons is to be carried out. A quadratic Bezier curve with the control triangle {V0 , V1 , V2 } matches the cubic Bezier curve with the control quadrangle {W0 , W1 , W2 , W3 } if and only if

W0 =V0,

1 2 W1 = V0 ⋅ + V1 ⋅ , 3 3

2 1 W2 = V1 ⋅ +V2 ⋅ , 3 3

W3 = V2 .

Thus, obtaining the control polygon of the cubic Bezier curve matching the quadratic Bezier curve can be represented. Example in the Figure 23 presents an application of our method to a natural binary bitmap image.

(a)

(b)

(c)

Figure 23: Skeleton representation of natural image: (a) binary bitmap, (b) polygonal figure approximation and the skeleton, (c) the control graph of the skeleton.

6

Experiments and Application

6.1

Experiments

The described method of continuous skeleton construction of a binary image was implemented and has passed multiple checks in different applications.

(b)

(a)

(c)

(d)

(e)

Figure 24: Test examples: (a) Billygoat, (b) leaf, (c) room, (d) neuron, (e) roots.

Theoretical estimates of computational complexity of algorithms, with all their importance, don’t give full conception about availability of algorithms in computer vision systems. So there is a necessity in experimental estimates derived from real working algorithms and from practical examples. There are not many publications describing such experiments. Usually there is no information about software implementation and algorithm running time at all (Manzanera et al., 1999) or there are only results of computing experiments with "toy" examples of very simple images (Deng et al., 2000). The most difficult examples of images and real time expenses for their skeletonization are presented in works (Ogniewicz & Kubler, 1995; Strzodka & Telea, 2004). These examples are shown in Figure 24. Comparison results of our algorithm with the algorithms described in these works are given in Tables 1 and 2. Quality of derived continuous skeletons is shown on examples in Figure 25, 26. The running time of our algorithm was estimated using Intel processor 1.6 GHertz and 512 Mb of memory. Time in tables is specified in seconds. Comparison with algorithm (Ogniewicz & Kubler, 1995) shows, that using MPP for image boundary approximation allows reducing dimension of a problem completely: number of elements in a polygonal figure skeleton is in 6-8 times less than in a corresponding Voronoi diagram of image boundary points. The general reduction of computational time (in 196 times) is partially explained by this dimension reduction of a problem, and partially by processors capacity increase as compared with SPARCstation-2. A new parallel discrete skeletonization algorithm is described in (Strzodka & Telea, 2004). As authors show, its running time is record for discrete algorithms. There are given results received by authors on GPU GeForce FX 5800 Ultra chip, containing tens independent computers working in a parallel mode. However, apparently from table 2, running time of our algorithm is above on 1-2 order. It is necessary to note that our algorithm can be parallelized too and its operation speed on multicore processors will grow.

(a) (b)

(c)

(d)

Figure 26: Polygonal boundaries of figures (red) and continuous skeletons (black): (a) leaf, (b) room, (c) Billygoat (internal), (d) Billygoat (external).

Figure 25: The fragment of the continuous skeleton for “neuron”.

sites edges vertices Time (sec)

OK 11104 31381 20303 9.82

CS 1874 3721 3730 0.05

OK/CS 5.92 8.43 5.44 196.4

Table 1: Comparison of our algorithm CS and algorithm OK (Ogniewicz & Kubler, 1995) for Billygoat example.

Dataset Leaf Room Neuron Roots

Size (pixels) 410× ×440=182040 413× ×506=208978 839× ×731=613309 1800× ×1810=3258000

ST 0.14 0.64 2.5 3.79

CS 0.02 0.03 0.1 0.41

ST/CS 70 21 25 9.22

Table 2: Comparison running time (sec) of our algorithm CS and algorithm ST (Strzodka & Telea, 2004).

6.2

Application to Shape Comparison of Flexible Objects

We consider the problem of shape comparison for elastic objects presented by binary bitmaps. Our approach to similarity measuring of such objects is based on the conception of a flexible object. A flexible object is defined as a planar graph with a family of circles centered on graph edges. A set of admissible deformations is defined for each flexible object. These deformations are described as a group of planar graph vertices transforms. We define the flexible objects similarity through matching and alignment within the group of admissible deformations. The regular method for approximation of the binary bitmap shape by the flexible object is based on skeleton. The flexible object is designed as a subgraph of continuous skeleton of the binary bitmap. The proposed approach is applied to a problem of palm shape recognition for personal biometrical identification. A human palm is a difficult object to classify. The person can’t repeat the same position of a palm even if he wants to. Two photos of the same palm and two photos of two different palms can have differences of the same range (Figure 27). Approaches to palm shape comparison based on alignment of their outline contours are unsuitable because they don’t preserve important invariants of the palm shape – finger’s width and curvature. The basis of our approach is the model of the so-called flexible object. Such object has a shape which can change within certain limits. The limits of these changes are described by special group of transforms. The transforms are organized in such a way that some elements of the shape are fixed and constant, and others can vary. With reference to a palm constant elements are metacarpus and phalanges of fingers, but changeable elements are joints. The problem of flexible object comparing is reduced to selection of such admissible transforms of these objects, in which their shapes will be the closest to one another. The difference of their shapes in this (the closest) position is accepted as a measure of the distinction of objects.

(b) (a) Figure 27: The silhouettes of palms (a) the first person, (b) the second person.

Let's consider a set of points T on the Euclidean plane R 2 , which is presented by a planar graph with tree type structure. This graph has a finite set of vertices, and its edges are continuous lines.

We bind with each point t ∈ T in graph T some circle ct with the center in this point. Let’s call this family of circles C = {ct , t ∈ T } a circular tree. A graph T we call an axial graph of the circular tree. And a union of all circles of family C (as point sets) we call a silhouette of the circular tree: 0

1 2 3

A B

A B

5

4

(a) (b) (c) (d) Figure 28: Approximation of a palm by flexible object: (a) the minimal perimeter polygon and its skeleton, (b) the skeletal base, (c) the circular tree, (d) bending points.

S = U ct t ∈T

The silhouette of the circular tree represents a closed connected set in Euclidean plane S ⊂ R 2 . The outline of this set is an envelope of the whole family of circles C . Among all vertices of a circular tree we extract some subset of the points P ⊂ T called bend points. We connect a range of angles between each couple of edges incident to bend point. We forbid to change a relative position of edges for the rest vertices of the graph. A change of angles in bend points in an admissible range is called a deformation of a circular tree. Such deformation implies not only the change of an axial graph, but also moving of a family of circles and a respective alteration of a circular silhouette of a tree. Let V be a group of deformations of a circular tree C . The couple of circular tree and its group of deformations we call a flexible object G = (C ,V ) . If some deformations of two circular trees make their silhouettes coincide then these trees are called equivalent. Let it be a binary bitmap (Figure 27). A construction of flexible objects which approximates this bitmap includes the following steps: 1. An approximation of the binary bitmap outline by the minimal perimeter polygon (Figure 28a). 2. A construction of the continuous skeleton of the polygon (Figure 28a). 3. A pruning of a skeleton to get the skeletal base of the polygon (Figure 28b). A skeletal base has a much more simple structure, than the skeleton of a polygon and is more stable to noise distortions connected with the source binary bitmap. Let’s choose a third degree vertex of a skeletal base graph which is incident to the branch of the thumb (the vertex A on Figure 28b). The branches of the thumb and a wrist are crossed in this vertex. The image of a wrist is extraneous information for palm shape description. Therefore we delete the branch of a wrist in the skeletal base. The obtained graph is an axial graph of a circular tree of a flexible object (Figure 28c). The vertex A is a root of the circular tree. Its circle is called a “root” circle. The next third degree vertex B we call a “center” of a palm (Figure 28c) and its circle is called a “middle” circle. The analysis of a real skeleton of a human palm shows, that it is enough to consider six bend points: two points of the thumb and one point for each of the rest four fingers (Figure 28d). We choose two bend points (0 and 1) of maximal curvature at thumb branch and four bend points (2-5) as crossing of finger branches and the “root” circle.

The measure of distance of flexible objects G1 and G2 is received from a static position of G1 and a deformation of G2 . Alignment of two palms is carried out by the following steps. 1. Coincidence of centers of “middle” circles (vertices B in Figure 28d). 2. Coincidence of directions from the centers of “middle” circles to the centers of “root” circles (vectors BA in Figure 28d). 3. Deformation of the axial graph of the second palm for coincidence with the axial graph of the first palm (Figure 29). For that we “rotate fingers” of the second palm (branches of axial graph) around bending points. The Hausdorf metrics can be used as a measure of coincidence of these branches.

(a)

(b)

Figure 29: Deformation of the circular tree: (a) rotation of branches, (b) moving of circles.

4. Construction of circular tree silhouettes as envelopes of a family of circles. 5. Comparing of silhouettes (Figure 30). The effective algorithm for computation of the areas of a symmetric difference is designed with the help of methods of computational geometry.

(a) (b) Figure 30: Comparison of silhouettes: (a) images of the same palm, (b) palms of different persons.

The measure of distance between silhouettes S1 and S2 is computed as

ρ ( S1 , S 2 ) =

Area ( S1 \ S 2 ∪ S 2 \ S1 ) . Area ( S1 ∪ S 2 )

The combination of two constructions – an outline and a skeleton – opens up opportunities for the comparison of objects which don’t require strictly fixed shapes for the matching method. The proposed method agrees well adjusted with common sense, it is easily visualized and allows efficient implementation. 6.3

Application to Spatial Reconstruction of Human Pose

The problem of reconstructing the spatial objects using several two-dimensional images is well-known and has many applications. In particular recognition of a posture requires reconstructing the spatial form of such a complex and variable object as a human body. The essence of our consideration is that the twodimensional images are supposed to be binary and to represent only silhouettes of a spatial object (Figure

31). Such a problem, in particular, arises in recognition of gestures by means of standard inexpensive equipment. For example simple WEB-cameras can hardly transfer the textural features of images and are able to reliably provide the silhouettes of the objects only. The lack of texture details makes it impossible to analyze images at texture level and apply well known object reconstruction methods based on automatic identification of matching points on stereo mate images. Obviously, the boundary points are the only points that may be reliably identified on the silhouette images. The problem is that the boundary points of a silhouette on one of the stereo mate images can have no matching points on the boundary of the silhouette on the other image. Thus, it is impossible to directly identify the matching points on the stereo mate silhouette images.

Figure 33: Stereo mate of initial images and their silhouettes.

Figure 32: Skeletons and radial functions of obtained silhouettes.

Ο

f

A

Ο′

g

A′

Figure 31: Stereo mate points found on skeletons.

Still one can try to identify the matching points making some assumptions on the nature of the original object. We propose to approximately represent human body as a union of the several "cylindrical" elements having local axial symmetry. Such objects are also called “generalized cylinders” or “tubular objects”. To be more precise a cylindrical element is a spatial body formed by a family of spheres with the centers on some curve. Such objects are called spatial fat curves. We are interested in the objects that can

be represented as a union of several fat curves. Such locally symmetric objects can be used as the models for the description of a human palm or a body. The proposed approach is based on revealing the symmetry axes of the locally symmetric objects. Although, these axes are invisible on the stereo mate images, they still can be calculated for each image by processing a silhouette presented on it. Considering the silhouettes of stereo mate images as the projections of spatial fat curves onto the corresponding planes, we can expect that the projection of the fat curves axes approximately coincide with the middle axes of the silhouettes. This is a key consideration for reconstructing a spatial form of a locally symmetric object based on its stereo mate silhouette images. We will consider some (invisible) points which are not the boundary points of the silhouettes as the common reference points of stereo mate images. Such reference points are provided by middle axes of the silhouettes constituting its skeleton. The structure of the proposed method is as follows. 1. First, we build a skeleton and radial function (Figure 32) for the silhouettes derived from the segmentation of the original images (Figure 31). The result is the set of inscribed circles centered on the medial axis. 2. After that, we set one to one correspondence of points of skeletons (Figure 33) and determine the coordinates of the spatial axes for the spatial body by epipolar geometry methods using stereo pairs of points of the skeletons. In this way we reconstruct the skeleton of a spatial object. For families of inscribed circles we calculate the radiuses of the spheres with centers on a spatial skeleton. 3. Thus, we obtain a description of the human body posture in the form of a spatial skeleton, as well as the envelope of the constructed surface for the family of spheres (Figure 34).

Figure 34: Reconstructed spatial object.

7

Conclusions

The continuous approach to image skeleton construction is superior to traditionally applied discrete methods on many criteria. 1. The continuous skeleton is described by the strict mathematical model. The discrete skeleton hasn’t such strict mathematical description; it is considered only as some analogue of a continuous skeleton. 2. Regularization of continuous skeletons, aimed at noise effects elimination, can be performed by strict mathematical methods; and as for discrete skeletons, it is done on the basis of heuristics.

3. The continuous skeleton along with the radial function gives more ample opportunities for object shape transformations. Comparison of continuous skeletons is reduced to a problem of planar graphs comparison by topological and metric criteria. 4. Running time of continuous skeletonization algorithm outperforms the best samples of discrete skeletonization algorithms by a factor of ten or even hundred. 5. The skeleton of a polygonal figure may be represented by straight line control graph of a compound Bezier curves. One major advantage is the simplicity of this description. Another advantage is the independence from the algorithm of skeleton construction. The proposed form of skeleton presents the tool for storing skeletons in geographical databases and computer graphics systems. 6. Payment for all advantages of continuous skeletons construction algorithm is the complexity of its software implementation that demands rather refined programming technique.

References Aichholzer, O., Aurenhammer F. (1996). Straight skeletons for general polygonal figures in the plane. Lecture Notes in Computer Science. Vol. 1090. Springer-Verlag, 1996, 117-126. Blum, H. (1967). A transformation for extracting new descriptors of shape. In Proc. Symposium Models for the perception of speech and visual form, MIT Press Cambridge MA, 1967. Costa, L., Cesar, R. (2001). Shape analysis and classification, CRC Press. Deng, W., Iyengar, S., Brener, N. (2000). A fast parallel thinning algorithm for the binary image skeletonization. The International Journal of High Performance Computing Applications, 14, No. 1, Spring 2000, pp. 65-81. Drysdale, R., Lee, D. (1978). Generalized Voronoi diagrams in the plane. Proc. 16th Ann. Allerton Conf. Commun. Control Comput., 1978, 833-842. Fortune, S. (1987). A sweepline algorithm for Voronoi diagrams. Algorithmica, 2 (1987), pp. 153-174. Karavelas, M.I. (2006). Voronoi Diagrams in Cgal, In 22nd European Workshop on Computational Geometry, pp. 229–232, 2006. Kirkpatrick, D. (1979). Efficient computation of continuous skeletons. Proc. 20th Ann. IEEE Symp. Foundations of Computer Science, 1979, 18-27. Lee, D. (1982). Medial axis transformation of a planar shape. IEEE Trans. Pat. Anal. Mach. Int. PAMI-4(4): 363-369, 1982. Lee, D.T., Schachter, B.J. (1980). Two Algorithms for Constructing a Delaunay Triangulation, Int. J. Comput. Inf. Sci., Vol. 9, pp. 219–242, 1980. Manzanera, A., Bernard, T., Preteux, F., Longue,t B. (1999). Ultra-fast skeleton based on an isotropic fully parallel algorithm. Proc. of Discrete Geometry for Computer Imagery, 1999. Mestetskiy, L. (2006). Skeletonization of a multiply connected polygonal domain based on its boundary adjacent tree. In Siberian journal of numerical mathematics, vol.9, N 3, 2006, 299-314, (in Russian). Mestetskiy, L. (2007). Shape comparison of flexible objects – similarity of palm silhouettes. In VISAPP’2007, 2nd Int. conf. on computer vision theory and applications, INSTICC Press, vol. IFP/IA, 2007, pp.390-393. Mestetskiy, L., (2009). Continuous Morphology of Binary Images: Figures, Skeletons and Circulas. FIZMATLIT, Moscow, 2009. (in Russian) Mestetskiy, L. (2010). Skeleton Representation Based on Compound Bezier Curves. In VISAPP’2010, 5th Int. conf. on computer vision theory and applications, INSTICC Press, vol.1, 2010, pp. 44-51. Mestetskiy, L., Semenov, A. (2008). Binary image skeleton - continuous approach. In VISAPP’2008, 3th Int. conf. on computer vision theory and applications, INSTICC Press, vol. 1, 2008, pp. 251-258. Ogniewicz, R., Kubler, O. (1995). Hierarchic Voronoi Skeletons. Pattern Recognition, vol. 28, no. 3, pp. 343-359, 1995. Siddiqi, K., Pizer, S. M. (2008). Medial Representations: Mathematics, Algorithms and Applications. Springer (2008). Srinivasan, V., Nackman, L., Tang, J., Meshkat, S. (1992). Automatic mesh generation using the symmetric axis transform of polygonal domains, Proc. of the IEEE, 80 (9) (1992), pp. 1485–1501. Strzodka, R., Telea, A. (2004). Generalized Distance Transforms and Skeletons in Graphics Hardware. Joint EUROGRAPHICS IEEE TCVG Symposium on Visualization (2004). Yap, C. (1987). An O(n log n) algorithm for the Voronoi diagram of the set of simple curve segments. Discrete Comput. Geom., 2(1987), pp. 365-393.