Modelling of Overlapping Circular Objects Based on Level Set Approach

Modelling of Overlapping Circular Objects Based on Level Set Approach Eva Dejnozkova and Petr Dokladal School of Mines of Paris, Centre of Mathematical Morphology 35, Rue Saint Honor´e 77 300 Fontainebleau, FRANCE [email protected]

Keywords: Shape analysis, computer vision, smoothing, segmentation, curvature, level set, part decomposition Abstract. The paper focuses on the extraction and modelling of circular objects embedded up to any extent. The proposed method is inspired from the continuous Level Set theory and consists of two stages. First, by using the local curvature and the normal vector on the boundaries are detected the shape parameters of the sought circular objects. Second, an area-based matching procedure detects such cases where several identified circles correspond to only one, partially ocluded object.

1

Introduction

The detection of occluded objects and the shape fitting are basic computer vision tasks. Many methods dealing with the detection and separation of overlapping objects can be found in the litterature. The Hough transform (HT) [1] and its extensions [2] represent a popular method for extracting analytic curves. However, its implementation is often memory consuming. Another group of methods is based on classical morphological tools. Meyer [3] proposes a method based on a bisectrice function of the distance to the complement. However, this method can only be used to separate objects, embedded up to a limited extent. A more recent algorithm by Talbot [4] computes the skeleton on the elliptical distance. This method is computationally expensive and the author does not explain up to which extent this algorithm works. The scale-space approach constitutes another separation technique. As an example, one can cite Lindeberg [5] who focuses on the junction detection by the normalized curvature in automatically selected scale-space. However, this method can lead to poor junction localization and requires an additional correction of the localization. Zhang [6] proposes a direct part decomposition of triangulated surfaces by thresholding the estimated gaussian curvature. The second problem, shape fitting, usually deals with some optimization method. A widely used approach for shape fitting is the minimization of the squared error (with constraints or not) [7], [8]. The problem of these methods is

the robustness and its numerical stability. Another approach using a Bayesian formulation is shown in Werman [9]. However, this approach results in optimizing nontrivial cost functions for the fitted model. This paper proposes a new method to detect and model embedded circular objects by using the local curvature of the object borders and tries to show that a local information can also be used for a global shape analysis. The parameter estimation algorithm is based on the continuous Level Set theory [10] which introduces the sub-pixel precision and thus allows to improve the numerical accuracy of the parameter estimation. In addition, the computation cost is reduced by the measuremements performed only on the narrow band of the objects contours. The second goal is to minimize the optimization effort of the shape fitting task by using a clustering method on a set with a reduced number of elements. The paper is organized as follows: the basic notions and principles are introduced first, followed by the algorithm description. Finally an application example used for an analysis of microscope photographs of polymer crystals is presented.

2

Basic Notions

Below we use the following notations and definitions. Let X be a discrete, binary object X: Z2 → {0, 1}, where points at 1 denote the objets and 0 the background. Let C denote a closed curve placed in R2 such that C is a continuous representation of the boundary of X. C is obtained from X by some interpolation method, see e.g. Siddiqi [11] or Shu [12]. For an easy manipulation, the curve C is defined implicitly by the signed distance function u : Z2 → R such that u(x) = d(x, C) (1) The distance u is assumed positive outside and negative inside the object X. C is therefore the zero-level set of u. To describe the curve C, it is useless to compute the distance u on the entire image. In order to limit the computation cost, the distance u is calculated only on the narrow band close to the curve. The narrow band is the set of points  NB = x x ∈ Z2 , |u(x)| < NBwidth (2)

where NBwidth > 0. Outside NB, the values of u are limited by putting u equal to ± NBwidth (exterior/interior). Let X be some set, and P(X) the set of all subsets of X. The mapping CCγ : X → P(X) denotes the set of connected components of X, with a given neighbourhood connectivity γ (i.e. 4- or 8-neighbourhood). Note that a γ-connected component of X is the equivalence class for γ-connected path of points in X. In the following, a circle is defined by a pair {c, r}, where c ∈ R2 stands for the centre and r ∈ R+ for the radius.

2.1

Local Curvature

The curvature analysis is a basic tool in computer vision. Defined in a given point as the inverse of the radius r of the osculating circle in this point, the curvature κ of a curve is proportional to the angular speed of the normal vector n0 travelling alongside the curve : κ = div n0 (3) An exhaustive discussion of the curvature representations can be found in Sapiro [13]. We focus on the curvature computed on the implicit representation of the curve. In terms of the curve implicit description by the distance function u, the curvature κ is given by: ∇u κ = div (4) |∇u| In this paper the curvature κ is used for two objectives: 1) smoothing and 2) radius estimation. The smoothing makes use of the traditional discretization scheme which is obtained by using central differences in Eq. (4). κ=

uxx u2y − 2uxuy uxy + uyy u2x (u2x + u2y )3/2

(5)

The main feature of this discretization scheme is that it allows to estimate the curvature even in singular points, i.e. the points where ux = uy = 0, by solving the limit case. However this scheme is less robust for the radius estimation because the surface deformations can influence the resulting level-set curvature. The radius estimation requires a better accuracy of the curvature measurement.We propose to compute first the normal vector n0 with a more sophisticated scheme taking the mean of four normal vectors obtained by one-sided differences as proposed by Sethian [10]. n00 = (n00x , n00y ) =

+ (u+ x , uy ) + 2 2 (u+ x ) + (uy )

+ and after normalization :


1/2 +

+ (u− x , uy )

+ 2 2 (u− x ) + (uy )

n0 =

− (u+ x , uy ) − 2 2 (u+ x ) + (uy )


1/2 +

n00 |n00 |


1/2 +

− (u− x , uy )

− 2 2 (u− x ) + (uy )


1/2

(6)

The divergence operator Eq. (3) is only applied after having numerically obtained n0 . In this case, the curvature estimation is based on the fine measurements of the changes in the normal vector direction and the resulting curvature estimation is more homogeneous. Nevertheless, this directional approach to the approximation is to be used carefully. The divergence operator can only be applied directly on such points where the normal vector n0 exists both in the point itself and in its neighbourhood. Otherwise the obtained numerical value is incorrect.

3

Algorithm

Before the actual identification, the contours are decomposed in parts to separate the fused objects. The algorithm consists in several steps, described in this section in the order they are applied. 3.1

Construction of the level-set function

The initial objects are defined as X: Z2 → {0, 1}. The boundary level-set function is the distance function u calculated by using Eq. (1). Next, the distance u has to be smoothed to eliminate the discretization effect and the segmentation artefacts (cf. Fig. 2). Recall that u is calculated with a sub-pixel accuracy. Unless one uses some higher-order interpolation to obtain the initial line C, the iso-distance lines will have a staircase-like aspect, unusable for global shape estimation based on the local curvature.

(a)

(b)

Fig. 1. Iso-distance lines before (a), and after smoothing (b).

The level-set smoothing can be considered as a choice of the appropriated scale-space used for the part decomposition. There exist many smoothing methods; see e.g. [14], [15] or [5]. We have adopted the smoothing governed by the equation called geometric heat flow [15]: ∂u = κ|∇u| ∂t

(7)

This choice has been made for the following reasons. It has been shown that any closed curve will be deformed to become convex and will converge to a circular form without developping self-intersections. The expected circular shape of the contours is then naturally preserved. Eq. (7) applies the stronger smoothing factor (see Fig. 1), the higher the local curvature κ is. The smoothing stops when some norm of the difference of the successive iterations is smaller than some arbitrary limit. 3.2

Boundary decomposition

The overlapping objects are separated by detecting the cusp points in the curve C. After the segmentation of C in these points one obtains a set of smooth circular arcs.

The segmentation is based on sign(κ) similarly to the methods proposed by [6] and [16]. The contour is decomposed in convex parts (κ > 0) by splitting it where κ changes sign and by dropping the concave parts. The concave parts correspond to the cusp points before the smoothing. Recall, that here the set of arcs is described implicitly by the set A of portions of the narrow band around the decomposed curve C. The set of arcs A = {A1 , A2 , . . .}, A ⊂ Z2 , is obtained as : A0 = CCN4 {x | x ∈ NB and κ(x) > 0}. The 4-connectivity N4 was used for this application. Optionally, the arc segments that are too short are filtered out: A = {Ai | Ai ∈ A0 , #Ai > const.}, where # stands for the cardinal of a set. 3.3

Detection of centres and radii of the circles

The centre coordinates of the circle approximating a given arc Ak ∈ A are calculated in all points of Ak where the curvature is not biased by the border effect (introduced by points close to the border of NB where the derivatives of u use neighbors from outside the narrow band). Let xi ∈ Ak be any such point. The circle approximating the iso-distance line in xi is given by: the radius ri = 1/κ(xi ) and the centre ci = xi +n0 (xi )/κ(xi ), ci ∈ R2 , where n0 (xi ) is the normal vector (Eq. (6)) of the iso-distance line in xi , and κ(xi ) the curvature. Recall that negative values of κ were filtered out (cf. section “Boundary decomposition”). Let C(Ak ) = {ci } denote the set of the circle centre coordinates, obtained for all xi ∈ Ak . The centre cˆk of the circle, approximating a given arc Ak , is obtained by taking the median: cˆk = med(C(Ak )). Finally, the radius is rˆk = med(de (ˆ ck , xi )), for all xi ∈ Ak , where de : R2 → R+ , denotes the euclidean distance from a to b calculated de (a, b) = [(ax − bx )2 + (ay − by )2 ]1/2 . Note that the computationaly expensive voting and following maxima searching processes of the standard HT are replaced by simple statistical measure. 3.4

Circles matching

Obviously, several arcs may form one circular object and have to be matched. We need some arbitrary condition authorizing to couple circles even if their centres and radii are not perfectly identical. This condition is a trade-off between the accuracy and the capacity to model circles fused to an unlimited extent. We use a classical clustering technique, proceeding iteratively by agglomerating the most similar pairs. The procedure uses the similarity matrix Ξ = {ξij } where ξij denotes the similarity of circles i and j. The similarity criterion ξ of two circles is the area of their intersection divided by the area of their union. The strictly positive values of ξ are allowed only for the circles of similar radii and position (i.e. low de ).

ξij =

(

S(A) S(Ai )+S(Aj )−S(A)

if de (ˆ ci , cˆj )
j

(8)

(a)

(b)

Fig. 2. Original image: (a) A microscope photograph of polymer crystals, (b) Segmentation of the original image.

where S(Ai ) and S(Aj ) are the areas of the i-th and j-th circle and S(A) the area of their intersection. The values ξij are calculated for i > j only to have a triangular matrix under the main diagonal. The matching algorithm reads: repeat while (maxi>j (ξij ) > 0) [k, l] := arg maxi>j (ξij ) // get the most similar pair k,l of all i,j Ak := Ak ∪ Al // merge the circle l to k cˆk := med(C(Ak )) // recompute the centre and radius rˆk := med(de (ˆ ck , xi )) for all xi ∈ Ak delete Al , line and column l from Ξ and recompute Ξ This algorithm is run on a reduced population set and is not computationally expensive. It stops as soon as there are no more pairs of circles verifying ξij > 0.

4

Experiment Results

The motivation of the study presented above was an automatic crystal growth analysis 1 . In the early stage of the growth, the crystals are circular. The goal is to successively study crystal properties such as the size or the degree of incrustation of artificial polymer crystals (Fig. 2). 4.1

Segmentation

Although the segmentation is not the objective of this paper, we briefly describe how the initial contours are obtained. Recall the basic operators from the mathematical morphology. Let f denote an image and F the family of images. Then εX , δX : F → F denote respectively the erosion and dilation by X. If no structuring element is given then the unitary disk is used. The segmentation strategy has been chosen according to the observations made on the original image (Fig. 2). One can see that the image background is almost flat. On the other hand, the range of intensity in the crystal interior can be high. The gray levels on the crystal borders can reach both low and high 1

The microscope images of polymer crystals were used with kind permission of the Centre for Material Forming (CEMEF), School of Mines of Paris, France

(a)

(b)

(c)

Fig. 3. (a) Arcs segmented by the boundary decomposition,(b) Circles detected from the arcs. (c) Result of the circles matching superposed on the original gray-scale image.

values. In order to overcome this problem we start the segmentation procedure by computing the morphological gradient which allows to extract the information about the contrast changes alongside the borders. Let f denote the original grey-scale image and fg = δ(f ) - ε(f ) the morphological gradient of f . Since the next objective is to segment the image by a threshold we first have to equalize the values in the crystal interior. For this purpose we use the hole filling operator HoleFill. HoleFill(fg ) fills the attraction basins of fg (corresponding to the agglomerated crystals): f1 = HoleFill(fg ). Note, that the HoleFill operator can be applied either to binary or grey-scale images. More details can be found in [17], for example. Finally, the following thresholding extracts the gradient crests delimiting the circular objects:  1, if f1 > T h f2 = 0, otherwise where T h > 0 (a convenient value for this application is T h = 15). In the next stage a morphological closing filter will smooth noisy borders: f3 = εX (δX (f2 )), where X is a disk of a four point radius. In order to eliminate the possible holes inside these objects, a following hole-filling operator is applied to fill the interior of the objetcs: f4 = HoleFill(f3 ). Finally, an area opening suppresses small, noisy objects in the background: f5 = AreaOpenN (f4 ), with N = 300 (to eliminate objects smaller than 300 points). f5 is the result of the segmentation (see Fig. 2 (b)). The binary objects are then submitted to boundary decomposition to extract the smooth arcs, see Fig. 3(a). For every extracted arc one osculating circle is found (Fig. 3(b)). Finally, if several circles correspond to only one circular object they are matched and replaced by only one circle. The parameters of the new circle are accordingly adjusted. See the results of the matching (superposed to the original image) at Fig. 3(c).

5

Conclusions

The paper shows the use of local curvature measure for a global shape analysis and gives a specific application example, where the curvature is used to separate circular objects fused theoretically up to any extent.

The proposed method uses a distance-based, implicit description of the contours for the estimation of the radii of circular objects. The measurements are performed only on a narrow band around the contours. Obviously, the curvature varies on different levels of the level set, even if measured in the normal direction. Nevertheless, the osculatory circles tangent to points laying on the normal are cocentric. Using larger narrow band gives birth to a more numerous population of candidates, and consequently to an increase of the accuracy. Concerning the radius estimation, it has been observed that higher curvature offers better accuracy for the radius estimation. One improvement consists in giving stronger weights to points belonging to level sets closer to the circle centre, and having therefore a higher curvature. The second one consists in using an asymmetric narrow band, larger towards the centre of the circles. In addition, this paper presents a matching procedure based on the analysis of the circle intersection area. The proposed matching condition allows to couple circles representing one binary object and, at the same time, to separate crystals embedded up to high degree of incrustation. The computation complexity of this technique remains quite low. The only computationally extensive step is the preprocessing where the distance function is iteratively smoothed. The needs of smoothing could be limited by improving the segmentation quality or by using some more sophisticated interpolation for the construction of the distance function [11]. However, the proposed method have a high degree of parallelism and can be efficiently implemented on a specific parallel hardware [18] without any constraint.

References 1. Hough, P.V.C.: Methods and means for recognizing complex patterns. US Pattent 3069654 (1962) 2. Atiquzzaman, M.: Coarse-to-fine search technique to detect circles in images. International Journal of Advanced Manufacturing Technology 15 (1999) 96–102 3. Meyer, F.: Cytologie quantitative et morphologie math´ematique. PhD thesis, Ecole des Mines de Paris (1979) 4. Talbot, H., Appleton, B.: Elliptical distance transform and the object splitting problem. In: International Symposium on Mathematical Morphology, Australia. (2002) 5. Lindeberg, T.: Scale-Space Theory In Computer Vision. Kluwer Academic Publishers, Monograph 1994 (1994) 6. Zhang, Y., Paik, J., Koschan, A., Abidi, M.A.: A simple and efficient algorithm for part decomposition of 3-d triangulated models based on curvature analysis. In: International Conference on Image Processing ICIP02. Volume III., Rochester, N.Y. USA (2002) 273–276 7. Fitzgibbon, A., Pilu, M., Fisher, R.: Direct least square fitting of ellipses. IEEE Transactions on Pattern Analysis and Machine Intelligence 21 (1999) 476–480 8. Gander, W., Golub, G.H., Strebel, R.: Least-squares fitting of circles and ellipses. In editorial board Bulletin Belgian Mathematical Society, ed.: Numerical analysis (in honour of Jean Meinguet). (1996) 63–84

9. Werman, M., Keren, D.: A bayesian method for fitting parametric and nonparametric models to noisy data. IEEE Transactions on Pattern Analysis and Machine Intelligence 23 (2001) 528–534 10. Sethian, J.: Level Set Methods. Cambridge University Press (1996) 11. Siddiqi, K., Kimia, B., Shu, C.W.: Geometric shock-capturing ENO schemes for subpixel interpolation, computation and curve evolution. Graphical models and image processing: GMIP 59 (1997) 278–301 12. Osher, S., Shu, C.W.: High-order Essentially Non-oscillatory schemes for HamiltonJacobi equations. SIAM Journal of Numerical Analysis 28 (1991) 907–922 13. Sapiro, G.: Geometric Partial Differential Equations and Image Analysis. Cambridge University Press (2000) 14. Leymarie, F., Levine, M.D.: Curvature morphology. Technical Report TR-CIM-891, Computer Vision and Robotics Laboratory, McGill University, Montreal, Quebec, Canada (1989) 15. Kimia, B., Siddiqi, K.: Geometric heat equation and nonlinear diffusion of shapes and images. Computer Vision and Image Understanding: CVIU 64 (1996) 305–322 16. Siddiqi, K., Kimia, B.: Parts of visual form: Computational aspect. IEEE Transactions on Pattern Analysis and Machine Intelligence (1995) 17. Serra, J.: Image analysis and mathematical morphology. Academic Press, London (1982) ENSMP - CMM Fontainebleau. 18. Dejnoˇzkov´ a, E., Dokl´ adal, P.: Asynchronous multi-core architecture for level set methods. In: International Conference on Acoustics, Speech and Signal Processing, Canada, IEEE (2004) Proceedings.