Two dimensional discrete statistical shape models

Two dimensional discrete statistical shape models construction Isameddine Boukhriss, Serge Miguet, and Laure Tougne Universit´e Lyon 2, Laboratoire LIRIS Bˆ atiment C, 5 av. Pierre Mends-France 69 676 Bron Cedex, France {isameddine.boukhriss,serge.miguet,laure.tougne}@univ-lyon2.fr http://liris.cnrs.fr

Abstract. In this paper, we present a discrete approach to 2D statistical shape models construction. An object is modeled in terms of landmark points on its contour. An optimized process of intrinsic discrete properties computes then curvilinear position, curvature and normal vector for each point. The approach uses after a dynamic programming process to match points of different shapes. By identifying matched points on a set of training examples, a statistical approach with principal component analysis, computes the mean object shape, the major modes of shape variation and extracts a compact and generic model. Key words: rigid registration, curvature, normals, shape matching, PCA, shape variation, deformable model.

1

Introduction

Models of shape are used widely in computer vision. They are of great interest in objects localization, tracking or classification. Many objects are non-rigid, requiring a deformable model in order to capture their shape variability. One such model is the Point Distribution Model (PDM). An object is modeled in terms of landmark points positioned on contours. By identifying such points on a set of training examples, a statistical approach (principal component analysis, or PCA) can be used to compute the mean object shape, and the major modes of shape variations. The standard PDM is based purely on linear statistics (the PCA assumes a Gaussian distribution of the training examples in shape space). For any particular mode of variation, the positions of landmark points can vary only along straight lines. Non-linear variation is achieved by a combination of two or more modes. Our approach encodes shapes with a set of particular points resulting from a polygonalization process. This guarantees the reversibility, in the construction process of the final model. However, shapes could have different numbers of points encoding their contours, which is not convenient for the PCA. This problem is solved with a controlled matching process which ensures that for every point of a given shape contour, we could find a destination point in any other contour. This procedure is explained in 3. In section 2, we show how we

2-9525435-1 © SITIS 2006

- 666 -

encode shapes contours and how we compute their discrete parameters. Section 4 explains how the final model is extracted. But first of all, let us talk about previous works about shape models. In our previous work [1], we focused on shape matching and recognition, but in this paper we extend the approach to extract deformable models from a set of shapes. Many approaches are proposed for shape models construction. For studying shape variations, these approaches could be ’hand-craft’, semiautomatic or automatic. Yuille and al [2] build up a model of a human eye using combinations of parameterized circles and arcs. This can be effective, but it is complicated, and a completely new solution is required for every new case. Staib and Duncan [3] represent shapes using fourier descriptors of closed curves. The complexity depends on coefficients and the generalization is not possible to open contours. Kass et al [4] introduced Active Contour Models (snakes) which are energy minimising curves. When no model is imposed, they are not optimal for locating objects which have a known shape. Alternative statistical approaches are described by Goodall [5] and Bookstein [6], they use statistical techniques for morphometric analysis. The most common modelling variability approach is to allow a prototype to vary according to some learned model. Park and al. [7] and Pentland and Sclaroff [8] represent the outlines or surfaces of prototype objects using finite element methods and describe variability in terms of vibrational modes. Turk and Pentland [9] use principal component analysis to describe the intensity patterns in face images in terms of a set of basis functions, or ‘eigenfaces’. Poggio and Jones [10] synthesize new views of an object from a set of example views. They fit the model to an unseen view by a stochastic optimization procedure. In [11], computing a curve atlas, based on deriving a correspondence between two curves, is presented. The optimal correspondence is found by a dynamic programming method based on a measure of similarity between the intrinsic properties of curves. Our approach has a common part with their technique in the matching process shown in section 3 which is a pre-stage to statistical analysis and model computing. But our technique is discrete, it does not subdivide uniformly contours and optimize correspondence between two sets of points with different sizes. The extraction and characterization of these points is explained in the following section. The section 3 deals with the process of matching between shapes. When this process is optimized on all shapes, we extract their statistical representative in section 4.

2

Data adjustment

Let us consider two binary figures F and F 0 , each of them contains only one connected component, without hole, called shape in the following and denoted respectively by f and f 0 . We proceed by a registration and scaling process. f and f 0 are aligned according to their maximal elongation (principal component associated to the first eigen vector) with maximum interior intersection. Let us now consider the respective borders of each shape, that is to say the set of points that belongs to the shape and that have at least one 4−background neighbor.

- 667 -

Note that we consider then 8−connected borders. Let us denote respectively by C and C 0 the borders of f and f 0 . We suppose that the border C contains n points and the border C 0 contains M points with N not necessary equals to m. These points are the extremities of the segments obtained by Debled’s linear polygonalization algorithm [12]. Let us denote by Pi with i from 0 to n the extremities of segments extracted from C, remark that P0 = Pn , and Pj0 with j 0 from 0 to m the extremities of segments extracted from C 0 with P00 = Pm . On 0 each point Pi and Pj we compute parameters that will help to associate a cost to each pair of points (Pi , Pj0 ). The first parameter to compute is the distance between the points of one pair. So, we consider the Euclidean distance: q d(Pi , Pj0 ) = (xPi − xPj0 )2 + (yPi − yPj0 )2 The two other parameters we compute compare locally the two borders C and C 0 . One measures the difference between the curvatures in the points Pi and Pj0 and the second compares the normal vectors. To the pair of points (Pi , Pj0 ) we associate the difference of curvatures : κ(Pi , Pj0 ) =| κPi − κPj0 | In order to avoid to associate points that would be near, with a quasi-similar local curvature but with inverse curve orientation, we compute a last parameter that compares the normal vectors at the points Pi and Pj0 . We associate to the pair of points (Pi , Pj0 ), the angle made by the two normal vectors : −−−−−→ −−−−→ \ α(Pi , Pj0 ) = c(Pi )Pi , c(Pj0 )Pj0

Fig. 1. Discrete parameters computation: polygonalization, curvature and normals. Figures a and b illustrate the polygonalization process. Figures c and d show osculator circles used in figure e to compute curvature=1/R and extract the normal as an extension of the line handling the radius.

- 668 -

Figure 1 illustrates the discrete parameters computation. We suppose, at this stage, that for each point Pi , Pj0 we know its curvature, its curvilinear coordinates and its normal. We will now quantify curves matching, for this reason, lets consider a mapping M of the two curves: M : [0, n] −→ [0, m], M (Pi ) = Pj0 . Our goal is to minimize aPmeasure of similarity on this mapping. Lets define this measure by φ[M ] = F (Pi , Pj0 ) where F is similarity cost function. F is computed by considering the distance parameter between each pair of points, the curvature difference and respective normal vectors angle. We have to precise how should F be quantified: F [0, n] × [0, m] −→ R+ F (Pi , Pj0 ) = d(Pi , Pj0 ) + κ(Pi , Pj0 ) + α(Pi , Pj0 ) Note that in order to make more robust and efficient optimization, we chose to add to coefficients k1 and k2 ²[0, 1] in order to control the influence of d(Pi , Pj0 ) and κ(Pi , Pj0 ) respectively in the optimization. By considering k1 = 1 − k2 : F (Pi , Pj0 ) = k1 d(Pi , Pj0 ) + k2 κ(Pi , Pj0 ) + α(Pi , Pj0 ). For more details see our previous work [1].

3

Matching optimization

This stage is important to get correspondence for every point Pi in C with Pj0 in C 0 . Before proceeding with the optimization process, based on dynamic programming, lets define the structure which will be used. This structure, as shown in figure 2 is a grid of intersections expressing the cost F (Pi , Pj0 ). So we can consider that our grid is n × m elements where each intersection of the axes joining Pi and Pj0 is the cost of matching Pi to Pj0 . This structure contains twice the same matrix for circular matching reason. When we suppose for example that P0 is matched to P00 , we update all other costs according to this choice. It is important to notice that the global cost function is monotic in the 0 sense that we could not match point Pi+1 with Pj−1 if we matched before Pi 0 to Pj as illustrated in the right part of figure 2. That is to say that the update for a cost between a pair of points (Pi , Pj0 ) could only be done by adding 0 0 min(cost(Pi−1 , Pj−1 ), cost(Pi−1 , Pj0 ), cost(Pi , Pj−1 )). This is the application of the direct acyclic graph (DAG). The idea is to find a path which permits the matching of all chosen points of polygonalization with the minimum cost [13]. The minimum cost path determines the pairs of points to be matched while going up and picking the positions i and j. We made tests on simple and complex shapes. For each pair of shapes, we proceed by a registration process, then we compute the intrinsic properties and

- 669 -

Fig. 2. Matching optimization with directed acyclic graph.

finally we search the best path candidate to join all points forming the corresponding contours. We can modify the final result, by modifying the associated weighted parameters to curvature and curvilinear coordinates. The complexity is linear in the registration and parameters computing processes but logarithmic in the optimization stage. Figure 3 illustrates the result we obtain on the examples of figure 1 (left) and a more complex shape matching (right).

Fig. 3. Simple and complex shapes matching examples.

4

Model extraction

The principal idea of the point distribution model (PDM) is that each form can be defined with a set of points. This form and its model of deformation,

- 670 -

computed over a base of the form, can be statistically studied. As we detailed in the previous sections, each form is described by a set of outline landmarks (xi , yi ). Lets define each form from the set by X1≤i≤n = (x1 , y1 , ..., xp , yp )T . As we said before, Xi and Xj6=i could have different sizes (number of landmarks). This is not ideal for statistical analysis. The matching process, already describe, resolves this problem by finding for each point pi of Ci (contour of Xi ) a point p0j in Cj0 (contour of Xj0 ) in a context of best fitting. If we suppose that pi is matched to two points p0j and p0k of Cj0 , pi should be twice added in a new vector ˆ i rather than Xi . Thus, we can construct new vectors X ˆ i and Xˆ 0 j with: X ˆ i ) = size(Xˆ 0 j ) = max(size(Xi ), size(Xj0 )) size(X This procedure can be generalized to n shapes by choosing size(X1≤i≤n ) = max(size(X1≤i≤n )). All new vectors will have the same size and are ready for the statistical analysis. The linear statistical analysis consists in transforming the original data within a linear transformation optimizing a geometrical or statistical criteria. The principal components analysis (PCA) is one of the most known methods based on statistics of second order, works of Cootes and al. [14] give more details on mathematical bases. We include here these results and adapt them to our problem. PCA is a factorial method of multidimensional data analysis. It decomposes a random vector V into uncorrelated components, orthogonal and best fitting the V distribution. These components are decreasingordered. N representatives of X create a cloud of n points called observations. In a geometrical context, the decomposition of X is defined by the eigen vectors of the covariance matrix C: n

C=

n

X 1X ¯ ¯ T ¯ (Xi − X)(X Xi i − X) , X = n i=1 i=1

¯ is the mean shape. C could be diagonalized to give (φi , λi )i=1,...,p , where X where φi are the eigen vectors and the λi are the eigen values. Hence, an observation X is defined by: ¯ + φb X=X where b = (b1 , ..., bp )t is the observation vector in the new base: b = φt (x − x ¯) The obtained decomposition can approximate the observations and quantify the error. We have just to select a number m, m ≤ p, of modes in the modal base to reconstruct an observation X: ¯ + φm bm X=X where φm is a sub matrix p × m of φ containing m eigen vectors and bm = (b1 , ..., bm )T is a representation in a m-dimensional space defined by the m principal components. New instances, plausible to learned observations, can be created

- 671 -

by choosing b in the limits. Typically, the variation interval [14] is: √ √ −3 λi ≤ bi ≤ +3 λi The new shape in figure 4 is created from the mean shape by combining two modes of variations.

Fig. 4. Basic idea of the algorithm: according to mean shape and principal modes of variations compute a model able to generate plausible new shapes.

Before carrying tests, some points require to be seen. The first point is that the number of segments obtained by polygonalization can vary according to the first chosen point. At worst of cases, we will have a segment more (i.e. a point more). This is not a problem considering that the matching deals with shapes not having the same number of points on their contours(landmarks). And on another side the contour of a form can be reconstructed, with the same way, from two different polygonalizations without any loss. A second aspect of our approach is that it is very fast compared to others considering that polygonalization is useful for encoding contour of a shape, is also used to deduce the curve in any point as well as the normal.

5

Results

Starting with a learning stage within a set of fishes, we proceed by the already described steps. We align shapes (scaling, rotation and translation) according to a common referential so that the sum of distances of each shape to the mean P ¯ 2 is minimised. The mean shape is initialized equal to the first D= |Xi − X| shape and updated each time we introduce a new one. We compute then different discrete parameters after a polygonalization process applied to each shape and the mean one. We encode thus all the shapes (their landmarks) by curvatures and normals. We denote here that the curvilinear abscissa is used to course contours.

- 672 -

Matching optimization is then done between shapes according to the mean shape. When all shapes are best matched, we compute the covariance matrix and we diagonalize it to obtain eigen values and vectors. The eigen properties describe how shapes vary linearly from the mean one. Eigen vectors give the directions of this variation and eigen values give the proportion. By analyzing different eigen values, we extract important modes of variation and we generate the model according to mean shape and the kept modes of variations. We keep always a number of modes which covers more than 90% of variations. With a set of fishes, a sample is shown in figure 5, we kept 24 modes of variation and by varying each mode, we obtain results in figure 6.

Fig. 5. A sample of learning data: a set of fishes.

Fig. 6. New statistical first row: new models according to the first √ generated models: √ mode of variation −3 λ1 ≤ b1 ≤ +3 λ1, the second row results from the second mode of variation and the final row is a combination of some retained modes of variation.

- 673 -

6

Conclusion

We have presented a 2D shape model construction method based on shape outlines. The approach is discrete and independent of scaling and initial positions of objects. Our contribution is basically at the process of matching in opposition to methods based on manual landmarking or uniform subdivision of contours. In the proposed method, the points result from a polygonalization process that allows object reconstruction and is adapted to any 2D shape. Note that noise could be a source of inefficiency specially in polygonalization process, but this can be avoided by fuzzy polygonalization [15]. Principal components analysis allows to generate models linearly, but has also its own limits. For any particular mode of variation, the positions of landmark points can vary only along straight lines. Non-linear variation is achieved by a combination of two or more modes. This situation is not ideal, firstly because the most compact representation of shape variability is not achieved, and secondly because implausible shapes can occur, when invalid combinations of deformations are used. Attempts have been made to combat this problem. Sozou et al’s Polynomial Regression PDM [16] approach allows landmark points to move along combinations of polynomial paths. Heap and Hogg’s Cartesian-Polar Hybrid PDM [17] makes use of polar coordinates to model bending deformations more accurately. Sozou et al [18] have also investigated using a multi-layer perceptron to provide a non-linear mapping from shape parameters to shape. We could use these researches to improve our method. Finally, extension to 3D is under study in spite of problem of landmark order that will not be determined in a similar way between shapes.

References 1. Boukhriss, I., Miguet, S., Tougne, L.: Tow-dimensional discrete shape matching and recogntiion. IWCIA (2006) 2. A. L. Yuille, D.S.C., Hallinan, P.: Feature extraction from faces using deformable templates. International Journal of Computer Vision 8(2) (1992) 99–112 3. Staib, L.H., Duncan, J.S.: Boundary finding with parametrically deformable models. IEEE Transactions on Pattern Analysis and Machine Intelligence 14(11) (1992) 1065–1075 4. M. Kass, A.W., Terzopoulos, D.: Snakes: Active contour models. In 1st International Conference on Computer Vision (June 1987) 259–268 5. Goodall, C.: Procrustes methods in the statistical analysis of shape. Journal of the Royal Statistical Society 53(2) (1991) 285–339 6. Bookstein., F.L.: Principal warps: Thin-plate splines and the decomposition of deformations. IEEE Transactions on Pattern Analysis and Machine Intelligence 11(6) (1989) 567–585 7. J. Park, D. Mataxas, A.Y., Axel, L.: Deformable models with parameter functions for cardiac motion analysis from tagged mri data. IEEE Transactions on Medical Imaging 15 (1996) 278–289 8. Pentland, A.P., Sclaroff, S.: Closed-form solutions for physically based modelling and recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 13(7) (1991) 715–729

- 674 -

9. Turk, M., Pentland, A.: Eigenfaces for recognition. Journal of Cognitive Neuroscience 3(1) (1991) 71–86 10. Jones, M.J., Poggio, T.: Multidimensional morphable models. In 6th International Conference on Computer Vision (1998) 683–688 11. Sebastian, B., Klein, P.N., Kimia, B.B., Crisco, J.J.: Constructing 2D Curve Atlases Thomas. IEEE Workshop on Mathematical Methods in Biomedical Image Analysis (2000) 70–77 12. Debled, I., Reveilles, J.P.: A linear algorithm for segmentation of digital curves. Third International Workshop on Parallel image analysis (1994) 13. Mathworld: Dag. http://mathworld.wolfram.com/AcyclicDigraph.html 14. Cootes, T., C.J.Taylor: Statistical models of appearance for computer vision. Technical report, Imaging Science and Biomedical Engineering, University of Manchester, http://www.isbe.man.ac.uk (March 2004) 15. Rennesson, I.D., Remy, J.L., Rouyer, J.: Segmentation of discrete curves into fuzzy segments. IWCIA 12 (2003) 16. Sozou, P., Cootes, T., Taylor, C., Di-Mauro, E.: A non-linear generalisation of pdms using polynomial regression. Volume 2. (1994) 397–406 17. Heap, T., Hogg, D.: Extending the point distribution model using polar coordinates. In: Computer Analysis of Images and Patterns. (1995) 130–137 18. Sozou, P., Cootes, T., Taylor, C., Di-Mauro, E.: Non-linear point distribution modelling using a multi-layer perceptron. Volume 1. (1995) 107–116

- 675 -