Augmented Reeb graphs for content-based retrieval of 3D ... - CiteSeerX

Augmented Reeb graphs for content-based retrieval of 3D mesh models Recherche par le contenu de maillages 3D par graphes de Reeb augmentés

Tony Tung Francis Schmitt

2003D010

2003

Département Traitement du Signal et des Images CNRS UMR 5141

Groupe Traitement Traitement et Interprétation des Images

Dépôt légal : 2003 – 4ème trimestre Imprimé à l’Ecole Nationale Supérieure des Télécommunications – Paris ISSN 0751-1345 ENST D (Paris) (France 1983-9999)

Augmented Reeb Graphs for Content-based Retrieval of 3D Mesh Models Recherche par le Contenu de Maillages 3D par Graphes de Reeb Augment´es Tony Tung

Francis Schmitt

Ecole Nationale Sup´erieure des T´el´ecommunications D´epartement TSI , CNRS UMR 5141 46, rue Barrault, 75634 Paris Cedex 13, France {TONY. TUNG , FRANCIS . SCHMITT}@ ENST. FR Abstract This article presents an improved method of 3D mesh models indexing for content-based retrieval in database with shape similarity and appearance queries. The approach is based on the Multiresolutional Reeb Graph matching presented in [Hilaga et al, SIGGRAPH’01]. The original method only takes into account topological information what is often not sufficient for effective matchings. Therefore we proposed to augment this graph with geometrical attributes. We also provide a new topological coherence condition to improve the graph matching. Moreover 2D appearance attributes and 3D features are extracted and merged to improve the estimation of the similarity between models. Besides, all these new attributes are user-dependent as they can be weighted by variable terms. We obtain a flexible multiresolutional and multicriteria representation called Augmented Reeb Graph (ARG). Good preliminary results have been obtained in a shape-based matching framework compared to existing methods based only on statistical measures. In addition, our study lead us to an innovative detail matching scheme based on the same approach as our Augmented Reeb Graph matching. R´esum´e Nous pr´esentons dans cet article une m´ethode am´elior´ee d’indexation d’objets 3D dans des bases de donn´ees avec requˆete par similarit´e de forme et d’aspect. L’approche repose sur l’appariement de graphes de Reeb multir´esolution present´es dans [Hilaga et al, SIGGRAPH’01]. La m´ethode originale ne prenait en compte que l’information topologique, ce qui est souvent insuffisant pour obtenir des appariements pr e´ cis. C’est pourquoi nous proposons d’enrichir le graphe a` l’aide d’attributs g´eom´etriques. Nous avons aussi ajouter une condition de coh´erence topologique suppl´ementaire pour am´eliorer leur mise en correspondance. De plus, des attributs visuels 2D et des caract´eristiques 3D sont extraits et fusionn´es pour am´eliorer l’estimation de la similarit´e entre les mod`eles. Par ailleurs, ces nouveaux attributs sont pond´erables selon le type de requˆetes d´esir´ees par les utilisateurs. Nous obtenons une repr´esentation souple, multi-crit`ere et multir´esolution appel´ee Graphe de Reeb Augment´e (GRA). Dans le cadre d’une recherche par similarit´e de forme, de bons r´esultats pr´eliminaires ont e´ t´e obtenus compar´es aux m´ethodes existantes bas´ees uniquement sur des mesures statistiques. En compl´ement, notre e´ tude nous a amen´e a` un sch´ema innovant d’appariement de d´etail reposant sur la mˆeme approche que les Graphes de Reeb Augment´es.

1. Introduction The development of new 3D applications has become easier and tend to generalize due to the permanent progress of numerical technologies. Database of 3D objects has begun to proliferate for multimedia applications [21] as well as for more industrial ones such as in CAO (e.g. motorcar companies). 3D objects indexing and content-based search appear as a natural answer for those who want to navigate in such huge database. As a consequence, queries have to be adapted to this new kind of data. Scientific litterature only presents few methods dealing with 3D shape similarity queries. The techniques derived from the 2D, such as color histogram or texture distribution taken from texture maps or from color images of the acquisition sequences of the object, are not well adapted to 3D queries. The approaches such as curvatures histograms [8] or EGI (Extended Gaussian Image) and its variants [9, 10] describe too locally the geometry of the objects. As a matter of fact they do not keep any topological information (mesh connectivity) and are highly sensitive to mesh modifications (noise on vertices position, decimation, etc...). In the other hand, global approaches such as cords histograms [5] or shape distributions [6] works quite well for the matching of primitive shapes but produce a too coarse description of the 3D objects shape to be a sharp retrieval tool when comparing similar complex shapes. Similar comments can be done for the Fourier based descriptors and moments calculations [3, 7]. To obtain an intuitive description of the shapes, the use of a skeletal or graph representation is very attractive. These approaches permit us to keep the topology of the objects which is a rich information for matching purpose. Infortunately, difficulties lies in the construction of the skeleton which is either time consuming, too sensitive to noise on the shape surface, or need a source point which favour a direction. Besides, most of the time no matching sheme is provided [11, 12, 13], or no multiresolutional approach [14, 15] whom the properties are useful for accurate similarity calculations and for various applications as presented in this article. The Multiresolutional Reeb Graph (MRG) [1] appears to be a very promising starting point of research, in particular because it is an efficient mean to exploit different techniques and merge the features as we will show in the next sec-

tions. The Reeb Graph of a 3D object is represented by a unidimensional skeleton. It is built using a fonction µ defined over the object surface considered as a compact manifold, and therefore relies on the mesh connectivity. The graph is robust in case of mesh modification thanks to the assumed integral function over the whole object [1]. The surface of the object is divided in regions according to value intervals of µ, and a node is associated to each connected component of the regions. The graph structure is then obtained by linking the nodes of the connected regions. Afterwards the MRG is constructed hierarchically from the finest level of resolution to the coarsest level. And similarity calculation between MRG are done from the coarsest level to the finest level. Keeping the advantages of the multiresolutional representation offers by the MRG, we have provided topological, geometrical and visual (colour and texture) informations to the node graphs to obtain a hierarchical description of the models, which is both global and local and is called Augmented Reeb Graph (ARG). Therefore the 3D objects indexing in database will consist on coding in feature vectors the information associated to the nodes of the Reeb graphs. Tests of shape based retrieval were performed uppon a database of high resolution 3D models [20, 21] and preliminary experiments were performed concerning the application of detail matching. The MRG construction is presented in the next section. The graph matching procedure and the choice of the nodes attributes are shown in section 3. Section 4 deals with the similarity calculation, and the results performed on our database are described in section 5. The detail matching scheme is presented in section 6. And we conclude in section 7 with a discussion about the possible improvement of the methods.

2. Multiresolutional Reeb Graph overview According to the Morse theory, a continuous function defined on a closed surface characterize the topology of the surface on its critical points [2]. Therefore, a Reeb graph can be obtained assuming a function µ calculated over the 3D object surface. The multiresolutional aspect results from the dichotomic discretization of the function values and from the hierarchical collection of Reeb graphs defined at each resolution.

2.1. The function µ Our study concerns objects which are 3D compact manifold meshes of triangles with vertices located in a cartesian frame R(x, y, z). The authors of [15] present different continuous functions which can be used for the Reeb graph construction (cf. Figure 1). Assuming a point v(x, y, z) on the surface S of an object, the height function µ defined as µ(v(x, y, z)) = z is well adapted for elongated stand models as human representations. This function is easy to compute but by definition has a strong orientation dependence. Therefore it is not suited for objects for which the points dispersal is perpendicular to the axis z. The function µ defined as the geodesic distance from curvature extrema is based on growing regions associated to local Gaussian curvatures on seed vertices. The results highly depend on the position of the seeds and require the local curvature calculation which is not obvious on every objects, especially those coming from the internet. The function defined as the distance of a surface point v to the center of mass G of the object µ(v) = d(G, v), where d is the Euclidean distance, is easy to calculate and stable to small noise. In the other hand, this function provides a Reeb Graph which is not flexible enough for our study, as explained next. The

g(v, p) from v to the others points {p|p ∈ S}: µ(v) =

Z

g(v, p)dS. p∈S The authors [1] chose to normalize the function with maxp∈S µ(p) for the denominator. To exploit the full dynamic of µ(v) over [0, 1], we have normalized the function with its min and max values: µ(v) − minp∈S µ(p) µN (v) = . maxp∈S µ(p) − minp∈S µ(p) The Reeb graph is obtained by partitionning the object surface following regular intervals of µ values, and linking the connected regions to each other. In each interval, a node is associated to a set of connected triangles (cf. Figure 2). Based on geodesic distance cal-

Figure 2. Reeb graph: 3D model (left), function µN (middle), Reeb graph; edges in green; nodes in red (right). (model from [21])

Figure 1. Example of µ functions: height function (left), distance to center of mass (middle), geodesic distance integral (right).

chosen function µ in [1] has rotation invariance property and is defined as the sum of the geodesic distance

culation, the resulting Reeb graph of a human model with arms lifted up in the air and a human model with arms along the body will look similar as maximal values of µ stay at the extremity of the arms. On the other hand, the function µ defined as the distance to the center of mass will lead to a different graph. As a matter of fact, it can be stated that the choice of the function µ depends on the user’s queries. 2.2. Multiresolutional aspect A graph of double or half resolution is respectively obtained by the division or the fusion of nodes [1]. A parent node generates n children nodes with n ≥ 1 at a higher level of resolution (cf. Figure 3).

belong to. Therefore neighbouring nodes belonging to a same branch have the same label. In this way, we ensure that a branch is entirely matched to a branch [1].

Figure 3. MRG construction by halving µ intervals. Parents nodes (left) have children nodes of the same color (right).

3. Augmented Reeb Graph matching The matching strategy of two MRGs is based on the detection of nodes with similar topological configurations in term of connections to their neighbours nodes and parents/children relations. The aim is to maximize the number of corresponding nodes. For this purpose a set of topological consistency criteria were stated. The pairs of nodes which verify the criteria are candidates for the matching. And a loss function determines the best matching nodes among the candidates. To improve the matching of the original MRG method, we have introduced an additive topological consistency criteria and added geometrical attributes to the nodes. Finally a function estimates the similarity between the two Augmented Reeb Graphs. 3.1. Addititional topological consistency We consider the graph comparison of two objects M and N . Topological consistency criteria permit to find the pairs of nodes (m, n), with m ∈ MRG(M ) and n ∈ MRG(N ), which could possibly be matched afterwards in the procedure. The nodes m and n are topologically consistent if: • the parents of m and n have been matched at the previous level of resolution • m and n are in the same interval of the fonction µN • m and n have the same label, meaning they both belong to branches which have been matched to each other. In deed, when two nodes are matched, a same label is propagated along the branchs they

It is also assumed that a node can be matched to a group of topologically consistent nodes [1]. However the simple statement of these criteria is not sufficient to ensure that two branchs will still be matched together when the resolution gets higher (cf. Figure 4). As a consequence, we add a new topological consistency criterion: if the parents of the neighbours of m and n have been matched, then they must have been matched together.

Figure 4. Matching nodes (left). At a higher level of resolution, branchs mixing can happend. To avoid this, the matching of the parents of the neighbours have to be tested (right).

3.2. Geometrical attributes From a topological point of view, the matching of the graphs could be acceptable. But in our framework these matching are not well adapted, because topologically speaking an arm worth a leg (they are limbs linked to the body), whereas we would like to get more accuracy in the matching to efficiently compare complex 3D models. The topological consistency criteria are such defined that if two nodes are not matched together at the coarsest level of resolution, whereas they should be, then there is no chance that their children nodes will be matched together at a finer resolution. Therefore it is crucial that nodes are matched the best they can from the beginning of the consistency tests. For this reason, we propose to provide localisation information to the nodes. A node m is associated to a set of connected triangles, so it can be located at their barycentric po-

sition (cf. Figures 2 and 3). We choose a spherical coordinate representation (r, θ, ϕ): • r(m) ≥ 0 is the distance to the node at the centre of the frame, • θ(m) = arctan • ϕ(m) = arccos

y
x z r




∈ [− π2 , π2 ], ∈ [0, π].

Using the example of a human model (cf. Figure 5) whom the axis x would get from the bottom to the top of the body, axis y oriented from left to right and axis z from behind to front, we can observe that θ allows to identify the left from the right, ϕ distinguish arms from legs and r could be decisive in a ambiguous configuration (an arm up in the air could be taken as an head). It becomes imperative that objects are oriented

a is the area of the set of triangles associated to a node. l stands for the complexity of a node and is linked to the number of its children as described in [1]. α, β, γ, δ and  are weighting terms which influence the attributes according to the type of desired queries. This aspect gives lots of freedom to the matching of nodes. Associated to the chosen function µ defined as the integral of geodesic distance, the flexibility of the method can be illustrated by the possibility to choose either to match models having exactly the same shapes, or to let more freedom to the matching and allow the matching between approximative similar shapes (typically two human models in different positions). And this can be done by simply adapting the weighting terms to the desired queries (cf. Figure 6). Besides, it is also possible

Figure 5. Nodes located in spherical coordinates (model from [21]).

towards the principal directions (x, y, z). For this purpose, we decide either to use the natural axis of the object centered to the centre of mass with a coordinates normalization, or to apply a classical spatial alignment technique such as the Principal Component Analysis to turn the object towards the principal axis [3]. 3.3. The loss function The loss function loss(m, n) quantifies the difference between two nodes m and n. It is applied on every groups of topologically consistent nodes and returns those which will be effectively matched by minimizing the function: + β |l(m)−l(n)| + loss(m, n) = α |a(m)−a(n)| 2 2 |r(m)−r(n)| |θ(m)−θ(n)| |ϕ(m)−ϕ(n)| γ +δ + . 2 2π 2π

Figure 6. Influence of the weight parameters on the matching. Users can choose if they want to match arms that are in the same position or not (models from [21]).

to replace the absolute values used to calculate the differences by quadratic distances, or any other dedicated functions to emphasize the differences or similarities between two nodes. Figure 7 shows the gain obtained in the matching of two graphs by introducing the localisation information in the nodes.

δ · min(θ(m), θ (n)) + ε · min(ϕ(m), ϕ(n)). To improve the retrieval of similar objects and to enlarge the context to queries with other aspects (geometric, colorimetric, etc...), we merge geometric features, global and local, and visual features extracted from the section of the object associated to each node m: • v(m) is the relative volume associated to m, • cords(m) is a statistic measure of the extent of the surface region associated to m, • curvm (m) and curvg (m) represent the mean and gaussian curvatures obtained on the triangulated surface associated to the node m,

Figure 7. Improvement of the graphs matching thanks to geometrical attributes: on the right, the arms and the head are well matched (models from [21]).

4. Similarity calculation Assuming the list of matching nodes, similarity functions will measure the similarity between the 3D models. First, a function calculates the similarity between each matched nodes. The original function introduced by [1] has been enriched with various features extracted from the mesh models. Using the same approach as the geometrical attributes, these 2D/3D extracted features are weighted to satisfy different types of queries and to significantly refine the similarity calculus between two objects. Then we obtain the Augmented Reeb Graph representation. The next sections present the new nodes similarity function, the introduced measures (relative volume, cords lengths, curvatures and colour) and the resulting graphs similarity function. 4.1. Similarity of nodes The calculation of the similarity function between two nodes sim(m, n) takes into account the same attributes as for the loss function loss(m, n): sim(m, n) = α · min(a(m), a(n))+ β · min(l(m), l(n)) + γ · min(r(m), r(n))+

• color(m) is a quantification of the texture/colour associated to the node. Features are extracted at the finest level of resolution and associated to the corresponding nodes. The multiresolutional aspect of the measures is ensured by the hierarchical relation between parents and children nodes. Our idea is to provide to the nodes all of the relevant characterizations that could be calculated on a set of triangles. Our choices may not be exhaustive but reflect the well-known existing techniques. Moreover, it is possible to favour one or several measures by adaptating the weighting terms. This again, let a lot of freedom to the queries of users. 4.2. Volume calculation Let’s consider the interval [µ1 , µ2 ] of the function µ associated to the node m. The relative volume v(m) is defined as: 1 vol(m) v(m) = · . R vol(S) vol(S) represents the total volume of the model, and vol(m) stands for the volume of the section of m. vol(S) is the sum of the tetrahedra formed by the oriented triangles of the mesh and the centre of mass of the object [4]. vol(m) is obtained by summing 1) the volume of the tetrahedra formed by the triangles associated to m and the barycenter of these triangles, 2) the volume of the tetrahedra formed by this barycenter, an edge of the mesh corresponding to the surface edge µ = µ1 associated to m, and the barycenter of the points associated to these edges, 3) the symetric volume for the surface edge µ = µ2 (cf. Figure 8). The

on Figure 9 that objects contains meaningful local informations to characterize the geometrical variations. The mean curvature is calculated on every points P of

Figure 8. Volume associated to a node obtained by summing the volume of the three types of tetrahedra included in the section.

factor R1 corresponds to the R level of resolution of P ARG(M ), leading to the relation m∈M v(m) = 1. Obviously, this volume calculation method is valid only if triangles can be oriented. Besides, an additive feature of the object section associated to the node m 3 could be the measure a(m) which has no dimension. v(m)2 4.3. Cords measurement The cords measure introduced in [5] is represented by a normalized histogram of the distance from the triangles centres to the centre of mass of the object. Applied to a node n at the finest level of resolution, the histogram cord(n) contains the distances between the triangle centres associated to the node n, and the position of triangle barycenter. The parent node m at coarser resolutions is thus deduced by applying the following formula: cords(m) =

X 1 1 · · R Nchild (m) n∈{children of

cords(n), m}

where Nchild (m) represents the number of children of m. This statistical measure gives the information on the extent of a set of triangles. Hence it is a signature of the mesh geometry associated to the node. The use of statistical methods based on histograms to describe an object, as the cords [5] or the shape distribution [6], is well adapted for simple shapes. In our context and from a certain level of resolution, nodes are mainly locally associated to shapes such as cylinders or paraboloids which justify the relevant use of these geometrical measures. 4.4. Curvatures measurement Local geometric properties of the surface are represented by mean and gaussian curvatures. We can see

Figure 9. Gaussian curvature calculated on each vertex of the mesh (model from [20]).

the mesh, according to the usual formula, by summing the angles (θN )i of the normals of the dihedra formed by the adjacent faces to P (i describes the N ei adjacent edges to P). Angles are weighted with the third of the mean area (Sm )i of the two faces (barycentric area): 1 X curvm (P) = N ei i

(θN )i 1 3 (Sm )i

!

,

The gaussian curvature at a point P is calculated using the angular deficit formula: curvg (P) =

2π − 1 3

P

P

j

j θj

Sj

,

Sj being the area of triangle j where P is one of its vertices, and θj is the angle at vertex P in the triangle j. The mean of these curvatures is calculated for each node at the finest resolution and afterwards propagated to the parents: curv∗ (m) =

X 1 1 · · R Nchild (m) n∈{children of

curv∗ (n), m}

where ∗ represents m or g. It is also possible to use only one histogram to represent the local curvature by combining the two curvatures (cf. Koenderink shape index [17, 8]) or to use any other characterization of local curvature [18, 19].

4.5. Texture extraction

• λ2 · (1 − kcords(m) − cords(n)k),

When browsing in a database of 3D objects, it is also useful to make queries uppon the texture or the colour of the models. The conjoint use of the shape and the texture is a highly discriminative criterion, but is still not much exploited in 3D. For this purpose, we use the information included in the texture maps associated to the 3D objects. These 2D features will naturally be merged to the 3D features previously presented. Our colour quantification is done using three normalized histograms on (R, G, B) (cf. Figure 10). The quantification of the texture is obtained locally

• λ3 · (1 − kcurv∗ (m) − curv∗ (n)k),

Figure 10. Colour histograms of the set of triangles associated to an ARG.

for each node of the ARG, and then hierarchically and globally for each level of resolution: color(m) =

X 1 1 · · R Nchild (m) n∈{children of

color(n). m}

4.6. Objects similarity Assuming two objects M and N , the function used to calculate their similarity is obtained by summing the similarity function of the matching pairs of nodes {m, n}: SIM (M, N ) =

X

sim(m, n).

{m,n}

The function SIM is maximal for identical objects. The 2D/3D features presented in the previous sub-sections have been introduced in the function sim(m, n), with weight parameters (λ1 , λ2 , λ3 , λ4 ) in accordance with the desired queries: • λ1 · (1 − kv(m) − v(n)k),

• λ4 · (1 − kcolor(m) − color(n)k), where the k.k norms can be Minskowski distances. Finally, we have obtained a flexible multiresolutional 3D indexing scheme, both global and local, and using various criteria illustrated by the different features extracted (topological, geometrical, and colorimetric).

5. Results For each object, the ARG structure with the node attributes are coded in a binary file. Its size is directly linked to the complexity of the object (linarly to the number of nodes). The ARG files stand for the feature vectors in the indexing framework, and their comparison in a database of 50 objects take only 1 to 2 seconds with a PC P4-2GHz. Vectors are computed in acceptable delay (∼10s for an object with 40000 triangles), the most time-consuming being the computation of the fonction µ over the object surface. For that we use the Dijkstra algorithm with a complexity in O(N log(N )), N being the number of vertices of the mesh. The 3D model database was created by reconstructing collection objects [16, 20], and downloading objects from the internet citefoo:3dfanta. All models have a high resolution (10000 a` 50000 triangles), have closed surface and oriented triangles. To show the advantages of the Augmented Reeb Graph, we have made our similarity tests between different classes of objects (cf. Figure 11). The particularity is that these classes could be easily confused by common approaches: statues, human models, elongated vases. Statues are representations of human beings but don’t look like the models of the human class. Some of them have a pedestal, a hat or miss an arm, etc. Vases have two handles and are thinner at the bottom part. It results that the ARG method is far more efficient to retrieve object classes, as can be seen on Figure 12, whereas the other tested methods make a confusion between statues and human models (cf. Figure 13). Those with the best results (Complex EGI [10], ratio S/V [4]) make a difference between the vases and the other classes. However on another set of vases, they are not efficient enough to distinguish the difference between vases with one or

Figure 11. Three classes of objects were used for our tests (models from [20, 21]).

Figure 12. Results of the similarity calculation on 15 models using the ARG matching method: similar models are retrieved and classes are separated.

two handles, whereas the ARG is. The Figure 14 illustrates the Reeb graph for one object of each class. We understand that the genus of the object is the main key of the graph matching. Besides, it is interesting to see that the similarity function can be a useful tool to compare objects with similar shapes, especially when associated to 2D and 3D features. For example, they could establish measurements between two objects depending on their volume, curvatures or visual differences. Moreover, it is also possible to use these informations to make classifications with respect to any of the criteria offer by the extracted features (color,size,...). In fact, as the technology of the computer keeps on growing faster (CPU, memory and HD size), Augmented Reeb Graph will allow us to integrate and merge all of the future most relevant shape descriptors.

6. Application to detail matching Our shape matching method has both the properties of a high-level descriptor thanks to the global representation with the multiresolutional Reeb graph, and a low-level descriptor with the local 2D and 3D features extracted at each level of resolution and for each node, especially at the finest level. This multiresolutional as-

Figure 13. Results of the four other tested methods: human models and statues are mixed.

pect can be exploited for many different purposes. Our current research work lead us to a promising application which consist on using the enriched nodes for de-

ideas presented in this article. We exploit the multiresolutional aspect and the topological consistency criteria of the nodes for matching shape details, and extract 2D/3D features to retrieve the most similar node. Detail matching scheme

Figure 14. Reeb graph of a human model, a vase and a statue at level of resolution 4 (models from [20, 21]).

tail matching as illustrated in Figure 15. As a matter of fact, the information included in the graph nodes are discriminant enough to be retrieve in a database. Moreover, the different representations of the nodes at each level of resolution, which include their children nodes, can ensure the unicity of the matching. An

Initialization: 1. Choose an object M . 2. Choose a level of resolution. 3. Choose the node m at this level of resolution which corresponds to the detail to be matched with other details in the database. Matching: 4. Compute the similarity function between m and every nodes n at the same level of resolution (or eventually at other close levels) among all the objects N in the database. 5. Test topological consistency on nodes similar to m with respect to a predefined threshold for the similarity function. 6. We consider now the subgraph of m and of all its children as the ARG of the detail of interest, that is the ARG of the truncated object region associated to the node m. We consider similarly the subgraphs of all the consistent nodes n as the ARGs of the details corresponding to nodes n. Then we apply the general matching algorithm described in section 3 to these subgraphs in order to determine the most similar details to the one chosen in M as query. For detail matching, two nodes m and n are topologically consistent if:

Figure 15. Nodes taken from Reeb graphs for detail matching (models from [21]).

approach of part matching was presented in [14], but the skeletal representation of the objects was not well adapted for this purpose. The lack of a multiresolutional strategy is obvious as a part of a skeleton could be a too coarse description (a cylinder could match anywhere). Our detail matching scheme relies on the

• the parents of m and n have been matched at the previous level of resolution, • m and n have the same branch label as defined in section 3.1, • if the parents of the neighbours of m and n have been matched, then they must have been matched together. The criteria on the intervals of µN is not use for the detail matching. Concerning the extracted features, only

the information relative to the node are taken into account, and not the information relative to the entire object. Then each node m contains the following information: • a(m) is the area of the triangle set associated to m, • vol(m) is the relative volume as defined in section 4.2 (the volume is no more normalized to the total volume of the model), • l(m) = max(m) − min(m) where max(m) and min(m) are the maximum and minimum of µN in m. We also add the attributes defined in the previous sections: cord(m), curv∗ (m) and color(m). The simililarity calculation between nodes for detail matching remains the same as defined in section 4.6, with weight parameters to be set by the users with respect to the queries.

7. Conclusion Keeping in sight our problematic of textured 3D object indexing and content-based retrieval, our study lead us to enhance the approach presented in [1] by introducing the Augmented Reeb Graph. Geometrical and colorimetric attributes are added to the topological consistency criteria which are also extended. The loss function calculation is consequently modified, and the similarity function is improved by merging the different 2D/3D features (geometrical and visual). Then, we obtain a multiresolutional and multicriteria shape descriptor. Our results demonstrate the ability of the ARG to retrieve similar objects classes in a database, and particularly objects with long appendices. However this method seems not so well adapted for compact objects as busts or faces. And actually the function µ works with meshes that have only one connected component, thus the objects should have been correctly modelized. Further works could be done to improve the characterization of the texture. For example by unfolding the texture associated to each node (patch), and reparametrizing it to extract spatial or frequency informations. Our method aims the interactivity and

the flexibility, but requires lots of parameters (weight terms of the different features). As a consequence, it could be interesting to apply optimisation techniques on parameters to define default values in case of specific applications for which learning sets could be provided. Perspectives of research also concern the choice of the function µ and the construction of the Reeb graph. It seems that methods using a robust graph (which allows topological perturbation) such as the Extended Reeb Graph [15] associated to our approach of Augmented Reeb Graph (with its multiresolutional aspect), could lead to interesting results. Also, other applications such as detail matching seems accessible with our approach but has not been developped yet and remains to be fully tested. Finally, we would like to point out again the flexible and robust aspects of our approach, whom the effectiveness and accuracy rely on the fusion of different techniques.

Acknowledgements This work has been partially supported by the SCULPTEUR European project IST-2001-35372 [20]. The authors would like to thank Pierre-Alexandre Pont for having coded the algorithm of [1] and for the additionnal topological consistency criterion, and Bianca Falcidieno and Marco Attene for their precious advice concerning the function µ.

References

http://www.tsi.enst.fr/3dmodels [17] J. Koenderink. Solid Shape, The MIT Press, 1990.

[1] M. Hilaga, Y. Shinagawa, T. Kohmura, T.L. Kunii. Topology Matching for Fully Automatic Similarity Estimation of 3D Shapes, ACM SIGGRAPH, Los Angeles, CA, USA, pp.203212, Aug. 2001. [2] Y. Shinagawa, T.L. Kunii and Y.L. Kergosien. Surface Coding Based on Morse Theory, IEEE Computer Graphics and Applications, Vol.11, No.5, pp.66-78, Sept. 1991. [3] D.V. Vranic, D. Saupe and J.Richter. Tools for 3D-object retrieval: Kahrune-Loeve Transform and spherical harmonics, IEEE Workshop Multimedia Signal Processing, Cannes, France, pp.293-298, Oct. 2001. [4] C. Zhang and T. Chen. Efficient Feature Extraction for 2D/3D Objects in Mesh Representation, ICIP, Thessaloniki, Greece, pp.935-938, Oct. 2001. [5] E. Paquet and M. Rioux. Nefertiti: a Query by Content Software for Three-Dimensional Models Databases Management, 3-D Digital Imaginig and Modeling, Proc. Int. Conf. on Recent Advances, pp.345-352, 1997. [6] R. Osada, T. Funkhouser, B. Chazelle, and D. Dobkin. Shape Distributions, ACM Trans. on Graphics, 21(4), pp.807-832, Oct. 2002. [7] K. Kazhdan and T. Funkhouser. Harmonic 3D Shape Matching, SIGGRAPH Technical Sketches, p.191, July 2002. [8] T. Zaharia, F. Prˆeteux. Indexation de maillages 3D par descripteurs de forme, RFIA, pp.48-57, Angers, France, Jan. 2002. [9] B.K.P. Horn. Extended Gaussian Image, Proc. of the IEEE, Vol.72, pp.1671-1686, 1984. [10] S. B. Kang, K. Ikeuchi. The complex EGI: a new representation for 3D pose determination, IEEE Trans on PAMI, Vol.16, No.3, pp.249-258, Mar. 1994. [11] J.A. Goldak, X. Yu, A. Knight, L. Dong. Constructing discrete medial axis of 3-d objects, International Journal of Computational Geometry and Applications, 1(3), pp.327339, 2001. [12] F. Leymarie and B. Kimia. The Shock Scaffold for Representing 3D Shape, Visual Form, Springer-Verlag, Lecture Notes in Computer Science, no.2059, pp.216-229, 2001. [13] A. Verroust, F. Lazarus. Extracting Skeletal Curves from 3D Scattered Data, The Visual Computer, Springer, 16(1), pp.1525, 2000. [14] H. Sundar, D. Silver, N. Gagvani, S. Dickinson. Skeleton Based Shape Matching and Retrieval, Shape Modelling International, Seoul, Korea, pp.130-142, May 2003. [15] S. Biasotti, S. Marini, M. Mortara, G. Patan`e, M. Spagnuolo, B. Falcidieno. 3D Shape Matching through Topological Structures, 11th Discrete Geometry for Computer Imagery conference, Springer-Verlag, Lecture Notes in Computer Science, no.2886, 2003. [16] C. Hern´andez Esteban and F. Schmitt. Silhouette and Stereo Fusion for 3D Object Modeling, 3DIM, 4th Int. Conf. on 3D Digital Imaging and Modeling, pp.46-53, 2003.

[18] D. Cohen-Steiner, J-M. Morvan. Approximation of Normal cycle, Rapport de recherche INRIA, Th`eme 2, no.4723, Feb. 2003. [19] Takeshi Masuda. Surface Curvature Estimation from the Signed Distance Field, 3DIM, 4th Int. Conf. on 3D Digital Imaging and Modeling, pp.361-368, 2003. [20] http://www.sculpteurweb.org [21] On-line 3D objects database: http://www.fantasticarts.com http://www.3dcafe.com

46, rue Barrault - 75634 Paris Cedex 13 - Tél. + 33 (0)1 45 81 77 77 - www.enst.fr Département TSI

Télécom Paris 2003

Ecole Nationale Supérieure des Télécommunications Groupe des Ecoles des Télécommunications - membre de ParisTech