Dipartimento di Informatica e Scienze dell

Dipartimento di Informatica e Scienze dell’Informazione

Surface Reconstruction: Online Mosaicing and Modeling with Uncertainty by Laura Papaleo

Theses Series DI SI, Un iv er sità d i G enov a V ia Dod ecan eso, 35, 16146 G enov a, I taly

DISI-TH-2004-XX

h t t p: / /ww w . di s i.u n i ge . it

Università degli Studi di Genova Dipartimento di Informatica e Scienze dell’Informazione Dottorato di Ricerca in Informatica

Ph.D. Thesis in Computer Science

Surface Reconstruction: Online Mosaicing and Modeling with Uncertainty by Laura Papaleo

March, 2004

1

Dottorato di Ricerca in Informatica Dipartimento di Informatica e Scienze dell’Informazione Università degli Studi di Genova DISI, Università di Genova Via Dodecaneso, 35 I-16146 Genova, Italy http://www.disi.unige.it/

Ph.D. Thesis in Computer Science Submitted by LAURA PAPALEO DISI, Università di Genova [email protected] Date of submission: March 2004

Title Surface Reconstruction: Online Mosaicing and Modeling with Uncertainty

Advisor: Prof. Enrico Puppo Dipartimento di Informatica e Scienze dell’Informazione Università di Genova [email protected]

Ext Reviewers: Prof. Vittorio Murino Dipartimento Scientifico e Tecnologico Università di Verona [email protected] Prof. Carlos Andújar Universitat Politecnica de Catalunya [email protected]

2

Abstract A burst of research has been made during the last decade on 3D Reconstruction and several interesting and well-behave algorithms have been developed. However, as scanning technologies improve their performance, reconstruction methods have to tackle new problems such as working with datasets of large dimension and building meshes almost in real-time. We pointed out a general need for formal analysis of the reconstruction problem and for methods which are able: (i) to elaborate huge and complex input datasets and to produce accurate results, (ii) to exploit all information provided by sensing devices, (iii) to transmit models quickly and accurately in order to visualize, search, and modify them using different devices (PDA, laptop, special devices, and so on). This PhD dissertation addresses the problem of reconstruction from two different points of view: Online Mosaicing and Modeling with uncertainty.

Online Mosaicing: the Thesis presents a Data Analysis approach which on the fly, starting from multiple acoustic/optical range images, elaborates the acquired unknown object by mosaicing multiple single frame meshes. In the context of the European ARROV project, we developed a 3D reconstruction pipeline, which provides a 3D model of an underwater scene from a sequence of range data captured by an acoustic camera mounted on board a Remotely Operated Vehicle (ROV). Our approach works on line by building the 3D model while the range images arrive from ROV. The method combines the range images in a sequence by minimizing the workload of the rendering system.

Modeling with uncertainty: The Thesis presents a general Surface Reconstruction framework which encapsulates the uncertainty of the sampled data, making no assumption on the shape of the surface to be reconstructed. Starting from the input points (either points clouds or multiple range images), we construct an Estimated Existence Function (EEF) that models

3

the space in which the desired surface could exist and, by the extraction of EEF critical points, we reconstructs the surface. The final goal is the development of a generic framework able to adapt the result to different kind of additional information coming from sensors, such as sampling conditions, normals, local curvature, and reliability of the data.

4

To Franco, the sun that drives out winter from my heart-

5

Tell me and I will forget, Show me and I will remember, Let me do it and I will understand [Confucius]

6

Acknowledgments I should say “Grazie” to many people for supporting me during these research years. Thanks to Enrico Puppo, who supports and suffer me, believing in my ideas, Giovanni Gallo and Alfredo Ferro, who suggested me to start this adventure and continue to sustain me. Franco, my husband and my best supporter in the world, who teaches me how organize my tasks and daily assists me in modeling my research offering me incredible ideas and realistic solutions. Since life is not just work, here there are some people who render these years so special: Angelo, my best friend of which a little part inside me will never die. Chiara, Esther and “gli Enry”, who helped me in difficult decisions and in feeling better during hard periods of my research. My Bai’ friends who facilitated my new life in Genova and Paolino because with his force taught me how life is short and wonderful and that we must live not simply exist. Thanks also to all my new friends who helped me in living in a new department and in understanding its rules: Viviana, Davide, Francesca, Marco, Emanuele, Barbara, Paola, Giorgio, Giovanna, Gabriella. Infine, grazie ai miei genitori, ai miei fratelli ed alla mia nonnina per aver creduto in me sempre e comunque, anche quando sembravo diversa, anche quando crescendo sembravo allontanarmi. Questa tesi la dedico a voi: siete sempre con me.

7

Table of Contents T A B LE OF C ON TEN TS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 1.

I NTR ODUC TI ON . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 0 1.1 1.2 1.3

2.

T H E R ECONS TRUC T ION PROC ES S . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 5 2.1 2.2 2.3 2.4

3.

RESEARCH MOTIVATION ......................................................................10 THESIS AIM AND GOALS ......................................................................12 THESIS STRUCTURE .............................................................................13 ACQUISITION AND REPRESENTATION ......................................................15 SCANNING TECHNIQUES .......................................................................17 REGISTRATION ...................................................................................21 INTEGRATION APPROACHES ..................................................................23

I NT EG RATI ON: S TATE -O F- TH E-AR T . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 5 3.1 COMPUTATIONAL GEOMETRY APPROACH ...............................................25 3.1 .1 α -sh a p e m e th o d s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 6 3.1 .2 S cu lp tu r in g M e th o d s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1 3.1 .3 M ed ia l A xi s Me th o d s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 8 3.1 .4 Lo ca l Appro a che s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 4 3.2 VOLUMETRIC METHODS .......................................................................59 3.2 .1 G r id-b a s ed a p p ro a ch es . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 9 3.2 .2 Rad ia l Ba s is Fun c t io n . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 8 3.2 .3 D e forma b l e m o d e ls a n d L e ve l s e t Me thods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 0

4.

ON L IN E MO SAIC ING . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 6 4.1 T H E ARROV P R O J E C T . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 6 4.2 MOSAICING FROM MULTIPLE RANGE IMAGES ..........................................89 4.3 T H E ARROV R E C O N S T R U C T I O N P I P E L I N E . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 0 4.3 .1 Da ta Cap tu r e . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 3 4.3 .2 Da ta Pr e-p ro ce ss in g . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 6 4.3 .3 R eg i s tra tion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 0 3 4.3 .4 G eo me t r ic Fu s io n . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 0 4 4.4 RESULTS ......................................................................................... 112

5.

MOD E LING WI T H UNC ER TAINT Y . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1 8 5.1 MULTI-SENSORS DATA FUSION ........................................................... 119 5.2 PROBLEM DEFINITION AND PROPOSED APPROACH ................................... 121 5.2 .1 Bu i ld in g the EEF Functio n . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 5.2 .2 C o mpu te r id g e p o in ts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 2 6 5.2 .3 Bu i ld in g the m e sh . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 3 1 5.3 THE ALGORITHM .............................................................................. 132 5.4 RESULTS ......................................................................................... 134

6.

CONCLU SIONS AND FU TUR E W ORKS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 4 0

7.

B IB L IOGRAP HY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 4 4

8

9

1. Introduction This chapter first presents the motivation for the research and then introduces the proposed frameworks and their key components. The challenges in developing these frameworks are discussed, and the original contributions of this research are presented. Finally, the organization of the remainder of this dissertation is outlined to guide the reader through its details.

1.1

Research Motivation

The problem of reconstructing 3D objects from sampled data has important applications in several fields among which virtual reality, animation, cultural heritage and reverse engineering are, maybe, the most popular [59]. In virtual reality, for example, generating an accurate 3D environment is essential for a human walk-through. In reverse engineering, 3D reconstruction is able to generate a CAD model from a real object. In robotic vision, a 3D simulation can help a robot to move in a unknown environment providing an correct mapping of the surrounding area. In image-guided surgery, 3D models reconstructed from CTs and MRIs can help a doctor to see anatomic structures and make an accurate diagnosis. 3D models are usually constructed from a set of measurements taken in 3D space. Laser range sensing and stereo vision are two popular methods for 3D measurement. Although stereo vision devices are much cheaper than laser range scanners, they are limited by measurement accuracy and range [86]. From an application-based point of view, two categories of tasks can be distinguished in 3D reconstruction: data analysis and surface reconstruction [64]. •

Data Analysis means that nothing is known about the surface from which the data originate. The task is to find the most reasonable solutions among many possibilities.

10

•

Surface Reconstruction means that the surface from which the data are sampled is known, say in form of a real model, and the goal is to get a computer-based description of this surface that is as accurate as possible. This knowledge may be used in the selection of a suitable algorithm.

A proper reconstruction of the desired surface in the latter case can only be expected if it is sufficiently sampled. Sufficiency depends on the particular method of surface reconstruction. It might be formulated in form of sampling theorems, which should give sufficient conditions that can be easily checked. The necessity of these theorems is present almost in all the algorithms described in the state-of-the-art section (see Chapter 3): all of them, for working in a satisfactory manner, make strong assumptions on the goodness of the sampling dataset, but do not formally explain under what conditions this happens. Exceptions are the works of Attali [8] (which gives a morphology-based sampling theorem at least for the 2D case), Bernardini, Bajaj [12] and Amenta et al.[2]. If data are improperly sampled, a reconstruction method may cause artifacts, which have to be dealt with. A common artifact is the presence of holes and spurious surface boundaries in the model. Like in classical sampling theory, pre-filtering e.g. in the sense of depth-pass filtering may help to reduce artifacts at the costs of loss of details. Another possibility is interactive correction by the user that may be helpful if artifacts occur at some few isolated locations. The opposite of insufficient sampling is that the sampling data are unnecessarily dense. This happens in particular if the surface is sampled with a uniform density. In that case, the sampling density required at fine details of the surface causes too many data points in regions of minor variation and often a simplification post-process is necessary. The actual framework in 3D Reconstruction outlines a general request of formal analysis of the problem and a need of methods, which are able: -

to elaborate huge and complex input datasets and to produce accurate results

-

to build models from input data which come with enriched properties (coordinates but also normal/curvature estimation on points, reliability of the data and so on.)

-

to transmit these models quickly and accurately (via web, for example) in order to visualize, search, modify them using different devices (PDA, laptop, special devices, and so on)

11

This dissertation contributes to solving some problems in two different directions: on one hand, it presents a Data Analysis approach to online and offline mosaicing of multiple range images (in the context of the ARROV project [5]). On the other hand, it reports our investigation on innovative Surface Reconstruction Methods, which try to encapsulate uncertainty of the sampled data, making assumptions on the shape of the surface to be reconstructed.

1.2

Thesis Aim and Goals

The use of three-dimensional digital models, in order to represent real objects and environments, is a powerful and smart solution: modeling is a creative but also cognitive strategy, which allows the users to visualize, modify and study 3D replica on a screen, far away from the real object, or to share representations of data and environments over the net. A burst of research has been made during the last decade on 3D Reconstruction and several interesting and well-behave algorithms have been developed. However, as scanning technologies improve their performance, reconstruction methods have to tackle new problems such as working with datasets of large dimension and building meshes almost in real-time. An optimal reconstruction method should takes into account not only positional information of points (spatial coordinates) but also enriched properties which modern sensors often provide such estimations on normal and curvature on scanned points or reliability of the received input points. This PhD dissertation addresses the problem of reconstruction from two different points of view: Online Mosaicing and Modeling with uncertainty. Online Mosaicing: the Thesis presents a Data Analysis approach which on the fly, starting from multiple acoustic/optical range images, elaborates the acquired unknown object by mosaicing multiple single frame meshes. In the context of the European ARROV project, we developed a 3D reconstruction pipeline that provides a 3D model of an underwater scene from a sequence of range data captured by an acoustic camera mounted on board a Remotely Operated Vehicle (ROV). Our approach works on line by building the 3D model while the range images arrive from ROV.

12

The method combines the range images in a sequence by minimizing the workload of the rendering system. Modeling with uncertainty: The Thesis presents a general Surface Reconstruction framework which encapsulates the uncertainty of the sampled data, making no assumption on the shape of the surface to be reconstructed. Starting from the input points (either points clouds or multiple range images), we construct an Estimated Existence Function (EEF) that models the space in which the desired surface could exist. The extraction of the EEF critical points by the use of a continuous method and a discrete one permits to reconstruct a surface, which correctly model the information given by the input. Moreover, the EEF can help/guide us in estimating the position and normal of missing data by exploiting spatial coherence. The final goal is the development of a generic framework able to adapt the result to different kind of additional information (enriched data) coming from sensors, such as sampling conditions, normals, local curvature, and reliability of the data. The implementation of both methods, based on space discretization by the use of a regular cell grid, has some advantages: 1. It can adapt requirements

the

output

resolution

in

order

to

fit

system

2. It can process large dataset progressively and successively merge all partial results 3. It can be simply parallelized.

1.3

Thesis Structure

This section explains the structure of the thesis: Chapter 2 presents the general pipeline for 3D reconstruction problems treating acquisition devices and methodologies, registration methods integration approaches in general, while Chapter 3 presents our proposed taxonomy of the existing integration methods. After these introductory Chapters, Chapter 4 presents the online 3D reconstruction pipeline we developed in the context of the European

13

Project ARROV [5] and Chapter 5 presents our novel approach for modeling with uncertainty in a Multi-Sensors Data Fusion framework. Both Chapters 4 and 5 show the results of our algorithms applied to significant datasets: for the ARROV pipeline, we tested the method on input data coming from an acoustic sensor, while for the Multi-Sensors approach we used standard input dataset (Stanford bunny, Happy Buddha…). Finally, Chapter 6 concludes this Dissertation and lists future works and open problems.

14

2. The Reconstruction Process The aim of this chapter is to introduce the standard reconstruction pipeline dealing with general concepts and methodologies for 3D data acquisition, registration and integration. It gives information on the different acquisition devices and on the standard registration techniques. Finally, it makes an initial subdivision of the existing integration methods that will be treated in details in the next chapter.

2.1

Acquisition and Representation

From an operative point of view, a digital 3D model is constructed with the definition of the following characteristics: 1. Geometry, description of the coordinates of vertices, 2. Connectivity, description of the relationships between vertices to form faces of the model 3. Photometry, colors attributes, normals or textures. Given the acquired points of the observed object surface, the most common definition of the topology is given by the use of a triangular mesh, which allows a simple way of model representation, visualization and manipulation. But a model can be presented also by the use of surface patches, more flexible but also more mathematically complex, using, for example, NURBS. The construction of 3D digital replicas requires performing a set of sequential and correlated phases that can be listed in the following order: 1. Acquisition Range sensors capture 3D surface measurements. Often, because of the sensor’s limited field of view or of the complexity of the object/scene to be scanned, multiple scans are required. Each view gives a set of 3D points on a certain given coordinates system (later in this chapter). 2. Registration In order to construct the entire model, the acquired data have to be

15

aligned transforming all the measurements into a common coordinate system. This operation has to be done with the minimal possible error. 3. Integration All data are merged to construct a single 3D surface model usually a triangle mesh. Successively some improvements on the model may be done, such as smoothing, simplification and so on (see chapter 3) for a complete taxonomy of the existing integration methods). 4. Color-Texture Processing Colors and reflectance properties have to be defined in order to extrapolate intrinsic physic object characteristics, which will be added to the synthetic model. Successively those reflectance values have to be added to the geometrical model, removing, for example, all the possible non-real shadings, or defining color for each screen pixel.

We are mainly interested in the Integration phase in which all the different correctly registered range images (or directly a cloud of points in space) are combined to construct an unambiguous topological model. The existing methods follow, essentially, two basic directions:

(1)

Reconstruction from unorganized points (points cloud) The main advantage of this type of approaches is that they are general and do not assume any knowledge of the object shape or topology [2], [3], [8], [9], [12], [14], [18], [19], [27], [31], [33], [34], [37], [44], [49], [51], [90], [106]. Unfortunately, for the same reasons, they are usually computationally expensive. Moreover, these methods work well in presence of smooth surfaces but not in the case of high curvature, zones and post processing operations are often necessary. To overcome these limitations, some methods (e.g. [44], [51]), start from a point cloud and extract additional information on the input (such as normals, k-neighbors set) inferring on proximity properties. Some others (e.g. [14] [90]), instead, consider point normals known a priori. In both cases those information are successively used in order to build a mesh as topologically correct as possible.

16

(2)

Reconstruction using the underlying structure of the acquired points (range images, volumetric data, contour data) Here, the underlying structure helps the method in the construction of the entire model [23], [26], [29], [52], [59], [75], [85], [101]. Normals, curvature, reliability of the data are strongly used in the integration phase. Unfortunately, some problems may arise in case of volumetric data: if the edge length of the cells grid is too big aliasing artifacts could be visible avoiding to obtain optimal result.

We now make a brief presentation on the existing scanning techniques and the existing methods on Registration of multiple range images. Finally, we concentrate on the existing Reconstruction Methods giving a initial taxonomy which will conclude this Chapter. In the next Chapter, instead, we will provide a complete survey on Integration Methods.

2.2

Scanning Techniques

A complete description of the existing acquisition techniques is out of the scope of this work but a brief explanation can help in understanding existing methodologies and heuristics. For details, see for example [26], [28]. Depending on the techniques used, the output of a scanning process can be simply a set of points (unstructured data), but it can be also a profile, a range image or a volumetric output (structured data) [81]. Trying to provide taxonomy, the standard acquisition techniques can be roughly divided into two main categories [28] (Figure 1): 1. Acquisition by contact, which is performed by touching the object surface on each relevant side with an ad-hoc instrument. These instruments are quite slow and cannot be used on some typology of objects. Moreover, they do not provide information on object appearance. 2. Acquisition without contact, which is performed by indirect techniques based on a certain energy source. The returned signal is measured by the use of digital cameras or special sensors. In this class, the optical and laser technologies are the most used (see Figure 5).

17

Shape Acquisition

Contact

Non destructive

CMM

Jointed arms

Non-contact

Destructive

Slicing

Reflective

Non optical

Microwave radar

Optical

Trasmissive

Industrial CT

Sonar

Fig ure 1 – Taxono my of the ex isting Acquisit ion Te chniques [28 ]

The optical technologies again can be divided into passive or active [28] (Figure 2). The last one (called also active sensing systems) can acquire data very fast and accurately: these are the reasons why they are the most popular existing technologies. Passive optical systems are, in general, based (i) on acquisition of many RGB images taken from various points, (ii) on the reconstruction of object by contours and, finally, (iii) on integration of such contours for the reconstruction of the model 3D. These systems determine the object coordinates only by the use of information contained in the acquired images (for example, photogrammetry and acquisition by silhouette). They are extremely economic, simple to use and produce a complete model; on the contrary, the quality and accuracy of the produced model can be quite low. Active optical systems are constituted by a source and a sensor, where the source emits a certain illuminant pattern and the sensor acquires returned marks reflected by the object surface.

18

Optical

Passive

Active

Imaging Radar Stereo

Shape from shading

Shape from silhouettes

Triangulation

Interferometry

Active stereo

Depth from focus/defocus Moire

Holography

Fig ure 2 - Taxono my of Opt ica l scanning t echniques [28]

The source scans regularly the space and the system returns a 2D matrix (range image), identifying the points on the surface. Among this type of systems we can list: •

Triangulation systems (Figure 3) where the object geometry is reconstructed by the use of three information: the pattern emission direction and the relative positions of both source and sensor (Figure 4). Either laser sources or light sources can be used as pattern emission sources. These systems reach a good level of accuracy, measuring many points in a small area and returning a 3D points cloud (x,y,z coordinates).

• •

Time of flight systems (imaging radar, interferometry, …) (Figure 3) which emit an impulse and use a sensor for measuring the time needed by this impulse to arrive at the surface and to come back at the device. They are, in general, less precise than triangulation ones but they can acquire wide surfaces on a single image. Moreover they tend to be more costly than triangulation-based sensors. In order to handle large bandwidths required by short pulses, sophisticated electronics that minimize sensitivity to variation in signal strength are needed [13].

19

F ig ur e 3 - Au tosyn ch ron ize d sca nn e r: d ua l-sc an ax is, equa lly us ed in t r ian g u la t ion & t im e o f f l igh t sys t e m s [ 1 3 ] .

In conclusion, scanning works best with well–behaved surfaces, which are smooth and have low reflectance [28]. Triangulation scanners, for example, have the common problem of shadowing: due to the separation of light source and detector, parts of a non–convex object may not be reached by the light from the projector or may not be seen by the detector.

Fig ure 4 - Coded light a pproach example (left) fully illuminated scene ta ken p r io r to proj ec t ion of t he st r ip e patt e rns , ( r igh t) g rey sc a le imag e of a s tr ip e pa ttern proj ected with a LCD proj ector. [13]

20

Fig ure 5 - Two Co lo r 3d scanners f rom Cybewa re [30]: Model 3030RGB/PS ( left) and Model 3030RGB/MS (rig ht). Bot h of them pro duce 14 ,580 po in ts per second, d ig it ize d to X Y Z and RG B co mpon en ts.

2.3

Registration

As we said before, in scanning complex objects, multiple scans are usually necessary and both the object and the scanner must be repositioned. Every range image is in its own local coordinate system and it is necessary to put all the range images into one common frame before entering the Integration Phase. The Registration problem is to find the rigid transformations of the scan sensor between the individual scans: often, especially for surfaces with little inherent structure, special markers are applied to the physical object and the desired transformation is exactly the one that matches a marker in one image onto the other. If no such external information can be used, or if refinement of the solution is necessary due to lack of precision, registration is done by an iterative optimization process that tries to match the point clouds of individual scans by minimizing distances of point pairs. It is generally assumed that the scans have enough overlap and are already roughly aligned (e.g. by interactively pairing prominent surface features) [54].

21

The standard approach, for matching multiple range images, is to register them pair-wise using variants of the iterated closest point method (ICP) [25].

2.3.1.1

ICP ALGORITHM

Let us suppose that we have two sets of 3D points which correspond to a single shape but are expressed in different reference frames. We will call one of these sets the data set V j , and the other the model set V i . Let G i j be the rigid transformation matrix (in homogeneous coordinates) that registers view j onto view i, Vi = Gij⋅Vj The registration consist in finding the 3-D transformation G i j which, when applied to the data set V j , minimizes the distance between the two point sets. In general point correspondences are unknown. For each point v from the set V j , there exists at least one point on the surface of V i which is closer to v than all other points in V i . This is the closest point. The basic idea behind the ICP algorithm is that, under certain conditions, closest points are a reasonable approximation to the true point correspondences. The ICP algorithm can be summarized as follows [22]: 1. For each point in V j , compute its closest point in V i ; 2. With the correspondence from step 1, compute the incremental transformation G i j ; 3. Apply G i j to the data V j ; 4. If the change in total mean square error is less than a threshold, terminate. Else, goto step one. For two point sets, the method determines a starting transformation and successively estimates correspondences between sample points. Then it updates the rigid transformation and repeats the first two steps until complete convergence. Ideally, the transformations are computed precisely and at the end all scans fit together perfectly. In practice, data are contaminated and have errors: surface points measured from different angles or distances result in slightly different sampled positions. So even with a good optimization procedure, it is likely that the n-th scan will not fit to the first. Multiple iterations are necessary to find a global optimum and to distribute the error evenly.

22

In order to overcome of ICP problems, many variants to ICP have been proposed, including: the use of thresholds to limit the maximum distance between points [107], the use of closest points in the direction of the local surface normal, Least Median of Squares estimation [98], rejecting matching on the surface boundaries [101], or the use of the X84 outlier rejection rule to discard false correspondences on the basis of their distance [42]. Besl and McKay [15] proved that the ICP algorithm as presented above is guaranteed to converge monotonically to a local minimum of the Mean Square Error. As for step 2, efficient, non-iterative solutions to this problem (known as the point set registration problem) were compared in [61], and the one based on Singular Value Decomposition was found to be the best in terms of accuracy and stability. However, a detailed discussion of these methods is actually beyond the scope of this dissertation.

2.4

Integration approaches

After data acquisition and registration phases, the next step is the generation of a single surface representation, usually an overall triangle mesh, from the acquired data. As we said before, general solutions should not assume any knowledge of the object shape or topology but possible approaches may strongly depend on the given type of input (e.g. point clouds, sections, multiple range images). It was quite difficult to determine a significant taxonomy of the existing surface reconstruction methods. Most of them, especially in the last few years, try to adopt hybrid solutions using different approaches in the same method. Regardless the underlying structure of data, approaches can be divided into two groups [64], depending on whether they produce an: 1. Interpolation of the input data 2. Approximation of the input data In the first case (interpolation), the vertices in the resulting mesh are the original sampled points. These methods, in some sense, rely on the accuracy of the input and use them as constraints for the construction of the final mesh [2], [3], [8], [9], [12], [18], [27], [31], [37], [44], [51], [90], [106].

23

The basic strategy under the interpolant approaches is to use the input points as the optimal geometric description of the scanned object, in other words as vertex set of the final mesh. In general a cloud of points with no other information is considered as input [37], [12], [18], [106], [31], [34] but, in some cases, also points clouds with additional information on the object structure or points proximity [2], [3], [8], [9] maybe processed. In addition, the modern scanning technologies often return the acquired points coordinates together with an estimation of the normal in each point: that is why some interpolant methods use also this information [90], [14], [44]. The interpolant methods can be divided into: •

Global Approaches that take into account the entire set of input points in order to derive an initial structure from which extract [12], [18], [31], [37], [90], [106] or on which construct [2], [3], [8], [9] the final mesh;

•

Local Approaches, that build portions of mesh by inferring on local characteristics of the input points [14], [44], [101].

In the second case (approximation), the vertices in the resulting mesh can be different from the original sampled points. The strategy of the approximation methods is to use the input points as guide for surface reconstruction. Especially for range data, an approximating rather than an interpolating mesh is desirable in order to get a result of moderate complexity [14], [23], [26], [29], [49], [50], [52]: in fact, in cases of input which comes with error, in the overlapping zones, where data are redundant, the error frequency is high and the interpolant methods can produce results with outliers. In these cases, approximation methods behave definitely better.

24

3. Integration: State-of-the-Art Be curious always, for knowledge will not acquire you; you must acquire it [Sudie Back]

This chapter tries to make an exhaustive description of the most representative 3D reconstructing methods developed in the last two decades. We have done our best to determine a good taxonomy of the existing methods. It was quite difficult because, especially in the last research years, many algorithms try to adopt hybrid solutions in order to get better results. We divided the reconstruction methods into two main categories (and some of them into subcategories): •

Computational Geometry Approaches: a method belongs to this class if it is based on computational geometry concepts (alpha-shapes, Delaunay Triangulation, Voronoi Diagram and Medial Axis and so on).

•

Volumetric Approaches: they try to define a function in space and to build the final mesh inferring on the space occupied.

The remain part of this chapter is dedicated to the explanation of these categories with examples and (when possible) methods comparisons.

3.1

Computational Geometry Approach

We consider a method to belong to this class if it basically takes into account the geometric structure of the object to be reconstructed and it is based upon computational geometry concepts (Voronoi Diagram, Delaunay Triangulation, Medial Axis, and so on). In the context of this macro-class, we can make another subdivision:

25

1. α -shape approaches are all the methods based on the concept of α-shape or correlated concepts [37], [12], [90]. 2. sculpturing approaches are methods which sculpture the model deleting tetrahedral from an initial structure [18], [106], [31], [9] 3. medial axis approaches are methods based on the construction of the final model by the use of medial axis of the object [2], [3], [8], [33], [34]. 4. local approaches are methods that operate in a local fashion [14], [27], [44],[49], [85], [101]. The remain part of this section explains in details all these subcategories.

3.1.1

α-SHAPE METHODS

The concept of α-shape was first introduced by Edelbrunner and Mücke [37] in 1994. It is an approach to formalize the intuitive notion of shape for 3D point sets. The alpha shape is a concrete geometric concept that is mathematically well defined: it is a generalization of the convex hull and a sub-graph of the Delaunay triangulation. Given a finite point set, a family of shapes can be derived from the Delaunay triangulation of the point set; a real parameter, alpha controls the desired level of detail. The set of all real alpha values leads to a whole family of shapes capturing the intuitive notion of crude vs. fine shapes of a point set.

F ig u r e 6 - D if f e re n t g r a p h u s ing d if f e r ent a lpha-va lu es and t he relat iv e a lphaba lls

26

For a given point set S⊂ℜ d and 0≤α≤∞, the α-complex C α of S is a simplicial subcomplex of the Delaunay triangulation of S DT(S). Consider a simplex ∆ T in DT(S): let be σ T and µ T the radius and the center of the circumsphere of ∆ T respectively. Given a certain α, ∆ T in DT(S)is in C α if -

σ T