Fusion of range camera and photogrammetry - Semantic Scholar

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART B: CYBERNETICS, VOL. 33, NO. 4, AUGUST 2003

667

Fusion of Range Camera and Photogrammetry: A Systematic Procedure for Improving 3-D Models Metric Accuracy Gabriele Guidi, Senior Member, IEEE, Jean-Angelo Beraldin, Stefano Ciofi, and Carlo Atzeni

Abstract—The generation of three-dimensional (3-D) digital models produced by optical technologies in some cases involves metric errors. This happens when small high-resolution 3-D images are assembled together in order to model a large object. In some applications, as for example 3-D modeling of Cultural Heritage, the problem of metric accuracy is a major issue and no methods are currently available for enhancing it. In this paper the authors present a procedure by which the metric reliability of the 3-D model, obtained through iterative alignments of many range maps, can be guaranteed to a known acceptable level. The goal is the integration of the 3-D range camera system with a close range digital photogrammetry technique. The basic idea is to generate a global coordinate system determined by the digital photogrammetric procedure, measuring the spatial coordinates of optical targets placed around the object to be modeled. Such coordinates, set as reference points, allow the proper rigid motion of few key range maps, including a portion of the targets, in the global reference system defined by photogrammetry. The other 3-D images are normally aligned around these locked images with usual iterative algorithms. Experimental results on a anthropomorphic test object, comparing the conventional and the proposed alignment method, are finally reported. Index Terms—Error analysis, measurement accuracy, shape measurement, 3-D modeling.

I. INTRODUCTION

R

ECENT technologies for digital three-dimensional acquisition have opened innovative applications for the conservation, reproduction, study and fruition of sculptural, architectural and archeological artworks [1]–[3]. Although the first examples of application of three-dimensional (3-D) digital scanning to Cultural Heritage have been demonstrated by the National Research Council of Canada since a number of years [4]–[8], a series of important applications have been described in recent years in many international papers, several of them concerning celebrated Italian artworks. Starting from 1997, some pioneer works have to be remembered.

Manuscript received November 22, 2001; revised January 20, 2003. This work was supported in part by Consiglio Nazionale delle Ricerche (CNR) with the program “short term mobility” 2001. This paper was recommended by Guest Editor A. Fusiello. G. Guidi, S. Ciofi, and C. Atzeni are with the Department of Electronics and Telecommunications, University of Florence, 3-50139 Florence, Italy (e-mail: [email protected]). J.-A. Beraldin is with the National Research Council Canada (NRCC), Ottawa, ON K1A 0R6, Canada (e-mail: [email protected]). Digital Object Identifier 10.1109/TSMCB.2003.814282

The first involved the florentine “Pietà” by Michelangelo, a marble sculpture more than two meters tall, kept in the Museum of the “Opera del Duomo” in Florence. Michelangelo sculptured this artwork for his own grave, but, unsatisfied, tried to destroy his work, that was recovered years later by one of his scholars. In order to perform an accurate study, the art historian Jack Wasserman required a 3-D digital model [9], that was realized by an IBM team. The IBM technology was based on a structured light projection camera equipped with six CCD, that allowed a lateral resolution of about 2 mm to be achieved [10]. In the same year an Italian–Canadian team performed the acquisition and modeling of the “Madonna col Bambino” by Andrea Pisano, a statue kept in the Cappella degli Scrovegni in Padova, using a laser range camera based on optical triangulation capable of 0.5 mm resolution [11]. In 1998 the “Digital Michelangelo Project” was developed in Italy by researchers of the Stanford University: three-dimensional models of Michelangelo’s statues, including the worldwide famous David, were realized using a triangulation laser scanner. A resolution of about 0.25 mm was reported [12]. The results of these first projects demonstrated that 3-D digital procedures, originally developed for industrial application, can be applied to 3-D acquisition of artistic Heritage as well, and that optical sensors were capable of scanning great size artworks with high resolution. Also, the projects allowed the refinement of techniques for aligning the acquisitions in the same reference system and for merging the measurements in a polygonal model from a high number of data [13]–[16]. Since then, a number of other applications to Heritage have been described. The digitalization of the “Maddalena” by Donatello, performed by the authors of this paper, represented another severe test for the technology. The wooden statue, kept in the Museum of the Opera del Duomo in Florence, was chosen just for its extreme technical difficulties, due to the material (wood optically “noncooperative”), and to the extremely complex surfaces of the long hair wrapping all around the body. These characteristics made very hard the acquisition and the subsequent processing [17]. The completion of a first model of the statue, exhibiting a range uncertainty from 0.02 to 0.2 mm, allowed us to focus a number of problems, that are typically encountered in high-quality digitizing of Heritage artworks, and are currently under study in a number of laboratories worldwide. These topics are related to the following: 1) reduction of the long time needed for completing a model [18], [19], possibly introducing automatic alignments without operator [20];

1083-4419/03$17.00 © 2003 IEEE

668

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART B: CYBERNETICS, VOL. 33, NO. 4, AUGUST 2003

2) need of photometric images to be associated to the geometric shape of the artwork [21], [22]; 3) visualization of a great size models requiring the introduction of special visual techniques in order to limit the explosive growth of the required memory [23], [24]. One crucial point that has so far received less attention in creating high quality 3-D models of Heritage is the problem of accuracy of the final model resulting from the alignment of acquired partial views of the scanned artwork. At our knowledge, no one of the published papers reports measures verifying the achieved accuracy of the model. In our experience on the “Maddalena” acquisition, we adopted verification through a conventional photogrammetry survey, in order to measure the accuracy of the final model. The results demonstrated that the process of alignment of partial acquisitions (whose number can be up to several hundreds) can yield not negligible metric errors, that can degrade the quality of the whole model [25]. The main objective of this research was the development of procedures for generation of high accuracy models of sculptural artworks through the integration of 3-D digital scanning and digital photogrammetry. Such a fusion can allow the overall error on the model to be a priori fixed. The same general concept of sensor fusion, implemented with different algorithms and without the specific purpose of maximizing the overall 3-D model accuracy, was first introduced for modeling of virtual environments [26], and in spatial applications [27]. II. TECHNICAL BACKGROUND Typically, a digital model is generated by merging a set of point clouds acquired by a range camera and framing the object to be modeled from different points of view, in order to cover the whole object surface. The key problem at this stage is related to the proper pose determination of each range map (which is referred in a reference system local to the camera) in a single global reference system. The techniques implemented for solving this problem are essentially based on two alternative approaches involving the following: 1) measurement of camera position and orientation during the range map acquisition through equipment like coordinate measurement machines (CMMs): in this case the range camera has to be coupled to a complex mechanical system which may be implemented with articulated arms or xyz micro positioning sensors. Although the measurement can be very accurate, this makes the whole system rather cumbersome for many applications. Alternatively the mechanical coupling can be substituted by a non contact device, as for example those based on magnetic tracking [28]. The implementation is in this case much simpler, but the measurement accuracy tends definitely to decrease; 2) alignment of partially superimposed point clouds, through iterative closet point (ICP) algorithms [29]–[31]. Such algorithms allow one to fix a range map as a reference, and to position an adjacent range map in order to minimize a cost function or to find the best match

Fig. 1. Schematical representation of different alignment situations: (a) Example of isotropic alignment constrains. (b) Lack of constrain along the vertical direction (3-D modeling of “Maddalena” by Donatello).

between common points, starting from an initial guess. The range camera in this case can be simply mounted over a tripod, and the overall system becomes extremely simple and therefore portable. In some applications, e.g., modeling ancient works of art, very rarely it is allowed to move the object to be modeled, and the need of a portable system involves the use of the second approach; in this case the problem of error propagation cannot be neglected [11]. Depending on the amount of three-dimensional features over the acquired surface and on the extent of overlapping areas between adjacent range maps, the convergence of an ICP algorithm may be more or less easy. The cost function involved in the optimal pose determination has many local minima in case of flat surfaces, while a good and well identifiable local minimum is found in most cases of high relief. Furthermore the measurement uncertainty, typical of each 3-D sensor, produces an artificial roughness, which appears superimposed to the true surface behavior. Although this is uniformly distributed over the whole measurement volume, it generally influences the ICP algorithm, producing a random alignment error that depends on the surface relief. This increases the probability for the ICP algorithm of finding convergence on a local minimum different from the correct one. A similar situation is encountered when the correlation of two equal signals immersed in noise is calculated. In that case the correlation peak is more jagged as the S/N ratio decreases. Error propagation produces therefore a displacement of aligned points with respect to the true surface points, that get worst as the number of images increases. The effect of such errors tends to be compensated in presence of closed surfaces, where the global alignment allows range maps to interact with each other like in the example of Fig. 1(a). But whenever the

GUIDI et al.: FUSION OF RANGE CAMERA AND PHOTOGRAMMETRY

geometry is elongated and possibly open along one direction, as we found during the modeling of statues [Fig. 1(b)], the alignment constrains with other parts of the model tend to be reduced, or in some cases to vanish completely. In these situations, as experimentally demonstrated during the generation of the 3-D model of the wooden statue “Maddalena” by Donatello [25] [Fig. 1(b)], the dimensional error along the longer dimension can be significant, and the use of an external reference may become fundamental for the metric accuracy of the final model. The approach described in this paper proposes the employment of digital photogrammetry as complementary measurement method, able to supply the spatial coordinates of a number of reference targets in a global reference system with respect to the object to be modeled, and a software procedure for rototranslating the range maps associated with such targets in the global coordinate system. III. MATERIALS AND METHODS Our goal in this work is to use the results of digital photogrammetry to lock some key range maps into place so that automatic alignment of the remaining ones becomes more accurate. The basic idea is to measure spatial coordinates of some calibrated targets located near the surface to be modeled, using both 3-D scanning and digital photogrammetry. Targets are placed around the object because touching the Heritage artworks surface is usually not allowed. Typically, 3-D range cameras supply small range maps representing a detail of the object, while photogrammetry can measure the whole object with the use of large image sensors and longer baselines. In the latter case the measured coordinates lie all in the same reference system defined by the digital photogrammetry procedure with an estimated error that is typically between 1:10 000 and 1:100 000 of the principal dimension of the framed area [32]. In other words, for a 2 m tall statue, we could expect an error lower than 200 m. This number is well within the error that can be found through the alignment of critical models. For example in [25] it was found an error of 4.5 mm for a statue 180 cm tall. In the literature, few papers actually address this issue of model accuracy. A. Range Camera The range camera used in this work is based on optical triangulation with fringe pattern projection and is produced by Optonet Srl, Brescia, Italy. The main components of the system are a LCD projector for pattern projection, a C-mount CCD camera, a few camera lenses, a variable baseline support, and a calibration fixture. Depending on the type of surface imaged, the field of view and accuracy of the 3-D camera are reconfigured. This process allows the operator to adapt the modeling process to different situations found in modeling of artworks. The 3-D coordinates evaluation in this system is based on the simultaneous use of Gray code projection, for generating an approximate estimation of the surface, and of phase shift of the finest pattern for a refinement of each 3-D point estimation, according to a method described in the literature [33], [34].

669

As any other active optical sensor based on triangulation, the measurement uncertainty depends inversely on the projectorcamera distance (baseline) and directly on the squared camerafield-of-view-distance (standoff). The displacement between the 3-D points measured by the scanner and their true coordinates follows a Gaussian distribution and it is common to statistically quantify the measurement uncertainty deviations ( , and ) respect to the three reference axes. This evaluation is performed at the calibration stage, usually before starting a measurement session, by measuring a known target located in known positions. The standard deviations are also evaluated in the following stages, for example at the end of a measurement session, or at the beginning of a session where a previous calibration setup is employed, by acquiring a verification object with a highly planar surface, and evaluating the deviation of the point cloud from the corresponding best fitting plane. The opto-geometrical setup chosen for the scanner gave a field of view of 255 184 mm on the focal plane, with an extent of 120 mm in depth, and the following standard deviation values at the calibration: m m m. Due to the handling and movement of the optical head, a value slightly worst was found at the end of the measurement of procedure, but anyway lower than 80 m. B. Three-Dimensional Modeling Platform Data registration was carried out with PolyworksTM, a commercial package (produced by Innovmetric, Quebec, Canada) which includes all the functionalities for a manual rough adjustment of adjacent range maps based on the selection of common points chosen by an operator, and a following refinement of such alignment through an ICP algorithm. A different piece of software included in the same package was used for generating a 3-D model from the aligned range maps. C. Photogrammetry A digital photogrammetry system was used based on a good quality digital camera and software for post processing the acquired images. The camera used is a Minolta Dimage 7, equipped with a CCD organized in 2560 1920 pixel (5.2 MPixel), and a lens with focal length ranging from 7.2 to 50.8 mm, equivalent to a 28–200 mm zoom in a 35 mm reflex camera. The camera has also the capability of saving images in a proprietary uncompressed format (raw format), that can be transformed in tiff files with a software tool provided by Minolta. This is an important feature when the image is employed for measurement purpose, because in this way any processing over the image can be avoided.

670

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART B: CYBERNETICS, VOL. 33, NO. 4, AUGUST 2003

TABLE I TARGET-TO-TARGET DISTANCES MEASURED ON PLATE A

After calibration, a set of convergent digital pictures of the object taken from different angles can be registered by first selecting homologous points over them, and then launching a bundle adjustment procedure, as described in [35]. Once the images are registered, the 3-D coordinates of specific points can be easily generated and saved in a text file. D. Targeting Plates Fig. 2. Custom calibration grid for photogrammetry of anthropomorphous objects, made with 32 surveyed 3-D targets.

The photogrammetry program is a commercial package (Shape Capture—Shape Quest Inc., Ontario, Canada) that allows calculating camera calibration with simple steps. Since we had the specific need of setting up the system for measuring anthropomorphous statues, we built a specific 3-D calibration grid, shown in Fig. 2. It covers a volume of approximately 1 m 1 m 2 m, and was made by fixing on two quasiperpendicular walls a set of randomly distributed black circular targets with a diameter of 10 mm. Each target was also designed with a little white hole in the middle, and its center was identified by a thin cross. This was done for making easier the targeting in a preliminary theodolite survey, carried out to have a reliable measurement of each point belonging to the 3-D grid. Such preliminary step gave a good estimation, with only 15 m of positional uncertainty. The laboratory is located at ground level and is on a thick concrete floor. The camera model used by Shape Capture takes into account 9 internal parameters including focal length, principal point and distortion parameters due to lens aberrations or image plane distortions, and 6 external parameters representing the position and orientation of the camera when the image is taken. These are evaluated by taking a set of 2–3 images of at least 8 control points belonging to a 3-D calibration grid. In our calibration procedure 32 control points were included (see Fig. 2).

The main role of these plates was to be easily recognizable with both 3-D scanning and photogrammetry, so they had to exhibit specific 3-D and optical properties. For this reason we designed them in order to have the following: 1) high contrast for making easier the target selection with the photogrammetry software; 2) high planarity, so that during the post processing of range maps its surface could be approximated with a fitting plane instead of employing the raw data set, typically affected by errors in the high contrast areas of the image; 3) high number of targets for making more robust the rototranslation procedure. The minimum number of points to univocally calculate a rototranslation (pose) of the range-map are of course three, but in this way the possible errors affecting the rototranslation matrix would be directly influenced by the uncertainty of each point measured in both the source and destination reference system. By properly choosing and increasing the number of points to be employed for calculating the rototranslation, the uncorrelated errors tend to cancel each other, leaving only systematic errors, that can be minimized thanks to calibration; 4) a known distance between targets, in order to give us the possibility to check the accuracy and deviation of the target position measured with both systems, in any stage of the processing (see Tables I–IV, Section IV).

GUIDI et al.: FUSION OF RANGE CAMERA AND PHOTOGRAMMETRY

671

TABLE II TARGET-TO-TARGET DISTANCES MEASURED ON PLATE B

TABLE III TARGET-TO-TARGET DISTANCES MEASURED ON PLATE C

TABLE IV TARGET-TO-TARGET DISTANCES MEASURED ON PLATE D

Therefore, we made these rectangular plates with 8 mm thick glass, ensuring an extremely good planarity (few tens of m per square meter), whose area was small enough (217 155 mm) to be framed in approximately half 3-D image with the range camera setup chosen. On this plane a set of 14 white 10 mm wide circular targets, spaced 40 mm apart, and located on three rows, as shown in Fig. 5, were printed. E. Sensor Fusion For integrating the information coming from different sensors, it was necessary first of all to extract from the range maps the 3-D coordinates of the points corresponding to the points measured through photogrammetry. For this purpose we developed a Matlab1 program for evaluating 3-D coordinates of each target centroid. The procedure starts with a manual selection of the range map containing the targeting plate. On the selected 3-D points the best-fit plane is evaluated, according to the principal component method [36]. Afterwards, the operator selects manually the small black area around each circular target. The centroid over the selected image is evaluated, giving the white target location over the two–dimensional (2-D) range map texture, with sub-pixel resolution. Finally such bidimensional information is projected on the best-fit plane, locating, with great accuracy, the 3-D coordinates of the target in the range map space. A text file containing such centroids list is the final output of this stage. Next step is to calculate the rototranslation matrix from two sets of experimental data representing the same point coordinates measured in two different reference systems. For this purpose a few methods are available in the literature [37], essentially based on two alternative approaches: 1

Matlab is a registered trademark of The MathWorks, Inc.

1) Iterative search of the minimal distance between rototranslated source points and destination points; 2) Closed form solution not requiring iteration. In order to have a simple and fast processing we chose the second way. A script in the Matlab environment was written, implementing a procedure based on Quaternions, originally proposed by Horn [38]. The two reference systems, shown in Fig. 3, are conventionally indicated as “left” and “right”. The sets of points are identified by the coordinates of their points in the two systems Left representation Right representation with right.

is for left and

for

672

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART B: CYBERNETICS, VOL. 33, NO. 4, AUGUST 2003

TABLE V SAMPLE DISTANCES BETWEEN THE TOP OF THE MODEL (TARGET #410) AND SOME SAMPLE POINTS AT THE BOTTOM OF THE MODEL (TARGETS #701–711), BEFORE AND AFTER THE INTEGRATION WITH PHOTOGRAMMETRY

Fig. 3. Left and right reference systems considered in the quaternion based procedure for rotranslating the range maps in the right positions.

Since the coordinate measurements have been done in the two systems in different times with different instruments (i.e., range camera and digital photogrammetry), random errors actually lead to two set of points that even after an ideal rototranslation cannot be exactly coincident. The purpose of the method is to obtain the 4 4 rototranslation matrix that allows the minimization of the residual error between the Right representation of points and the roto-translated Left representation of the same points. of the two The method calculates first the centroids and point sets. For example for the Left system we have

Then the procedure subtracts each centroid from the left and right representation of the points, respectively, obtaining two baricentered sets: Fig. 4. Subset of the five images used for the photogrammetric process. The areas framed by the range camera are highlighted in (a).

lying in the reference systems indicated as and in Fig. 3. is created, whose elements are Afterwards, a 3 3 matrix the sum of products of coordinates measured in the two systems

where and Once the matrix is available, a 4 4 matrix can be calculated, whose components are a combination of the compo(see the equation at the bottom of the page). nents of

It can be demonstrated [38] that a proper combination of the components of the eigenvector

corresponding to the most positive eigenvalue of , and represented here as a quaternion, gives the rotation part of the rototranslation matrix as shown in the equation at the bottom of the next page.) Once the rotation is known, the translation is evaluated as the difference between the centroid of the destination set (left) and the rotated source set. The final stage of this process is the generation of a set of roto-translation matrices, one for each 3-D image, in a format compliant with the Polyworks software, which was used for the final alignment.

GUIDI et al.: FUSION OF RANGE CAMERA AND PHOTOGRAMMETRY

Fig. 5.

673

Targets numbering employed in the bundle adjustment procedure as referred to in Tables I–V: (a) Plate A. (b) Plate B. (c) Plate C. (d) Plate D.

IV. RESULTS AND DISCUSSION The purpose of the experiment described in this section was to compare, in terms of global model accuracy, the performances of a classic approach implemented by a commercial software based on automatic alignment [39] with the proposed one. For this reason a test set was arranged in our lab, with a test object simulating a typical anthropomorphic statues. It was modeled with the two approaches, and the final position of some key points was compared for evaluating the enhancements provided by the proposed method (see Table V). The test object is actually a mannequin put on a wooden holder, 160 cm tall and 40 cm wide. The whole object surface was first acquired with about 150 partially superimposed range maps (40% overlap), then aligned in the conventional way. Some difficulties were found on the wooden holder, for the lack of 3-D features, and at the connection between the holder and the bust where the stalk gets thinner. However a converging reg-

istration that allowed generation of the 3-D model is shown in Fig. 7(a). Four target plates were then located around the object, being careful to avoid any contact between the plates and the mannequin, as should be done for any ancient statue. The plates positions, highlighted by dashed rectangles in Fig. 4(a), were determined by choosing surface portions at the two far ends of the object (near the neck and at the bottom), in order to fix the final model height, and in a couple of central areas that in the preliminary alignment gave a few problems due to the 3-D featureless surface of the wooden holder. The targeting plates were acquired with the range camera, framing in each image most of the plate together with an object surface portion large enough to be properly aligned with previously acquired range maps. From these acquisitions the target centroids were obtained with the Matlab procedure described in Section III.E. In this particular case only 12 of the 14 targets were included in the range map. In any case their number was

674

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART B: CYBERNETICS, VOL. 33, NO. 4, AUGUST 2003

sufficient to calculate the rototranslation matrix from the range camera reference system to the global one. Leaving the targeting plates in place, a 1 m long stadia rod was added to the scenario in order to give a metric reference. With the previously calibrated photo camera 5 images were taken, targeting the test object frontal side from 5 different locations: bottom right, bottom center, bottom left, top right and top left. The test object-camera distance was approximately 1.5 m, such as the distance between adjacent recording locations. In Fig. 4 two of the five images are displayed. These images were then processed with the Shape Capture program, that allowed to locate the points corresponding to the different targets centroids. In Figs. 5 the numbering assigned to the targets over the different plates is reported. 1) 400–413 Plate A; 2) 500–513 Plate B; 3) 600–613 Plate C; 4) 700–717 Plate D. From the picture it is evident that for Plate D a different material and another target diameter were used, leaving the target-totarget distance unchanged (40 mm). Image registration with ShapeCapture allowed the generation of the 3-D coordinates of each target set in the global coordinate system defined by photogrammetry, where the stadia rod was chosen as the axis. From their 3-D plot (Fig. 6) a first qualitative confirmation of the correct target measurements was found, but, in order to have a more quantitative feedback, the target-to-target distances, theoretically equal to 40 mm, were calculated for the four plates. These results are reported in Tables I–IV, starting from the 3-D coordinates extracted by photogrammetry and by the range maps. For each targeting plate they show a measured average distance very close to 40 mm with both methods, and a small standard deviation either by 3-D scanning (from 78 to 199 m) or by photogrammetry (from 169 to 258 m). Although the average value of the standard deviation is double for photogrammetry with respect to 3-D scanning, it must be taken into account that the range camera records data from a small volume (255 184 120 mm); whereas with photogrammetry a framed area of about 2 1.5 m is considered. The latter result confirms the higher relative accuracy of photogrammetry, that can be profitably employed as a reference for enhancing the 3-D model global accuracy. Once the measurements were tested and verified, the rototranslation procedure described in Section III-V was applied to the four range maps including the targeting plates. Starting from these, all the remaining 150 were then aligned, generating a second version of the model apparently equal to the first one. In order to measure possible metric differences along the height of the two models, the range maps corresponding to plates A and D were also aligned to the first model, obtaining the result shown in Fig. 7(b). On this “dummy” model some control measurements between a point near the model top (target #410) and a few points of target D (at the bottom of the test object) were performed. The results of this test are reported in Table V. In the fourth column the differences between homologous distances before and after the photogrammetry driven alignment are reported. It

Fig. 6. Three-dimensional plot of the points extracted from the photogrammetric procedure.

Fig. 7. Models generated by merging the 3-D data: (a) Only with 3-D scans alignement. (b) With the procedure described in the paper. A set of top-to-bottom differences were in this way corrected, as shown in Table V.

is evident that some distances are compressed and others elongated. This is due to a major alignment difficulty at the thin connection between the bust and the wooden holder of the test object. In addition, the random errors propagation contributes to further displace such points. Thanks to ICP-photogrammetry

GUIDI et al.: FUSION OF RANGE CAMERA AND PHOTOGRAMMETRY

integration an overall error larger than 3 mm was cancelled (e.g., dist. 410–701 and 410–709 in Table V). V. CONCLUSION Conventional methods don’t include any means for evaluating and possibly correcting the metric accuracy errors on a 3-D model. The results obtained demonstrate a practical approach to reduce the overall 3-D model error to a known value. This feature is specially important in applications where unpredictable modeling errors are not acceptable, as in dimensional monitoring of Cultural Heritage through 3-D modeling. The method proposes to exploit two complementary 3-D measurement techniques getting the best performances from each sensor: digital photogrammetry for a good relative accuracy on few key points and range cameras for a high point density. In particular digital photogrammetry is used to generate a set of spatial references in a single coordinate system. Such references allow the rigid motion of some crucial portions of a surface in definite and very accurate positions. The choice of these portions is made in order to limit the overall metric error within the values attainable with photogrammetry that, as known, is very low. For example in our experiment a piece of surface near the head and another one at the bottom of the test object were chosen. Once the surfaces defining the object ends are locked in their positions, the registration of the other range maps cannot change the overall error that results in this way, known “a-priori” before starting the ICP procedure. The level of accuracy that has been measured on a “photogrammetry driven” alignment may be one order of magnitude better than the alignment with no external reference, especially when hundreds of images are used to generate a 3-D model, or when critical situation are encountered (e.g., lack of superposition between adjacent images due to physical constrains or flatness of properly superimposed surfaces). In the presented experimental results a correction of a few millimeters was obtained over a test object 1.6 m tall. Models of large objects, structures and environments are possible, but we are convinced that the fusion of the current 3-D techniques with other measurement methods can be the key for enhancing overall accuracy of 3-D models. ACKNOWLEDGMENT The authors wish to thank S.-El Hakim, from NRC Canada, for his useful help in the camera calibration, I. Chiaverini and D. Ostuni from the Civil Engineering Department—University of Florence, for their contribution in making an accurate theodolite survey of the 3-D grid used in the photogrammetry calibration, and Valentina Damato for the help provided during the range camera acquisitions. REFERENCES [1] G. Guidi, M. Pieraccini, S. Ciofi, C. Atzeni, J.-A. Beraldin, and S. Lazzari, “Immagini Digitali Tridimensionali Di Beni Artistici,” Alta Freq., vol. 13, no. 2, pp. 30–36, 2001.

675

[2] A. C. Addison, “Emerging trends in virtual heritage,” IEEE Multimedia, vol. 7, pp. 22–25, Apr./June 2000. [3] E. Berndt and J. C. Teixeira, “Cultural heritage in the mature era of computer graphics,” IEEE Comput. Graph. Applicat., vol. 20, pp. 36–37, Jan./Feb. 2000. [4] P. Boulanger, M. Rioux, J. Taylor, and F. Livingstone, “Automatic replication and recording of museum artifacts,” in Proc. 12th Int. Symp. Conservation Restoration Cultural Property, Tokyo, Japan, Sept. 29–Oct. 1 1988, pp. 131–147. [5] R. Baribeau, M. Rioux, and G. Godin, “Recent advances in the use of a laser scanner in the examination of paintings,” in Proc. Restoration’92, Amsterdam, The Netherlands, 1992, pp. 69–73. [6] R. Baribeau, L. Cournoyer, G. Godin, and M. Rioux, “Color three-dimensional modeling of museum objects,” in Imaging the Past, Electronic Imaging and Computer Graphics in Museum and Archaeology, London, U.K.: British Museum, 1996, pp. 199–209. Occasional Paper Number 114. [7] J. Taylor, G. Forester, F. Livingstone, and R. Baribeau, “AMUSE: 3-D color imaging, remote access and display,” in Proc. EVA’96, 1996, pp. 9.1–9.10. [8] F. Livingstone, K. Evans, J. Taylor, M. Rioux, and H. Arsenault-Desfossés, “The 3rd dimension: A new way of seeing in cyberspace,” in Proc. EVA’97, Paris, 1997, pp. 10.1–10.13. [9] J. Abouaf, “Florentine Pieta: Can visualization solve the 450-year-old mystery?,” IEEE Comput. Graph. Appl., vol. 19, pp. 6–10, Jan./Feb. 1999. [10] F. Bernardini, H. Rushmeier, I. M. Martin, J. Mittleman, and G. Taubin, “Building a digital model of Michelangelo’s Florentine Pieta,” IEEE Comput. Graph. Applicat., vol. 22, pp. 59–67, 2002. [11] J.-A. Beraldin, F. Blais, L. Cournoyer, M. Rioux, F. Bernier, and N. Harrison, “Portable digital 3-D imaging system for remote sites,” Proc. IEEE Int. Symp. Circuits Systems, vol. 5, pp. V-488–V-493, 1998. [12] M. Levoy, “Digital michelangelo project,” in Proc. Computer Graphics Forum, vol. 18, 1999, p. 4. [13] J.-A. Beraldin, L. Cournoyer, M. Rioux, F. Blais, S. F. El-Hakim, and G. Godin, “Object model creation from multiple range images: Acquisition, calibration, model building and verification,” in Proc. Intl. Conf. Recent Advances 3-D Digital Imaging Modeling, Ottawa, ON, Canada, 1997, pp. 326–333. [14] M. Levoy, S. Rusinkiewicz, M. Ginzton, J. Ginsberg, K. Pulli, D. Koller, S. Anderson, J. Shade, B. Curless, L. Pereira, J. Davis, and D. Fulk, “The digital Michelangelo project: 3-D scanning of large statues,” in Proc. ACM SIGGRAPH Conf. Computer Graphics, 2000, pp. 131–144. [15] B. Curless and M. Levoy, “A volumetric method for building complex models from range images,” in Proc. SIGGRAPH’96 Computer Graphics , 1996, pp. 303–312. [16] M. Rioux, J.-A. Beraldin, F. Blais, G. Godin, and L. Cournoyer, “High resolution digital 3-d imaging of large structures,” in SPIE Proc. ThreeDimensional Image Capture, vol. 3023, 1997, pp. 109–118. [17] G. Guidi, M. Pieraccini, S. Ciofi, V. Damato, J.-A. Beraldin, and C. Atzeni, “Tridimensional digitizing of Donatello’s Maddalena,” Proc. IEEE Int. Conf. Image Processing, vol. 1, pp. 578–581, 2001. [18] R. Scopigno, P. Pingi, C. Rocchini, P. Cignoni, and C. Montani, “3-D scanning and rendering cultural heritage artifacts on a low budget,” presented at the Proc. Eur. Workshop High Performance Graphics Systems Applications, Cineca, Bologna, Italy, Oct. 16–17, 2000. [19] C. Rocchini, P. Cignoni, C. Montani, P. Pingi, and R. Scopigno, “A low cost 3-D scanner based on structured light,” in Proc. Computer Graphics Forum, vol. 20, 2001, pp. C/299–C/308. [20] M. Andreetto, R. Bernardini, G. M. Cortelazzo, and L. Lucchese, “Toward automatic modeling of 3-D cultural heritage,” Proc. IEEE Int. Conf. Image Processing, vol. 1, pp. 574–577, 2001. [21] F. Bernardini, I. M. Martin, and H. Rushmeier, “High-quality texture reconstruction from multiple scans,” IEEE Trans.Vis. Comput. Graph., vol. 7, pp. 318–332, Oct./Dec. 2001. [22] G. Godin, J.-A. Beraldin, M. Rioux, M. Levoy, L. Cournoyer, and F. Blais, “An assessment of laser range measurement of marble surfaces,” in Proc. 5th Conf. Optical 3-D Measurement Techniques, Vienna, Austria, Oct. 1–4, 2001. [23] P. Cignoni, C. Montani, E. Puppo, and R. Scopigno, “Multiresolution representation and visualization of volume data,” IEEE Trans. Visual. Comput. Graphics, vol. 3, pp. 352–369, 1997. [24] P. Cignoni, E. Puppo, and R. Scopigno, “Representation and visualization of terrain surfaces at variable resolution,” Vis. Comput., vol. 13, no. 5, pp. 199–217, 1997.

676

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART B: CYBERNETICS, VOL. 33, NO. 4, AUGUST 2003

[25] G. Guidi, D. Ostuni, F. Costantino, M. Pieraccini, G. Tucci, and J.-A. Beraldin, “Photogrammetry and 3-D scanning: Assessment of metric accuracy for the digital model of Donatello’s Maddalena,” presented at the Proc. Italy-Canada Workshop 3-D Digital Imaging Modeling Applications: Heritage, Industry, Medicine & Land, Padua, Venetia, Italy, Apr. 3–4, 2001. [26] S. F. El-Hakim, C. Brenner, and G. Roth, “A multi-sensor approach to creating accurate virtual environments,” ISPRS J. Photogramm. Remote Sensing, vol. 53, pp. 379–391, 1998. [27] F. Blais, J.-A. Beraldin, S. F. El-Hakim, and L. Cournoyer, “Comparison of pose estimation methods of a 3-D laser tracking system using triangulation and protogrammetry techniques,” in Proc. SPIE, vol. 4309, San Jose, CA, Jan. 22–23, 2001, pp. 185–194. [28] F. E. Raab et al., “Magnetic position and orientation tracking system,” IEEE Trans. Aerosp. Electron. Syst., vol. 5, pp. 709–717, 1979. [29] Y. Chen and G. Medioni, “Object modeling by registration of multiple range images,” Image Vis. Comput., vol. 10, no. 3, pp. 145–155, 1992. [30] P. J. Besl and N. D. McKay, “A method for registration of 3-D shapes,” IEEE Trans. Pattern Anal. Machine Intell., vol. 14, pp. 239–256, Feb. 1992. [31] S. Rusinkiewicz and M. Levoy, “Efficient variants of the ICP algorithm,” in Proc. 3rd Intl. Conf. 3-D Digital Imaging Modeling, 2001, pp. 145–152. [32] C. S. Fraser, “Network design,” in Close Range Photogrammetry and Machine Vision, K. B. Atkinson, Ed, Caithness, U.K.: Whittles, 2001, pp. 256–282. [33] G. Sansoni, S. Corini, S. Lazzari, R. Rodella, and F. Docchio, “Three-dimensional imaging of surfaces for industrial applications: Integration of structured light projection, Gray code projection, and projector-camera calibration for improved performance,” in Proc. SPIE, vol. 2661, pp. 88–96. [34] G. Sansoni, M. Carocci, and R. Rodella, “3-D vision based on the combination of Gray code and phase shift light projection,” Appl. Opt., vol. 38, no. 31, pp. 6565–6573, 1999. [35] S.-F. El-Hakim, J.-A. Beraldin, G. Godin, and P. Boulanger, “Two 3-D sensors for environment modeling and virtual reality: Calibration and multi-view registration,” Int. Arch. Photogramm. Remote Sensing, vol. 31, no. B5, pp. 140–146, 1996. [36] D. H. Ballard and C. M. Brown, Computer Vision. Englewood Cliffs, NJ: Prentice-Hall, 1982, p. 523. [37] P. R. Wolf, Elements of Photogrammetry. New York: McGraw-Hill, 1974. [38] B. K. P. Horn, “Closed-form solution of absolute orientation using unit quaternions,” J. Opt. Soc. Am. A, vol. 4, no. 4, pp. 629–642, 1987. [39] M. Soucy and D. Laurendeau, “A general surface approach to the integration of a set of range views,” IEEE Trans. Pattern Anal. Machine Intell., vol. 4, pp. 344–358, Apr. 1995.

Gabriele Guidi (M’91–SM’99) received the E.E. degree in 1988 from the University of Florence, Florence, Italy and the Ph.D. degree in bioengineering in 1992 from the University of Bologna, Bologna, Italy. He joined the University of Florence as Researcher in 1994, developing, for more than ten years, novel ultrasound equipment for biomedical applications. In the last three years, he reoriented his activity on applications of imaging technologies to the field of Cultural Heritage conservation focusing his research on optoelectronics methodologies for 3-D modeling. Dr. Guidi is a member of the 3-Dim Conference scientific committee, and of the IEEE Signal Processing, Geoscience and Remote Sensing, and Systems, Man, and Cybernetics Societies.

Jean-Angelo Beraldin received the B.Eng. degree from the Université de Sherbrooke, Sherbrooke, QC, Canada, and the M.A.Sc. degree from the University of Ottawa, Ottawa, ON, Canada, in 1984 and 1986, respectively, both in electrical engineering. His current research interests include sensor systems engineering, signal processing hardware and software for 3-D vision systems, optoelectronic aspects of range sensors, as well as metrology. He is on many program conference committees like 3-DIM and 3-DPVT. He is very active in the field of Heritage and has given many tutorials throughout the world on active 3-D vision and heritage applications of 3-D vision.

Stefano Ciofi was born in Florence, Italy, in 1974. He received the E.E. degree from the University of Florence, Florence, Italy, in 2001. He is currently pursuing the Ph.D. degree in electronic systems engineering at the University of Florence. His research interests include the application to Cultural Heritage of three-dimensional scanning techniques.

Carlo Atzeni received the degree in physics from the University of Florence, Florence, Italy in 1965. He has been working as Full Professor of Microelectronics at the University of Florence since 1980. His scientific activity has been mainly concerned with signal processing techniques and devices, with emphasis on imaging systems, particularly based on microwave and ultrasound, with application to radar and medical diagnostics. Recently, he has organized an important research project for the Italian Ministry of Research, concerning remote monitoring and control of architectural Heritage, and promoted the application of 3-D modeling techniques to Italian Heritage conservation.