OBJECT BASED DIGITAL NON-METRIC IMAGES

Photogrammetry and Remote Sensing

OBJECT BASED DIGITAL NON-METRIC IMAGES ACCURACY

Assistant PhD student Eng. Valeria Ersilia Oniga Lecturer PhD Eng. Constantin Chirilă “Gheorghe Asachi” Technical University of Iasi, Department of “Terrestrial Measurements and Cadastre”, Romania

ABSTRACT In order to obtain the 3D model of an object, many methods and data sources have been used, like: terrestrial laser scanner, traditional technologies of terrestrial photogrammetry, digital non-metric images and surveying techniques. 3D modelling using multiple non-metric images represents one of the most attractive and provocative researching domain in photogrammetry, since, in recent years non-metric cameras have known a great development. In this paper the accuracy of the “PhotoModeler Scanner” software from Eos Systems, Inc. in 3D reconstruction of an object, based on non-metric digital images, acquired with a Canon EOS Rebel XSi/450D digital camera was tested. The aim of this research is to determine the degree of confidence that can be given to non-metric digital images when using them to reconstruct an object in 3D. To obtain the goal, a sphere has been photographed with three different focal lengths: 18 mm, 35 mm and 55 mm. For this case study, the digital camera was calibrated using both a 3D object and a planar target point array. Finally, the 3D models obtained by digital close-range photogrammetry were compared with the ones measured with the help of a CMM (Coordinate Measuring Machine) produced by Aberlink, with a 2µm precision, the accuracy of this 3D models being evaluated. Keywords: close-range photogrammetry, 3D reconstruction, sphere, CMM, calibration INTRODUCTION In recent years non-metric digital cameras are used to obtain metric information from our environment, due to the technological developments of this cameras, in areas such as accident reconstruction, industrial inspection and some heritage recording projects, but also metric cameras are still of great use in areas such as architecture. In order to achieve accurate information of the three-dimensional (3D) world, the camera calibration process is an important task. The purpose of camera calibration is to describe the projection model that relates both coordinate systems, and to identify the intrinsic camera parameters so it can be used as a measurement device. Such parameters are usually calculated from a calibration target that contains easily and accurately detectable features in the captured image. There are many methods for this process that have been developed by several authors, such as: Tsai [1] Heikkilä [2],[3], Bakstein & Halir [4] and Zhang [5]. A review of methods and some software for digital camera calibration are presented in [6].

13th International Multidisciplinary Scientific GeoConference SGEM 2013

This paper presents Heikkilä and Silven’s method to determine the parameters of a Canon EOS Rebel XSi/450D digital non-metric camera using a 3D calibration object and also the calibration process using a two-dimensional calibration field. Calibration using 3D calibration objects yields very efficient results, although the calibration elements must be accurate and require an elaborate configuration [5]. A non-metric camera is a camera whose interior orientation is completely or partially unknown and frequently unstable. All “off the shelf” or “amateur” cameras belong to this category are perhaps rather easily classified by the lack of fiducial marks [7]. The motivation of this paper is to determine the degree of confidence of a non-metric camera to reconstruct an object in 3D. MATHERIALS AND METHOD For this study, a Canon EOS Rebel XSi /450D digital SLR camera (12.2 Mega pixel), equipped with a 22.2 mm by 14.8 mm image sensor and a Canon EF-S 18-55mm IS lens, was used. The digital image has a resolution of 4272 x 2848 pixels with a 5.2 µm pixel size. A 3D target consisting in a number of 42 points placed in the corners of 9 wood cubes with different heights and on a board was used as a 3D calibration object for the calibration model. This target was then attached to a room wall in order to make image observations. Relative movements of targets on walls and so forth must be considered if the calibration range is to remain operative for a long period. A coordinate measuring machine (CMM), produced by Aberlink, with an uncertainty within the working space of 2 µm, was used to place the target in the world coordinate system (Figure 1 a,b). The 42 control points coordinates were measured in the (X, Y, Z) coordinate system, corresponding to the world coordinate system, with the XOY plane, the board plane and the OZ axis perpendicular to the board plane as it can be seen in Figure 1c. For each control point were measured four points, the software automatically determining the best fit circle of the four points and then computing, all automatically the coordinates of the circle center and the circle diameter. The coordinates of the control points were registered automatically to a computer. For the calibration process using a 3D calibration target, the camera calibration toolbox for Matlab (version 3.0) implementing the Heikkila & Silven’s method was used. This Matlab toolbox is available at [8] and utilizes a new bias correction procedure for circular control points and a nonrecursive method for reversing the distortion model. Also, the technique developed by Heikkila & Silven (1997) first extracts initial estimates of the camera parameters using a closed-form solution (DLT-Direct linear transformation) and then a nonlinear least-squares estimation is applied to define the interior orientation and compute the distortion parameters. The model uses two coefficients for both radial and decentering distortion, and the method works with single or multiple images and with 2D or 3D calibration grids [5]. The two-dimensional calibration field consist of a planar object point array, namely 100 points, distributed on 10 rows and 10 columns, printed on an A4 sheet. This planar sheet was placed on a flat surface that is a board with a uniform texture and color, as shown in

Photogrammetry and Remote Sensing

Figure 1 d, this shortening the time to determine the parameters of the camera used. The camera was placed on a tripod to ensure the camera and image stability.

(a)

(b)

(c)

(d)

Figure 1: (a), (b) Measuring the coordinates of the control points using a CMM (c) 3D calibration object in the 3D world coordinate system, (d) the planar testfield RESULTS AND DISCUSSION 1 The Canon EOS Rebel XSi /450D digital camera calibration using the 3D calibration target The calibration process was carried out using the 3D test field and 7 images acquired with the digital camera at 18mm focal length, 5 images at 35mm focal length and 5 images at 55mm focal length. The radial and decentering distortion profiles for the Canon EOS Rebel XSi/450D digital camera, were computed based on the calibration process resulted values for each focal length. To model the radial distortion (Δr) the odd-order polynomial Δr = k1r3 + k2r5 was used, where r is the radial distance. The decentering distortion P(r) was graphically represented in a manner analogous to the radial distortion, using the function P( r )  r 2  p12  p22 . Table 1: Physical Canon EOS Rebel XSi /450D digital camera parameters calculated for 3 focal lengths using a 3D calibration target Focal length f[mm] f=18 mm 18.4207 f=35 mm 34.5876 f=55 mm 52.5034

k1[mm] 5.98895•10-4 1.46237•10-4 3.77911•10-5

k2[mm] -1.33324•10-6 -6.82524•10-7 -3.48779•10-7

p1[mm] -7.12068•10-5 -1.07228•10-4 -6.78719•10-5

p2[mm] -1.46256•10-4 -1.83018•10-5 -1.41437•10-5

(a) (b) Figure 2: Radial and decentering distortion profiles for the Canon 450D digital camera, computed for 3 different focal lengths using a 3D calibration target

13th International Multidisciplinary Scientific GeoConference SGEM 2013

2 The Canon EOS Rebel XSi /450D digital camera calibration using the 2D calibration target The images of the 2D calibration target, taken from the four sides of the planar target at various rotations (12 for each focal length), were processed using the “PhotoModeler Scanner” software, which automatically identified the points of the planar object point array (4 control points and 96 extra points) and calculated the calibration parameters of the camera using the "Camera Calibration Project ". In the case of the calibration process performed for the 18mm focal length, the average photo point coverage was 87%, the maximum error of marking the points was 0.7288 pixels and the minimum error of 0.0806 pixels (the overall residual was 0.113 pixels, the maximum RMS was 0.457 pixels, the minimum RMS was 0.068 pixels). In the case of the calibration process performed for the 35mm focal length, the average photo point coverage was 82%, the maximum error was 0.4632 pixels and the minimum error of 0.0674 pixels (the overall residual was 0.109 pixels, the maximum RMS was 0.225 pixels, the minimum RMS was 0.044 pixels). In the case of the calibration process performed for the 55mm focal length, the average photo point coverage was 87%, the maximum error was 0.3631 pixels and the minimum error of 0.0632 pixels (overall residual-0.125, maximum RMS-0.306 pixels, minimum RMS-0.048 pixels). The resulting parameters for the 3 focal lengths (18 mm, 35 mm and 55 mm) are found in Table 2. Table 2: Physical Canon EOS Rebel XSi /450D digital camera parameters calculated for 3 focal lengths using a 2D calibration target Focal length f[mm] k1[mm] k2[mm] p1[mm] p2[mm] f=18 mm 18.4673 5.8790•10-4 -1.3600•10-6 -1.3200•10-4 2.2070•10-5 f=35 mm 34.8640 3.9260•10-5 -1.2290•10-7 -9.3810•10-5 2.2380•10-5 f=55 mm 52.6404 -2.1990•10-5 -1.743•10-8 -6.3230•10-5 3.0850•10-5

(a)

(b)

Figure 3: Radial and decentering distortion profiles for the Canon 450D digital camera, computed for three different focal lengths using a 2D calibration target 3. 3D reconstruction of a sphere In order to determine the degree of confidence that can be given to non-metric digital cameras when using them to reconstruct an object in 3D (in these experiments a small size object), we chose a sphere whose 3D model was created based on the digital images taken with the non-metric camera, Canon EOS Rebel XSi /450D. First, the sphere was measured with a precision of 2 μm using a CMM, then was placed on a 2D grid of

Photogrammetry and Remote Sensing

squares and photographed all around with 3 different focal length (f = 18 mm, 35 mm and 55 mm) being taken 12 images for each focal length, from 12 different camera positions distributed circularly around the object. The artificial targets placed all over the object surface and also the corners of the 2D grid used to scale the object, were marked on each image individually (Figure 4 a). By processing the data, using the Bundle adjustment algorithm implemented in the Photomodeler Scanner software, the 3D coordinates of all referenced points were computed, their measurement precision was estimated and also the spatial locations and orientation angles of each camera were calculated. Thus, were computed the 3D coordinates of a number of 185 targets, of which a total of 20 points represents the 2D grid corners (Figure 4 a,b). Knowing that the 2D grid size is 10 mm, the object was scaled using a 100 mm line whose ends were clearly identified in the images.

(a)

(b)

(c)

Figure 4: (a)The artificial control points placed on the sphere surface, a screen shot of the 3D Viewer with the final set of three-dimensional control points and cameras displayed, (b) perspective view, (c) top view To approximate the mathematical shape of the object, namely with a sphere, the leastsquares method was used. The starting point was the implicit equation of a sphere, with the form: ( x  a)2  ( y  b)2  ( z  c)2  r 2  0

(1)

where: r- the sphere radius and (a,b,c)-the sphere center coordinates. The following function will be considered:

Fi  ri 2  r 2  ( xi  a)2  ( yi  b)2  ( zi  c)2  r 2

(2)

The equation (2) can be used for initial values of the sphere center coordinates and the sphere radius calculation. The equation for a point Pi on the sphere circumference is:

Fi (a, b, c, r )  xi 2  yi 2  zi 2  2axi  2byi  2czi  (a 2  b2  c2  r 2 )  0

(3)

We note the term (a2+b2+c-r2) with ρ and the equations system for n set of data points, written in matrix form is:

 2  x1   2  x2  ...   2  xn

2  y1

2  z1

2  y2

2  z2

...

...

2  yn

2  zn

2 2 2 1   ao   x1  y1  z1   F1         1   bo   x22  y22  z22   F2      ...  co   ...   ...      1   o  x 2  y 2  z 2   Fn  n n  n

(4)

13th International Multidisciplinary Scientific GeoConference SGEM 2013

In order to solve this system by the least-squares method, Fi=0 is considered. The equations system (4) can be written in matrix form as: BX+L=0. The system solution is the X vector of the following form: X=-(BTB)-1BTL. The initial estimates for the sphere center coordinates and the sphere radius, in the case of 18 mm focal length and 3D target parameters, are: ao  81.2510mm; bo  172.3773mm; co  -50.2532mm; ro  49.8119mm

The sphere that best fit the measured points is obtained by minimizing the distances (perpendiculars) drawn from each point to the sphere circumference. The Gauss Newton method is used to arrive at the final values for center coordinates and radius. The function is linearized by a Taylor series expansion around four provisional values (a0, b0, c0, r0) and the first order terms are detained:

 G   G   G   G  Gi (a, b, c, r )  Gi (ao , bo , co , ro )   i  da   i  db   i  dc   i  dr  ...  a o  b o  c o  r o

(5)

where: Gi(a,b,c,r)=di=ri-r; ao, bo, co and ro – the provisional values of the unknowns obtains by solving the equations system (4);

ri  ( xi  ao )2  ( yi  bo )2  ( zi  co )2

- the distances drawn from each point to the

sphere center. By conveniently grouping the terms, the unknown parameters of the sphere corrections equations system results, written in matrix form [9]:

Br,4 X 4,1  Lr,1  Vr,1 , with Pr,r  ( x1  ao )  r1   ( x2  ao ) where: Br ,4   r2  ...   ( xr  ao )  rr 

( y1  bo ) r1

( z1  co ) r1

( y2  bo ) r2

( z2  co ) r2

... ( yr  bo ) rr

... ( zr  co ) rr

(6)  1   da   v1       db  1   v2  , X  , , V  4 , 1 r , 1   dc   ...  ...       dr    vr   1 

1 / s 2   2 2 2 t1   ( x1  ao )  ( y1  bo )  ( z1  co )  ro     Lr,1  ............................................................. , P  ...   r ,r   0  ( x  a )2  ( y  b )2  ( z  c )2  r   r o r o r o o  

0   ... ...   2 0 1 / st  r 

...

The weights matrix has on its main diagonal the inverses of the squares of the spatial errors corresponding to the artificial control points and all the other elements equal to zero. The normal equations system is:

Nn,n X n,1  Tn,1  On,1 , where N  BT PB and T  BT PL

(7)

Photogrammetry and Remote Sensing

1 1 from where the unknowns vector is: X n,1   Nn,nTn,1  Qn,n  Tn,1 ( Q  N )

The determination precision of the sphere radius from the unknown matrix is expressed with the mean square error:

[V T PV ] sr  so Q4,4  0.069mm, where so  , with r  165 and n  4 r n

(8)

The process is iterative until all unknowns do not change significantly. A routine in the MATLAB programming language was written. First a *. txt file was created containing a column with the detailed point number, three columns with the coordinates (X, Y, Z) in the local system defined in the Photomodeler Scanner software and three columns containing the measurement accuracies of the three coordinates. For the points measured on digital images taken with 18 mm focal length and the parameters obtained with the 3D calibration target, a total number of three iterations was effectuated, obtaining the maximum initial correction (after the first iteration) of 0.2276 mm and the minimum correction of 0.01µm, the final values for the sphere center coordinates and the sphere radius, determined with a precision of 0.069mm, being:

a  81.2168mm; b  172.4016mm; c  -50.1849mm; r  49.8171mm The sphere radius measured with the help of a CMM has a value of 49.6190 mm. The spheres rays derived from measurements made on digital images taken with three focal lengths and the differences between them and the one measured using the CMM are listed in Table 3. Table 3: The spheres rays approximated by the measured points on the digital images using the parameters of the 2D and 3D calibration field, as well as the residual errors Focal length

18mm 35mm 55mm

Sphere radius 2D target 3D target parameters parameters

50.0032 49.9473 49.7621

49.8171 49.6260 49.5504

RMSE [mm] 2D target 3D target parameters parameters

0.3842 0.3283 0.1431

0.1981 0.0070 -0.0686

RMSE [%] 2D target 3D target parameters parameters

0.7743 0.6616 0.2884

0.3992 0.0141 -0.1383

CONCLUSION All lenses, whether of a metric, semi-metric or non-metric photogrammetric cameras, suffer from aberrations, that are known as radial and decentering distortion. Before any camera system is used for photogrammetric projects, the intrinsic parameters of the camera: focal length (f), optical center point (u0,v0), correction of radial distortion (k1, k2), correction of tangential distortion (p1, p2) and the medium image scale factor su, as well as the extrinsic parameters (rij, X0, Y0, Z0) should be determined in the calibration process. A non-metric camera, the Canon EOS Rebel XSi /450D digital SLR camera, was calibrated in this article using both 2D and 3D calibration target.

13th International Multidisciplinary Scientific GeoConference SGEM 2013

The accuracy in 3D reconstruction of an object is directly proportional to the scale factor, so it is recommended to use a 2D grid to be photographed simultaneously with the object. The best accuracy was obtained with the calibration parameters calculated based on the 3D calibration target. Using artificial control points placed all over the object surface, a much better measurement precision can be obtained than in the case of the natural control points. The minimum 3D reconstruction error of the sphere was 7 µm, representing 1:7000 from the object dimension and the maximum 3D reconstruction error was 0. 38 mm, representing 1:130 from the object dimension. These experiments represent an analysis of the degree of confidence when using a nonmetric camera to reconstruct an object in 3D, because in areas such as camera based 3D measurements and robot vision, high geometrical accuracy is needed. Thus, the analysis performed on the 3D models of the sphere, it can be said that the degree of confidence that can be given to a non-metric digital camera in the process of 3D reconstruction of an object is 99%, considering that the maximum RMS error is approximately 1%. REFERENCES [1] Tsai, R.Y. A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses, IEEE Int. Journal Robotics and Automation, Vol. 3(4), pp. 323-344, 1987. [2] Heikkila J., Silven O. A four-step camera calibration procedure with implicit image correction. In CVPR, 1997. [3] Heikkila J. – Geometric camera calibration using circular control points, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 22, No. 10, pp. 10661077, October 2000. [4] Bakstein, H., Halir, R. Camera Calibration with a Simulated Three Dimensional Calibration Object, Czech Pattern Recognition Workshop 2000, Tomáš Svoboda (Ed.), Peršlák, Czech Republic, February 2–4, 2000. [5] Zhang, Z. A flexible new technique for camera calibration, IEEE Trans. on PAMI, Vol. 22(11), pp. 1330-1334, 2000. [6] Remondino, F., Fraser, C. Digital camera calibration methods: considerations and comparisons, IAPRS Volume XXXVI, Part 5, Dresden 25-27 September 2006. [7] Faig, W. Photogrammetric Potentials of non-metric cameras- Report of ISP Working Group V/2 18th ISP Congress, Helsinki, Photogrammetric Engineering and Remote Sensing: 47-49, 1976. [8] www.ee.oulu.fi/~jth/calibr/ [9] Oniga V. E., Chirila C. Complex roof structures generalized representation established by approximating the mathematical shape using the least-squares method, Scientific Journal „Mathematical Modelling in Civil Engineering”, Vol.8. no.4, pp. 178187, December 2012, Bucharest, Romania, ISSN 2066-6926, 2012.