Improved Direct Linear Transformation for Parameter ...

algorithms Article

Improved Direct Linear Transformation for Parameter Decoupling in Camera Calibration Zhenqing Zhao 1 , Dong Ye 1, *, Xin Zhang 1 , Gang Chen 1 and Bin Zhang 2 1 2

*

School of Electrical Engineering and Automation, Harbin Institute of Technology, Heilongjiang 150001, China; [email protected] (Z.Z.); [email protected] (X.Z.); [email protected] (G.C.) Capital Aerospace Machinery Company, Beijing 100000, China; [email protected] Correspondence: [email protected]; Tel.: +86-186-4603-2603 (ext. 123); Fax: +451-8641-7581

Academic Editor: Faisal N. Abu-Khzam Received: 11 December 2015; Accepted: 25 April 2016; Published: 29 April 2016

Abstract: For camera calibration based on direct linear transformation (DLT), the camera’s intrinsic and extrinsic parameters are simultaneously calibrated, which may cause coupling errors in the parameters and affect the calibration parameter accuracy. In this paper, we propose an improved direct linear transformation (IDLT) algorithm for calibration parameter decoupling. This algorithm uses a linear relationship of calibration parameter errors and obtains calibration parameters by moving a three-dimensional template. Simulation experiments were conducted to compare the calibration accuracy of DLT and IDLT algorithms with image noise and distortion. The results show that the IDLT algorithm calibration parameters achieve higher accuracy because the algorithm removes the coupling errors. Keywords: camera calibration; direct linear transformation; parameter decoupling

1. Introduction With the technological developments of digital cameras and microprocessors, computer vision has been widely applied to robot navigation, surveillance, three-dimensional (3D) reconstruction and other fields for its high speed, high accuracy and non-contact nature. To obtain improved 3D information from a two-dimensional (2D) image, it is necessary to calibrate the intrinsic parameters of the camera, such as the focal distance and optical center point, as well its extrinsic parameters, such as rotation and translation, which relate the world coordinate system to the camera coordinate system. Over the last decade, numerous studies have focused on this area. In [1], the authors proposed an efficient approach for the dynamic calibration of multiple cameras. In [2], the authors proposed a calibration algorithm based on line images. In [3], the authors used one-dimensional (1D) information to calibrate the parameters. All of these algorithms make camera calibration faster and more convenient. To obtain high accuracy parameter results, high accuracy 3D or 2D templates can be used. These algorithms include direct linear transformation (DLT) [4], the Tsai calibration method [5] and the Zhang calibration method [6]. In space rendezvous and docking, as well as in visual tracking applications, it is necessary to obtain the specific extrinsic and intrinsic parameters of the camera. At the same time, with the wide application of the coordinate measuring machine (CMM) [7,8], high precision, large range 3D templates have become more readily applied. The DLT algorithm is more suitable for these applications. The DLT algorithm is based on the perspective projection between 3D space points and 2D image points. It calculates a transformation matrix and obtains the camera’s intrinsic and extrinsic parameters according to the parameter decomposition. With this model, only one 3D template in one position is required for calculation; therefore, the template size, number of feature points and relative distance between the template and camera are critical [9–11]. Algorithms 2016, 9, 31; doi:10.3390/a9020031

www.mdpi.com/journal/algorithms

Algorithms 2016, 9, 31

2 of 15

Algorithms 2016, 9, 31

2 of 15

Camera calibration with image noise and distortion remains a difficult task. Algorithms typically solve Camera the camera parameters by analysis; then, they solve theamodel oftask. image noise and distortion by calibration with image noise and distortion remains difficult Algorithms typically optimization [6,12,13]. All traditional algorithms visual such solve the camera parameters by analysis;optimization then, they solve the modelapplied of imagetonoise andsystems, distortion by as the modified Newton algorithm, the Levenberg–Marquardt algorithm and the genetic algorithm, require a optimization [6,12,13]. All traditional optimization algorithms applied to visual systems, such as the good initial solution to optimize. Therefore, the analytic solution is not only robust to image noise, but modified Newton algorithm, the Levenberg–Marquardt algorithm and the genetic algorithm, require a good initial for solution to optimize. Therefore, the analytic solution is not only robust to image noise, also effective addressing distortion. but also effective for addressing distortion. When the calibration data contain noise and distortion, coupling errors exist between the camera’s When the calibration data contain noise and distortion, coupling exist parameter between themay be extrinsic and intrinsic parameters. This means that an error in anerrors extrinsic camera’s extrinsic and intrinsic parameters. This means that an error in an extrinsic parameter may compensated by an error in an intrinsic parameter [14–16]. Existing parameter decoupling methods use be compensated by an error in an intrinsic parameter [14–16]. Existing parameter decoupling calibration models without considering extrinsic parameters. These include vanishing points [15,17,18], methods use calibration models without considering extrinsic parameters. These include vanishing straight lines [16,19], and cross ratios [20,21]. In the present study, the image re-projection error from points [15,17,18], straight lines [16,19], and cross ratios [20,21]. In the present study, the image rethe mathematical model and the camera pinhole geometric model were analyzed. The results show projection error from the mathematical model and the camera pinhole geometric model were that the coupling error causes small re-projection error variance and a large parameter analyzed. The results show thatathe coupling error causes a small re-projection errorcalibration variance and a error Atparameter the sameerror time,variance. this variance related to this the template number feature points largevariance. calibration At theissame time, variance issize, related to theoftemplate and between therelative template and camera. size,relative numberdistance of feature points and distance betweenTo theimprove templatethe andcalibration camera. Toaccuracy improve of the camera parameters, a decoupling algorithm of intrinsic and extrinsic is proposed the calibration accuracy of the camera parameters, a decoupling algorithmparameters of intrinsic and extrinsic based on DLT. parameters is proposed based on DLT. The remainder remainder ofof this paper is organized as follows. The relationship between the couplingthe error, The this paper is organized as follows. The relationship between coupling template size and relative distance between the template and camera is described in Section 2. The error, template size and relative distance between the template and camera is described in Section 2. proposed improved DLT DLT (IDLT) algorithm for camera calibration is proposed in Section 3. The 3. The The proposed improved (IDLT) algorithm for camera calibration is proposed in Section experimental results are presented in Section 4, and the conclusions are given in Section 5. experimental results are presented in Section 4, and the conclusions are given in Section 5. Parameter Coupling 2.2. Parameter CouplingAnalysis Analysis 2.1.Camera Camera Model Model 2.1. T The camera model is shown in Figure 1. A 3D point is denoted by Pi = [ X i Yi Zi ]” . A 2D image ıT The camera model is shown in Figure 1. A 3D point is denoted by P “ . A Xi Yi Zi i T point is denoted by pi = [ui vi ] . The ” relationship ıT between 3D point Pi and its image projection, 2Dp image point is denoted by pi “ ui νi . The relationship between 3D point Pi and its image i , is given by: projection, pi , is given by: r X + r12iY`i r+12rY13i Z `ir+ 13 tZxi `t x u0x`11a xi rr11 X ui u=i u“ 0 +a Xi `r32 Yi `r33 Zi `tz 31 r31 X i + r32Yi + r33 Z i + t z (1) (1) r21 X ir+ r22i Y r Z +t `i r+ 22 Y23i `ri 23 Zyi `ty 21 X vi ν=i v“ 0 +νa 0y` ay r X `r32 Y `r33 Z `tz

31 r i Y + r i Z + t i r31 X i + z 32 i 33 i

where ax = f /dx , ay = f /dy and f is the camera focal distance. In addition, dx , dy are pixel sizes in the where ax = f/dx, ay = f/dy and f is the camera focal distance. In addition, dx, dy are pixel sizes in the horizontal verticaldirections; directions;(u(u 0 , v0 ) is the optical center point of the image; and R, T are the horizontal and and vertical 0, v0) is the optical center point of the image; and R, T are the rotation andtranslation translationvector, vector, respectively, which relate the world coordinate rotation matrix matrix and respectively, which relate the world coordinate systemsystem to the to the camera coordinate system. Furthermore, r is the i-th row and j-th column element of R, camera coordinate system. Furthermore, rij isij the i-th row and j-th column element of R, and txand , ty, tztx , ty , tz are therotation rotationmatrix matrixisisrepresented represented Euler angles, arethe theelements elements of of T. T. For simplicity, simplicity, the byby Euler angles, ψ ,ψ,θ θ, , φφ,, which represent the rotation around the respective x, y and z axes. which represent the rotation around the respective x, y and z axes.

Figure 1. Camera model. Figure 1. Camera model.

Algorithms 2016, 9, 31

3 of 15

Owing to the influence of image noise and distortion, the calibration parameters, image points and space points do not completely conform to Equation (1). Thus, we have: `r12 Y`r13 Z`t x ∆ui “ ui ´ u0 ´ a x rr11 X X `r32 Y `r33 Z`tz 31

(2) ∆νi “

r X `r Y `r23 Z`ty νi ´ ν0 ´ ay r21 X`r22 32 Y `r33 Z`tz 31 n ÿ

min f pu, νq “

p∆u2i ` ∆νi2 q

(3)

i “1

Equation (2) describes the calculation of image re-projection errors. Small re-projection errors are desired, which means that the calibration results satisfy Equation (3). However, the question arises of if a small image re-projection error will lead to the high calibration accuracy of the camera parameters. Analysis of the relationship between the image re-projection error and the calibration accuracy is therefore required. 2.2. Error Coupling Analysis The two equations in Equation (2) have the same form. We thus analyze the first equation; all conclusions can apply to the second equation. We assume that u0 has error 4u0 and that tx has error 4tx . From Equation (2), we have: r X `r Y `r Z `pt `∆t x q

x ∆u1 i “ ui ´ pu0 ` ∆u0 q ´ a x 11 ri X12`ri32 Y13`ri33 Z ` 31 i i i tz ax “ ∆ui ´ p∆u0 ` r X `r32∆tYx ` r33 Z `tz q 31

i

i

(4)

i

when ∆t x “ ´pr31 Xi ` r32 Yi ` r33 Zi ` tz q{a x ∆u0 , we have ∆u1 i “ ∆ui . The image re-projection error will not change. However, because the space point coordinate values are different, other image point re-projection errors will be changed. We assume that: ∆t x “ ´ r31 X1 `r32 Ya1x`r33 Z1 `tz ∆u0 (5) “ ´ r31 Xi `r32 Yaix`r33 Zi `tz ∆u0 ´

r31 dXi `r32 dYi `r33 dZi ∆u0 ax

where dXi = X1 ´ Xi , dYi = Y1 ´ Yi , dZi = Z1 ´ Zi , and image re-projection errors are given by: ∆u1 i “ ∆ui ` ∆u2 i with ∆u2 i “

r31 dXi ` r32 dYi ` r33 dZi ∆u0 r31 Xi ` r32 Yi ` r33 Zi ` tz

(6)

Based on Equation (6), if dZi is very small and tz is very large, the change in image re-projection errors will be very small. This is because coupling 4u0 and 4tx reduces the image re-projection error. In order to reduce the coupling effect, better results can be obtained if the 3D template is large and close to the camera. In fact, the template size is limited, and the calibration distance is limited by the field of view and the lens parameter. With respect to focal distance and translation vector, we assume ax has error 4ax and tz has error 4tz . From Equation (2), we have: ∆u1 i “ ui ´ u0 ´

pa x ` ∆a x qCi Bi ` ∆tz

(7)

where Bi “ r31 Xi ` r32 Yi ` r33 Zi and Ci “ r11 Xi ` r12 Yi ` r13 Zi . We assume that ∆a x {∆tz “ a x {B1 , and we obtain ∆u1 1 “ ∆u1 . Other image re-projection errors are: ∆u1 i “ ∆ui ` ∆u2 i with ∆u2 i “ ´

a x Ci ∆tz a x Ci ∆tz ˆ «´ B1 pBi ` ∆tz q Bi B1

(8)

Algorithms 2016, 9, 31

4 of 15

Δui′ = Δui + Δui′′ with Δui′′ = − Algorithms 2016, 9, 31

ax Ci Δt z a C Δt ≈− x i × z B1 ( Bi + Δt z ) Bi B1

(8) 4 of 15

From Equation (1), we have ax Ci / Bi ≈ (ui − u0 ). We assume that the Euler angles of the world coordinate system relative to the camera coordinate system are zero. By setting the first space point From Equation (1), we have a x Ci {Bi « pui ´ u0 q. We assume that the Euler angles of the world Z1 = 1 , we have: coordinate system relative to the camera coordinate system are zero. By setting the first space point Z1 “ 1, we have: u −u Δ u i′′ = − i ui ´0 uΔ0t z (9) 2 ∆u i “ ´ t z ∆tz (9) tz u ) is small and t is large, the change in the image re-projection Based (9), if if(upu i − Basedon onEquation Equation (9), i ´0 u0 q is small and ztz is large, the change in the image re-projection

△t4 z t reduces thethe image re-projection error. error errorwill willbe bevery verysmall. small.This Thisisisbecause becausecoupling coupling△a 4xaand image re-projection error. x and z reduces To camera. Toreduce reducethe thecoupling couplingeffect, effect,better betterresults resultscan canbe beobtained obtainedififthe the3D 3Dtemplate templateisiscloser closertotothe the camera. Similar translation vector vector coupling couplingerror errorare arerelated relatedto Similartotothe thedistortion, distortion,the the camera camera focal focal distance and translation tothe theimage imagepoint point location. Therefore, under influence distortion, camera focal distance location. Therefore, under thethe influence of of distortion, thethe camera focal distance and and translation vector coupling error increase calibration error. translation vector coupling error maymay increase the the calibration error. The Thecoupling couplingerror errorisispresent presentininthe thegeometric geometricmodel. model.The Thecamera cameramodel modelisisalso alsocalled calleda apinhole pinhole model; model;that thatis,is,the thespace spacepoint, point,image imagepoint pointand andcamera cameraorigin originare arelocated locatedon onthe thesame sameline. line.Figure Figure2 2 shows center point pointof ofthe theimage imageand andthe thetranslation translation vector. When showsthe thecoupling couplingbetween between the the optical center vector. When the the optical center point of image c is offset, the origin the camera coordinate shifts. To optical center point of image Oc isOoffset, the origin of theof camera coordinate system system shifts. To continue continue fitting the model, pinholethe model, space i, under camera coordinate will fitting the pinhole spacethe point, Pi , point, under P the camerathe coordinate system willsystem produce an produce an offset. The space point under the world coordinate system is fixed so that the relationship offset. The space point under the world coordinate system is fixed so that the relationship between the between the world system coordinate camerasystem coordinate system changes. world coordinate and system camera and coordinate changes.

Figure2.2.Relationship Relationshipbetween betweenthe theprincipal principalpoint pointand andtranslation. translation. Figure

Figure Figure3 3shows showsthe thecoupling couplingbetween betweenthe thecamera camerafocal focaldistance distanceand andthe thetranslation translationvector. vector.The The camera focal distance is calibrated based on the relative distance between multiple space points. camera focal distance is calibrated based on the relative distance between multiple space points. When When the camera distance changes, is the origin of the camera coordinate system, , movesto the camera focalfocal distance changes, thatthat is the origin of the camera coordinate system, OcO, cmoves toO' O' c, the world coordinate system moves along the direction of the optical axis of the camera because c , the world coordinate system moves along the direction of the optical axis of the camera because the distance 2. When the Euler angles are zero, the value of the optical axis the distanceisisfixed fixedbetween betweenPP1 1and andPP 2 . When the Euler angles are zero, the value of the optical axis direction directionisis ttzz .. In summary, under the influence of image noise and distortion, when the size of the templates, number of feature points and relative distance between the template and camera are certain, coupling errors occur in the camera’s intrinsic and extrinsic parameters that affect the calibration parameter accuracy. Therefore, to improve the accuracy of the calibration parameters, it is necessary to remove the parameter coupling errors.

Algorithms 2016, 9, 31 Algorithms 2016, 9, 31

5 of 15 5 of 15

Figure 3. Relationship between the focal distance translation. Figure 3. Relationship between the focal distance and and translation.

3.InImproved Transformation summary,Direct underLinear the influence of image noise and distortion, when the size of the templates, number of feature points and relative distance between the template and camera are certain, coupling 3.1.occur DirectinLinear Transformation errors the camera’s intrinsic and extrinsic parameters that affect the calibration parameter accuracy.The Therefore, to improve accuracy the parameters, it is necessary remove DLT algorithm datesthe to the work ofof[4] orcalibration earlier. In [13,22], the authors analyzedto the definition the of parameter errors. the worldcoupling coordinate system and proposed the use of data normalization to reduce the noise impact on the camera calibration parameters. In [10,23], the authors proposed combining the DLT algorithm 3. Improved Direct Linear Transformation with an optimization algorithm to address the camera distortion models. The DLT algorithm consists of two steps: (1) homogeneous equation solving; and (2) parameter 3.1. factorization. Direct Linear Transformation Space point P and its image p are related by homography M: The DLT algorithm dates to the work of [4] or earlier. In [13,22], the authors analyzed the sp and “ MP with M the “ K[R|T] definition of the world coordinate system proposed use of data normalization to reduce the(10) noise impact on the camera calibration parameters. In [10,23], the authors proposed combining the where K is the camera intrinsic matrix. mij is the i-th row and j-th column element of M. DLT algorithm with an optimization algorithm to address the camera distortion models. The 12 parameters in M3ˆ4 are unknown in the matrix M'12ˆ1 . From Equation (10), we have: The DLT algorithm consists of two steps: (1) homogeneous equation solving; and (2) parameter factorization. Space point P and its image p are related by homography M: AM’ “ 0 with ||M'|| “ C (11) sp = MP with M = K[R|T] (10) where C is a constant, and: where K is the camera intrinsic matrix. mij is the i-th row and j-th column element of M. The 12 « ff « ff T matrix T . From Equation parameters in M3×4 are unknown inPthe M'12×1 weT have: 0 ´uP P T 0 (10), ´uP A“ A“ (12) T T ´νP T 0 AM’ P T = 0´νP with ||M'|| = C0 P (11)

where CBecause is a constant, and: (11) is a homogeneous equation, we assume that m34 is equal to one. The M Equation matrix can be solved by singularT value decomposition. We can then obtain the intrinsic and extrinsic P  PT 0 −uPT  0 −uPT  = = A parameters of the camera Athrough factorization. (12)    T T −vPT  −vPT  0 P 0 P a tz “ 1{ m31 2 ` m32 2 ` m33 2 (13) Because Equation (11) is a homogeneous equation, we assume that m34 is equal to one. The M matrix can be solved by singular value decomposition. We can then obtain the intrinsic and extrinsic u0 “ t2z pm11 m31 ` m12 m32 ` m13 m33 q (14) parameters of the camera through factorization.

Algorithms 2016, 9, 31

6 of 15

ν0 “ t2z pm21 m31 ` m22 m32 ` m23 m33 q

(15)

b a x “ t2z pm12 m33 ´ m13 m32 q2 ` pm11 m33 ´ m13 m31 q2 ` pm11 m32 ´ m12 m31 q2 b ay “ t2z pm22 m33 ´ m23 m32 q2 ` pm21 m33 ´ m23 m31 q2 ` pm21 m32 ´ m22 m31 q2

(17)

t x “ tz pm14 ´ u0 q{a x

(18)

ty “ tz pm24 ´ ν0 q{ay

(19)

r11 “ tz pm11 ´ u0 m31 q{a x r21 “ tz pm21 ´ ν0 m31 q{a x r31 “ tz m31

r12 “ tz pm12 ´ u0 m32 q{a x r22 “ tz pm22 ´ ν0 m32 q{a x r32 “ tz m32

r13 “ tz pm13 ´ u0 m33 q{a x r23 “ tz pm23 ´ ν0 m33 q{a x r33 “ tz m33

(16)

(20)

Accordingly, at least six non-coplanar feature points and their corresponding image points—the camera focal distance, optical center point, rotation matrix and translation vector—can be solved according to Equations (11)–(20). However, with the influence of image noise and distortion, mij contains errors. From Equations (18) and (19), we have: t x ` ∆t x “

ptz `∆tz qpm14 `∆m14 ´u0 ´∆u0 q a x `∆a x

ty ` ∆ty “

ptz `∆tz qpm24 ´u0 `∆m24 ´∆u0 q ay `∆ay

where 4tx , 4tz , 4m14 , 4ty and 4m24 are errors. When t x « ∆t x “

tz p∆m14 ´∆u0 q a x `∆a x

∆ty “

tz p∆m24 ´∆ν0 q ay `∆ay

`

∆tz pm14 ´u0 q a x `∆a x

`

∆tz pm24 ´ν0 q ay `∆ay

tz pm14 ´u0 q a x `∆a x , ty

`

∆tz p∆m14 ´∆u0 q a x `∆a x

`

∆tz p∆m24 ´∆ν0 q ay `∆ay

(21)

«

tz pm24 ´ν0 q ay `∆ay ,

we have:

(22)

In Equation (22), the third molecule is much smaller than the others and can be ignored. When tz ą t x and ∆u0 " ∆m14 , we have: ∆t x “ ´tz ∆u0 {a x (23) At the same time, when tz ą ty and ∆ν0 " ∆m24 , we have: ∆ty “ ´tz ∆ν0 {ay

(24)

With respect to the coupling error between the focal distance and translation vector, to simplify the analysis model, we assume that the Euler angles of the world coordinate system relative to the camera coordinate system are zero. Then, we have: »

a x {tz — M“– 0 0

0 ay {tz 0

u0 {tz ν0 {tz 1{tz

fi pt x a x ` tz u0 q{tz ffi pty ay ` tz ν0 q{tz fl 1

(25)

From Equation (13), we have: tz “ 1{m33

(26)

a x “ t2z m11 m33 “ m11 tz ay “ t2z m22 m33 “ m22 tz

(27)

From Equations (16) and (17), we have:

Algorithms 2016, 9, 31

7 of 15

a x = t z2 m11m33 = m11t z

(27)

a y = t z2 m22 m33 = m22 t z

Algorithms 2016, 9, 31

7 of 15

Owing to the influence of image noise and distortion, M contains errors. From Equation (27), we thusOwing have: to the influence of image noise and distortion, M contains errors. From Equation (27), we thus have: ax + Δax = m11t z + m11Δt z + Δm11t z + Δm11Δt z a x ` ∆a x “ m11 tz ` m11 ∆tz ` ∆m11 tz ` ∆m11 ∆tz (28) (28) ay + Δay = m22t z + m22 Δt z + Δm22t z + Δm22 Δt z ay ` ∆a “ m t ` m ∆t ` ∆m t ` ∆m ∆t y 22 z 22 z 22 z 22 z where4 △a 11, △a y and △m22 are errors. Δm Δt z and Δm Δt z 22are smaller than the others where axx,, △t 4tzz, ,△m 4m ∆tzmuch are much smaller than the 11 , 4ay and 4m22 are errors.11 ∆m 11 ∆tz and22∆m and can be ignored. Then, we have: others and can be ignored. Then, we have:

Δa = m Δt z + Δm11t z ∆a x “x m11 11 ∆tz ` ∆m11 tz ay m = m∆t Δt z +∆m Δm22ttz ∆ayΔ“ 22 22 z ` 22 z

(29) (29)

Onaccount accountofof∆tΔt"  Δm and Δt z ∆m Δm22, ,we wehave: have: On z z ∆m1111 and ∆tz " 22

Δa = m Δt ∆a x “x m1111∆tzz a y =mm22∆t Δtz ∆aΔ y “ 22 z

(30) (30)

Equations (23), (24) and (30) show that a linear relationship exists between the calibration Equations (23), (24) and (30) show that a linear relationship exists between the calibration parameters after ignoring some minor errors. We use a simulation to illustrate the size of some of the parameters after ignoring some minor errors. We use a simulation to illustrate the size of some ignored minor errors. In the simulation, we have ax = 1454.5, ay = 1454.5, u0 = 700 pixel and v0 = 512 pixel; of the ignored minor errors. In the simulation, we have ax = 1454.5, ay = 1454.5, u0 = 700 pixel and moreover, the size of the pattern is 0.7 m × 0.7 m × 0.3 m. The relationship of the world coordinate v0 = 512 pixel; moreover, the size of the pattern is 0.7 m ˆ 0.7 m ˆ 0.3 m. The relationship of the system relative to the camera coordinate system relationship is represented by R = [20, 20, 20] (°) and world coordinate system relative to the camera coordinate system relationship is represented by T = [−0.25, −0.25, tz] (mm). Gaussian noise with a zero mean and a 0.1-pixel standard deviation is R = [20, 20, 20] (˝ ) and T = [´0.25, ´0.25, tz ] (mm). Gaussian noise with a zero mean and a 0.1-pixel added to the projected image points. The analysis of Equations (23) and (24) is shown in Figure 4. The standard deviation is added to the projected image points. The analysis of Equations (23) and (24) is difference between △ax, △ay and the results of Equations (23) and (24) is small. The analysis of shown in Figure 4. The difference between 4ax , 4ay and the results of Equations (23) and (24) is small. Equation (30) is shown in Figure 5. Because the Euler angles are not equal to 0°, △tx, △ty and the result The analysis of Equation (30) is shown in Figure 5. Because the Euler angles are not equal to 0˝ , 4tx , of Equation (30) are different. However, the difference is small. 4ty and the result of Equation (30) are different. However, the difference is small. 1.4

1.4

1

The error of tx

1.2

The approximation value of tx error

The error of t y (mm)

The error of t x (mm)

1.2

0.8 0.6 0.4 0.2 0 0

The error of ty The approximation value of ty error

1 0.8 0.6 0.4 0.2

0.5

1

1.5 2 tz(m)

(a)

2.5

3

3.5

0 0

0.5

1

1.5 2 tz(m)

2.5

3

3.5

(b)

Figure erroranalysis; analysis;(b) (b)ty terror analysis. y error Figure 4. 4. Error Errorapproximate approximate analysis: analysis: (a) (a) ttxx error analysis.

Under the influence of image noise and lens distortion, the M matrix of the DLT algorithm contains errors. These errors further affect the accuracy of the calibration parameters. Through the above analysis, there is a linear coupling relationship between ∆t x and ∆u0 , ∆ty and ∆ν0 , ∆a x and ∆tz and ∆ay and ∆tz after ignoring some minor errors.

Algorithms Algorithms2016, 2016,9,9,31 31

8 8ofof1515

1.4

2

The error of ax

1.2

The approximation value of the ax error The error of a y

The error of a x

1

1.5

0.8 0.6 0.4

The error of ay The approximation value of the ay error

1

0.5

0.2 0 1

1.5

2

2.5

3

3.5

0 1

1.5

2

2.5

3

3.5

tz (m)

tz (m)

(b)

(a)

ay aerror FigureFigure 5. Error analysis: (a) ax(a)error comparison; (b) (b) comparison. 5. approximate Error approximate analysis: a x error comparison; y error comparison. 3.2. Improved Direct Linear Transformation Under the influence of image noise and lens distortion, the M matrix of the DLT algorithm contains errors. These errors further affect the accuracy of the calibration parameters. Through the Owing to the linear relationship that exists between the intrinsic and extrinsic parameter errors, above analysis, there is a linear coupling relationship between Δt x and Δu0 , Δt y and Δv0 , Δax all of these errors are unknown, and the specific error value cannot be directly solved. We assume that ay and and3DΔtemplate t z and Δonly Δt z along after ignoring some the moves the z axis. a1 x ,minor a1 y , u1 0errors. , ν1 0 , t1 x , t1 y and t1 z1 are calibration values of the DLT algorithm before the z axis translation. a2 x , a2 y , u2 0 , ν2 0 , t2 x , t2 y and t1 z2 are calibration 3.2. Improved Direct Linear Transformation values of the DLT algorithm after the z axis translation. From Equations (23) and (24), we have:

Owing to the linear relationship that exists between the intrinsic and extrinsic parameter errors, ∆t1 x “ t1 x ´ t x “ ´tz1 pu1 0 ´ u0 q{a x (31) all of these errors are unknown, and the specific error value cannot be directly solved. We assume ∆t2 x “ t2 x ´ t x “ ´tz2 pu2 0 ´ u0 q{a x that the 3D template only moves along the z axis. ax′ , a′y , u0′ , v0′ , t x′ , t y′ and t z′1 are calibration 1 values of the DLT algorithm before z taxis translation. ax′′ , y a′′y , u0′′ , v0′′ , t x′′ , t ′′y and tz′2 are ∆t1 y “the t1 y ´ y “ ´t z1 pν 0 ´ ν0 q{a (32) 2 2 2 ∆t y “ t yafter ´ ty the “ ´tz z2axis pν 0translation. ´ ν0 q{ay From Equations (23) and (24), calibration values of the DLT algorithm we have: where a , a , u , ν , t , t , t and t are true values. Because the translation occurs only along the z x

y

0

0

x

y

z2

z1

axis, we set tz2 “ ntz1 . From EquationsΔ(31) ′ (32), we ′have: t x′ = tand x − t x = −t z1 (u0 − u0 ) a x Δt ′′ = t ′′ − t = −t (u ′′ − u ) a t1 x ´ tx2 x x tz1x nu2z 20 ´0 u1 0 0 x “ p ´ u0 q n´ (v0′1− v0 ) a y Δt1y′ = t ′y −atxy = −tnz1 ´

(31) (33)

(32) Δtty′′2 = t ′′y −tt y =nν −t2z 2 (´ v′′ −1 v ) a y t1 y ´ y 0 0 ν 00 z1 “ p ´ ν0 q (34) 1 t a xare true n ´ 1values. Because the translation occurs only where ax , a y , u0 , v0 , t x , t y , t zn1 ´and z2 1 2 From relationship exists t z 2 = ntaz1 . linear From Equations (31) and (32),between we have: pnu 0 ´ u 0 q{pn ´ 1q and along the z Equation axis, we set(33), 1 2 pt x ´ t x q{pn ´ 1q. By repeatedly moving the 3D template, u0 can be solved with a linear ′′ t z1 nu0′′ − (34). t x′ − tby u0′ least-squares fit. Similarly, ν0 can be solved x Equation (33) = ( − u0 ) n − 1 ax n −1 From Equations (31) and (32), we have:

− v0′2 0 ´ u1 0 q nt1 x ´tt′y2−x t ′′y t z1 nv tz 0′′npu − v0 ) “=t x ´( n ´ 1n − 1 ax a xn − 1 n ´ 1

(34) (35)

From Equation (33), a linear relationship nt1 y ´ t2 y exists between tz npν2 0 ´(nu ν1 00′′q− u0′ ) (n − 1) and (t x′ − t x′′) (n − 1) . By “ ty ´ (36) n ´u10 can be solved ay n ´ 1a linear least-squares fit. Similarly, v0 with repeatedly moving the 3D template, can A be solved Equation (34). exists between npu2 ´ u1 q{pn ´ 1q and pnt1 x ´ t2 x q{n ´ 1, linearby relationship 0 0 1 1 ´(32), From Equations (31) we have: 2 q{pn npν2 ´ ν q{pn ´ 1q and pntand t ´ 1q. Thus, t , t can be solved by Equations (35) and (36). y y x y 0

0

nt x′ − t x′′ t n(u0′′ − u0′ ) = tx − z n −1 a n −1 a1 x ´ a x “ pr11 a x ` r31 ux0 qpt1 z1 ´ tz1 q{tz1 a2 x ´ a x “ pr11 a x ` r31 u0 qpt1 z2 ´ tz2 q{tz2

From Equation (30), we have:

(35) (37)

Algorithms 2016, 9, 31

9 of 15

From Equation (37), we have: pr11 a x ` r31 u0 qppt1 z2 ´ t1 z1 q ´ ptz2 ´ tz1 qq ´ ptz2 a2 x ´ tz1 a1 x ´ a x ptz2 ´ tz1 qq “ 0

(38)

We set dz1 “ tz2 ´ tz1 . We then obtain: pr11 a x ` r31 u0 q

a2 x ´ a1 x t1 z2 ´ t1 z1 ´ dz1 ´ tz1 ´ a2 x ` a x “ 0 dz1 dz1

(39)

Because pa2 x ´ a1 x q{dz1 is small, we use t1 z1 to replace tz1 . Then, we have: pr11 a x ` r31 u0 q

t1 z2 ´ t1 z1 ´ dz1 a2 x ´ a1 x ´ pt1 z1 ` a2 x q ` a x “ 0 dz1 dz1

(40)

A linear relationship exists between pt1 z2 ´ t1 z1 ´ dz1 q{dz1 and t1 z1 pa2 x ´ a1 x q{dz1 ` a2 x . By repeatedly moving the 3D template, a x can be solved with a linear least-squares fit. With respect to tz1 , from Equation (37), we have: npt1 z1 ´ tz1 q ∆a1 x “ ∆a2 x t1 z2 ´ ntz1

(41)

∆a1 x t1 z2 ´ n∆a2 x t1 z1 “ tz1 pn∆a1 x ´ n∆a2 x q

(42)

Then, A linear relationship exists between ∆a1 x t1 z2 ´ n∆a2 x t1 z1 and n∆a1 x ´ n∆a2 x . By repeatedly moving the template, tz1 can be solved with a linear least-squares fit. For ay , we have: pr22 ay ` r32 ν0 q

a2 y ´ a1 y t1 z2 ´ t1 z1 ´ dz1 ´ pt1 z1 ` a2 y q ` a y “ 0 dz1 dz1

∆a1 y t1 z2 ´ n∆a2 y t1 z1 “ tz1 pn∆a1 y ´ n∆a2 y q

(43) (44)

A linear relationship exists between t1 z2 ´ t1 z1 ´ dz1 {dz1 and pa2 y ´ a1 y qt1 z1 {dz1 ` a2 y and 1 1 1 2 2 y t z2 ´ n∆a y t z1 and n∆a y ´ n∆a y . By repeatedly moving the template, ay , tz1 can be solved with a linear least-squares fit. In sum, the 3D template along the z axis to the translation performs a DLT algorithm at each location. With the results of the DLT, u0 , ν0 , t x , ty , a x , ay and tz1 can again be solved with a linear least-squares fit by Equations (33)–(36), (40) and (42)–(44). ∆a1

4. Experimental Section The simulation and physical experiment parameters were set as follows. The camera focal length was 12 mm. The image resolution was 1400 ˆ 1024. The pixel size was 7.4 µm ˆ 7.4 µm. 4.1. Simulation Experiment The size of the 3D template was 0.7 m ˆ 0.7 m ˆ 0.3 m and contained a pattern of 8 ˆ 8 ˆ 3 points. The rotation matrix and translation vector were R = [10, 10, 10] (˝ ) and T = [´0.35, ´0.35, tz ] (m), tz = 1.2–3.2 m. Gaussian noise with a 0 mean and a 0.01–0.5-pixel standard deviation was added to the projected image points. The 3D template moved 0.1 m at a time. The IDLT algorithm was calculated after 18 times of movement. A total of 100 independent tests was performed for each noise to obtain the parameter error mean and standard deviation. The error mean plus three times the standard deviation was used to represent the calibration error. Because the calibration error became larger as tz increased, we only analyzed the results at tz “ 1.2m. The results of the DLT and IDLT calibration are shown in Figure 6. For u0 and v0 , the errors

projected image points. The 3D template moved 0.1 m at a time. The IDLT algorithm was calculated after 18 times of movement. A total of 100 independent tests was performed for each noise to obtain the parameter error mean and standard deviation. The error mean plus three times the standard deviation was used to represent the calibration error. Because the calibration error became larger as tz increased, we only analyzed the results at Algorithms 2016, 9, 31 15 t z = 1.2 m. The results of the DLT and IDLT calibration are shown in Figure 6. For u0 and 10 v0 of , the errors of IDLT are less than 10% of the error of DLT. The errors in t x and t y are less than 0.1 mm. ofFor IDLT of theof error of are DLT. The errors ty are less than 0.1 mm. For a x , ay x and ay less ax ,are andthan t z , 10% the errors IDLT less than 60%in oft the error of DLT. and tz , the errors of IDLT are less than 60% of the error of DLT. 1.4

0.7

1

The error of t x (mm)

The error of u 0 (pixel)

1.2

0.8

DLT IDLT

0.8 0.6 0.4 0.2 0 0

DLT IDLT

0.6 0.5 0.4 0.3 0.2 0.1

0.1

0.2 0.3 Noise (pixel)

0.4

0 0

0.5

0.1

(a)

The error of t y (mm)

The error of v 0 (pixel)

0.6

0.8 0.6 0.4 0.2

0.5

0.4

0.5

0.5 0.4 0.3 0.2 0.1

0.1

0.2 0.3 Noise (pixel)

0.4

0 0

0.5

0.1

0.2 0.3 Noise (pixel)

(d)

2.5

1.4 DLT IDLT

1.2 The error of t z (mm)

The error of a x

0.4

DLT IDLT

(c)

2

0.5

0.7 DLT IDLT

1

0 0

0.4

(b)

1.4 1.2

0.2 0.3 Noise (pixel)

1.5

1

DLT IDLT

1 0.8 0.6 0.4

0.5 0.2 0 0

0.1

0.2 0.3 Noise (pixel)

0.4

0.5

(e)

0 0

0.1

0.2 0.3 Noise (pixel)

(f) Figure 6. Cont.

Algorithms 2016, 9, 31

11 of 15

Algorithms 2016, 9, 31 Algorithms 2016, 9, 31

11 of 15 11 of 15

2.5

2 1.5

1.2 1.4 Theoferror of t z(mm) The error t z(mm)

Theoferror The error a y of a y

2.5 2

1.4 DLT IDLT DLT IDLT

1.5 1 1 0.5

1 1.2

DLT IDLT DLT IDLT

0.81 0.6 0.8 0.4 0.6 0.2 0.4

0.5 0 0

0.1

0.4

0.5

0 0.2 0

0.1

0.5

0.1

0.2(g) 0.3 Noise (pixel)

0.4

0.5

0 0

0.2 0.3 Noise (pixel)

0.4

0 0

0.2 0.3 Noise (pixel)

0.1

0.2(h) 0.3 Noise (pixel)

0.4

0.5

Figure 6. Calibration error improved DLT (IDLT) (g) analysis between direct linear translation (DLT) and (h) with noise: (a) error of u0 ; (b) error of v0 ; (c) error of t x ; (d) error of t y ; (e) error of ax ; (f) error of Figure6.6.Calibration Calibrationerror erroranalysis analysisbetween betweendirect directlinear lineartranslation translation(DLT) (DLT)and andimproved improvedDLT DLT(IDLT) (IDLT) Figure ay error t z based on(a)aerror error of(b)aerror error of terror on; (d) . y ; (h)of x ; (g)of z based t u ; v ; (c) of t of ; (e) error of a ; (f) error with noise: y error of a x ; (f) x error of tof with noise: (a) error of u0 0; (b) error of v0 ;0(c) error of t x ; x(d) error of ty ; (e) z based on a xon ; (g)aerror ay ; (h) tz based ay . on a y . ay ; (h)oferror t z based error of error of ton x ; (g) of z based

Figure 7 shows the calibration result with noise and distortion. Gaussian noise with a 0 mean and a 0.1-pixel standard deviation is added to the projected image points. Image distortion comes Figure7 7shows showsthe the calibration result with noise distortion. Gaussian a 0 mean Figure calibration result with noise andand distortion. Gaussian noisenoise withwith a 0 mean and from ideal image points (ui , vi ) and real image points (ui′, vi′) with k = 2.915 × 10−9 − 2.915 × 10−8 (the a 0.1-pixel standard deviation is added the projected distortion aand 0.1-pixel standard deviation is added to the to projected image image points.points. ImageImage distortion comes comes from image distortion 1–10 pixels (1400, when points u01 =, ν700 ). 1 q(uwith ´8−8(the fromimage ideal image ui , vi real )atand real0)image k = 2.915 − 2.915 × 10 (the ideal pointsispoints pu image points pu = 2.915 ˆ 10´×910 ˆ 10 ´−92.915 i , νi q (and i i i′, vi′) kwith image u0u“ 700). 2 imagedistortion distortionisis1–10 1–10pixels pixelsatat(1400, (1400,0)0)when 0 = 700 ). ′ uwhen i = ui + (ui − u0 )kr (45) 22 2 ′ =uviui`++pu ( v − v ) kr ( u − u ) kr u1 i vu“ ´ u qkr 0 ii′ = i ii 00 i (45) (45) 22 ν1 i v“ νvi `+pν ν0)qkr i´ ′ = ( v − v Owing to the influence of image distortion, the calibration errors of DLT are relatively large. 0 kr i i i However, the calibration errorsimage of IDLT are relatively small. In particular, the errors of u0 and v0 Owing large. Owingto tothe theinfluence influenceof of imagedistortion, distortion, the the calibration calibration errors errors of of DLT DLT are are relatively relatively large. are less than 0.7 pixels, whereas the maximum error ofsmall. DLT isIn44 pixels. The errors ofofaof are However, the calibration errors are relatively particular, the errors u and va0y are x 0and However, the calibration errorsof ofIDLT IDLT are relatively small. In particular, the errors u0 and v0 less than 0.7 pixels, whereas the maximum error of DLT is 44 pixels. The errors of a and a are less a less than 3. Because the pixels are square, the errors of a and have the same form. The error in x y y x are less than 0.7 pixels, whereas the maximum error of DLT is 44 pixels. The errors of ax and a y are than 3.larger Because the pixels are square, the errors of athat ais the same The error in tz is x and y have t t y . form. tless is than that in t , . The main reason is t larger than t x ,form. y are square, the errors of a z and a z x than 3. Because the pixels same The error in y have the x larger than that in t x , ty . The main reason is that tz is larger than t x , ty . t z is larger than that in t x , t y . The main reason is that t z is larger than t x , t y . 30

25 20

15 Theoferror of t x (mm) The error t x (mm)

Theoferror of u 0 (pixel) The error u 0 (pixel)

30 25

15 DLT IDLT DLT IDLT

20 15 15 10 10 5 05 0 0 0

0.5 0.5

1 1.5 Distortion parameter k

2

1 (a) 1.5 Distortion parameter k

2

(a)

10

DLT IDLT DLT IDLT

10

2.5

5 5 0 0

0.5

-8

x 10

2.5 -8

x 10

Figure 7. Cont.

0 0

0.5

1 1.5 Distortion parameter k

2

1 (b) 1.5 Distortion parameter k

2

(b)

2.5 -8

x 10

2.5 -8

x 10

Algorithms Algorithms2016, 2016,9,9,31 31

12 12ofof15 15

50

25 DLT IDLT

20 The error of t y (mm)

The error of v 0 (pixel)

40

30

20

10

DLT IDLT

15

10

5

0 0

0.5

1 1.5 Distortion parameter k

2

0 0

2.5

0.5

-8

x 10

(c) 30 DLT IDLT The error of t z(mm)

The error of a x

25

15

10

5

-8

DLT IDLT

20 15 10

0.5

1 1.5 Distortion parameter k

2

0 0

2.5

0.5

-8

x 10

(e) 30

DLT IDLT

25 The error of t z(mm)

15

10

5

0 0

1 1.5 Distortion parameter k

2

2.5 -8

x 10

(f)

20 The error of a y

2.5 x 10

5

0 0

25

2

(d)

25

20

1 1.5 Distortion parameter k

DLT IDLT

20 15 10 5

0.5

1 1.5 Distortion parameter k

(g)

2

2.5 -8

x 10

0 0

0.5

1 1.5 Distortion parameter k

2

2.5 -8

x 10

(h)

Figure7.7.Calibration Calibrationerror erroranalysis analysisbetween between DLT and IDLT with noise distortion: (a) error Figure DLT and IDLT with noise andand distortion: (a) error of u0of; t u ; (b) error of v ; (c) error of t ; (d) error of ; (e) error of a ; (f) error of t based on ayx;; 0 x x t z based on a x ;z (g) error of a (b)0 error of v0 ; (c) error of t x ; (d) error of ty ; (e) errory of a x ; (f) error of (h) on ay .of t z based on a y . (g)error errorofoftz abased y ; (h) error

4.2. 4.2.Physical PhysicalExperiment Experiment We Weused usedaalight-emitting light-emittingdiode diode(LED) (LED)as asthe thespace spacepoint pointfixed fixedon onaacoordinate coordinatemeasurement measurement machine, as shown in Figure 8. The size of the plane was 0.7 m ˆ 0.7 m, which containeda apattern pattern machine, as shown in Figure 8. The size of the plane was 0.7 m × 0.7 m, which contained of of 8 ˆ 8 points. There were 20 planes; the data for three of these planes were used for one 8 × 8 points. There were 20 planes; the data for three of these planes were used for one DLT DLT calculation. calculation. The The DLT DLTresults resultsare areshown shownin inTable Table1.1.The Thefluctuations fluctuationsin inthe theEuler Eulerangle angleerrors errorsare areless less ˝ , and they are not very volatile. The most volatile values are t and v . The fluctuation in t is than 0.06 y y ty 0 v0. The fluctuation in than 0.06°, and they are not very volatile. The most volatile values are ty and less thanthan 28.0328.03 mm, mm, and the in v0 isin less 2.59than pixels. results of results the IDLT is less andfluctuation the fluctuation v0 than is less 2.59The pixels. The ofalgorithm the IDLT are shown in Table 2, where the calibration parameter values are different from those of DLT. algorithm are shown in Table 2, where the calibration parameter values are different from

those of DLT.

Algorithms 2016, 9, 31

13 of 15

Algorithms 2016, 9, 31

13 of 15

Figure Figure 8. 8. Photograph Photograph of of the the physical physical experiment. experiment. Table 1. DLT results of the physical experiment. Table 1. DLT results of the physical experiment. ax

ax

1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8

1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7

1735.90 1735.90 1736.97 1736.97 1738.26 1738.26 1736.34 1736.34 1737.88 1737.88 1738.41 1738.41 1737.42 1737.42 1737.75 1737.21 1737.75 1737.17 1737.21 1736.76 1737.17 1737.09 1736.76 1737.19 1736.94 1737.09 1737.05 1737.19 1736.63 1736.94 1736.33 1736.79 1737.05 1736.63 1736.33

8

1736.79 ax

1738.11

ψ φ θ tx ty tz ˝ ˝ (mm) ay tx (mm) (mm) ty (mm) (mm) tz (mm) (°)ψ ( ) (°) θ ( )(°) (pixel) (pixel) 1735.56 701.09 522.07 −363.52 −319.10 1705.86 1.11 −0.13 1.36 1735.56 701.09 522.07 ´363.52 ´319.10 1705.86 1.11 ´0.13 1736.67 700.97 522.59 −363.67 −317.70 1806.84 1.101.10−0.13´0.13 1.36 1736.67 700.97 522.59 ´363.67 ´317.70 1806.84 1737.96 701.66 523.22 −364.68 −316.48 1908.12 1.08 −0.13 1.36 ´316.48 1908.12 1.08 ´0.13 1737.96 701.66 523.22 ´364.68 1736.05 701.59 701.59 523.46 523.46 −364.90 ´364.90 −314.87 ´314.87 2005.89 2005.89 1.071.07−0.13 ´0.13 1736.05 1.36 1737.58 700.65 700.65 523.91 523.91 −364.05 ´364.05 −313.54 ´313.54 2107.60 2107.60 1.061.06−0.13 ´0.13 1737.58 1.36 ´311.73 2208.32 1.06 ´0.12 1738.15 700.71 523.93 ´364.39 1738.15 700.71 523.93 −364.39 −311.73 2208.32 1.06 −0.12 1.36 1737.20 700.71 524.02 ´364.65 ´310.01 2307.00 1.05 ´0.12 1737.20 700.71 524.02 −364.65 −310.01 2307.00 1.05 −0.12 1.36 ´308.38 2407.47 1.05 ´0.12 1737.60 700.58 524.17 ´364.72 1737.07 700.58 700.71 524.17 524.16 −364.72 ´365.17 −308.38 ´306.53 2407.47 2506.68 1.051.05−0.12 ´0.13 1737.60 1.36 1737.05 700.54 524.10 ´365.18 ´304.62 2606.64 1737.07 700.71 524.16 −365.17 −306.53 2506.68 1.051.05−0.13´0.12 1.36 1736.63 700.66 524.31 ´365.61 ´303.12 2705.92 1.04 ´0.12 1737.05 700.54 524.10 −365.18 −304.62 2606.64 1.05 −0.12 1.35 ´302.02 2806.35 1.03 ´0.13 1737.00 700.74 524.76 ´365.99 1736.63 1.35 1737.11 700.66 700.97 524.31 524.85 −365.61 ´366.63 −303.12 ´300.39 2705.92 2906.44 1.041.03−0.12 ´0.13 ´298.18 2806.35 3006.04 1.031.03−0.13 ´0.13 1736.86 700.74 700.96 524.76 524.60 −365.99 ´366.89 −302.02 1737.00 1.35 ´296.22 3106.31 1.04 ´0.13 1736.98 700.83 524.52 ´366.94 1737.11 700.97 524.85 −366.63 −300.39 2906.44 1.03 −0.13 1.35 1736.56 700.75 524.32 ´367.06 ´294.05 3205.58 1.05 ´0.13 1736.86 1.36 1736.27 700.96 700.88 524.60 524.43 −366.89 ´367.58 −298.18 ´292.43 3006.04 3304.94 1.031.04−0.13 ´0.13 ´291.07 3106.31 3405.76 1.041.03−0.13 ´0.14 1736.77 700.83 701.40 524.52 524.66 −366.94 ´368.87 −296.22 1736.98 1.36 1736.56 700.75 524.32 −367.06 −294.05 3205.58 1.05 −0.13 1.35 1736.27 Table 700.88 524.43 −292.43 3304.94 1.04 −0.13 1.35 2. IDLT results −367.58 of the physical experiment. ay

1736.77 ay

u0 u0 (pixel)

v0 v0 (pixel)

1.36 1.36 1.36 1.36 1.36 1.36 1.36 1.36 1.36 1.35 1.35 1.35 1.35 1.36 1.36 1.35 1.35 1.35

701.40 524.66 −368.87 −291.07 3405.76 1.03 −0.14 1.35 u0 (pixel) v0 (pixel) tx (mm) ty (mm) Tz (mm)

IDLT results 555.55 of the physical experiment.´352.39 1738.33Table 2.696.27 ´358.82 ax

φ (˝ )

ay

u0 (pixel)

v0 (pixel)

tx (mm)

ty (mm)

1708.23

Tz (mm)

1738.11 1738.33accuracy 696.27of the555.55 −358.82 the −352.39 To compare the calibration two algorithms, results1708.23 of IDLT and the first set of DLT data were used to calculate the re-projection errors of 20 planes. Each plane image error was To compare the calibration the two deviation algorithms, resultspoint of IDLT and the first set described by the mean plus threeaccuracy times theofstandard of the image re-projection errors. of DLT data were used to calculate the re-projection errors of 20 planes. Each plane image error was The results are shown in Figure 9. The calculation errors of DLT increase with the number of planes. described bythat the the mean plus three times the standard deviation of the imagebut point errors. This shows DLT calibration results are optimal for some planes, notre-projection for all planes. The The results are shown in Figure 9. The calculation errors DLT increase withisthe number planes. calibration errors of IDLT decrease with the number of of planes. The reason because theofdistance This shows the point DLT calibration optimal some but the not effect for allofplanes. The between thethat image and opticalresults center are point of thefor image is planes, small, and distortion calibration is small. errors of IDLT decrease with the number of planes. The reason is because the distance between the image point and optical center point of the image is small, and the effect of distortion is small.

Algorithms 2016, 9, 31 Algorithms 2016, 9, 31

14 of 15 14 of 15

3

DLT IDLT 0 The error of v (pixel)

The error of u (pixel)

2.5

5 DLT IDLT

2 1.5

-10

1

-15

0.5 0 0

-5

5

10 plane

15

20

-20 0

(a)

5

10 plane

15

20

(b)

Figure9.9.Re-projection Re-projectionerrors errorsof of20 20planes planeswith withDLT DLTand andIDLT IDLTcalibration calibrationresults: results:(a) (a)uuofofprojection projection Figure image errors; (b) v of projection image errors. image errors; (b) v of projection image errors.

5. Conclusions 5. Conclusions Based on a camera model, the DLT algorithm uses linear equations to calculate the intrinsic and Based on a camera model, the DLT algorithm uses linear equations to calculate the intrinsic extrinsic camera parameters. Because the camera’s intrinsic and extrinsic parameters are and extrinsic camera parameters. Because the camera’s intrinsic and extrinsic parameters are simultaneously calibrated, the coupling error of the calibration parameter affects the calibration simultaneously calibrated, the coupling error of the calibration parameter affects the calibration accuracy. In this paper, we analyzed the principles of intrinsic and extrinsic parameter error coupling, accuracy. In this paper, we analyzed the principles of intrinsic and extrinsic parameter error coupling, determined a linear coupling relationship between the intrinsic parameter calibration error and the determined a linear coupling relationship between the intrinsic parameter calibration error and the translation vector (extrinsic parameters) calibration error and proposed the IDLT algorithm. The translation vector (extrinsic parameters) calibration error and proposed the IDLT algorithm. The IDLT algorithm uses the linear coupling relationship to calculate the calibration parameters of the IDLT algorithm uses the linear coupling relationship to calculate the calibration parameters of the camera. The results of simulations and experiments show that there are significantly fewer calibration camera. The results of simulations and experiments show that there are significantly fewer calibration parameter errors using the IDLT algorithm than there are using DLT with noise and distortion. parameter errors using the IDLT algorithm than there are using DLT with noise and distortion. Acknowledgments: This work was supported by the National Science Foundation of China (Grant Acknowledgments: ThisNatural work was supported by theof National ScienceProvince Foundation of China (Grant No. 51075095) No. 51075095) and the Science Foundation Heilongjiang (Grant No. E201045). and the Natural Science Foundation of Heilongjiang Province (Grant No. E201045). Author Contributions: Zhenqing Zhao proposed the method and wrote the manuscript. Xin Zhang performed Author Contributions: Zhenqing Zhao proposed the method and wrote the manuscript. Xin Zhang performed thedata dataanalysis. analysis. Dong contributed toconception the conception this study. Gangand Chen Binhelped Zhang helped the Dong Ye Ye contributed to the of thisofstudy. Gang Chen Bin and Zhang perform perform thewith analysis with constructive discussions. the analysis constructive discussions. Conflicts ConflictsofofInterest: Interest:The Theauthors authorsdeclare declareno noconflict conflictofofinterest. interest.

References References 1.1. 2.2. 3.3. 4.4.

5.5. 6.6. 7.7.

8.

Chen, S.J.S.J. AnAn efficient approach for dynamic calibration of multiple cameras. IEEE Trans. Chen,I.H.; I.H.;Wang, Wang, efficient approach for dynamic calibration of multiple cameras. IEEEAutom. Trans. Sci. Eng. 2009, 6, 187–194. [CrossRef] Autom. Sci. Eng. 2009, 6, 187–194. Ly, Ly,D.S.; D.S.;Demonceaux, Demonceaux,C.; C.;Vasseur, Vasseur,P.;P.;Pegard, Pegard,C.C.Extrinsic Extrinsiccalibration calibrationofofheterogeneous heterogeneouscameras camerasby byline line images. images.Mach. Mach.Vis. Vis.Appl. Appl. 2014, 2014, 25, 25,1601–1614. 1601–1614. [CrossRef] Peng, Li,Li, L. Camera calibration using using one-dimensional information and its applications in both controlled Peng,E.;E.; L. Camera calibration one-dimensional information and its applications in both and uncontrolled environments. Pattern Recogn. 2010, 43, 1188–1198. [CrossRef] controlled and uncontrolled environments. Pattern Recogn. 2010, 43, 1188–1198. Abdel-Aziz, Karara, H.M. H.M.Direct Directlinear linear transformation from comparator coordinates into object Abdel-Aziz, Y.I.; Y.I.; Karara, transformation from comparator coordinates into object space space coordinates in close-range photo grammetry. Proceedingsofofthe the Symposium Symposium on coordinates in close-range photo grammetry. In In Proceedings on Close-Range Close-Range Photogrammetry, Photogrammetry.University UniversityofofIllinois IllinoisatatUrbana-Champaign, Urbana-Champaign,Champaign, Champaign,IL, IL,USA, USA,1971; 1971;pp. pp.1–18. 1–18. Tsai, R.Y. A versatile camera calibration technique for high-accuracy 3D machine vision metrology Tsai, R.Y. A versatile camera calibration technique for high-accuracy 3D machine vision metrologyusing using off-the-shelf off-the-shelfTV TVcameras camerasand andlenses. lenses.IEEE IEEEJ.J.Robot RobotAutom Autom1987, 1987,3,3,323–344. 323–344.[CrossRef] Zhang, Zhang,Z.Y. Z.Y.AAflexible flexiblenew newtechnique techniquefor forcamera cameracalibration. calibration.IEEE IEEETrans. Trans.Pattern PatternAnal. Anal.Mach. Mach.Intell. Intell.2000, 2000,22, 22, 1330–1334. [CrossRef] 1330–1334. Chen, Chen,G.; G.;Guo, Guo,Y.B.; Y.B.;Wang, Wang,H.P.; H.P.;Ye, Ye,D.; D.;Gu, Gu,Y.F. Y.F.Stereo Stereovision visionsensor sensorcalibration calibrationbased basedon onrandom randomspatial spatial points given by CMM. Optik 2012, 123, 731–734. [CrossRef] points given by CMM. Optik 2012, 123, 731–734.

Samper, D.; Santolaria, J.; Brosed, F.J.; Majarena, A.C.; Aguilar, J.J. Analysis of Tsai calibration method using two- and three-dimensional calibration objects. Mach. Vis. Appl. 2013, 24, 117–131.

Algorithms 2016, 9, 31

8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23.

15 of 15

Samper, D.; Santolaria, J.; Brosed, F.J.; Majarena, A.C.; Aguilar, J.J. Analysis of Tsai calibration method using two- and three-dimensional calibration objects. Mach. Vis. Appl. 2013, 24, 117–131. [CrossRef] Sun, W.; Cooperstock, J.R. An empirical evaluation of factors influencing camera calibration accuracy using three publicly available techniques. Mach. Vis. Appl. 2006, 17, 51–67. [CrossRef] Huang, J.H.; Wang, Z.; Xue, Q.; Gao, J.M. Calibration of a camera projector measurement system and error impact analysis. Meas. Sci. Technol. 2012, 23, 125402. [CrossRef] Zhou, F.Q.; Cui, Y.; Wang, Y.X.; Liu, L.; Gao, H. Accurate and robust estimation of camera parameters using RANSAC. Opt. Laser Eng. 2013, 51, 197–212. [CrossRef] Ricolfe-Viala, C.; Sanchez-Salmeron, A. Camera calibration under optimal conditions. J. Opt. Soc. Am. 2011, 19, 10769–10775. [CrossRef] [PubMed] Hartley, R.; Zisserman, A. Multiple View Geometry in Computer Vision; Cambridge University Press: Cambridge, UK, 2000; pp. 110–115. Batista, J.; Araujo, H.; Almeida, A.T. Iterative multistep explicit camera calibration. IEEE Trans. Robot. Autom. 1999, 15, 897–917. [CrossRef] Ricolfe-Viala, C.; Sanchez-Salmeron, A.; Valera, A. Efficient lens distortion correction for decoupling in calibration of wide angle lens cameras. IEEE Sens. J. 2013, 13, 854–863. [CrossRef] Guillemaut, J.Y.; Aguado, A.S.; Illingworth, J. Using points at infinity for parameter decoupling in camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 2005, 27, 265–270. [CrossRef] [PubMed] Caprile, B.; Torre, V. Using vanishing points for camera calibration. Int. J. Comput. Vis. 1990, 4, 127–140. [CrossRef] He, B.W.; Zhou, X.L.; Li, Y.F. A new camera calibration method from vanishing points in a vision system. Trans. Inst. Meas. Control 2011, 33, 806–822. [CrossRef] Devernay, F.; Faugeras, O. Straight lines have to be straight. Mach. Vis. Appl. 2001, 13, 14–24. [CrossRef] Zhang, G.J.; He, J.J.; Yang, X.M. Calibrating camera radial distortion with cross-ratio invariability. Opt. Laser Technol. 2003, 35, 457–461. [CrossRef] Li, D.D.; Wen, G.J.; Hui, B.W.; Qiu, S.H.; Wang, W.F. Cross-ratio invariant based line scan camera geometric calibration with static linear data. Opt. Laser Eng. 2014, 62, 119–125. [CrossRef] Hartley, R.I. In defense of the eight-point algorithm. IEEE Trans. Pattern Anal. Mach. Intell. 1997, 19, 580–593. [CrossRef] Xu, Q.Y.; Ye, D.; Chen, H.; Che, R.S.; Chen, G.; Huang, Y. A valid camera calibration based on the maximum likelihood using virtual stereo calibration pattern. In Proceedings of the International Conference on Sensing, Computing and Automation, Chongqing, China, 8–11 May 2006; pp. 2346–2351. © 2016 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC-BY) license (http://creativecommons.org/licenses/by/4.0/).