Calibration of Target Positions using the Conformal Model and Geometric Algebra R. J. Valkenburg, N. S. Alwesh Industrial Research Ltd., P.O. Box 2225, Auckland, New Zealand.
[email protected],
[email protected]
Abstract This paper addresses the problem of calibrating the 3D positions of multiple stationary point targets based on the motion analysis of a group of rigidly co-located calibrated cameras. A non-linear solution is developed using the conformal model of Euclidean 3D space. The algorithm is expressed entirely within the computational framework of geometric algebra and geometric calculus. Experiments are performed to evaluate the algorithm and the results show that it performs well and is robust in the presence of noise.
Keywords: calibration, geometric algebra, geometric calculus, conformal model 1
Introduction
Measuring the position of many targets over a large area using a total station, for example, is a time consuming task. When this must be performed frequently it can be expensive. In this paper we describe a calibration algorithm to rapidly measure stationary target positions. A group of rigidly colocated calibrated cameras are moved to several positions and images of the targets are acquired. The image target locations are transformed to lines which are used as input data to the algorithm. The algorithm uses the lines to estimate the 3D target locations. The algorithm is developed using the conformal model of Euclidean 3D space E3 and implemented with geometric algebra. The paper describes a re-expression of an algorithm implemented in the homogeneous model [1] to the conformal model with additional improvements. One goal of the paper, in addition to describing the target calibration algorithm, is to assess the usefulness of the conformal model and geometric algebra in a reasonably complex practical problem.
1.1 Geometric Algebra and Calculus The concepts and notation of geometric algebra (GA), geometric calculus (GC), and the conformal model used in this paper are briefly introduced in this section. For a proper treatment of GA and GC refer to one of the growing list of references [2, 3, 4]. GA is a powerful algebraic structure which subsumes many other algebraic structures and techniques such as vector algebra, exterior algebra,
quaternions, complex numbers, spinor algebra, and reciprocal frames. GC extends analysis to functions on a geometric algebra into a geometric algebra. GC also subsumes and unifies many techniques such as differential forms, differential geometry, and symbolic calculus. A geometric algebra over R with signature (p, q) (p positive and q negative basis elements) is denoted Gp,q . When q = 0 we write Gp and when the signature and dimension do not matter we simply write G. Gr denotes the grade r elements of G. Let A, B, C ∈ G and X = x1 ∧ x2 ∧ . . . ∧ xr ∈ Gr be a blade. The grade r part of A is given by hAir ∈ Gr and the scalar part given the special notation hAi = hAi0 ∈ G0 ≡ R. The part of A containing the grades in B is denoted hAiB . The dual of A is given by A∗ = AI −1 = AcI −1 where I ∈ Gp+q is the pseudoscalar. The reverse of X is ˜ = xr ∧ . . . ∧ x2 ∧ x1 . If S is a linear given by X operator, the outermorphism S is defined by SX = r Sx1 ∧. . .∧Sx
r ∈ G . The adjoint of S, denoted S, is defined by S(A)B = hAS(B)i ,P ∀A, B ∈ G. The scalar product is defined A ∗ B = r,s hhAir hBis i, left contraction inner product [5] defined AcB = P r,s hhAi P r hBis is−r and outer product defined A∧ B = r,s hhAir hBis is+r . The following important equation relates the inner and outer products: Ac(BcC) = (A ∧ B)cC. The commutator product is defined A × B = 12 (AB − BA). A multivector p ˜ A blade Λ reprenorm is given by |A| = A ∗ A. senting a subspace Λ will be uppercase if it is in direct form (e.g. x ∈ Λ ⇔ x∧Λ = 0) and lowercase if it is in dual form (e.g. x ∈ Λ ⇔ xcλ = 0). The multivector derivative operator is denoted ∂X and has the algebraic properties of the multivector X. The operator hA∂X i = A∗∂X is the A-derivative or
directional derivative in direction A. ∂˙X AB˙ means differentiate B = B(X) with respect to X while regarding A as constant. Refer to [2, 6] for more details. In this paper we use the conformal model of E3 implemented using the algebra G4,1 . G14,1 has a standard orthogonal basis {e1 , e2 , e3 , e+ , e− } where e2k = e2+ = 1 and e2− = −1. It is usually more convenient to work with the basis {e0 , e1 , e2 , e3 , e} where e0 = (e− − e+ )/2 and e = e− + e+ are null vectors with e0 ce = −1. e0 represents the origin and e represents the point at infinity. The conformal model can be regarded as containing the homogeneous model commonly used in computer vision. It allows a richer set of objects to be implemented directly: flats (flat point, line, plane), rounds (point pair, circle, sphere), orientations, tangents (point, tangent line, tangent plane), and conformal transformations (rotor, translator, motor) and all their duals. The algebraic distinction between all these objects makes the conformal model easier to work with than the standard model of E3 implemented using the G3 and homogeneous model implemented using G4 . The pseudoscalars of G3 and G4,1 are given by I3 = e1 ∧ e2 ∧ e3 and I = e0 ∧ I3 ∧ e. It is also convenient to define I4 = e0 ∧ I3 and E = e ∧ e0 . A vector is represented v = v1 e1 + v2 e2 + v3 e3 ∈ G13 and a point is represented p = p + e0 + 12 p2 e ∈ G14,1 where p ∈ G13 is a vector. A dual sphere centered at point p with radius ρ is given by s = p− 12 ρ2 e. A point can be regarded as a dual sphere with zero radius. A line is represented p∧v∧e ∈ G34,1 where p is a point and v is a direction vector. Given a line Λ, the direction vector v can be extracted v = EcΛ. A line is normalised by the mapping Λ → Λ/ |Λ| which physically results in it having a unit direction vector. A translator is defined 1 1 T = exp − t ∧ e = 1 − te 2 2 where t ∈ G13 is the translation vector. A rotor is defined 1 R = exp − B 2 where B ∈ G23 is the angle of rotation (B/ |B| characterises the plane and direction of the angle and |B| is the angle magnitude). A motor M = T R ∈ G0,2,4 4,1 is a single orthogonal element providing both rotation and translation. For example, a line Λo is ˜. rotated then translated by Λ = M Λo M
1.2 Problem Statement We have a set of Nc rigidly co-located calibrated cameras, referred to as the camera group. The
cameras are organised to approximate an omnidirectional camera with their optical centres as coincident as physically possible, and the image planes providing maximum coverage with as little overlap as possible. The geometric relationship between each camera is known, so the camera group can be characterised by a single moving coordinate system denoted CSM. For example, CSM may be identified with the camera coordinate system of one of the cameras. The cameras are synchronised so that a single frame capture event will grab Nc images. A world coordinate system, denoted CSW, is defined in terms of K stationary point targets (e.g. one target is at the origin, another on the x-axis and another in the xy-plane). The camera group CSM is moved to N arbitrary 0,2,4 positions in CSW. Let the motor Mn ∈ G4,1 represent the pose (orientation and position) of CSM in CSW for the nth position. At each position a set of Nc images of the targets are captured (one from each camera). Not all the targets will necessary be visible as the sensors do not give complete coverage and targets may also be occluded. Some targets may also be visible in more than one image because of small amounts of overlap. A target pixel location is extracted using a subpixel estimator then backprojected through the camera model to get a ray in the camera coordinate system. The ray is rotated and translated to provide a normalised line Λoj ∈ G34,1 in CSM ready for subsequent processing. For each line Λoj the associated target label k = k(j) and camera group position index n = n(j) are known and stored with the line. The problem can now be stated as: Given a set of measured labelled lines Λoj , j ∈ J in CSM captured from N positions we wish to recover the position of the K targets in CSW.
2
Solution using Geometric Algebra
The solution involves three steps: (1) Calculate initial estimates of Mn , n = 1, . . . , N , using a closed form algorithm, (2) Calculate accurate estimates of Mn , by non-linear minimisation, (3) Reconstruct the target positions in CSW by triangulation using Λoj , Mn . Initially all we have is the lines expressed in CSM, we do not know where the targets are and so can not define CSW, and also do not know the positions of CSM expressed in some common coordinate system. Therefore we temporarily identify CSW with one position of CSM to avoid the obvious ambiguity when both CSW and CSM are unknown.
2.1 Initial estimate of Mn , n = 1, . . . , N
where S(x, Λ) = x − (xcΛ)cΛ.
The initial estimate is based on approximating the camera group by a more idealised omnidirectional camera in which all the optical centres are coincident and positioned at the origin of CSM. This immediately allows many standard results to be applied directly. These results can be expressed elegantly in terms of GA. A line in CSM going through the origin e0 can be represented Λ = e0 ∧ v ∧ e = Ev where v ∈ G13 is the direction vector.
Lemma 1. Let Λj ∈ G34,1 , jP∈ J be a set of normalised lines and S(x) = j∈J S(x, Λj ). If SI3 6= 0 then the point q ∈ G14,1 closest to all the lines in the least squares sense is given by the center of the normalised dual sphere
The direction vectors u, v from two positions of CSM associated with a common target satisfy ucEv = 0 where E = AR is the well known essential transformation. Antisymmetric A, defined by Av = (v ∧ t)cI3 = vc(tcI3 ), encodes information about the relative position ˜ encodes t. Orthogonal R, defined Rv = Rv R the relative orientation R. As the symmetric transformation EE = AA = AA it follows that tcEEt = 0. Therefore, given E, an estimate of t (and hence A) can be recovered up to a scale factor as the unit eigenvector associated with the smallest eigenvector of EE. Let u1 , u2 , u3 be a basis of G13 and note RE = A. An estimate of R is given by the unit eigenrotor of L associated with the largest eigenvalue, where L : G0,2 → G0,2 is 3 3 defined 3 X LR = (Auk )R(Euk ) k=1
By considering the lines at each position of CSM, estimates of the relative pose (with unit translation) between the positions of CSM can be obtained. These can be transformed into CSW to give initial estimates of Rn and the unit tn . Any known distances (yardsticks) can be used to rescale tn to consistent values. The motors are then given by Mn = Tn Rn where Tn = 1 − 21 tn e. For more details refer to [7].
2.2 Accurate estimate of Mn , n = 1, . . . , N In this section we define a suitable objective (error) function and its gradient so that the problem can be formulated as a standard non-linear optimisation. The lines associated with a given target k will nearly intersect. Image noise and quantisation, calibration errors, camera modelling errors etc will prevent them from intersecting exactly. The objective function is based on finding the motors Mn to minimise the dispersion of Λj about their nominal intersection point. First, we present a number of useful general results about dual spheres and lines. The squared distance between the center of a dual sphere s ∈ G14,1 and a normalised line Λ ∈ G34,1 is given by d2 (s, Λ) = scS(s, Λ)
s=−
S(I3 )cI4 S(I3 )cI3
(1)
The sum of squared distances d2 from each line Λj to q is given by d2 = −
S(I4 )cI4 S(I3 )cI3
(2)
A proof is presented in [7]. The condition SI3 = 0 will only occur under rare circumstances such as all the lines are parallel. This result can be used directly to define a suitable objective function. Each line Λj in CSW is obtained by rotating and translating the corresponding measured line in CSM and is given by ˜ n , n = n(j) Λj = M n Λo M (3) j
where Mn is the pose of CSM in CSW associated with the j th line. Let d2k (M1 , . . . , MN ) be the sum of squared distances from the lines associated with the k th target (Λj , j ∈ Jk ). In general, d2k will involve lines from each position of CSM, n = 1 . . . N , so d2k depends on Mn , n = 1 . . . N . The objective function is then defined by summing the errors for each target d2 = d2 (M1 , . . . , MN ) =
K X
d2k
k=1
Next we consider the gradient of d2 which can be expressed in terms of the multivector derivatives ∂Mn d2 . As ∂ Mn d 2 =
K X
∂Mn d2k
(4)
k=1
it suffices to initially consider the contribution of P just the k th target. Let Sk (p) = j∈Jk S(p, Λj ) denote the sum associated with the k th target. If we let Nk = S k (I4 )cI4 ∈ R and Dk = S k (I3 )cI3 ∈ R then d2k = −Nk /Dk . By the product rule ∂Mn d2k = −
∂ M n Nk D k − N k ∂ M n D k Dk2
(5)
For clarity, we will temporarily drop the subscripts n and k by identifying M with Mn , d2 with d2k , J = ∪Jk with Jk , and S(p) with Sk (p). To calculate ∂M d2 it suffices to evaluate ∂M N and ∂M D which have similar structures. We will make use of the following result [7],
Lemma 2. Let L : G1 → G1 be a linear operator V which depends on a multivector X. Let Ar = i∈I ai and Cr be r-blades independent of X, and define Ei so Ar = ai ∧ Ei . Then X ˙ i )c(L(Ei )cCr ) ∂X L(Ar )cCr = ∂˙X L(a i∈I
Applying Lemma 2 with L = S and X = M gives X ˙ i )c(S(Ei )cCr ) ∂M S(Ar )cCr = ∂˙M S(a i∈I
=
XX
˙ i , Λj )c(S(Ei )cCr ) ∂˙M S(a
j∈J i∈I
Observe that S(Ei )cCr ∈ G14,1 . If u, v ∈ G14,1 and Λ ∈ G34,1 then S(u, Λ)cv = ucv − (ucΛ)c(vcΛ) and it is a simple matter to show ˙ ˙ ∂˙M S(u, Λ)cv = ∂˙M ΛcB(u, v, Λ) where B(u, v, Λ) = − u ∧ (vcΛ) − v ∧ (ucΛ) ∈ G34,1 (6) If B ∈ G34,1 , and Λ ∈ G34,1 depends on M (i.e. D E ˜ then ΛcB = M Λo M ˜ B which can Λ = M Λo M)
be readily differentiated with respect to M [7], D E D E ˙ ˜B ˜ ˜ ∂˙M ΛcB = Λo M − MBM Λo M M M D E ˜ =2 M (Λ × B) (7)
updating the derivatives ∂Mn Nk where n = n(j). Similarly we get ∂ Mn D k = 2
3 D X X
˜ n (Λj × Bikj ) M
j∈Jkn i=1
j∈JM i∈I
where Bikj = B(ai , S k (Ei )cCr , Λj ) ∈ G34,1 . Let Jkn denote the set of lines associated with the k th target and nth position. For each target, only a few lines will be associated with a specific motor Mn . Normally there will be just one. If the target was occluded or located where there was no image coverage there will be none. If it is located in a region of overlap there might be 2 or more. Hence Jkn will only have a few elements. Reintroducing the subscripts n and k, and applying Lemma 3 to Nk = S k (I4 )cI4 gives ∂ M n Nk = 2
3 D E X X ˜ n (Λj × Bikj ) M
j∈Jkn i=0
Mn
(8)
where Bikj = B(ei , S k (Ei )cI4 , Λj ) ∈ G34,1 . Calculating the derivatives ∂Mn Nk is implemented efficiently by iterating over all lines j ∈ ∪Jk and
(9)
2.2.1 Constraints Because the optical centers of the cameras are very close the distances between them do not provide effective yardsticks to scale the problem. This can be remedied by adding additional larger yardsticks such as the distance between two targets. The normalised dual sphere s with center q in Lemma 1, representing the closest point to all the lines, has radius |q|. It follows that for the k th and lth targets, sk − sl = q k − q l . Hence if the distance between targets qk and ql is dkl we can impose the constraint kl = |tkl | − dkl = 0 where tkl = sk −sl . The expression ∂Mn 2kl is given by ∂Mn 2kl = 2kl ∂Mn |tkl | =
We have the following result,
M
Mn
where Bikj = B(ei , S k (Ei )cI3 , Λj ) ∈ G34,1 . Using (4), (5), (8), and (9) we can calculate ∂Mn d2 , n = 1 . . . N.
M
Lemma 3. If JM denotes the set of lines which depend on M then E X XD ˜ (Λj × Bikj ) ˙ r )cCr = 2 M ∂˙M S(A
E
=
2kl ˙ ˙ ∂M tkl ctkl |tkl | n
2kl ˙ (∂Mn s˙ k ctkl − ∂˙Mn s˙ l ctkl ) |tkl |
Note that, sk ctkl =
Sk (I3 )c(tkl cI4 ) Nk = Sk (I3 )cI3 Dk
Using the product rule we can express ∂˙Mn s˙ k ctkl in terms of ∂Mn Nk and ∂Mn Dk and use of Lemma 3 to calculate these derivatives. Similarly for ∂˙Mn s˙ l ctkl . 2.2.2 Parameterisation These multivector derivatives are not suitable for using directly with standard optimisation tools such as the NAG library which make use of information such as gradients and Jacobian matrices. The following parameterisation framework allows us to interface with such tools. Let X(α) ∈ G where α ∈ Rp be a parameterisation of a multivector, s(X) ∈ R be a scalar valued function of a multivector, and define g : Rp → R by g(α) = s(X(α)). The gradient ∇α g is given by [∇α g]i = ∂αi g = (∂αi X(α)) ∗ (∂X s(X))
A parameterisation of a motor M : R6 → G0,2,4 4,1 can be expressed M (γ) = M (α, β) = T (β)R(α) where α, β ∈ R3 , γ = (αT β T )T ∈ R6 , R(α) is a rotor parameterisation, and T (β) is a translator parameterisation. One useful rotor parameterisation R : R3 → G0,2 3 is given by |α| B(α) |α| − sin R(α) = cos 2 2 |α| P3 where B(α) = k=1 αk ek I3 ∈ G23 (note |B(α)| = |α|). The derivative ∂αi R(α) is given by αi |α| |α| αi B(α) ∂αi R(α) = − sin − cos 2 2 2 |α| 2 2 |α| ! ei I3 |α| B(α)αi − sin − 3 2 |α| |α|
Figure 1: Model of room showing targets as small spheres and positions of camera group as coordinate systems (axis length is 1 m).
Similarly, the obvious translator parameterisation T : R3 → G0,2 is given by T (β) = 1 − 21 t(β)e where P3 4,1 t(β) = k=1 βk ek ∈ G13 , and ∂βi T (β) = − 21 ei e. This gives ∂αi M (α) = T (β)∂αi R(α) and ∂βi M (β) = ∂βi T (β)R(α) = − 21 ei eR(α). The gradient ∇γ d2 is given by ∇γ d2 i = (∂γi M (γ)) ∗ (∂M d2 (M ))
In this way the multivector derivatives ∂Mn d2 , n = 1, . . . , N can be readily used to form a 6N gradient vector. An important point is that the derivative of the parameterisation is completely decoupled from the multivector derivative and the exact choice of parameterisation can be delayed and easily changed.
Given the objective function, constraints, their gradients, and the initial estimate, the problem can be formulated as a standard optimisation with barrier functions, or a constrained optimisation.
2.3 Reconstruct the target positions Given the lines Λoj in CSM, and poses Mn , n = 1, . . . , N of CSM in CSW it is a simple matter to reconstruct target positions in CSW. First the lines are mapped into CSW using (3). The lines associated with the k th target, Λj , j ∈ Jk , are then triangulated using (1) in Lemma 1. 1 qk = sk + (sk csk )e, 2
sk = −
S k (I3 )cI4 S k (I3 )cI3
As previously mentioned CSW was temporary identified with one of the positions of CSM. The true CSW can now be constructed out of the targets and the target positions mapped into this CSW.
1 2 3
rms 0.0074 0.0063 0.0019
avg 0.0062 0.0057 0.0019
max 0.0263 0.0137 0.0023
Table 1: (1) Error between the estimated target positions and the measured lines, (2) Error between the estimated and surveyed target positions, and (3) Error in the target constraints at the estimated target positions. Errors in meters.
3
Results
The algorithm was tested on some measured line data. The camera group comprised Nc = 6 calibrated cameras mounted on a mechanical rig so the image planes were parallel with the faces of a cube. The camera group parameters were obtained from the calibration files for each camera and their relative poses. Each camera had an image size of 640 × 480 pixels with a field of view of approximately 90◦ for the image width. The K = 24 targets were located on the walls and ceiling of a room and their positions surveyed to about 5mm accuracy using a total station. Figure 1 shows a model of the room with the targets shown as small spheres. Two constraints were introduced by selecting two pairs of targets. The camera group was moved to N = 22 positions in the room (shown as coordinate systems in Figure 1) and line data Λoj , j ∈ ∪K k=1 Jk gathered as described in Section 1.2. The algorithm was then applied to the lines and the results are presented in Table 1. Rows 1 and 2 summarise the errors in the final estimates of the reconstructed target positions. These results are typical for multiple runs with the same camera group and target configuration. The results are good, with positional errors of a few millimetres
over a reasonably large volume. Since the reported estimation error includes the uncertainty of the surveyed target positions, the actual error may be even smaller. Row 3 shows that the constraints are satisfied well at the solution. The accuracy of the target calibration is dependent on good camera intrinsic calibration and calibration of the pose of each camera in CSM. The intrinsic camera parameters are easy to calibrate to high accuracy. Unfortunately the pose of the cameras in the group is not as easy to accurately calibrate. All the cameras point in different directions and share little, if any, common view. While using the mechanical data from the rig certainly provides a good initial estimate it does not take into consideration any deviations of the camera coordinate system from their nominal positions. Issues such as mechanical sensor placement in the camera body and sensor tilt with respect to the optical axis need to be taken into consideration when defining the position of the camera coordinate system. The camera group pose calibration is performed by a procedure similar to the target calibration algorithm described in this paper but relies on having a known surveyed target test field. Any errors in the test field will result in errors in the group calibration. Initially we found evaluating geometric derivatives difficult primarily because multivectors, in particular ∂X , do not commute in general. However, once we realised the strong interplay between GA and GC things became much easier. Despite appearances, the only real differentiation in this paper are a few trivial applications of the product rule, and Eqn. (7) which is a fairly well known result.
4
Conclusions
The conformal model, expressed in terms of the Geometric Algebra G4,1 , proved very useful for the theoretical and practical development of the calibration algorithm. In general, expressions were simpler than those for the homogeneous model. The ability to differentiate with respect to a motor (rather than a rotor and a translation vector) to calculate ∂Mn d2 significantly simplified the implementation of the gradient. The choice of parameterisation can be delayed and readily changed without affecting ∂Mn d2 . The ability to represent many objects and operators (e.g. vectors, points, lines, dual spheres, motors) as elements of a single algebra simplified the software implementation of the algorithm. The extension of linear operators to multivectors via the outermorphism operator allows a compact representation of results (e.g. (1) and (2)). The ex-
pression of inversion in terms of duality simplifies expressions involving inverses, and in particular allows them to be more easily differentiated. For example, Eqn (1) for the closest point to a number of lines is expressed in terms of duality. The formulation is more elegant than equivalent formulation in terms of matrix theory, where it is most natural to involve a matrix inverse. The error function used in the algorithm is simple and elegant but does not accurately reflect the sources of noise in the measured data such as the supixel image locations of the targets. It would be useful to develop new error functions which better model noise and compare the results. In addition, it would be good to examine techniques to perform the camera group pose calibration simultaneously with the target location calibration to reduce the effects of target measurement error during camera group pose calibration.
5
Acknowledgements
This work was supported by the N.Z. Foundation for Research, Science and Technology.
References [1] R. Valkenburg, X. Lin, and R. Klette, “Selfcalibration of target positions using geometric algebra,” in Proceedings of Image and Vision Computing New Zealand, pp. 221–226, Nov 2004. [2] D. Hestenes and G. Sobczyk, Clifford algebra to geometric calculus. D. Reidel Publishing Company, 1984. [3] D. Hestenes, New Foundations for Classical Mechanics. Kluwer Academic, second ed., 1999. [4] C. Doran and A. Lasenby, Geometric Algebra for Physicists. Cambridge University Press, 2003. [5] L. Dorst, C. Doran, and J. Lasenby, eds., Applications of Geometric Algebra in Computer Science and Engineering, ch. 2, pp. 37–48. Birkhauser, 2002. [6] E. Hitzer, “Multivector differential calculus,” Advances in Applied Clifford Algebras, vol. 12, pp. 125–182, December 2002. [7] R. J. Valkenburg, “Some techniques in geometric algebra for computer vision,” Tech. Rep. 8713000-1-03, Industrial Research Limited, Auckland, New Zealand, August 2003.
