IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 17, NO. 12, DECEMBER 2008
2393
Camera Calibration With Three Noncollinear Points Under Special Motions Zijian Zhao, Yuncai Liu, Senior Member, IEEE, and Zhengyou Zhang, Fellow, IEEE
Abstract—Plane-based (2-D) camera calibration is becoming a hot research topic in recent years because of its flexibility. However, at least four image points are needed in every view to denote the coplanar feature in the 2-D camera calibration. Can we do the camera calibration by using the calibration object that only has three points? Some 1-D camera calibration techniques use the setup of three collinear points with known distances, but it is a kind of special conditions of calibration object setup. How about the general setup—three noncollinear points? We propose a new camera calibration algorithm based on the calibration objects with three noncollinear points. Experiments with simulated data and real images are carried out to verify the theoretical correctness and numerical robustness of our results. Because the objects with three noncollinear points have special properties in camera calibration, they are midway between 1-D and 2-D calibration objects. Our method is actually a new kind of camera calibration algorithm. Index Terms—Camera calibration, Euclidean structure, homography, intrinsic parameters, three noncollinear points.
I. INTRODUCTION AMERA calibration is a very important and necessary technique in computer vision, especially in 3-D reconstruction (extracting 3-D metric information form 2-D images). Recently, plane-based camera calibration [2]–[4], [6]–[9], [11], [15] is becoming a hot research topic for its flexibility. In planebased calibration, the imaged circular points (the circular points are two complex conjugate points at infinity) are computed, which gives the Euclidean structure and are used to induce the constraints of the absolute conic. Zhang’s calibration method [15] uses a planar pattern board (point features) as the calibration object and extracts the imaged circular points (Euclidean structure) of the plane through the world-to-image homographies. To compute the world-to-image homographies, we need at least four image points in every view. To recover the imaged circular points directly, circle features are used in many recent works. Meng [3] applies a calibration pattern that is made up of a circle and a set of lines through its centers, and computes the vanishing line and its intersection with the projected circle. Wu
C
Manuscript received February 26, 2008; revised July 24, 2008. Current version published November 12, 2008. This work was supported in part by the National Nature Science Foundation of China (60833009) and in part by the National Key Basic Research and Development Program (2006CB303103). The associate editor coordinating the review of this manuscript and approving it for publication was Prof. Dan Schonfeld. Z. Zhao is with the Institute of Image Processing and Pattern Recognition, Shanghai JiaoTong University, Shanghai, 200240 China (e-mail: zj_zhao@sjtu. edu.cn). Y. Liu is with the Institute of Image Processing and Pattern Recognition, Shanghai JiaoTong University, Shanghai 200240, China (e-mail:
[email protected]). Z. Zhang is with Microsoft Research, Redmond, WA 98052-6399 USA (e-mail:
[email protected]). Digital Object Identifier 10.1109/TIP.2008.2005562
[4] describes the associated lines of two coplanar circles, and gives the quasi-affine invariance for camera calibration. Gurdjos [6] proposes a general method for more than two parallel circles and extends Wu’s work. Kim and Gurdjos [7], [8] pay more attention to confocal conics (including concentric circles), which can be used to compute the Euclidean structure of the supporting plane. All above methods using conic features need at least five image points in every view to fit a projected conic, though more than six image points are usually needed for a conic. As mentioned above, the calibration objects, which provide at least four image points in one view, are needed in the planebased camera calibration. So, can only three space points be used in camera calibration? Zhang [16] has proposed a camera calibration method by using 1-D objects (three collinear points). Wu and Hu have also given the different descriptions on the camera calibration based on three collinear points in [5], [20], and [21]. The setup of three collinear points is a special condition. How about the general condition of three noncollinear space points? To our knowledge, there does not exist any calibration technique about it reported in the recent literatures. Therefore, our paper’s topic is about how to do the camera calibration by using three noncollinear points. Three noncollinear points do not provide the properties of cross ratio [5] as three collinear points do, so we will establish constraints in a different way. By rotating all the three points around certain axes with 180 degrees, we can construct a number of imaginary quadrilaterals. Using these quadrilaterals, we can get enough constraints for camera calibration. In this paper, we will introduce two kinds of conditions on how to rotate the three noncollinear points and construct imaginary quadrilaterals. We show that the Euclidean structure can be extracted from the images of these constructed quadrilaterals. In other words, although the calibration object only has three noncollinear points, we can find the constraints on the imaged circular points and recover the Euclidean structure, when the calibration object performs a synthetic rigid motion that consists of a series of arbitrary rigid motions and rotations with certain rotation axes. When our camera calibration method is used in a hand-eye robot system, we find that the special motion idea is more suitable for the calibrations (camera calibration and hand-eye calibration [14]) of the hand-eye robot system than other methods, because it provides calibration data for both camera calibration and hand-eye calibration. We also provide an algorithm of camera calibration using three noncollinear points under general motions in this paper, although it is just for multicamera calibration with the mass computation of projection depths. This paper is organized as follows. Section II gives some preliminaries such as the camera model, homography and geometric description of arbitrary three noncollinear points. We will
1057-7149/$25.00 © 2008 IEEE Authorized licensed use limited to: Shanghai Jiao Tong University. Downloaded on December 21, 2008 at 06:43 from IEEE Xplore. Restrictions apply.
2394
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 17, NO. 12, DECEMBER 2008
introduce how to solve the camera calibration problem with the special motions of calibration object in Section III. The multicamera calibration method is introduced in Appendix A. The experimental results with both simulated data and real images are provided in Section IV. In this section, we also report experiments on applying our camera calibration method to the calibrations of a hand-eye robot system. Section V is some concluding remarks. II. PRELIMINARIES A. Camera Model and Homography be the Euclidean coordinates of a world Let be its 3-D homogeneous coordipoint nates, and be the 2-D homogeneous coordinates of its projection in the image plane. and are related by the following equation in the pinhole model: with
(1) Fig. 1. Illustration of three noncollinear points.
where is a scale factor (projection depth of ); is the , the princamera intrinsic matrix, with the focal length cipal point and the skew factor ; is the camera extrinsic matrix, that is the rotation and translation from the world is referred as the frame to the camera frame; camera projection matrix. If we assume that the world space is restricted to its x-y plane, the world-to-image homography then will be expressed as (2) where
are the first two columns of
.
B. Geometric Description of Three Noncollinear Points As shown in Fig. 1, there are arbitrary three noncollinear points ( and ) in the 3-D space. Their relative geometric , where is the structure can be described as a vector norm of is the norm of , and is the angle which and form. According to Fig. 1 and (1), the three points ( and ) will in a single give three constraints based on the vector view. If all three points are described in the camera coordinate system according to (1), we will have eight unknowns including three projection depths and five intrinsic parameters in one observation view. Given observations of the three noncollinear equations and unknowns. It points, we then have seems that camera calibration is impossible. However, if calibration object performs rotations with nonparallel rotation axes, we can solve the camera calibration problem by considering every two adjacent observation together. It will be introduced in the next section. III. CAMERA CALIBRATION WITH THREE NONCOLLINEAR POINTS
erals by rotating the three points, the rotation axes must be perpendicular to the supporting plane or in the supporting plane. According to the difference of the rotation axes, we will introduce two kinds of conditions on solving camera calibration problem in Sections III-A and III-B. A. Rotation Axis Perpendicular to the Supporting Plane Refer to Fig. 2(a). Line (an arbitrary normal of the supporting plane ) is a rotation axis perpendicular to the supporting plane that is determined by the three noncollinear , and point is the intersection of line and plane points . Assume that the positions of before and after rotaand tion around line with 180 degrees are , and their image points are and . Then the image point of can be determined by the following equation: (3) where , they are three image lines. Theoretically, these lines intersect at the same point . However, no three lines exactly intersect at the same point in practice due to noise in image data, the least squares method (lsq) is used to compute the point as shown in (3). As shown in Fig. 2(b), the three imaginary quadri, and laterals are all parallelograms, and the space lines and are the diagonals of them. In is also the midpoint of the three lines. geometry, the point Thus, we have
As shown in Fig. 1, three noncollinear points can determine a supporting plane. If we want to construct a number of quadrilatAuthorized licensed use limited to: Shanghai Jiao Tong University. Downloaded on December 21, 2008 at 06:43 from IEEE Xplore. Restrictions apply.
(4)
ZHAO et al.: CAMERA CALIBRATION WITH THREE NONCOLLINEAR POINTS UNDER SPECIAL MOTIONS
2395
Fig. 2. Three noncollinear points rotate around an axis perpendicular to the supporting plane with 180 degrees: (a) the rotation axis l and the plane intersect at the point O ; (b) construct three imaginary quadrilaterals ( ; and ).
A B A B A C A C
According to (1) and (4), we get
B C B C
According to (1), we also have
(5) (6) (7) are the unknown where projection depthes of all space points. By performing cross and product on both sides of (5)–(7) with respectively, we have
Substituting
by (8), (9), and (10) gives (11) (12) (13)
with
In turn, we obtain
(8)
(9)
(10) Refer to Fig. 2(b). We choose the following constraints to solve the camera calibration problem:
Equations (11)–(13), which are the basic constraints for camera calibration, contain the unknown intrinsic parameters and the unknown depth . Although is unknown, we can eliminate it by performing the method which is introduced in , which is a 3 3 symmetrical matrix, Section III-C. describes the image of the absolute conic. Therefore, we have totally six unknowns in (11)–(13). B. Rotation Axis in the Supporting Plane Only three lines in the supporting plane can be used as the rotation axes in this condition. As shown in Fig. 3(a), they are line
Authorized licensed use limited to: Shanghai Jiao Tong University. Downloaded on December 21, 2008 at 06:43 from IEEE Xplore. Restrictions apply.
2396
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 17, NO. 12, DECEMBER 2008
Fig. 3. Three noncollinear points rotate around an axis in the supporting plane with 180 degrees: (a) line be used as the rotation axis; (b) construct an imaginary quadrilateral .
A C A B
and . For example, if we choose as the rotation axis, the positions of point line will not change during the rotation around line with 180 degrees and still be . Only the position of is to after rotation. Point is the interchanged from and . The image point of section of line can be computed as . In Fig. 3(b), the imaginary quadrilateral is , then the technique we describe not a parallelogram (if . will not work), but the point is the midpoint of line Then we can have
A B B C ;
;
and
C A
, each of them can
(20) (21) with
(14) From line , the position of can also be determined according to the geometric properties of the quadrilateral . We can get (15) with . According to (1), we can have the following equations from (14) and (15)
By performing the same method in Section III-A to the equations above, we obtain
C. Closed-Form Solution for the Camera Intrinsic Matrix In this section, we will introduce how to solve the constraint equations for the camera intrinsic matrix with above two conditions. Because (11), (12), and (13) and (19), (20), and (21) are similar in the formation, we will only discuss how to solve (11), (12), and (13). The same method can be applied to solve (19), (20), and (21). Let
(16)
(17)
(18) Choosing the same constraints as in Section III-A and substiby (16), (17), and (18) gives the simtuting ilar constraint equations for camera calibration to (11), (12), and (13) (19)
is a symmetric, and can be defined by a 6-D vector
Let (11), (12), and (13) becomes
, then
(22) with
, and .
Authorized licensed use limited to: Shanghai Jiao Tong University. Downloaded on December 21, 2008 at 06:43 from IEEE Xplore. Restrictions apply.
ZHAO et al.: CAMERA CALIBRATION WITH THREE NONCOLLINEAR POINTS UNDER SPECIAL MOTIONS
2397
image. According to (1), we have the following equation for and : three noncollinear points
(25) with is the th column of . As shown in Fig. 1, we can choose the relative to denote the three points’ positions, then coordinate (25) becomes
where
Fig. 4. Calibration object performs a synthetic rigid motion. (a) See Section III-A; (b) see Section III-B (the camera is stationary).
According to different choice of the rotation axes, we can subby (8), (9), and (10), or (16), (17), and stitute (18). Then we have (26)
Performing cross product on both sides of (22) with have
, we
depend on the choice of the where the values of rotation axes. From (26), we obtain
(27) According to (27), the extrinsic parameters are computed
In turn, we obtain
with
(23) where
When the calibration object performs a synthetic rigid motion that consists of a series of arbitrary rigid motions and rotations around the certain rotation axes (as shown in Fig. 4), images of the three noncollinear points are observed. By stacking such equations as (23) we have
(24) is a matrix. If (if , then ), we will have a unique solution up to a scale factor. Once is estimated, we can get the matrix . The intrinsic can then be obtained by Cholesky factorizacamera matrix tion and matrix inversion. where
is the
th column of matrix
. E. Nonlinear Optimization Due to the existence of random noise, the above solution for camera intrinsic and extrinsic parameters is not robust. Therefore, we can refine it through the maximum likelihood inference. Generally, we assume that the image points are corrupted by independent and identically distributed Gaussian noise. By images of the calibration object on which there are given , the maximum likelihood three noncollinear points estimate can be obtained by minimizing the following function:
D. Closed-Form Solution for the Extrinsic Camera Parameters Once obtaining the camera intrinsic matrix compute the camera extrinsic parameters
, we can easily for every
Authorized licensed use limited to: Shanghai Jiao Tong University. Downloaded on December 21, 2008 at 06:43 from IEEE Xplore. Restrictions apply.
(28)
2398
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 17, NO. 12, DECEMBER 2008
where and are the projections of point and onto the image according to (1) and (26). Minimizing (28) is an optimization problem, so we can solve it by using the algorithm as described in [12]. The initial guess required can be obtained from of the closed-form solution above. F. Degenerate Configurations In this section, we will study the configurations in which additional images do not provide more constraints for the camera intrinsic parameters. As shown in Fig. 4, after the calibration object performs a synthetic rigid motion that consists of a series of arbitrary rigid motions and rotations with certain rotation axes, a number of images are captured before and after the rotations. If the arbitrary motion between two adjacent rotations is a pure translation, the images captured before and after the second rotation will not provide additional constraints. In the following, we will give a more detailed description of degenerate conditions. Proposition 1: If the two supporting planes of imaginary quadrilaterals constructed by two rotations are parallel to each other, then the images of the imaginary quadrilaterals constructed by the second rotation do not provide additional constraints for camera calibration. It is clear that parallel planes have the same circular points on the absolute conic, so Proposition 1 is self-evident. In fact, the motions of calibration object in Proposition 1 also satisfy the condition for unique solution of hand-eye geometry [14]. If the calibration objects perform special motions as shown in Fig. 4, the camera calibration algorithm described above can be used for both single camera calibration and multicamera calibration. How to calibrate cameras if the calibration objects perform general motions? In Appendix A, we give the answer to the question. IV. EXPERIMENTS The techniques of camera calibration described in previous sections have been implemented and experimented on simulated data and real images respectively. Especially, we describe our technique’s application in a hand-eye robot system. A. Simulation Results In the simulation experiment, the camera has the following setup: . The image resolution is 1024 768. The three noncollinear points of the calibration object form a triangle and have the relative geo. The rotation axis is metric structure vector defined as the normal of the supporting plane defined by the three noncollinear points, and the intersection point of it and the supporting plane is the midpoint of the edge facing the angle . Noise influence. In the experiment, the calibration object and ) is simulated by the (three noncollinear points computer and performs a series of rotations and arbitrary motions. A number of synthetic images (22 images) are captured. Gaussian noise of zero mean and standard deviation is added to the projected image points. The estimated camera parameters are compared with the ground truth, and the relative errors
Fig. 5. Calibration results at different noise levels.
are measured as in [16]. We vary the noise level from 0.1 to 1 pixel. For each noise level, 100 independent trials are performed. The results shown in Fig. 5 are the average and the standard deviation. Fig. 5(a) shows the average results refined by the nonlinear minimization, and Fig. 5(b) displays the STD of the results refined by the nonlinear minimization. We can see that error results of our method increase almost linearly with the noise level. Our camera calibration method is robust and efficient. Influence of the number of images used. In this experiment, we investigate the performance with respect to the number of the images used in calibration. The number of the images used for calibration varies from 4 to 22. For all images, the directions of rotation axes in all rotations and the arbitrary motions between every two adjacent rotations are altered and chosen at random. For each number, 100 dependent trials are performed. Independent Gaussian noise with mean 0 and standard deviation 0.2 is added in every trial. The average results are shown in Fig. 6(a), and the STD results are shown in Fig. 6(b). As shown in this figure, when more images are used, the errors will decrease. However, once the number of images is more than 6, the increase of accuracy will slow down. Sensitivity to the in the relative geometric structure vector. The experiment also investigates the performance of our camera calibration method with respect to the angle in . The cosine value of varies from the vector to 0.9. Twenty-two images are used at every cosine value of . Gaussion noise with mean 0 and standard deviation 0.3 is added to the projected image points. For each cosine value of , we perform 100 trials and compute the average errors. The result is shown in Fig. 7. When lies into the interval , best performance seems to be achieved. Note that , but in practice we only the angle can be in the interval
Authorized licensed use limited to: Shanghai Jiao Tong University. Downloaded on December 21, 2008 at 06:43 from IEEE Xplore. Restrictions apply.
ZHAO et al.: CAMERA CALIBRATION WITH THREE NONCOLLINEAR POINTS UNDER SPECIAL MOTIONS
2399
Fig. 8. Images captured by two CCD cameras with eight mm lens: (a) Point Grey FLEA; (b) WATEC 902H.
Fig. 6. Calibration results with different image numbers.
Fig. 9. Reconstruction results in the first real experiment.
TABLE I COMPARISON OF TWO CAMERA CALIBRATION METHODS
Fig. 7. Calibration results at different cosine values of theta.
set the angle in the interval the obtuse angle in geometry.
to avoid the trouble of
B. Experiment Results With Real Images To show the validity of our technique for camera calibration, we carried out camera calibration experiments with two kinds of CCD cameras using different calibration objects (clock dial and triangle board). In the first experiment, we use a clock dial as the calibration object and make its clock hands performing the rotations as described in Section III-A. There are three noncollinear points on two clock hands. Two Point Grey FLEA cameras are used to capture the images of them as shown in Fig. 8. The image resolution is 1024 768. We apply the proposed calibration method to these images and estimate the camera parameters with final optimization. To verify the estimated camera parameters, we apply them to the reconstruction of a model plane. As shown in Fig. 9, we take nine pairs of images captured by two cameras and give the reconstructions of the corner points on the model plane at nine different views. We can clearly see that the reconstructed points are indeed coplanar. In addition, the computed average distance between every two adjacent points is 30.21 mm, which
accords well with the ground truth 30 mm. This shows indirectly that the estimated camera parameters are accurate and our camera calibration technique is workable. For comparison, we also use the camera calibration algorithm in [15] to calibrate the Point Grey FLEA camera. Table I gives the calibration results of our method and Zhang’s method. In the second experiment, we take a triangle board to perform rotations around its longest edge as described in Section III-B. Two WATEC 902H cameras are used to take images of the calibration object [in Fig. 8(b)]. The image resolution is 768 576. By using these images, we do the camera calibration for the two cameras and get the final results of camera parameters. Then we do the reconstruction of a cup. Fig. 10(a) and (b) are the images of the cup captured by the two WATEC 902H cameras. In two images, the symbols “ ” denote the corners detected by using Harris’ method [19]. Fig. 10(c) and (d) give the reconstruction results of the cup shown at two different views. In the figures, the reconstruction results look realistic, which owe to the accurate camera parameters and our camera calibration algorithm. C. Application in a Hand-Eye Robot System We apply our camera calibration method in the calibrations of a hand-eye robot system (a MOTOMAN-HP3 robot with two Point Grey FLEA cameras). As shown in Fig. 11, the calibration object with three noncollinear points is mounted on the end-effector of the robot arm. The two cameras form a stereo camera system. We can control the robot arm to perform the motions that satisfy Proposition 1. From Section III-F, we know that the
Authorized licensed use limited to: Shanghai Jiao Tong University. Downloaded on December 21, 2008 at 06:43 from IEEE Xplore. Restrictions apply.
2400
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 17, NO. 12, DECEMBER 2008
Fig. 12. Comparison between the real robot track and the observed robot track.
Fig. 10. Reconstruction results in the second real experiment: (a) image 1; (b) image 2; (c) front view; (d) back view.
Fig. 13. Multicamera calibration results at different noise levels.
D. Experiments of Multicamera Calibration Algorithm
Fig. 11. Hand-eye robot system.
motions satisfy Proposition 1 can also make sure of having the unique solution for hand-eye calibration. Therefore, by using the calibration data that the robot motions provide, camera calibration and hand-eye calibration can both achieve satisfied results. It is the unique advantage of the special motion idea in our camera calibration algorithm. After having the camera parameters and the hand-eye parameters, we can transform the positions in camera coordinate to the positions in robot base coordinate. Then the robot arm’s actions can be directed by using the stereo camera system. To show the accuracy of calibration results, we move the robot arm to six positions in the experiment. Suppose denote the six end-effector positions in the robot base coordinate system (they are read from the robot condenote the same positions in troller) and the camera coordinate system (they are computed in the stereo reconstruction according to the camera parameters). After having the hand-eye parameters, we can also compute the transfrom the camera coordinate to the robot formation matrix base coordinate. Then we can get . from the Fig. 12 shows the real robot track observed robot controller and the robot track is 2.8 mm, by the cameras. The average error of which indirectly shows that our camera calibration algorithm is valid and workable.
To test the multicamera calibration method described in Appendix A, the simulation experiments are carried out to show its property to noise influence. Six cameras, which have the same setup as described in Section IV-A, are used in the simulation. The calibration object with three noncollinear points performs general motions, and every camera captures 22 images of the object. Gaussian noise of zero mean and standard deviation is added to all projected image points. The relative errors are measured at all noise levels from 0.1 to 1 pixel. 100 independent trials are performed at every noise level. Fig. 13 shows the average results of all six cameras in all intrinsic parameters. We can note that the error results in Fig. 13 are smaller than those in Fig. 5(a). Because the multicamera calibration algorithm has large data redundancy of image points, its ability in combating noise is more robust. V. CONCLUSION In this paper, we investigate the possibility of camera calibration using the object only having three noncollinear points. We show that when the calibration object performs a certain rigid motion as described in Section III, we can constructs a number of imaginary quadrilaterals for camera calibration. These quadrilaterals encode the metric information of their supporting planes, i.e., the Euclidean structure. According to the geometric properties of the imaginary quadrilaterals, we can have the distance constraints to solve the camera intrinsic parameters, and only even if we do not have the metric values of know the ratio of them (see in Section III-C). Geometrically speaking, the calibration object with three noncollinear points
Authorized licensed use limited to: Shanghai Jiao Tong University. Downloaded on December 21, 2008 at 06:43 from IEEE Xplore. Restrictions apply.
ZHAO et al.: CAMERA CALIBRATION WITH THREE NONCOLLINEAR POINTS UNDER SPECIAL MOTIONS
has the supporting plane and belongs to the 2-D calibration objects. However, the plane-based camera calibration techniques [2]–[4], [6]–[9], [11], [15] cannot just take three space points to solve the problems. Algebraically, the camera calibration using three noncollinear points can provide one and a half constraint equations for every view unlike the camera calibration based on the 1-D calibration objects [5], [16]. Therefore, the calibration objects with three noncollinear points are actually midway between 1-D and 2-D calibration objects, and our algorithm is a new kind of camera calibration algorithm. In Appendix, we also introduce how to do multicamera calibration by using three noncollinear points under general motions. It consists in first estimating projection depths. Our camera calibration algorithm has been tested with both synthetic data and real images, and very satisfactory results are obtained, especially in the real application. However, we have not yet considered the lens distortion in our camera calibration algorithm. In the future, we are planning to work on the problem. Eventually, the reader may wonder where to find the calibration objects with three noncollinear points. There are many objects that can be used for calibration in the real world, especially the clock dial with two clock hands. In practice, We can attach a triangle rig to a robotic end-effector and have the robot arm to perform desired motions. If the triangle rig is visible from any directions, our technique can be used to calibrate multiple cameras mounted apart from each other.
2401
, then we can get according to (30)
(31) where and are unknown scales. Without loss of generality, we can fix both projective and Euclidean coordinate frames to the first camera. Then the transformation matrix can be restricted to the following form: (32) represents the plane at infinity. We where and , have known that then we can obtain the following equations from (31) and (32):
(33)
(34) APPENDIX A. Multicamera Calibration Algorithm cameras, we can calibrate For a system composing of the cameras by using the three noncollinear points under general motions. Suppose are the image points of captured by the th camera. Then we can construct the scaled measurement matrix
.. .
.. .
(29)
are the projection depths estimated by where Sturm’s method [20]. It is actually complex in estimating these projection scales, especially for a large number of cameras. To cameras are needed. The satisfy the rank 4 condition, factorization [20] of (29) recovers the motion and the shape up to a 4 4 projective transformation
(30) where and If we set
, . and
, and is the 1st camera’s intrinsic pawhere rameter matrix. Given three or more images for every camera, in a least-squares sense. Then is obwe can determine and matrix inversion. tained by Cholesky factorization of Then is also computed from (33) and (34). is obtained, we can get by using the following Once equations: (35) as a least-squares solution for this over-deWe can obtain termined linear equation, and this completes the computation of . All camera matrices are recovered by . The first 3 3 sub-matrix of may be decomposed into the orthonormal rotation matrix and the upper triangular calibration matrix by RQ matrix decomposition. Then the position vector is computed according to (1). REFERENCES [1] O. Faugeras, Three-Dimension Computer Vision: A Geometric Viewpoint. Cambridge, MA: MIT Press, 1993. [2] B. Triggs, “Autocalibration from planar scenes,” in Proc. ECCV, 1998, pp. 89–105. [3] X. Meng and Z. Hu, “A new easy camera calibration technique based on circular points,” Pattern Recognit., vol. 36, no. 5, pp. 1155–1164, 2003. [4] Y. Wu, H. Zhu, Z. Hu, and F. Wu, “Camera calibration from quasiaffine invariance of two parallel circles,” in Proc. ECCV, 2004, pp. 190–202. [5] F. Wu, Z. Hu, and H. Zhu, “Camera calibration with moving one-dimensional objects,” Pattern Recognit., vol. 38, no. 5, pp. 755–765, 2005.
Authorized licensed use limited to: Shanghai Jiao Tong University. Downloaded on December 21, 2008 at 06:43 from IEEE Xplore. Restrictions apply.
2402
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 17, NO. 12, DECEMBER 2008
N
[6] P. Gurdjos, P. Sturm, and Y. Wu, “Euclidean structure from 2 parallel circles: Theory and algorithms,” in Proc. ECCV, 2006, pp. 238–252. [7] J. Kim, P. Gurdjos, and I. Kweon, “Geometric and algebraic constraints of projected concentric circles and their applications to camera calibration,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 4, pp. 637–642, Apr. 2005. [8] P. Gurdjos, J. Kim, and I. Kweon, “Euclidean structure from confocal conics: Theory and application to camera calibration,” in Proc. CVPR, 2006, pp. 1214–1221. [9] P. Sturm and S. Maybank, “On plane-based camera calibration: A general algorithm, singularities, applications,” in Proc. CVPR, 1999, pp. 1432–1437. [10] R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, 2nd ed. Cambridge, U.K.: Cambridge Univ. Press, 2003. [11] Q. Chen, H. Wu, and T. Wada, “Camera calibration with two arbitrary coplanar circles,” in Proc. ECCV, 2004, pp. 521–532. [12] J. C. Lagarias, J. A. Reeds, M. H. Wright, and P. E. Wright, “Convergence properties of the Nelder-Mead simplex method in low dimensions,” SIAM J. Optim., vol. 9, no. 1, pp. 112–147, 1998. [13] P. Hammarstedt, P. Sturm, and A. Heyden, “Degenerate cases and closed-form solution for camera calibration with one-dimension objects,” in Proc. ICCV, 2005, pp. 317–324. [14] H. H. Chen, “A screw motion approach to uniqueness analysis of head-eye geometry,” in Proc. CVPR, 1991, pp. 145–151. [15] Z. Zhang, “A flexible new technique for camera calibration,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 11, pp. 1330–1334, Nov. 2000. [16] Z. Zhang, “Camera calibration with one-dimensional objects,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 26, no. 7, pp. 892–899, Jul. 2004. [17] X. Cao and H. Foroosh, “Camera calibration using symmetric objects,” IEEE Trans. Image Process., vol. 15, no. 10, pp. 3614–3619, Oct. 2006. [18] K. Wong, R. Mendonca, and R. Cipolla, “Camera calibration from surfaces of revolution,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 25, no. 2, pp. 147–161, Feb. 2003. [19] C. Harris and M. Stephens, “A combined corner and edge detector,” in Proc. Alvey Conf., 1987, pp. 189–192. [20] L. Wang, F. Wu, and Z. Hu, “Multi-camera calibration with one-dimensional object under general motions,” in Proc. ICCV, 2007, pp. 1–7. [21] F. Wu, Mathematical Method in Computer Vision. Beijing, China: Science Press, 2008. [22] P. Sturm and B. Triggs, “A factorization based algorithm for multi-image projective structure and motion,” in Proc. ECCV, 1996, pp. 709–720. Zijian Zhao received the M.S. degree in electrical engineering from Shandong University, in 2005, and the Ph.D. degree in image processing and pattern recognition from Shanghai Jiao Tong University, in 2008. His research interests include computer vision, robot vision, and computer assisted orthopedic surgery.
Yuncai Liu (SM’90) received the Ph.D. degree from the Department of Electrical and Computer Science Engineering, University of Illinois at Urbana-Champaign, Urbana, in 1990. He was an Associate Researcher at the Beckman Institute of Science and Technology from 1990 to 1991. In 1991, he was a system consultant and then a chief consultant of research at Sumitomo Electric Industries, Ltd., Japan. In October 2000, he jointed the Shanghai Jiao Tong University as a Distinguished Professor. His research interests are in image processing and computer vision, especially in motion estimation, feature detection and matching, and image registration. He has also made progress in the research of intelligent transportation systems.
Zhengyou Zhang (F’05) received the B.S. degree in electronic engineering from the University of Zhejiang, China, in 1985, the M.S. degree in computer science from the University of Nancy, France, in 1987, the Ph.D. degree in computer science from the University of Paris XI, France, in 1990, and the Doctor of Science (Habilitation à diriger des recherches) diploma from the University of Paris XI, France, in 1994. He has been with INRIA (French National Institute for Research in Computer Science and Control) for 11 years and was a Senior Research Scientist from 1991 until he joined Microsoft Research in March 1998. In 1996–1997, he spent one-year sabbatical as an Invited Researcher at ATR (Advanced Telecommunications Research Institute International), Kyoto, Japan. He is now a Principal Researcher with Microsoft Research, Redmond, WA, and manages the humancomputer interaction and multimodal collaboration group. He holds more than 50 U.S. patents and has about 40 patents pending. He also holds a few Japanese patents for his inventions during his sabbatical at ATR. He has published over 160 papers in refereed international journals and conferences, has edited three special issues, and has co-authored three books: 3-D Dynamic Scene Analysis: A Stereo Based Approach (Heidelberg, 1992); Epipolar Geometry in Stereo, Motion and Object Recognition (Kluwer, 1996); and Computer Vision (Science Publishers, 2003). He is a member of the IEEE Computer Society Fellows Committee since 2005, Chair of IEEE Technical Committee on Autonomous Mental Development, and a member of IEEE Technical Committee on Multimedia Signal Processing. He is currently an Associate Editor of several international journals, including the IEEE TRANSACTIONS ON MULTIMEDIA, the International Journal of Computer Vision (IJCV), the International Journal of Pattern Recognition and Artificial Intelligence (IJPRAI), and the Machine Vision and Applications journal (MVA). He served on the editorial board of the IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE from 2000 to 2004, among others. He has been on the organization or program committees for numerous international conferences, and was a Program Co-Chair of the Asian Conference on Computer Vision (ACCV2004), 2004, Jeju Island, Korea, a Technical Co-Chair of the International Workshop on Multimedia Signal Processing (MMSP06), 2006, Victoria, BC, Canada, and a Program Co-Chair of the International Workshop on Motion and Video Computing (WMVC07), 2007, Austin, TX.
Authorized licensed use limited to: Shanghai Jiao Tong University. Downloaded on December 21, 2008 at 06:43 from IEEE Xplore. Restrictions apply.
