Abstract. In this paper, a simple method for ellipse detection is proposed and applied in central catadioptric camera calibration. Similar to many previous ...
RANSAC based ellipse detection with application to catadioptric camera calibration
Abstract. In this paper, a simple method for ellipse detection is proposed and applied in central catadioptric camera calibration. Similar to many previous approaches, it consists of two phases. Firstly it locates ellipse center candidates using center symmetry of ellipses, and the detected edge points are grouped into several subsets according to the center candidates. Then all the ellipses are fitted by performing RANSAC for each subset. We also present an approach for calibrating a central catadioptric camera based on the bounding ellipse of the catadioptric image. Using the proposed ellipse detection method, we can easily detect the bounding ellipse. As a result, a self-calibration can be realized. Experiments show the proposed method is effective. Keywords: Ellipse detection, central catadioptric camera, camera calibration.
1. Introduction Detection of ellipses or circles from an image of a scene is very useful in many vision applications such as object location, tracking, and camera calibration etc. Hutter and Brewer [1] fit an ellipse to the image of the vehicle wheel for vehicle pose determination. Kwolek [2] uses an ellipse model to approximate the image of the human head in head tracking. Ying et al [3, 4] use the images of spheres, which are ellipses, to calibrate central catadioptric cameras. For detecting structures in images, Hough transform and RANSAC are generally used robust techniques. However, an ellipse has five parameters which include location, shape, and orientation. So directly applying Hough transform to ellipse detection requires a five-dimensional accumulator. It is impractical because of the huge computation and storage. On the other hand, it is well known that RANSAC also needs a huge computation if the ratio of the inliers is low. In papers [5,6], ellipse detection is decomposed into two stages. The first stage detects the ellipse center, and the second stage determines the other parameters. The method in [5] requires accurate calculation of the gradients and tangents of the edge pixels, which is sensitive to the image noise and time-consuming. The method in [6] firstly determines two symmetry axes by the Hough transform following the horizontal and vertical scanning, whose cross is the ellipse center, and then uses the geometry symmetry to determine the other parameters. Xu etal.[7] develop the Randomized Hough Transform which randomly selects five points in each iteration and use them to vote on the ellipse parameters. The accuracy and speed of this algorithm depend on the ratio of the inliers, which is similar to RANSAC. If the ratio is low, the number of the random sampling should be large, and then the computation is heavy. Many computer vision applications expect a large field of view such as robot navigation, surveillance, teleconferencing and virtual reality etc. One effective way to enlarge the field of view is to combine mirrors with conventional cameras, which is called a catadioptric imaging system [8]. In catadioptric systems, a single effective viewpoint is highly desirable due to its superior and useful geometric properties [9,10]. A catadioptric system with a unique viewpoint is called a central
catadioptric system. A central catadioptric system [8] can be built by setting a parabolic mirror in front of an orthographic camera, or a hyperbolic, elliptical, planar mirror in front of a perspective camera, where the single viewpoint constraint can be fulfilled via a careful alignment of the mirror and the camera. Now the calibration of central catadioptric cameras has been an active research field. Previous calibration methods [3,4,10,11] for central catadioptric cameras are mainly based on the projections of lines. Nearly all these approaches need conic fitting since a line in space is projected to a conic in a central catadioptric image, and the accuracy of the calibration highly depends on the accuracy of the conic fitting. The conic is called the line image. In general, only a small segment of the conic is visible in the catadioptric image due to the partial occlusion, which makes the conic estimation hard to accomplish. Wu et al [12] present a calibration method of no conic estimation, but it is mainly for para-catadioptric cameras. Moreover, there are few algorithms for self-calibration. Kang[13] proposes a nonlinear self-calibration method to calibrate a para-catadioptric system by tracking feature correspondences across multiple views. Since the omni-directional images have large distortion, finding feature correspondences is not a trivial task. In this work, similar to several previous approaches, ellipse detection is decomposed into two phases. Firstly ellipse center candidates are detected using center symmetry of ellipses, and the detected edge points are grouped into several subsets according to the center candidates. Then all the ellipses are fitted by performing RANSAC for each subset. Using the proposed ellipse detection method, we can easily detect the bounding ellipse of the catadioptric image. As a result, a simple self-calibration method is presented. Section 2 shows the ellipse detection. Section 3 describes the proposed calibration method. Experimental results are reported in Section 4, and followed are some conclusions in Section 5.
2. Ellipse detection An ellipse is a conic, and can be generally represented as follows:
ax 2 + 2bxy + cy 2 + 2dx + 2ey + f = 0
(1)
Given an image, the task of the ellipse detection is to decide whether the image contains an ellipse, and to determine the ellipse equation (1) if an ellipse is present. Similar to previous approaches [5,6], we decompose the ellipse detection into two phases. The ellipse center candidates are detected in the first phase, and other parameters are determined in the second phase. It is well known that the ellipse is symmetrical on the ellipse center, and the middle point of any two symmetric points is the ellipse center. Since each pair of symmetric points will cast a vote on the center, while each pair of dissymmetrical points casts a vote dispersedly, the ellipse center can be detected by the voting from the middle point of each two points. For each ellipse center candidate, there is an associated subset of symmetric points. In the second phase, we use RANSAC to estimate the ellipse equation (1) for each ellipse candidate. RANSAC is an iterative method for robust fitting of a model from a set of observed data which contains many outliers. Assume that the ratio of the inliers is ρ , the minimal data set for determining the model includes k data points, and the probability that there is a good sampling among the K samplings is z , then [14]
K=
log(1 − z ) log(1 − ρ k )
(2)
From the equation (2), we can see that the number K increases with the ratio ρ decreasing and with the data number k increasing. Generally speaking, the number k should be 5 for ellipse parameter estimation since an ellipse has 5 parameters. So the sampling number K will be very large and the computation is impractical if the ratio of the inliers is very low. Evidently, the ratio of inliers in each subset is very high for each ellipse center candidate, and the point number of the minimal data set for determining the ellipse is 3 because only three geometry parameters need to be determined. So the parameters can be detected accurately and quickly by RANSAC. Finally, we need to verify whether each detected ellipse candidate is a real ellipse since we use a threshold in detecting the center candidates. For each ellipse candidate, we can determine the following parameters from the equation (1):
a b
d
E = ac − b , F = a + c, H = b
c
e
d
e
f
2
(3)
An ellipse should subject to the condition [15]:
H ≠ 0, E > 0, FH ≤ 0 . Where, the ellipse is a circle if
(4)
E= 0 .
3. Central catadioptric camera calibration based on the boundary ellipse 3.1 Central catadioptric camera Geyer and Daniilidis [9] show that the central catadioptric imaging process is equivalent to the following two-step mapping by a sphere (see Fig. 1): 1).Under the viewing sphere coordinate system O-xyz, a 3D point X = [x, y, z]T is projected to a point Xs on the unit sphere centered at the viewpoint O by Xs = [x / r, y / r, z / r]T , r =|| X || . 2).The point Xs on the viewing sphere is projected to a point m on the catadioptric image plane
Π by a pinhole camera through the perspective center O c . In this camera system, the optical axes of the pinhole camera is the line OcO , and thus its principal point is the intersection, p = [u 0 , v0 ,1]T , of the line OcO with the image plane Π . The distance from point O to Oc , ξ =|| O − OC || , is called the mirror parameter, which determines the mirror used in the central catadioptric camera. The mirror is a paraboloid if ξ = 1 , an ellipsoid or a hyperboloid if 0 < ξ < 1 , and a plane if ξ = 0 . The details can be found in Ref.[9]. In this paper, we assume 0 < ξ ≤ 1 , i.e. do not consider the case of plane mirror. Let the intrinsic matrix of the pinhole camera be
⎡ rf K = ⎢⎢ 0 ⎢⎣ 0
s u0 ⎤ f v0 ⎥⎥ . 0 1 ⎥⎦
(5)
Where f is the effective focal length; r is the aspect ratio; p =[u0,v0,1]T is the principal point; s is the parameter describing the skew of the two image axes. Then the catadioptric image of a space point X is
⎡X X m = λ K[I, ξe] ⎢ ⎣ 1 where
λ
is a scalar,
⎤ ⎥ = λ K( X X + ξe) ⎦
,
(6)
I is an identical matrix, e = [0, 0,1]T , and ξe is the coordinate vector of the
sphere center O under the pinhole coordinate system Oc-xcyczc. In the catadioptric camera calibration, there are totally six parameters [ f,r,s,u0,v0,ξ] to be determined.
Oc
yc
ξ
xc
zc
O y
z
X
Xs X
L
u
o v
m
p C
Π
Figure 1. Central catadioptric camera
3.2 Calibration based on the boundary ellipse It’s well known that the mirror boundary of the central catadioptric camera is a circle and the projection of the mirror boundary is an ellipse (see Fig.2). The optical axis is perpendicular to the plane containing the circle and goes through the center of the circle, so the center of the bounding ellipse is the principal point ( u 0
v0
1) . T
In [13, 16], partial intrinsic parameters of the camera can be
determined from the bounding ellipse of the catadioptric image. In fact, since both the eccentricity ε of the mirror and the field of view (FOV) are usually known, we can determine all intrinsic parameters from the bounding ellipse. Here the mirror parameter ξ can be obtained from the eccentricity ε of the mirror as [9]:
ξ=
2ε 1+ε 2
(7)
Let the distance from the optical center O c to the center O b of the boundary circle be h and the radius of the circle be q. As Fig.2 shows, q and h can be decided from FOV, i.e. q = sin(θ), h = ξ + cos(θ) , where θ = FOV
.
2
Assume that the coordinate of a point on the mirror boundary under the perspective coordinate system Oc x c y c z c is X = ( x, y, h ) , and its image point is m = ( u, v,1) . T
T
Then,
K −1m =
1 X and h
XT X q2 m ωm = m K K m = 2 = 1 + 2 h h T
T
−T
.
(8)
−1
Where ω = K − T K −1 is the image of the absolute conic (IAC) of the pinhole camera. Then the bounding ellipse can be expressed as
mT Cm = 0
(9)
⎛0 0 0⎞ 1 q2 ⎜ ⎟ , λ = 1 + 2 , E3 = 0 0 0 . where C ≈ (ω − λ E3 ), ω11 = ⎜ ⎟ rf h ⎜0 0 1⎟ ⎝ ⎠ In addition, the catadioptric image of the 3D mirror point X 0 = (q,0, h)T is
m0 =
1 ⎛ rfq KX0 = ⎜ + u0 h ⎝ h
T
⎞ v0 1⎟ ⎠
(10)
It is one intersection of the bounding ellipse and the line v = v 0 on the image plane. So we can determine the item ω11 of the IAC from the image point m 0 , and the IAC can be uniquely determined by the item ω11 and the bounding ellipse equation (9). Thus, all camera intrinsic parameters can be obtained by using Choleskey decomposition of the IAC. Oc c
y zc ξ O
y
xc h
x
θ q O
o
z b
X0
u
v p
m0 Π
Figure 2. Mirror boundary and its projection
4．Experimental results Fig.3a
shows
an
image
with
a
resolution
http://mail.isr.uc.pt/~carloss/software/software.htm,
of
which
1024×768
is
acquired
downloaded by
an
from
uncalibrated
paracatadioptric camera [10]. Its field of view (FOV) is 180 degree [17]. The edge map for this image is extracted using the Canny edge operator and is shown in Fig.3b. The detected ellipse center and the points symmetrical on the center are shown in Fig.3c in red and blue colors respectively. The boundary ellipse detected by RANSAC is shown in Fig.3a. The image is calibrated using the detected boundary ellipse. The rectified image using the calibration result is shown in Fig.3d. Fig.4a shows another image with a resolution of 2048×1536, which is taken by a catadioptric system consisting of a perspective camera with a hyperbolic mirror. The mirror is designed by the Center for Machine Perception, Czech Technical University, its FOV is 217.2 degree, and the eccentricity of the hyperbolic mirror is 1.302, corresponding to ξ = 0.966. Fig.4b shows the edge map. Fig.4c shows the detected ellipse center and the symmetrical point set. The boundary ellipse detected by RANSAC is shown in Fig.4a, and Fig.4d shows the rectified image using the calibration result. We can see from the figures that there are many outliers among the symmetrical point sets of the two images, and the ellipse detection using RANSAC is very accurate. From the rectified images, we can see that the rectified lines are roughly straight, which shows that the calibration approach is effective.
(a) The image and the boundary ellipse
(b) The edge map
100 200
300 400
500 600
700 100
(c) The ellipse center and the symmetrical point set
200
300
400
500
600
700
800
(d) The rectified image
Figure 3. the paracatadioptric camera
900
1000
(a) The image and the boundary ellipse
(b) The edge map
200 400
600
800 1000
1200 1400 200
(c) The ellipse center and the symmetrical point set
400
600
800
1000
1200
1400
1600
1800
2000
(d). The rectified image
Figure 4. the hypercatadioptric camera
5. Conclusion In this work, a simple method for ellipse detection is proposed and applied in catadioptric camera calibration. Similar to several previous approaches, the ellipse detection is decomposed into two phases. Firstly ellipse center candidates are detected using center symmetry of ellipses, and the detected edge points are grouped into several subsets according to the center candidates. Then all the ellipses are fitted by performing RANSAC for each subset. Using the proposed ellipse detection method, we can easily detect the bounding ellipse of the catadioptric image. As a result, a simple self-calibration method for central catadioptric cameras is presented, which can be used in some applications where high accuracy of the calibration is not required. Experiments demonstrate the efficiency of the proposed method. Of course, the calibration accuracy can be improved by an optimizing process automatically. This is our future work. REFERENCES 1.
Marcus Hutter and Nathan Brewer. Matching 2-D Ellipses to 3-D Circles with Application to Vehicle Pose Identification. IVCNZ 2009，153-158
2.
Kwolek. B. Stereo-vision based head tracking using color and ellipse fitting in a particle filter. ECCV 2004, vol. 4, pp. 192-204.
3.
X.H. Ying and Z.Y. Hu, “Catadioptric camera calibration using geometric invariants”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 10, pp.1260-1271, 2004.
4.
Xianghua Ying, Hongbin Zha, Identical projective geometric properties of central catadioptric line images and sphere images with applications to calibration, IJCV, vol 78, pp.89-105, 2008
5.
A.Aguado, M.E.Montiel and M.S.Nixon. On using directional information for parameter space decomposition in ellipse detection. Pattern recognition, 29(3):369-381, 1996
6.
C.T. Ho and L.H.Chen, A fast ellipse/circle detector using geometric symmetry. Pattern recognition 28(1):117-124, 1995.
7.
L. Xu, E.Oja. Randomized Hough Transform(RHT): Basic Mechanisms, Algorithms and Complexities. Computer Vision, Graphics, and Image Processing: Image Understanding. 1993, 57: 131-151.
8.
S. Baker and S. Nayer, A theory of single-viewpoint catadioptric image formation, IJCV, vol.35, no.2, pp.175-196, 1999
9.
C. Geyer and K. Daniilidis, Catadioptric projective geometry, IJCV, vol.45, no.3, pp 223-243, 2001
10.
J.P. Barreto and H. Araujo, “ Geometric properties of central catadioptric line images and their application in calibration”, IEEE Trans on PAMI, vol. 27, no.8, pp.1327-1333, 2005.
11.
Fuchao Wu, Fuqing Duan, Zhanyi Hu, Yihong Wu, A new linear algorithm for calibrating central catadioptric cameras, Pattern Recognition, Vol 41, pp.3166-3172, 2008
12.
Y. Wu, Y. Li, and Z. Hu, Easy calibration for para-catadioptric-like camera, IEEE International Conference on Intelligent
13.
S.Kang, Catadioptric self calibration. CVPR,Vol.I,pp.201-207,2000
14.
M. A. Fischler, R. C. Bolles. Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image
Robots and Systems, pp.5719-5724,2006
Analysis and Automated Cartography. Comm. of the ACM, Vol 24, pp 381-395, 1981 15.
Donald L. Vossler, Exploring Analytic Geometry with Mathematica, Academic Press, 1999
16.
C. Geyer and K. Daniilidis, Catadioptric camera calibration, Proc.7th ICCV, Vol.I, pp.398-404,1999
17.
J.P.Barreto, H.Araujo, Fitting conics to paracatadioptric projection of lines. Computer Vision and Image Understanding, 101(3), 151–165, 2006
