Multiple View Geometry

Multiple View Geometry in computer vision

Chapter 6: Camera Models

Michael Hofmann

Categorization • Camera mapping: matrices with particular properties

x = PX • Finite cameras – finite camera center

• Cameras at infinity – camera center is lying on plane at infinity

Categorization • Finite cameras – – – –

Basic pinhole model CCD camera Finite projective camera General projective camera

• Cameras at infinity – Affine cameras • • • •

Orthographic projection Scaled orthographic projection Weak perspective projection General affine camera

– General cameras at infinity

Finite cameras (pinhole model) • Mapping from points in space onto a plane (Z = f) • R3 to R2: X = ( X , Y , Z )T a ( fX / Z , fY / Z )T • Using homogeneous coordinates X    fX   f Y      Z  a  fY  =     Z   1  

f

X 0   Y  0  Z 1 0  1

x = PX P = diag ( f , f ,1)[I

0]

Finite cameras • Principal point offset

X = ( X , Y , Z )T a ( fX / Z + p x , fY / Z + p y )T

– Coordinate origin in image plane X    fX + Zp x   f   Y   + a fY Zp y =  Z       Z   1  

px py 1

X 0   Y  0  Z   0  1

x = K [I

f f K =   • CCD camera/Finite projective camera – non-square pixels / skew parameter

α x s K =  αy 

x0  y0  1 

α x = fmx α y = fm y x0 = mx p x y0 = m y p y

0]X cam f

px  p y  1 

Finite cameras • Extrinsic camera parameters (rotation & translation) – transformation from world coordinate to camera coordinate frame ~ ~ ~ (camera orientation & camera center) X cam = R ( X − C) – in homogeneous coordinates X cam

X ~   ~  R − RC Y   R − RC = X   =  1  Z   0 0  0 1  

– projection

[

x = KR I

]

~ −C X

P = K [R t ]

~ t = − RC

Finite cameras • Finite projective camera

[

P = KR I

~ −C

]

α x s K =  αy 

x0  y0  1 

– Left hand 3x3 sub-matrix ( KR ) is non-singular (RQ matrix decomposition: M = KR )

• General projective camera – remove non-singularity restriction – arbitrary homogeneous 3 x 4 matrix of rank 3 – 11 degrees of freedom (5 intrinsic, 6 extrinsic)

Projective camera properties • Camera center – P has 1-dim. null-space (rank 3, 4 columns), PC = 0 C is camera center as homogeneous 4-vector

X(λ ) = λA + (1 − λ )C

• Column points

⇒

P = ( p1

x = PX(λ ) = λPA + (1 − λ ) PC = λPA

p2

p3

p4 )

– p1 , p2 , p3 are the vanishing points of the WC axes – p4 is the image of the coordinate origin

X ,Y , Z

Projective camera properties • Principal plane –

P 3 is plane through camera center parallel to image plane (set of points X which are imaged on line at infinity)

PX = ( x, y,0)T

iff

P 3T X = 0

• Axis planes – P1 , P 2 are planes through camera center, represent set of points that maps to image lines x=0 or y=0 resp.

(P1T X = 0, P 2T X = 0 resp.)

 P1T   2T  P = P   P 3T   

Projective camera properties • Principal point – intersection point of principal axis & image plane (project principal axis vector onto image plane)

P = [M

p4 ] ⇒

x0 = Mm 3T

• Principal axis vector – m 3T points in direction of principal axis, but: direction ambiguity! – v = det( M )m 3T is directed towards the front of the camera • This holds for Pcam = K [I 0] ; unaffected by rotation, since det(R)>0

Action on points • Back-projection to rays – Given a point x in the image plane, what 3D ray is it on? – Representation: join of two points • Camera center C + + • the point P x ( P

= PT (PP T ) -1 ) , since P( P + x) = Ix = x – Therefore: X(λ ) = P + x + λC

• Depth of points ~

~

3T – w = m ( X − C) can be interpreted as the depth of X from C in direction of principal ray (for normalized P, det M > 0, m 3 = 1 ) – Normalization:

d ( X ; P) =

sgn(det M ) w T m3

Decomposition of P • Camera center – point, for which – Algebraically:

PC = 0

=> SVD of

X = det([p 2 , p 3 , p 4 ])

Y = − det([p1 , p 3 , p 4 ])

Z = det([p1 , p 2 , p 4 ])

T = − det([p1 , p 2 , p 3 ])

P

• Extrinsic/intrinsic parameters

[

] [

]

~ ~ – P = M − MC = K R − RC – Decompose M = KR using RQ-decomposition

Cameras at infinity • Affine camera – An affine camera has a camera matrix P in which the last row is of the form (0,0,0,1). – Points at infinity are mapped to points at infinity.

Affine camera • Tracking back (along the principal ray) while zooming in  r1T  2T Pt = KQ r r 3T 

~ − r1T (C − tr 3 )  d t d 0 2T ~ 3  − r (C − tr ) = K  ~  − r 3T (C − tr 3 ) 

dt d0

magnification factor k=dt/d0

 r1T  2T P∞ = lim Pt = K r t →∞  0T 

~ − r1T C )  2T ~  − r C ) d 0 

1T  r  r 2T  1 r 3T

~ − r1T C  2T ~  −r C d t 

~ d t = −r 3T C + t depth of camera center wrt. world origin

Affine camera • Error – All points on plane through world origin perpendicular to principal axis direction r 3 are unchanged.  α r1 + β r 2 + ∆ r 3   X =  – Error of a 3D point at a distance ∆  • image of the cameras

x proj

⇒

x + (d 0 + ∆)~ x0   K 2×2 ~  =  d0 + ∆  



Pt =0 and P∞ x affine

1



x + d0~ x0   K 2×2 ~   = d0  

d0 + ∆ ~ ~ ~ xaffine − x0 = ( x proj − ~ x0 ) d0

(dehomogenized)

• The effect of the affine approximation to the true camera matrix is to move the image of a point X radially towards or away from the principal point by a factor equal to d 0 + ∆ d 0

Hierarchy of affine cameras • Orthographic projection R P= T 0

1T 1 0 0 0  r t  2T  0 1 0 0 = r   1 0 0 0 1  0T

t1   t2  1 

• Scaled orthographic projection k P =  

 r  k  r 2T 1  0T 1T

t1   r1T   t 2  = r 2T 1   0T

t1   t2  1 k 

( X , Y , Z ,1)T a ( X , Y ,1)T (dropping the Z-coordinate)

Hierarchy of affine cameras • Weak perspective projection α x P =  αy 

 r  r 2T  1  0T 1T

t1   t2  1 

• Affine camera (general form) α x s P =  αy 

 r  r 2T  1  0T 1T

t1   m11  t 2  = m21 1   0

m12

m13

m22 0

m23 0

( M 2×3 rank 2)

t1  t 2  1 

Affine camera • Properties – Principal plane is plane at infinity (also optical center, since it lies on principal plane) – Any camera matrix for which the principal plane is the plane at infinity is affine – Parallel world lines are mapped to parallel image lines – The vector d satisfying M 2×3d = 0 is the direction of parallel T projection, and dT ,0 the camera center, since

(

d PA   = 0 0

)