Since the warp W is uniquely determined by positions of the shape landmarks, it suffices to apply Eq. (1) L times, once for the spatial position xl, l = 1,...,L of each ...
Adaptive and Constrained Algorithms for Inverse Compositional Active Appearance Model Fitting — CVPR 2008 Paper Supplemental Material — George Papandreou and Petros Maragos School of Electrical and Computer Engineering National Technical University of Athens, Greece http://cvsp.cs.ntua.gr
1. Including Priors into Flexible Warp-based Inverse Compositional Algorithms In this note we expand the discussion of Section 4.1 of the main paper on computing the inverse-compositional to additive parameter update (4 + n) × (4 + n) Jacobian matrix Jp˜ . Full details are given for the particularly interesting case of the thin-plate spline warp [2]. ˜ , which holds for all points x in the image ˜ + Jp˜ d˜ p) ≈ W W(x; −d˜ p); p Our starting point is the relationship W(x; p plane to first order in d˜ p [1, 4]. Differientiation w.r.t. d˜ p yields ∂W ∂W ∂W Jp˜ . ≈− ˜ (x;˜p) ˜ (x;˜p=0) ∂p ∂x (x;˜p) ∂ p |{z} | {z } | | {z } (4+n)×(4+n) {z } 2×(4+n)
2×2
(1)
2×(4+n)
Equation (1) gives 2 × (4 + n) constraints per image point x. Since the warp W is uniquely determined by positions of the shape landmarks, it suffices to apply Eq. (1) L times, once for the spatial position xl , l = 1, . . . , L of each landmark on the mean shape s0 . Putting together the L resulting terms in a single block matrix equation yields ∂W ∂W ∂W ˜ ˜ ∂x (x1 ;˜p) ∂ p (x1 ;˜p=0) ∂ p (x1 ;˜p) .. .. . Jp˜ (2) ≈ − . . |{z} ∂W ∂W ∂W (4+n)×(4+n) ˜ ˜ ∂p ∂x ∂p (xL ;˜ p) (xL ;˜ p) (xL ;˜ p=0) | | {z } {z } 2L×(4+n)
2L×(4+n)
∂W the (2L) × (4 + n) stacked matrix of derivatives on the left-hand-side and as ⊙ Denoting as ∂W ˜ ∂p ∂x (x1:L ;˜ p) (x1:L ;˜ p) ∂W the stacked block-by-block matrix product on the right-hand-side of the previous equation, we can write it more ˜ ∂p (x1:L ;0)
compactly as
∂W ∂W ∂W ⊙ Jp˜ . ≈− ˜ (x1:L ;˜p) ˜ (x1:L ;0) ∂p ∂x (x1:L ;˜p) ∂p |{z} | | {z } (4+n)×(4+n) {z } 2L×(4+n)
(3)
2L×(4+n)
Solving this with the method of least squares yields the Jacobian estimate Jp˜ = −
−1 ∂W ∂W ∂W ∂W T ⊙ , ˜ (x1:L ;˜p) ∂ p ˜ (x1:L ;˜p) ˜ (x1:L ;0) ∂p ∂x (x1:L ;˜p) ∂p
(4)
which is Eq. (22) of our main paper. We move forward and show how the matrices involved in Eq. (4) can be computed. Regarding the (2L) × (4 + n) ∂W ∂W ˜ ) = St W(x, p) and , we need compute the ∂ p˜ Jacobian. Applying the chain rule on W(x, p matrix ∂ p˜ (x1:L ;˜ p)
(x;˜ p)
considering separately the similarity t and deformation p parameters gives ∂W ∂S ∂S · = ∂t ∂x (W(x,p);t) (W(x,p);t) ˜ (x;˜p) ∂p
∂W ∂p
(x;p)
(5)
Taking advantage of the fact that we only need to evaluate the quantities above on the landmark positions xl , it is easy to show (c.f . [4, Sec. 4.1.2]) that h ∂W = sp ˜ (x1:L ;˜p) ∂p
s+ p
1x
1+ x
(1 + t1 ) s1
. . . sn + t2 s+ 1
i , . . . s+ n
(6)
Pn where sp = s0 + i=1 pi si is the deformed shape, given the parameters p, s+ denotes the shape s rotated counter-clockwise by 90o and 1x = [ 1 0 ··· 1 0 ]T is the shape with 1’s in the x-coordinate and 0’s in the y-coordinate. ∂W Regarding the 2L × 2 matrix ∂x , we need compute the Jacobian ∂W . Application of the chain rule on ∂x (x1:L ;˜ p) (x;˜ p) ˜ ) = St W(x, p) gives W(x, p ∂W ∂S ∂W 1 + t1 −t2 ∂W = = . (7) t2 1 + t1 ∂x (x;p) ∂x (x;˜p) ∂x (W(x,t);t) ∂x (x;p) depends on the warp family under consideration. For the often used Computation of the deformation field Jacobian ∂W ∂x (x;p)
thin-plate spline warp [2], we can write the warp function W(x, p) in the form of a generalized linear model (c.f . [3, App. F]) W(x, p) = W (p) k(x) , (8) | {z } |{z} 2×(L+3) (L+3)×1
where the vector k(x) is given by
k(x) = U (|x − x1 |)
...
U (|x − xL |)
1
x y
T
,
(9)
U (r) = r2 ln r2 is the spline kernel, and W (p) is determined by requiring that the warp maps s0 to sp . The final result is 2 2(1 + ln rx,1 )(x − x1 )T .. . dk(x) ∂W 2(1 + ln r2 )(x − xL )T x,L (10) = W (p) , = W (p) ∂x (x;p) dx x 0 0 1 0 0 1 where rx,l = kx − xl k2 . We need to evaluate ∂W for each landmark point xl . Since the term k(x) does not depend on ∂x (x;p) the shape parameter p, dk(x) dx can be pre-computed and be subsequently used in every AAM iteration. x
The cost of computing the Jacobian matrix Jp˜ can be analyzed as follows: (a) Computing the (2L) × (4 + n) matrix ∂W ∂W is O (nL). (b) Computing the 2L × 2 matrix ∂x is O (L). (c) Forming the stacked block-by-block ˜ ∂p (x1:L ;˜ p) (x1:L ;˜ p) ⊙ ∂W is O (nL). (d) Forming the (4 + n) × (4 + n) least-squares system matrix matrix product ∂W ˜ ∂x ∂p (x1:L ;˜ p) (x1:L ;0) T ∂W ∂W is O n2 L . (e) Inverting the same system matrix is O n3 . Overall, the last two operations ˜ ˜ ∂p ∂p (x1:L ;˜ p) (x1:L ;˜ p) dominate the cost of the procedure, whose overall complexity is thus O n2 L + n3 .
References [1] S. Baker, R. Gross, and I. Matthews. Lucas-Kanade 20 years on: A unifying framework - Part 4. Technical Report CMU-RI-TR-04-14, Robotics Institute, CMU, 2004. [2] F. L. Bookstein. Principal warps: Thin-plate splines and the decomposition of deformations. IEEE Tr. on PAMI, 11(6):567–585, 1989. [3] T. F. Cootes and C. J. Taylor. Statistical models of appearance for computer vision. http://www.isbe.man.ac. uk/˜bim/Models/app_models.pdf, 2004. [4] I. Matthews and S. Baker. Active appearance models revisited. Int. J. of Comp. Vision, 60(2):135–164, 2004.
