[5] Stratou, Giota and Ghosh, Abhijeet and Debevec, Paul and Morency, Louis-Philippe. Effect of illumination on automatic expression recognition: a novel 3D re-.
Functional Faces: Groupwise Dense Correspondence using Functional Maps 1
1
2
1
Chao Zhang , William A.P. Smith , Arnaud Dessein , Nick Pears , and Hang Dai 1
1
Dept. of Computer Science, The University of York, UK 2 MB/LaBRI, Universit´e de Bordeaux, France
Goal
Visualising Functional Maps Error
Qualitative Results
Filtering Feature Matches
Problem Exact point-to-point correspondence in a set of 3D face shapes may not exist, e.g. eyebrows. Only several fiducial points are well-defined. Goal Propose to optimise dense correspondence intrinsically in a groupwise fashion. Contributions First, we propose a groupwise variant functional maps [1]. Second, we heuristically design a set of real-valued functions that are appropriate specifically to the problem of face correspondence. Finally, we show how functional maps provide a constraint that can be used to filter potential feature matches between faces.
We visualise the error of correspondence by using a SVD decomposition of the functional maps [2]. Three kinds of functional maps are compared: group-truth, pairwise, and groupwise. Only the first two modes are shown. Areas undergoing large error are highlighted in the figure.
Texture transfer [4]
We use functional maps to restrict the tentative feature matches. A Euclidean distance in the functional embedded space is used as a threshold.
Pairwise Functional Maps Functions Several pairs of correspondent functions f and g from both shapes. Basis and coefficients LB basis M and N and coefficients a and b of functions f and g are computed. Functional Map The unknown functional map C can be found given enough constraints of type Cai = bi. Visualizing Maps Ground-truth: within-person (left) and between-person (right). Not much di↵erence! 10
0.8
10
0.8
20
0.6
20
0.6
30
0.4
30
0.4
40
0.2
40
0.2
50
0
50
0
60
-0.2
60
-0.2
70
-0.4
70
-0.4
80
-0.6
80
-0.6
-0.8
90
-0.8
90 100
-1
20
40
60
80
100
40
60
80
Group FM shape
Mouth area not nice match due to inner mouth part
texture
Expression morphing [5]
Texture
Ground Pairwise truth
Groupwise
Ground truth
Source shape
Pairwise Groupwise
i,j
Correspondence functions We propose several possible functions for face correspondence task.
Quantitative Results
Conclusions
Geodesic error We compare geodesic error of our method with state-of-the-art methods, using a subset of 6 face shapes of the same person [6]. (b)' (a)' Source'shape'and'texture' Intermediate'func4ons'
(c)'
(d)'
(e)'
(f)'
1
0.9
1. Functional maps representation brings efficiency and flexibility to building groupwise correspondence. 2. Imposing hard orthogonality constraint is beneficial to both within-person and between-person face correspondence. Code: www-users.cs.york.ac.uk/~wsmith/CVPR16
0.8
References
% Correspondences
0.6
[1] Ovsjanikov, Maks and Ben-Chen, Mirela and Solomon, Justin and Butscher, Adrian and Guibas, Leonidas Functional maps: a flexible representation of maps between shapes. TOG, 2012.
Pairwise [21]
0.5
Group−wise (ours) 0.4
Pairwise+ICP [21] Group+ICP (ours)
0.3
[2] Ovsjanikov, Maks and Ben-Chen, Mirela and Chazal, Frederic and Guibas, Leonidas Analysis and visualization of maps between shapes. CGF, 2013.
Non−rigid ICP [1] 0.2
(h)' (g)' Shape5derived'func4ons'
(i)'
(j)'
(k)'
(l)'
(m)'
(n)'
CSP [11]
[3] Manopt, a Matlab http://www.manopt.org
0.1
0 0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
Geodesic error
Texture consistency (ave. color variance)
i,j
Assuming functional map matrices are orthonormal, the manifold optimisation is solved with Manopt [3].
Target 2
First, the choice of functions is not exploited and it will be quite useful if we can learn some optimal weights for them before hand. Second, converting functional maps to point-to-point correspondence is quite engineering so far with the help of TPS fitting. In the future, we’d like to see an elegant way to achieve this goal.
0.7
Minimal Set Optimisation With two facts Cij = Cji1 and Cki = Ckj Cji, all the maps between two shapes can be reconstructed by using Cji = Cj1C1i = Cj1Ci11 Note. We emphasise that the solution is independent of the chosen reference shape. Hard Orthonormality Constraint Besides functional preservation, we also make use of operator commutativity. The objective function could be written as: X X "= kCij Pj Pik2F + ↵ kQj Cij Cij Qik2F
Target 1
Limitations
100
Groupwise Functional Maps
NR-ICP
Error heat map (K = 2)
Error heat map (K = 1)
100 20
Eyebrows not matching It is up to the ref. shape
(p)' (o)' Texture5derived'func4ons'
(q)'
(r)'
(s)'
(t)'
(u)'
(v)'
Method Texture Consis. Run time Non-rigid ICP .0103 6 hours Ours NN .0101 55 seconds Ours TPS .0099 120 seconds
Toolbox
for
Optimization
on
Manifolds,
[4] Paysan, Pascal and Knothe, Reinhard and Amberg, Brian and Romdhani, Sami and Vetter, Thomas A 3D face model for pose and illumination invariant face recognition. AVSS, 2009. [5] Stratou, Giota and Ghosh, Abhijeet and Debevec, Paul and Morency, Louis-Philippe E↵ect of illumination on automatic expression recognition: a novel 3D relightable facial database. FG, 2011. [6] Yin, Lijun and Wei, Xiaozhou and Sun, Yi and Wang, Jun and Rosato, Matthew J A 3D facial expression database for facial behavior research FG, 2006
Acknowledgement The authors would like to thank The British Machine Vision Association (BMVA) for funding the travel. LATEX Tik Zposter
