Functional Faces: Groupwise Dense Correspondence Using Functional Maps

In this paper we present a method for computing dense correspondence between a set of 3D face meshes using functional maps. The functional maps paradigm brings with it a number of advantages for face correspondence. First, it allows us to combine various notions of correspondence. We do so by proposing a number of face-specific functions, suited to either within-or between-subject correspondence. Second, we propose a groupwise variant of the method allowing us to compute cycle-consistent functional maps between all faces in a training set. Since functional maps are of much lower dimension than point-to-point correspondences, this is feasible even when the input meshes are very high resolution. Finally, we show how a functional map provides a geometric constraint that can be used to filter feature matches between non-rigidly deforming surfaces.

[1]  Jun Wang,et al.  A 3D facial expression database for facial behavior research , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[2]  Andrea Cavallaro,et al.  Matching 3D Faces with Partial Data , 2008, BMVC.

[3]  Leonidas J. Guibas,et al.  Functional map networks for analyzing and exploring large shape collections , 2014, ACM Trans. Graph..

[4]  Maks Ovsjanikov,et al.  Functional maps , 2012, ACM Trans. Graph..

[5]  Andrew Jones,et al.  Driving High-Resolution Facial Scans with Video Performance Capture , 2014, ACM Trans. Graph..

[6]  Xavier Bresson,et al.  Functional correspondence by matrix completion , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Li Zhang,et al.  Spacetime faces: high resolution capture for modeling and animation , 2004, SIGGRAPH 2004.

[8]  Timothy F. Cootes,et al.  A minimum description length approach to statistical shape modeling , 2002, IEEE Transactions on Medical Imaging.

[9]  Norimichi Tsumura,et al.  Image-based skin color and texture analysis/synthesis by extracting hemoglobin and melanin information in the skin , 2003, ACM Trans. Graph..

[10]  Sami Romdhani,et al.  Optimal Step Nonrigid ICP Algorithms for Surface Registration , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Bamdev Mishra,et al.  Manopt, a matlab toolbox for optimization on manifolds , 2013, J. Mach. Learn. Res..

[12]  Alexander M. Bronstein,et al.  Coupled quasi‐harmonic bases , 2012, Comput. Graph. Forum.

[13]  Maks Ovsjanikov,et al.  Supervised Descriptor Learning for Non-Rigid Shape Matching , 2014, ECCV Workshops.

[14]  Leonidas J. Guibas,et al.  An Optimization Approach to Improving Collections of Shape Maps , 2011, Comput. Graph. Forum.

[15]  Leonidas J. Guibas,et al.  Consistent Shape Maps via Semidefinite Programming , 2013, SGP '13.

[16]  Paul E. Debevec,et al.  Effect of illumination on automatic expression recognition: A novel 3D relightable facial database , 2011, Face and Gesture 2011.

[17]  Andrea J. van Doorn,et al.  Surface shape and curvature scales , 1992, Image Vis. Comput..

[18]  Sami Romdhani,et al.  A 3D Face Model for Pose and Illumination Invariant Face Recognition , 2009, 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance.

[19]  Massimo Mauro,et al.  Multi-class semantic segmentation of faces , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[20]  Stefanos Zafeiriou,et al.  300 Faces in-the-Wild Challenge: The First Facial Landmark Localization Challenge , 2013, 2013 IEEE International Conference on Computer Vision Workshops.

[21]  Andriy Myronenko,et al.  Point Set Registration: Coherent Point Drift , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Touradj Ebrahimi,et al.  Using face morphing to protect privacy , 2013, 2013 10th IEEE International Conference on Advanced Video and Signal Based Surveillance.

[23]  Richard Szeliski,et al.  Multi-image matching using multi-scale oriented patches , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[24]  Leonidas J. Guibas,et al.  Image Co-segmentation via Consistent Functional Maps , 2013, 2013 IEEE International Conference on Computer Vision.

[25]  Michael Möller,et al.  Point-wise Map Recovery and Refinement from Functional Correspondence , 2015, VMV.

[26]  Hanspeter Pfister,et al.  Face transfer with multilinear models , 2005, ACM Trans. Graph..

[27]  Matthew Turk,et al.  A Morphable Model For The Synthesis Of 3D Faces , 1999, SIGGRAPH.

[28]  Michael M. Bronstein,et al.  MADMM: A Generic Algorithm for Non-smooth Optimization on Manifolds , 2015, ECCV.

[29]  Hao Li,et al.  Global Correspondence Optimization for Non‐Rigid Registration of Depth Scans , 2008, Comput. Graph. Forum.

[30]  Michael J. Black,et al.  Coregistration: Simultaneous Alignment and Modeling of Articulated 3D Shape , 2012, ECCV.

[31]  Alexander M. Bronstein,et al.  Numerical Geometry of Non-Rigid Shapes , 2009, Monographs in Computer Science.