Fully automatic expression-invariant face correspondence

We consider the problem of computing accurate point-to-point correspondences among a set of human face scans with varying expressions. Our fully automatic approach does not require any manually placed markers on the scan. Instead, the approach learns the locations of a set of landmarks present in a database and uses this knowledge to automatically predict the locations of these landmarks on a newly available scan. The predicted landmarks are then used to compute point-to-point correspondences between a template model and the newly available scan. To accurately fit the expression of the template to the expression of the scan, we use as template a blendshape model. Our algorithm was tested on a database of human faces of different ethnic groups with strongly varying expressions. Experimental results show that the obtained point-to-point correspondence is both highly accurate and consistent for most of the tested 3D face models.

[1]  G Learned-MillerErik.  Data Driven Image Models through Continuous Joint Alignment , 2006 .

[2]  Michael G. Strintzis,et al.  Bilinear Models for 3-D Face and Facial Expression Recognition , 2008, IEEE Transactions on Information Forensics and Security.

[3]  Erik G. Learned-Miller,et al.  Data driven image models through continuous joint alignment , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Alexander M. Bronstein,et al.  Expression-Invariant Representations of Faces , 2007, IEEE Transactions on Image Processing.

[5]  Jochen Lang,et al.  Wavelet Model-based Stereo for Fast, Robust Face Reconstruction , 2011, 2011 Canadian Conference on Computer and Robot Vision.

[6]  Jim Austin,et al.  3D face landmark labelling , 2010, 3DOR '10.

[7]  Ghassan Hamarneh,et al.  A Survey on Shape Correspondence , 2011, Comput. Graph. Forum.

[8]  Ron Kimmel,et al.  Generalized multidimensional scaling: A framework for isometry-invariant partial surface matching , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[9]  Ioannis A. Kakadiaris,et al.  Using Facial Symmetry to Handle Pose Variations in Real-World 3D Face Recognition , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Chang Shu,et al.  Consistent parameterization and statistical analysis of human head scans , 2009, The Visual Computer.

[11]  Trevor F. Cox,et al.  Multidimensional Scaling, Second Edition , 2000 .

[12]  Arman Savran,et al.  Bosphorus Database for 3D Face Analysis , 2008, BIOID.

[13]  Leonidas J. Guibas,et al.  Robust single-view geometry and motion reconstruction , 2009, ACM Trans. Graph..

[14]  Chang Shu,et al.  Automatic Locating of Anthropometric Landmarks on 3D Human Models , 2006, Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06).

[15]  Sami Romdhani,et al.  Face Identification by Fitting a 3D Morphable Model Using Linear Shape and Texture Error Functions , 2002, ECCV.

[16]  Ioannis A. Kakadiaris,et al.  3D Facial Landmark Detection & Face Registration A 3D Facial Landmark Model & 3D Local Shape Descriptors Approach , 2010 .

[17]  M. Iqbal Saripan,et al.  3D facial expression recognition using maximum relevance minimum redundancy geometrical features , 2012, EURASIP Journal on Advances in Signal Processing.

[18]  NairPrathap,et al.  3-D face detection, landmark localization, and registration using a point distribution model , 2009 .

[19]  William T. Freeman,et al.  Understanding belief propagation and its generalizations , 2003 .

[20]  Patrick J. Flynn,et al.  Multiple Nose Region Matching for 3D Face Recognition under Varying Facial Expression , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Shervin Mehryar,et al.  Automatic landmark detection for 3D face image processing , 2010, IEEE Congress on Evolutionary Computation.

[22]  Xing Zhang,et al.  Reshaping 3D facial scans for facial appearance modeling and 3D facial expression analysis , 2011, Face and Gesture 2011.

[23]  Timothy F. Cootes,et al.  Active Appearance Models , 1998, ECCV.

[24]  Ioannis A. Kakadiaris,et al.  Three-Dimensional Face Recognition in the Presence of Facial Expressions: An Annotated Deformable Model Approach , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Federica Marcolin,et al.  3D human face description: landmarks measures and geometrical features , 2012, Image Vis. Comput..

[26]  Wen Gao,et al.  Efficient 3D reconstruction for face recognition , 2005, Pattern Recognit..

[27]  Maurício Pamplona Segundo,et al.  Automatic Face Segmentation and Facial Landmark Detection in Range Images , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[28]  Jun Wang,et al.  A 3D facial expression database for facial behavior research , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[29]  Sridha Sridharan,et al.  Least squares congealing for unsupervised alignment of images , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Chang Shu,et al.  Landmark-free posture invariant human shape correspondence , 2011, The Visual Computer.

[31]  Michael C. Hout,et al.  Multidimensional Scaling , 2003, Encyclopedic Dictionary of Archaeology.

[32]  Y. Gao.  Efficiently comparing face images using a modified Hausdorff distance , 2001 .

[33]  Hanspeter Pfister,et al.  Face transfer with multilinear models , 2005, ACM Trans. Graph..

[34]  Matthew Turk,et al.  A Morphable Model For The Synthesis Of 3D Faces , 1999, SIGGRAPH.

[35]  Chang Shu,et al.  Semi-Automatic Prediction of Landmarks on Human Models in Varying Poses , 2010, 2010 Canadian Conference on Computer and Robot Vision.

[36]  Anil K. Jain,et al.  Automatic feature extraction for multiview 3D face recognition , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[37]  Lijun Yin,et al.  3D facial behaviour analysis and understanding , 2012, Image Vis. Comput..

[38]  Kun Tang,et al.  Automatic landmark annotation and dense correspondence registration for 3D human facial images , 2012, BMC Bioinformatics.

[39]  Thomas Vetter,et al.  Registration of expressions data using a 3D morphable model , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[40]  Anil K. Jain,et al.  Deformation Modeling for Robust 3D Face Matching , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Sami Romdhani,et al.  Efficient, robust and accurate fitting of a 3D morphable model , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[42]  Zoran Popovic,et al.  The space of human body shapes: reconstruction and parameterization from range scans , 2003, ACM Trans. Graph..

[43]  Peter H. Tu,et al.  Semi-supervised facial landmark annotation , 2012, Comput. Vis. Image Underst..

[44]  Andrea Cavallaro,et al.  3-D Face Detection, Landmark Localization, and Registration Using a Point Distribution Model , 2009, IEEE Transactions on Multimedia.

[45]  Alberto Del Bimbo,et al.  3D facial expression recognition using SIFT descriptors of automatically detected keypoints , 2011, The Visual Computer.

[46]  Mongi A. Abidi,et al.  Surface matching by 3D point's fingerprint , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[47]  Jiawei Han,et al.  Data Mining: Concepts and Techniques , 2000 .

[48]  Ioannis A. Kakadiaris,et al.  Automatic 3D Facial Region Retrieval from Multi-pose Facial Datasets , 2009, 3DOR@Eurographics.

[49]  Ron Kimmel,et al.  On Bending Invariant Signatures for Surfaces , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[50]  Robert P.W. Duin,et al.  PRTools3: A Matlab Toolbox for Pattern Recognition , 2000 .

[51]  Hao Li,et al.  Realtime performance-based facial animation , 2011, ACM Trans. Graph..

[52]  M. Pauly,et al.  Example-based facial rigging , 2010, ACM Trans. Graph..

[53]  Marc Pouget,et al.  Smooth surfaces, umbilics, lines of curvatures, foliations, ridges and the medial axis: a concise overview , 2004 .

[54]  Anil K. Jain,et al.  Matching 2.5D face scans to 3D models , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[55]  Thomas Vetter,et al.  Expression invariant 3D face recognition with a Morphable Model , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[56]  Jorge Nocedal,et al.  On the limited memory BFGS method for large scale optimization , 1989, Math. Program..

[57]  Anuj Srivastava,et al.  Statistical Shape Analysis , 2014, Computer Vision, A Reference Guide.