Fitting a 3D Morphable Model to Edges: A Comparison Between Hard and Soft Correspondences

We propose a fully automatic method for fitting a 3D morphable model to single face images in arbitrary pose and lighting. Our approach relies on geometric features (edges and landmarks) and, inspired by the iterated closest point algorithm, is based on computing hard correspondences between model vertices and edge pixels. We demonstrate that this is superior to previous work that uses soft correspondences to form an edge-derived cost surface that is minimised by nonlinear optimisation.

[1]  Andrew W. Fitzgibbon,et al.  Robust Registration of 2D and 3D Point Sets , 2003, BMVC.

[2]  Deva Ramanan,et al.  Face detection, pose estimation, and landmark localization in the wild , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Matthew Turk,et al.  A Morphable Model For The Synthesis Of 3D Faces , 1999, SIGGRAPH.

[4]  Bernhard Egger,et al.  A Monte Carlo Strategy to Integrate Detection and Model-Based Face Analysis , 2013, GCPR.

[5]  Mark Pauly,et al.  Dynamic 3D avatar creation from hand-held video input , 2015, ACM Trans. Graph..

[6]  William A. P. Smith,et al.  Statistical 3D face shape estimation from occluding contours , 2016, Comput. Vis. Image Underst..

[7]  Marc Levoy,et al.  Efficient variants of the ICP algorithm , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[8]  William J. Christmas,et al.  Fitting 3D Morphable Face Models using local features , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[9]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Thomas Vetter,et al.  Face Recognition Based on Fitting a 3D Morphable Model , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Thomas Vetter,et al.  Probabilistic Modeling and Visualization of the Flexibility in Morphable Models , 2009, IMA Conference on the Mathematics of Surfaces.

[12]  Andrew W. Fitzgibbon,et al.  What Shape Are Dolphins? Building 3D Morphable Models from 2D Images , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Terence Sim,et al.  The CMU Pose, Illumination, and Expression Database , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[15]  Vincent Lepetit,et al.  Pose Priors for Simultaneously Solving Alignment and Correspondence , 2008, ECCV.

[16]  Thomas F. Coleman,et al.  An Interior Trust Region Approach for Nonlinear Minimization Subject to Bounds , 1993, SIAM J. Optim..

[17]  Sami Romdhani,et al.  Estimating 3D shape and texture using pixel intensity, edges, specular highlights, texture constraints and a prior , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[18]  Larry S. Davis,et al.  Model-based object pose in 25 lines of code , 1992, International Journal of Computer Vision.

[19]  Philip David,et al.  SoftPOSIT: Simultaneous Pose and Correspondence Determination , 2002, International Journal of Computer Vision.

[20]  Xiangyu Zhu,et al.  High-fidelity Pose and Expression Normalization for face recognition in the wild , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Marwan Mattar,et al.  Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments , 2008 .

[22]  Andrew W. Fitzgibbon,et al.  Reconstructing High Quality Face-Surfaces using Model Based Stereo , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[23]  N. T. Hoai-Phuong,et al.  Optimization under Composite Monotonic Constraints and Constrained Optimization over the Efficient Set , 2006 .

[24]  Thomas Vetter,et al.  3D Reconstruction of Human Faces from Occluding Contours , 2007, MIRAGE.

[25]  Raghu Machiraju,et al.  Model-based 3D face capture with shape-from-silhouettes , 2003, 2003 IEEE International SOI Conference. Proceedings (Cat. No.03CH37443).

[26]  Thomas Vetter,et al.  Optimal landmark detection using shape models and branch and bound , 2011, 2011 International Conference on Computer Vision.

[27]  Oswald Aldrian,et al.  Inverse Rendering of Faces with a 3D Morphable Model , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Melvyn L. Smith,et al.  Facial Geometry Estimation Using Photometric Stereo and Profile Views , 2009, ICB.

[29]  Thabo Beeler,et al.  Real-time high-fidelity facial performance capture , 2015, ACM Trans. Graph..

[30]  Volker Blanz,et al.  Face recognition based on a 3D morphable model , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[31]  Bernhard Schölkopf,et al.  Automatic 3D face reconstruction from single images or video , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[32]  Stephen P. Boyd,et al.  Disciplined Convex Programming , 2006 .

[33]  Ira Kemelmacher-Shlizerman,et al.  Total Moving Face Reconstruction , 2014, ECCV.

[34]  William J. Christmas,et al.  Fitting 3D Morphable Models using Local Features , 2015, arXiv.org.

[35]  Alan Brunton,et al.  Review of statistical shape spaces for 3D data with comparative analysis for human faces , 2012, Comput. Vis. Image Underst..

[36]  Sami Romdhani,et al.  A 3D Face Model for Pose and Illumination Invariant Face Recognition , 2009, 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance.