Augmenting a 3D morphable model of the human head with high resolution ears

Abstract We present a parts-based 3D Morphable Model (3DMM) of the full human head, with particular emphasis on modelling the complex shape of the ear as a flexible, high-resolution separate part. The 3D ear model part undergoes an iterative process of refinement that employs data augmentation using a 2D image dataset with landmarked ears. Evaluations using several performance metrics validate the training process and the resulting model. We make the new ear model and our reconstructed training dataset publicly available. We merge the trained high-resolution 3DMM of the ear with a publicly-available 3DMM of the full head that has a much lower resolution in the ear regions. The resulting parts-based 3DMM provides more shape variation and more shape detail in the ears, and we demonstrate a higher fidelity overall model fit to raw data.

[1]  Marc Alexa,et al.  As-rigid-as-possible surface modeling , 2007, Symposium on Geometry Processing.

[2]  Zhanfu An,et al.  Deep Transfer Network with 3D Morphable Models for Face Recognition , 2018, 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018).

[3]  Paul J. Besl,et al.  Method for registration of 3-D shapes , 1992, Other Conferences.

[4]  Ryan Schmidt,et al.  meshmixer: an interface for rapid mesh composition , 2010, SIGGRAPH '10.

[5]  Patrick Pérez,et al.  MoFA: Model-Based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[6]  Thomas F. Coleman,et al.  An Interior Trust Region Approach for Nonlinear Minimization Subject to Bounds , 1993, SIAM J. Optim..

[7]  Thomas Vetter,et al.  A morphable model for the synthesis of 3D faces , 1999, SIGGRAPH.

[8]  Simon Lucey,et al.  Real-time avatar animation from a single image , 2011, Face and Gesture 2011.

[9]  Stefanos Zafeiriou,et al.  Large Scale 3D Morphable Models , 2017, International Journal of Computer Vision.

[10]  Peter Peer,et al.  Ear recognition: More than a survey , 2016, Neurocomputing.

[11]  Sami Romdhani,et al.  Optimal Step Nonrigid ICP Algorithms for Surface Registration , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Martin Styner,et al.  Evaluation of 3D Correspondence Methods for Model Building , 2003, IPMI.

[13]  Stefanos Zafeiriou,et al.  The unconstrained ear recognition challenge , 2017, 2017 IEEE International Joint Conference on Biometrics (IJCB).

[14]  Stefanos Zafeiriou,et al.  Deformable Models of Ears in-the-Wild for Alignment and Recognition , 2017, 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017).

[15]  Deva Ramanan,et al.  Face detection, pose estimation, and landmark localization in the wild , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Christian Rössl,et al.  Laplacian surface editing , 2004, SGP '04.

[17]  Andriy Myronenko,et al.  Point Set Registration: Coherent Point Drift , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Craig T. Jin,et al.  Generating a morphable model of ears , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[19]  Christoph Busch,et al.  Ear biometrics: a survey of detection, feature extraction and recognition methods , 2012, IET Biom..

[20]  Christian Theobalt,et al.  Reconstruction of Personalized 3D Face Rigs from Monocular Video , 2016, ACM Trans. Graph..

[21]  J. Gower.  Generalized procrustes analysis , 1975 .

[22]  Arun Ross,et al.  A survey on ear biometrics , 2013, CSUR.

[23]  William Smith,et al.  A 3D Morphable Model of Craniofacial Shape and Texture Variation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[24]  Mark Meyer,et al.  Implicit fairing of irregular meshes using diffusion and curvature flow , 1999, SIGGRAPH.