Statistical Modeling of Craniofacial Shape and Texture

We present a fully-automatic statistical 3D shape modeling approach and apply it to a large dataset of 3D images, the Headspace dataset , thus generating the first public shape-and-texture 3D morphable model (3DMM) of the full human head. Our approach is the first to employ a template that adapts to the dataset subject before dense morphing. This is fully automatic and achieved using 2D facial landmarking, projection to 3D shape, and mesh editing. In dense template morphing, we improve on the well-known Coherent Point Drift algorithm, by incorporating iterative data-sampling and alignment. Our evaluations demonstrate that our method has better performance in correspondence accuracy and modeling ability when compared with other competing algorithms. We propose a texture map refinement scheme to build high quality texture maps and texture model. We present several applications that include the first clinical use of craniofacial 3DMMs in the assessment of different types of surgical intervention applied to a craniosynostosis patient group.

[1]  Nick Pears,et al.  A 2D Morphable Model of Craniofacial Profile and Its Application to Craniosynostosis , 2017, MIUA.

[2]  Qiang Ji,et al.  Facial Landmark Detection: A Literature Survey , 2018, International Journal of Computer Vision.

[3]  Anuj Srivastava,et al.  Statistical Shape Analysis , 2014, Computer Vision, A Reference Guide.

[4]  Alessandro Verri,et al.  Fitting 3D morphable models using implicit representations , 2007, J. Virtual Real. Broadcast..

[5]  Christian Theobalt,et al.  Reconstruction of Personalized 3D Face Rigs from Monocular Video , 2016, ACM Trans. Graph..

[6]  Marc Alexa,et al.  As-rigid-as-possible surface modeling , 2007, Symposium on Geometry Processing.

[7]  Thomas Vetter,et al.  Face Recognition Based on Fitting a 3D Morphable Model , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Jim Austin,et al.  A Machine-Learning Approach to Keypoint Detection and Landmarking on 3D Meshes , 2012, International Journal of Computer Vision.

[9]  Fred L. Bookstein,et al.  Principal Warps: Thin-Plate Splines and the Decomposition of Deformations , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Flavio Prieto,et al.  Fully automatic expression-invariant face correspondence , 2013, Machine Vision and Applications.

[11]  Stefanos Zafeiriou,et al.  Deformable Models of Ears in-the-Wild for Alignment and Recognition , 2017, 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017).

[12]  Deva Ramanan,et al.  Face detection, pose estimation, and landmark localization in the wild , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Kathleen M. Robinette,et al.  Principles of Fit to Optimize Helmet Sizing , 2006 .

[14]  Timo Bolkart,et al.  Statistical Analysis of 3D Faces in Motion , 2013, 2013 International Conference on 3D Vision.

[15]  Jochen Lang,et al.  Wavelet Model-based Stereo for Fast, Robust Face Reconstruction , 2011, 2011 Canadian Conference on Computer and Robot Vision.

[16]  Frank B. ter Haar,et al.  3D Face Model Fitting for Recognition , 2008, ECCV.

[17]  Zhanfu An,et al.  Deep Transfer Network with 3D Morphable Models for Face Recognition , 2018, 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018).

[18]  Lijun Yin,et al.  A high-resolution 3D dynamic facial expression database , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[19]  Yiying Tong,et al.  FaceWarehouse: A 3D Facial Expression Database for Visual Computing , 2014, IEEE Transactions on Visualization and Computer Graphics.

[20]  D. Kendall SHAPE MANIFOLDS, PROCRUSTEAN METRICS, AND COMPLEX PROJECTIVE SPACES , 1984 .

[21]  Bernhard Egger,et al.  Morphable Face Models - An Open Framework , 2017, 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018).

[22]  Hanspeter Pfister,et al.  Face transfer with multilinear models , 2005, ACM Trans. Graph..

[23]  Wojciech Matusik,et al.  A statistical model for synthesis of detailed facial geometry , 2006, ACM Trans. Graph..

[24]  Matthew Turk,et al.  A Morphable Model For The Synthesis Of 3D Faces , 1999, SIGGRAPH.

[25]  Fei Yang,et al.  Expression flow for 3D-aware face component transfer , 2011, ACM Trans. Graph..

[26]  Laurens van der Maaten,et al.  Accelerating t-SNE using tree-based algorithms , 2014, J. Mach. Learn. Res..

[27]  D'arcy W. Thompson,et al.  On Growth and Form , 1917, Nature.

[28]  Simon Lucey,et al.  Real-time avatar animation from a single image , 2011, Face and Gesture 2011.

[29]  Derek Bradley,et al.  Rigid stabilization of facial expressions , 2014, ACM Trans. Graph..

[30]  Luc Van Gool,et al.  Optimal Regions for Linear Model-Based 3D Face Reconstruction , 2010, ACCV.

[31]  Stefanos Zafeiriou,et al.  Large Scale 3D Morphable Models , 2017, International Journal of Computer Vision.

[32]  Xiaoming Liu,et al.  Nonlinear 3D Face Morphable Model , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[33]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[34]  Thomas Vetter,et al.  Probabilistic Joint Face-Skull Modelling for Facial Reconstruction , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[35]  Sami Romdhani,et al.  A 3D Face Model for Pose and Illumination Invariant Face Recognition , 2009, 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance.

[36]  Thomas Gerig,et al.  Gaussian Process Morphable Models , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[38]  Stefanos Zafeiriou,et al.  A 3D Morphable Model Learnt from 10,000 Faces , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Fei Yang,et al.  Facial expression editing in video using a temporally-smooth factorization , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[40]  Ivana Kolingerová,et al.  Hairstyles modeling for police identikits , 2015, SCCG.

[41]  Sami Romdhani,et al.  Optimal Step Nonrigid ICP Algorithms for Surface Registration , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[42]  Martin Styner,et al.  Evaluation of 3D Correspondence Methods for Model Building , 2003, IPMI.

[43]  Gérard G. Medioni,et al.  Object modelling by registration of multiple range images , 1992, Image Vis. Comput..

[44]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[45]  Zhou Wang,et al.  Multiscale structural similarity for image quality assessment , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[46]  William Smith,et al.  Symmetric Shape Morphing for 3D Face and Head Modelling , 2018, 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018).

[47]  Thomas Vetter,et al.  Modeling the Remaining Flexibility of Partially Fixed Statistical Shape Models , 2008 .

[48]  Timothy F. Cootes,et al.  Combining point distribution models with shape models based on finite element analysis , 1994, Image Vis. Comput..

[49]  Zhou Wang,et al.  Information Content Weighting for Perceptual Image Quality Assessment , 2011, IEEE Transactions on Image Processing.

[50]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[51]  Timothy F. Cootes,et al.  Active Appearance Models , 1998, ECCV.

[52]  William Smith,et al.  A 3D Morphable Model of Craniofacial Shape and Texture Variation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[53]  Timo Bolkart,et al.  A Groupwise Multilinear Correspondence Optimization for 3D Faces , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[54]  Andriy Myronenko,et al.  Point Set Registration: Coherent Point Drift , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[55]  Paul J. Besl,et al.  Method for registration of 3-D shapes , 1992, Other Conferences.