Statistical modeling of facial aging based on 3D scans

This thesis presents an approach to the modeling of facial aging using and extending the Morphable Model technique. For modeling the face variation across individuals, facial expressions, and physical attributes, we collected 3D face scans of 298 persons. The 3D face scans where acquired with a structured light 3D scanner, which we improved in collaboration with the manufacturer to achieve superior geometry and texture quality. Moreover, we developed an efficient way to measure fine skin structure and reflection properties with the scanner. The collected face scans have been used to build the Basel Face Model, a new publicly available Morphable Model. Using the 3D scans we learn the correlation between physical attributes such as weight, height, and especially age and faces. With the learned correlation, we present a novel way to simultaneously manipulate different attributes and demonstrate the capability to model changes caused by aging. Using the attributes of the face model in conjunction with a skull model developed in the same research group, we present a method to reconstruct faces from skull shapes which considers physical attributes, as the body weight, age etc. The most important aspect of facial aging that can not be simulated with the Morphable Model is the appearance of facial wrinkles. In this work we present a novel approach to synthesize age wrinkles based on statistics. Our wrinkle synthesis consists of two main parts: The learning of a generative model of wrinkle constellations, and the modeling of their visual appearance. For learning the constellations we use kernel density estimation of manually labeled wrinkles to estimate the wrinkle occurrence probability. To learn the visual appearance of wrinkles we use the fine scale skin structure captured with our improved scanning method. Our results show that the combination of the attribute fitting based aging and the wrinkle synthesis, facilitate a simulation of visually convincing progressive aging. The method is without restrictions applicable to any face that can be represented by the Morphable Model.

[1]  Timothy F. Cootes,et al.  The Use of Active Shape Models for Locating Structures in Medical Images , 1993, IPMI.

[2]  Aaron Hertzmann,et al.  Eurographics/ Acm Siggraph Symposium on Computer Animation (2006) Learning a Correlated Model of Identity and Pose-dependent Body Shape Variation for Real-time Synthesis , 2022 .

[3]  K. Torrance,et al.  Theory for off-specular reflection from roughened surfaces , 1967 .

[4]  Pieter Peers,et al.  Rapid Acquisition of Specular and Diffuse Normal Maps from Polarized Spherical Gradient Illumination , 2007 .

[5]  James C. Gee,et al.  Generalized n-D Ck B-Spline Scattered Data Approximation with Confidence Values , 2006, MIAR.

[6]  Thomas Vetter,et al.  Expression invariant 3D face recognition with a Morphable Model , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[7]  Hans-Peter Seidel,et al.  Image-based reconstruction of spatial appearance and geometric detail , 2003, TOGS.

[8]  Thomas Vetter,et al.  Face Recognition Based on Fitting a 3D Morphable Model , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Wojciech Matusik,et al.  Multi-scale capture of facial geometry and motion , 2007, ACM Trans. Graph..

[10]  Henrik Wann Jensen.  Digital face cloning , 2003, SIGGRAPH '03.

[11]  Zoran Popovic,et al.  The space of human body shapes: reconstruction and parameterization from range scans , 2003, ACM Trans. Graph..

[12]  Song-Chun Zhu,et al.  A Multi-Resolution Dynamic Model for Face Aging Simulation , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Wojciech Matusik,et al.  A statistical model for synthesis of detailed facial geometry , 2006, SIGGRAPH 2006.

[14]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[15]  Arman Savran,et al.  Bosphorus Database for 3D Face Analysis , 2008, BIOID.

[16]  Patrick J. Flynn,et al.  An evaluation of multimodal 2D+3D face biometrics , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Thomas Vetter,et al.  Skin Detail Analysis for Face Recognition , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Hans-Peter Seidel,et al.  Creating Face Models from Vague Mental Images , 2006, Comput. Graph. Forum.

[19]  Francesco Santini,et al.  Face Reconstruction from Skull Shapes and Physical Attributes , 2009, DAGM-Symposium.

[20]  J T Richtsmeier,et al.  Brief communication: a sample of pediatric skulls available for study. , 1997, American journal of physical anthropology.

[21]  Nadia Magnenat-Thalmann,et al.  Simulating wrinkles and skin aging , 1999, The Visual Computer.

[22]  Volker Blanz,et al.  Automatische Rekonstruktion der dreidimensionalen Form von Gesichtern aus einem Einzelbild , 2000 .

[23]  Christopher J. Solomon,et al.  Aging the human face - a statistically rigorous approach , 2005 .

[24]  Alexei A. Efros,et al.  Image quilting for texture synthesis and transfer , 2001, SIGGRAPH.

[25]  Steve Marschner,et al.  A practical model for subsurface light transport , 2001, SIGGRAPH.

[26]  Li Yi-bo,et al.  The Wrinkle Generation Method for Facial Reconstruction Based on Extraction of Partition Wrinkle Line Features and Fractal Interpolation , 2007, Fourth International Conference on Image and Graphics (ICIG 2007).

[27]  G. Lemperle,et al.  A classification of facial wrinkles. , 2001, Plastic and reconstructive surgery.

[28]  Christopher J. Solomon,et al.  A person-specific, rigorous aging model of the human face , 2006, Pattern Recognit. Lett..

[29]  Zicheng Liu,et al.  Image-based surface detail transfer , 2004, IEEE Computer Graphics and Applications.

[30]  Sami Romdhani,et al.  Optimal Step Nonrigid ICP Algorithms for Surface Registration , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[31]  C N Stephan,et al.  Building faces from dry skulls: are they recognized above chance rates? , 2001, Journal of forensic sciences.

[32]  Nikolai I. Chernov,et al.  Least Squares Fitting of Circles , 2005, Journal of Mathematical Imaging and Vision.

[33]  Irfan A. Essa,et al.  Real-time Photo-Realistic Physically Based Rendering of Fine Scale Human Skin Structure , 2001, Rendering Techniques.

[34]  Peter Schröder,et al.  Discrete conformal mappings via circle patterns , 2005, TOGS.

[35]  Brett Allen,et al.  Exploring the space of human body shapes: data-driven synthesis under anthropometric control , 2004 .

[36]  P. Ekman,et al.  The Repertoire of Nonverbal Behavior: Categories, Origins, Usage, and Coding , 1969 .

[37]  Yu Zhang,et al.  Face modeling and editing with statistical local feature control models , 2008 .

[38]  Cassidy J. Curtis,et al.  An art-directed wrinkle system for CG character clothing and skin , 2007, Graph. Model..

[39]  Hanspeter Pfister,et al.  Face transfer with multilinear models , 2005, SIGGRAPH 2005.

[40]  M. Gross,et al.  Analysis of human faces using a measurement-based skin reflectance model , 2006, ACM Trans. Graph..

[41]  Kristina Scherbaum.  Learning Based Prediction and 3 D-Visualisation of Children ’ s Facial Growth , 2005 .

[42]  Jan Kautz,et al.  Visio-lization: generating novel facial images , 2009, ACM Trans. Graph..

[43]  Yun Fu,et al.  Age Synthesis and Estimation via Faces: A Survey , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Sami Romdhani,et al.  Face Identification by Fitting a 3D Morphable Model Using Linear Shape and Texture Error Functions , 2002, ECCV.

[45]  C. M. Elliott,et al.  Computation of geometric partial differential equations and mean curvature flow , 2005, Acta Numerica.

[46]  Diego F. Nehab,et al.  Efficiently combining positions and normals for precise 3D geometry , 2005, SIGGRAPH 2005.

[47]  Reinhard Knothe,et al.  A global-to-local model for the representation of human faces , 2009 .

[48]  Hans-Peter Seidel,et al.  Geometry-based Muscle Modeling for Facial Animation , 2001, Graphics Interface.

[49]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[50]  Thomas Vetter,et al.  Curvature guided surface registration using level sets , 2007 .

[51]  Thomas Vetter,et al.  A statistical deformation prior for non-rigid image and shape registration , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[52]  Edward H. Adelson,et al.  The Design and Use of Steerable Filters , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[53]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2004 .

[54]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[55]  Ming Li,et al.  Modeling Expressive Wrinkles of Face For Animation , 2007, Fourth International Conference on Image and Graphics (ICIG 2007).

[56]  Thomas Vetter,et al.  A morphable model for the synthesis of 3D faces , 1999, SIGGRAPH.

[57]  Jun Wang,et al.  A 3D facial expression database for facial behavior research , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[58]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[59]  Hans-Peter Seidel,et al.  Prediction of Individual Non‐Linear Aging Trajectories of Faces , 2007, Comput. Graph. Forum.

[60]  M. Lüthi.  A machine learning approach to statistical shape models with applications to medical image analysis , 2010 .

[61]  Cristina Conde,et al.  An Automatic 2D, 2.5D & 3D Score-Based Fusion Face Verification System , 2007, 2006 International Workshop on Computer Architecture for Machine Perception and Sensing.

[62]  Maria Petrou,et al.  Colour photometric stereo: simultaneous reconstruction of local gradient and colour of rough textured surfaces , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[63]  Hiroshi Ando,et al.  Extraction and manipulation of wrinkles and spots for facial image synthesis , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[64]  Geraldine S. Cheok,et al.  Recovering Spheres from 3D Point Data , 2006, 35th IEEE Applied Imagery and Pattern Recognition Workshop (AIPR'06).

[65]  Takaaki Kuratate,et al.  A simple method for modeling wrinkles on human skin , 2002, 10th Pacific Conference on Computer Graphics and Applications, 2002. Proceedings..

[66]  Henrik Wann Jensen,et al.  Light diffusion in multi-layered translucent materials , 2005, ACM Trans. Graph..

[67]  Roger Y. Tsai,et al.  A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses , 1987, IEEE J. Robotics Autom..

[68]  Thomas Vetter,et al.  Weight, Sex, and Facial Expressions: On the Manipulation of Attributes in Generative 3D Face Models , 2009, ISVC.

[69]  Zicheng Liu,et al.  Expressive expression mapping with ratio images , 2001, SIGGRAPH.

[70]  James Davis,et al.  Facial type, expression, and viseme generation , 2007, SIGGRAPH '07.

[71]  Markus H. Gross,et al.  Pose-space animation and transfer of facial details , 2008, SCA '08.

[72]  Yanfeng Sun,et al.  D Face Animation Based on Morphable Model ? , 2004 .