Pose-space animation and transfer of facial details

This paper presents a novel method for real-time animation of highly-detailed facial expressions based on a multi-scale decomposition of facial geometry into large-scale motion and fine-scale details, such as expression wrinkles. Our hybrid animation is tailored to the specific characteristics of large- and fine-scale facial deformations: Large-scale deformations are computed with a fast linear shell model, which is intuitively and accurately controlled through a sparse set of motion-capture markers or user-defined handle points. Fine-scale facial details are incorporated using a novel pose-space deformation technique, which learns the correspondence of sparse measurements of skin strain to wrinkle formation from a small set of example poses. Our hybrid method features real-time animation of highly-detailed faces with realistic wrinkle formation, and allows both large-scale deformations and fine-scale wrinkles to be edited intuitively. Furthermore, our pose-space representation enables the transfer of facial details to novel expressions or other facial models.

[1]  Lance Williams,et al.  Performance-driven facial animation , 1990, SIGGRAPH.

[2]  Tsuneya Kurihara,et al.  Modeling deformable human hands from medical images , 2004, SCA '04.

[3]  Hanspeter Pfister,et al.  Face transfer with multilinear models , 2005, ACM Trans. Graph..

[4]  Hanspeter Pfister,et al.  Face transfer with multilinear models , 2005, SIGGRAPH 2005.

[5]  Wojciech Matusik,et al.  A statistical model for synthesis of detailed facial geometry , 2006, ACM Trans. Graph..

[6]  Matthew Turk,et al.  A Morphable Model For The Synthesis Of 3D Faces , 1999, SIGGRAPH.

[7]  Nadia Magnenat-Thalmann,et al.  Simulation of static and dynamic wrinkles of skin , 1996, Proceedings Computer Animation '96.

[8]  Ronald Fedkiw,et al.  Automatic determination of facial muscle activations from sparse motion capture marker data , 2005, ACM Trans. Graph..

[9]  Yu Zhang,et al.  Realistic and efficient wrinkle simulation using an anatomy-based face model with adaptive refinement , 2005, International 2005 Computer Graphics.

[10]  John P. Lewis,et al.  Real‐Time Weighted Pose‐Space Deformation on the GPU , 2006, Comput. Graph. Forum.

[11]  David Salesin,et al.  Resynthesizing facial animation through 3D model-based tracking , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[12]  Jovan Popović,et al.  Deformation transfer for triangle meshes , 2004, SIGGRAPH 2004.

[13]  Thomas Vetter,et al.  A morphable model for the synthesis of 3D faces , 1999, SIGGRAPH.

[14]  Stefan Van Niekerk,et al.  Playable universal capture , 2006, SIGGRAPH Sketches.

[15]  Wojciech Matusik,et al.  Multi-scale capture of facial geometry and motion , 2007, ACM Trans. Graph..

[16]  Olga Sorkine-Hornung,et al.  Context‐Aware Skeletal Shape Deformation , 2007, Comput. Graph. Forum.

[17]  M. Otaduy,et al.  Multi-scale capture of facial geometry and motion , 2007, ACM Trans. Graph..

[18]  Pieter Peers,et al.  Rapid Acquisition of Specular and Diffuse Normal Maps from Polarized Spherical Gradient Illumination , 2007 .

[19]  Jovan Popovic,et al.  Deformation transfer for triangle meshes , 2004, ACM Trans. Graph..

[20]  Kari Pulli,et al.  Real-time enveloping with rotational regression , 2007, ACM Trans. Graph..

[21]  Li Zhang,et al.  Spacetime faces: high resolution capture for modeling and animation , 2004, SIGGRAPH 2004.

[22]  Frederic I. Parke,et al.  A parametric model for human faces. , 1974 .

[23]  Alex Pentland,et al.  Modeling, tracking and interactive animation of faces and heads//using input from video , 1996, Proceedings Computer Animation '96.

[24]  Nadia Magnenat-Thalmann,et al.  A computational skin model: fold and wrinkle formation , 2002, IEEE Transactions on Information Technology in Biomedicine.

[25]  John P. Lewis,et al.  Pose Space Deformation: A Unified Approach to Shape Interpolation and Skeleton-Driven Deformation , 2000, SIGGRAPH.

[26]  Dimitris N. Metaxas,et al.  Optical Flow Constraints on Deformable Models with Applications to Face Tracking , 2000, International Journal of Computer Vision.

[27]  Dinesh K. Pai,et al.  EigenSkin: real time large deformation character skinning in hardware , 2002, SCA '02.

[28]  Richard K. Beatson,et al.  Reconstruction and representation of 3D objects with radial basis functions , 2001, SIGGRAPH.

[29]  Mark Meyer,et al.  Key Point Subspace Acceleration and soft caching , 2007, SIGGRAPH 2007.

[30]  Daniel Thalmann,et al.  Joint-dependent local deformations for hand animation and object grasping , 1989 .

[31]  Wojciech Matusik,et al.  A statistical model for synthesis of detailed facial geometry , 2006, SIGGRAPH 2006.

[32]  Ronald Fedkiw,et al.  Automatic determination of facial muscle activations from sparse motion capture marker data , 2005, SIGGRAPH '05.

[33]  Raghu Raghavan,et al.  A kinematic-variational model for animating skin with wrinkles , 2005, Comput. Graph..

[34]  Jing Xiao,et al.  Vision-based control of 3D facial animation , 2003, SCA '03.

[35]  Eric Enderton,et al.  A system for efficient rendering of human skin , 2007, SIGGRAPH '07.

[36]  Lance Williams,et al.  Performance-driven facial animation , 1990, SIGGRAPH Courses.

[37]  Markus H. Gross,et al.  Simulating facial surgery using finite element models , 1996, SIGGRAPH.

[38]  Mathieu Desbrun,et al.  Learning controls for blend shape based realistic facial animation , 2003, SIGGRAPH '03.

[39]  Tomaso A. Poggio,et al.  Reanimating Faces in Images and Video , 2003, Comput. Graph. Forum.

[40]  David Salesin,et al.  Synthesizing realistic facial expressions from photographs , 1998, SIGGRAPH.

[41]  Jun-yong Noh,et al.  A Survey of Facial Modeling and Animation Techniques , 2001 .

[42]  Mark Meyer,et al.  Discrete Differential-Geometry Operators for Triangulated 2-Manifolds , 2002, VisMath.

[43]  Jun-yong Noh,et al.  Expression cloning , 2001, SIGGRAPH.

[44]  Eric Enderton,et al.  Efficient Rendering of Human Skin , 2007 .