Real-time MPEG-4 facial animation with 3D scalable meshes

Abstract In this paper the complete framework of a software package for real-time animation of 3D facial models according to the MPEG-4 specifications is described. The focus is mainly on the algorithmic solution that has been adopted for scaling the polygon mesh to a level of complexity suited to the specific hw/sw profile of the platform that hosts the decoder. The basic functionality of this system is its ability to deform virtual faces in response to MPEG-4 Facial Animation Parameters (FAP) according to “animation rules” whose computation is based on very simple “semantic” information previously associated with the model. The software architecture is modular and each module, implementing a specific functionality, can be easily plugged in or unplugged. In this paper, we begin with a general presentation of the system and then we present a brief description of its main modules: the face model, the mesh animation, the mesh calibration, the mesh simplification and the FAP decoding. While for the other modules a deeper analysis is left to referenced bibliography previously published by the authors, a comprehensive description is given here of the mesh simplification block together with experimental evidence of its efficiency. The algorithm we have adopted for mesh simplification is based on iterative edge contraction with the introduction of specific geometry constraints due to the particular nature of the 3D mesh representing an MPEG-4 animated face. In this paper it will be shown how rather complex 3D facial models, optionally textured, can be simplified in real-time in order to adapt the rendering performance to the hardware capabilities, whilst still maintaining compliance with the MPEG-4 specifications on facial animation.

[1]  E. Catmull,et al.  Recursively generated B-spline surfaces on arbitrary topological meshes , 1978 .

[2]  Demetri Terzopoulos,et al.  Physically-based facial modelling, analysis, and animation , 1990, Comput. Animat. Virtual Worlds.

[3]  Parke,et al.  Parameterized Models for Facial Animation , 1982, IEEE Computer Graphics and Applications.

[4]  Ruey-Song Huang,et al.  3-D facial model estimation from single front-view facial image , 2002, IEEE Trans. Circuits Syst. Video Technol..

[5]  Frederic I. Parke,et al.  A parametric model for human faces. , 1974 .

[6]  A. Murat Tekalp,et al.  Face and 2-D mesh animation in MPEG-4 , 2000, Signal Process. Image Commun..

[7]  Fabio Lavagetto,et al.  The facial animation engine: toward a high-level interface for the design of MPEG-4 compliant animated faces , 1999, IEEE Trans. Circuits Syst. Video Technol..

[8]  Chuck Hansen,et al.  Eurographics '97 , 1998, COMG.

[9]  Hugues Hoppe,et al.  New quadric metric for simplifying meshes with appearance attributes , 1999, Proceedings Visualization '99 (Cat. No.99CB37067).

[10]  R. Schaback.  Creating Surfaces from Scattered Data Using Radial Basis Functions , 1995 .

[11]  A. Guéziec.  Surface simplification inside a tolerance volume , 1996 .

[12]  Rémi Ronfard,et al.  Full‐range approximation of triangulated polyhedra. , 1996, Comput. Graph. Forum.

[13]  J. Clark,et al.  Recursively generated B-spline surfaces on arbitrary topological meshes , 1978 .

[14]  Daniel Thalmann,et al.  Simulation of Facial Muscle Actions Based on Rational Free Form Deformations , 1992, Comput. Graph. Forum.

[15]  Norman I. Badler,et al.  Animating facial expressions , 1981, SIGGRAPH '81.

[16]  Jovan Popovic,et al.  Progressive simplicial complexes , 1997, SIGGRAPH.

[17]  Jörn Ostermann,et al.  3D talking head customization by adapting a generic model to one uncalibrated picture , 2001, ISCAS 2001. The 2001 IEEE International Symposium on Circuits and Systems (Cat. No.01CH37196).

[18]  Michael Garland,et al.  Surface simplification using quadric error metrics , 1997, SIGGRAPH.

[19]  Keith Waters,et al.  A muscle model for animation three-dimensional facial expression , 1987, SIGGRAPH.

[20]  Hugues Hoppe,et al.  Progressive meshes , 1996, SIGGRAPH.

[21]  Fabio Lavagetto,et al.  MPEG-4: Audio/video and synthetic graphics/audio for mixed media , 1997, Signal Process. Image Commun..

[22]  Zongmin Wu,et al.  Compactly supported positive definite radial functions , 1995 .

[23]  Fabio Lavagetto,et al.  A SOLUTION FOR MODEL-INDEPENDENT ANIMATION OF MPEG-4 FACES , 2001 .