Generating Visemes for Realistic Animation

Efficient, realistic face animation is still a challenge. A system is proposed that yields realistic visemes for speech animation. This paper discusses the extraction of these visemes. It starts from real 3D face dynamics, observed at frame rate for thousands of points on the faces of speaking actors. A generic 3D mesh is fitted to the data throughout 3D time sequences. This is based on a combination of morphing and tracking techniques. The actual animation is the subject of a companion paper.

[1]  Thomas S. Huang,et al.  Explanation-based facial motion tracking using a piecewise Bezier volume deformation model , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[2]  D. Massaro,et al.  Perceiving Talking Faces , 1995 .

[3]  Jun-yong Noh,et al.  Expression cloning , 2001, SIGGRAPH 2001.

[4]  Gérard Bailly,et al.  MOTHER: a new generation of talking heads providing a flexible articulatory control for video-realistic speech animation , 2000, INTERSPEECH.

[5]  A. Montgomery,et al.  Physical characteristics of the lips underlying vowel lipreading performance. , 1983, The Journal of the Acoustical Society of America.

[6]  David Banks,et al.  Interactive shape metamorphosis , 1995, I3D '95.

[7]  Tony Ezzat,et al.  Visual Speech Synthesis by Morphing Visemes , 2000, International Journal of Computer Vision.

[8]  Demetri Terzopoulos,et al.  Physically-based facial modelling, analysis, and animation , 1990, Comput. Animat. Virtual Worlds.

[9]  E. Owens,et al.  Visemes observed by hearing-impaired and normal-hearing adult viewers. , 1985, Journal of speech and hearing research.

[10]  Eric Vatikiotis-Bateson,et al.  The moving face during speech communication , 1998 .

[11]  John R. Wright,et al.  Synthesis of Speaker Facial Movement to Match Selected Speech Sequences , 1994 .

[12]  Christoph Bregler,et al.  Video Rewrite: Driving Visual Speech with Audio , 1997, SIGGRAPH.

[13]  Jun-yong Noh,et al.  Expression cloning , 2001, SIGGRAPH.

[14]  Matthew Brand,et al.  Voice puppetry , 1999, SIGGRAPH.

[15]  Thaddeus Beier,et al.  Feature-based image metamorphosis , 1998 .

[16]  David Salesin,et al.  Synthesizing realistic facial expressions from photographs , 1998, SIGGRAPH.

[17]  Stephen M. Omohundro,et al.  Nonlinear Image Interpolation using Manifold Learning , 1994, NIPS.