3D facial animation from Chinese text

The reality and controllability of the facial animation are enhanced in our work by modeling mouth in a parametric approach described by 7 parameters: superior and anterior bend for upper and lower lip, width of the mouth, weight of radium of lips. These parameters were calculated by tracking only 4 points around the mouth from video, and the cost of computation is low. The coarticulation model of Cohen and Massaro was adopted to generate the natural key frames and the transitional frames. Considering the difference between Chinese and English, an algorithm was introduced to calculate the coefficients of dominance functions. Based on the muscle-based facial model and the parametric mouth model, the 3D head can talk vividly.

[1]  Shigeo Morishima,et al.  Physics-based muscle model for mouth shape control , 1996, Proceedings 5th IEEE International Workshop on Robot and Human Communication. RO-MAN'96 TSUKUBA.

[2]  D. Massaro.  Speech Perception By Ear and Eye: A Paradigm for Psychological Inquiry , 1989 .

[3]  Jörn Ostermann,et al.  User evaluation: Synthetic talking faces for interactive services , 1999, The Visual Computer.

[4]  Akikazu Takeuchi,et al.  Speech Dialogue With Facial Displays: Multimodal Human-Computer Conversation , 1994, ACL.

[5]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[6]  Chun Chen,et al.  Robust Real-Time Face Tracking and Modeling from Video , 2003, ICCSA.

[7]  Raymond D. Kent,et al.  Coarticulation in recent speech production models , 1977 .

[8]  Mark Steedman,et al.  Animated conversation: rule-based generation of facial expression, gesture & spoken intonation for multiple conversational agents , 1994, SIGGRAPH.

[9]  Hans-Peter Seidel,et al.  Speech Synchronization for Physics-Based Facial Animation , 2002, WSCG.

[10]  J. Kelso,et al.  A qualitative dynamic analysis of reiterant speech production: phase portraits, kinematics, and dynamic modeling. , 1985, The Journal of the Acoustical Society of America.

[11]  Keith Waters,et al.  A coordinated muscle model for speech animation , 1995 .

[12]  Michael M. Cohen,et al.  Modeling Coarticulation in Synthetic Visual Speech , 1993 .

[13]  Alex Pentland,et al.  3D Modeling of Human Lip Motion , 1998, ICCV.

[14]  Sherman Wilcox,et al.  Dynamic Modeling of Phonetic Structure , 1992 .

[15]  Wang Zhiming,et al.  A dynamic viseme model for personalizing a talking head , 2002, 6th International Conference on Signal Processing, 2002..

[16]  Alex Pentland,et al.  3D modeling and tracking of human lip motions , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[17]  James M. Rehg,et al.  Computer Vision for Human–Machine Interaction: Visual Sensing of Humans for Active Public Interfaces , 1998 .

[18]  J. Ohya,et al.  Realistic 3D mouth animation using a minimal number of parameters , 1996, Proceedings 5th IEEE International Workshop on Robot and Human Communication. RO-MAN'96 TSUKUBA.

[19]  Speech dialogue with facial displays , 1994, CHI '94.