Automatic Generation of Non-Verbal Facial Expressions from Speech

Speech synchronized facial animation that controls only the movement of the mouth is typically perceived as wooden and unnatural. We propose a method to generate additional facial expressions such as movement of the head, the eyes, and the eyebrows fully automatically from the input speech signal. This is achieved by extracting prosodic parameters such as pitch flow and power spectrum from the speech signal and using them to control facial animation parameters in accordance to results from paralinguistic research.

[1]  M. Cranach,et al.  Human ethology : claims and limits of a new discipline : contributions to the Colloquium , 1982 .

[2]  Editors , 1986, Brain Research Bulletin.

[3]  Brian Wyvill,et al.  Speech and expression: a computer solution to face animation , 1986 .

[4]  J. P. Lewis,et al.  Automated lip-synch and speech synthesis for character animation , 1987, CHI '87.

[5]  Nicole Chovil.  Discourse‐oriented facial displays in conversation , 1991 .

[6]  Daniel Thalmann,et al.  SMILE: A Multilayered Facial Animation System , 1991, Modeling in Computer Graphics.

[7]  Michael M. Cohen,et al.  Modeling Coarticulation in Synthetic Visual Speech , 1993 .

[8]  Daniel Thalmann,et al.  Models and Techniques in Computer Animation , 2014, Computer Animation Series.

[9]  Keith Waters,et al.  Computer facial animation , 1996 .

[10]  Roxane Bertrand,et al.  About the relationship between eyebrow movements and Fo variations , 1996, Proceeding of Fourth International Conference on Spoken Language Processing. ICSLP '96.

[11]  Mark Steedman,et al.  Generating Facial Expressions for Speech , 1996, Cogn. Sci..

[12]  Horace Ho-Shing Ip,et al.  Script-based facial gesture and speech animation using a NURBS based face model , 1996, Comput. Graph..

[13]  Matthew Brand,et al.  Voice puppetry , 1999, SIGGRAPH.

[14]  A. Paeschke,et al.  F0-CONTOURS IN EMOTIONAL SPEECH , 1999 .

[15]  Matthew Stone,et al.  Living Hand to Mouth: Psychological Theories about Speech and Gesture in Interactive Dialogue Systems , 1999 .

[16]  K. Scherer,et al.  THE EFFECTS OF EMOTIONS ON VOICE QUALITY , 1999 .

[17]  Jonas Beskow,et al.  Developing a 3D-agent for the august dialogue system , 1999, AVSP.

[18]  D. Barr.  Trouble in mind: paralinguistic indices of effort and uncertainty in communication , 2001 .

[19]  Hans-Peter Seidel,et al.  Geometry-based Muscle Modeling for Facial Animation , 2001, Graphics Interface.

[20]  Hans-Peter Seidel,et al.  Face to Face: From Real Humans to Realistic Facial Animation , 2001 .

[21]  Björn Granström,et al.  Timing and interaction of visual cues for prominence in audiovisual speech perception , 2001, INTERSPEECH.

[22]  Hans-Peter Seidel,et al.  Speech Synchronization for Physics-Based Facial Animation , 2002, WSCG.