A Facial Animation Interactive Framework with Facial Expressions, Lip Synchronization and Eye Behavior

In this paper we describe our approach of generating convincing and empathetic facial animation. Our goal is to develop a robust facial animation platform that can be scalable, usable and easily extended, in order to allow integration of research on the area and also the direct incorporation of the characters in interactive applications, such as Embodied Conversational Agents and games. We have developed a framework capable of animating MPEG-4 parametrized faces easily through high-level description of facial actions and behaviors. We also present a case study which integrates computer vision techniques in order to provide interaction between the user and a character, that interacts with different facial actions according to detected events in the application.

[1]  Paula Salgado Lucena Rodrigues.  Um sistema de GeraçãO de Expressões Faciais DINâMICAS EM ANIMAções Faciais 3D com Processamento de fala , 2007 .

[2]  Steve DiPaola,et al.  Facial actions as visual cues for personality: Research Articles , 2006 .

[3]  Mark Steedman,et al.  APML, a Markup Language for Believable Behavior Generation , 2004, Life-like characters.

[4]  Koray Balci.  Xface: MPEG-4 based open source toolkit for 3D Facial Animation , 2004, AVI.

[5]  Justine Cassell,et al.  BEAT: the Behavior Expression Animation Toolkit , 2001, Life-like characters.

[6]  Catherine Pelachaud,et al.  From Greta's mind to her face: modelling the dynamics of affective states in a conversational embodied agent , 2003, Int. J. Hum. Comput. Stud..

[7]  Fernando Pereira,et al.  MPEG-4 facial animation technology: survey, implementation, and results , 1999, IEEE Trans. Circuits Syst. Video Technol..

[8]  Fabio Pianesi,et al.  Synthetic characters as multichannel interfaces , 2005, ICMI '05.

[9]  Norman I. Badler,et al.  Eyes alive , 2002, ACM Trans. Graph..

[10]  G. Sommavilla,et al.  EMOTIONAL TALKING HEAD : THE DEVELOPMENT OF " LUCIA " Piero Cosi , Emanuela , 2007 .

[11]  Steve DiPaola,et al.  Face Modeling and Animation Language for MPEG-4 XMT Framework , 2007, IEEE Transactions on Multimedia.

[12]  Steve DiPaola,et al.  A framework for socially communicative faces for game and interactive learning applications , 2007, Future Play.

[13]  Ken Perlin,et al.  Layered compositing of facial expression , 1997, SIGGRAPH '97.

[14]  Nicolas Courty,et al.  Embodied in a Look: Bridging the Gap between Humans and Avatars , 2003, IVA.

[15]  Algirdas Pakstas,et al.  MPEG-4 Facial Animation: The Standard,Implementation and Applications , 2002 .

[16]  Fabio Pianesi,et al.  Xface open source project and smil-agent scripting language for creating and animating embodied conversational agents , 2007, ACM Multimedia.

[17]  Hung-Hsuan Huang,et al.  The design of a generic framework for integrating ECA components , 2008, AAMAS.

[18]  Soraia Raupp Musse,et al.  Providing expressive gaze to virtual animated characters in interactive applications , 2008, CIE.

[19]  Ken Perlin,et al.  [Computer Graphics]: Three-Dimensional Graphics and Realism , 2022 .

[20]  Lloyd Rutledge.  SMIL 2.0: XML for Web Multimedia , 2001, IEEE Internet Comput..

[21]  Carlo Drioli,et al.  INTERFACE Toolkit: A New Tool for Building IVAs , 2005, IVA.

[22]  Norman I. Badler,et al.  FacEMOTE: qualitative parametric modifiers for facial animations , 2002, SCA '02.

[23]  Abe Mamdani,et al.  Multi-modal embodied agents scripting , 2002, Proceedings. Fourth IEEE International Conference on Multimodal Interfaces.

[24]  Sung Yong Shin,et al.  An example-based approach for facial expression cloning , 2003, SCA '03.

[25]  Ronald Fedkiw,et al.  Automatic determination of facial muscle activations from sparse motion capture marker data , 2005, ACM Trans. Graph..

[26]  Igor S. Pandžić,et al.  Autonomous Speaker Agent , 2004 .

[27]  Parke,et al.  Parameterized Models for Facial Animation , 1982, IEEE Computer Graphics and Applications.

[28]  Stefan Kopp,et al.  The Behavior Markup Language: Recent Developments and Challenges , 2007, IVA.

[29]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[30]  Alice Wang,et al.  Assembling an expressive facial animation system , 2007, Sandbox '07.