Towards 3D Communications: Real Time Emotion Driven 3D Virtual Facial Animation

In this paper, we provide a novel algorithm for building real time 3D virtual facial animation of avatar by drawing facial expression features of real characters. This algorithm first (automatically) labels the feature points of user's facial imagines drew from camera. Then sequentially estimates the emotional state of user's face expressions by Hidden Markov Model. At last construct the avatar's facial animation according to these emotional states. We explore the complex expression space by outputting synthetic facial animations blended by 6 standard facial expressions.

[1]  P. Ekman.  Expression and the Nature of Emotion , 1984 .

[2]  Tsuhan Chen,et al.  Video-based face recognition using adaptive hidden Markov models , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[3]  Monson H. Hayes,et al.  Face Recognition Using An Embedded HMM , 1999 .

[4]  PanticMaja,et al.  A Survey of Affect Recognition Methods , 2009 .

[5]  Tim Cootes,et al.  An Introduction to Active Shape Models , 2000 .

[6]  Zhihong Zeng,et al.  A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions , 2009, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Radoslaw Niewiadomski,et al.  Expressions of Empathy in ECAs , 2008, IVA.

[8]  N. Magnenat-Thalmann,et al.  Automatic face cloning and animation using real-time facial feature tracking and speech acquisition , 2001, IEEE Signal Process. Mag..

[9]  Ruigang Yang,et al.  Photo Realistic 3D Cartoon Face Modeling Based on Active Shape Model , 2009, Trans. Edutainment.

[10]  Salil Deena,et al.  Speech-Driven Facial Animation Using a Shared Gaussian Process Latent Variable Model , 2009, ISVC.

[11]  Tapio Takala,et al.  An Approach for Creating and Blending Synthetic Facial Expressions of Emotion , 2009, IVA.

[12]  John P. Lewis,et al.  Expressive Facial Animation Synthesis by Learning Speech Coarticulation and Expression Spaces , 2006, IEEE Transactions on Visualization and Computer Graphics.

[13]  Radoslaw Niewiadomski,et al.  Multimodal Complex Emotions: Gesture Expressivity and Blended Facial Expressions , 2006, Int. J. Humanoid Robotics.

[14]  Rafael A. Calvo,et al.  Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications , 2010, IEEE Transactions on Affective Computing.

[15]  Xiaogang Jin,et al.  Data-driven facial expression synthesis via Laplacian deformation , 2010, Multimedia Tools and Applications.