Emotional Intensity-based Facial Expression Cloning for Low Polygonal Applications

People instinctively recognize facial expression as a key to nonverbal communication, which has been confirmed by many different research projects. A change in intensity or magnitude of even one specific facial expression can cause different interpretations. A systematic method for generating facial expression syntheses, while mimicking realistic facial expressions and intensities, is a strong need in various applications. Although manually produced animation is typically of high quality, the process is slow and costly-therefore, often unrealistic for low polygonal applications. In this paper, we present a simple and efficient emotional-intensity-based expression cloning process for low-polygonal-based applications, by generating a customized face, as well as by cloning facial expressions. We define intensity mappings to measure expression intensity. Once a source expression is determined by a set of suitable parameter values in a customized 3D face and its embedded muscles, expressions for any target face(s) can be easily cloned by using the same set of parameters. Through experimental study, including facial expression simulation and cloning with intensity mapping, our research reconfirms traditional psychological findings. Additionally, we discuss the method's overall usability and how it allows us to automatically adjust a customized face with embedded facial muscles while mimicking the user's facial configuration, expression, and intensity.

[1]  Marco Fratarcangeli,et al.  Facial motion cloning with radial basis functions in MPEG-4 FBA , 2007, Graph. Model..

[2]  Dong Liang,et al.  A facial expression recognition system based on supervised locally linear embedding , 2005, Pattern Recognit. Lett..

[3]  Thomas Vetter,et al.  A morphable model for the synthesis of 3D faces , 1999, SIGGRAPH.

[4]  Adrien Bartoli,et al.  Pools of AAMs: Towards Automatically Fitting any Face Image , 2008, BMVC.

[5]  Keith Waters,et al.  Computer facial animation , 1996 .

[6]  Hyeong-Seok Ko,et al.  Performance-driven muscle-based facial animation , 2001, Comput. Animat. Virtual Worlds.

[7]  Nicu Sebe,et al.  Facial expression recognition from video sequences: temporal and static modeling , 2003, Comput. Vis. Image Underst..

[8]  Christoph Bregler,et al.  Analysis, Synthesis, and Retargeting of Facial Expressions , 2004 .

[9]  P. Ekman.  Emotions Revealed: Recognizing Faces and Feelings to Improve Communication and Emotional Life , 2003 .

[10]  Lawrence K. Saul,et al.  Think Globally, Fit Locally: Unsupervised Learning of Low Dimensional Manifold , 2003, J. Mach. Learn. Res..

[11]  Ana Paiva,et al.  Pathematic agents: rapid development of believable emotional agents in intelligent virtual environments , 1999, AGENTS '99.

[12]  Igor S. Pandzic,et al.  MPEG-4 Facial Animation , 2002 .

[13]  Baining Guo,et al.  Geometry-driven photorealistic facial expression synthesis , 2003, IEEE Transactions on Visualization and Computer Graphics.

[14]  Demetrios J Halazonetis,et al.  The spontaneous smile in dynamic motion. , 2005, American journal of orthodontics and dentofacial orthopedics : official publication of the American Association of Orthodontists, its constituent societies, and the American Board of Orthodontics.

[15]  Shaogang Gong,et al.  Appearance Manifold of Facial Expression , 2005, ICCV-HCI.

[16]  M. Yachida,et al.  Facial expression recognition and its degree estimation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[17]  C. Izard.  The face of emotion , 1971 .

[18]  Paul Ekman,et al.  Emotions inside out. 130 Years after Darwin's "The Expression of the Emotions in Man and Animal". , 2003, Annals of the New York Academy of Sciences.

[19]  Algirdas Pakstas,et al.  MPEG-4 Facial Animation: The Standard,Implementation and Applications , 2002 .

[20]  M. Trivedi,et al.  Manifold analysis of facial gestures for face recognition , 2003, WBMA '03.

[21]  Parke,et al.  Parameterized Models for Facial Animation , 1982, IEEE Computer Graphics and Applications.

[22]  Nira Dyn,et al.  Image Warping by Radial Basis Functions: Application to Facial Expressions , 1994, CVGIP Graph. Model. Image Process..

[23]  Sergios Theodoridis,et al.  Pattern Recognition, Third Edition , 2006 .

[24]  P. Ekman,et al.  Pan-Cultural Elements in Facial Displays of Emotion , 1969, Science.

[25]  Barry-John Theobald,et al.  Real-time expression cloning using appearance models , 2007, ICMI '07.

[26]  P. Ekman,et al.  Facial action coding system: a technique for the measurement of facial movement , 1978 .

[27]  Catherine Pelachaud,et al.  Multimodal embodied agents , 2002, The Knowledge Engineering Review.

[28]  Ioannis Pitas,et al.  Facial Expression Recognition in Image Sequences Using Geometric Deformation Features and Support Vector Machines , 2007, IEEE Transactions on Image Processing.

[29]  Alex Pentland,et al.  Facial expression recognition using a dynamic model and motion energy , 1995, Proceedings of IEEE International Conference on Computer Vision.

[30]  P. Ekman,et al.  Unmasking the face : a guide to recognizing emotions from facial clues , 1975 .

[31]  Timothy F. Cootes,et al.  Active Appearance Models , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  Mohammed Yeasin,et al.  Recognition of facial expressions and measurement of levels of interest from video , 2006, IEEE Transactions on Multimedia.

[33]  Gwen Littlewort,et al.  Dynamics of Facial Expression Extracted Automatically from Video , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[34]  Thaddeus Beier,et al.  Feature-based image metamorphosis , 1998 .

[35]  Vatcharaporn Esichaikul,et al.  Fuzzy-C-Mean Determines the Principle Component Pairs to Estimate the Degree of Emotion from Facial Expressions , 2005, FSKD.

[36]  Stephen Lehmann,et al.  The Science of Happiness: How Our Brains Make Us Happy-and What We Can Do to Get Happier , 2006 .

[37]  Shigeo Morishima,et al.  Emotion space for analysis and synthesis of facial expression , 1993, Proceedings of 1993 2nd IEEE International Workshop on Robot and Human Communication.

[38]  Shigeo Morishima,et al.  Emotion modeling in speech production using emotion space , 1996, Proceedings 5th IEEE International Workshop on Robot and Human Communication. RO-MAN'96 TSUKUBA.

[39]  Maja Pantic,et al.  Expert system for automatic analysis of facial expressions , 2000, Image Vis. Comput..

[40]  P. Ekman.  Facial expression and emotion. , 1993, The American psychologist.

[41]  John Yen,et al.  Emotionally expressive agents , 1999, Proceedings Computer Animation 1999.