Factors Influencing the Perception of Realism in Synthetic Facial Expressions

One way to synthesize facial expressions is to change an image to represent the desired emotion and it is useful in entertainment, diagnostic and psychiatric disorders therapy applications. Despite several existing approaches, there is little discussion about factors that contribute or hinder the perception of realism in synthetic facial expressions images. After presenting an approach for facial expressions synthesis through the deformation of facial features, this paper provides an evaluation by 155 volunteers regarding the realism of synthesized images. The proposed facial expression synthesis aims to generate new images using two source images (neutral and expressive face) and changing the expression in a target image (neutral face). The results suggest that assignment of realism depends on the type of image (real or synthetic). However, the synthesis presents images that can be considered realistic, especially for the expression of happiness. Finally, while factors such as color difference between subsequent regions and unnatural-sized facial features contribute to less realism, other factors such as the presence of wrinkles contribute to a greater assignment of realism to images.

[1]  Volker Blanz,et al.  Automated 3D Face Reconstruction from Multiple Images Using Quality Measures , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Fátima de Lourdes dos Santos Nunes,et al.  Generating Facial Emotions for Diagnosis and Training , 2015, 2015 IEEE 28th International Symposium on Computer-Based Medical Systems.

[3]  Xiaofei He,et al.  Coupled Dictionary Learning for the Detail-Enhanced Synthesis of 3-D Facial Expressions , 2016, IEEE Transactions on Cybernetics.

[4]  Xi Zhou,et al.  Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network , 2018, ECCV.

[5]  Klaus Mathiak,et al.  Recognition Profile of Emotions in Natural and Virtual Faces , 2008, PloS one.

[6]  Matan Sela,et al.  Learning Detailed Face Reconstruction from a Single Image , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Bing Zhou,et al.  Facial expression cloning with elastic and muscle models , 2014, J. Vis. Commun. Image Represent..

[8]  P. Ekman,et al.  Constants across cultures in the face and emotion. , 1971, Journal of personality and social psychology.

[9]  Shinya Nakano,et al.  A natural smile synthesis from an artificial smile , 2009, SIGGRAPH '09.

[10]  Hassan Aghaeinia,et al.  Diverse videos synthesis using manifold-based parametric motion model for facial understanding , 2016, IET Image Process..

[11]  Josephine Sullivan,et al.  One millisecond face alignment with an ensemble of regression trees , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Jun'ichiro Seyama,et al.  The Uncanny Valley: Effect of Realism on the Impression of Artificial Human Faces , 2007, PRESENCE: Teleoperators and Virtual Environments.

[13]  Daniel Cohen-Or,et al.  Bringing portraits to life , 2017, ACM Trans. Graph..

[14]  Wei Wei,et al.  A realistic dynamic facial expression transfer method , 2012, Neurocomputing.

[15]  Hao Wang,et al.  Affective interaction based on person-independent facial expression space , 2008, Neurocomputing.

[16]  Xin Li,et al.  Face alive icon , 2007, J. Vis. Lang. Comput..

[17]  Qionghai Dai,et al.  A Data-Driven Approach for Facial Expression Retargeting in Video , 2014, IEEE Transactions on Multimedia.

[18]  R Kikinis,et al.  Deformable modeling of facial tissue for craniofacial surgery simulation. , 1998, Computer aided surgery : official journal of the International Society for Computer Aided Surgery.

[19]  Jesús P. Mena-Chalco,et al.  Banco de dados de faces 3D , 2011 .

[20]  Peter Robinson,et al.  A facial affect mapping engine , 2014, IUI Companion '14.

[21]  Irwin King,et al.  Facial expression synthesis by radial basis function network and image warping , 1996, Proceedings of International Conference on Neural Networks (ICNN'96).

[22]  S. Carey,et al.  Why faces are and are not special: an effect of expertise. , 1986, Journal of experimental psychology. General.

[23]  Coskun Bayrak,et al.  Facial animation framework for web and mobile platforms , 2011, 2011 IEEE 13th International Conference on e-Health Networking, Applications and Services.

[24]  Doris Y. Tsao,et al.  Mechanisms of face perception. , 2008, Annual review of neuroscience.

[25]  Jun-yong Noh,et al.  Expression cloning , 2001, SIGGRAPH 2001.

[26]  LinLin Shen,et al.  Facial expression synthesis with direction field preservation based mesh deformation and lighting fitting based wrinkle mapping , 2018, Multimedia Tools and Applications.

[27]  Tomaso A. Poggio,et al.  Reanimating Faces in Images and Video , 2003, Comput. Graph. Forum.

[28]  Greg Humphreys,et al.  Physically Based Rendering: From Theory to Implementation , 2004 .

[29]  Zicheng Liu,et al.  Expressive expression mapping with ratio images , 2001, SIGGRAPH.

[30]  Hui Chen,et al.  Geometry-Contrastive Generative Adversarial Network for Facial Expression Synthesis , 2018, ArXiv.

[31]  Yiqiang Chen,et al.  SVR-based facial texture driving for realistic expression synthesis , 2004, Third International Conference on Image and Graphics (ICIG'04).

[32]  Gary Bishop,et al.  Measuring the perceived visual realism of images , 2003 .

[33]  John McDonald,et al.  Photo-realistic facial expression synthesis , 2005, Image Vis. Comput..

[34]  Wei Wei,et al.  Facial expression transfer method based on frequency analysis , 2016, Pattern Recognit..

[35]  Anastasios Delopoulos,et al.  The MUG facial expression database , 2010, 11th International Workshop on Image Analysis for Multimedia Interactive Services WIAMIS 10.

[36]  Thomas Vetter,et al.  Expression invariant 3D face recognition with a Morphable Model , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.