Facial expression transfer method based on frequency analysis

We propose a novel expression transfer method based on an analysis of the frequency of multi-expression facial images. We locate the facial features automatically and describe the shape deformations between a neutral expression and non-neutral expressions. The subtle expression changes are important visual clues to distinguish different expressions. These changes are more salient in the frequency domain than in the image domain. We extract the subtle local expression deformations for the source subject, coded in the wavelet decomposition. This information about expressions is transferred to a target subject. The resulting synthesized image preserves both the facial appearance of the target subject and the expression details of the source subject. This method is extended to dynamic expression transfer to allow a more precise interpretation of facial expressions. Experiments on Japanese Female Facial Expression (JAFFE), the extended Cohn-Kanade (CK+) and PIE facial expression databases show the superiority of our method over the state-of-the-art methods. The subtle expression changes are salient in the frequency domain.A local expression deformation transfer method based on frequency analysis.Dynamic expression synthesis for the source subject using expression manifold.A unified framework automatic dynamic expression transfer method.

[1]  Jun-yong Noh,et al.  Expression cloning , 2001, SIGGRAPH.

[2]  Jian Sun,et al.  Blessing of Dimensionality: High-Dimensional Feature and Its Efficient Compression for Face Verification , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[4]  C. Darwin.  The Expression of the Emotions in Man and Animals , .

[5]  J. Cohn,et al.  Deciphering the Enigmatic Face , 2005, Psychological science.

[6]  Ahmed M. Elgammal,et al.  Nonlinear Shape and Appearance Models for Facial Expression Analysis and Synthesis , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[7]  Takeo Kanade,et al.  The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[8]  R Kikinis,et al.  Deformable modeling of facial tissue for craniofacial surgery simulation. , 1998, Computer aided surgery : official journal of the International Society for Computer Aided Surgery.

[9]  Ioannis A. Kakadiaris,et al.  Feature fusion for facial landmark detection , 2014, Pattern Recognit..

[10]  Fernando De la Torre,et al.  Supervised Descent Method and Its Applications to Face Alignment , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Timothy F. Cootes,et al.  Interpreting face images using active appearance models , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[12]  Zicheng Liu,et al.  Expressive expression mapping with ratio images , 2001, SIGGRAPH.

[13]  Keith Waters,et al.  Computer facial animation , 1996 .

[14]  Narendra Ahuja,et al.  Facial expression decomposition , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[15]  Deva Ramanan,et al.  Face detection, pose estimation, and landmark localization in the wild , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  S. Demleitner.  [Communication without words]. , 1997, Pflege aktuell.

[17]  Josephine Sullivan,et al.  One millisecond face alignment with an ensemble of regression trees , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Norman I. Badler,et al.  Animating facial expressions , 1981, SIGGRAPH '81.

[19]  Daijin Kim,et al.  Tensor-Based AAM with Continuous Variation Estimation: Application to Variation-Robust Face Recognition , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Jian Sun,et al.  Face Alignment at 3000 FPS via Regressing Local Binary Features , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Xiaogang Wang,et al.  Deep Learning Face Representation by Joint Identification-Verification , 2014, NIPS.

[22]  Min Chen,et al.  Facial expression recognition in dynamic sequences: An integrated approach , 2014, Pattern Recognit..

[23]  Luiz Velho,et al.  Expression Transfer between Photographs through Multilinear AAM's , 2006, 2006 19th Brazilian Symposium on Computer Graphics and Image Processing.

[24]  Fernando De la Torre,et al.  Bilinear Kernel Reduced Rank Regression for Facial Expression Synthesis , 2010, ECCV.

[25]  A. Mehrabian.  Communication without words , 1968 .

[26]  Erik Hjelmås,et al.  Face Detection: A Survey , 2001, Comput. Vis. Image Underst..

[27]  Takeo Kanade,et al.  Multi-PIE , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[28]  B. Heisele.  Face Detection , 2001 .

[29]  Jing Xiao,et al.  Multi-View AAM Fitting and Construction , 2008, International Journal of Computer Vision.

[30]  Simon Baker,et al.  Active Appearance Models Revisited , 2004, International Journal of Computer Vision.

[31]  Keith Waters,et al.  A muscle model for animation three-dimensional facial expression , 1987, SIGGRAPH.

[32]  Ivo L. Abraham,et al.  Board of Review , 2003 .

[33]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[34]  Demetri Terzopoulos,et al.  Realistic modeling for facial animation , 1995, SIGGRAPH.

[35]  Vinay Bettadapura,et al.  Face Expression Recognition and Analysis: The State of the Art , 2012, ArXiv.

[36]  Keith Waters,et al.  Computer Facial Animation, Second Edition , 1996 .

[37]  Timothy F. Cootes,et al.  Active Appearance Models , 1998, ECCV.

[38]  David Salesin,et al.  Synthesizing realistic facial expressions from photographs , 1998, SIGGRAPH.

[39]  Demetri Terzopoulos,et al.  Multilinear Analysis of Image Ensembles: TensorFaces , 2002, ECCV.

[40]  Jian Sun,et al.  Face Alignment by Explicit Shape Regression , 2012, International Journal of Computer Vision.

[41]  Maja Pantic,et al.  Automatic Analysis of Facial Expressions: The State of the Art , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[42]  Zhigang Deng,et al.  Computer Facial Animation: A Survey , 2008 .

[43]  Qi Tian,et al.  Multiview Face Recognition: From TensorFace to V-TensorFace and K-TensorFace , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[44]  Li Zhuo,et al.  Learning realistic facial expressions from web images , 2013, Pattern Recognit..

[45]  Wei Wei,et al.  A realistic dynamic facial expression transfer method , 2012, Neurocomputing.

[46]  M. Taner Eskil,et al.  Facial expression recognition based on anatomy , 2014, Comput. Vis. Image Underst..

[47]  Jenn-Jier James Lien,et al.  Facial expression recognition system based on rigid and non-rigid motion separation and 3D pose estimation , 2009, Pattern Recognit..

[48]  P. Ekman,et al.  Constants across cultures in the face and emotion. , 1971, Journal of personality and social psychology.

[49]  Markus H. Gross,et al.  Simulating facial surgery using finite element models , 1996, SIGGRAPH.