Expression Classification and Intensity Estimation by Expression Manifold Synthesis

Facial expression and its dynamic property play an important role in interpreting and conveying emotions. Recently facial expression analysis has been an active topic in both psychology and computer vision. Most previous investigations have focused on the recognition of static images with intense expressions. Different from the previous work, we present an expression synthesis method for both expression classification and intensity estimation. By means of synthesising expression manifolds from neutral faces, the dynamic variations in facial expression can be modelled and analysed. Eigentransformation is utilised on both shape and expression details in generating novel expressions. Expression classification is performed on the expanded training sets with synthesised expression landmarks, and the intensity can be estimated with synthesised expression manifolds. Comprehensive experimental results conducted on the extended Cohn-Kanade database are reported.

[1]  Shaogang Gong,et al.  Facial expression recognition based on Local Binary Patterns: A comprehensive study , 2009, Image Vis. Comput..

[2]  Xiaogang Wang,et al.  Face sketch synthesis and recognition , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[3]  Gene H. Golub,et al.  Singular value decomposition and least squares solutions , 1970, Milestones in Matrix Computation.

[4]  Oksam Chae,et al.  Facial expression recognition using Local Directional Pattern (LDP) , 2010, 2010 IEEE International Conference on Image Processing.

[5]  Mohammad Reza Mohammadi,et al.  Intensity Estimation of Spontaneous Facial Action Units Based on Their Sparsity Properties , 2016, IEEE Transactions on Cybernetics.

[6]  Beat Fasel,et al.  Automati Fa ial Expression Analysis: A Survey , 1999 .

[7]  P. Ekman.  Facial expressions of emotion: an old controversy and new findings. , 1992, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[8]  Md. Zia Uddin,et al.  An enhanced independent component-based human facial expression recognition from video , 2009, IEEE Transactions on Consumer Electronics.

[9]  Shihong Lao,et al.  Logit-RankBoost with pruning for face recognition , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[10]  MengChu Zhou,et al.  Image Ratio Features for Facial Expression Recognition Application , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[11]  Yi-Ping Hung,et al.  Intensity Rank Estimation of Facial Expressions Based on a Single Image , 2013, 2013 IEEE International Conference on Systems, Man, and Cybernetics.

[12]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[13]  Y. V. Venkatesh,et al.  Facial expression recognition using radial encoding of local Gabor features and classifier synthesis , 2012, Pattern Recognit..

[14]  Maja Pantic,et al.  Facial action recognition for facial expression analysis from static face images , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[15]  K. Mardia,et al.  Statistical Shape Analysis , 1998 .

[16]  P. Ekman,et al.  Facial action coding system: a technique for the measurement of facial movement , 1978 .

[17]  John McDonald,et al.  Automatic estimation of the dynamics of facial expression using a three-level model of intensity , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[18]  Aurobinda Routray,et al.  Automatic facial expression recognition using features of salient facial patches , 2015, IEEE Transactions on Affective Computing.

[19]  Maja Pantic,et al.  Dynamics of facial expression: recognition of facial actions and their temporal segments from face profile image sequences , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[20]  Maja Pantic,et al.  Automatic Analysis of Facial Expressions: The State of the Art , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Takeo Kanade,et al.  Recognizing Action Units for Facial Expression Analysis , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  WenAn Tan,et al.  Gabor feature-based face recognition using supervised locality preserving projection , 2007, Signal Process..

[23]  Takeo Kanade,et al.  The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[24]  Anil K. Jain,et al.  Handbook of Face Recognition, 2nd Edition , 2011 .

[25]  Qingshan Liu,et al.  RankBoost with l1 regularization for facial expression recognition and intensity estimation , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[26]  J. Cohn,et al.  Deciphering the Enigmatic Face , 2005, Psychological science.

[27]  Takeo Kanade,et al.  Comprehensive database for facial expression analysis , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[28]  G. Baudat,et al.  Generalized Discriminant Analysis Using a Kernel Approach , 2000, Neural Computation.