Combining appearance and geometric features for facial expression recognition

This paper introduces a method for facial expression recognition combining appearance and geometric facial features. The proposed framework consistently combines multiple facial representations at both global and local levels. First, covariance descriptors are computed to represent regional features combining various feature information with a low dimensionality. Then geometric features are detected to provide a general facial movement description of the facial expression. These appearance and geometric features are combined to form a vector representation of the facial expression. The proposed method is tested on the CK+ database and shows encouraging performance.

[1]  Zhihong Zeng,et al.  A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Oliver G. B. Garrod,et al.  Facial expressions of emotion are not culturally universal , 2012, Proceedings of the National Academy of Sciences.

[3]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[4]  A. Mehrabian.  Communication without words , 1968 .

[5]  Junzhou Huang,et al.  Pose-Free Facial Landmark Fitting via Optimized Part Mixtures and Cascaded Deformable Shape Model , 2013, 2013 IEEE International Conference on Computer Vision.

[6]  Hui Yu,et al.  Perception-driven facial expression synthesis , 2012, Comput. Graph..

[7]  Philippe G Schyns,et al.  Reply to Sauter and Eisner: Differences outweigh commonalities in the communication of emotions across human cultures , 2013, Proceedings of the National Academy of Sciences.

[8]  Maja Pantic,et al.  A Dynamic Appearance Descriptor Approach to Facial Actions Temporal Modeling , 2014, IEEE Transactions on Cybernetics.

[9]  Kenji Mase,et al.  Recognition of Facial Expression from Optical Flow , 1991 .

[10]  Aggelos K. Katsaggelos,et al.  Automatic facial expression recognition using facial animation parameters and multistream HMMs , 2006, IEEE Transactions on Information Forensics and Security.

[11]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[12]  Matti Pietikäinen,et al.  A comparative study of texture measures with classification based on featured distributions , 1996, Pattern Recognit..

[13]  Wen Gao,et al.  Learned local Gabor patterns for face representation and recognition , 2009, Signal Process..

[14]  Maja Pantic,et al.  Local Evidence Aggregation for Regression-Based Facial Point Detection , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Maja Pantic,et al.  Dynamics of facial expression: recognition of facial actions and their temporal segments from face profile image sequences , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[16]  Xiaoyang Tan,et al.  Fusing Gabor and LBP Feature Sets for Kernel-Based Face Recognition , 2007, AMFG.

[17]  Maja Pantic,et al.  Decision Level Fusion of Domain Specific Regions for Facial Action Recognition , 2014, 2014 22nd International Conference on Pattern Recognition.

[18]  Gamini Dissanayake,et al.  Optical flow based analyses to detect emotion from human facial image data , 2010, Expert Syst. Appl..

[19]  Zhihong Zeng,et al.  A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions , 2009, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Qiang Ji,et al.  Facial Action Unit Recognition by Exploiting Their Dynamic and Semantic Relationships , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  W. Förstner,et al.  A Metric for Covariance Matrices , 2003 .

[22]  Maja Pantic,et al.  Fully Automatic Recognition of the Temporal Phases of Facial Actions , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[23]  Luiz Eduardo Soares de Oliveira,et al.  Facial expression recognition using ensemble of classifiers , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[24]  Fatih Murat Porikli,et al.  Region Covariance: A Fast Descriptor for Detection and Classification , 2006, ECCV.

[25]  Michael J. Lyons,et al.  Automatic Classification of Single Facial Images , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  Takeo Kanade,et al.  The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[27]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[28]  P. Ekman,et al.  Constants across cultures in the face and emotion. , 1971, Journal of personality and social psychology.

[29]  Honghai Liu,et al.  Regression-Based Facial Expression Optimization , 2014, IEEE Transactions on Human-Machine Systems.

[30]  Stefanos Zafeiriou,et al.  Local normal binary patterns for 3D facial action unit detection , 2012, 2012 19th IEEE International Conference on Image Processing.

[31]  Shaogang Gong,et al.  Facial expression recognition based on Local Binary Patterns: A comprehensive study , 2009, Image Vis. Comput..

[32]  Siu-Yeung Cho,et al.  A local experts organization model with application to face emotion recognition , 2009, Expert Syst. Appl..