Facial Expression Recognition Based on Local Fourier Coefficients and Facial Fourier Descriptors

The recent boom of mass media communication (such as social media and mobiles) has boosted more applications of automatic facial expression recognition (FER). Thus, human facial expressions have to be encoded and recognized through digital devices. However, this process has to be done under recurrent problems of image illumination changes and partial occlusions. Therefore, in this paper, we propose a fully automated FER system based on Local Fourier Coefficients and Facial Fourier Descriptors. The combined power of appearance and geometric features is used for describing the specific facial regions of eyes-eyebrows, nose and mouth. All based on the attributes of the Fourier Transform and Support Vector Machines. Hence, our proposal overcomes FER problems such as illumination changes, partial occlusion, image rotation, redundancy and dimensionality reduction. Several tests were performed in order to demonstrate the efficiency of our proposal, which were evaluated using three standard databases: CK+, MUG and TFEID. In addition, evaluation results showed that the average recognition rate of each database reaches higher performance than most of the state-of-the-art techniques surveyed in this paper.

[1]  P. Ekman Universals and cultural differences in facial expressions of emotion. , 1972 .

[2]  Xiong Chen,et al.  Facial expression recognition from image sequences using twofold random forest classifier , 2015, Neurocomputing.

[3]  Raphael C.-W. Phan,et al.  Facial Expression Recognition in the Encrypted Domain Based on Local Fisher Discriminant Analysis , 2013, IEEE Transactions on Affective Computing.

[4]  Y. V. Venkatesh,et al.  Facial expression recognition using radial encoding of local Gabor features and classifier synthesis , 2012, Pattern Recognit..

[5]  Timo Ahonen,et al.  Local phase quantization for blur-insensitive image analysis , 2012, Image Vis. Comput..

[6]  Lijiang Chen,et al.  Facial expression recognition considering individual differences in facial structure and texture , 2014, IET Comput. Vis..

[7]  Hélio Pedrini,et al.  Geometrical Features and Active Appearance Model Applied to Facial Expression Recognition , 2016, Int. J. Image Graph..

[8]  Masahide Kaneko,et al.  Facial Expression Recognition Using Facial-component-based Bag of Words and PHOG Descriptors , 2010 .

[9]  Edilson de Aguiar,et al.  Facial expression recognition with Convolutional Neural Networks: Coping with few data and the training sample order , 2017, Pattern Recognit..

[10]  Andrea Cavallaro,et al.  Automatic Analysis of Facial Affect: A Survey of Registration, Representation, and Recognition , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Mahesh M. Goyani,et al.  Multi-Level Haar Wavelet Based Facial Expression Recognition using Logistic Regression , 2017, Int. J. Next Gener. Comput..

[12]  Manasi S. Patwardhan,et al.  Survey on real-time facial expression recognition techniques , 2016, IET Biom..

[13]  Takeo Kanade,et al.  Comprehensive database for facial expression analysis , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[14]  Chiou-Ting Hsu,et al.  Dual Subspace Nonnegative Graph Embedding for Identity-Independent Expression Recognition , 2015, IEEE Transactions on Information Forensics and Security.

[15]  Zhaohui Wu,et al.  Facial expression recognition based on meta probability codes , 2013, Pattern Analysis and Applications.

[16]  Hélio Pedrini,et al.  Effects of cultural characteristics on building an emotion classifier through facial expression analysis , 2015, J. Electronic Imaging.

[17]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[18]  Shaogang Gong,et al.  Facial expression recognition based on Local Binary Patterns: A comprehensive study , 2009, Image Vis. Comput..

[19]  Héctor M. Pérez Meana,et al.  A sub-block-based eigenphases algorithm with optimum sub-block size , 2013, Knowl. Based Syst..

[20]  Sang Hyun Park,et al.  Facial expression recognition based on local region specific features and support vector machines , 2016, Multimedia Tools and Applications.

[21]  Takeo Kanade,et al.  Facial Expression Recognition , 2011, Handbook of Face Recognition.

[22]  Stefanos Zafeiriou,et al.  Incremental Face Alignment in the Wild , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Ze-Nian Li,et al.  Recognition of facial expressions based on salient geometric features and support vector machines , 2016, Multimedia Tools and Applications.

[24]  Alaa Eleyan,et al.  Facial expression recognition based on image pyramid and single-branch decision tree , 2017, Signal, Image and Video Processing.

[25]  Qingxuan Jia,et al.  Weighted Feature Gaussian Kernel SVM for Emotion Recognition , 2016, Comput. Intell. Neurosci..

[26]  Stefanos Zafeiriou,et al.  A Comprehensive Performance Evaluation of Deformable Face Tracking “In-the-Wild” , 2016, International Journal of Computer Vision.

[27]  Maja Pantic,et al.  Facial Expression Recognition , 2009, Encyclopedia of Biometrics.

[28]  Vinod Chandran,et al.  Random Gabor based templates for facial expression recognition in images with facial occlusion , 2014, Neurocomputing.

[29]  Junmo Kim,et al.  Joint Fine-Tuning in Deep Neural Networks for Facial Expression Recognition , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[30]  Masahide Kaneko,et al.  Facial Expression Recognition Based on Facial Region Segmentation and Modal Value Approach , 2014, IEICE Trans. Inf. Syst..

[31]  Masahide Kaneko,et al.  顔部品の「Bag of Words」とPHOG記述子を用いた顔表情認識 , 2010 .

[32]  Anastasios Delopoulos,et al.  The MUG facial expression database , 2010, 11th International Workshop on Image Analysis for Multimedia Interactive Services WIAMIS 10.

[33]  Jake K. Aggarwal,et al.  Facial expression recognition with temporal modeling of shapes , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[34]  Wonjun Hwang,et al.  Face Recognition System Using Multiple Face Model of Hybrid Fourier Feature Under Uncontrolled Illumination Variation , 2011, IEEE Transactions on Image Processing.

[35]  Mohammad Reza Mohammadi,et al.  PCA-based dictionary building for accurate facial expression recognition via sparse representation , 2014, J. Vis. Commun. Image Represent..

[36]  Masahide Kaneko,et al.  Methodical Analysis of Western-Caucasian and East-Asian Basic Facial Expressions of Emotions Based on Specific Facial Regions , 2017 .