Recognition of Facial Expressions under Varying Conditions Using Dual-Feature Fusion

Facial expression recognition plays an important role in communicating the emotions and intentions of human beings. Facial expression recognition in uncontrolled environment is more difficult as compared to that in controlled environment due to change in occlusion, illumination, and noise. In this paper, we present a new framework for effective facial expression recognition from real-time facial images. Unlike other methods which spend much time by dividing the image into blocks or whole face image, our method extracts the discriminative feature from salient face regions and then combine with texture and orientation features for better representation. Furthermore, we reduce the data dimension by selecting the highly discriminative features. The proposed framework is capable of providing high recognition accuracy rate even in the presence of occlusions, illumination, and noise. To show the robustness of the proposed framework, we used three publicly available challenging datasets. The experimental results show that the performance of the proposed framework is better than existing techniques, which indicate the considerable potential of combining geometric features with appearance-based features.

[1]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[2]  N. Ahmed,et al.  Discrete Cosine Transform , 1996 .

[3]  Tetsuya Takiguchi,et al.  Multithreading cascade of SURF for facial expression recognition , 2016, EURASIP J. Image Video Process..

[4]  Erfan Zangeneh,et al.  Facial expression recognition by using differential geometric features , 2018 .

[5]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[6]  Dejey Dharma,et al.  Novel directional patterns and a Generalized Supervised Dimension Reduction System (GSDRS) for facial emotion recognition , 2018, Multimedia Tools and Applications.

[7]  Uros Mlakar,et al.  Automated facial expression recognition based on histograms of oriented gradient feature vector differences , 2015, Signal Image Video Process..

[8]  Chen-Chiung Hsieh,et al.  Effective semantic features for facial expressions recognition using SVM , 2015, Multimedia Tools and Applications.

[9]  Josephine Sullivan,et al.  One millisecond face alignment with an ensemble of regression trees , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Pingzhi Fan,et al.  Performance evaluation of score level fusion in multimodal biometric systems , 2010, Pattern Recognit..

[11]  Zheng Zhang,et al.  Pose-invariant face recognition using facial landmarks and Weber local descriptor , 2015, Knowl. Based Syst..

[12]  Maja Pantic,et al.  Web-based database for facial expression analysis , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[13]  Hung-Hsu Tsai,et al.  Facial expression recognition using a combination of multiple facial features and support vector machine , 2018, Soft Comput..

[14]  Jörgen Ahlberg,et al.  Fast facial expression recognition using local binary features and shallow neural networks , 2018, The Visual Computer.

[15]  Ayyaz Hussain,et al.  Illumination invariant facial expression recognition using selected merged binary patterns for real world images , 2018 .

[16]  Shiguang Shan,et al.  AU-inspired Deep Networks for Facial Expression Feature Learning , 2015, Neurocomputing.

[17]  Ze-Nian Li,et al.  Recognition of facial expressions based on salient geometric features and support vector machines , 2016, Multimedia Tools and Applications.

[18]  Tamás D. Gedeon,et al.  Static facial expression analysis in tough conditions: Data, evaluation protocol and benchmark , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[19]  Jake K. Aggarwal,et al.  Facial expression recognition with temporal modeling of shapes , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[20]  Ayyaz Hussain,et al.  Reliable facial expression recognition for multi-scale images using weber local binary image based cosine transform features , 2017, Multimedia Tools and Applications.

[21]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[22]  Adnan Acan,et al.  ECG classification using three-level fusion of different feature descriptors , 2018, Expert Syst. Appl..

[23]  Sang Hyun Park,et al.  Facial expression recognition based on local region specific features and support vector machines , 2016, Multimedia Tools and Applications.

[24]  Ioannis Pitas,et al.  Facial Expression Recognition in Image Sequences Using Geometric Deformation Features and Support Vector Machines , 2007, IEEE Transactions on Image Processing.

[25]  Tetsuya Takiguchi,et al.  Rotation-reversal invariant HOG cascade for facial expression recognition , 2017, Signal Image Video Process..

[26]  Yueh-Min Huang,et al.  Facial expression recognition using optimized active regions , 2018, Human-centric Computing and Information Sciences.

[27]  Vinod Chandran,et al.  Facial expression recognition experiments with data from television broadcasts and the World Wide Web , 2014, Image Vis. Comput..

[28]  Sung Wook Baik,et al.  Facial appearance and texture feature-based robust facial expression recognition framework for sentiment knowledge discovery , 2017, Cluster Computing.

[29]  Takeo Kanade,et al.  The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[30]  Cláudio Rosito Jung,et al.  Facial expression recognition using temporal POEM features , 2017, Pattern Recognit. Lett..

[31]  Haitao Zhao,et al.  An efficient unconstrained facial expression recognition algorithm based on Stack Binarized Auto-encoders and Binarized Neural Networks , 2017, Neurocomputing.

[32]  Mohammed Yeasin,et al.  Recognition of facial expressions and measurement of levels of interest from video , 2006, IEEE Transactions on Multimedia.

[33]  N. Altman An Introduction to Kernel and Nearest-Neighbor Nonparametric Regression , 1992 .

[34]  Xiaohui Yuan,et al.  Conditional convolution neural network enhanced random forest for facial expression recognition , 2018, Pattern Recognit..

[35]  Haitao Zhao,et al.  A visual attention based ROI detection method for facial expression recognition , 2018, Neurocomputing.

[36]  Gary W. Small,et al.  Five days at outdoor education camp without screens improves preteen skills with nonverbal emotion cues , 2014, Comput. Hum. Behav..

[37]  Biao Yang,et al.  Facial expression recognition based on dual-feature fusion and improved random forest classifier , 2017, Multimedia Tools and Applications.

[38]  Mathias Kölsch,et al.  Analysis of rotational robustness of hand detection with a Viola-Jones detector , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[39]  Matti Pietikäinen,et al.  IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2009, TPAMI-2008-09-0620 1 WLD: A Robust Local Image Descriptor , 2022 .

[40]  Qingshan Liu,et al.  Spatio-temporal convolutional features with nested LSTM for facial expression recognition , 2018, Neurocomputing.