Sample awareness-based personalized facial expression recognition

The behavior of the current emotion classification model to recognize all test samples using the same method contradicts the cognition of human beings in the real world, who dynamically change the methods they use based on current test samples. To address this contradiction, this study proposes an individualized emotion recognition method based on context awareness. For a given test sample, a classifier that was deemed the most suitable for the current test sample was first selected from a set of candidate classifiers and then used to realize the individualized emotion recognition. The Bayesian learning method was applied to select the optimal classifier and then evaluate each candidate classifier from the global perspective to guarantee the optimality of each candidate classifier. The results of the study validated the effectiveness of the proposed method.

[1]  Xi Zhao,et al.  An efficient multimodal 2D + 3D feature-based approach to automatic facial expression recognition , 2015, Comput. Vis. Image Underst..

[2]  Lijun Yin,et al.  A high-resolution 3D dynamic facial expression database , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[3]  Qiang Ji,et al.  Fusion of visible and thermal images for facial expression recognition , 2014, Frontiers of Computer Science.

[4]  Fabio Roli,et al.  Methods for dynamic classifier selection , 1999, Proceedings 10th International Conference on Image Analysis and Processing.

[5]  Jun Yu,et al.  Facial Expression Recognition by Fusing Gabor and Local Binary Pattern Features , 2017, MMM.

[6]  Luiz Eduardo Soares de Oliveira,et al.  Fusion of feature sets and classifiers for facial expression recognition , 2013, Expert Syst. Appl..

[7]  Yang Yu,et al.  Diversity Regularized Ensemble Pruning , 2012, ECML/PKDD.

[8]  Emmanuel Dellandréa,et al.  Automatic 3D Facial Expression Recognition Based on a Bayesian Belief Net and a Statistical Facial Feature Model , 2010, 2010 20th International Conference on Pattern Recognition.

[9]  George D. C. Cavalcanti,et al.  META-DES: A dynamic ensemble selection framework using meta-learning , 2015, Pattern Recognit..

[10]  Bo Sun,et al.  Facial expression recognition in the wild based on multimodal texture features , 2016, J. Electronic Imaging.

[11]  Ioannis A. Kakadiaris,et al.  3D facial expression recognition: A perspective on promises and challenges , 2011, Face and Gesture 2011.

[12]  Luiz Eduardo Soares de Oliveira,et al.  Dynamic selection of classifiers - A comprehensive review , 2014, Pattern Recognit..

[13]  Liming Chen,et al.  3D facial expression recognition via multiple kernel learning of Multi-Scale Local Normal Patterns , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[14]  Yong Du,et al.  Facial Expression Recognition Based on Deep Evolutional Spatial-Temporal Networks , 2017, IEEE Transactions on Image Processing.

[15]  Liming Chen,et al.  Automatic 3D facial expression recognition using geometric scattering representation , 2015, 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[16]  Ji-Sang Yoo,et al.  Facial Expression Classification Using Deep Convolutional Neural Network , 2017 .

[17]  James C. Bezdek,et al.  Decision templates for multiple classifier fusion: an experimental comparison , 2001, Pattern Recognit..

[18]  Séverine Dubuisson,et al.  Dynamic Pose-Robust Facial Expression Recognition by Multi-View Pairwise Conditional Random Forests , 2016, IEEE Transactions on Affective Computing.

[19]  Stefanos Zafeiriou,et al.  Recognition of 3D facial expression dynamics , 2012, Image Vis. Comput..

[20]  Muhammad Hameed Siddiqi,et al.  Accurate and robust facial expression recognition system using real-time YouTube-based datasets , 2018, Applied Intelligence.

[21]  Bernd Bischl,et al.  On the combination of locally optimal pairwise classifiers , 2007, Eng. Appl. Artif. Intell..

[22]  Yunhong Wang,et al.  Texture and Geometry Scattering Representation-Based Facial Expression Recognition in 2D+3D Videos , 2018, ACM Trans. Multim. Comput. Commun. Appl..

[23]  Luiz Eduardo Soares de Oliveira,et al.  A framework for dynamic classifier selection oriented by the classification problem difficulty , 2018, Pattern Recognit..

[24]  T. Mussweiler,et al.  A tool for thought! When comparative thinking reduces stereotyping effects , 2009 .

[25]  Ioannis A. Kakadiaris,et al.  4D facial expression recognition , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[26]  Junping Du,et al.  Reliable Crowdsourcing and Deep Locality-Preserving Learning for Expression Recognition in the Wild , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Hassen Drira,et al.  4-D Facial Expression Recognition by Learning Geometric Deformations , 2014, IEEE Transactions on Cybernetics.

[28]  Anima Majumder,et al.  Automatic Facial Expression Recognition System Using Deep Network-Based Data Fusion , 2018, IEEE Transactions on Cybernetics.

[29]  Gian Luca Marcialis,et al.  A study on the performances of dynamic classifier selection based on local accuracy estimation , 2005, Pattern Recognit..

[30]  Yoshua Bengio,et al.  Challenges in representation learning: A report on three machine learning contests , 2013, Neural Networks.

[31]  Basilio Sierra,et al.  Dynamic selection of the best base classifier in One versus One , 2015, Knowl. Based Syst..

[32]  Bartosz Krawczyk,et al.  Dynamic classifier selection for one-class classification , 2016, Knowl. Based Syst..

[33]  Qun Dai,et al.  An efficient ordering-based ensemble pruning algorithm via dynamic programming , 2015, Applied Intelligence.

[34]  Fabio Roli,et al.  Dynamic classifier selection based on multiple classifier behaviour , 2001, Pattern Recognit..

[35]  Tin Kam Ho,et al.  The Random Subspace Method for Constructing Decision Forests , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[36]  Ludmila I. Kuncheva,et al.  A Bound on Kappa-Error Diagrams for Analysis of Classifier Ensembles , 2013, IEEE Transactions on Knowledge and Data Engineering.

[37]  Yunlong Liu,et al.  Expression recognition method based on evidence theory and local texture , 2016, Multimedia Tools and Applications.

[38]  Danyang Li,et al.  Ensemble of Deep Neural Networks with Probability-Based Fusion for Facial Expression Recognition , 2017, Cognitive Computation.

[39]  Tong Zhang,et al.  A Deep Neural Network-Driven Feature Learning Method for Multi-view Facial Expression Recognition , 2016, IEEE Transactions on Multimedia.

[40]  Edilson de Aguiar,et al.  Facial expression recognition with Convolutional Neural Networks: Coping with few data and the training sample order , 2017, Pattern Recognit..

[41]  Xinghai Yang,et al.  Research of Emotion Recognition Based on Speech and Facial Expression , 2013 .

[42]  Giorgio Valentini,et al.  Applications of Supervised and Unsupervised Ensemble Methods , 2009, Applications of Supervised and Unsupervised Ensemble Methods.

[43]  Yunhong Wang,et al.  Automatic 4D Facial Expression Recognition Using Dynamic Geometrical Image Network , 2018, 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018).

[44]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[45]  Robert Sabourin,et al.  LoGID: An adaptive framework combining local and global incremental learning for dynamic selection of ensembles of HMMs , 2012, Pattern Recognit..

[46]  André Carlos Ponce de Leon Ferreira de Carvalho,et al.  Multiclass SVM Model Selection Using Particle Swarm Optimization , 2006, 2006 Sixth International Conference on Hybrid Intelligent Systems (HIS'06).

[47]  Robert Sabourin,et al.  From dynamic classifier selection to dynamic ensemble selection , 2008, Pattern Recognit..

[48]  Jingying Chen,et al.  Deep peak-neutral difference feature for facial expression recognition , 2018, Multimedia Tools and Applications.

[49]  Amar Mitiche,et al.  Classifier combination for hand-printed digit recognition , 1993, Proceedings of 2nd International Conference on Document Analysis and Recognition (ICDAR '93).

[50]  Danyang Li,et al.  An ensemble convolutional echo state networks for facial expression recognition , 2015, 2015 International Conference on Affective Computing and Intelligent Interaction (ACII).

[51]  Danyang Li,et al.  MRMR-based ensemble pruning for facial expression recognition , 2017, Multimedia Tools and Applications.

[52]  Rama Chellappa,et al.  FaceNet2ExpNet: Regularizing a Deep Face Recognition Net for Expression Recognition , 2016, 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017).

[53]  Shiguang Shan,et al.  Multi-Channel Pose-Aware Convolution Neural Networks for Multi-View Facial Expression Recognition , 2018, 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018).

[54]  Tony R. Martinez,et al.  An instance level analysis of data complexity , 2014, Machine Learning.

[55]  L. Chen,et al.  Cognitive gravitation model for classification on small noisy data , 2013, Neurocomputing.

[56]  Hong Zhang,et al.  Facial expression recognition via learning deep sparse autoencoders , 2018, Neurocomputing.

[57]  Zhiyong Feng,et al.  Multi-Modal Emotion Recognition Fusing Video and Audio , 2013 .

[58]  Xiaoyi Jiang,et al.  Dynamic classifier ensemble model for customer classification with imbalanced class distribution , 2012, Expert Syst. Appl..

[59]  Liming Chen,et al.  Muscular Movement Model-Based Automatic 3D/4D Facial Expression Recognition , 2015, IEEE Transactions on Multimedia.

[60]  Danyang Li,et al.  RTCRelief-F: an effective clustering and ordering-based ensemble pruning algorithm for facial expression recognition , 2018, Knowledge and Information Systems.

[61]  Ludmila I. Kuncheva,et al.  Switching between selection and fusion in combining classifiers: an experiment , 2002, IEEE Trans. Syst. Man Cybern. Part B.

[62]  Hassen Drira,et al.  3D dynamic expression recognition based on a novel Deformation Vector Field and Random Forest , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[63]  Stéphane Mallat,et al.  Invariant Scattering Convolution Networks , 2012, IEEE transactions on pattern analysis and machine intelligence.

[64]  Yoshua Bengio,et al.  Challenges in Representation Learning: A Report on Three Machine Learning Contests , 2013, ICONIP.

[65]  Liming Chen,et al.  Muscular Movement Model Based Automatic 3D Facial Expression Recognition , 2015, MMM.

[66]  Ludmila I. Kuncheva,et al.  Measures of Diversity in Classifier Ensembles and Their Relationship with the Ensemble Accuracy , 2003, Machine Learning.

[67]  G. Baruchello.  A classification of classics. Gestalt psychology and the tropes of rhetoric , 2015 .