Unsupervised emotion recognition algorithm based on improved deep belief model in combination with probabilistic linear discriminant analysis

Since the initial weight matrix between the last hidden layer of the network and the classification layer is usually generated randomly, the weight matrix does not have the discrimination ability to accurately classify the facial expression recognition, which results in that the features obtained by the weight matrix mapping cannot be guaranteed to be suitable for classification tasks. To solve this problem, a novel linear discriminant deep belief network is proposed in this paper. Firstly, the traditional linear discriminant analysis method is improved, and a new type of inter-class dispersion matrix is designed to solve the rank limitation problem in the traditional Linear Discriminant Analysis Method (LDA). Then, the weight matrix between the last hidden layer and the classification layer of the deep belief network is initialized by the improved linear discriminant analysis method, so that the network is more suitable for the classification task. In the experiments, our proposed deep network obtains respectively the recognition rates of 78.26% and 94.48% on the JAFFE database and the Extended Cohn-Kanade database. In addition, using our proposed algorithm for aggregating linear discriminant analysis into a deep belief network, we were able to produce an accuracy of 81.03% on the challenge test set.

[1]  Cüneyt Güzelis,et al.  A new facial expression recognition based on curvelet transform and online sequential extreme learning machine initialized with spherical clustering , 2014, Neural Computing and Applications.

[2]  Antonios Danelakis,et al.  A spatio-temporal wavelet-based descriptor for dynamic 3D facial expression retrieval and recognition , 2016, The Visual Computer.

[3]  Xin Geng,et al.  A multi-task model for simultaneous face identification and facial expression recognition , 2016, Neurocomputing.

[4]  Bo Sun,et al.  Facial expression recognition in the wild based on multimodal texture features , 2016, J. Electronic Imaging.

[5]  Haibin Yan,et al.  Transfer subspace learning for cross-dataset facial expression recognition , 2016, Neurocomputing.

[6]  Soo-Young Lee,et al.  Hierarchical committee of deep convolutional neural networks for robust facial expression recognition , 2016, Journal on Multimodal User Interfaces.

[7]  Rajendran Parthiban,et al.  Joint facial expression recognition and intensity estimation based on weighted votes of image sequences , 2017, Pattern Recognit. Lett..

[8]  Rajendran Parthiban,et al.  Spatiotemporal feature extraction for facial expression recognition , 2016, IET Image Process..

[9]  Tong Zhang,et al.  A Deep Neural Network-Driven Feature Learning Method for Multi-view Facial Expression Recognition , 2016, IEEE Transactions on Multimedia.

[10]  Nicu Sebe,et al.  Learning Personalized Models for Facial Expression Analysis and Gesture Recognition , 2016, IEEE Transactions on Multimedia.

[11]  A. Young,et al.  Differences in holistic processing do not explain cultural differences in the recognition of facial expression , 2017, Quarterly journal of experimental psychology.

[12]  Yibin Li,et al.  Facial Expression Recognition with Fusion Features Extracted from Salient Facial Areas , 2017, Sensors.

[13]  Honggang Zhang,et al.  Joint Patch and Multi-label Learning for Facial Action Unit and Holistic Expression Recognition , 2016, IEEE Transactions on Image Processing.

[14]  Zheru Chi,et al.  Facial Expression Recognition in Video with Multiple Feature Fusion , 2018, IEEE Transactions on Affective Computing.

[15]  Ioannis Pitas,et al.  Facial Expression Recognition in Image Sequences Using Geometric Deformation Features and Support Vector Machines , 2007, IEEE Transactions on Image Processing.

[16]  Edilson de Aguiar,et al.  Facial expression recognition with Convolutional Neural Networks: Coping with few data and the training sample order , 2017, Pattern Recognit..

[17]  Hyunseung Choo,et al.  A Novel Maximum Entropy Markov Model for Human Facial Expression Recognition , 2016, PloS one.

[18]  Y. V. Venkatesh,et al.  Facial expression recognition using radial encoding of local Gabor features and classifier synthesis , 2012, Pattern Recognit..

[19]  Kaiqi Huang,et al.  Multi angle optimal pattern-based deep learning for automatic facial expression recognition , 2017, Pattern Recognit. Lett..

[20]  Haibo Li,et al.  Sparse Kernel Reduced-Rank Regression for Bimodal Emotion Recognition From Facial Expression and Speech , 2016, IEEE Transactions on Multimedia.

[21]  P. Niedenthal,et al.  Fashioning the Face: Sensorimotor Simulation Contributes to Facial Expression Recognition , 2016, Trends in Cognitive Sciences.

[22]  Janez Brest,et al.  Multi-Objective Differential Evolution for feature selection in Facial Expression Recognition systems , 2017, Expert Syst. Appl..

[23]  Jian Sun,et al.  Multimodal 2D+3D Facial Expression Recognition With Deep Fusion Convolutional Neural Network , 2017, IEEE Transactions on Multimedia.

[24]  Sunil Kumar,et al.  Extraction of informative regions of a face for facial expression recognition , 2016, IET Comput. Vis..

[25]  Qiuqi Ruan,et al.  Facial expression recognition using sparse local Fisher discriminant analysis , 2016, Neurocomputing.

[26]  Giancarlo Fortino,et al.  A facial expression recognition system using robust face features from depth videos and deep learning , 2017, Comput. Electr. Eng..

[27]  Matti Pietikäinen,et al.  Dynamic Facial Expression Recognition With Atlas Construction and Sparse Representation , 2016, IEEE Transactions on Image Processing.

[28]  Yong Man Ro,et al.  Partial Matching of Facial Expression Sequence Using Over-Complete Transition Dictionary for Emotion Recognition , 2016, IEEE Transactions on Affective Computing.

[29]  Danyang Li,et al.  Ensemble of Deep Neural Networks with Probability-Based Fusion for Facial Expression Recognition , 2017, Cognitive Computation.

[30]  Shiguang Shan,et al.  Learning Expressionlets via Universal Manifold Model for Dynamic Facial Expression Recognition , 2015, IEEE Transactions on Image Processing.

[31]  Tom Manly,et al.  Facial expression recognition across the adult life span , 2003, Neuropsychologia.

[32]  H. Przuntek,et al.  Facial expression recognition in people with medicated and unmedicated Parkinson’s disease , 2003, Neuropsychologia.

[33]  Hung-Hsu Tsai,et al.  Facial expression recognition using a combination of multiple facial features and support vector machine , 2018, Soft Comput..

[34]  Shiguang Shan,et al.  Learning Expressionlets on Spatio-temporal Manifold for Dynamic Facial Expression Recognition , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[35]  Liming Chen,et al.  Muscular Movement Model-Based Automatic 3D/4D Facial Expression Recognition , 2015, IEEE Transactions on Multimedia.

[36]  A. Young,et al.  Understanding the recognition of facial identity and facial expression , 2005, Nature Reviews Neuroscience.