Facial expression recognition algorithm based on parameter adaptive initialization of CNN and LSTM

In view of the high dimensionality, nonrigidity, multiscale variation and the influence of illumination and angle on facial expressions, it is quite difficult to obtain facial expression images or videos using computers and analyze facial morphology and changes to accurately obtain the emotional changes of the subjects. Existing facial expression recognition algorithms have the following problems in the application process: the existing shallow feature extraction model has lost a lot of effective feature information and low recognition accuracy. The facial expression recognition method based on deep learning has problems such as overfitting, gradient explosion and parameter initialization. Therefore, this paper develops a facial expression recognition algorithm based on the deep learning method. An adaptive model parameter initialization based on the multilayer maxout network linear activation function is proposed to initialize the convolutional neural network (CNN) and the long–short-term memory network (LSTM) method. It can effectively overcome the gradient disappearance and gradient explosion problems in the deep learning model training process. At the same time, the convolutional neural network with an LSTM memory unit is used to extract the related information from the image sequence, and the facial expression judgment is based on a single-frame image and historical-related information. However, the top-level structure of the CNN model is a fully connected feedforward neural network, which undertakes the task of expression classification. Therefore, the SVM classification method replaces the top-level classifier to further improve the expression classification accuracy. Experiments show that the facial expression recognition method proposed in this paper not only accurately identifies various expressions but also has good adaptive ability. This is because the method achieves the adaptive initialization of the parameters of the deep learning model construction process and also analyzes the relevance of the expression database expression, thereby improving the accuracy of expression recognition.

[1]  Edilson de Aguiar,et al.  Facial expression recognition with Convolutional Neural Networks: Coping with few data and the training sample order , 2017, Pattern Recognit..

[2]  M. Hariharan,et al.  Recognition of Facial Expression Using Haar Wavelet Transform , 2013 .

[3]  Tetsuya Takiguchi,et al.  Facial expression recognition with multithreaded cascade of rotation-invariant HOG , 2015, 2015 International Conference on Affective Computing and Intelligent Interaction (ACII).

[4]  Matti Pietikäinen,et al.  Facial expression recognition from near-infrared videos , 2011, Image Vis. Comput..

[5]  Joanne Pransky The Pransky interview - Martin Haegele, Head of Department Robotics and Assistive Systems, Fraunhofer IPA , 2018, Ind. Robot.

[6]  Ping Liu,et al.  Facial Expression Recognition via a Boosted Deep Belief Network , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Edilson de Aguiar,et al.  A Facial Expression Recognition System Using Convolutional Networks , 2015, 2015 28th SIBGRAPI Conference on Graphics, Patterns and Images.

[8]  Vasiliki Vouloutsi,et al.  Emotions and self-regulation , 2018, Oxford Scholarship Online.

[9]  Ting Zhang,et al.  Facial Expression Recognition Based on Deep Learning: A Survey , 2017 .

[10]  Shuicheng Yan,et al.  Peak-Piloted Deep Network for Facial Expression Recognition , 2016, ECCV.

[11]  S. Demleitner [Communication without words]. , 1997, Pflege aktuell.

[12]  Oliveira-SantosThiago,et al.  Facial expression recognition with Convolutional Neural Networks , 2017 .

[13]  Junmo Kim,et al.  Joint Fine-Tuning in Deep Neural Networks for Facial Expression Recognition , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[14]  Yong Du,et al.  Facial Expression Recognition Based on Deep Evolutional Spatial-Temporal Networks , 2017, IEEE Transactions on Image Processing.

[15]  Jun Wang,et al.  A 3D facial expression database for facial behavior research , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[16]  George Kurian,et al.  Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation , 2016, ArXiv.

[17]  Yongzhao Zhan,et al.  Pose-robust feature learning for facial expression recognition , 2015, Frontiers of Computer Science.

[18]  Mohamed Daoudi,et al.  A Novel Space-Time Representation on the Positive Semidefinite Cone for Facial Expression Recognition , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[19]  Shiguang Shan,et al.  Deeply Learning Deformable Facial Action Parts Model for Dynamic Expression Analysis , 2014, ACCV.

[20]  Marc Pollefeys,et al.  Fast 3D Reconstruction of Faces with Glasses , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[22]  Hasan Demirel,et al.  Improved SIFT matching for pose robust facial expression recognition , 2011, Face and Gesture 2011.

[23]  Maja Pantic,et al.  Discriminative Shared Gaussian Processes for Multiview and View-Invariant Facial Expression Recognition , 2015, IEEE Transactions on Image Processing.

[24]  Shaogang Gong,et al.  Facial expression recognition based on Local Binary Patterns: A comprehensive study , 2009, Image Vis. Comput..

[25]  Yoshua Bengio,et al.  Challenges in representation learning: A report on three machine learning contests , 2013, Neural Networks.

[26]  Lianwen Jin,et al.  A New Facial Expression Recognition Method Based on Local Gabor Filter Bank and PCA plus LDA , 2006 .

[27]  Yiying Tong,et al.  FaceWarehouse: A 3D Facial Expression Database for Visual Computing , 2014, IEEE Transactions on Visualization and Computer Graphics.

[28]  ErhanDumitru,et al.  Challenges in representation learning , 2015 .

[29]  Shiguang Shan,et al.  Learning Expressionlets on Spatio-temporal Manifold for Dynamic Facial Expression Recognition , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Rama Chellappa,et al.  FaceNet2ExpNet: Regularizing a Deep Face Recognition Net for Expression Recognition , 2016, 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017).

[31]  Cha Zhang,et al.  Image based Static Facial Expression Recognition with Multiple Deep Network Learning , 2015, ICMI.