Facial expression recognition based on a mlp neural network using constructive training algorithm

This paper presents a constructive training algorithm for Multi Layer Perceptron (MLP) applied to facial expression recognition applications. The developed algorithm is composed by a single hidden-layer using a given number of neurons and a small number of training patterns. When the Mean Square Error MSE on the Training Data TD is not reduced to a predefined value, the number of hidden neurons grows during the neural network learning. Input patterns are trained incrementally until all patterns of TD are presented and learned. The proposed MLP constructive training algorithm seeks to find synthesis parameters as the number of patterns corresponding for subsets of each class to be presented initially in the training step, the initial number of hidden neurons, the number of iterations during the training step as well as the MSE predefined value. The suggested algorithm is developed in order to classify a facial expression. For the feature extraction stage, a biological vision-based facial description, namely Perceived Facial Images PFI has been applied to extract features from human face images. To evaluate, the proposed approach is tested on three databases which are the GEMEP FERA 2011, the Cohn-Kanade facial expression and the facial expression recognition FER-2013 databases. Compared to the fixed MLP architecture and the literature review, experimental results clearly demonstrate the efficiency of the proposed algorithm.

[1]  Nathan Intrator,et al.  Complex cells and Object Recognition , 1997 .

[2]  M. Taner Eskil,et al.  Facial expression recognition based on anatomy , 2014, Comput. Vis. Image Underst..

[3]  Junfei Qiao,et al.  A node pruning algorithm for feedforward neural network based on neural complexity , 2010, 2010 International Conference on Intelligent Control and Information Processing.

[4]  Shiqing Zhang,et al.  Facial expression recognition using local binary patterns and discriminant kernel locally linear embedding , 2012, EURASIP Journal on Advances in Signal Processing.

[5]  Radu Tudor Ionescu,et al.  Local Learning to Improve Bag of Visual Words Model for Facial Expression Recognition , 2013 .

[6]  Sina Mohseni,et al.  Facial expression recognition using DCT features and neural network based decision tree , 2013, Proceedings ELMAR-2013.

[7]  Veikko Surakka,et al.  Feature-based detection of facial landmarks from neutral and expressive facial images , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Plamen P. Angelov,et al.  PANFIS: A Novel Incremental Learning Machine , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[9]  Gwen Littlewort,et al.  Learning spatiotemporal features by using independent component analysis with application to facial expression recognition , 2012, Neurocomputing.

[10]  Giovanna Castellano,et al.  An iterative pruning algorithm for feedforward neural networks , 1997, IEEE Trans. Neural Networks.

[11]  Yuan Luo,et al.  Facial expression recognition based on fusion feature of PCA and LBP with SVM , 2013 .

[12]  Luiz Eduardo Soares de Oliveira,et al.  Fusion of feature sets and classifiers for facial expression recognition , 2013, Expert Syst. Appl..

[13]  Qingshan Liu,et al.  Exploring facial expressions with compositional features , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[14]  Qiuqi Ruan,et al.  Facial expression recognition based on tensor local linear discriminant analysis , 2012, 2012 IEEE 11th International Conference on Signal Processing.

[15]  M. Chtourou,et al.  MLP neural network based face recognition system using constructive training algorithm , 2012, 2012 International Conference on Multimedia Computing and Systems.

[16]  Liming Chen,et al.  Face recognition under varying facial expression based on Perceived Facial Images and local feature matching , 2012, 2012 International Conference on Information Technology and e-Services.

[17]  S. S. Sridhar,et al.  Improved Adaptive Learning Algorithm for Constructive Neural Networks , 2011 .

[18]  Khashayar Khorasani,et al.  Facial expression recognition using constructive feedforward neural networks , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[19]  Radu Tudor Ionescu,et al.  Objectness to improve the bag of visual words model , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[20]  Qiuqi Ruan,et al.  Tensor rank one differential graph preserving analysis for facial expression recognition , 2012, Image Vis. Comput..

[21]  Gwen Littlewort,et al.  The motion in emotion — A CERT based approach to the FERA emotion challenge , 2011, Face and Gesture 2011.

[22]  George A. Papakostas,et al.  Lattice Computing Extension of the FAM Neural Classifier for Human Facial Expression Recognition , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[23]  Jake K. Aggarwal,et al.  Spontaneous facial expression recognition: A robust metric learning approach , 2014, Pattern Recognit..

[24]  Bir Bhanu,et al.  Facial expression recognition using emotion avatar image , 2011, Face and Gesture 2011.

[25]  Takeo Kanade,et al.  Comprehensive database for facial expression analysis , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[26]  Rainer Lienhart,et al.  An extended set of Haar-like features for rapid object detection , 2002, Proceedings. International Conference on Image Processing.

[27]  Hubert Konik,et al.  Framework for reliable, real-time facial expression recognition for low resolution images , 2013, Pattern Recognit. Lett..

[28]  Zhen Wang,et al.  Facial Expression Recognition Based on Local Phase Quantization and Sparse Representation , 2012, ICNC.

[29]  Nadia Bianchi-Berthouze,et al.  Emotion recognition by two view SVM_2K classifier on dynamic facial expression features , 2011, Face and Gesture 2011.

[30]  P. Ekman Facial expressions of emotion: an old controversy and new findings. , 1992, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[31]  Zahir M. Hussain,et al.  Automatic facial expression recognition: feature extraction and selection , 2010, Signal, Image and Video Processing.

[32]  Mohamed Chtourou,et al.  Efficient MLP constructive training algorithm using a neuron recruiting approach for isolated word recognition system , 2011, Int. J. Speech Technol..

[33]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[34]  Maja Pantic,et al.  The first facial expression recognition and analysis challenge , 2011, Face and Gesture 2011.

[35]  Junfei Qiao,et al.  A novel pruning algorithm for self-organizing neural network , 2009, IJCNN.

[36]  K. S. Venkatesh,et al.  Emotion recognition from geometric facial features using self-organizing map , 2014, Pattern Recognit..

[37]  Yajie Tian,et al.  Handbook of face recognition , 2003 .

[38]  Zhaohui Wu,et al.  Facial expression recognition based on meta probability codes , 2013, Pattern Analysis and Applications.

[39]  Ioan Marius Bilasco,et al.  Intelligent pixels of interest selection with application to facial expression recognition using multilayer perceptron , 2013, Signal Process..

[40]  Nicu Sebe,et al.  Multimodal Human Computer Interaction: A Survey , 2005, ICCV-HCI.

[41]  Nasrollah Moghaddam Charkari,et al.  Audiovisual emotion recognition using ANOVA feature selection method and multi-classifier neural networks , 2014, Neural Computing and Applications.

[42]  Chokri Ben Amar,et al.  Facial Expression Recognition Based on Perceived Facial Images and Local Feature Matching , 2013, ICIAP.

[43]  Tamás D. Gedeon,et al.  Emotion recognition using PHOG and LPQ features , 2011, Face and Gesture 2011.

[44]  Min Chen,et al.  Facial expression recognition in dynamic sequences: An integrated approach , 2014, Pattern Recognit..

[45]  Debasmita Chakrabarti,et al.  Facial Expression Recognition Using Eigenspaces , 2013 .

[46]  Roberto Paredes,et al.  Dimensionality reduction by minimizing nearest-neighbor classification error , 2011, Pattern Recognit. Lett..

[47]  Yongzhao Zhan,et al.  A neural-AdaBoost based facial expression recognition system , 2014, Expert Syst. Appl..

[48]  C. Theekapun,et al.  Facial Expression Recognition Based on , 2008 .

[49]  Liming Chen,et al.  Textured 3D face recognition using biological vision-based facial representation and optimized weighted sum fusion , 2011, CVPR 2011 WORKSHOPS.

[50]  Hamid Sadeghi,et al.  Facial expression recognition using geometric normalization and appearance representation , 2013, 2013 8th Iranian Conference on Machine Vision and Image Processing (MVIP).

[51]  ZhanYongzhao,et al.  A neural-AdaBoost based facial expression recognition system , 2014 .

[52]  Chien-Cheng Lee,et al.  Facial Affect Recognition Using Regularized Discriminant Analysis-Based Algorithms , 2010, EURASIP J. Adv. Signal Process..

[53]  Zhen Li,et al.  Emotion recognition from an ensemble of features , 2011, Face and Gesture 2011.

[54]  Zhaoyu Wang,et al.  Analyses of a Multimodal Spontaneous Facial Expression Database , 2013, IEEE Transactions on Affective Computing.

[55]  Di Huang Robust face recognition based on three dimensional data , 2011 .

[56]  Derong Liu,et al.  A constructive algorithm for feedforward neural networks with incremental training , 2002 .

[57]  Y. V. Venkatesh,et al.  Facial expression recognition using radial encoding of local Gabor features and classifier synthesis , 2012, Pattern Recognit..

[58]  Jean Meunier,et al.  Emotion recognition using dynamic grid-based HoG features , 2011, Face and Gesture 2011.

[59]  Pravin Chandra,et al.  CONSTRUCTIVE NEURAL NETWORKS: A REVIEW , 2010 .

[60]  R.P. Lippmann,et al.  Pattern classification using neural networks , 1989, IEEE Communications Magazine.

[61]  Aly A. Farag,et al.  Facial expression recognition based on geometric and optical flow features in colour image sequences , 2012 .

[62]  Fernando José Von Zuben,et al.  A constructive algorithm to synthesize arbitrarily connected feedforward neural networks , 2012, Neurocomputing.

[63]  Shuicheng Yan,et al.  Accumulated motion images for facial expression recognition in videos , 2011, Face and Gesture 2011.

[64]  K. Scherer,et al.  Introducing the Geneva Multimodal Emotion Portrayal (GEMEP) corpus , 2010 .

[65]  Yong Man Ro,et al.  Using color texture sparsity for facial expression recognition , 2013, 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[66]  Nizar Bouguila,et al.  Face detection and facial expression recognition using simultaneous clustering and feature selection via an expectation propagation statistical learning framework , 2013, Multimedia Tools and Applications.

[67]  Shaogang Gong,et al.  Facial expression recognition based on Local Binary Patterns: A comprehensive study , 2009, Image Vis. Comput..

[68]  Eiji Hayashi,et al.  Consciousness Field Behavior Modules Level 0 Level 1 Level 2 Level 3 Level 4 Valued Sensation Field Primitive Sensation Basic Awake Consciousness Stable Emotions Detour Search Approach Avoid Move Sleep , 2022 .