Human facial expression recognition using curvelet feature extraction and normalized mutual information feature selection

To recognize expressions accurately, facial expression systems require robust feature extraction and feature selection methods. In this paper, a normalized mutual information based feature selection technique is proposed for FER systems. The technique is derived from an existing method, that is, the max-relevance and min-redundancy (mRMR) method. We, however, propose to normalize the mutual information used in this method so that the domination of the relevance or of the redundancy can be eliminated. For feature extraction, curvelet transform is used. After the feature extraction and selection the feature space is reduced by employing linear discriminant analysis (LDA). Finally, hidden Markov model (HMM) is used to recognize the expressions. The proposed FER system (CNF-FER) is validated using four publicly available standard datasets. For each dataset, 10-fold cross validation scheme is utilized. CNF-FER outperformed the existing well-known statistical and state-of-the-art methods by achieving a weighted average recognition rate of 99 % across all the datasets.

[1]  Ferdinando Silvestro Samaria,et al.  Face recognition using Hidden Markov Models , 1995 .

[2]  Martin D. Levine,et al.  Fully automated recognition of spontaneous facial expressions in videos using random forest classifiers , 2014, IEEE Transactions on Affective Computing.

[3]  B. Scholkopf,et al.  Fisher discriminant analysis with kernels , 1999, Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468).

[4]  J. Astola,et al.  Anew Improved Tactic to Extract Facial Expression Based on Genetic Algorithm and Wvdf , 2012 .

[5]  Michael J. Lyons,et al.  Coding facial expressions with Gabor wavelets , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[6]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Feng Chen,et al.  Facial expression recognition and its application based on curvelet transform and PSO-SVM , 2013 .

[8]  Vinay Bettadapura,et al.  Face Expression Recognition and Analysis: The State of the Art , 2012, ArXiv.

[9]  Beat Fasel,et al.  Automatic facial expression analysis: a survey , 2003, Pattern Recognit..

[10]  Zhe L. Lin,et al.  Nonparametric Context Modeling of Local Appearance for Pose- and Expression-Robust Facial Landmark Localization , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Jerzy Martyna,et al.  Spontaneous Facial Expression Recognition: Automatic Aggression Detection , 2012, HAIS.

[12]  Eun-Soo Kim,et al.  Facial expression recognition using active contour-based face detection, facial movement-based feature extraction, and non-linear feature selection , 2015, Multimedia Systems.

[13]  Laurent Demanet,et al.  Fast Discrete Curvelet Transforms , 2006, Multiscale Model. Simul..

[14]  David L. Donoho,et al.  Curvelets, multiresolution representation, and scaling laws , 2000, SPIE Optics + Photonics.

[15]  Mark A. Hall,et al.  Correlation-based Feature Selection for Machine Learning , 2003 .

[16]  Ryotaro Kamimura,et al.  Structural enhanced information and its application to improved visualization of self-organizing maps , 2011, Applied Intelligence.

[17]  Shiguang Shan,et al.  Learning Expressionlets on Spatio-temporal Manifold for Dynamic Facial Expression Recognition , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Shangfei Wang,et al.  Spontaneous Facial Expression Recognition Based on Feature Point Tracking , 2011, 2011 Sixth International Conference on Image and Graphics.

[19]  Goran Martinović,et al.  Emotion Recognition System by a Neural Network Based Facial Expression Analysis , 2013 .

[20]  Chong-Ho Choi,et al.  Input feature selection for classification problems , 2002, IEEE Trans. Neural Networks.

[21]  Friedhelm Schwenker,et al.  A Multiple Classifier System Approach for Facial Expressions in Image Sequences Utilizing GMM Supervectors , 2010, 2010 20th International Conference on Pattern Recognition.

[22]  Young-Koo Lee,et al.  Hierarchical Recognition Scheme for Human Facial Expression Recognition Systems , 2013, Sensors.

[23]  Fuhui Long,et al.  Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Hector Perez-Meana,et al.  Face recognition and verification using histogram equalization , 2010 .

[25]  Jacek M. Zurada,et al.  Normalized Mutual Information Feature Selection , 2009, IEEE Transactions on Neural Networks.

[26]  Sebastian Mika,et al.  Kernel Fisher Discriminants , 2003 .

[27]  Takeo Kanade,et al.  Comprehensive database for facial expression analysis , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[28]  G. Baudat,et al.  Generalized Discriminant Analysis Using a Kernel Approach , 2000, Neural Computation.

[29]  Shiqing Zhang,et al.  Robust Facial Expression Recognition via Compressive Sensing , 2012, Sensors.

[30]  Oksam Chae,et al.  Local Directional Number Pattern for Face Analysis: Face and Expression Recognition , 2013, IEEE Transactions on Image Processing.

[31]  Deepak Ghimire,et al.  Geometric Feature-Based Facial Expression Recognition in Image Sequences Using Multi-Class AdaBoost and Support Vector Machines , 2013, Sensors.

[32]  Ioannis Pitas,et al.  Facial Expression Recognition in Image Sequences Using Geometric Deformation Features and Support Vector Machines , 2007, IEEE Transactions on Image Processing.

[33]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[34]  KANCHAN LATA KASHYAP,et al.  STUDY AND ANALYSIS OF STATISTICAL FEATURES OF FACE EXPRESSION IN NOISY ENVIRONMENT , 2012 .

[35]  I K Fodor,et al.  A Survey of Dimension Reduction Techniques , 2002 .

[36]  Shaogang Gong,et al.  Facial expression recognition based on Local Binary Patterns: A comprehensive study , 2009, Image Vis. Comput..

[37]  Roberto Battiti,et al.  Using mutual information for selecting features in supervised neural net learning , 1994, IEEE Trans. Neural Networks.

[38]  Shiqing Zhang,et al.  Facial Expression Recognition Based on Local Binary Patterns and Kernel Discriminant Isomap , 2011, Sensors.

[39]  Huan Liu,et al.  Feature Selection for Classification , 1997, Intell. Data Anal..

[40]  J. Russell Core affect and the psychological construction of emotion. , 2003, Psychological review.

[41]  Ping Liu,et al.  Facial Expression Recognition via a Boosted Deep Belief Network , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[42]  L. Baum,et al.  An inequality and associated maximization technique in statistical estimation of probabilistic functions of a Markov process , 1972 .

[43]  Fei Chen,et al.  A Natural Visible and Infrared Facial Expression Database for Expression Recognition and Emotion Inference , 2010, IEEE Transactions on Multimedia.

[44]  Md. Zia Uddin,et al.  An enhanced independent component-based human facial expression recognition from video , 2009, IEEE Transactions on Consumer Electronics.

[45]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[46]  Alamgir Hossain Anew Improved Tactic to Extract Facial Expression Based on Genetic Algorithm and WVDF , 2012 .

[47]  Mahip M. Bartere,et al.  Facial Emotion Recognition in Videos Using Hmm , 2013 .