Automated facial expression recognition based on histograms of oriented gradient feature vector differences

This article proposes an efficient automated method for facial expression recognition based on the histogram of oriented gradient (HOG) descriptor. This subject-independent method was designed for recognizing six prototyping emotions. It recognizes emotions by calculating differences on a level of feature descriptors between a neutral expression and a peak expression of an observed person. The parameters for the HOG descriptor were determined by using a genetic algorithm. Support vector machines (SVM) were applied during the recognition phase, whereat one SVM classifier was trained for one emotion. Each classifier was trained using difference vectors obtained by subtraction of HOG feature vectors calculated for the neutral and apex emotion subjects image. The proposed method was tested by using a leave-one-subject-out validation strategy for 106 subjects on 1232 images from the Cohn Kanade, and for 10 subjects on 192 images from the JAFFE database. A mean recognition rate of 95.64 % was obtained using the Cohn Kanade database, which is higher than the recognition rates for almost all other single-image- or video-based methods for facial emotion recognition.

[1]  Shaogang Gong,et al.  Robust facial expression recognition using local binary patterns , 2005, IEEE International Conference on Image Processing 2005.

[2]  Luc Van Gool,et al.  Hough Forest-Based Facial Expression Recognition from Video Sequences , 2010, ECCV Workshops.

[3]  Matti Pietikäinen,et al.  Boosted multi-resolution spatiotemporal descriptors for facial expression recognition , 2009, Pattern Recognit. Lett..

[4]  Eun-Soo Kim,et al.  Facial expression recognition using active contour-based face detection, facial movement-based feature extraction, and non-linear feature selection , 2014, Multimedia Systems.

[5]  Takeo Kanade,et al.  Comprehensive database for facial expression analysis , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[6]  Nicu Sebe,et al.  Authentic Facial Expression Analysis , 2004, FGR.

[7]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[8]  Carlos Orrite-Uruñuela,et al.  HOG-Based Decision Tree for Facial Expression Classification , 2009, IbPRIA.

[9]  Michael J. Lyons,et al.  Coding facial expressions with Gabor wavelets , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[10]  Peter Robinson,et al.  Real-Time Inference of Complex Mental States from Facial Expressions and Head Gestures , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[11]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[12]  Zahir M. Hussain,et al.  Automatic facial expression recognition: feature extraction and selection , 2010, Signal, Image and Video Processing.

[13]  P. Geetha,et al.  Evolutionary Computational Method of Facial Expression Analysis for Content-based Video Retrieval using 2-Dimensional Cellular Automata , 2010, ArXiv.

[14]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[15]  Takeo Kanade,et al.  Facial Expression Analysis , 2011, AMFG.

[16]  Maja Pantic,et al.  Fully Automatic Recognition of the Temporal Phases of Facial Actions , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[17]  Claude C. Chibelushi,et al.  Recognition of Facial Expressions in the Presence of Occlusion , 2001, BMVC.

[18]  Christopher Joseph Pal,et al.  Facial Expression Analysis Based on High Dimensional Binary Features , 2014, ECCV Workshops.

[19]  Aliaa A. A. Youssif,et al.  Spontaneous Facial Expression Recognition Based on Histogram of Oriented Gradients Descriptor , 2014, Comput. Inf. Sci..

[20]  Takeo Kanade,et al.  The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[21]  Chih-Hung Wu,et al.  A real-valued genetic algorithm to optimize the parameters of support vector machine for predicting bankruptcy , 2007, Expert Syst. Appl..

[22]  Simon Lucey,et al.  Face alignment through subspace constrained mean-shifts , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[23]  Min Chen,et al.  Facial expression recognition in dynamic sequences: An integrated approach , 2014, Pattern Recognit..

[24]  Ioannis Pitas,et al.  Comparison of ICA approaches for facial expression recognition , 2009, Signal Image Video Process..

[25]  Mohammed Yeasin,et al.  Recognition of facial expressions and measurement of levels of interest from video , 2006, IEEE Transactions on Multimedia.

[26]  Maja Pantic,et al.  Case-based reasoning for user-profiled recognition of emotions from face images , 2004, 2004 IEEE International Conference on Multimedia and Expo (ICME) (IEEE Cat. No.04TH8763).

[27]  Caifeng Shan,et al.  Local features based facial expression recognition with face registration errors , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[28]  W. Zheng,et al.  Facial expression recognition using kernel canonical correlation analysis (KCCA) , 2006, IEEE Transactions on Neural Networks.

[29]  P. Ekman,et al.  Measuring facial movement , 1976 .

[30]  Sandip Sen,et al.  Using real-valued genetic algorithms to evolve rule sets for classification , 1994, Proceedings of the First IEEE Conference on Evolutionary Computation. IEEE World Congress on Computational Intelligence.

[31]  Takeo Kanade,et al.  Facial Expression Recognition , 2011, Handbook of Face Recognition.

[32]  Fernando De la Torre,et al.  Facial Expression Analysis , 2011, Visual Analysis of Humans.

[33]  Timothy F. Cootes,et al.  Face Recognition Using Active Appearance Models , 1998, ECCV.

[34]  Nicu Sebe,et al.  Facial expression recognition from video sequences: temporal and static modeling , 2003, Comput. Vis. Image Underst..

[35]  Ioannis Pitas,et al.  An analysis of facial expression recognition under partial facial image occlusion , 2008, Image Vis. Comput..

[36]  Garrison W. Cottrell,et al.  Representing Face Images for Emotion Classification , 1996, NIPS.

[37]  Gwen Littlewort,et al.  Learning spatiotemporal features by using independent component analysis with application to facial expression recognition , 2012, Neurocomputing.

[38]  Wei Liu,et al.  Feature fusion of HOG and WLD for facial expression recognition , 2013, Proceedings of the 2013 IEEE/SICE International Symposium on System Integration.