Automatic facial expression recognition: feature extraction and selection

In this paper, we investigate feature extraction and feature selection methods as well as classification methods for automatic facial expression recognition (FER) system. The FER system is fully automatic and consists of the following modules: face detection, facial detection, feature extraction, selection of optimal features, and classification. Face detection is based on AdaBoost algorithm and is followed by the extraction of frame with the maximum intensity of emotion using the inter-frame mutual information criterion. The selected frames are then processed to generate characteristic features using different methods including: Gabor filters, log Gabor filter, local binary pattern (LBP) operator, higher-order local autocorrelation (HLAC) and a recent proposed method called HLAC-like features (HLACLF). The most informative features are selected based on both wrapper and filter feature selection methods. Experiments on several facial expression databases show comparisons of different methods.

[1]  A. Mehrabian.  Communication without words , 1968 .

[2]  David G. Stork,et al.  Pattern Classification , 1973 .

[3]  D J Field,et al.  Relations between the statistics of natural images and the response properties of cortical cells. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[4]  Takio Kurita,et al.  A New Scheme for Practical Flexible and Intelligent Vision Systems , 1988, MVA.

[5]  Roberto Battiti,et al.  Using mutual information for selecting features in supervised neural net learning , 1994, IEEE Trans. Neural Networks.

[6]  Lalit M. Patnaik,et al.  Genetic algorithms: a survey , 1994, Computer.

[7]  Tai Sing Lee,et al.  Image Representation Using 2D Gabor Wavelets , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Matti Pietikäinen,et al.  A comparative study of texture measures with classification based on featured distributions , 1996, Pattern Recognit..

[9]  Toby Berger,et al.  Reliable On-Line Human Signature Verification Systems , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Ron Kohavi,et al.  Wrappers for Feature Subset Selection , 1997, Artif. Intell..

[11]  R Kahavi,et al.  Wrapper for feature subset selection , 1997 .

[12]  Michael J. Lyons,et al.  Coding facial expressions with Gabor wavelets , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[13]  Takeo Kanade,et al.  Comprehensive database for facial expression analysis , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[14]  Irina Rish,et al.  An empirical study of the naive Bayes classifier , 2001 .

[15]  Beat Fasel,et al.  Automati Fa ial Expression Analysis: A Survey , 1999 .

[16]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[17]  Li Wang,et al.  Facial Expression Recognition Using HLAC Features and WPCA , 2005, ACII.

[18]  Takeo Kanade,et al.  Facial Expression Analysis , 2011, AMFG.

[19]  Takahiro Toyoda,et al.  Texture Classification Using Extended Higher Order Local Autocorrelation Features , 2005 .

[20]  Fuhui Long,et al.  Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Masoud Nikravesh,et al.  Feature Extraction - Foundations and Applications , 2006, Feature Extraction.

[22]  S. Lajevardi,et al.  Facial expression recognition from image sequences using optimized feature selection , 2008, 2008 23rd International Conference Image and Vision Computing New Zealand.

[23]  Margaret Lech,et al.  Averaged Gabor Filter Features for Facial Expression Recognition , 2008, 2008 Digital Image Computing: Techniques and Applications.

[24]  S. Lajevardi,et al.  Novel higher-order local autocorrelation-like feature extraction methodology for facial expression recognition , 2010 .