Multi-Layer Sparse Representation for Weighted LBP-Patches Based Facial Expression Recognition

In this paper, a novel facial expression recognition method based on sparse representation is proposed. Most contemporary facial expression recognition systems suffer from limited ability to handle image nuisances such as low resolution and noise. Especially for low intensity expression, most of the existing training methods have quite low recognition rates. Motivated by sparse representation, the problem can be solved by finding sparse coefficients of the test image by the whole training set. Deriving an effective facial representation from original face images is a vital step for successful facial expression recognition. We evaluate facial representation based on weighted local binary patterns, and Fisher separation criterion is used to calculate the weighs of patches. A multi-layer sparse representation framework is proposed for multi-intensity facial expression recognition, especially for low-intensity expressions and noisy expressions in reality, which is a critical problem but seldom addressed in the existing works. To this end, several experiments based on low-resolution and multi-intensity expressions are carried out. Promising results on publicly available databases demonstrate the potential of the proposed approach.

[1]  Alaa Eleyan,et al.  Gabor wavelet transform based facial expression recognition using PCA and LBP , 2014, 2014 22nd Signal Processing and Communications Applications Conference (SIU).

[2]  Ping Liu,et al.  Improving facial expression analysis using histograms of Log-Transformed Nonnegative Sparse Representation with a Spatial Pyramid Structure , 2013, 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[3]  J. Lyons.  Coding facial expression with Gabor wavelets , 1998 .

[4]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[5]  Matti Pietikäinen,et al.  A comparative study of texture measures with classification based on featured distributions , 1996, Pattern Recognit..

[6]  Nong Sang,et al.  A Facial Expression Recognition Method by Fusing Multiple Sparse Representation Based Classifiers , 2013, ISNN.

[7]  Mohammed Yeasin,et al.  From facial expression to level of interest: a spatio-temporal approach , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[8]  Jean-Philippe Thiran,et al.  Modelling human perception of static facial expressions , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[9]  G. Alpers,et al.  Happy mouth and sad eyes: scanning emotional facial expressions. , 2011, Emotion.

[10]  Matti Pietikäinen,et al.  Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Shaogang Gong,et al.  Robust facial expression recognition using local binary patterns , 2005, IEEE International Conference on Image Processing 2005.

[12]  P. Ekman.  Darwin, Deception, and Facial Expression , 2003, Annals of the New York Academy of Sciences.

[13]  Massimo Fornasier,et al.  Compressive Sensing , 2015, Handbook of Mathematical Methods in Imaging.

[14]  Zi-Lu Ying,et al.  A new method for facial expression recognition based on sparse representation plus LBP , 2010, 2010 3rd International Congress on Image and Signal Processing.

[15]  Yueting Zhuang,et al.  Sparse representation using nonnegative curds and whey , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[16]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Masahide Kaneko,et al.  Facial Expression Recognition Using Facial-component-based Bag of Words and PHOG Descriptors , 2010 .

[18]  Takeo Kanade,et al.  The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[19]  Takeo Kanade,et al.  Comprehensive database for facial expression analysis , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[20]  Shiqing Zhang,et al.  Robust Facial Expression Recognition via Compressive Sensing , 2012, Sensors.

[21]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[22]  Min Chen,et al.  Facial expression recognition in dynamic sequences: An integrated approach , 2014, Pattern Recognit..

[23]  Qi Jia,et al.  A sparse representation approach for local feature based expression recognition , 2011, 2011 International Conference on Multimedia Technology.

[24]  Junzhou Huang,et al.  Learning with dynamic group sparsity , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[25]  Takeo Kanade,et al.  Recognizing Action Units for Facial Expression Analysis , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  Gwen Littlewort,et al.  Dynamics of Facial Expression Extracted Automatically from Video , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[27]  David G. Stork,et al.  Pattern Classification (2nd ed.) , 1999 .

[28]  Y. V. Venkatesh,et al.  Facial expression recognition using radial encoding of local Gabor features and classifier synthesis , 2012, Pattern Recognit..

[29]  Cüneyt Güzelis,et al.  A new facial expression recognition based on curvelet transform and online sequential extreme learning machine initialized with spherical clustering , 2014, Neural Computing and Applications.

[30]  Shaogang Gong,et al.  Facial expression recognition based on Local Binary Patterns: A comprehensive study , 2009, Image Vis. Comput..

[31]  Zhihong Zeng,et al.  A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Dian Tjondronegoro,et al.  Facial Expression Recognition Using Facial Movement Features , 2011, IEEE Transactions on Affective Computing.

[33]  Aggelos K. Katsaggelos,et al.  Automatic facial expression recognition using facial animation parameters and multistream HMMs , 2006, IEEE Transactions on Information Forensics and Security.

[34]  Ying-li Tian,et al.  Evaluation of Face Resolution for Expression Analysis , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[35]  Michael J. Lyons,et al.  Coding facial expressions with Gabor wavelets , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[36]  Jun Ou,et al.  Automatic Facial Expression Recognition Using Gabor Filter and Expression Analysis , 2010, 2010 Second International Conference on Computer Modeling and Simulation.

[37]  R.G. Baraniuk,et al.  Compressive Sensing [Lecture Notes] , 2007, IEEE Signal Processing Magazine.

[38]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..