Recognition of occluded facial expressions using a Fusion of Localized Sparse Representation Classifiers

Facial expression recognition (FER) methods that are localized have a distinct advantage over whole face methods in situations where part of the face is occluded. We present a new algorithm that first classifies local regions of the face around each eye and the mouth using a Sparse Representation Classifier (SRC). Then, it dynamically determines the set of local regions that best classifies the facial expression by using the representation error characteristics from each local region. This algorithm is termed Fusion of Local Sparse Representation Classifiers (FLSRC). In an experiment where the mouth region is occluded, the FLSRC recognition rate was 93.4% compared to 72.8% using SRC with all available pixels and rates < 60% using Gabor or PCA based methods. FLSRC significantly outperforms other methods when large facial regions are occluded. For example, when approximately 25% of the image is occluded, FLSRC gives a recognition rate of 85% compared to < 65% for other methods.

[1]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[2]  Maja Pantic,et al.  Automatic Analysis of Facial Expressions: The State of the Art , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Shane F. Cotter,et al.  Sparse Representation for accurate classification of corrupted and occluded facial expressions , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[4]  D. Massaro,et al.  Featural evaluation, integration, and judgment of facial affect. , 1997, Journal of experimental psychology. Human perception and performance.

[5]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Ioannis Pitas,et al.  Application of non-negative and local non negative matrix factorization to facial expression recognition , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[7]  David G. Stork,et al.  Pattern classification, 2nd Edition , 2000 .

[8]  Marian Stewart Bartlett,et al.  Classifying Facial Actions , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Michael J. Lyons,et al.  Coding facial expressions with Gabor wavelets , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[10]  Zhengyou Zhang,et al.  Feature-Based Facial Expression Recognition: Sensitivity Analysis and Experiments with A Multilayer Perceptron , 1999, Int. J. Pattern Recognit. Artif. Intell..

[11]  Beat Fasel,et al.  Automatic facial expression analysis: a survey , 2003, Pattern Recognit..

[12]  Garrison W. Cottrell,et al.  Representing Face Images for Emotion Classification , 1996, NIPS.

[13]  Michael J. Lyons,et al.  Automatic Classification of Single Facial Images , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Lianwen Jin,et al.  A New Facial Expression Recognition Method Based on Local Gabor Filter Bank and PCA plus LDA , 2006 .

[15]  E.J. Candes,et al.  An Introduction To Compressive Sampling , 2008, IEEE Signal Processing Magazine.

[16]  Ganesh K. Venayagamoorthy,et al.  Recognition of facial expressions using Gabor wavelets and learning vector quantization , 2008, Eng. Appl. Artif. Intell..

[17]  Ioannis Pitas,et al.  Facial expression analysis under partial occlusion , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[18]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.