Using Moments on Spatiotemporal Plane for Facial Expression Recognition

In this paper, we propose a novel approach to capture the dynamic deformation caused by facial expressions. The proposed method is concentrated on the spatiotemporal plane which is not well explored. It uses the moments as features to describe the movements of essential components such as eyes and mouth on vertical time plane. The system we developed can automatically recognize the expression on images as well as on image sequences. The experiments are performed on 348 sequences from 95 subjects in Cohn-Kanade database and obtained good results as high as 96.1% in 7-class recognition for frames and 98.5% in 6-class for sequences.

[1]  J. N. Bassili.  Emotion recognition: the role of facial movement and the relative importance of upper and lower areas of the face. , 1979, Journal of personality and social psychology.

[2]  Shang-Hong Lai,et al.  Learning partially-observed hidden conditional random fields for facial expression recognition , 2009, CVPR.

[3]  Gwen Littlewort,et al.  Fully Automatic Facial Action Recognition in Spontaneous Behavior , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[4]  Enrique Muñoz,et al.  Recognising facial expressions in video sequences , 2007, Pattern Analysis and Applications.

[5]  Chi Chung Ko,et al.  Using moment invariants and HMM in facial expression recognition , 2000, 4th IEEE Southwest Symposium on Image Analysis and Interpretation.

[6]  Matti Pietikäinen,et al.  Boosted multi-resolution spatiotemporal descriptors for facial expression recognition , 2009, Pattern Recognit. Lett..

[7]  M. Bartlett,et al.  Machine Analysis of Facial Expressions , 2007 .

[8]  Yi Ji,et al.  Facial Expression Recognition by Automatic Facial Parts Position Detection with Boosted-LBP , 2009, 2009 Fifth International Conference on Signal Image Technology and Internet Based Systems.

[9]  Shaogang Gong,et al.  Facial expression recognition based on Local Binary Patterns: A comprehensive study , 2009, Image Vis. Comput..

[10]  P. Ekman,et al.  Facial action coding system: a technique for the measurement of facial movement , 1978 .

[11]  Bernhard Schölkopf,et al.  Face Detection - Efficient and Rank Deficient , 2004, NIPS.

[12]  Takeo Kanade,et al.  Comprehensive database for facial expression analysis , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).