Facial Micro-Expression Detection in Hi-Speed Video Based on Facial Action Coding System (FACS)

SUMMARY Facial micro-expressions are fast and subtle facial motions that are considered as one of the most useful external signs for detecting hidden emotional changes in a person. However, they are not easy to detect and measure as they appear only for a short time, with small muscle contraction in the facial areas where salient features are not available. We propose a new computer vision method for detecting and measuring timing characteristics of facial micro-expressions. The core of this method is based on a descriptor that combines pre-processing masks, histograms and concatenation of spatial-temporal gradient vectors. Presented 3D gradient histogram descriptor is able to detect and measure the timing characteristics of the fast and subtle changes of the facial skin surface. This method is specifically designed for analysis of videos recorded using a hi-speed 200fps camera. Final classification of micro expressions is done by using a k-meanclassifier and a votingprocedure. The Facial Action CodingSystem was utilized to annotate the appearance and dynamics of the expressions in our new hi-speed micro-expressions video database. The efficiency of the

[1]  Fei Chen,et al.  A Natural Visible and Infrared Facial Expression Database for Expression Recognition and Emotion Inference , 2010, IEEE Transactions on Multimedia.

[2]  M. Bartlett,et al.  Machine Analysis of Facial Expressions , 2007 .

[3]  Javier R. Movellan,et al.  Tracking Motion, Deformation, and Texture Using Conditionally Gaussian Processes , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  S. Porter,et al.  Reading Between the Lies , 2008, Psychological science.

[5]  Maja Pantic,et al.  Action unit detection using sparse appearance descriptors in space-time video volumes , 2011, Face and Gesture 2011.

[6]  Serge J. Belongie,et al.  Behavior recognition via sparse spatio-temporal features , 2005, 2005 IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance.

[7]  Daijin Kim,et al.  Automated Face Analysis: Emerging Technologies and Research , 2009 .

[8]  Gwen Littlewort,et al.  Automatic Recognition of Facial Actions in Spontaneous Expressions , 2006, J. Multim..

[9]  Yajie Tian,et al.  Handbook of face recognition , 2003 .

[10]  Mubarak Shah,et al.  A 3-dimensional sift descriptor and its application to action recognition , 2007, ACM Multimedia.

[11]  Maja Pantic,et al.  Detecting facial actions and their temporal segments in nearly frontal-view face image sequences , 2005, 2005 IEEE International Conference on Systems, Man and Cybernetics.

[12]  R. Cowie,et al.  The description of naturally occurring emotional speech , 2003 .

[13]  Takeo Kanade,et al.  Comprehensive database for facial expression analysis , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[14]  Qiang Ji,et al.  Facial Action Unit Recognition by Exploiting Their Dynamic and Semantic Relationships , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  P. Ekman,et al.  Facial action coding system: a technique for the measurement of facial movement , 1978 .

[16]  Sotiris Malassiotis,et al.  Real-time 2D+3D facial action and expression recognition , 2010, Pattern Recognit..

[17]  Qingshan Liu,et al.  Boosting encoded dynamic features for facial expression recognition , 2009, Pattern Recognit. Lett..

[18]  Yuichi Ohta,et al.  Facial micro-expressions recognition using high speed camera and 3D-gradient descriptor , 2009, ICDP.

[19]  Barbara Caputo,et al.  Local velocity-adapted motion events for spatio-temporal recognition , 2007, Comput. Vis. Image Underst..

[20]  Cordelia Schmid,et al.  A Spatio-Temporal Descriptor Based on 3D-Gradients , 2008, BMVC.

[21]  P. Ekman.  Facial expressions of emotion: an old controversy and new findings. , 1992, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[22]  Simon Lucey,et al.  Investigating Spontaneous Facial Action Recognition through AAM Representations of the Face , 2007 .

[23]  Maja Pantic,et al.  A Dynamic Texture-Based Approach to Recognition of Facial Actions and Their Temporal Models , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.