High-Speed Video System for Micro-Expression Detection and Recognition

Micro-expressions play an essential part in understanding non-verbal communication and deceit detection. They are involuntary, brief facial movements that are shown when a person is trying to conceal something. Automatic analysis of micro-expression is challenging due to their low amplitude and to their short duration (they occur as fast as 1/15 to 1/25 of a second). We propose a fully micro-expression analysis system consisting of a high-speed image acquisition setup and a software framework which can detect the frames when the micro-expressions occurred as well as determine the type of the emerged expression. The detection and classification methods use fast and simple motion descriptors based on absolute image differences. The recognition module it only involves the computation of several 2D Gaussian probabilities. The software framework was tested on two publicly available high speed micro-expression databases and the whole system was used to acquire new data. The experiments we performed show that our solution outperforms state of the art works which use more complex and computationally intensive descriptors.

[1]  Matti Pietikäinen,et al.  A Spontaneous Micro-expression Database: Inducement, collection and baseline , 2013, 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[2]  P. Ekman,et al.  What the face reveals : basic and applied studies of spontaneous expression using the facial action coding system (FACS) , 2005 .

[3]  P. Ekman,et al.  Constants across cultures in the face and emotion. , 1971, Journal of personality and social psychology.

[4]  L. Fleischer Telling Lies Clues To Deceit In The Marketplace Politics And Marriage , 2016 .

[5]  Frédo Durand,et al.  Eulerian video magnification for revealing subtle changes in the world , 2012, ACM Trans. Graph..

[6]  Ron Kimmel,et al.  A Deep Learning Perspective on the Origin of Facial Expressions , 2017, ArXiv.

[7]  KokSheik Wong,et al.  Spontaneous Subtle Expression Detection and Recognition based on Facial Strain , 2016, Signal Process. Image Commun..

[8]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[9]  S. Porter,et al.  Reading Between the Lies , 2008, Psychological science.

[10]  E. A. Haggard,et al.  Micromomentary facial expressions as indicators of ego mechanisms in psychotherapy , 1966 .

[11]  Yuichi Ohta,et al.  Facial micro-expressions recognition using high speed camera and 3D-gradient descriptor , 2009, ICDP.

[12]  A. Rusu,et al.  Investigation of Facial Micro-expressions of Emotions in Psychopathy – A Case Study of an Individual in Detention , 2015 .

[13]  Guoying Zhao,et al.  CASME II: An Improved Spontaneous Micro-Expression Database and the Baseline Evaluation , 2014, PloS one.

[14]  Leslie P. Gartner,et al.  Textbook of Head and Neck Anatomy , 1982 .

[15]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[16]  Simon Lucey,et al.  Deformable Model Fitting by Regularized Landmark Mean-Shift , 2010, International Journal of Computer Vision.

[17]  Qi Wu,et al.  CASME database: A dataset of spontaneous micro-expressions collected from neutralized faces , 2013, 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[18]  Jun Yu,et al.  Spontaneous facial micro-expression detection based on deep learning , 2016, 2016 IEEE 13th International Conference on Signal Processing (ICSP).

[19]  Yuichi Ohta,et al.  Facial Micro-Expression Detection in Hi-Speed Video Based on Facial Action Coding System (FACS) , 2013, IEICE Trans. Inf. Syst..

[20]  Sergio Escalera,et al.  Automatic Recognition of Deceptive Facial Expressions of Emotion , 2017, ArXiv.

[21]  Trevor Hastie,et al.  Multi-class AdaBoost ∗ , 2009 .

[22]  Qi Wu,et al.  Effects of the duration of expressions on the recognition of microexpressions , 2012, Journal of Zhejiang University SCIENCE B.

[23]  Vinay Bettadapura,et al.  Face Expression Recognition and Analysis: The State of the Art , 2012, ArXiv.

[24]  J. Russell,et al.  Universality Revisited , 2013 .

[25]  KokSheik Wong,et al.  Subtle Expression Recognition Using Optical Strain Weighted Features , 2014, ACCV Workshops.

[26]  Matti Pietikäinen,et al.  Recognising spontaneous facial micro-expressions , 2011, 2011 International Conference on Computer Vision.

[27]  Matti Pietikäinen,et al.  Towards Reading Hidden Emotions: A Comparative Study of Spontaneous Micro-Expression Spotting and Recognition Methods , 2015, IEEE Transactions on Affective Computing.