Micro-Expression Spotting Using the Riesz Pyramid

Facial micro-expressions (MEs) are fast and involuntary facial expressions which reveal people hidden emotions. ME spotting refers to the process of finding the temporal locations of rapid facial movements from a video sequence. However, detecting these events is difficult due to their short durations and low intensities. Also, a distinction must be made between MEs and eye-related movements (blinking, eye-gaze change, etc). Taking inspiration from video magnification techniques, we design a workflow for automatically spotting MEs based on the Riesz pyramid. In addition, we propose a filtering and masking scheme that segment motions of interest without producing undesired artifacts or delays. Furthermore, the system is able to differentiate between MEs and eye movements. Experiments are carried out on two databases containing videos of spontaneous micro-expressions. Finally, we show that our method is able to outperform other methods from the state of the art in this challenging task.

[1]  Guoying Zhao,et al.  Quantifying Micro-expressions with Constraint Local Model and Local Binary Pattern , 2014, ECCV Workshops.

[2]  Yong Man Ro,et al.  Micro-Expression Recognition with Expression-State Constrained Spatio-Temporal Feature Representations , 2016, ACM Multimedia.

[3]  Guoying Zhao,et al.  CASME II: An Improved Spontaneous Micro-Expression Database and the Baseline Evaluation , 2014, PloS one.

[4]  Frédo Durand,et al.  Quaternionic Representation of the Riesz Pyramid for Video Magnification , 2014 .

[5]  Moi Hoon Yap,et al.  Objective Micro-Facial Movement Detection Using FACS-Based Regions and Baseline Evaluation , 2016, 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018).

[6]  Maja Pantic,et al.  Optimization Problems for Fast AAM Fitting in-the-Wild , 2013, 2013 IEEE International Conference on Computer Vision.

[7]  Yong Man Ro,et al.  Subtle Facial Expression Recognition Using Adaptive Magnification of Discriminative Facial Motion , 2015, ACM Multimedia.

[8]  Matti Pietikäinen,et al.  Towards Reading Hidden Emotions: A Comparative Study of Spontaneous Micro-Expression Spotting and Recognition Methods , 2015, IEEE Transactions on Affective Computing.

[9]  Sung Yong Shin,et al.  General Construction of Time-Domain Filters for Orientation Data , 2002, IEEE Trans. Vis. Comput. Graph..

[10]  Frédo Durand,et al.  Eulerian video magnification for revealing subtle changes in the world , 2012, ACM Trans. Graph..

[11]  Wen-Jing Yan,et al.  How Fast are the Leaked Facial Expressions: The Duration of Micro-Expressions , 2013 .

[12]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[13]  P. Ekman,et al.  What the face reveals : basic and applied studies of spontaneous expression using the facial action coding system (FACS) , 2005 .

[14]  Sungsoo Park,et al.  Subtle facial expression recognition using motion magnification , 2009, Pattern Recognit. Lett..

[15]  Dimitri Van De Ville,et al.  Multiresolution Monogenic Signal Analysis Using the Riesz–Laplace Wavelet Transform , 2009, IEEE Transactions on Image Processing.

[16]  Frédo Durand,et al.  Phase-based video motion processing , 2013, ACM Trans. Graph..

[17]  Matti Pietikäinen,et al.  A Spontaneous Micro-expression Database: Inducement, collection and baseline , 2013, 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[18]  KokSheik Wong,et al.  Automatic apex frame spotting in micro-expression database , 2015, 2015 3rd IAPR Asian Conference on Pattern Recognition (ACPR).

[19]  Dmitry B. Goldgof,et al.  Macro- and micro-expression spotting in long videos using spatio-temporal strain , 2011, Face and Gesture 2011.

[20]  Frédo Durand,et al.  Riesz pyramids for fast phase-based video magnification , 2014, 2014 IEEE International Conference on Computational Photography (ICCP).

[21]  Matti Pietikäinen,et al.  Spotting Rapid Facial Movements from Videos Using Appearance-Based Feature Difference Analysis , 2014, 2014 22nd International Conference on Pattern Recognition.