Detection of Micro-expression Recognition Based on Spatio-Temporal Modelling and Spatial Attention

My PhD project aims to make contributions in the affective computing application to assist in the depression diagnosis by micro-expression recognition. My motivation is the similarities of the low-intensity facial expressions in micro-expressions and the low-intensity facial expressions (`frozen face?) in people with psycho-motor retardation caused by depression. It will focus on, firstly, investigating spatio-temporal modelling and attention systems for micro-expression recognition (MER) and, secondly, exploring the role of micro-expressions in automated depression analysis by improving deep learning architectures to detect low-intensity facial expressions. This work will investigate different deep learning architectures (e.g. Temporal Convolutional Networks (TCNN) or Gate Recurrent Unit (GRU)) and validate the results on publicly available micro-expression benchmark datasets to quantitatively analyse the robustness and accuracy of MER's contribution to improving automatic depression analysis. Moreover, video magnification as a way to enhance small movements will be combined with the deep learning methods to address the low-intensity issues in MER.

[1]  P. Ekman,et al.  Nonverbal Leakage and Clues to Deception †. , 1969, Psychiatry.

[2]  Min Xu,et al.  Image Based Facial Micro-Expression Recognition Using Deep Learning on Small Datasets , 2017, 2017 International Conference on Digital Image Computing: Techniques and Applications (DICTA).

[3]  D. K. Mooney Activity Measurement in Psychology and Medicine. , 1993 .

[4]  Guoying Zhao,et al.  A Main Directional Mean Optical Flow Feature for Spontaneous Micro-Expression Recognition , 2016, IEEE Transactions on Affective Computing.

[5]  Stefano Zapperi,et al.  Role of the Number of Microtubules in Chromosome Segregation during Cell Division , 2015, PloS one.

[6]  Shu Zhan,et al.  Facial micro-expression recognition based on the fusion of deep learning and enhanced optical flow , 2018, Multimedia Tools and Applications.

[7]  Yong Man Ro,et al.  Micro-Expression Recognition with Expression-State Constrained Spatio-Temporal Feature Representations , 2016, ACM Multimedia.

[8]  John See,et al.  Efficient Spatio-Temporal Local Binary Patterns for Spontaneous Facial Micro-Expression Recognition , 2015, PloS one.

[9]  Descriptors Higher Annual Meeting of the International Communication Association , 1974 .

[10]  Matti Pietikäinen,et al.  Recognising spontaneous facial micro-expressions , 2011, 2011 International Conference on Computer Vision.

[11]  Vlado Menkovski,et al.  Micro-expression detection in long videos using optical flow and recurrent neural networks , 2019, 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019).

[12]  Abhishek Das,et al.  Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[13]  Guoying Zhao,et al.  CASME II: An Improved Spontaneous Micro-Expression Database and the Baseline Evaluation , 2014, PloS one.

[14]  Gregory D. Hager,et al.  Temporal Convolutional Networks for Action Segmentation and Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Nicholas Costen,et al.  SAMM: A Spontaneous Micro-Facial Movement Dataset , 2018, IEEE Transactions on Affective Computing.

[16]  Eric Granger,et al.  Emotion Recognition with Spatial Attention and Temporal Softmax Pooling , 2019, ICIAR.

[17]  Yongxin Zhu,et al.  Recognizing Facial Expressions Using a Shallow Convolutional Neural Network , 2019, IEEE Access.

[18]  John See,et al.  LBP with Six Intersection Points: Reducing Redundant Information in LBP-TOP for Micro-expression Recognition , 2014, ACCV.

[19]  Wen-Jing Yan,et al.  How Fast are the Leaked Facial Expressions: The Duration of Micro-Expressions , 2013 .

[20]  Matti Pietikäinen,et al.  Facial Micro-Expression Recognition Using Spatiotemporal Local Binary Pattern with Integral Projection , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[21]  Guoying Zhao,et al.  Spatiotemporal Recurrent Convolutional Networks for Recognizing Spontaneous Micro-Expressions , 2019, IEEE Transactions on Multimedia.

[22]  Frédo Durand,et al.  Eulerian video magnification for revealing subtle changes in the world , 2012, ACM Trans. Graph..

[23]  Huai-Qian Khor,et al.  Enriched Long-Term Recurrent Convolutional Network for Facial Micro-Expression Recognition , 2018, 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018).