SHCFNet on Micro-expression Recognition System

Micro expression is a facial feature that can reflect the most real emotional state hidden in the human heart. This is a very short process and difficult to capture accurately. Based convolutional network, a new network architecture (SHCFNet) is proposed to extract the spatial-temporal feature of peak frames, the optical flow between onset and apex frame and its derivative (optical strain). The proposed network stacks these features from the outcomes of the previous layer. Then, the stacked feature is merged with the convolution feature of the previous layer, which enhances the learnability of neurons. The performance of the proposed SHCFNet are evaluated on four benchmark datasets: CASME I, CASME II, SAMM and SMIC, and compared with other advanced networks.

[1]  Yuichi Ohta,et al.  Facial micro-expressions recognition using high speed camera and 3D-gradient descriptor , 2009, ICDP.

[2]  P. Ekman,et al.  Constants across cultures in the face and emotion. , 1971, Journal of personality and social psychology.

[3]  Dmitry B. Goldgof,et al.  Towards macro- and micro-expression spotting in video using strain patterns , 2009, 2009 Workshop on Applications of Computer Vision (WACV).

[4]  Guoying Zhao,et al.  CASME II: An Improved Spontaneous Micro-Expression Database and the Baseline Evaluation , 2014, PloS one.

[5]  Shaogang Gong,et al.  Facial expression recognition based on Local Binary Patterns: A comprehensive study , 2009, Image Vis. Comput..

[6]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[7]  Xiaolan Fu,et al.  SMEConvNet: A Convolutional Neural Network for Spotting Spontaneous Facial Micro-Expression From Long Videos , 2018, IEEE Access.

[9]  Nicholas Costen,et al.  SAMM: A Spontaneous Micro-Facial Movement Dataset , 2018, IEEE Transactions on Affective Computing.

[10]  Wei-Chuen Yau,et al.  OFF-ApexNet on Micro-expression Recognition System , 2018, Signal Process. Image Commun..

[11]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[13]  Young-Koo Lee,et al.  Hierarchical Recognition Scheme for Human Facial Expression Recognition Systems , 2013, Sensors.

[14]  Radhika M. Pai,et al.  Combining temporal interpolation and DCNN for faster recognition of micro-expressions in video sequences , 2016, 2016 International Conference on Advances in Computing, Communications and Informatics (ICACCI).

[15]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Matti Pietikäinen,et al.  A Spontaneous Micro-expression Database: Inducement, collection and baseline , 2013, 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[17]  Qi Wu,et al.  CASME database: A dataset of spontaneous micro-expressions collected from neutralized faces , 2013, 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[18]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[19]  J. Koenderink.  Q… , 2014, Les noms officiels des communes de Wallonie, de Bruxelles-Capitale et de la communaute germanophone.

[20]  E. A. Haggard,et al.  Micromomentary facial expressions as indicators of ego mechanisms in psychotherapy , 1966 .

[21]  Guoying Zhao,et al.  A Main Directional Mean Optical Flow Feature for Spontaneous Micro-Expression Recognition , 2016, IEEE Transactions on Affective Computing.

[22]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  P. Ekman,et al.  The Repertoire of Nonverbal Behavior: Categories, Origins, Usage, and Coding , 1969 .

[24]  Sridhar Godavarthy.  Micro Expression Detection Using Strain Patterns , 2009 .

[25]  Qi Wu,et al.  The Machine Knows What You Are Hiding: An Automatic Micro-expression Recognition System , 2011, ACII.