Intensity Enhancement Via Gan for Multimodal Facial Expression Recognition

Face expression recognition (FER) on low intensity is not well studied in the literature. This paper investigates this new problem and presents a novel Generative Adversarial Network (GAN) based multimodal approach to it. The method models the tasks of intensity enhancement and expression recognition jointly, ensuring that the synthesize faces not only present expression of high intensity, but also truly contribute to promoting the performance of FER. Extensive experiments are conducted on the BU-3DFE and BU-4DFE datasets. State-of-the-art FER performance clearly validates the effectiveness of the proposed method.

[1]  Takio Kurita,et al.  Facial expression intensity estimation using Siamese and triplet networks , 2018, Neurocomputing.

[2]  Jun Wang,et al.  3D Facial Expression Recognition Based on Primitive Surface Feature Distribution , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[3]  Jun Wang,et al.  A 3D facial expression database for facial behavior research , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[4]  James Jenn-Jier Lien,et al.  A Multi-Method Approach for Discriminating Between Similar Facial Expressions, Including Expression Intensity Estimation , 1998 .

[5]  Mohammad H. Mahoor,et al.  DISFA: A Spontaneous Facial Action Intensity Database , 2013, IEEE Transactions on Affective Computing.

[6]  R. Adolphs,et al.  Impaired Judgments of Sadness But Not Happiness Following Bilateral Amygdala Damage , 2004, Journal of Cognitive Neuroscience.

[7]  Liming Chen,et al.  Fast and Light Manifold CNN based 3D Facial Expression Recognition across Pose Variations , 2018, ACM Multimedia.

[8]  Liming Chen,et al.  Unsupervised Domain Adaptation with Regularized Optimal Transport for Multimodal 2D+3D Facial Expression Recognition , 2018, 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018).

[9]  Lijun Yin,et al.  Analyzing Facial Expressions Using Intensity-Variant 3D Data For Human Computer Interaction , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[10]  Ping Liu,et al.  Facial Expression Recognition via a Boosted Deep Belief Network , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Shang-Hong Lai,et al.  Emotion-Preserving Representation Learning via Generative Adversarial Network for Multi-View Facial Expression Recognition , 2018, 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018).

[12]  Jian Sun,et al.  Multimodal 2D+3D Facial Expression Recognition With Deep Fusion Convolutional Neural Network , 2017, IEEE Transactions on Multimedia.

[13]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[14]  Di Huang,et al.  Discriminative Attention-based Convolutional Neural Network for 3D Facial Expression Recognition , 2019, 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019).

[15]  Lijun Yin,et al.  A high-resolution 3D dynamic facial expression database , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[16]  Hasan Demirel,et al.  Facial Expression Recognition Using 3D Facial Feature Distances , 2007, ICIAR.

[17]  Li Fei-Fei,et al.  Perceptual Losses for Real-Time Style Transfer and Super-Resolution , 2016, ECCV.

[18]  Shaogang Gong,et al.  Facial expression recognition based on Local Binary Patterns: A comprehensive study , 2009, Image Vis. Comput..

[19]  Kai-Tai Song,et al.  Facial expression recognition based on mixture of basic expressions and intensities , 2012, 2012 IEEE International Conference on Systems, Man, and Cybernetics (SMC).