Attack Can Benefit: An Adversarial Approach to Recognizing Facial Expressions under Noisy Annotations

The real-world Facial Expression Recognition (FER) datasets usually exhibit complex scenarios with coupled noise annotations and imbalanced classes distribution, which undoubtedly impede the development of FER methods. To address the aforementioned issues, in this paper, we propose a novel and flexible method to spot noisy labels by leveraging adversarial attack, termed as Geometry Aware Adversarial Vulnerability Estimation (GAAVE). Different from existing state-of-the-art methods of noisy label learning (NLL), our method has no reliance on additional information and is thus easy to generalize to the large-scale real-world FER datasets. Besides, the combination of Dataset Splitting module and Subset Refactoring module mitigates the impact of class imbalance, and the Self-Annotator module facilitates the sufficient use of all training data. Extensive experiments on RAF-DB, FERPlus, AffectNet, and CIFAR-10 datasets validate the effectiveness of our method. The stabilized enhancement based on different methods demonstrates the flexibility of our proposed GAAVE.

[1]  Chen Chen,et al.  Delving into the Adversarial Robustness of Federated Learning , 2023, AAAI.

[2]  Mofei Song,et al.  Toward Stable Co-Saliency Detection and Object Co-Segmentation , 2022, IEEE Transactions on Image Processing.

[3]  Bo Li,et al.  Federated Learning with Label Distribution Skew via Logits Calibration , 2022, ICML.

[4]  Mofei Song,et al.  Re-Thinking the Relations in Co-Saliency Detection , 2022, IEEE Transactions on Circuits and Systems for Video Technology.

[5]  Weihong Deng,et al.  Learn From All: Erasing Attention Consistency for Noisy Label Facial Expression Recognition , 2022, ECCV.

[6]  Bo Li,et al.  Towards Efficient Data Free Blackbox Adversarial Attack , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Shuang Wu,et al.  Detecting Camouflaged Object in Frequency Domain , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Bo Li,et al.  Towards Practical Certifiable Patch Defense with Vision Transformer , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Chunhua Shen,et al.  DENSE: Data-Free One-Shot Federated Learning , 2021, NeurIPS.

[10]  Shouhong Ding,et al.  Highly Efficient Natural Image Matting , 2021, BMVC.

[11]  Jilin Li,et al.  Detecting Adversarial Patch Attacks through Global-local Consistency , 2021, AdvM @ ACM Multimedia.

[12]  Tao Wang,et al.  Distract Your Attention: Multi-Head Cross Attention Network for Facial Expression Recognition , 2021, Biomimetics.

[13]  Mofei Song,et al.  Disentangled High Quality Salient Object Detection , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[14]  Qingshan Liu,et al.  Learning Deep Global Multi-Scale and Local Attention Features for Facial Expression Recognition in the Wild , 2021, IEEE Transactions on Image Processing.

[15]  Darshan Gera,et al.  Consensual Collaborative Training And Knowledge Distillation Based Facial Expression Recognition Under Noisy Annotations , 2021, International Journal of Engineering Trends and Technology.

[16]  Lv Tang CoSformer: Detecting Co-Salient Object with Transformers , 2021, ArXiv.

[17]  Z. Chai,et al.  Feature Decomposition and Reconstruction Learning for Effective Facial Expression Recognition , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Tao Mei,et al.  Dive into Ambiguity: Latent Distribution Mining and Pairwise Uncertainty Estimation for Facial Expression Recognition , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Qi Wu,et al.  Jo-SRC: A Contrastive Approach for Combating Noisy Labels , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Song-Chun Zhu,et al.  Learning to Amend Facial Expression Representation via De-albino and Affinity , 2021, ArXiv.

[21]  Pengxiang Wu,et al.  Learning with Feature-Dependent Label Noise: A Progressive Approach , 2021, ICLR.

[22]  Hongxia Yang,et al.  Understanding the Interaction of Adversarial Training with Noisy Labels , 2021, ArXiv.

[23]  Xiaojun Qi,et al.  Facial Expression Recognition in the Wild via Deep Attentive Center Loss , 2021, 2021 IEEE Winter Conference on Applications of Computer Vision (WACV).

[24]  Bo Li,et al.  CLASS: Cross-Level Attention and Supervision for Salient Objects Detection , 2020, ACCV.

[25]  Darshan Gera,et al.  Landmark Guidance Independent Spatio-channel Attention and Complementary Context Information based Facial Expression Recognition , 2020, Pattern Recognit. Lett..

[26]  Hwanjun Song,et al.  Learning From Noisy Labels With Deep Neural Networks: A Survey , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[27]  Varun Kanade,et al.  How benign is benign overfitting? , 2020, ICLR.

[28]  Gang Niu,et al.  Dual T: Reducing Estimation Error for Transition Matrix in Label-noise Learning , 2020, NeurIPS.

[29]  Xiaojun Qi,et al.  Discriminant Distribution-Agnostic Loss for Facial Expression Recognition in the Wild , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[30]  Jianfei Yang,et al.  Suppressing Uncertainties for Large-Scale Facial Expression Recognition , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Zhengxing Sun,et al.  Co-saliency Detection Based on Hierarchical Consistency , 2019, ACM Multimedia.

[32]  Binqiang Zhao,et al.  O2U-Net: A Simple Noisy Label Detection Approach for Deep Neural Networks , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[33]  Jae-Gil Lee,et al.  Prestopping: How Does Early Stopping Help Generalization against Label Noise? , 2019, ArXiv.

[34]  James Bailey,et al.  Symmetric Cross Entropy for Robust Learning With Noisy Labels , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[35]  Bo Li,et al.  Detecting Robust Co-Saliency with Recurrent Co-Attention Neural Network , 2019, IJCAI.

[36]  Zhengxing Sun,et al.  SuperVAE: Superpixelwise Variational Autoencoder for Salient Object Detection , 2019, AAAI.

[37]  Weihong Deng,et al.  Unequal-Training for Deep Face Recognition With Long-Tailed Noisy Data , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Jianfei Yang,et al.  Region Attention Networks for Pose and Occlusion Robust Facial Expression Recognition , 2019, IEEE Transactions on Image Processing.

[39]  Bo Li,et al.  Two-B-real Net: Two-branch Network for Real-time Salient Object Detection , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[40]  N. Silberman,et al.  Learning From Noisy Labels by Regularized Estimation of Annotator Confusion , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Yang Song,et al.  Class-Balanced Loss Based on Effective Number of Samples , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[42]  Xingrui Yu,et al.  How does Disagreement Help Generalization against Label Corruption? , 2019, ICML.

[43]  Mert R. Sabuncu,et al.  Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels , 2018, NeurIPS.

[44]  Shan Li,et al.  Deep Facial Expression Recognition: A Survey , 2018, IEEE Transactions on Affective Computing.

[45]  Yang Liu,et al.  MobileFaceNets: Efficient CNNs for Accurate Real-time Face Verification on Mobile Devices , 2018, CCBR.

[46]  Xingrui Yu,et al.  Co-teaching: Robust training of deep neural networks with extremely noisy labels , 2018, NeurIPS.

[47]  Li Fei-Fei,et al.  MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels , 2017, ICML.

[48]  Mohammad H. Mahoor,et al.  AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild , 2017, IEEE Transactions on Affective Computing.

[49]  Junping Du,et al.  Reliable Crowdsourcing and Deep Locality-Preserving Learning for Expression Recognition in the Wild , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[50]  Aleksander Madry,et al.  Towards Deep Learning Models Resistant to Adversarial Attacks , 2017, ICLR.

[51]  Yoshua Bengio,et al.  A Closer Look at Memorization in Deep Networks , 2017, ICML.

[52]  Shai Shalev-Shwartz,et al.  Decoupling "when to update" from "how to update" , 2017, NIPS.

[53]  Richard Nock,et al.  Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[54]  Ajinkya More,et al.  Survey of resampling techniques for improving classification performance in unbalanced datasets , 2016, ArXiv.

[55]  Emad Barsoum,et al.  Training deep networks for facial expression recognition with crowd-sourced label distribution , 2016, ICMI.

[56]  Yuxiao Hu,et al.  MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition , 2016, ECCV.

[57]  Qingming Huang,et al.  Relay Backpropagation for Effective Learning of Deep Convolutional Neural Networks , 2015, ECCV.

[58]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[59]  Mohammed Bennamoun,et al.  Cost-Sensitive Learning of Deep Feature Representations From Imbalanced Data , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[60]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[61]  Jason Yosinski,et al.  Deep neural networks are easily fooled: High confidence predictions for unrecognizable images , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[62]  Joan Bruna,et al.  Intriguing properties of neural networks , 2013, ICLR.

[63]  Yoshua Bengio,et al.  Challenges in representation learning: A report on three machine learning contests , 2013, Neural Networks.

[64]  Qingshan Liu,et al.  Learning active facial patches for expression analysis , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[65]  Markus Flierl,et al.  Graph-Preserving Sparse Nonnegative Matrix Factorization With Application to Facial Expression Recognition , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[66]  Skyler T. Hawk,et al.  Presentation and validation of the Radboud Faces Database , 2010 .

[67]  Takeo Kanade,et al.  The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[68]  Bo Li,et al.  Shape Matters: Deformable Patch Attack , 2022, ECCV.

[69]  Weihong Deng,et al.  Relative Uncertainty Learning for Facial Expression Recognition , 2021, NeurIPS.

[70]  Hyung-Jeong Yang,et al.  Pyramid With Super Resolution for In-the-Wild Facial Expression Recognition , 2020, IEEE Access.

[71]  Martial Hebert,et al.  Learning to Model the Tail , 2017, NIPS.

[72]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[73]  Bo Li Group-Wise Deep Object Co-Segmentation With Co-Attention Recurrent Neural Network , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).