Contrastive Learning with Hallucinating Data for Long-Tailed Face Recognition

Face recognition has been well studied over the past decades. Most existing methods focus on optimizing the loss functions or improving the feature embedding networks. However, the long-tailed distribution problem, i.e, most of the samples belong to a few identities, while the remaining identities only have limited samples, is less explored, where these datasets are not fully utilized. In this paper, we propose a learning framework to balance the long-tailed distribution problem in public face datasets. The proposed framework learns the diversity from head identity samples to generate more samples for identifying persons’ identities in the tail. The generated samples are used to finetune face recognition models through a contrastive learning process. The proposed framework can be adapted to any feature embedding networks or combined with different loss functions. Experiments on both constrained and unconstrained datasets have proved the efficiency of the proposed framework.

[1]  Anil K. Jain,et al.  Probabilistic Face Embeddings , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[2]  Rama Chellappa,et al.  Synthesis-based recognition of low resolution faces , 2011, 2011 International Joint Conference on Biometrics (IJCB).

[3]  Stefanos Zafeiriou,et al.  ArcFace: Additive Angular Margin Loss for Deep Face Recognition , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Patrick J. Flynn,et al.  Toward facial re-identification: Experiments with data from an operational surveillance camera plant , 2016, 2016 IEEE 8th International Conference on Biometrics Theory, Applications and Systems (BTAS).

[5]  Andriy Mnih,et al.  Disentangling by Factorising , 2018, ICML.

[6]  Yu Qiao,et al.  A Discriminative Feature Learning Approach for Deep Face Recognition , 2016, ECCV.

[7]  Wei Liu,et al.  Super-Identity Convolutional Neural Network for Face Hallucination , 2018, ECCV.

[8]  Kaiming He,et al.  Momentum Contrast for Unsupervised Visual Representation Learning , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  N. Pavesic,et al.  Principal Gabor filters for face recognition , 2009, 2009 IEEE 3rd International Conference on Biometrics: Theory, Applications, and Systems.

[10]  Manuele Bicego,et al.  Using hidden Markov models and wavelets for face recognition , 2003, 12th International Conference on Image Analysis and Processing, 2003.Proceedings..

[11]  Andrew Zisserman,et al.  Multicolumn Networks for Face Recognition , 2018, BMVC.

[12]  Geoffrey E. Hinton,et al.  A Simple Framework for Contrastive Learning of Visual Representations , 2020, ICML.

[13]  Kilian Q. Weinberger,et al.  Distance Metric Learning for Large Margin Nearest Neighbor Classification , 2005, NIPS.

[14]  Xing Ji,et al.  CosFace: Large Margin Cosine Loss for Deep Face Recognition , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[15]  Omkar M. Parkhi,et al.  VGGFace2: A Dataset for Recognising Faces across Pose and Age , 2017, 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018).

[16]  Wei Xu,et al.  Deep Joint Face Hallucination and Recognition , 2016, ArXiv.

[17]  Li Shen,et al.  Comparator Networks , 2018, ECCV.

[18]  Xiaoming Liu,et al.  Multi-Task Convolutional Neural Network for Pose-Invariant Face Recognition , 2017, IEEE Transactions on Image Processing.

[19]  Bhiksha Raj,et al.  SphereFace: Deep Hypersphere Embedding for Face Recognition , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Ali Razavi,et al.  Data-Efficient Image Recognition with Contrastive Predictive Coding , 2019, ICML.

[21]  Yuxiao Hu,et al.  MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition , 2016, ECCV.

[22]  Xiaoming Liu,et al.  Disentangled Representation Learning GAN for Pose-Invariant Face Recognition , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Oriol Vinyals,et al.  Representation Learning with Contrastive Predictive Coding , 2018, ArXiv.

[24]  James Philbin,et al.  FaceNet: A unified embedding for face recognition and clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).