WebFace260M: A Benchmark for Million-Scale Deep Face Recognition

In this paper, we contribute a new million-scale recognition benchmark, containing uncurated 4M identities/260M faces (WebFace260M) and cleaned 2M identities/42M faces (WebFace42M) training data, as well as an elaborately designed time-constrained evaluation protocol. Firstly, we collect 4M name lists and download 260M faces from the Internet. Then, a Cleaning Automatically utilizing Self-Training pipeline is devised to purify the tremendous WebFace260M, which is efficient and scalable. To our best knowledge, the cleaned WebFace42M is the largest public face recognition training set in the community. Referring to practical deployments, Face Recognition under Inference Time conStraint (FRUITS) protocol and a new test set with rich attributes are constructed. Moreover, we gather a large-scale masked face sub-set for biometrics assessment under COVID-19. For a comprehensive evaluation of face matchers, three recognition tasks are performed under standard, masked and unbiased settings, respectively. Equipped with this benchmark, we delve into million-scale face recognition problems. Enabled by WebFace42M, we reduce 40% failure rate on the challenging IJB-C set and rank the 3rd among 430 entries on NIST-FRVT. Even 10% data (WebFace4M) shows superior performance compared with the public training set. The proposed benchmark shows enormous potential on standard, masked and unbiased face recognition scenarios.

[1]  Stefanos Zafeiriou,et al.  Masked Face Recognition Challenge: The InsightFace Track Report , 2021, 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW).

[2]  Jiwen Lu,et al.  Masked Face Recognition Challenge: The WebFace260M Track Report , 2021, ArXiv.

[3]  Shiguang Shan,et al.  MFR 2021: Masked Face Recognition Competition , 2021, 2021 IEEE International Joint Conference on Biometrics (IJCB).

[4]  T. Spector,et al.  Association of social distancing and face mask use with risk of COVID-19 , 2021, Nature Communications.

[5]  Jiwen Lu,et al.  Pseudo Facial Generation with Extreme Poses for Face Recognition , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Lei Zhang,et al.  Virtual Fully-Connected Layer: Training a Large-Scale Face Recognition Dataset with Limited Computational Resources , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Feiyue Huang,et al.  Consistent Instance False Positive Improves Fairness in Face Recognition , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Fei-Yue Wang,et al.  Learning from the Web: Webly Supervised Meta-Learning for Masked Face Recognition , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[9]  Arjan Kuijper,et al.  Extended evaluation of the effect of real and simulated masks on face recognition performance , 2021, IET Biom..

[10]  Gang Zhang,et al.  Dynamic Class Queue for Large Scale Face Recognition In the Wild , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Tao Mei,et al.  Towards NIR-VIS Masked Face Recognition , 2021, IEEE Signal Processing Letters.

[12]  Dalong Du,et al.  WebFace260M: A Benchmark Unveiling the Power of Million-Scale Deep Face Recognition , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  A. Morales,et al.  A Comprehensive Study on Face Recognition Biases Beyond Demographics , 2021, IEEE Transactions on Technology and Society.

[14]  Hongming Shan,et al.  When Age-Invariant Face Recognition Meets Face Age Synthesis: A Multi-Task Learning Framework , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Krishan Kumar,et al.  Deep Neural Architecture for Face mask Detection on Simulated Masked Face Dataset against Covid-19 Pandemic , 2021, 2021 International Conference on Computing, Communication, and Intelligent Systems (ICCCIS).

[16]  Patrick Grother,et al.  Ongoing Face Recognition Vendor Test (FRVT) Part 6B: Face recognition accuracy with face masks using post-COVID-19 algorithms , 2020 .

[17]  Yonghong Tian,et al.  Masked Face Recognition with Latent Part Detection , 2020, ACM Multimedia.

[18]  Yonghong Tian,et al.  Masked Face Recognition with Generative Data Augmentation and Domain Constrained Ranking , 2020, ACM Multimedia.

[19]  Debing Zhang,et al.  Partial FC: Training 10 Million Identities on a Single Machine , 2020, 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW).

[20]  Arijit Raychowdhury,et al.  Masked Face Recognition for Secure Authentication , 2020, ArXiv.

[21]  Yonghyun Kim,et al.  BroadFace: Looking at Tens of Thousands of People at Once for Face Recognition , 2020, ECCV.

[22]  Shuicheng Yan,et al.  Towards Age-Invariant Face Recognition , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Tao Mei,et al.  Semi-Siamese Training for Shallow Face Learning , 2020, ECCV.

[24]  W. Hariri,et al.  Efficient masked face recognition method during the COVID-19 pandemic , 2020, Signal, Image and Video Processing.

[25]  Sixue Gong,et al.  Mitigating Face Recognition Bias via Group Adaptive Classifier , 2020, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Weihong Deng,et al.  Global-Local GCN: Large-Scale Label Noise Cleansing for Face Recognition , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Irene Kotsia,et al.  RetinaFace: Single-Shot Multi-Level Face Localisation in the Wild , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Guodong Guo,et al.  Hierarchical Pyramid Diverse Attention Networks for Face Recognition , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Yonghyun Kim,et al.  GroupFace: Learning Latent Groups and Constructing Group-Based Representations for Face Recognition , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  E. Kostelich,et al.  To mask or not to mask: Modeling the potential for face mask use by the general public to curtail the COVID-19 pandemic , 2020, Infectious Disease Modelling.

[31]  Feiyue Huang,et al.  CurricularFace: Adaptive Curriculum Learning Loss for Deep Face Recognition , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Dahua Lin,et al.  Learning to Cluster Faces via Confidence and Connectivity Estimation , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Kaiming He,et al.  Designing Network Design Spaces , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Dong Cao,et al.  Domain Balancing: Face Recognition on Long-Tailed Domains , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Hao Wu,et al.  Masked Face Recognition Dataset and Application , 2020, ArXiv.

[36]  Anil K. Jain,et al.  Towards Universal Representation Learning for Deep Face Recognition , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Shifeng Zhang,et al.  Mis-classified Vector Guided Softmax Loss for Face Recognition , 2019, AAAI.

[38]  Weihong Deng,et al.  Mitigating Bias in Face Recognition Using Skewness-Aware Reinforcement Learning , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Sixue Gong,et al.  Jointly De-Biasing Face Recognition and Demographic Attribute Estimation , 2019, ECCV.

[40]  Quoc V. Le,et al.  Self-Training With Noisy Student Improves ImageNet Classification , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Debing Zhang,et al.  Lightweight Face Recognition Challenge , 2019, 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).

[42]  Hailin Shi,et al.  Co-Mining: Deep Face Recognition With Noisy Labels , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[43]  Mei Wang,et al.  Fair Loss: Margin-Aware Reinforcement Learning for Deep Face Recognition , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[44]  Wei Liu,et al.  Occlusion Robust Face Recognition Based on Mask Learning With Pairwise Differential Siamese Network , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[45]  Weihong Deng,et al.  Unequal-Training for Deep Face Recognition With Long-Tailed Noisy Data , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[46]  Kai Zhao,et al.  RegularFace: Deep Face Recognition via Exclusive Regularization , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[47]  Xiangyu Zhu,et al.  AdaptiveFace: Adaptive Margin and Sampling for Face Recognition , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[48]  Jiwen Lu,et al.  UniformFace: Learning Deep Equidistributed Representation for Face Recognition , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[49]  Quoc V. Le,et al.  EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks , 2019, ICML.

[50]  Xiaogang Wang,et al.  P2SGrad: Refined Gradients for Optimizing Deep Face Models , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[51]  Kan Chen,et al.  Billion-scale semi-supervised learning for image classification , 2019, ArXiv.

[52]  Xiaogang Wang,et al.  AdaCos: Adaptively Scaling Cosine Logits for Effectively Learning Deep Face Representations , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[53]  Anil K. Jain,et al.  Probabilistic Face Embeddings , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[54]  Hao Wang,et al.  Decorrelated Adversarial Learning for Age-Invariant Face Recognition , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[55]  Lei Yang,et al.  Learning to Cluster Faces on an Affinity Graph , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[56]  Fan Zhang,et al.  Noise-Tolerant Paradigm for Training Face Recognition CNNs , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[57]  Mei Wang,et al.  Racial Faces in the Wild: Reducing Racial Bias by Information Maximization Adaptation Network , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[58]  Jian Liu,et al.  iQIYI-VID: A Large Dataset for Multi-modal Person Identification , 2018, ArXiv.

[59]  Fei Wang,et al.  The Devil of Face Recognition is in the Noise , 2018, ECCV.

[60]  Fang Zhao,et al.  Towards Pose Invariant Face Recognition in the Wild , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[61]  Yang Liu,et al.  MobileFaceNets: Efficient CNNs for Accurate Real-time Face Verification on Mobile Devices , 2018, CCBR.

[62]  Anil K. Jain,et al.  IARPA Janus Benchmark - C: Face Dataset and Protocol , 2018, 2018 International Conference on Biometrics (ICB).

[63]  Xing Ji,et al.  CosFace: Large Margin Cosine Loss for Deep Face Recognition , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[64]  Stefanos Zafeiriou,et al.  ArcFace: Additive Angular Margin Loss for Deep Face Recognition , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[65]  Jian Cheng,et al.  Additive Margin Softmax for Face Verification , 2018, IEEE Signal Processing Letters.

[66]  Lei Yang,et al.  Accelerated Training for Massive Classification via Dynamic Class Selection , 2018, AAAI.

[67]  Omkar M. Parkhi,et al.  VGGFace2: A Dataset for Recognising Faces across Pose and Age , 2017, 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018).

[68]  Gang Sun,et al.  Squeeze-and-Excitation Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[69]  Weihong Deng,et al.  Cross-Age LFW: A Database for Studying Cross-Age Face Recognition in Unconstrained Environments , 2017, ArXiv.

[70]  Stefanos Zafeiriou,et al.  AgeDB: The First Manually Collected, In-the-Wild Age Database , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[71]  Stefanos Zafeiriou,et al.  Marginal Loss for Deep Face Recognition , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[72]  Kaiming He,et al.  Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour , 2017, ArXiv.

[73]  Ira Kemelmacher-Shlizerman,et al.  Level Playing Field for Million Scale Face Recognition , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[74]  Bhiksha Raj,et al.  SphereFace: Deep Hypersphere Embedding for Face Recognition , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[75]  Xiaogang Wang,et al.  Residual Attention Network for Image Classification , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[76]  Jian Cheng,et al.  NormFace: L2 Hypersphere Embedding for Face Verification , 2017, ACM Multimedia.

[77]  Bo Chen,et al.  MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications , 2017, ArXiv.

[78]  Kihyuk Sohn,et al.  Improved Deep Metric Learning with Multi-class N-pair Loss Objective , 2016, NIPS.

[79]  Zhuowen Tu,et al.  Aggregated Residual Transformations for Deep Neural Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[80]  Carlos D. Castillo,et al.  UMDFaces: An annotated face dataset for training deep networks , 2016, 2017 IEEE International Joint Conference on Biometrics (IJCB).

[81]  Yu Qiao,et al.  A Discriminative Feature Learning Approach for Deep Face Recognition , 2016, ECCV.

[82]  Yuxiao Hu,et al.  MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition , 2016, ECCV.

[83]  Anil K. Jain,et al.  Clustering Millions of Faces by Identity , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[84]  Carlos D. Castillo,et al.  Frontal to profile face verification in the wild , 2016, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).

[85]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[86]  Ira Kemelmacher-Shlizerman,et al.  The MegaFace Benchmark: 1 Million Faces for Recognition at Scale , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[87]  Tieniu Tan,et al.  A Light CNN for Deep Face Representation With Noisy Labels , 2015, IEEE Transactions on Information Forensics and Security.

[88]  James Philbin,et al.  FaceNet: A unified embedding for face recognition and clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[89]  David A. Shamma,et al.  YFCC100M , 2015, Commun. ACM.

[90]  Xiaogang Wang,et al.  DeepID3: Face Recognition with Very Deep Neural Networks , 2015, ArXiv.

[91]  Xiaogang Wang,et al.  Deeply learned face representations are sparse, selective, and robust , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[92]  Shengcai Liao,et al.  Learning Face Representation from Scratch , 2014, ArXiv.

[93]  Stefan Winkler,et al.  A data-driven approach to cleaning large face datasets , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[94]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[95]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[96]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[97]  Xiaogang Wang,et al.  Deep Learning Face Representation from Predicting 10,000 Classes , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[98]  Ming Yang,et al.  Web-scale training for face identification , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[99]  Xiaogang Wang,et al.  Deep Learning Face Representation by Joint Identification-Verification , 2014, NIPS.

[100]  Ming Yang,et al.  DeepFace: Closing the Gap to Human-Level Performance in Face Verification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[101]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[102]  Tal Hassner,et al.  Face recognition in unconstrained videos with matched background similarity , 2011, CVPR 2011.

[103]  Marwan Mattar,et al.  Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments , 2008 .

[104]  Yann LeCun,et al.  Learning a similarity metric discriminatively, with application to face verification , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[105]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[106]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[107]  Stefanos Zafeiriou,et al.  Sub-center ArcFace: Boosting Face Recognition by Large-Scale Noisy Web Faces , 2020, ECCV.

[108]  Weihong Deng,et al.  Cross-Pose LFW : A Database for Studying Cross-Pose Face Recognition in Unconstrained Environments , 2018 .

[109]  B. ackground.  TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE , 2018 .

[110]  Andrew Zisserman,et al.  Deep Face Recognition , 2015, BMVC.

[111]  A. Krizhevsky.  ImageNet Classification with Deep Convolutional Neural Networks , 2022 .