Towards Gender-Neutral Face Descriptors for Mitigating Bias in Face Recognition

State-of-the-art deep networks implicitly encode gender information while being trained for face recognition. Gender is often viewed as an important attribute with respect to identifying faces. However, the implicit encoding of gender information in face descriptors has two major issues: (a.) It makes the descriptors susceptible to privacy leakage, i.e. a malicious agent can be trained to predict the face gender from such descriptors. (b.) It appears to contribute to gender bias in face recognition, i.e. we find a significant difference in the recognition accuracy of DCNNs on male and female faces. Therefore, we present a novel `Adversarial Gender De-biasing algorithm (AGENDA)' to reduce the gender information present in face descriptors obtained from previously trained face recognition networks. We show that AGENDA significantly reduces gender predictability of face descriptors. Consequently, we are also able to reduce gender bias in face verification while maintaining reasonable recognition performance.

[1]  Carlos D. Castillo,et al.  Accuracy comparison across face recognition algorithms: Where are we on measuring race bias? , 2019, ArXiv.

[2]  Kevin W. Bowyer,et al.  Issues Related to Face Recognition Accuracy Varying Based on Race and Skin Tone , 2020, IEEE Transactions on Technology and Society.

[3]  Ang Li,et al.  DeepObfuscator: Adversarial Training Framework for Privacy-Preserving Image Classification , 2019, ArXiv.

[4]  Kai Zhang,et al.  How Does Gender Balance In Training Data Affect Face Recognition Accuracy? , 2020, 2020 IEEE International Joint Conference on Biometrics (IJCB).

[5]  Zhenyu Wu,et al.  Towards Privacy-Preserving Visual Recognition via Adversarial Training: A Pilot Study , 2018, ECCV.

[6]  P. Drozdowski,et al.  Demographic Bias in Biometrics: A Survey on an Emerging Challenge , 2020, IEEE Transactions on Technology and Society.

[7]  Timnit Gebru,et al.  Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification , 2018, FAT.

[8]  Anil K. Jain,et al.  Ethnicity identification from face images , 2004, SPIE Defense + Commercial Sensing.

[9]  James Philbin,et al.  FaceNet: A unified embedding for face recognition and clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[11]  Yun Fu,et al.  Age Synthesis and Estimation via Faces: A Survey , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Carlos D. Castillo,et al.  An All-In-One Convolutional Neural Network for Face Analysis , 2016, 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017).

[13]  Anil K. Jain,et al.  IARPA Janus Benchmark - C: Face Dataset and Protocol , 2018, 2018 International Conference on Biometrics (ICB).

[14]  Roope Raisamo,et al.  Evaluation of Gender Classification Methods with Automatically Detected and Aligned Faces , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Jieyu Zhao,et al.  Balanced Datasets Are Not Enough: Estimating and Mitigating Gender Bias in Deep Image Representations , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[16]  Anil K. Jain,et al.  Face Recognition Performance: Role of Demographic Information , 2012, IEEE Transactions on Information Forensics and Security.

[17]  Kevin W. Bowyer,et al.  Is Face Recognition Sexist? No, Gendered Hairstyles and Biology Are , 2020, BMVC.

[18]  Ming Yang,et al.  DeepFace: Closing the Gap to Human-Level Performance in Face Verification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Yuxiao Hu,et al.  MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition , 2016, ECCV.

[20]  Carlos D. Castillo,et al.  Deep Features for Recognizing Disguised Faces in the Wild , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[21]  Kevin Bowyer,et al.  Characterizing the Variability in Face Recognition Accuracy Relative to Race , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[22]  Maja Pantic,et al.  Investigating Bias in Deep Face Analysis: The KANFace Dataset and Empirical Study , 2020, Image Vis. Comput..

[23]  Carlos D. Castillo,et al.  The Do’s and Don’ts for CNN-Based Face Verification , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[24]  Sepp Hochreiter,et al.  Self-Normalizing Neural Networks , 2017, NIPS.

[25]  Stefanos Zafeiriou,et al.  ArcFace: Additive Angular Margin Loss for Deep Face Recognition , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Xiaogang Wang,et al.  Deep Learning Face Attributes in the Wild , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[27]  Carlos D. Castillo,et al.  Triplet probabilistic embedding for face verification and clustering , 2016, 2016 IEEE 8th International Conference on Biometrics Theory, Applications and Systems (BTAS).

[28]  Kevin W. Bowyer,et al.  Analysis of Gender Inequality In Face Recognition Accuracy , 2020, 2020 IEEE Winter Applications of Computer Vision Workshops (WACVW).

[29]  Arun Ross,et al.  Privacy of Facial Soft Biometrics: Suppressing Gender But Retaining Identity , 2014, ECCV Workshops.

[30]  Kush R. Varshney,et al.  Fairness GAN , 2018, IBM J. Res. Dev..

[31]  Alex Pentland,et al.  Face recognition using eigenfaces , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[32]  Blake Lemoine,et al.  Mitigating Unwanted Biases with Adversarial Learning , 2018, AIES.

[33]  Carlos D. Castillo,et al.  How are attributes expressed in face DCNNs? , 2020, 2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020).

[34]  Carlos D. Castillo,et al.  UMDFaces: An annotated face dataset for training deep networks , 2016, 2017 IEEE International Joint Conference on Biometrics (IJCB).

[35]  Oliver Thomas,et al.  Discovering Fair Representations in the Data Domain , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Connor J. Parde,et al.  Deep convolutional neural networks in the face of caricature , 2018, Nature Machine Intelligence.

[37]  Arun Ross,et al.  Soft biometric privacy: Retaining biometric utility of face images while perturbing gender , 2017, 2017 IEEE International Joint Conference on Biometrics (IJCB).

[38]  Richa Singh,et al.  Deep Learning for Face Recognition: Pride or Prejudiced? , 2019, ArXiv.

[39]  Connor J. Parde,et al.  Single Unit Status in Deep Convolutional Neural Network Codes for Face Identification: Sparseness Redefined , 2020, ArXiv.

[40]  Andrew Zisserman,et al.  Turning a Blind Eye: Explicit Removal of Biases and Variation from Deep Neural Network Embeddings , 2018, ECCV Workshops.

[41]  Carlos D. Castillo,et al.  A Fast and Accurate System for Face Detection, Identification, and Verification , 2018, IEEE Transactions on Biometrics, Behavior, and Identity Science.

[42]  Chen Gao,et al.  Why Can't I Dance in the Mall? Learning to Mitigate Scene Bias in Action Recognition , 2019, NeurIPS.

[43]  Mei Wang,et al.  Racial Faces in the Wild: Reducing Racial Bias by Information Maximization Adaptation Network , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[44]  Arun Ross,et al.  Gender Privacy: An Ensemble of Semi Adversarial Networks for Confounding Arbitrary Gender Classifiers , 2018, 2018 IEEE 9th International Conference on Biometrics Theory, Applications and Systems (BTAS).

[45]  Alice J. O'Toole,et al.  Face recognition algorithms and the other-race effect: computational mechanisms for a developmental contact hypothesis , 2002, Cogn. Sci..

[46]  Toniann Pitassi,et al.  Learning Adversarially Fair and Transferable Representations , 2018, ICML.

[47]  Daniela Rus,et al.  Uncovering and Mitigating Algorithmic Bias through Learned Latent Structure , 2019, AIES.