Measuring the Gender and Ethnicity Bias in Deep Models for Face Recognition

We explore the importance of gender and ethnic attributes in the decision-making of face recognition technologies. Our work is in part motivated by the new European regulation for personal data protection, which forces data controllers to avoid discriminative hazards while managing sensitive data like biometric data. The experiments in this paper are aimed to study what extent sensitive data like gender or ethnic origin attributes are present in the most common face recognition networks. For this, our experiments include two popular pre-trained networks: VGGFace and Resnet50. Both pre-trained models are able to classify gender and ethnicity easily (over 95% of performance) even suppressing 80% of the neurons in their embedding layers. The experimentation is conducted on a publicly available database known as Labeled Faces in the Wild with more than 13000 images of faces with a huge range of poses, ages, races and nationalities.

[1]  Julian Fiérrez,et al.  Multi-biometric template protection based on Homomorphic Encryption , 2017, Pattern Recognit..

[2]  Arun Ross,et al.  What Else Does Your Biometric Data Reveal? A Survey on Soft Biometrics , 2016, IEEE Transactions on Information Forensics and Security.

[3]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  James Philbin,et al.  FaceNet: A unified embedding for face recognition and clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Omkar M. Parkhi,et al.  VGGFace2: A Dataset for Recognising Faces across Pose and Age , 2017, 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018).

[6]  Julian Fiérrez,et al.  Multiple classifiers in biometrics. part 1: Fundamentals and review , 2018, Inf. Fusion.

[7]  Julian Fiérrez,et al.  Soft Biometrics and Their Application in Person Recognition at a Distance , 2014, IEEE Transactions on Information Forensics and Security.

[8]  Anil K. Jain,et al.  Pushing the frontiers of unconstrained face detection and recognition: IARPA Janus Benchmark A , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Julian Fierrez,et al.  Facial Soft Biometrics for Recognition in the Wild: Recent Works, Annotation, and COTS Evaluation , 2018, IEEE Transactions on Information Forensics and Security.

[10]  Andrew Zisserman,et al.  Deep Face Recognition , 2015, BMVC.

[11]  Julian Fierrez,et al.  Facial soft biometric features for forensic face recognition. , 2015, Forensic science international.

[12]  Seth Flaxman,et al.  European Union Regulations on Algorithmic Decision-Making and a "Right to Explanation" , 2016, AI Mag..

[13]  Arun Ross,et al.  Semi-adversarial Networks: Convolutional Autoencoders for Imparting Privacy to Face Images , 2017, 2018 International Conference on Biometrics (ICB).

[14]  Hugo Proença,et al.  Biometric recognition in surveillance scenarios: a survey , 2016, Artificial Intelligence Review.

[15]  Honglak Lee,et al.  Learning to Align from Scratch , 2012, NIPS.

[16]  Carlos D. Castillo,et al.  Deep Learning for Understanding Faces: Machines May Be Just as Good, or Better, than Humans , 2018, IEEE Signal Processing Magazine.

[17]  Sharath Pankanti,et al.  Biometric Recognition: Security and Privacy Concerns , 2003, IEEE Secur. Priv..

[18]  Julian Fiérrez,et al.  Multiple classifiers in biometrics. Part 2: Trends and challenges , 2018, Inf. Fusion.

[19]  Julian Fierrez,et al.  EXPLORING FACIAL REGIONS IN UNCONSTRAINED SCENARIOS : EXPERIENCE ON ICB-RW , 2018 .