Fairness in Biometrics: A Figure of Merit to Assess Biometric Verification Systems

Machine learning-based (ML) systems are being largely deployed since the last decade in a myriad of scenarios impacting several instances in our daily lives. With this vast sort of applications, aspects of fairness start to rise in the spotlight due to the social impact that this can get in minorities. In this work aspects of fairness in biometrics are addressed. First, we introduce the first figure of merit that is able to evaluate and compare fairness aspects between multiple biometric verification systems, the so-called Fairness Discrepancy Rate (FDR). A use case with two synthetic biometric systems is introduced and demonstrates the potential of this figure of merit in extreme cases of fair and unfair behavior. Second, a use case using face biometrics is presented where several systems are evaluated compared with this new figure of merit using three public datasets exploring gender and race demographics.

[1]  R. Malpass,et al.  Recognition for faces of own and other race. , 1969, Journal of personality and social psychology.

[2]  James Philbin,et al.  FaceNet: A unified embedding for face recognition and clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Sixue Gong,et al.  DebFace: De-biasing Face Recognition , 2019, ArXiv.

[4]  Sébastien Marcel,et al.  Periocular biometrics in mobile environment , 2015, 2015 IEEE 7th International Conference on Biometrics Theory, Applications and Systems (BTAS).

[5]  Karl Ricanek,et al.  MORPH: a longitudinal image database of normal adult age-progression , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[6]  Max Welling,et al.  The Variational Fair Autoencoder , 2015, ICLR.

[7]  Anil K. Jain,et al.  IARPA Janus Benchmark - C: Face Dataset and Protocol , 2018, 2018 International Conference on Biometrics (ICB).

[8]  Toniann Pitassi,et al.  Learning Fair Representations , 2013, ICML.

[9]  Krishna P. Gummadi,et al.  Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatment , 2016, WWW.

[10]  Yun Fu,et al.  Face Recognition: Too Bias, or Not Too Bias? , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[11]  Kevin Bowyer,et al.  Characterizing the Variability in Face Recognition Accuracy Relative to Race , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[12]  Nicholas M. Orlans,et al.  NIST Special Databse 32 - Multiple Encounter Dataset II (MEDS-II) , 2011 .

[13]  Mingliang Chen,et al.  Towards Threshold Invariant Fair Classification , 2020, UAI.

[14]  Josef Kittler,et al.  Group-specific score normalization for biometric systems , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[15]  Harry Wechsler,et al.  Face Verification Subject to Varying (Age, Ethnicity, and Gender)Demographics Using Deep Learning , 2016 .

[16]  Yuxiao Hu,et al.  MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition , 2016, ECCV.

[17]  Stefan Bauer,et al.  On the Fairness of Disentangled Representations , 2019, NeurIPS.

[18]  Weihong Deng,et al.  Mitigate Bias in Face Recognition using Skewness-Aware Reinforcement Learning , 2019, ArXiv.

[19]  Dana Michalski,et al.  The Impact of Age and Threshold Variation on Facial Recognition Algorithm Performance Using Images of Children , 2018, 2018 International Conference on Biometrics (ICB).

[20]  Mei Wang,et al.  Racial Faces in the Wild: Reducing Racial Bias by Information Maximization Adaptation Network , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[21]  John J. Howard,et al.  Demographic Effects in Facial Recognition and Their Dependence on Image Acquisition: An Evaluation of Eleven Commercial Systems , 2019, IEEE Transactions on Biometrics, Behavior, and Identity Science.

[22]  Nathan Srebro,et al.  Equality of Opportunity in Supervised Learning , 2016, NIPS.

[23]  Carlos Eduardo Scheidegger,et al.  Certifying and Removing Disparate Impact , 2014, KDD.

[24]  Rolf P. Würtz,et al.  Face Recognition with Disparity Corrected Gabor Phase Differences , 2012, ICANN.

[25]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[26]  Carlos D. Castillo,et al.  Accuracy comparison across face recognition algorithms: Where are we on measuring race bias? , 2019, ArXiv.

[27]  Tony Doyle,et al.  Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy , 2017, Inf. Soc..

[28]  Tiago de Freitas Pereira.  Learning How To Recognize Faces In Heterogeneous Environments , 2019 .