Robustness Disparities in Commercial Face Detection

Facial detection and analysis systems have been deployed by large companies and 1 critiqued by scholars and activists for the past decade. Critiques that focus on 2 system performance analyze disparity of the system’s output, i.e., how frequently is 3 a face detected for different Fitzpatrick skin types or perceived genders. However, 4 we focus on the robustness of these system outputs under noisy natural perturba5 tions. We present the first of its kind detailed benchmark of the robustness of two 6 such systems: Amazon Rekognition and Microsoft Azure. We use both standard 7 and recently released academic facial datasets to quantitatively analyze trends in 8 robustness for each. Qualitatively across all the datasets and systems, we find that 9 photos of individuals who are older, masculine presenting, of darker skin type, or 10 have dim lighting are more susceptible to errors than their counterparts in other 11 identities. 12

[1]  Sebastian Benthall,et al.  Racial categories in machine learning , 2018, FAT.

[2]  Vitaly Shmatikov,et al.  Can we still avoid automatic face detection? , 2016, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).

[3]  Thomas G. Dietterich,et al.  Benchmarking Neural Network Robustness to Common Corruptions and Perturbations , 2018, ICLR.

[4]  Micah Goldblum,et al.  LowKey: Leveraging Adversarial Attacks to Protect Social Media Users from Facial Recognition , 2021, ICLR.

[5]  Emily Denton,et al.  Towards a critical race methodology in algorithmic fairness , 2019, FAT*.

[6]  Nathan Srebro,et al.  Equality of Opportunity in Supervised Learning , 2016, NIPS.

[7]  Zhe Zhao,et al.  Data Decisions and Theoretical Implications when Adversarially Learning Fair Representations , 2017, ArXiv.

[8]  Morgan Klaus Scheuerman,et al.  Gender Recognition or Gender Reductionism?: The Social Implications of Embedded Gender Recognition Systems , 2018, CHI.

[9]  Juan Carlos Niebles,et al.  Representation Learning with Statistical Independence to Mitigate Bias. , 2019 .

[10]  Shai Ben-David,et al.  Empirical Risk Minimization under Fairness Constraints , 2018, NeurIPS.

[11]  Takeo Igarashi,et al.  Exploring a Makeup Support System for Transgender Passing based on Automatic Gender Recognition , 2021, CHI.

[12]  Boi Faltings,et al.  Non-Discriminatory Machine Learning through Convex Fairness Criteria , 2018, AAAI.

[13]  T. Fitzpatrick.  The validity and practicality of sun-reactive skin types I through VI. , 1988, Archives of dermatology.

[14]  Olga Russakovsky,et al.  Towards Fairness in Visual Recognition: Effective Strategies for Bias Mitigation , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Toniann Pitassi,et al.  Learning Adversarially Fair and Transferable Representations , 2018, ICML.

[16]  Sahil Singla,et al.  Fairness Through Robustness: Investigating Robustness Disparity in Deep Learning , 2021, FAccT.

[17]  Bruce A. Draper,et al.  An introduction to the good, the bad, & the ugly face recognition challenge problem , 2011, Face and Gesture 2011.

[18]  Inioluwa Deborah Raji,et al.  Actionable Auditing: Investigating the Impact of Publicly Naming Biased Performance Results of Commercial AI Products , 2019, AIES.

[19]  Radha Poovendran,et al.  Google's Cloud Vision API is Not Robust to Noise , 2017, 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA).

[20]  Harry Wechsler,et al.  Face Verification Subject to Varying (Age, Ethnicity, and Gender)Demographics Using Deep Learning , 2016 .

[21]  Caroline Pantofaru,et al.  A Step Toward More Inclusive People Annotations for Fairness , 2021, AIES.

[22]  Alice J. O'Toole,et al.  Demographic effects on estimates of automatic face recognition performance , 2011, Face and Gesture 2011.

[23]  White Guy.  Facial Recognition Is Accurate, if You’re a White Guy , 2018 .

[24]  Yang Song,et al.  Age Progression/Regression by Conditional Adversarial Autoencoder , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Ed H. Chi,et al.  Fairness without Demographics through Adversarially Reweighted Learning , 2020, NeurIPS.

[26]  Timnit Gebru,et al.  Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification , 2018, FAT.

[27]  Mayank Vatsa,et al.  On the Robustness of Face Recognition Algorithms Against Attacks and Bias , 2020, AAAI.

[28]  Alex A. Ahmed.  Bridging Social Critique and Design: Building a Health Informatics Tool for Transgender Voice , 2019, CHI Extended Abstracts.

[29]  Aaron Roth,et al.  Convergent Algorithms for (Relaxed) Minimax Fairness , 2020, ArXiv.

[30]  Alexandra Chouldechova,et al.  The Frontiers of Fairness in Machine Learning , 2018, ArXiv.

[31]  Albert Gordo,et al.  Towards Measuring Fairness in AI: The Casual Conversations Dataset , 2021, IEEE Transactions on Biometrics, Behavior, and Identity Science.

[32]  John Langford,et al.  A Reductions Approach to Fair Classification , 2018, ICML.

[33]  Hee Jung Ryu,et al.  InclusiveFaceNet: Improving Face Attribute Detection with Race and Gender Diversity , 2017 .

[34]  Guillermo Sapiro,et al.  Minimax Pareto Fairness: A Multi Objective Perspective , 2020, ICML.

[35]  Jieyu Zhao,et al.  Balanced Datasets Are Not Enough: Estimating and Mitigating Gender Bias in Deep Image Representations , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[36]  Krishna P. Gummadi,et al.  Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatment , 2016, WWW.

[37]  Toniann Pitassi,et al.  Fairness through awareness , 2011, ITCS '12.

[38]  Toniann Pitassi,et al.  Learning Fair Representations , 2013, ICML.

[39]  Alice J. O'Toole,et al.  FRVT 2006 and ICE 2006 large-scale results , 2007 .

[41]  Tal Hassner,et al.  Age and Gender Estimation of Unfiltered Faces , 2014, IEEE Transactions on Information Forensics and Security.

[42]  Krishna P. Gummadi,et al.  Fairness Constraints: A Flexible Approach for Fair Classification , 2019, J. Mach. Learn. Res..

[43]  Anil K. Jain,et al.  Face Recognition Performance: Role of Demographic Information , 2012, IEEE Transactions on Information Forensics and Security.

[44]  Weihong Deng,et al.  Mitigating Bias in Face Recognition Using Skewness-Aware Reinforcement Learning , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Ben Y. Zhao,et al.  Fawkes: Protecting Privacy against Unauthorized Deep Learning Models , 2020, USENIX Security Symposium.

[46]  Sujit Gujar,et al.  FNNC: Achieving Fairness through Neural Networks , 2018, IJCAI.

[47]  John J. Howard,et al.  Demographic Effects in Facial Recognition and Their Dependence on Image Acquisition: An Evaluation of Eleven Commercial Systems , 2019, IEEE Transactions on Biometrics, Behavior, and Identity Science.

[48]  Carlos Eduardo Scheidegger,et al.  Certifying and Removing Disparate Impact , 2014, KDD.

[49]  Anil K. Jain,et al.  A longitudinal study of automatic face recognition , 2015, 2015 International Conference on Biometrics (ICB).

[50]  Krishna P. Gummadi,et al.  Fairness Constraints: Mechanisms for Fair Classification , 2015, AISTATS.

[51]  Amos J. Storkey,et al.  Censoring Representations with an Adversary , 2015, ICLR.

[52]  Oliver Thomas,et al.  Discovering Fair Representations in the Data Domain , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).