Fair Normalizing Flows

Fair representation learning is an attractive approach that promises fairness of downstream predictors by encoding sensitive data. Unfortunately, recent work has shown that strong adversarial predictors can still exhibit unfairness by recovering sensitive attributes from these representations. In this work, we present Fair Normalizing Flows (FNF), a new approach offering more rigorous fairness guarantees for learned representations. Specifically, we consider a practical setting where we can estimate the probability density for sensitive groups. The key idea is to model the encoder as a normalizing flow trained to minimize the statistical distance between the latent representations of different groups. The main advantage of FNF is that its exact likelihood computation allows us to obtain guarantees on the maximum unfairness of any potentially adversarial downstream predictor. We experimentally demonstrate the effectiveness of FNF in enforcing various group fairness notions, as well as other attractive properties such as interpretability and transfer learning, on a variety of challenging real-world datasets.

[1]  Vishnu Naresh Boddeti,et al.  Mitigating Information Leakage in Image Representations: A Maximum Entropy Approach , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Stefano Ermon,et al.  A Theory of Usable Information Under Computational Constraints , 2020, ICLR.

[3]  Junping Du,et al.  Anomaly Detection Using Local Kernel Density Estimation and Context-Based Regression , 2020, IEEE Transactions on Knowledge and Data Engineering.

[4]  Stanislav Pidhorskyi,et al.  Generative Probabilistic Novelty Detection with Adversarial Autoencoders , 2018, NeurIPS.

[5]  Yoav Goldberg,et al.  Adversarial Removal of Demographic Attributes from Text Data , 2018, EMNLP.

[6]  Suleyman S. Kozat,et al.  Online Anomaly Detection With Minimax Optimal Density Estimation in Nonstationary Environments , 2018, IEEE Transactions on Signal Processing.

[7]  Greg Ver Steeg,et al.  Controllable Guarantees for Fair Outcomes via Contrastive Information Estimation , 2021, AAAI.

[8]  Koray Kavukcuoglu,et al.  Pixel Recurrent Neural Networks , 2016, ICML.

[9]  Hugo Larochelle,et al.  MADE: Masked Autoencoder for Distribution Estimation , 2015, ICML.

[10]  Graham Neubig,et al.  Controllable Invariance through Adversarial Feature Learning , 2017, NIPS.

[11]  Toniann Pitassi,et al.  Learning Fair Representations , 2013, ICML.

[12]  Geoff Gordon,et al.  Inherent Tradeoffs in Learning Fair Representations , 2019, NeurIPS.

[13]  Cheng Soon Ong,et al.  Costs and Benefits of Fair Representation Learning , 2019, AIES.

[14]  Jette Henderson,et al.  CERTIFAI: A Common Framework to Provide Explanations and Analyse the Fairness and Robustness of Black-box Models , 2020, AIES.

[15]  Novi Quadrianto,et al.  Null-sampling for Interpretable and Fair Representations , 2020, ECCV.

[16]  Stefan Bauer,et al.  On the Fairness of Disentangled Representations , 2019, NeurIPS.

[17]  Peter Kairouz,et al.  Learning Generative Adversarial RePresentations (GAP) under Fairness and Censoring Constraints , 2019, ArXiv.

[18]  Stefano Ermon,et al.  Learning Controllable Fair Representations , 2018, AISTATS.

[19]  Cynthia Dwork,et al.  Calibrating Noise to Sensitivity in Private Data Analysis , 2006, TCC.

[20]  Aaron Roth,et al.  Differentially Private Fair Learning , 2018, ICML.

[21]  Salvatore J. Stolfo,et al.  A comparative evaluation of two algorithms for Windows Registry Anomaly Detection , 2005, J. Comput. Secur..

[22]  Toniann Pitassi,et al.  Learning Adversarially Fair and Transferable Representations , 2018, ICML.

[23]  Pascal Vincent,et al.  A Closer Look at the Optimization Landscapes of Generative Adversarial Networks , 2019, ICLR.

[24]  Vitaly Shmatikov,et al.  Overlearning Reveals Sensitive Attributes , 2019, ICLR.

[25]  Tapani Raiko,et al.  Semi-supervised detection of collective anomalies with an application in high energy particle physics , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[26]  Rishabh Singh,et al.  Interpreting Neural Network Judgments via Minimal, Stable, and Symbolic Corrections , 2018, NeurIPS.

[27]  Mark Alfano,et al.  The philosophical basis of algorithmic recourse , 2020, FAT*.

[28]  T. Raiko,et al.  Novelty detection by nonlinear factor analysis for structural health monitoring , 2010, 2010 IEEE International Workshop on Machine Learning for Signal Processing.

[29]  Moritz Hardt,et al.  Tight Bounds for Learning a Mixture of Two Gaussians , 2014, STOC.

[30]  Krishna P. Gummadi,et al.  Fairness Constraints: Mechanisms for Fair Classification , 2015, AISTATS.

[31]  B. Nachman,et al.  Anomaly detection with density estimation , 2020, Physical Review D.

[32]  T. Brennan,et al.  Evaluating the Predictive Validity of the Compas Risk and Needs Assessment System , 2009 .

[33]  Amos J. Storkey,et al.  Censoring Representations with an Adversary , 2015, ICLR.

[34]  Max Welling,et al.  The Variational Fair Autoencoder , 2015, ICLR.

[35]  Shakir Mohamed,et al.  Variational Inference with Normalizing Flows , 2015, ICML.

[36]  Samy Bengio,et al.  Density estimation using Real NVP , 2016, ICLR.

[37]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[38]  Yang Liu,et al.  Actionable Recourse in Linear Classification , 2018, FAT.

[39]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[40]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[41]  Jun Sakuma,et al.  Fairness-aware Learning through Regularization Approach , 2011, 2011 IEEE 11th International Conference on Data Mining Workshops.

[42]  Armando Solar-Lezama,et al.  Probabilistic verification of fairness properties via concentration , 2018, Proc. ACM Program. Lang..

[43]  Yoshua Bengio,et al.  NICE: Non-linear Independent Components Estimation , 2014, ICLR.

[44]  David A. Clifton,et al.  A Framework for Novelty Detection in Jet Engine Vibration Data , 2007 .

[45]  Heiga Zen,et al.  WaveNet: A Generative Model for Raw Audio , 2016, SSW.

[46]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .

[47]  Andrew D. Selbst,et al.  Big Data's Disparate Impact , 2016 .

[48]  Bernhard Schölkopf,et al.  Algorithmic Recourse: from Counterfactual Explanations to Interventions , 2020, FAccT.

[49]  Premkumar Natarajan,et al.  Invariant Representations through Adversarial Forgetting , 2019, AAAI.

[50]  Toniann Pitassi,et al.  Flexibly Fair Representation Learning by Disentanglement , 2019, ICML.

[51]  J. Zico Kolter,et al.  Learning perturbation sets for robust machine learning , 2020, ICLR.

[52]  Kristina Lerman,et al.  Learning Fair and Interpretable Representations via Linear Orthogonalization , 2019, ArXiv.

[53]  Benny Pinkas,et al.  Fairness in the Eyes of the Data: Certifying Machine-Learning Models , 2020, AIES.

[54]  Aws Albarghouthi,et al.  FairSquare: probabilistic verification of program fairness , 2017, Proc. ACM Program. Lang..

[55]  Linda F. Wightman.  LSAC National Longitudinal Bar Passage Study. LSAC Research Report Series. , 1998 .

[56]  Aditya Krishna Menon,et al.  The cost of fairness in binary classification , 2018, FAT.

[57]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[58]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[59]  Yizhou Sun,et al.  Learning Fair Representations via an Adversarial Framework , 2019, ArXiv.

[60]  Caterina Urban,et al.  Perfectly parallel fairness certification of neural networks , 2020, Proc. ACM Program. Lang..

[61]  Yuxin Ding,et al.  Host-based intrusion detection using dynamic and static behavioral models , 2003, Pattern Recognit..

[62]  Toniann Pitassi,et al.  Fairness through awareness , 2011, ITCS '12.

[63]  Heinrich Jiang,et al.  Uniform Convergence Rates for Kernel Density Estimation , 2017, ICML.

[64]  Rob Brekelmans,et al.  Invariant Representations without Adversarial Training , 2018, NeurIPS.

[65]  Sameer Singh,et al.  An approach to novelty detection applied to the classification of image regions , 2004, IEEE Transactions on Knowledge and Data Engineering.

[66]  P. Angelov,et al.  A fast approach to novelty detection in video streams using recursive density estimation , 2008, 2008 4th International IEEE Conference Intelligent Systems.

[67]  Diptikalyan Saha,et al.  Verifying Individual Fairness in Machine Learning Models , 2020, UAI.

[68]  Nathan Srebro,et al.  Equality of Opportunity in Supervised Learning , 2016, NIPS.

[69]  Miguel Nicolau,et al.  One-Class Classification for Anomaly Detection with Kernel Density Estimation and Genetic Programming , 2016, EuroGP.

[70]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[71]  Prafulla Dhariwal,et al.  Glow: Generative Flow with Invertible 1x1 Convolutions , 2018, NeurIPS.

[72]  Alex Graves,et al.  Conditional Image Generation with PixelCNN Decoders , 2016, NIPS.

[73]  Peter A. Flach,et al.  FACE: Feasible and Actionable Counterfactual Explanations , 2020, AIES.

[74]  A. Lo,et al.  Consumer Credit Risk Models Via Machine-Learning Algorithms , 2010 .

[75]  Martin Vechev,et al.  Learning Certified Individually Fair Representations , 2020, NeurIPS.