A General Framework for Fair Regression

Fairness, through its many forms and definitions, has become an important issue facing the machine learning community. In this work, we consider how to incorporate group fairness constraints into kernel regression methods, applicable to Gaussian processes, support vector machines, neural network regression and decision tree regression. Further, we focus on examining the effect of incorporating these constraints in decision tree regression, with direct applications to random forests and boosted trees amongst other widespread popular inference techniques. We show that the order of complexity of memory and computation is preserved for such models and tightly binds the expected perturbations to the model in terms of the number of leaves of the trees. Importantly, the approach works on trained models and hence can be easily applied to models in current use and group labels are only required on training data.

[1]  V. Silvério.  Affirmative action in the United States and India: a comparative perspective , 2006 .

[2]  L. Dumont.  Homo Hierarchicus: The Caste System and Its Implications , 1966 .

[3]  A. O'Hagan,et al.  Bayes–Hermite quadrature , 1991 .

[4]  Nathan Srebro,et al.  Equality of Opportunity in Supervised Learning , 2016, NIPS.

[5]  Jun Sakuma,et al.  Fairness-aware Learning through Regularization Approach , 2011, 2011 IEEE 11th International Conference on Data Mining Workshops.

[6]  Xiangliang Zhang,et al.  Decision Theory for Discrimination-Aware Classification , 2012, 2012 IEEE 12th International Conference on Data Mining.

[7]  Carl E. Rasmussen,et al.  Active Learning of Model Evidence Using Bayesian Quadrature , 2012, NIPS.

[8]  Jon M. Kleinberg,et al.  On Fairness and Calibration , 2017, NIPS.

[9]  Pierre Geurts,et al.  Extremely randomized trees , 2006, Machine Learning.

[10]  Matt J. Kusner,et al.  Counterfactual Fairness , 2017, NIPS.

[11]  Toon Calders,et al.  Classifying without discriminating , 2009, 2009 2nd International Conference on Computer, Control and Communication.

[12]  Krishna P. Gummadi,et al.  Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatment , 2016, WWW.

[13]  Matt J. Kusner,et al.  The Sensitivity of Counterfactual Fairness to Unmeasured Confounding , 2019, UAI.

[14]  Toniann Pitassi,et al.  Fairness through awareness , 2011, ITCS '12.

[15]  Thomas B. Schön,et al.  Linearly constrained Gaussian processes , 2017, NIPS.

[16]  Seth Neel,et al.  A Convex Framework for Fair Regression , 2017, ArXiv.

[17]  Philip S. Yu,et al.  Top 10 algorithms in data mining , 2007, Knowledge and Information Systems.

[18]  Adam Tauman Kalai,et al.  Decoupled Classifiers for Group-Fair and Efficient Machine Learning , 2017, FAT.

[19]  Toon Calders,et al.  Three naive Bayes approaches for discrimination-free classification , 2010, Data Mining and Knowledge Discovery.

[20]  Neil D. Lawrence,et al.  Deep Gaussian Processes , 2012, AISTATS.

[21]  Toon Calders,et al.  Discrimination Aware Decision Tree Learning , 2010, 2010 IEEE International Conference on Data Mining.

[22]  Toon Calders,et al.  Controlling Attribute Effect in Linear Regression , 2013, 2013 IEEE 13th International Conference on Data Mining.

[23]  Andreas Holzinger,et al.  Data Mining with Decision Trees: Theory and Applications , 2015, Online Inf. Rev..

[24]  J Elith,et al.  A working guide to boosted regression trees. , 2008, The Journal of animal ecology.

[25]  Seth Neel,et al.  Rawlsian Fairness for Machine Learning , 2016, ArXiv.

[26]  Aaron Roth,et al.  Equal Opportunity in Online Classification with Partial Feedback , 2019, NeurIPS.

[27]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[28]  Bernhard Schölkopf,et al.  Avoiding Discrimination through Causal Reasoning , 2017, NIPS.

[29]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[30]  Maria-Florina Balcan,et al.  Envy-Free Classification , 2018, NeurIPS.

[31]  Hany Farid,et al.  The accuracy, fairness, and limits of predicting recidivism , 2018, Science Advances.

[32]  Krishna P. Gummadi,et al.  From Parity to Preference-based Notions of Fairness in Classification , 2017, NIPS.

[33]  Steven Mills,et al.  Fair Forests: Regularized Tree Induction to Minimize Model Bias , 2017, AIES.

[34]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[35]  Shai Ben-David,et al.  Empirical Risk Minimization under Fairness Constraints , 2018, NeurIPS.

[36]  COMPAS Risk Scales : Demonstrating Accuracy Equity and Predictive Parity Performance of the COMPAS Risk Scales in Broward County , 2016 .

[37]  Franco Turini,et al.  k-NN as an implementation of situation testing for discrimination discovery and prevention , 2011, KDD.

[38]  Erez Shmueli,et al.  Algorithmic Fairness , 2020, ArXiv.

[39]  Pratik Gajane,et al.  On formalizing fairness in prediction with machine learning , 2017, ArXiv.