Fair Structure Learning in Heterogeneous Graphical Models

Inference of community structure in probabilistic graphical models may not be consistent with fairness constraints when nodes have demographic attributes. Certain demographics may be over-represented in some detected communities and under-represented in others. This paper defines a novel `1-regularized pseudo-likelihood approach for fair graphical model selection. In particular, we assume there is some community or clustering structure in the true underlying graph, and we seek to learn a sparse undirected graph and its communities from the data such that demographic groups are fairly represented within the communities. Our optimization approach uses the demographic parity definition of fairness, but the framework is easily extended to other definitions of fairness. We establish statistical consistency of the proposed method for both a Gaussian graphical model and an Ising model for, respectively, continuous and binary data, proving that our method can recover the graphs and their fair communities with high probability.

[1]  Alexandre d'Aspremont,et al.  Model Selection Through Sparse Max Likelihood Estimation Model Selection Through Sparse Maximum Likelihood Estimation for Multivariate Gaussian or Binary Data , 2022 .

[2]  A. Rinaldo,et al.  Consistency of spectral clustering in stochastic block models , 2013, 1312.2050.

[3]  Alexandra Chouldechova,et al.  The Frontiers of Fairness in Machine Learning , 2018, ArXiv.

[4]  Noureddine El Karoui,et al.  Operator norm consistent estimation of large-dimensional sparse covariance matrices , 2008, 0901.3220.

[5]  Shai Ben-David,et al.  Empirical Risk Minimization under Fairness Constraints , 2018, NeurIPS.

[6]  H. Zou,et al.  Positive Definite $\ell_1$ Penalized Estimation of Large Covariance Matrices , 2012, 1208.5702.

[7]  Romain Takeo Bouyer,et al.  Artist gender representation in music streaming , 2020, ISMIR.

[8]  Riccardo Bellazzi,et al.  Patient similarity for precision medicine: A systematic review , 2018, J. Biomed. Informatics.

[9]  Daniel Pérez Palomar,et al.  Structured Graph Learning Via Laplacian Spectral Constraints , 2019, NeurIPS.

[10]  Alexander Felfernig,et al.  Recommender Systems: An Overview , 2011, AI Mag..

[11]  K. Khare,et al.  A convex pseudolikelihood framework for high dimensional partial correlation estimation with convergence guarantees , 2013, 1307.5381.

[12]  Peng Zhao,et al.  On Model Selection Consistency of Lasso , 2006, J. Mach. Learn. Res..

[13]  N. Meinshausen,et al.  High-dimensional graphs and variable selection with the Lasso , 2006, math/0608017.

[14]  Jun Sakuma,et al.  Enhancement of the Neutrality in Recommendation , 2012, Decisions@RecSys.

[15]  L. Amaral,et al.  The web of human sexual contacts , 2001, Nature.

[16]  Jiaxi Ying,et al.  Algorithms for Learning Graphs in Financial Markets , 2020, ArXiv.

[17]  Genevera I. Allen,et al.  A Log-Linear Graphical Model for inferring genetic networks from high-throughput sequencing data , 2012, 2012 IEEE International Conference on Bioinformatics and Biomedicine.

[18]  Luca Oneto,et al.  Fairness in Machine Learning , 2020, INNSBDDL.

[19]  Kathryn B. Laskey,et al.  Stochastic blockmodels: First steps , 1983 .

[20]  M. Hassner,et al.  The use of Markov Random Fields as models of texture , 1980 .

[21]  Sandeep Kumar,et al.  A Unified Framework for Structured Graph Learning via Spectral Constraints , 2019, J. Mach. Learn. Res..

[22]  John Riedl,et al.  Item-based collaborative filtering recommendation algorithms , 2001, WWW '01.

[23]  Dean P. Foster,et al.  Clustering Methods for Collaborative Filtering , 1998, AAAI 1998.

[24]  Wotao Yin,et al.  Global Convergence of ADMM in Nonconvex Nonsmooth Optimization , 2015, Journal of Scientific Computing.

[25]  Prosenjit Gupta,et al.  Clustering-based recommender system using principles of voting theory , 2014, 2014 International Conference on Contemporary Computing and Informatics (IC3I).

[26]  J. Borwein,et al.  Two-Point Step Size Gradient Methods , 1988 .

[27]  Pei Wang,et al.  Learning networks from high dimensional binary data: An application to genomic instability data , 2009, 0908.3882.

[28]  George Michailidis,et al.  Estimation of Graphical Models through Structured Norm Minimization , 2016, J. Mach. Learn. Res..

[29]  Hinrich Schütze,et al.  Book Reviews: Foundations of Statistical Natural Language Processing , 1999, CL.

[30]  Xiaodong Li,et al.  Robust and Computationally Feasible Community Detection in the Presence of Arbitrary Outlier Nodes , 2014, ArXiv.

[31]  E. Levina,et al.  Asymptotic Properties of the Joint Neighborhood Selection Method for Estimating Categorical Markov Networks , 2012 .

[32]  Mohit Singh,et al.  Multi-Criteria Dimensionality Reduction with Applications to Fairness , 2019, NeurIPS.

[33]  Dit-Yan Yeung,et al.  Collaborative Deep Learning for Recommender Systems , 2014, KDD.

[34]  Thomas Hofmann,et al.  Efficient Structure Learning of Markov Networks using L1-Regularization , 2007 .

[35]  Lingrui Gan,et al.  Bayesian Joint Estimation of Multiple Graphical Models , 2019, NeurIPS.

[36]  Pierre Hansen,et al.  NP-hardness of Euclidean sum-of-squares clustering , 2008, Machine Learning.

[37]  E. Ceulemans,et al.  A Partial Correlation Screening Approach for Controlling the False Positive Rate in Sparse Gaussian Graphical Models , 2019, Scientific Reports.

[38]  Elena L. Glassman,et al.  Feature engineering for clustering student solutions , 2014, L@S.

[39]  Christian Haas,et al.  Fairness in Machine Learning: A Survey , 2020, ACM Comput. Surv..

[40]  Weidong Liu,et al.  Adaptive Thresholding for Sparse Covariance Matrix Estimation , 2011, 1102.2237.

[41]  E. Levina,et al.  Joint estimation of multiple graphical models. , 2011, Biometrika.

[42]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[43]  Patrick Pérez,et al.  Discrete Markov image modeling and inference on the quadtree , 2000, IEEE Trans. Image Process..

[44]  Piyush Srivastava,et al.  Exact recovery in the Ising blockmodel , 2016, The Annals of Statistics.

[45]  Jing Ma,et al.  Joint Structural Estimation of Multiple Graphical Models , 2016, J. Mach. Learn. Res..

[46]  James R. Foulds,et al.  HyPER: A Flexible and Extensible Probabilistic Framework for Hybrid Recommender Systems , 2015, RecSys.

[47]  Tengyao Wang,et al.  A useful variant of the Davis--Kahan theorem for statisticians , 2014, 1405.0680.

[48]  Amit Kumar,et al.  A simple linear time (1 + /spl epsiv/)-approximation algorithm for k-means clustering in any dimensions , 2004, 45th Annual IEEE Symposium on Foundations of Computer Science.

[49]  Qinbao Song,et al.  A Fast Clustering-Based Feature Subset Selection Algorithm for High-Dimensional Data , 2013, IEEE Transactions on Knowledge and Data Engineering.

[50]  Garry Robins,et al.  An introduction to exponential random graph (p*) models for social networks , 2007, Soc. Networks.

[51]  Florentina Bunea,et al.  High-Dimensional Inference for Cluster-Based Graphical Models , 2018, J. Mach. Learn. Res..

[52]  Pranjal Awasthi,et al.  Guarantees for Spectral Clustering with Fairness Constraints , 2019, ICML.

[53]  Ruslan Salakhutdinov,et al.  Probabilistic Matrix Factorization , 2007, NIPS.

[54]  Yehuda Koren,et al.  Collaborative filtering with temporal dynamics , 2009, KDD.

[55]  J. Lafferty,et al.  High-dimensional Ising model selection using ℓ1-regularized logistic regression , 2010, 1010.0311.

[56]  Pradeep Ravikumar,et al.  Graphical Models via Generalized Linear Models , 2012, NIPS.

[57]  Eugen Pircalabelu,et al.  Community-Based Group Graphical Lasso , 2020, J. Mach. Learn. Res..

[58]  R. Tibshirani,et al.  Sparse inverse covariance estimation with the graphical lasso. , 2008, Biostatistics.

[59]  Pei Wang,et al.  Partial Correlation Estimation by Joint Sparse Regression Models , 2008, Journal of the American Statistical Association.

[60]  Elizaveta Levina,et al.  On semidefinite relaxations for the block model , 2014, ArXiv.

[61]  P. Zhao,et al.  A path following algorithm for Sparse Pseudo-Likelihood Inverse Covariance Estimation (SPLICE) , 2008, 0807.3734.

[62]  Carlos Eduardo Scheidegger,et al.  Certifying and Removing Disparate Impact , 2014, KDD.

[63]  Silvio Lattanzi,et al.  Fair Clustering Through Fairlets , 2018, NIPS.

[64]  Carlos Castillo,et al.  Exploring Artist Gender Bias in Music Recommendation , 2020, ComplexRec-ImpactRS@RecSys.

[65]  Trevor Hastie,et al.  Applications of the lasso and grouped lasso to the estimation of sparse graphical models , 2010 .

[66]  M. Yuan,et al.  Model selection and estimation in regression with grouped variables , 2006 .

[67]  Jonathan L. Herlocker,et al.  Clustering items for collaborative filtering , 1999 .

[68]  Mohit Singh,et al.  The Price of Fair PCA: One Extra Dimension , 2018, NeurIPS.

[69]  Su-In Lee,et al.  Learning graphical models with hubs , 2014, J. Mach. Learn. Res..

[70]  Deepak K. Agarwal,et al.  A Flexible, Scalable and Efficient Algorithmic Framework for Primal Graphical Lasso , 2011, 1110.5508.

[71]  Guang Cheng,et al.  Simultaneous Clustering and Estimation of Heterogeneous Graphical Models , 2016, J. Mach. Learn. Res..

[72]  Mehrbakhsh Nilashi,et al.  Collaborative filtering recommender systems , 2013 .

[73]  Liang Zhang,et al.  MODELING ITEM-ITEM SIMILARITIES FOR PERSONALIZED RECOMMENDATIONS ON YAHOO! FRONT PAGE , 2011, 1111.0416.

[74]  Matthias Jarke,et al.  A Clustering Approach for Collaborative Filtering Recommendation Using Social Network Analysis , 2011, J. Univers. Comput. Sci..

[75]  Kevin P. Murphy,et al.  Sparse Gaussian graphical models with unknown block structure , 2009, ICML '09.

[76]  E. Ising Beitrag zur Theorie des Ferromagnetismus , 1925 .

[77]  Xiaodong Li,et al.  Convex Relaxation Methods for Community Detection , 2018, Statistical Science.

[78]  Xiangnan He,et al.  Bias and Debias in Recommender System: A Survey and Future Directions , 2020, ACM Trans. Inf. Syst..

[79]  George Michailidis,et al.  ESTIMATING HETEROGENEOUS GRAPHICAL MODELS FOR DISCRETE DATA WITH AN APPLICATION TO ROLL CALL VOTING. , 2015, The annals of applied statistics.

[80]  Patrick Danaher,et al.  The joint graphical lasso for inverse covariance estimation across multiple classes , 2011, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[81]  Nisheeth K. Vishnoi,et al.  Ranking with Fairness Constraints , 2017, ICALP.

[82]  E. Levina,et al.  Joint Structure Estimation for Categorical Markov Networks , 2010 .

[83]  Krishna P. Gummadi,et al.  Fairness Constraints: Mechanisms for Fair Classification , 2015, AISTATS.

[84]  Yufeng Liu,et al.  Joint estimation of multiple precision matrices with common structures , 2015, J. Mach. Learn. Res..

[85]  Su-In Lee,et al.  Learning Sparse Gaussian Graphical Models with Overlapping Blocks , 2016, NIPS.