Fairness in Ranking: A Survey

In the past few years, there has been much work on incorporating fairness requirements into algorithmic rankers, with contributions coming from the data management, algorithms, information retrieval, and recommender systems communities. In this survey we give a systematic overview of this work, offering a broad perspective that connects formalizations and algorithmic approaches across subfields. An important contribution of our work is in developing a common narrative around the value frameworks that motivate specific fairness-enhancing interventions in ranking. This allows us to unify the presentation of mitigation objectives and of algorithmic techniques to help meet those objectives or identify trade-offs.

[1]  Abolfazl Asudeh,et al.  Fairly evaluating and scoring items in a data set , 2020, Proc. VLDB Endow..

[2]  Jaana Kekäläinen,et al.  Cumulated gain-based evaluation of IR techniques , 2002, TOIS.

[3]  A. Davis Black Feminist Thought: Knowledge, Consciousness and the Politics of Empowerment , 1993 .

[4]  Toshihiro Kamishima,et al.  Nantonac collaborative filtering: recommendation based on order responses , 2003, KDD '03.

[5]  Julia Stoyanovich,et al.  Causal intersectionality for fair ranking , 2020, ArXiv.

[6]  Christopher J. C. Burges,et al.  From RankNet to LambdaRank to LambdaMART: An Overview , 2010 .

[7]  Luca Oneto,et al.  Fairness in Machine Learning , 2020, INNSBDDL.

[8]  Julia Stoyanovich,et al.  Balanced Ranking with Diversity Constraints , 2019, IJCAI.

[9]  Alexandra Chouldechova,et al.  A snapshot of the frontiers of fairness in machine learning , 2020, Commun. ACM.

[10]  Suresh Venkatasubramanian,et al.  On the (im)possibility of fairness , 2016, ArXiv.

[11]  Abolfazl Asudeh,et al.  A Nutritional Label for Rankings , 2018, SIGMOD Conference.

[12]  Joel W. Cohen,et al.  The Medical Expenditure Panel Survey: A National Information Resource to Support Healthcare Cost Research and Inform Policy and Practice , 2009, Medical care.

[13]  K. Crenshaw Mapping the margins: intersectionality, identity politics, and violence against women of color , 1991 .

[14]  Yashodhan Kanoria,et al.  Joint Seat Allocation 2018: An algorithmic perspective , 2019, ArXiv.

[15]  Thorsten Joachims,et al.  Policy Learning for Fairness in Ranking , 2019, NeurIPS.

[16]  Kush R. Varshney,et al.  Fair Transfer Learning with Missing Protected Attributes , 2019, AIES.

[17]  Nicole Immorlica,et al.  Online auctions and generalized secretary problems , 2008, SECO.

[18]  Jon M. Kleinberg,et al.  Selection Problems in the Presence of Implicit Bias , 2018, ITCS.

[19]  R. A. Bradley,et al.  RANK ANALYSIS OF INCOMPLETE BLOCK DESIGNS THE METHOD OF PAIRED COMPARISONS , 1952 .

[20]  Abbe Mowshowitz,et al.  Bias on the web , 2002, CACM.

[21]  Carlos Castillo,et al.  Reducing Disparate Exposure in Ranking: A Learning To Rank Approach , 2018, WWW.

[22]  C. Spearman The proof and measurement of association between two things. , 2015, International journal of epidemiology.

[23]  Meike Zehlike,et al.  Matching code and law: achieving algorithmic fairness with optimal transport , 2017, Data Mining and Knowledge Discovery.

[24]  F. Maxwell Harper,et al.  The MovieLens Datasets: History and Context , 2016, TIIS.

[25]  Pasquale Lops,et al.  Semantics-aware Content-based Recommender Systems , 2014, CBRecSys@RecSys.

[26]  Martin Ester,et al.  A matrix factorization technique with trust propagation for recommendation in social networks , 2010, RecSys '10.

[27]  Nisheeth K. Vishnoi,et al.  Ranking with Fairness Constraints , 2017, ICALP.

[28]  Hang Li,et al.  Learning to Rank for Information Retrieval and Natural Language Processing , 2011, Synthesis Lectures on Human Language Technologies.

[29]  Krishna P. Gummadi,et al.  A Moral Framework for Understanding of Fair ML through Economic Models of Equality of Opportunity , 2018, ArXiv.

[30]  Ivan Kitanovski,et al.  FairSearch: A Tool For Fairness in Ranked Search Results , 2018, WWW.

[31]  Julia Stoyanovich,et al.  Measuring Fairness in Ranked Outputs , 2016, SSDBM.

[32]  Krishna P. Gummadi,et al.  iFair: Learning Individually Fair Data Representations for Algorithmic Decision Making , 2018, 2019 IEEE 35th International Conference on Data Engineering (ICDE).

[33]  Lu Zhang,et al.  On Discrimination Discovery and Removal in Ranked Data using Causal Graph , 2018, KDD.

[34]  Evaggelia Pitoura,et al.  Diversity in Big Data: A Review , 2017, Big Data.

[35]  Tie-Yan Liu,et al.  Learning to rank: from pairwise approach to listwise approach , 2007, ICML '07.

[36]  Robin D. Burke,et al.  Fairness and discrimination in recommendation and retrieval , 2019, RecSys.

[37]  Anika Saxena,et al.  ENGINEERING STUDENTS , 2020 .

[38]  Ed H. Chi,et al.  Fairness in Recommendation Ranking through Pairwise Comparisons , 2019, KDD.

[39]  Krishna P. Gummadi,et al.  Two-Sided Fairness for Repeated Matchings in Two-Sided Markets: A Case Study of a Ride-Hailing Platform , 2019, KDD.

[40]  Hang Li Learning to Rank for Information Retrieval and Natural Language Processing , 2011, Synthesis Lectures on Human Language Technologies.

[41]  Francesco Bonchi,et al.  Algorithmic Bias: From Discrimination Discovery to Fairness-aware Data Mining , 2016, KDD.

[42]  Jun Sakuma,et al.  Recommendation Independence , 2018, FAT.

[43]  Nisheeth K. Vishnoi,et al.  Interventions for ranking in the presence of implicit bias , 2020, FAT*.

[44]  D. Gifford 1963 , 2018, The British Film Catalogue.

[45]  Roberto Veneziani,et al.  Equality of When , 2017 .

[46]  Toniann Pitassi,et al.  Fairness through awareness , 2011, ITCS '12.

[47]  D. Fitch,et al.  Review of "Algorithms of oppression: how search engines reinforce racism," by Noble, S. U. (2018). New York, New York: NYU Press. , 2018, CDQR.

[48]  Neide Mayumi Osada,et al.  Black Feminist Thought: Knowledge, Consciousness, and the Politics of Empowerment , 2008 .

[49]  Abolfazl Asudeh,et al.  Designing Fair Ranking Schemes , 2017, SIGMOD Conference.

[50]  Thorsten Joachims,et al.  Fairness of Exposure in Rankings , 2018, KDD.

[51]  Toniann Pitassi,et al.  Learning Fair Representations , 2013, ICML.

[52]  J. Stoker,et al.  The Department of Health and Human Services. , 1999, Home healthcare nurse.

[53]  Krishna P. Gummadi,et al.  Equity of Attention: Amortizing Individual Fairness in Rankings , 2018, SIGIR.

[54]  1961 , 2018, Documents on Irish Foreign Policy vol. XI.

[55]  Helen Nissenbaum,et al.  Bias in computer systems , 1996, TOIS.

[56]  S. Shields,et al.  Gender: An Intersectionality Perspective , 2008 .

[57]  Linda F. Wightman LSAC National Longitudinal Bar Passage Study. LSAC Research Report Series. , 1998 .

[58]  Ricardo Baeza-Yates,et al.  FA*IR: A Fair Top-k Ranking Algorithm , 2017, CIKM.

[59]  Thorsten Joachims,et al.  Accurately interpreting clickthrough data as implicit feedback , 2005, SIGIR '05.

[60]  E. Rasmussen Evaluation in Information Retrieval , 2002 .

[61]  Gerhard Weikum,et al.  Privacy through Solidarity: A User-Utility-Preserving Framework to Counter Profiling , 2017, SIGIR.

[62]  Natalia Kovalyova,et al.  Data feminism , 2020, Information, Communication & Society.

[63]  Krishna P. Gummadi,et al.  Operationalizing Individual Fairness with Pairwise Fair Representations , 2019, Proc. VLDB Endow..

[64]  Matthew Richardson,et al.  Predicting clicks: estimating the click-through rate for new ads , 2007, WWW '07.

[65]  Fernando Diaz,et al.  Towards a Fair Marketplace: Counterfactual Evaluation of the trade-off between Relevance, Fairness & Satisfaction in Recommendation Systems , 2018, CIKM.

[66]  A. Sen,et al.  Equality of What? Tanner Lecture on Human Values , 1979 .

[67]  Sergei Vassilvitskii,et al.  Generalized distances between rankings , 2010, WWW '10.

[68]  Robin D. Burke,et al.  Fairness and Discrimination in Retrieval and Recommendation , 2019, SIGIR.

[69]  Rachel K. E. Bellamy,et al.  AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Bias , 2018, ArXiv.

[70]  H. V. Jagadish,et al.  Online Set Selection with Fairness and Diversity Constraints , 2018, EDBT.

[71]  Robin D. Burke,et al.  Multisided Fairness for Recommendation , 2017, ArXiv.

[72]  Thomas S. Ferguson,et al.  Who Solved the Secretary Problem , 1989 .

[73]  David Lindley,et al.  Dynamic Programming and Decision Theory , 1961 .