Pareto-Optimal Fairness-Utility Amortizations in Rankings with a DBN Exposure Model

In recent years, it has become clear that rankings delivered in many areas need not only be useful to the users but also respect fairness of exposure for the item producers. We consider the problem of finding ranking policies that achieve a Pareto-optimal tradeoff between these two aspects. Several methods were proposed to solve it; for instance a popular one is to use linear programming with a Birkhoff-von Neumann decomposition. These methods, however, are based on a classical Position Based exposure Model (PBM), which assumes independence between the items (hence the exposure only depends on the rank). In many applications, this assumption is unrealistic and the community increasingly moves towards considering other models that include dependences, such as the Dynamic Bayesian Network (DBN) exposure model. For such models, computing (exact) optimal fair ranking policies remains an open question. In this paper, we answer this question by leveraging a new geometrical method based on the so-called expohedron proposed recently for the PBM (Kletti et al., WSDM'22). We lay out the structure of a new geometrical object (the DBN-expohedron), and propose for it a Carathéodory decomposition algorithm of complexity $O(n^3)$, where n is the number of documents to rank. Such an algorithm enables expressing any feasible expected exposure vector as a distribution over at most n rankings; furthermore we show that we can compute the whole set of Pareto-optimal expected exposure vectors with the same complexity $O(n^3)$. Our work constitutes the first exact algorithm able to efficiently find a Pareto-optimal distribution of rankings. It is applicable to a broad range of fairness notions, including classical notions of meritocratic and demographic fairness. We empirically evaluate our method on the TREC2020 and MSLR datasets and compare it to several baselines in terms of Pareto-optimality and speed.

[1]  Michael D. Ekstrand,et al.  Overview of the TREC 2021 Fair Ranking Track , 2023, TREC.

[2]  J. Renders,et al.  Introducing the Expohedron for Efficient Pareto-optimal Fairness-Utility Amortizations in Repeated Rankings , 2022, Web Search and Data Mining.

[3]  Thorsten Joachims,et al.  Fairness in Ranking under Uncertainty , 2021, NeurIPS.

[4]  Harrie Oosterhuis,et al.  Computationally Efficient Optimization of Plackett-Luce Ranking Models for Relevance and Fairness , 2021, SIGIR.

[5]  Georgia Koutrika,et al.  Fairness in rankings and recommendations: an overview , 2021, The VLDB Journal.

[6]  Julia Stoyanovich,et al.  Fairness in Ranking: A Survey , 2021, ArXiv.

[7]  Thorsten Joachims,et al.  User Fairness, Item Fairness, and Diversity for Rankings in Two-Sided Markets , 2020, ICTIR.

[8]  Thibaut Thonet,et al.  Multi-grouping Robust Fair Ranking , 2020, SIGIR.

[9]  Thorsten Joachims,et al.  Controlling Fairness and Bias in Dynamic Learning-to-Rank , 2020, SIGIR.

[10]  Bhaskar Mitra,et al.  Evaluating Stochastic Rankings with Expected Exposure , 2020, CIKM.

[11]  Miroslav Dudík,et al.  Fair Regression: Quantitative Definitions and Reduction-based Algorithms , 2019, ICML.

[12]  A. Polyanskii,et al.  Perron and Frobenius Meet Carathéodory , 2019, Electron. J. Comb..

[13]  Krishna P. Gummadi,et al.  Equity of Attention: Amortizing Individual Fairness in Rankings , 2018, SIGIR.

[14]  Thorsten Joachims,et al.  Fairness of Exposure in Rankings , 2018, KDD.

[15]  Fanny Dufossé,et al.  Notes on Birkhoff-von Neumann decomposition of doubly stochastic matrices , 2016 .

[16]  M. de Rijke,et al.  Click Models for Web Search , 2015, Click Models for Web Search.

[17]  Tao Qin,et al.  Introducing LETOR 4.0 Datasets , 2013, ArXiv.

[18]  Toniann Pitassi,et al.  Fairness through awareness , 2011, ITCS '12.

[19]  Olivier Chapelle,et al.  Expected reciprocal rank for graded relevance , 2009, CIKM.

[20]  Olivier Chapelle,et al.  A dynamic bayesian network click model for web search ranking , 2009, WWW '09.

[21]  Naoto Miyoshi,et al.  m-Balanced words: A generalization of balanced words , 2004, Theor. Comput. Sci..

[22]  Jaana Kekäläinen,et al.  Cumulated gain-based evaluation of IR techniques , 2002, TOIS.

[23]  S. Robertson The probability ranking principle in IR , 1997 .

[24]  I. Olkin,et al.  Inequalities: Theory of Majorization and Its Applications , 1980 .

[25]  R. Plackett The Analysis of Permutations , 1975 .

[26]  C. Carathéodory Über den Variabilitätsbereich der Koeffizienten von Potenzreihen, die gegebene Werte nicht annehmen , 1907 .