FARA: Future-aware Ranking Algorithm for Fairness Optimization

Ranking systems are the key components of modern Information Retrieval (IR) applications, such as search engines and recommender systems. Besides the ranking relevance to users, the exposure fairness to item providers has also been considered an important factor in ranking optimization. Many fair ranking algorithms have been proposed to jointly optimize both ranking relevance and fairness. However, we find that most existing fair ranking methods adopt greedy algorithms that only optimize rankings for the next immediate session or request. As shown in this paper, such a myopic paradigm could limit the upper bound of ranking optimization and lead to suboptimal performance in the long term. To this end, we propose \textbf{FARA}, a novel \textbf{F}uture-\textbf{A}ware \textbf{R}anking \textbf{A}lgorithm for ranking relevance and fairness optimization. Instead of greedily optimizing rankings for the next immediate session, FARA plans ahead by jointly optimizing multiple ranklists together and saving them for future sessions. Specifically, FARA first uses the Taylor expansion to investigate how future ranklists will influence the overall fairness of the system. Then, based on the analysis of the Taylor expansion, FARA adopts a two-phase optimization algorithm where we first solve an optimal future exposure planning problem and then construct the optimal ranklists according to the optimal future exposure planning. Theoretically, we show that FARA is optimal for ranking relevance and fairness joint optimization. Empirically, our extensive experiments on three semi-synthesized datasets show that FARA is efficient, effective, and can deliver significantly better ranking performance compared to state-of-the-art fair ranking methods. We make our implementation public at \href{https://github.com/Taosheng-ty/QP_fairness/}{https://github.com/Taosheng-ty/QP\_fairness/}.

[1]  J. M. Phillips,et al.  Mitigating Exploitation Bias in Learning to Rank with an Uncertainty-aware Empirical Bayes Approach , 2023, ArXiv.

[2]  Michael D. Ekstrand,et al.  Overview of the TREC 2022 Fair Ranking Track , 2023, TREC.

[3]  Qingyao Ai,et al.  Marginal-Certainty-Aware Fair Ranking Algorithm , 2022, WSDM.

[4]  Michael D. Ekstrand,et al.  Measuring Fairness in Ranked Results: An Analytical and Empirical Comparison , 2022, SIGIR.

[5]  E. Bagheri,et al.  Gender Fairness in Information Retrieval Systems , 2022, SIGIR.

[6]  Qingyao Ai,et al.  Can Clicks Be Both Labels and Features?: Unbiased Behavior Feature Collection and Uncertainty-aware Learning to Rank , 2022, SIGIR.

[7]  Nicolas Usunier,et al.  Fast online ranking with fairness of exposure , 2022, FAccT.

[8]  T. Joachims,et al.  Fair Ranking as Fair Division: Impact-Based Individual Fairness in Ranking , 2022, KDD.

[9]  M. de Rijke,et al.  Fairness of Exposure in Light of Incomplete Exposure Estimation , 2022, SIGIR.

[10]  Bhaskar Mitra,et al.  Joint Multisided Exposure Fairness for Recommendation , 2022, SIGIR.

[11]  M. de Rijke,et al.  Probabilistic Permutation Graph Search: Black-Box Optimization for Fairness in Ranking , 2022, SIGIR.

[12]  Hossein A. Rahmani,et al.  CPFair: Personalized Consumer and Producer Fairness Re-ranking for Recommender Systems , 2022, SIGIR.

[13]  Qingyao Ai,et al.  Effective Exposure Amortizing for Fair Top-k Recommendation , 2022, ArXiv.

[14]  Gourab K. Patro,et al.  Fair ranking: a critical review, challenges, and future directions , 2022, FAccT.

[15]  Pascal Van Hentenryck,et al.  End-to-End Learning for Fair Ranking Systems , 2021, WWW.

[16]  Masoud Mansoury,et al.  Understanding and mitigating multi-sided exposure bias in recommender systems , 2021, SIGWEB Newsl..

[17]  Qingyao Ai,et al.  ULTRA: An Unbiased Learning To Rank Algorithm Toolbox , 2021, CIKM.

[18]  Yunqi Li,et al.  Towards Personalized Fairness based on Causal Notion , 2021, SIGIR.

[19]  Thorsten Joachims,et al.  Fairness and Control of Exposure in Two-sided Markets , 2021, ICTIR.

[20]  Chirag Shah,et al.  Addressing Bias and Fairness in Search Systems , 2021, SIGIR.

[21]  Yingqiang Ge,et al.  FAIR: Fairness‐aware information retrieval evaluation , 2021, J. Assoc. Inf. Sci. Technol..

[22]  Harrie Oosterhuis Computationally Efficient Optimization of Plackett-Luce Ranking Models for Relevance and Fairness , 2021, SIGIR.

[23]  Yudong Tan,et al.  TFROM: A Two-sided Fairness-Aware Recommendation Model for Both Customers and Providers , 2021, SIGIR.

[24]  Julia Stoyanovich,et al.  Fairness in Ranking: A Survey , 2021, ArXiv.

[25]  M. de Rijke,et al.  Unifying Online and Counterfactual Learning to Rank: A Novel Counterfactual Estimator that Effectively Utilizes Online Interventions , 2021, Proceedings of the 14th ACM International Conference on Web Search and Data Mining.

[26]  Qingyao Ai,et al.  Maximizing Marginal Fairness for Dynamic Learning to Rank , 2021, WWW.

[27]  Thorsten Joachims,et al.  Controlling Fairness and Bias in Dynamic Learning-to-Rank , 2020, SIGIR.

[28]  M. de Rijke,et al.  Policy-Aware Unbiased Learning to Rank for Top-k Rankings , 2020, SIGIR.

[29]  Bhaskar Mitra,et al.  Evaluating Stochastic Rankings with Expected Exposure , 2020, CIKM.

[30]  Huazheng Wang,et al.  Variance Reduction in Gradient Exploration for Online Learning to Rank , 2019, SIGIR.

[31]  Sahin Cem Geyik,et al.  Fairness-Aware Ranking in Search & Recommendation Systems with Application to LinkedIn Talent Search , 2019, KDD.

[32]  Thorsten Joachims,et al.  Policy Learning for Fairness in Ranking , 2019, NeurIPS.

[33]  Thorsten Joachims,et al.  Estimating Position Bias without Intrusive Interventions , 2018, WSDM.

[34]  Carlos Castillo,et al.  Reducing Disparate Exposure in Ranking: A Learning To Rank Approach , 2018, WWW.

[35]  Huazheng Wang,et al.  Efficient Exploration of Gradient Space for Online Learning to Rank , 2018, SIGIR.

[36]  Krishna P. Gummadi,et al.  Equity of Attention: Amortizing Individual Fairness in Rankings , 2018, SIGIR.

[37]  W. Bruce Croft,et al.  Unbiased Learning to Rank with Unbiased Propensity Estimation , 2018, SIGIR.

[38]  Thorsten Joachims,et al.  Fairness of Exposure in Rankings , 2018, KDD.

[39]  Marc Najork,et al.  Position Bias Estimation for Unbiased Learning to Rank in Personal Search , 2018, WSDM.

[40]  Abolfazl Asudeh,et al.  Designing Fair Ranking Schemes , 2017, SIGMOD Conference.

[41]  Ricardo Baeza-Yates,et al.  FA*IR: A Fair Top-k Ranking Algorithm , 2017, CIKM.

[42]  Nisheeth K. Vishnoi,et al.  Ranking with Fairness Constraints , 2017, ICALP.

[43]  Fabrizio Silvestri,et al.  Post-Learning Optimization of Tree Ensembles for Efficient Ranking , 2016, SIGIR.

[44]  Tao Qin,et al.  Introducing LETOR 4.0 Datasets , 2013, ArXiv.

[45]  Tie-Yan Liu,et al.  Learning to rank for information retrieval , 2009, SIGIR.

[46]  Nick Craswell,et al.  An experimental comparison of click position-bias models , 2008, WSDM '08.

[47]  James Allan,et al.  A comparison of statistical significance tests for information retrieval evaluation , 2007, CIKM '07.

[48]  Matthew Richardson,et al.  Predicting clicks: estimating the click-through rate for new ads , 2007, WWW '07.

[49]  R. Jackson Inequalities , 2007, Algebra for Parents.

[50]  Filip Radlinski,et al.  Minimally Invasive Randomization for Collecting Unbiased Preferences from Clickthrough Logs , 2006, AAAI 2006.

[51]  Thorsten Joachims,et al.  Accurately interpreting clickthrough data as implicit feedback , 2005, SIGIR '05.

[52]  Jaana Kekäläinen,et al.  Cumulated gain-based evaluation of IR techniques , 2002, TOIS.

[53]  S. Robertson The probability ranking principle in IR , 1997 .

[54]  Ashudeep Singh Fairness of Exposure for Ranking Systems , 2021 .

[55]  John Fulcher,et al.  Computational Intelligence: An Introduction , 2008, Computational Intelligence: A Compendium.