Top-K Ranking from Pairwise Comparisons: When Spectral Ranking is Optimal

We explore the top-$K$ rank aggregation problem. Suppose a collection of items is compared in pairs repeatedly, and we aim to recover a consistent ordering that focuses on the top-$K$ ranked items based on partially revealed preference information. We investigate the Bradley-Terry-Luce model in which one ranks items according to their perceived utilities modeled as noisy observations of their underlying true utilities. Our main contributions are two-fold. First, in a general comparison model where item pairs to compare are given a priori, we attain an upper and lower bound on the sample size for reliable recovery of the top-$K$ ranked items. Second, more importantly, extending the result to a random comparison model where item pairs to compare are chosen independently with some probability, we show that in slightly restricted regimes, the gap between the derived bounds reduces to a constant factor, hence reveals that a spectral method can achieve the minimax optimality on the (order-wise) sample size required for top-$K$ ranking. That is to say, we demonstrate a spectral method alone to be sufficient to achieve the optimality and advantageous in terms of computational complexity, as it does not require an additional stage of maximum likelihood estimation that a state-of-the-art scheme employs to achieve the optimality. We corroborate our main results by numerical experiments.

[1]  D. Hunter MM algorithms for generalized Bradley-Terry models , 2003 .

[2]  Sergio Verdú,et al.  Generalizing the Fano inequality , 1994, IEEE Trans. Inf. Theory.

[3]  Arun Rajkumar,et al.  A Statistical Convergence Perspective of Algorithms for Rank Aggregation from Pairwise Data , 2014, ICML.

[4]  R. Duncan Luce,et al.  Individual Choice Behavior: A Theoretical Analysis , 1979 .

[5]  Andrew Caplin,et al.  Aggregation and Social Choice: A Mean Voter Theorem , 1991 .

[6]  Matthias Grossglauser,et al.  Fast and Accurate Inference of Plackett-Luce Models , 2015, NIPS.

[7]  Nebojsa Jojic,et al.  Efficient Ranking from Pairwise Comparisons , 2013, ICML.

[8]  Martin J. Wainwright,et al.  Stochastically Transitive Models for Pairwise Comparisons: Statistical and Computational Issues , 2015, IEEE Transactions on Information Theory.

[9]  Sergey Brin,et al.  The Anatomy of a Large-Scale Hypertextual Web Search Engine , 1998, Comput. Networks.

[10]  Teh-Hsing Wei,et al.  The algebraic foundations of ranking theory , 1952 .

[11]  Bruce E. Hajek,et al.  Minimax-optimal Inference from Partial Rankings , 2014, NIPS.

[12]  R. A. Bradley,et al.  RANK ANALYSIS OF INCOMPLETE BLOCK DESIGNS THE METHOD OF PAIRED COMPARISONS , 1952 .

[13]  Nir Ailon,et al.  Active Learning Ranking from Pairwise Preferences with Almost Optimal Query Complexity , 2011, NIPS.

[14]  Devavrat Shah,et al.  Ranking: Compare, don't score , 2011, 2011 49th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[15]  R. Plackett The Analysis of Permutations , 1975 .

[16]  Mark Braverman,et al.  Noisy sorting without resampling , 2007, SODA '08.

[17]  Brian Eriksson,et al.  Learning to Top-K Search using Pairwise Comparisons , 2013, AISTATS.

[18]  Moni Naor,et al.  Rank aggregation methods for the Web , 2001, WWW '01.

[19]  Robert D. Nowak,et al.  Active Ranking using Pairwise Comparisons , 2011, NIPS.

[20]  Francesco Ricci,et al.  Group recommendations with rank aggregation and collaborative filtering , 2010, RecSys '10.

[21]  L. R. Ford Solution of a Ranking Problem from Binary Comparisons , 1957 .

[22]  Craig Boutilier,et al.  Learning Mallows Models with Pairwise Preferences , 2011, ICML.

[23]  Yuxin Chen,et al.  Spectral MLE: Top-K Rank Aggregation from Pairwise Comparisons , 2015, ICML.

[24]  Alan Agresti,et al.  Categorical Data Analysis , 2003 .

[25]  David C. Parkes,et al.  A Statistical Decision-Theoretic Framework for Social Choice , 2014, NIPS.

[26]  C. L. Mallows NON-NULL RANKING MODELS. I , 1957 .

[27]  Sebastiano Vigna,et al.  Spectral ranking , 2009, Network Science.

[28]  P.-C.-F. Daunou,et al.  Mémoire sur les élections au scrutin , 1803 .

[29]  David C. Parkes,et al.  Generalized Method-of-Moments for Rank Aggregation , 2013, NIPS.

[30]  Paul N. Bennett,et al.  Pairwise ranking aggregation in a crowdsourced setting , 2013, WSDM.

[31]  Devavrat Shah,et al.  Rank Centrality: Ranking from Pairwise Comparisons , 2012, Oper. Res..

[32]  Eyke Hüllermeier,et al.  Preference-Based Rank Elicitation using Statistical Models: The Case of Mallows , 2014, ICML.

[33]  Martin J. Wainwright,et al.  Simple, Robust and Optimal Ranking from Pairwise Comparisons , 2015, J. Mach. Learn. Res..