Active Ranking from Pairwise Comparisons and the Futility of Parametric Assumptions

We consider sequential or active ranking of a set of n items based on noisy pairwise comparisons. Items are ranked according to the probability that a given item beats a randomly chosen item, and ranking refers to partitioning the items into sets of pre-specified sizes according to their scores. This notion of ranking includes as special cases the identification of the top-k items and the total ordering of the items. We first analyze a sequential ranking algorithm that counts the number of comparisons won, and uses these counts to decide whether to stop, or to compare another pair of items, chosen based on confidence intervals specified by the data collected up to that point. We prove that this algorithm succeeds in recovering the ranking using a number of comparisons that is optimal up to logarithmic factors. This guarantee does not require any structural properties of the underlying pairwise probability matrix, unlike a significant body of past work on pairwise ranking based on parametric models such as the Thurstone or BradleyTerry-Luce models. It has been a long-standing open question as to whether or not imposing these parametric assumptions allow for improved ranking algorithms. Our second contribution settles this issue in the context of the problem of active ranking from pairwise comparisons: by means of tight lower bounds, we prove that perhaps surprisingly, these popular parametric modeling choices offer little statistical advantage.

[1]  Matthias Grossglauser,et al.  Robust Active Ranking from Sparse Noisy Comparisons , 2015, ArXiv.

[2]  Nir Ailon,et al.  Active Learning Ranking from Pairwise Preferences with Almost Optimal Query Complexity , 2011, NIPS.

[3]  Brian Eriksson,et al.  Learning to Top-K Search using Pairwise Comparisons , 2013, AISTATS.

[4]  Matthew J. Salganik,et al.  Wiki surveys : Open and quantifiable social data collection ∗ , 2012 .

[5]  Shie Mannor,et al.  Action Elimination and Stopping Conditions for the Multi-Armed Bandit and Reinforcement Learning Problems , 2006, J. Mach. Learn. Res..

[6]  Eyke Hüllermeier,et al.  Online Rank Elicitation for Plackett-Luce: A Dueling Bandits Approach , 2015, NIPS.

[7]  Yuxin Chen,et al.  Spectral MLE: Top-K Rank Aggregation from Pairwise Comparisons , 2015, ICML.

[8]  Sébastien Bubeck,et al.  Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems , 2012, Found. Trends Mach. Learn..

[9]  Robert D. Nowak,et al.  Sparse Dueling Bandits , 2015, AISTATS.

[10]  David C. Parkes,et al.  Computing Parametric Ranking Models via Rank-Breaking , 2014, ICML.

[11]  Robert D. Nowak,et al.  Active Ranking using Pairwise Comparisons , 2011, NIPS.

[12]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[13]  E. Paulson.  A Sequential Procedure for Selecting the Population with the Largest Mean from $k$ Normal Populations , 1964 .

[14]  D. Hunter.  MM algorithms for generalized Bradley-Terry models , 2003 .

[15]  Kannan Ramchandran,et al.  A Case for Ordinal Peer-evaluation in MOOCs , 2013 .

[16]  L. Thurstone.  A law of comparative judgment. , 1994 .

[17]  R. Duncan Luce,et al.  Individual Choice Behavior: A Theoretical Analysis , 1979 .

[18]  Martin J. Wainwright,et al.  Stochastically Transitive Models for Pairwise Comparisons: Statistical and Computational Issues , 2015, IEEE Transactions on Information Theory.

[19]  Raphaël Féraud,et al.  Generic Exploration and K-armed Voting Bandits , 2013, ICML.

[20]  Charu C. Aggarwal,et al.  Recommender Systems: The Textbook , 2016 .

[21]  Devavrat Shah,et al.  Iterative ranking from pair-wise comparisons , 2012, NIPS.

[22]  J. Dana,et al.  Transitivity of preferences. , 2011, Psychological review.

[23]  Martin J. Wainwright,et al.  Estimation from Pairwise Comparisons: Sharp Minimax Bounds with Topology Dependence , 2015, J. Mach. Learn. Res..

[24]  A. Culyer.  Thurstone’s Law of Comparative Judgment , 2014 .

[25]  R. A. Bradley,et al.  RANK ANALYSIS OF INCOMPLETE BLOCK DESIGNS THE METHOD OF PAIRED COMPARISONS , 1952 .

[26]  P.-C.-F. Daunou,et al.  Mémoire sur les élections au scrutin , 1803 .

[27]  Sébastien Bubeck,et al.  Multiple Identifications in Multi-Armed Bandits , 2012, ICML.

[28]  Martin J. Wainwright,et al.  Simple, Robust and Optimal Ranking from Pairwise Comparisons , 2015, J. Mach. Learn. Res..

[29]  Aurélien Garivier,et al.  On the Complexity of Best-Arm Identification in Multi-Armed Bandit Models , 2014, J. Mach. Learn. Res..

[30]  Eyke Hüllermeier,et al.  Top-k Selection based on Adaptive Sampling of Noisy Preferences , 2013, ICML.

[31]  Harry Joe,et al.  Majorization, entropy and paired comparisons , 1988 .

[32]  Zhenghao Chen,et al.  Tuned Models of Peer Assessment in MOOCs , 2013, EDM.

[33]  Xi Chen,et al.  Competitive analysis of the top-K ranking problem , 2016, SODA.

[34]  Thorsten Joachims,et al.  The K-armed Dueling Bandits Problem , 2012, COLT.

[35]  A. Tversky,et al.  Substitutability and similarity in binary choices , 1969 .

[36]  Bruce E. Hajek,et al.  Minimax-optimal Inference from Partial Rankings , 2014, NIPS.