Spectral Ranking using Seriation

We describe a seriation algorithm for ranking a set of items given pairwise comparisons between these items. Intuitively, the algorithm assigns similar rankings to items that compare similarly with all others. It does so by constructing a similarity matrix from pairwise comparisons, using seriation methods to reorder this matrix and construct a ranking. We first show that this spectral seriation algorithm recovers the true ranking when all pairwise comparisons are observed and consistent with a total order. We then show that ranking reconstruction is still exact when some pairwise comparisons are corrupted or missing, and that seriation based spectral ranking is more robust to noise than classical scoring methods. Finally, we bound the ranking error when only a random subset of the comparions are observed. An additional benefit of the seriation formulation is that it allows us to solve semi-supervised ranking problems. Experiments on both synthetic and real datasets demonstrate that seriation based spectral ranking achieves competitive and in some cases superior performance compared to classical ranking methods.

[1]  P.-C.-F. Daunou,et al.  Mémoire sur les élections au scrutin , 1803 .

[2]  E. Zermelo.  Die Berechnung der Turnier-Ergebnisse als ein Maximumproblem der Wahrscheinlichkeitsrechnung , 1929 .

[3]  P. Moran.  On the method of paired comparisons. , 1947, Biometrika.

[4]  R. A. Bradley,et al.  RANK ANALYSIS OF INCOMPLETE BLOCK DESIGNS THE METHOD OF PAIRED COMPARISONS , 1952 .

[5]  R. A. Bradley,et al.  Rank Analysis of Incomplete Block Designs: I. The Method of Paired Comparisons , 1952 .

[6]  R. Duncan Luce,et al.  Individual Choice Behavior , 1959 .

[7]  P. J. Huber.  Pairwise Comparison and Ranking: Optimum Properties of the Row Sum Procedure , 1963 .

[8]  E. Barbeau.  Perron's Result and a Decision on Admissions Tests , 1986 .

[9]  G. Stewart,et al.  Matrix Perturbation Theory , 1990 .

[10]  Henryk Wozniakowski,et al.  Estimating the Largest Eigenvalue by the Power and Lanczos Algorithms with a Random Start , 1992, SIAM J. Matrix Anal. Appl..

[11]  J. Kuczy,et al.  Estimating the Largest Eigenvalue by the Power and Lanczos Algorithms with a Random Start , 1992 .

[12]  James P. Keener,et al.  The Perron-Frobenius Theorem and the Ranking of Football Teams , 1993, SIAM Rev..

[13]  Eli Upfal,et al.  Computing with Noisy Information , 1994, SIAM J. Comput..

[14]  Yoram Singer,et al.  Learning to Order Things , 1997, NIPS.

[15]  Bruce Hendrickson,et al.  A Spectral Algorithm for Seriation and the Consecutive Ones Problem , 1999, SIAM J. Comput..

[16]  Yoram Singer,et al.  An Efficient Boosting Algorithm for Combining Preferences by , 2013 .

[17]  M. KleinbergJon.  Authoritative sources in a hyperlinked environment , 1999 .

[18]  Rajeev Motwani,et al.  The PageRank Citation Ranking : Bringing Order to the Web , 1999, WWW 1999.

[19]  Santosh S. Vempala,et al.  Semi-definite relaxations for minimum bandwidth and other vertex-ordering problems , 2000, Theor. Comput. Sci..

[20]  Moni Naor,et al.  Rank aggregation methods for the Web , 2001, WWW '01.

[21]  Dimitris Achlioptas,et al.  Fast computation of low rank matrix approximations , 2001, STOC '01.

[22]  Thorsten Joachims,et al.  Optimizing search engines using clickthrough data , 2002, KDD.

[23]  D. Hunter.  MM algorithms for generalized Bradley-Terry models , 2003 .

[24]  Thomas L. Saaty,et al.  Decision-making with the AHP: Why is the principal eigenvector necessary , 2003, Eur. J. Oper. Res..

[25]  Tom Minka,et al.  TrueSkillTM: A Bayesian Skill Rating System , 2006, NIPS.

[26]  James R. Lee,et al.  An improved approximation ratio for the minimum linear arrangement problem , 2007, Inf. Process. Lett..

[27]  Claire Mathieu,et al.  Electronic Colloquium on Computational Complexity, Report No. 144 (2006) How to rank with few errors A PTAS for Weighted Feedback Arc Set on Tournaments , 2006 .

[28]  Dimitris Achlioptas,et al.  Fast computation of low-rank matrix approximations , 2007, JACM.

[29]  Mark Braverman,et al.  Noisy sorting without resampling , 2007, SODA '08.

[30]  Ling Huang,et al.  Spectral Clustering with Perturbed Data , 2008, NIPS.

[31]  Mikhail Belkin,et al.  Consistency of spectral clustering , 2008, 0804.0678.

[32]  Nicolas de Condorcet.  Essai Sur L'Application de L'Analyse a la Probabilite Des Decisions Rendues a la Pluralite Des Voix , 2009 .

[33]  Michael I. Jordan,et al.  On the Consistency of Ranking Algorithms , 2010, ICML.

[34]  Ohad Shamir,et al.  Spectral Clustering on a Budget , 2011, AISTATS.

[35]  Yuan Yao,et al.  Statistical ranking and combinatorial Hodge theory , 2008, Math. Program..

[36]  Robert D. Nowak,et al.  Active Ranking using Pairwise Comparisons , 2011, NIPS.

[37]  Nir Ailon,et al.  Active Learning Ranking from Pairwise Preferences with Almost Optimal Query Complexity , 2011, NIPS.

[38]  Devavrat Shah,et al.  Iterative ranking from pair-wise comparisons , 2012, NIPS.

[39]  Michael I. Jordan,et al.  The Asymptotics of Ranking Algorithms , 2012, ArXiv.

[40]  Nebojsa Jojic,et al.  Efficient Ranking from Pairwise Comparisons , 2013, ICML.

[41]  Alexandre d'Aspremont,et al.  Convex Relaxations for Permutation Problems , 2013, SIAM J. Matrix Anal. Appl..

[42]  Arun Rajkumar,et al.  A Statistical Convergence Perspective of Algorithms for Rank Aggregation from Pairwise Data , 2014, ICML.

[43]  Tengyao Wang,et al.  A useful variant of the Davis--Kahan theorem for statisticians , 2014, 1405.0680.

[44]  Mihai Cucuringu,et al.  Sync-Rank: Robust Ranking, Constrained Ranking and Rank Aggregation via Eigenvector and Semidefinite Programming Synchronization , 2015, ArXiv.

[45]  Mihai Cucuringu,et al.  Sync-Rank: Robust Ranking, Constrained Ranking and Rank Aggregation via Eigenvector and SDP Synchronization , 2015, IEEE Transactions on Network Science and Engineering.

[46]  Sebastiano Vigna,et al.  Spectral ranking , 2009, Network Science.