Sync-Rank: Robust Ranking, Constrained Ranking and Rank Aggregation via Eigenvector and SDP Synchronization

We consider the classical problem of establishing a statistical ranking of a set of n items given a set of inconsistent and incomplete pairwise comparisons between such items. Instantiations of this problem occur in numerous applications in data analysis (e.g., ranking teams in sports data), computer vision, and machine learning. We formulate the above problem of ranking with incomplete noisy information as an instance of the group synchronization problem over the group SO(2) of planar rotations, whose usefulness has been demonstrated in numerous applications in recent years in computer vision and graphics, sensor network localization and structural biology. Its least squares solution can be approximated by either a spectral or a semidefinite programming (SDP) relaxation, followed by a rounding procedure. We perform extensive numerical simulations on both synthetic and real-world data sets, which show that our proposed method compares favorably to other ranking methods from the recent literature. Existing theoretical guarantees on the group synchronization problem imply lower bounds on the largest amount of noise permissible in the data while still achieving an approximate recovery of the ground truth ranking. We propose a similar synchronization-based algorithm for the rank-aggregation problem, which integrates in a globally consistent ranking many pairwise rank-offsets or partial rankings, given by different rating systems on the same set of items, an approach which yields significantly more accurate results than other aggregation methods, including Rank-Centrality, a recent state-of-the-art algorithm. Furthermore, we discuss the problem of semi-supervised ranking when there is available information on the ground truth rank of a subset of players, and propose an algorithm based on SDP which is able to recover the ranking of the remaining players, subject to such hard constraints. Finally, synchronization-based ranking, combined with a spectral technique for the densest subgraph problem, makes it possible to extract partial rankings that other methods are not able to find, in other words, to identify the rank of a small subset of players whose pairwise rank comparisons are less noisy than the rest of the data. We discuss a number of related open questions which we defer for future investigation.

[1]  Amit Singer,et al.  Tightness of the maximum likelihood semidefinite relaxation for angular synchronization , 2014, Math. Program..

[2]  Devavrat Shah,et al.  Iterative ranking from pair-wise comparisons , 2012, NIPS.

[3]  Yuan Yao,et al.  Statistical ranking and combinatorial Hodge theory , 2008, Math. Program..

[4]  Mark Braverman,et al.  Noisy sorting without resampling , 2007, SODA '08.

[5]  U. Feige,et al.  On the densest k-subgraph problems , 1997 .

[6]  Stanley Wasserman,et al.  Social Network Analysis: Methods and Applications , 1994 .

[7]  Nicolas de Condorcet Essai Sur L'Application de L'Analyse a la Probabilite Des Decisions Rendues a la Pluralite Des Voix , 2009 .

[8]  Nicolas Boumal,et al.  A Riemannian low-rank method for optimization over semidefinite matrices with block-diagonal constraints , 2015, ArXiv.

[9]  David HARViLLE,et al.  The Use of Linear-Model Methodology to Rate High School or College Football Teams , 1977 .

[10]  Bruce Hendrickson,et al.  A Spectral Algorithm for Seriation and the Consecutive Ones Problem , 1999, SIAM J. Comput..

[11]  Paul Van Dooren,et al.  Iterative Filtering in Reputation Systems , 2010, SIAM J. Matrix Anal. Appl..

[12]  Katya Scheinberg,et al.  Block Coordinate Descent Methods for Semidefinite Programming , 2012 .

[13]  Devavrat Shah,et al.  Efficient rank aggregation using partial data , 2012, SIGMETRICS '12.

[14]  Tie-Yan Liu Learning to Rank for Information Retrieval , 2009, Found. Trends Inf. Retr..

[15]  D. Féral,et al.  The Largest Eigenvalue of Rank One Deformation of Large Wigner Matrices , 2006, math/0605624.

[16]  Noga Alon,et al.  Finding a large hidden clique in a random graph , 1998, SODA '98.

[17]  Claire Mathieu,et al.  Electronic Colloquium on Computational Complexity, Report No. 144 (2006) How to rank with few errors A PTAS for Weighted Feedback Arc Set on Tournaments , 2006 .

[18]  Alexandre d'Aspremont,et al.  SerialRank: Spectral Ranking using Seriation , 2014, NIPS.

[19]  Shipeng Yu,et al.  Ranking annotators for crowdsourced labeling tasks , 2011, NIPS.

[20]  C. L. Mallows NON-NULL RANKING MODELS. I , 1957 .

[21]  Mark Newman,et al.  Networks: An Introduction , 2010 .

[22]  Ling Liu,et al.  A reputation-based trust model for peer-to-peer ecommerce communities , 2003, EC.

[23]  Amit Singer,et al.  Eigenvector Synchronization, Graph Rigidity and the Molecule Problem , 2011, Information and inference : a journal of the IMA.

[24]  J. Lasserre,et al.  Handbook on Semidefinite, Conic and Polynomial Optimization , 2012 .

[25]  Nebojsa Jojic,et al.  Efficient Ranking from Pairwise Comparisons , 2013, ICML.

[26]  Gary L. Miller,et al.  Scalable Constrained Clustering: A Generalized Spectral Method , 2016, ArXiv.

[27]  C. Lee Giles,et al.  CiteSeer: an automatic citation indexing system , 1998, DL '98.

[28]  Kenneth J. Koehler,et al.  An application of a biased version of the Bradley-Terry-Luce model to professional basketball results , 1982 .

[29]  Ling Liu,et al.  A reputation-based trust model for peer-to-peer e-commerce communities , 2003, EEE International Conference on E-Commerce, 2003. CEC 2003..

[30]  U. Feige,et al.  On the Densest K-subgraph Problem , 1997 .

[31]  T. L. Saaty A Scaling Method for Priorities in Hierarchical Structures , 1977 .

[32]  R. Duncan Luce,et al.  Individual Choice Behavior: A Theoretical Analysis , 1979 .

[33]  R. A. Bradley,et al.  RANK ANALYSIS OF INCOMPLETE BLOCK DESIGNS , 1952 .

[34]  Mason A. Porter,et al.  Dynamic network centrality summarizes learning in the human brain , 2012, J. Complex Networks.

[35]  Rajeev Motwani,et al.  The PageRank Citation Ranking : Bringing Order to the Web , 1999, WWW 1999.

[36]  R. A. Bradley,et al.  RANK ANALYSIS OF INCOMPLETE BLOCK DESIGNS THE METHOD OF PAIRED COMPARISONS , 1952 .

[37]  Anton van den Hengel,et al.  Semidefinite Programming , 2014, Computer Vision, A Reference Guide.

[38]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[39]  Bin Bi,et al.  Iterative Learning for Reliable Crowdsourcing Systems , 2012 .

[40]  James R. Lee,et al.  An improved approximation ratio for the minimum linear arrangement problem , 2007, Inf. Process. Lett..

[41]  Lydia B. Chilton,et al.  The labor economics of paid crowdsourcing , 2010, EC '10.

[42]  Jon Kleinberg,et al.  Authoritative sources in a hyperlinked environment , 1999, SODA '98.

[43]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[44]  K. Talluri,et al.  The Theory and Practice of Revenue Management , 2004 .

[45]  Ernesto Estrada,et al.  The Structure of Complex Networks: Theory and Applications , 2011 .

[46]  Shou-De Lin,et al.  Time-Aware Ranking in Dynamic Citation Networks , 2011, 2011 IEEE 11th International Conference on Data Mining Workshops.

[47]  Shuzhong Zhang,et al.  Complex Quadratic Optimization and Semidefinite Programming , 2006, SIAM J. Optim..

[48]  Amit Singer,et al.  A Cheeger Inequality for the Graph Connection Laplacian , 2012, SIAM J. Matrix Anal. Appl..

[49]  Ian Davidson,et al.  On constrained spectral clustering and its applications , 2012, Data Mining and Knowledge Discovery.

[50]  M. Kendall,et al.  ON THE METHOD OF PAIRED COMPARISONS , 1940 .

[51]  Amit Singer,et al.  Exact and Stable Recovery of Rotations for Robust Synchronization , 2012, ArXiv.

[52]  David C. Parkes,et al.  Computing Parametric Ranking Models via Rank-Breaking , 2014, ICML.

[53]  Robert D. Nowak,et al.  Active Ranking using Pairwise Comparisons , 2011, NIPS.

[54]  Rongjie Lai,et al.  A Splitting Method for Orthogonality Constrained Problems , 2014, J. Sci. Comput..

[55]  P. J. Huber Pairwise Comparison and Ranking: Optimum Properties of the Row Sum Procedure , 1963 .

[56]  Ming Ma A Matrix Approach to Asset Pricing in Foreign Exchange Market , 2006 .

[57]  Anil N. Hirani,et al.  Least Squares Ranking on Graphs , 2010, 1011.1716.

[58]  Raj Rao Nadakuditi,et al.  The singular values and vectors of low rank perturbations of large rectangular random matrices , 2011, J. Multivar. Anal..

[59]  Alexandre d'Aspremont,et al.  Convex Relaxations for Permutation Problems , 2013, SIAM J. Matrix Anal. Appl..

[60]  David F. Gleich,et al.  SVD based term suggestion and ranking system , 2004, Fourth IEEE International Conference on Data Mining (ICDM'04).

[61]  Le Song,et al.  Learning Social Infectivity in Sparse Low-rank Networks Using Multi-dimensional Hawkes Processes , 2013, AISTATS.

[62]  Yoram Singer,et al.  An Efficient Boosting Algorithm for Combining Preferences by , 2013 .

[63]  Amit Singer,et al.  Approximating the little Grothendieck problem over the orthogonal and unitary groups , 2013, Mathematical Programming.

[64]  Tie-Yan Liu,et al.  Learning to rank for information retrieval , 2009, SIGIR.

[65]  Yaron Lipman,et al.  Sensor network localization by eigenvector synchronization over the euclidean group , 2012, TOSN.

[66]  Roberto Turrin,et al.  Performance of recommender algorithms on top-n recommendation tasks , 2010, RecSys '10.

[67]  A. Singer,et al.  Vector diffusion maps and the connection Laplacian , 2011, Communications on pure and applied mathematics.

[68]  David C. Parkes,et al.  Generalized Method-of-Moments for Rank Aggregation , 2013, NIPS.

[69]  S. Osher,et al.  Statistical ranking using the $l^{1}$-norm on graphs , 2013 .

[70]  Ofer Zeitouni,et al.  A law of large numbers for finite-range dependent random matrices , 2006 .

[71]  Stella Yu,et al.  Angular Embedding: A Robust Quadratic Criterion , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[72]  A. Singer Angular Synchronization by Eigenvectors and Semidefinite Programming. , 2009, Applied and computational harmonic analysis.

[73]  Amy Nicole Langville,et al.  Google's PageRank and beyond - the science of search engine rankings , 2006 .