Ranking and synchronization from pairwise measurements via SVD

Given a measurement graph $G= (V,E)$ and an unknown signal $r \in \mathbb{R}^n$, we investigate algorithms for recovering $r$ from pairwise measurements of the form $r_i - r_j$; $\{i,j\} \in E$. This problem arises in a variety of applications, such as ranking teams in sports data and time synchronization of distributed networks. Framed in the context of ranking, the task is to recover the ranking of $n$ teams (induced by $r$) given a small subset of noisy pairwise rank offsets. We propose a simple SVD-based algorithmic pipeline for both the problem of time synchronization and ranking. We provide a detailed theoretical analysis in terms of robustness against both sampling sparsity and noise perturbations with outliers, using results from matrix perturbation and random matrix theory. Our theoretical findings are complemented by a detailed set of numerical experiments on both synthetic and real data, showcasing the competitiveness of our proposed algorithms with other state-of-the-art methods.

[1]  Anil N. Hirani,et al.  Least Squares Ranking on Graphs , 2010, 1011.1716.

[2]  Emmanuel J. Candès,et al.  The Power of Convex Relaxation: Near-Optimal Matrix Completion , 2009, IEEE Transactions on Information Theory.

[3]  Emmanuel J. Candès,et al.  Matrix Completion With Noise , 2009, Proceedings of the IEEE.

[4]  Devavrat Shah,et al.  Rank Centrality: Ranking from Pairwise Comparisons , 2012, Oper. Res..

[5]  David F. Gleich,et al.  Rank aggregation via nuclear norm minimization , 2011, KDD.

[6]  Takeo Kanade,et al.  Shape and motion from image streams under orthography: a factorization method , 1992, International Journal of Computer Vision.

[7]  Can M. Le,et al.  Sparse random graphs: regularization and concentration of the Laplacian , 2015, ArXiv.

[8]  Ren-Cang Li On perturbations of matrix pencils with real spectra , 1994 .

[9]  Raja Giryes,et al.  Ranking Recovery from Limited Comparisons using Low-Rank Matrix Completion , 2018, ArXiv.

[10]  Emmanuel J. Candès,et al.  Templates for convex cone problems with applications to sparse signal recovery , 2010, Math. Program. Comput..

[11]  Arpit Agarwal,et al.  Accelerated Spectral Ranking , 2018, ICML.

[12]  Cristopher Moore,et al.  A physical model for efficient ranking in networks , 2017, Science Advances.

[13]  R. A. Bradley,et al.  RANK ANALYSIS OF INCOMPLETE BLOCK DESIGNS THE METHOD OF PAIRED COMPARISONS , 1952 .

[14]  A. Bandeira,et al.  Sharp nonasymptotic bounds on the norm of random matrices with independent entries , 2014, 1408.6185.

[15]  P.R. Kumar,et al.  Distributed Clock Synchronization over Wireless Networks: Algorithms and Analysis , 2006, Proceedings of the 45th IEEE Conference on Decision and Control.

[16]  Amit Singer,et al.  Eigenvector Synchronization, Graph Rigidity and the Molecule Problem , 2011, Information and inference : a journal of the IMA.

[17]  Yuan Yao,et al.  Statistical ranking and combinatorial Hodge theory , 2008, Math. Program..

[18]  Franz J. Király,et al.  Modeling outcomes of soccer matches , 2018, Machine Learning.

[19]  Deborah Estrin,et al.  Optimal and Global Time Synchronization in Sensornets , 2003 .

[20]  Madeleine Udell,et al.  Why Are Big Data Matrices Approximately Low Rank? , 2017, SIAM J. Math. Data Sci..

[21]  Ewout van den Berg,et al.  1-Bit Matrix Completion , 2012, ArXiv.

[22]  Ronen Basri,et al.  A Survey on Structure from Motion , 2017, ArXiv.

[23]  A. Singer,et al.  Vector diffusion maps and the connection Laplacian , 2011, Communications on pure and applied mathematics.

[24]  Johan A.K. Suykens,et al.  Deformed Laplacians and spectral ranking in directed networks , 2015, Applied and Computational Harmonic Analysis.

[25]  Prateek Jain,et al.  Low-rank matrix completion using alternating minimization , 2012, STOC '13.

[26]  S. Boucheron,et al.  Concentration inequalities : a non asymptotic theory of independence , 2013 .

[27]  Sahin Albayrak,et al.  Spectral Analysis of Signed Graphs for Clustering, Prediction and Visualization , 2010, SDM.

[28]  Dehui Yang,et al.  Modeling and recovering non-transitive pairwise comparison matrices , 2015, 2015 International Conference on Sampling Theory and Applications (SampTA).

[29]  Amit Singer,et al.  Synchronization over Cartan Motion Groups via Contraction , 2016, SIAM J. Appl. Algebra Geom..

[30]  Arun Rajkumar,et al.  When can we rank well from comparisons of \(O(n\log(n))\) non-actively chosen pairs? , 2016, COLT.

[31]  V. N. Bogaevski,et al.  Matrix Perturbation Theory , 1991 .

[32]  H. Yau,et al.  Spectral statistics of Erdős–Rényi graphs I: Local semicircle law , 2011, 1103.1919.

[33]  Sewoong Oh,et al.  Learning from Comparisons and Choices , 2017, J. Mach. Learn. Res..

[34]  Rajeev Motwani,et al.  The PageRank Citation Ranking : Bringing Order to the Web , 1999, WWW 1999.

[35]  M. Davenport ONE-BIT MATRIX COMPLETION FOR PAIRWISE COMPARISON MATRICES , 2013 .

[36]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2008, Found. Comput. Math..

[37]  Maksims Volkovs,et al.  New learning methods for supervised and unsupervised preference aggregation , 2014, J. Mach. Learn. Res..

[38]  Simon DeDeo,et al.  Social Feedback and the Emergence of Rank in Animal Society , 2015, bioRxiv.

[39]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[40]  Subhajit Sanyal,et al.  Multi-objective ranking of comments on web , 2012, WWW.

[41]  Daniel B. Larremore,et al.  Systematic inequality and hierarchy in faculty hiring networks , 2015, Science Advances.

[42]  Devavrat Shah,et al.  Iterative ranking from pair-wise comparisons , 2012, NIPS.

[43]  Yaron Lipman,et al.  Sensor network localization by eigenvector synchronization over the euclidean group , 2012, TOSN.

[44]  Alexandre d'Aspremont,et al.  Spectral Ranking using Seriation , 2014, J. Mach. Learn. Res..

[45]  Yonina C. Eldar,et al.  Phase Retrieval via Matrix Completion , 2011, SIAM Rev..

[46]  Dong Liu,et al.  Robust late fusion with rank minimization , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[47]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[48]  Bin Yu,et al.  Impact of regularization on spectral clustering , 2016 .

[49]  A. Singer Angular Synchronization by Eigenvectors and Semidefinite Programming. , 2009, Applied and computational harmonic analysis.

[50]  Zhao Kang,et al.  Top-N Recommender System via Matrix Completion , 2016, AAAI.

[51]  Mihai Cucuringu,et al.  Sync-Rank: Robust Ranking, Constrained Ranking and Rank Aggregation via Eigenvector and Semidefinite Programming Synchronization , 2015, ArXiv.

[52]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[53]  P. Wedin Perturbation bounds in connection with singular value decomposition , 1972 .

[54]  Alexandre d'Aspremont,et al.  SerialRank: Spectral Ranking using Seriation , 2014, NIPS.

[55]  H. Weyl Das asymptotische Verteilungsgesetz der Eigenwerte linearer partieller Differentialgleichungen (mit einer Anwendung auf die Theorie der Hohlraumstrahlung) , 1912 .

[56]  Hemant Tyagi,et al.  An extension of the angular synchronization problem to the heterogeneous setting , 2020, Foundations of Data Science.

[57]  Mikhail Belkin,et al.  Unperturbed: spectral analysis beyond Davis-Kahan , 2017, ALT.