Rank Aggregation via Heterogeneous Thurstone Preference Models

We propose the Heterogeneous Thurstone Model (HTM) for aggregating ranked data, which can take the accuracy levels of different users into account. By allowing different noise distributions, the proposed HTM model maintains the generality of Thurstone's original framework, and as such, also extends the Bradley-Terry-Luce (BTL) model for pairwise comparisons to heterogeneous populations of users. Under this framework, we also propose a rank aggregation algorithm based on alternating gradient descent to estimate the underlying item scores and accuracy levels of different users simultaneously from noisy pairwise comparisons. We theoretically prove that the proposed algorithm converges linearly up to a statistical error which matches that of the state-of-the-art method for the single-user BTL model. We evaluate the proposed HTM model and algorithm on both synthetic and real data, demonstrating that it outperforms existing methods.

[1]  M. Kendall.  Rank Correlation Methods , 1949 .

[2]  Michael I. Jordan,et al.  Advances in Neural Information Processing Systems 30 , 1995 .

[3]  Lirong Xia,et al.  Learning Mixtures of Random Utility Models , 2018, AAAI.

[4]  Farzad Farnoud,et al.  HyDRA: gene prioritization via hybrid distance-score rank aggregation , 2015, Bioinform..

[5]  Bruce E. Hajek,et al.  Minimax-optimal Inference from Partial Rankings , 2014, NIPS.

[6]  Thorsten Joachims,et al.  Bayesian Ordinal Peer Grading , 2015, L@S.

[7]  Chih-Jen Lin,et al.  A Bayesian Approximation Method for Online Ranking , 2011, J. Mach. Learn. Res..

[8]  Tingting Zhang,et al.  Efficient Algorithm for Sparse Tensor-variate Gaussian Graphical Models via Gradient Descent , 2017, AISTATS.

[9]  Prateek Jain,et al.  Low-rank matrix completion using alternating minimization , 2012, STOC '13.

[10]  John Guiver,et al.  Bayesian inference for Plackett-Luce ranking models , 2009, ICML '09.

[11]  Bassem A. Hassan,et al.  Gene prioritization through genomic data fusion , 2006, Nature Biotechnology.

[12]  Jian Ma,et al.  Covariate Adjusted Precision Matrix Estimation via Nonconvex Optimization , 2018, ICML.

[13]  Jian Ma,et al.  Speeding Up Latent Variable Gaussian Graphical Model Estimation via Nonconvex Optimization , 2017, NIPS.

[14]  L. Thurstone.  A law of comparative judgment. , 1994 .

[15]  Martin J. Wainwright,et al.  Restricted strong convexity and weighted matrix completion: Optimal bounds with noise , 2010, J. Mach. Learn. Res..

[16]  Thorsten Joachims,et al.  Methods for ordinal peer grading , 2014, KDD.

[17]  R. Luce,et al.  Individual Choice Behavior: A Theoretical Analysis. , 1960 .

[18]  Devavrat Shah,et al.  Rank Centrality: Ranking from Pairwise Comparisons , 2012, Oper. Res..

[19]  Yuxin Chen,et al.  Spectral MLE: Top-K Rank Aggregation from Pairwise Comparisons , 2015, ICML.

[20]  Devavrat Shah,et al.  Iterative ranking from pair-wise comparisons , 2012, NIPS.

[21]  R. A. Bradley,et al.  Rank Analysis of Incomplete Block Designs: I. The Method of Paired Comparisons , 1952 .

[22]  Nebojsa Jojic,et al.  Efficient Ranking from Pairwise Comparisons , 2013, ICML.

[23]  Moni Naor,et al.  Rank aggregation methods for the Web , 2001, WWW '01.

[24]  ChengXiang Zhai,et al.  High-Dimensional Variance-Reduced Stochastic Gradient Expectation-Maximization Algorithm , 2017, ICML.

[25]  M. Kendall,et al.  Rank Correlation Methods , 1949 .

[26]  Paul N. Bennett,et al.  Pairwise ranking aggregation in a crowdsourced setting , 2013, WSDM.

[27]  Zhaoran Wang,et al.  High Dimensional EM Algorithm: Statistical Optimization and Asymptotic Normality , 2015, NIPS.

[28]  Mark Braverman,et al.  Noisy sorting without resampling , 2007, SODA '08.

[29]  Philip L. H. Yu,et al.  Bayesian analysis of order-statistics models for ranking data , 2000 .

[30]  R. A. Bradley,et al.  RANK ANALYSIS OF INCOMPLETE BLOCK DESIGNS , 1952 .

[31]  E. Zermelo.  Die Berechnung der Turnier-Ergebnisse als ein Maximumproblem der Wahrscheinlichkeitsrechnung , 1929 .

[32]  D. Hunter.  MM algorithms for generalized Bradley-Terry models , 2003 .

[33]  R. Duncan Luce,et al.  Individual Choice Behavior: A Theoretical Analysis , 1979 .

[34]  Milan Vojnovic,et al.  Parameter Estimation for Generalized Thurstone Choice Models , 2016, ICML.

[35]  Joel A. Tropp,et al.  User-Friendly Tail Bounds for Sums of Random Matrices , 2010, Found. Comput. Math..

[36]  Francesco Ricci,et al.  Group recommendations with rank aggregation and collaborative filtering , 2010, RecSys '10.

[37]  Martin J. Wainwright,et al.  Noisy matrix decomposition via convex relaxation: Optimal rates in high dimensions , 2011, ICML.

[38]  Matthew Lease,et al.  Learning to rank from a noisy crowd , 2011, SIGIR.

[39]  Xiao Zhang,et al.  A Unified Framework for Nonconvex Low-Rank plus Sparse Matrix Recovery , 2018, AISTATS.