A New Probabilistic Model for Rank Aggregation

This paper is concerned with rank aggregation, which aims to combine multiple input rankings to get a better ranking. A popular approach to rank aggregation is based on probabilistic models on permutations, e.g., the Luce model and the Mallows model. However, these models have their limitations in either poor expressiveness or high computational complexity. To avoid these limitations, in this paper, we propose a new model, which is defined with a coset-permutation distance, and models the generation of a permutation as a stagewise process. We refer to the new model as coset-permutation distance based stagewise (CPS) model. The CPS model has rich expressiveness and can therefore be used in versatile applications, because many different permutation distances can be used to induce the coset-permutation distance. The complexity of the CPS model is low because of the stagewise decomposition of the permutation probability and the efficient computation of most coset-permutation distances. We apply the CPS model to supervised rank aggregation, derive the learning and inference algorithms, and empirically study their effectiveness and efficiency. Experiments on public datasets show that the derived algorithms based on the CPS model can achieve state-of-the-art ranking accuracy, and are much more efficient than previous algorithms.

[1]  Javed A. Aslam,et al.  Relevance score normalization for metasearch , 2001, CIKM '01.

[2]  Dan Roth,et al.  Unsupervised rank aggregation with distance-based models , 2008, ICML '08.

[3]  H. E. Daniels,et al.  Rank Correlation and Population Models , 1950 .

[4]  R. Duncan Luce,et al.  Individual Choice Behavior , 1959 .

[5]  M. Fligner,et al.  Distance Based Ranking Models , 1986 .

[6]  D. Critchlow Metric Methods for Analyzing Partially Ranked Data , 1986 .

[7]  John D. Lafferty,et al.  Cranking: Combining Rankings Using Conditional Probability Models on Permutations , 2002, ICML.

[8]  Tao Qin,et al.  LETOR: Benchmark Dataset for Research on Learning to Rank for Information Retrieval , 2007 .

[9]  R. Manmatha,et al.  Modeling score distributions for combining the outputs of search engines , 2001, SIGIR '01.

[10]  Moni Naor,et al.  Rank aggregation methods for the Web , 2001, WWW '01.

[11]  Nesar Ahmad,et al.  Soft Computing Techniques for Rank Aggregation on the World Wide Web , 2003, World Wide Web.

[12]  L. Thurstone A law of comparative judgment. , 1994 .

[13]  M. M. Sufyan Beg Parallel rank aggregation for theWorld Wide Web , 2004 .

[14]  Ronald Fagin,et al.  Efficient similarity search and classification via rank aggregation , 2003, SIGMOD '03.

[15]  R. Plackett The Analysis of Permutations , 1975 .

[16]  Jaana Kekäläinen,et al.  Cumulated gain-based evaluation of IR techniques , 2002, TOIS.

[17]  P. Diaconis Group representations in probability and statistics , 1988 .

[18]  Javed A. Aslam,et al.  Models for metasearch , 2001, SIGIR '01.

[19]  C. L. Mallows NON-NULL RANKING MODELS. I , 1957 .