Efficient gradient descent algorithm for sparse models with application in learning-to-rank

Recently, learning-to-rank has attracted considerable attention. Although significant research efforts have been focused on learning-to-rank, it is not the case for the problem of learning sparse models for ranking. In this paper, we consider the sparse learning-to-rank problem. We formulate it as an optimization problem with the @?"1 regularization, and develop a simple but efficient iterative algorithm to solve the optimization problem. Experimental results on four benchmark datasets demonstrate that the proposed algorithm shows (1) superior performance gain compared to several state-of-the-art learning-to-rank algorithms, and (2) very competitive performance compared to FenchelRank that also learns a sparse model for ranking.

[1]  Jin Zhang,et al.  An efficient incremental method for generating equivalence groups of search results in information retrieval and queries , 2012, Knowl. Based Syst..

[2]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1997, EuroCOLT.

[3]  Qiang Wu,et al.  McRank: Learning to Rank Using Multiple Classification and Gradient Boosting , 2007, NIPS.

[4]  Ling Li,et al.  Ordinal Regression by Extended Binary Classification , 2006, NIPS.

[5]  Julien Mairal,et al.  Convex optimization with sparsity-inducing norms , 2011 .

[6]  Yong Tang,et al.  Learning to rank with document ranks and scores , 2011, Knowl. Based Syst..

[7]  Ryan M. Rifkin,et al.  Value Regularization and Fenchel Duality , 2007, J. Mach. Learn. Res..

[8]  Tao Qin,et al.  Robust sparse rank learning for non-smooth ranking measures , 2009, SIGIR 2009.

[9]  Yoram Singer,et al.  An Efficient Boosting Algorithm for Combining Preferences by , 2013 .

[10]  Tie-Yan Liu,et al.  Learning to rank: from pairwise approach to listwise approach , 2007, ICML '07.

[11]  Tie-Yan Liu,et al.  Listwise approach to learning to rank: theory and algorithm , 2008, ICML '08.

[12]  Stephen E. Robertson,et al.  SoftRank: optimizing non-smooth rank metrics , 2008, WSDM '08.

[13]  Yanchun Zhang,et al.  Random walk based rank aggregation to improving web search , 2011, Knowl. Based Syst..

[14]  Jaana Kekäläinen,et al.  Cumulated gain-based evaluation of IR techniques , 2002, TOIS.

[15]  Klaus Obermayer,et al.  Support vector learning for ordinal regression , 1999 .

[16]  Arkadi Nemirovski,et al.  EFFICIENT METHODS IN CONVEX PROGRAMMING , 2007 .

[17]  Xiaowei Yang,et al.  A Kernel Fuzzy c-Means Clustering-Based Fuzzy Support Vector Machine Algorithm for Classification Problems With Outliers or Noises , 2011, IEEE Transactions on Fuzzy Systems.

[18]  Ali Emrouznejad,et al.  Aggregating preference ranking with fuzzy Data Envelopment Analysis , 2010, Knowl. Based Syst..

[19]  Jie Wu,et al.  Sparse Learning-to-Rank via an Efficient Primal-Dual Algorithm , 2013, IEEE Transactions on Computers.

[20]  Jieping Ye,et al.  An accelerated gradient method for trace norm minimization , 2009, ICML '09.

[21]  Koby Crammer,et al.  Pranking with Ranking , 2001, NIPS.

[22]  Filip Radlinski,et al.  A support vector method for optimizing average precision , 2007, SIGIR.

[23]  Hang Li,et al.  AdaRank: a boosting algorithm for information retrieval , 2007, SIGIR.

[24]  Yurii Nesterov,et al.  Introductory Lectures on Convex Optimization - A Basic Course , 2014, Applied Optimization.

[25]  S. Sathiya Keerthi,et al.  Efficient algorithms for ranking with SVMs , 2010, Information Retrieval.

[26]  Gregory N. Hullender,et al.  Learning to rank using gradient descent , 2005, ICML.

[27]  Tao Qin,et al.  LETOR: A benchmark collection for research on learning to rank for information retrieval , 2010, Information Retrieval.

[28]  Yong Tang,et al.  Transductive learning to rank using association rules , 2011, Expert Syst. Appl..

[29]  Chris Buckley,et al.  OHSUMED: an interactive retrieval evaluation and new large test collection for research , 1994, SIGIR '94.