Robust Learning to Rank Based on Portfolio Theory and AMOSA Algorithm

Effectiveness is the most important factor considered in the ranking models yielded by algorithms of learning to rank (LTR). Most of the related ranking models only focus on improving the average effectiveness but ignore robustness. When a ranking model ignores robustness, the effectiveness for many queries is possibly very poor although the average effectiveness for all queries is relatively high. Therefore, Wang <italic>et al.</italic> first consider robustness in their ranking models. However, the robustness formula defined by Wang <italic>et al.</italic> cannot characterize those queries whose effectiveness are hurt seriously in comparison with the baseline model. In order to overcome this shortcoming, we propose a novel formula of characterizing robustness based on portfolio theory, and construct a multiobjective optimization model of the robust LTR in which the formula is used. Based on this model, we propose an approach of risk-sensitive and robust LTR, named as <inline-formula> <tex-math notation="LaTeX">$\text{R}^{ 2}$ </tex-math></inline-formula>Rank, which is based on the framework of archived multiobjective simulated annealing algorithm and the idea of preference ranking organization method for enrichment evaluation. The experimental results show that the ranking models produced by our proposed <inline-formula> <tex-math notation="LaTeX">$\text{R}^{ 2}$ </tex-math></inline-formula>Rank approach are better in both effectiveness and robustness than those produced by three state-of-the-art LTR approaches.

[1]  Xin Li,et al.  Incorporating robustness into web ranking evaluation , 2009, CIKM.

[2]  Guang-Bin Huang,et al.  Learning to Rank with Extreme Learning Machine , 2013, Neural Processing Letters.

[3]  Brian D. Davison,et al.  Learning to rank for freshness and relevance , 2011, SIGIR.

[4]  Ryoji Kataoka,et al.  Robust Online Learning to Rank via Selective Pairwise Approach Based on Evaluation Measures , 2013, Inf. Media Technol..

[5]  Timos K. Sellis,et al.  Learning to rank user intent , 2011, CIKM '11.

[6]  W. Bruce Croft,et al.  Linear feature-based models for information retrieval , 2007, Information Retrieval.

[7]  Jun Wang,et al.  Portfolio theory of information retrieval , 2009, SIGIR.

[8]  Ujjwal Maulik,et al.  A Simulated Annealing-Based Multiobjective Optimization Algorithm: AMOSA , 2008, IEEE Transactions on Evolutionary Computation.

[9]  Kevyn Collins-Thompson,et al.  Reducing the risk of query expansion via robust constrained optimization , 2009, CIKM.

[10]  Thomas Hofmann,et al.  Learning to Rank with Nonsmooth Cost Functions , 2006, NIPS.

[11]  Tie-Yan Liu,et al.  Future directions in learning to rank , 2010, Yahoo! Learning to Rank Challenge.

[12]  Ke Wang,et al.  A Cooperative Coevolution Framework for Parallel Learning to Rank , 2015, IEEE Transactions on Knowledge and Data Engineering.

[13]  Katja Hofmann,et al.  Online Learning to Rank: Absolute vs. Relative , 2015, WWW.

[14]  Brian D. Davison,et al.  Multi-objective optimization in learning to rank , 2011, SIGIR '11.

[15]  Yoram Singer,et al.  An Efficient Boosting Algorithm for Combining Preferences by , 2013 .

[16]  Xueqi Cheng,et al.  Top-k learning to rank: labeling, ranking and evaluation , 2012, SIGIR '12.

[17]  Katja Hofmann,et al.  Information Retrieval manuscript No. (will be inserted by the editor) Balancing Exploration and Exploitation in Listwise and Pairwise Online Learning to Rank for Information Retrieval , 2022 .

[18]  Thorsten Joachims,et al.  Optimizing search engines using clickthrough data , 2002, KDD.

[19]  Shuaiqiang Wang,et al.  Directly optimizing evaluation measures in learning to rank based on the clonal selection algorithm , 2008, SIGIR '08.

[20]  Qiang Wu,et al.  McRank: Learning to Rank Using Multiple Classification and Gradient Boosting , 2007, NIPS.

[21]  Paul N. Bennett,et al.  Robust ranking models via risk-sensitive optimization , 2012, SIGIR '12.

[22]  William W. Cohen,et al.  A Meta-Learning Approach for Robust Rank Learning , 2008 .

[23]  Vadim Mottl,et al.  A Bayesian Approach to Sparse Learning-to-Rank for Search Engine Optimization , 2015, MLDM.

[24]  Domingo Giménez,et al.  Parameterized Schemes of Metaheuristics: Basic Ideas and Applications With Genetic Algorithms, Scatter Search, and GRASP , 2013, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[25]  Hang Li,et al.  A Short Introduction to Learning to Rank , 2011, IEICE Trans. Inf. Syst..

[26]  Jonas Schmitt.  Portfolio Selection Efficient Diversification Of Investments , 2016 .

[27]  Raffaele Perego,et al.  QuickScorer: A Fast Algorithm to Rank Documents with Additive Ensembles of Regression Trees , 2015, SIGIR.

[28]  Qiang Wu,et al.  Adapting boosting for information retrieval measures , 2010, Information Retrieval.

[29]  Hongyuan Zha,et al.  Leveraging Auxiliary Data for Learning to Rank , 2012, TIST.

[30]  Tong Zhang,et al.  Subset Ranking Using Regression , 2006, COLT.

[31]  Maksims Volkovs,et al.  Learning to rank with multiple objective functions , 2011, WWW.

[32]  Jie Wu,et al.  Sparse Learning-to-Rank via an Efficient Primal-Dual Algorithm , 2013, IEEE Transactions on Computers.

[33]  Olivier Chapelle,et al.  Expected reciprocal rank for graded relevance , 2009, CIKM.

[34]  Jaana Kekäläinen,et al.  Cumulated gain-based evaluation of IR techniques , 2002, TOIS.

[35]  Witold Pedrycz,et al.  A Gradient-Descent-Based Approach for Transparent Linguistic Interface Generation in Fuzzy Models , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[36]  Rodrygo L. T. Santos,et al.  The whens and hows of learning to rank for web search , 2012, Information Retrieval.

[37]  Reza Baradaran Kazemzadeh,et al.  PROMETHEE: A comprehensive literature review on methodologies and applications , 2010, Eur. J. Oper. Res..

[38]  Hang Li,et al.  AdaRank: a boosting algorithm for information retrieval , 2007, SIGIR.

[39]  Craig MacDonald,et al.  Effect of dynamic pruning safety on learning to rank effectiveness , 2012, SIGIR '12.

[40]  Franco Scarselli,et al.  SortNet: Learning to Rank by a Neural Preference Function , 2011, IEEE Transactions on Neural Networks.

[41]  Tie-Yan Liu,et al.  Learning to rank: from pairwise approach to listwise approach , 2007, ICML '07.

[42]  Sanghamitra Bandyopadhyay,et al.  Multiobjective Simulated Annealing for Fuzzy Clustering With Stability and Validity , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[43]  Wei-Pang Yang,et al.  Learning to Rank for Information Retrieval Using Genetic Programming , 2007 .

[44]  Yong Tang,et al.  FSMRank: Feature Selection Algorithm for Learning to Rank , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[45]  Shuaiqiang Wang,et al.  An immune programming-based ranking function discovery approach for effective information retrieval , 2010, Expert Syst. Appl..