Learning non-linear ranking functions for web search using probabilistic model building GP

Ranking the set of search results according to their relevance to a user query is an important task in an Information Retrieval (IR) systems such as a Web Search Engine. Learning the optimal ranking function for this task is a challenging problem because one must consider complex non-linear interactions between numerous factors such as the novelty, authority, contextual similarity, etc. of thousands of documents that contain the user query. We model this task as a non-linear ranking problem, for which we propose Rank-PMBGP, an efficient algorithm to learn an optimal non-linear ranking function using Probabilistic Model Building Genetic Programming. We evaluate the proposed method using the LETOR dataset, a standard benchmark dataset for training and evaluating ranking functions for IR. In our experiments, the proposed method obtains a Mean Average Precision (MAP) score of 0.291, thereby significantly outperforming a non-linear baseline approach that uses Genetic Programming.

[1]  Pável Calado,et al.  A combined component approach for finding collection-adapted ranking functions based on genetic programming , 2007, SIGIR.

[2]  Kotaro Hirasawa,et al.  Comparison between Genetic Network Programming (GNP) and Genetic Programming (GP) , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[3]  J. Friedman.  Stochastic gradient boosting , 2002 .

[4]  Mark Wineberg,et al.  A Representation Scheme To Perform Program Induction in a Canonical Genetic Algorithm , 1994, PPSN.

[5]  William F. Punch.  HOW EFFECTIVE ARE MULTIPLE POPULATIONS IN GENETIC PROGRAMMING , 1998 .

[6]  Thorsten Joachims,et al.  Optimizing search engines using clickthrough data , 2002, KDD.

[7]  Rafal Salustowicz,et al.  Probabilistic Incremental Program Evolution , 1997, Evolutionary Computation.

[8]  Y. Freund,et al.  Discussion of the Paper \additive Logistic Regression: a Statistical View of Boosting" By , 2000 .

[9]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[10]  Tao Qin,et al.  Query-level loss functions for information retrieval , 2008, Inf. Process. Manag..

[11]  Tao Qin,et al.  FRank: a ranking method with fidelity loss , 2007, SIGIR.

[12]  Hinrich Schütze,et al.  Introduction to information retrieval , 2008 .

[13]  Ralf Herbrich,et al.  Large margin rank boundaries for ordinal regression , 2000 .

[14]  CHENGXIANG ZHAI,et al.  A study of smoothing methods for language models applied to information retrieval , 2004, TOIS.

[15]  LETOR : A Benchmark Collection LETOR : A Benchmark Collection for Learning to Rank for Information Retrieval by Microsoft Research Asia , an incomplete draft , 2008 .

[16]  Wei-Pang Yang,et al.  Designing a classifier by a layered multi-population genetic programming approach , 2007, Pattern Recognit..

[17]  Gregory N. Hullender,et al.  Learning to rank using gradient descent , 2005, ICML.

[18]  Tao Qin,et al.  LETOR: Benchmark Dataset for Research on Learning to Rank for Information Retrieval , 2007 .

[19]  Weiguo Fan,et al.  Genetic Programming-Based Discovery of Ranking Functions for Effective Web Search , 2005, J. Manag. Inf. Syst..

[20]  Yoram Singer,et al.  An Efficient Boosting Algorithm for Combining Preferences by , 2013 .

[21]  Danushka Bollegala,et al.  RankDE: learning a ranking function for information retrieval using differential evolution , 2011, GECCO '11.

[22]  Jürgen Schmidhuber,et al.  Probabilistic Incremental Program Evolution: Stochastic Search Through Program Space , 1997, ECML.

[23]  Hitoshi Iba,et al.  Genetic Programming 1998: Proceedings of the Third Annual Conference , 1999, IEEE Trans. Evol. Comput..

[24]  Stephen E. Robertson,et al.  Overview of the Okapi projects , 1997, J. Documentation.

[25]  Thore Graepel,et al.  Large Margin Rank Boundaries for Ordinal Regression , 2000 .

[26]  Tie-Yan Liu,et al.  Listwise approach to learning to rank: theory and algorithm , 2008, ICML '08.

[27]  Tie-Yan Liu,et al.  Generalization analysis of listwise learning-to-rank algorithms , 2009, ICML '09.

[28]  Brian D. Davison,et al.  Topical link analysis for web search , 2006, SIGIR.

[29]  Peter A. N. Bosman,et al.  Grammar Transformations in an EDA for Genetic Programming , 2004 .

[30]  Azadeh Shakery,et al.  Relevance Propagation for Topic Distillation UIUC TREC 2003 Web Track Experiments , 2003, TREC.

[31]  Qiang Yang,et al.  Exploiting the hierarchical structure for link analysis , 2005, SIGIR '05.

[32]  Tao Qin,et al.  A study of relevance propagation for web search , 2005, SIGIR '05.

[33]  Klaus Obermayer,et al.  Support vector learning for ordinal regression , 1999 .

[34]  Rajeev Motwani,et al.  The PageRank Citation Ranking : Bringing Order to the Web , 1999, WWW 1999.

[35]  Koby Crammer,et al.  Pranking with Ranking , 2001, NIPS.

[36]  Danushka Bollegala,et al.  Probabilistic model building GP with Belief propagation , 2012, 2012 IEEE Congress on Evolutionary Computation.

[37]  Lars Schmidt-Thieme,et al.  Swarming to rank for information retrieval , 2009, GECCO.

[38]  Yi Chang,et al.  Yahoo! Learning to Rank Challenge Overview , 2010, Yahoo! Learning to Rank Challenge.

[39]  Robert I. McKay,et al.  Sampling Bias in Estimation of Distribution Algorithms for Genetic Programming Using Prototype Trees , 2010, PRICAI.

[40]  Martin Pelikan,et al.  An introduction and survey of estimation of distribution algorithms , 2011, Swarm Evol. Comput..

[41]  Tie-Yan Liu,et al.  Learning to rank: from pairwise approach to listwise approach , 2007, ICML '07.

[42]  Wei-Pang Yang,et al.  Learning to Rank for Information Retrieval Using Genetic Programming , 2007 .

[43]  Shingo Mabu,et al.  A Novel Graph-Based Estimation of the Distribution Algorithm and its Extension Using Reinforcement Learning , 2014, IEEE Transactions on Evolutionary Computation.

[44]  Hitoshi Iba,et al.  A Bayesian Network Approach to Program Generation , 2008, IEEE Transactions on Evolutionary Computation.

[45]  P. Ross,et al.  An Adverse Interaction between the Crossover Operator and a Restriction on Tree Depth , 1995 .

[46]  P. Ross,et al.  An adverse interaction between crossover and restricted tree depth in genetic programming , 1996 .

[47]  Weiguo Fan,et al.  Discovery of context-specific ranking functions for effective information retrieval using genetic programming , 2004, IEEE Transactions on Knowledge and Data Engineering.

[48]  Harris Wu,et al.  The effects of fitness functions on genetic programming-based ranking discovery forWeb search , 2004, J. Assoc. Inf. Sci. Technol..

[49]  Weiguo Fan,et al.  Personalization of search engine services for effective retrieval and knowledge management , 2000, ICIS.