Early exit optimizations for additive machine learned ranking systems

Some commercial web search engines rely on sophisticated machine learning systems for ranking web documents. Due to very large collection sizes and tight constraints on query response times, online efficiency of these learning systems forms a bottleneck. An important problem in such systems is to speedup the ranking process without sacrificing much from the quality of results. In this paper, we propose optimization strategies that allow short-circuiting score computations in additive learning systems. The strategies are evaluated over a state-of-the-art machine learning system and a large, real-life query log, obtained from Yahoo!. By the proposed strategies, we are able to speedup the score computations by more than four times with almost no loss in result quality.

[1]  Howard R. Turtle,et al.  Query Evaluation: Strategies and Optimizations , 1995, Inf. Process. Manag..

[2]  Ronald Fagin,et al.  Combining Fuzzy Information from Multiple Systems , 1999, J. Comput. Syst. Sci..

[3]  Filip Radlinski,et al.  A support vector method for optimizing average precision , 2007, SIGIR.

[4]  Berkant Barla Cambazoglu,et al.  Quantifying performance and quality gains in distributed web search engines , 2009, SIGIR.

[5]  W. Bruce Croft,et al.  Optimization strategies for complex queries , 2005, SIGIR '05.

[6]  Clifford Stein,et al.  Introduction to Algorithms, 2nd edition. , 2001 .

[7]  Donna Harman,et al.  Retrieving Records from a Gigabyte of Text on a Minicomputer Using Statistical Ranking. , 1990 .

[8]  Eric W. Brown,et al.  Fast evaluation of structured queries for information retrieval , 1995, SIGIR '95.

[9]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[10]  Alistair Moffat,et al.  Vector-space ranking with effective early termination , 2001, SIGIR '01.

[11]  Ronald Fagin,et al.  Combining fuzzy information: an overview , 2002, SGMD.

[12]  Qiang Wu,et al.  McRank: Learning to Rank Using Multiple Classification and Gradient Boosting , 2007, NIPS.

[13]  Dik Lun Lee,et al.  Implementations of Partial Document Ranking Using Inverted Files , 1993, Information Processing & Management.

[14]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[15]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[16]  Shlomo Moran,et al.  Predictive caching and prefetching of query results in search engines , 2003, WWW '03.

[17]  Alistair Moffat,et al.  Pruned query evaluation using pre-computed impacts , 2006, SIGIR.

[18]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[19]  Michael Persin,et al.  Document filtering for fast ranking , 1994, SIGIR '94.

[20]  Ronald L. Rivest,et al.  Introduction to Algorithms , 1990 .

[21]  Sergei Vassilvitskii,et al.  Top-k aggregation using intersections of ranked inputs , 2009, WSDM '09.

[22]  Hongyuan Zha,et al.  A General Boosting Method and its Application to Learning Ranking Functions for Web Search , 2007, NIPS.

[23]  R. Tibshirani,et al.  Generalized Additive Models , 1991 .

[24]  Alistair Moffat,et al.  Self-indexing inverted files for fast text retrieval , 1996, TOIS.

[25]  Tong Zhang,et al.  Subset Ranking Using Regression , 2006, COLT.

[26]  Dennis DeCoste,et al.  Anytime Interval-Valued Outputs for Kernel Machines: Fast Support Vector Machine Classification via Distance Geometry , 2002, ICML.

[27]  Chris Buckley,et al.  Optimization of inverted vector searches , 1985, SIGIR '85.

[28]  Marc Najork,et al.  Hits on the web: how does it compare? , 2007, SIGIR.

[29]  Stephen E. Robertson,et al.  Optimisation methods for ranking functions with multiple parameters , 2006, CIKM '06.

[30]  Tie-Yan Liu,et al.  Adapting ranking SVM to document retrieval , 2006, SIGIR.

[31]  Tao Qin,et al.  LETOR: Benchmark Dataset for Research on Learning to Rank for Information Retrieval , 2007 .

[32]  Thomas H. Cormen,et al.  Introduction to algorithms [2nd ed.] , 2001 .