Improving Dynamic Index Pruning via Linear Programming

Dynamic index pruning techniques are commonly used to speed up query processing in Web search engines. In this work, we propose a linear programming technique which can further improve the performance of the state-of-the-art dynamic index pruning techniques. The experiments we conducted demonstrate that the proposed technique achieves reduction in terms of the disk access, index decompression, and scoring costs compared to the well-known Max-Score technique.

[1]  Torsten Suel,et al.  Faster top-k document retrieval using block-max indexes , 2011, SIGIR.

[2]  Marcus Fontoura,et al.  Evaluation strategies for top-k queries over memory-resident inverted indexes , 2011, Proc. VLDB Endow..

[3]  Svein Erik Bratsberg,et al.  Efficient Compressed Inverted Index Skipping for Disjunctive Text-Queries , 2011, ECIR.

[4]  Simon Jonassen,et al.  Efficient query processing in distributed search engines , 2013, SIGF.

[5]  Craig MacDonald,et al.  Upper-bound approximations for dynamic pruning , 2011, TOIS.

[6]  Torsten Suel,et al.  Optimizing top-k document retrieval strategies for block-max indexes , 2013, WSDM.

[7]  Hongfei Yan,et al.  Optimized top-k processing with global page scores on block-max indexes , 2012, WSDM '12.

[8]  Howard R. Turtle,et al.  Query Evaluation: Strategies and Optimizations , 1995, Inf. Process. Manag..

[9]  Surajit Chaudhuri,et al.  Interval-based pruning for top-k processing over compressed lists , 2011, 2011 IEEE 27th International Conference on Data Engineering.

[10]  Svein Erik Bratsberg,et al.  Intra-query Concurrent Pipelined Processing for Distributed Full-Text Retrieval , 2012, ECIR.

[11]  W. Bruce Croft,et al.  Optimization strategies for complex queries , 2005, SIGIR '05.

[12]  Berkant Barla Cambazoglu,et al.  Query forwarding in geographically distributed search engines , 2010, SIGIR.

[13]  Craig MacDonald,et al.  Learning to predict response times for online query scheduling , 2012, SIGIR '12.

[14]  Andrei Z. Broder,et al.  Efficient query evaluation using a two-level retrieval process , 2003, CIKM '03.

[15]  Svein Erik Bratsberg,et al.  Improving the performance of pipelined query processing with skipping—and its comparison to document-wise partitioning , 2013, World Wide Web.

[16]  Mauricio Marín,et al.  Efficient Parallel Block-Max WAND Algorithm , 2013, Euro-Par.