Ranked queries over sources with Boolean query interfaces without ranking support

Many online or local data sources provide powerful querying mechanisms but limited ranking capabilities. For instance, PubMed allows users to submit highly expressive Boolean keyword queries, but ranks the query results by date only. However, a user would typically prefer a ranking by relevance, measured by an Information Retrieval (IR) ranking function. The naive approach would be to submit a disjunctive query with all query keywords, retrieve the returned documents, and then re-rank them. Unfortunately, such an operation would be very expensive due to the large number of results returned by disjunctive queries. In this paper we present algorithms that return the top results for a query, ranked according to an IR-style ranking function, while operating on top of a source with a Boolean query interface with no ranking capabilities (or a ranking capability of no interest to the end user). The algorithms generate a series of conjunctive queries that return only documents that are candidates for being highly ranked according to a relevance metric. Our approach can also be applied to other settings where the ranking is monotonic on a set of factors (query keywords in IR) and the source query interface is a Boolean expression of these factors. Our comprehensive experimental evaluation on the PubMed database and TREC dataset show that we achieve order of magnitude improvement compared to the current baseline approaches.

[1]  Gerhard Weikum,et al.  Top-k Query Evaluation with Probabilistic Guarantees , 2004, VLDB.

[2]  Stephen E. Robertson,et al.  Okapi at TREC-3 , 1994, TREC.

[3]  P. W. Jones,et al.  Bandit Problems, Sequential Allocation of Experiments , 1987 .

[4]  Stephen E. Robertson,et al.  GatfordCentre for Interactive Systems ResearchDepartment of Information , 1996 .

[5]  Panagiotis G. Ipeirotis,et al.  Efficient Ranked Queries on Sources with Boolean Query Interfaces , 2009 .

[6]  Jayant Madhavan,et al.  Google's Deep Web crawl , 2008, Proc. VLDB Endow..

[7]  Michael McGill,et al.  Introduction to Modern Information Retrieval , 1983 .

[8]  Michael K. Bergman.  White Paper: The Deep Web: Surfacing Hidden Value , 2001 .

[9]  Petros Zerfos,et al.  Downloading textual hidden web content through keyword queries , 2005, Proceedings of the 5th ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL '05).

[10]  David H. Wolpert,et al.  Bandit problems and the exploration/exploitation tradeoff , 1998, IEEE Trans. Evol. Comput..

[11]  Ihab F. Ilyas,et al.  A survey of top-k query processing techniques in relational database systems , 2008, CSUR.

[12]  Ronald Fagin,et al.  Comparing top k lists , 2003, SODA '03.

[13]  Luis Gravano,et al.  Distributed Search over the Hidden Web: Hierarchical Database Sampling and Selection , 2002, VLDB.

[14]  James P. Callan,et al.  Query-based sampling of text databases , 2001, TOIS.

[15]  B. Huberman,et al.  The Deep Web : Surfacing Hidden Value , 2000 .

[16]  Jongseok Lee,et al.  Exploration and Exploitation in the Presence of Network Externalities , 2003, Manag. Sci..