Relevance and Overlap Aware Text Collection Selection

In an environment of distributed text collections, the first step in the information retrieval process is to identify which of all available collections are more relevant to a given query and should thus be accessed to answer the query. Collection selection is difficult due to the varying relevance of sources as well as the overlap between these sources. Previous collection selection methods have considered relevance of the collections but have ignored overlap among collections. They thus make the unrealistic assumption that the collections are all effectively disjoint. In this paper, we describe two new approaches: (i) COSCO which handles collection overlap, and (ii) ROSCO, which builds on COSCO to handle both collection relevance and collection overlap. We start by developing methods for estimating the statistics concerning size, relevance, and overlap that are necessary to support collection selection. We then explain how COSCO and ROSCO select text collections based upon these statistics. Finally, we demonstrate the effectiveness of COSCO and ROSCO by comparing them to major text collection selection algorithms (CORI and RDDE) under a variety of scenarios. Our evaluation is based on a set of 8 testbeds drawn from online scientific paper collections that vary systematically across relevance, overlap and size.