Automatic Combination of Multiple RankedRetrieval

Retrieval performance can often be improved signiicantly by using a number of diierent retrieval algorithms and combining the results, in contrast to using just a single retrieval algorithm. This is because diierent retrieval algorithms, or retrieval experts, often emphasize diierent document and query features when determining relevance and therefore retrieve diierent sets of documents. However, it is unclear how the diierent experts are to be combined, in general, to yield a superior overall estimate. We propose a method by which the relevance estimates made by diierent experts can be automatically combined to result in superior retrieval performance. We apply the method to two expert combination tasks. The applications demonstrate that the method can identify high performance combinations of experts and also is a novel means for determining the combined eeectiveness of experts.