Improving the Efficiency of Retrieval Effectiveness Evaluation: Finding a Few Good Topics with Clustering?

We consider the issue of using fewer topics in the effectiveness evaluation of information retrieval systems. Previous work has shown that using fewer topics is theoretically possible; one of the main issues that remains to be solved is how to find such a small set of a few good topics. To this aim, in this paper we try a novel approach based on clustering of topics. We consider various algorithms, metrics, and various features of topics that can be helpful in identifying such a set.