We took part in the HARD track, with an active learning method to choose which document snippets to show the user for relevance feedback (compared to baseline feedback using snippets from the top-ranked documents). The active learning method is described, and some prior experiments with the Reuters collection are summarised. We also invited user feedback on phrases chosen from the top retrieved documents, and made some use of the ‘relt’ relevant texts provided as part of the metadata. Unfortunately, our results on the HARD task were not good: in most runs, feedback hurt performance, and the active learning feedback hurt more than the baseline feedback. The only runs that improved slightly on the no-feedback runs were a couple of baseline feedback runs.
[1]
Stephen E. Robertson,et al.
Okapi at TREC-5
,
1996,
TREC.
[2]
Stephen E. Robertson,et al.
Microsoft Cambridge at TREC-9: Filtering Track
,
2000,
TREC.
[3]
Marcel Worring,et al.
NIST Special Publication
,
2005
.
[4]
David A. Cohn,et al.
Active Learning with Statistical Models
,
1996,
NIPS.
[5]
David J. C. MacKay,et al.
Information-Based Objective Functions for Active Data Selection
,
1992,
Neural Computation.
[6]
Stephen E. Robertson,et al.
Okapi/Keenbow at TREC-8
,
1999,
TREC.
[7]
Stephen E. Robertson,et al.
Microsoft Cambridge at TREC 2002: Filtering Track
,
2002,
TREC.
[8]
Stephen E. Robertson,et al.
Okapi at TREC-3
,
1994,
TREC.