Hummingbird submitted ranked result sets for all Monolingual Information Retrieval tasks of the Cross-Language Evaluation Forum (CLEF) 2002. Enabling stemming in SearchServer increased average precision by 16 points in Finnish, 9 points in German, 4 points in Spanish, 3 points in Dutch, 2 points in French and Italian, and 1 point in Swedish and English. Accent-indexing increased average precision by 3 points in Finnish and 2 points in German, but decreased it by 2 points in French and 1 point in Italian and Swedish. Treating apostrophes as word separators increased average precision by 3 points in French and 1 point in Italian. Confidence intervals produced using the bootstrap percentile method were found to be very similar to those produced using the standard method; both were of similar width to rank-based intervals for differences in average precision, but substantially narrower for differences in Precision@10.
[1]
Stephen Tomlinson,et al.
Hummingbird's Fulcrum SearchServer at TREC-9
,
2000,
TREC.
[2]
K. Sparck Jones,et al.
A Probabilistic Model of Information Retrieval : Development and Status
,
1998
.
[3]
Stephen E. Robertson,et al.
Okapi at TREC-3
,
1994,
TREC.
[4]
Douglas A. Wolfe,et al.
Nonparametric Statistical Methods
,
1973
.
[5]
Michael R. Chernick,et al.
Bootstrap Methods: A Practitioner's Guide
,
1999
.
[6]
Stephen Tomlinson.
Stemming Evaluated in 6 Languages by Hummingbird SearchServerTM at CLEF 2001
,
2001,
CLEF.
[7]
Amit Singhal,et al.
AT&T at TREC-7
,
1998,
TREC.
[8]
James T. Wassell,et al.
Bootstrap Methods: A Practitioner's Guide
,
2001,
Technometrics.