Creating a Dutch Information Retrieval Test Corpus

This paper describes the first large-scale evaluation of information retrieval systems using Dutch documents and queries. We describe in detail the characteristics of the Dutch test data, which is part of the official CLEF multilingual test corpus, and give an overview of the experimental results of companies and research institutions that participated in the first official Dutch CLEF experiments. Judging from these experiments, the handling of languagespecific issues of Dutch, like for instance simple morphology and compound nouns, significantly improves the performance of information retrieval systems in many cases. Careful examination of the test collection shows that it serves as a reliable tool for the evaluation of information retrieval systems in the future.

[1]  Mirna Adriani.  English-Dutch CLIR Using Query Translation Techniques , 2001, CLEF.

[2]  Stephen P. Harter,et al.  Variations in Relevance Assessments and the Measurement of Retrieval Effectiveness , 1996, J. Am. Soc. Inf. Sci..

[3]  Douglas W. Oard.  The CLEF 2001 Interactive Track , 2001, CLEF.

[4]  James Mayfield,et al.  JHU/APL Experiments at CLEF: Translation Resources and Score Normalization , 2001, CLEF.

[5]  Michael E. Lesk,et al.  Relevance assessments and retrieval system evaluation , 1968, Inf. Storage Retr..

[6]  Tefko Saracevic,et al.  RELEVANCE: A review of and a framework for the thinking on the notion in information science , 1997, J. Am. Soc. Inf. Sci..

[7]  Wessel Kraaij,et al.  TNO at CLEF-2001: Comparing Translation Resources , 2001, CLEF.

[8]  Justin Zobel,et al.  How reliable are the results of large-scale information retrieval experiments? , 1998, SIGIR '98.

[9]  Annius Groenink,et al.  Minimalistic Test Runs of the Eidetica Indexer , 2001, CLEF.

[10]  Marvin Brünner,et al.  Some Terms are more Interchangeable than Others , 2001, CLEF.

[11]  Isabelle Moulinier,et al.  Thomson Legal and Regulatory at CLEF 2001: Monolingual and Bilingual Experiments , 2001, CLEF.

[12]  Ellen M. Voorhees.  Variations in relevance judgments and the measurement of retrieval effectiveness , 2000, Inf. Process. Manag..

[13]  Ellen M. Voorhees,et al.  The Ninth Text REtrieval Conference (TREC-9) , 2001 .

[14]  Maarten de Rijke,et al.  The University of Amsterdam at CLEF 2003 , 2001, CLEF.

[15]  C. J. van Rijsbergen,et al.  Report on the need for and provision of an 'ideal' information retrieval test collection , 1975 .

[16]  Peter van der Weerd,et al.  First Experiments with CLEF , 2001, CLEF.

[17]  Stephen Tomlinson.  Hummingbird's Fulcrum SearchServer at CLEF2001 , 2001, CLEF.

[18]  Stefano Mizzaro.  Relevance: the whole history , 1997 .

[19]  Brian Vickery,et al.  Techniques of information retrieval , 1970 .

[20]  David A. Hull.  Using statistical testing in the evaluation of retrieval experiments , 1993, SIGIR.