GeoCLEF 2008: the CLEF 2008 Cross-Language Geographic Information Retrieval Track Overview

GeoCLEF is an evaluation task running under the scope of the Cross Language Evaluation Forum (CLEF). The purpose of GeoCLEF is to test and evaluate cross-language geographic information retrieval (GIR). The GeoCLEF 2008 task presented twenty-five geographically challenging search topics for English, German and Portuguese. Eleven participants submitted 131 runs, based on a variety of approaches, including sample documents, named entity extraction and ontology based retrieval. The evaluation methodology and results are presented in the paper.

[1]  Fredric C. Gey,et al.  GeoCLEF 2008: the CLEF 2008 Cross-Language Geographic Information Retrieval Track Overview , 2008, Conference and Labs of the Evaluation Forum.

[2]  Carol Peters,et al.  Multilingual Information Access for Text, Speech and Images, 5th Workshop of the Cross-Language Evaluation Forum, CLEF 2004, Bath, UK, September 15-17, 2004, Revised Selected Papers , 2005, CLEF.

[3]  Gobinda G. Chowdhury,et al.  TREC: Experiment and Evaluation in Information Retrieval , 2007 .

[4]  Mark Sanderson,et al.  Information retrieval system evaluation: effort, sensitivity, and reliability , 2005, SIGIR '05.

[5]  Diana Santos,et al.  The Key to the First CLEF with Portuguese: Topics, Questions and Answers in CHAVE , 2004, CLEF.

[6]  José Luis Vicedo González,et al.  TREC: Experiment and evaluation in information retrieval , 2007, J. Assoc. Inf. Sci. Technol..

[7]  Fredric C. Gey,et al.  GeoCLEF 2008: The CLEF 2008 Cross-Language Geographic Information Retrieval Track Overview , 2008, CLEF.

[8]  Paulo Rocha,et al.  Portuguese at CLEF , 2004 .

[9]  David A. Hull.  Using statistical testing in the evaluation of retrieval experiments , 1993, SIGIR.

[10]  Sven Hartrumpf,et al.  GikiP at GeoCLEF 2008: Joining GIR and QA Forces for Querying Wikipedia , 2008, CLEF.

[11]  Mário J. Silva,et al.  Challenges and resources for evaluating geographical IR , 2005, GIR '05.

[12]  E. Ziegel.  Introduction to the Theory and Practice of Econometrics , 1989 .

[13]  Fredric C. Gey,et al.  Challenges to Evaluation of Multilingual Geographic Information Retrieval in GeoCLEF , 2007, EVIA@NTCIR.

[14]  Fredric C. Gey,et al.  Accessing Multilingual Information Repositories, 6th Workshop of the Cross-Language Evalution Forum, CLEF 2005, Vienna, Austria, 21-23 September, 2005, Revised Selected Papers , 2006, CLEF.

[15]  Christiane Eilders.  The role of news factors in media use , 1996 .

[16]  Carol Peters,et al.  Accessing multilingual information repositories : 6th Workshop of the Cross-Language Evalution [i.e. Evaluation] Forum, CLEF 2005, Vienna, Austria, 21-23 September, 2005 ; revised selected papers , 2006 .

[17]  Giorgio Maria Di Nunzio,et al.  A Statistical and Graphical Methodology for Comparing Bilingual to Monolingual Cross-Language Information Retrieval , 2008, Information Access through Search Engines and Digital Libraries.

[18]  Ellen M. Voorhees,et al.  Retrieval System Evaluation , 2005 .

[19]  Diana Santos,et al.  The place of place in geographical IR , 2006, GIR.

[20]  Fredric C. Gey,et al.  An Evaluation Resource for Geographic Information Retrieval , 2008, LREC.

[21]  Carol Peters,et al.  Cross-Language Evaluation Forum: Objectives, Results, Achievements , 2004, Information Retrieval.

[22]  Carol Peters,et al.  Advances in Multilingual and Multimodal Information Retrieval, 8th Workshop of the Cross-Language Evaluation Forum, CLEF 2007, Budapest, Hungary, September 19-21, 2007, Revised Selected Papers , 2008, CLEF.

[23]  D. Whitefield,et al.  A review of: “Practical Nonpararnetric Statistics. By W. J. CONOVER. (New York: Wiley, 1971.) [Pl" x+462.] £5·25. , 1972 .

[24]  Peter Willett,et al.  Readings in information retrieval , 1997 .

[25]  Jean Tague-Sutcliffe,et al.  The Pragmatics of Information Retrieval Experimentation Revisited , 1997, Inf. Process. Manag..

[26]  Giorgio Maria Di Nunzio,et al.  DIRECT: A System for Evaluating Information Access Components of Digital Libraries , 2005, ECDL.

[27]  Xing Xie,et al.  MSRA Columbus at GeoCLEF 2006 , 2006, CLEF.

[28]  Maristella Agosti.  Information Access through Search Engines and Digital Libraries , 2008, The Information Retrieval Series.

[29]  Eckhard Bick,et al.  Providing Internet Access to Portuguese Corpora: the AC/DC Project , 2000, LREC.

[30]  W. J. Conover,et al.  Practical Nonparametric Statistics , 1972 .

[31]  Fredric C. Gey,et al.  ENSM-SE at CLEF 2006 : Fuzzy Proximity Method with an Adhoc Influence Function in Evaluation of Multilingual and Multi-modal Information Retrieval 7th Workshop of the Cross-Language Evaluation Forum, CLEF 2006, Alicante, Spain , 2007 .

[32]  Diana Santos,et al.  Portuguese at CLEF 2005: Reflections and Challenges , 2005, CLEF.

[33]  Michael Kluck,et al.  Inside the Evaluation Process of the Cross-Language Evaluation Forum (CLEF): Issues of Multilingual Topic Creation and Multilingual Relevance Assessment , 2002, LREC.

[34]  Sven Hartrumpf,et al.  Getting Geographical Answers fromWikipedia: the GikiP pilot at CLEF , 2008, CLEF.

[35]  Justin Zobel,et al.  How reliable are the results of large-scale information retrieval experiments? , 1998, SIGIR '98.

[36]  Diana Santos,et al.  Portuguese at CLEF 2005 , 2005, CLEF.

[37]  Xing Xie,et al.  Query Parsing Task for GeoCLEF2007 Report , 2007, CLEF.

[38]  Fredric C. Gey,et al.  GeoCLEF: the CLEF 2005 Cross-Language Geographic Information Retrieval Track , 2005, CLEF.