CLEF 2009 Ad Hoc Track Overview: Robust - WSD Task

The Robust-WSD at CLEF 2009 aims at exploring the contribution ofWord Sense Disambiguation to monolingual and multilingual Information Retrieval. The organizers of the task provide documents and topics which have been automatically tagged with Word Senses from WordNet using several state-of-the-art Word Sense Disambiguation systems. The Robust-WSD exercise follows the same design as in 2008. It uses two languages often used in previous CLEF campaigns (English, Spanish). Documents were in English, and topics in both English and Spanish. The document collections are based on the widely used LA94 and GH95 news collections. All instructions and datasets required to replicate the experiment are available from the organizers website (http://ixa2.si.ehu.es/clirwsd/). The results show that some top-scoring systems improve their IR and CLIR results with the use of WSD tags, but the best scoring runs do not use WSD.

[1]  Peter Willett,et al.  Readings in information retrieval , 1997 .

[2]  Cyril Cleverdon,et al.  The Cranfield tests on index language devices , 1997 .

[3]  Carol Peters,et al.  CLEF 2003 Methodology and Metrics , 2003, CLEF.

[4]  Carol Peters,et al.  Comparative Evaluation of Multilingual Information Access Systems , 2003, Lecture Notes in Computer Science.

[5]  Ellen M. Voorhees,et al.  The TREC robust retrieval track , 2005, SIGF.

[6]  C. Peters,et al.  Comparative Evaluation of Multilingual Information Access Systems: 4th Workshop of the Cross-Language Evaluation Forum, CLEF 2003, Trondheim, Norway, August ... Papers (Lecture Notes in Computer Science) , 2005 .

[7]  Stephen E. Robertson,et al.  On GMAP: and other transformations , 2006, CIKM '06.

[8]  Piek T. J. M. Vossen,et al.  SemEval-2007 Task 01: Evaluating WSD on Cross-Language Information Retrieval , 2007, Fourth International Workshop on Semantic Evaluations (SemEval-2007).

[9]  Kurt Bilde,et al.  En forskningsartikel: This is a test , 2007 .

[10]  Eneko Agirre,et al.  UBC-ALM: Combining k-NN with SVD for WSD , 2007, SemEval@ACL.

[11]  Martha Palmer,et al.  SemEval-2007 Task-17: English Lexical Sample, SRL and All Words , 2007, Fourth International Workshop on Semantic Evaluations (SemEval-2007).

[12]  Hwee Tou Ng,et al.  NUS-PT: Exploiting Parallel Texts for Word Sense Disambiguation in the English All-Words Tasks , 2007, Fourth International Workshop on Semantic Evaluations (SemEval-2007).

[13]  Carol Peters,et al.  Advances in Multilingual and Multimodal Information Retrieval, 8th Workshop of the Cross-Language Evaluation Forum, CLEF 2007, Budapest, Hungary, September 19-21, 2007, Revised Selected Papers , 2008, CLEF.

[14]  Arantxa Otegi,et al.  IXA at CLEF 2008 Robust-WSD Task: using Word Sense Disambiguation for (Cross Lingual) Information Retrieval , 2008, CLEF.

[15]  Maristella Agosti.  Information Access through Search Engines and Digital Libraries , 2008, The Information Retrieval Series.

[16]  Giorgio Maria Di Nunzio,et al.  How robust are multilingual information retrieval systems? , 2008, SAC '08.

[17]  Carol Peters,et al.  CLEF 2008: Ad Hoc Track Overview , 2008, CLEF.

[18]  Giorgio Maria Di Nunzio,et al.  A Statistical and Graphical Methodology for Comparing Bilingual to Monolingual Cross-Language Information Retrieval , 2008, Information Access through Search Engines and Digital Libraries.

[19]  Gilles Falquet,et al.  UniGe at CLEF 2009 Robust WSD Task (abstract only) , 2009, CLEF.

[20]  Ángel F. Zazo Rodríguez,et al.  REINA at CLEF 2009 Robust-WSD Task: Partial Use of WSD Information for Retrieval , 2009, CLEF.

[21]  Roman Kern,et al.  Evaluation of Axiomatic Approaches to Crosslanguage Retrieval , 2009, CLEF.

[22]  Iryna Gurevych,et al.  Combining Probabilistic and Translation-Based Models for Information Retrieval based on Word Sense Annotations , 2009, CLEF.

[23]  Arantxa Otegi,et al.  Using Semantic Relatedness and Word Sense Disambiguation for (CL)IR , 2009, CLEF.

[24]  Carol Peters,et al.  Evaluating Systems for Multilingual and Multimodal Information Access, 9th Workshop of the Cross-Language Evaluation Forum, CLEF 2008, Aarhus, Denmark, September 17-19, 2008, Revised Selected Papers , 2009, CLEF.

[25]  Viviane Pereira Moreira,et al.  UFRGS@CLEF2009: Retrieval by Numbers , 2009, CLEF.

[26]  Roman Kern,et al.  Application of Axiomatic Approaches to Crosslanguage Retrieval , 2009, CLEF.

[27]  Annalina Caputo,et al.  UNIBA-SENSE @ CLEF 2009: Robust WSD task , 2009, CLEF.