INEX 2002 - 2006: Understanding XML Retrieval Evaluation

Evaluating the effectiveness of XML retrieval requires building test collections where the evaluation paradigms are provided according to criteria that take into account structural aspects. The INitiative for the Evaluation of XML retrieval (INEX) was set up in 2002, and aimed to establish an infrastructure and to provide means, in the form of large test collections and appropriate scoring methods, for evaluating the effectiveness of content-oriented XML retrieval. This paper describes the evaluation methodology developed in INEX, with particular focus on how evaluation metrics and the notion of relevance are treated.

[1]  Sihem Amer-Yahia,et al.  XML search: languages, INEX and scoring , 2006, SGMD.

[2]  RaghavanVijay,et al.  A critical investigation of recall and precision as measures of retrieval system performance , 1989 .

[3]  Gabriella Kazai,et al.  Evaluating the effectiveness of content-oriented XML retrieval , 2003 .

[4]  Andrew Trotman,et al.  Wanted : Element Retrieval Users , 2005 .

[5]  Birger Larsen,et al.  Report on the INEX 2004 interactive track , 2005, SIGF.

[6]  Gabriella Kazai.  Initiative for the Evaluation of XML Retrieval , 2009 .

[7]  Gabriella Kazai,et al.  The overlap problem in content-oriented XML retrieval evaluation , 2004, SIGIR '04.

[8]  Andrew Trotman,et al.  Report on the INEX 2005 workshop on element retrieval methodology , 2005, SIGF.

[9]  Jaana Kekäläinen,et al.  Cumulated gain-based evaluation of IR techniques , 2002, TOIS.

[10]  Jaana Kekäläinen,et al.  Using graded relevance assessments in IR evaluation , 2002, J. Assoc. Inf. Sci. Technol..

[11]  CarmelDavid,et al.  XML and information retrieval , 2000 .

[12]  Mounia Lalmas,et al.  Providing consistent and exhaustive relevance assessments for XML retrieval evaluation , 2004, CIKM '04.

[13]  Mounia Lalmas,et al.  Investigating the exhaustivity dimension in content-oriented XML element retrieval evaluation , 2006, CIKM '06.

[14]  Gerhard Weikum,et al.  Intelligent Search on XML Data: Applications, Languages, Models, Implementations, and Benchmarks , 2003 .

[15]  Ludovic Denoyer,et al.  The Wikipedia XML Corpus , 2006, INEX.

[16]  Vijay V. Raghavan,et al.  A critical investigation of recall and precision as measures of retrieval system performance , 1989, TOIS.

[17]  Gabriella Kazai,et al.  A Study of the Assessment of Relevance for the INEX'02 Test Collection , 2004, ECIR.

[18]  Ludovic Denoyer,et al.  The XML Wikipedia Corpus , 2006 .

[19]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[20]  Gabriella Kazai,et al.  eXtended cumulated gain measures for the evaluation of content-oriented XML retrieval , 2006, TOIS.

[21]  Andrew Trotman,et al.  Narrowed Extended XPath I (NEXI) , 2004, INEX.

[22]  Gabriella Kazai,et al.  Overview of the Initiative for the Evaluation of XML retrieval (INEX) 2002 , 2002, INEX Workshop.

[23]  Andrew Trotman,et al.  Strict and vague interpretation of XML-retrieval queries , 2006, SIGIR.

[24]  Gerhard Weikum,et al.  Intelligent Search on XML Data , 2003, Lecture Notes in Computer Science.