The Interpretation of CAS

There has been much debate over how to interpret the structure in queries that contain structural hints. At INEX 2003 and 2004, there were two interpretations: SCAS in which the user specified target element was interpreted strictly, and VCAS in which it was interpreted vaguely. But how many ways are there that the query could be interpreted? In the investigation at INEX 2005 (discussed herein) four different interpretations were proposed, and compared on the same queries. Those interpretations (SSCAS, SVCAS, VSCAS, and VVCAS) are the four interpretations possible by interpreting the target elements, and the support elements, either strictly or vaguely. An analysis of the submitted runs shows that those that share an interpretation of the target element correlate – that is, the previous decision to divide CAS into the SCAS and VCAS (as done at INEX 2003 and 2004) was sound. The analysis is supported by the fact that the best performing VSCAS run was submitted to the VVCAS task and the best performing SVCAS run was submitted to the SSCAS task.

[1]  Andrew Trotman,et al.  INEX 2005 guidelines for topic development , 2005 .

[2]  Birger Larsen,et al.  The Interactive Track at INEX 2004 , 2004, INEX.

[3]  Andrew Trotman,et al.  Wanted : Element Retrieval Users , 2005 .

[4]  Andrew Trotman,et al.  Narrowed Extended XPath I (NEXI) , 2004, INEX.

[5]  Andrew Trotman,et al.  NEXI, Now and Next , 2004, INEX.

[6]  Ellen M. Voorhees,et al.  The Philosophy of Information Retrieval Evaluation , 2001, CLEF.

[7]  Shlomo Geva,et al.  NLPX at INEX 2004 , 2004, INEX.

[8]  Justin Zobel,et al.  How reliable are the results of large-scale information retrieval experiments? , 1998, SIGIR '98.

[9]  Donna K. Harman,et al.  Overview of the first TREC conference , 1993, SIGIR.

[10]  James A. Thom,et al.  Users and Assessors in the Context of INEX: Are Relevance Dimensions Relevant? , 2005, ArXiv.

[11]  Shlomo Geva,et al.  Fine Tuning INEX , 2005 .

[12]  Mounia Lalmas,et al.  Providing consistent and exhaustive relevance assessments for XML retrieval evaluation , 2004, CIKM '04.

[13]  Mounia Lalmas,et al.  Interface pour l'évaluation de systèmes de recherche sur des documents XML , 2004, CORIA.

[14]  Gabriella Kazai,et al.  Advances in XML Information Retrieval and Evaluation: 4th International Workshop of the Initiative for the Evaluation of XML Retrieval, INEX 2005, Dagstuhl ... Papers (Lecture Notes in Computer Science) , 2006 .

[15]  N. Fuhr.  PAN-Uncovering Plagiarism , Authorship , and Social Software Misuse ImageCLEF 2013-Cross Language Image Annotation and Retrieval INEX-INitiative for the Evaluation of XML retrieval , 2002 .