Seven Years of INEX Interactive Retrieval Experiments - Lessons and Challenges

This paper summarizes a major effort in interactive search investigation, the INEX i-track, a collective effort run over a seven-year period. We present the experimental conditions, report some of the findings of the participating groups, and examine the challenges posed by this kind of collective experimental effort.

[1]  Gabriella Kazai,et al.  Overview of the Initiative for the Evaluation of XML retrieval (INEX) 2002 , 2002, INEX Workshop.

[2]  Peter Ingwersen,et al.  The Application of Work Tasks in Connection with the Evaluation of Interactive Information Retrieval Systems: Empirical Results , 1999, MIRA.

[3]  Arjen P. de Vries,et al.  Relevant contextual features in XML retrieval , 2006, IIiX.

[4]  Norbert Fuhr,et al.  Overview of the INEX 2009 Interactive Track , 2009, INEX.

[5]  Nils Pharo.  The effect of granularity and order in XML element retrieval , 2008, Inf. Process. Manag..

[6]  Andrew Trotman,et al.  Focused Retrieval and Evaluation, 8th International Workshop of the Initiative for the Evaluation of XML Retrieval, INEX 2009, Brisbane, Australia, December 7-9, 2009, Revised and Selected Papers , 2010, INEX.

[7]  Joemon M. Jose,et al.  How users assess Web pages for information seeking , 2005, J. Assoc. Inf. Sci. Technol..

[8]  Ludovic Denoyer,et al.  The XML Wikipedia Corpus , 2006 .

[9]  Mounia Lalmas,et al.  Advances in XML Information Retrieval, Third International Workshop of the Initiative for the Evaluation of XML Retrieval, INEX 2004, Dagstuhl Castle, Germany, December 6-8, 2004, Revised Selected Papers , 2005, INEX.

[10]  Birger Larsen,et al.  Users, structured documents and overlap: interactive searching of elements and the influence of context on search behaviour , 2006, IIiX.

[11]  Andrew Trotman,et al.  Comparative Evaluation of XML Information Retrieval Systems: 5th International Workshop of the Initiative for the Evaluation of XML Retrieval, INEX 2006 Dagstuhl Castle, Germany, December 17-20, 2006 Revised and Selected Papers , 2005 .

[12]  Pia Borlund,et al.  Experimental components for the evaluation of interactive information retrieval systems , 2000, J. Documentation.

[13]  Nils Pharo,et al.  Context Matters: An Analysis of Assessments of XML Documents , 2005, CoLIS.

[14]  Norbert Fuhr,et al.  Overview of the INEX 2010 Interactive Track , 2010, INEX.

[15]  Jaana Kekäläinen,et al.  IR evaluation methods for retrieving highly relevant documents , 2000, SIGIR '00.

[16]  Norbert Fuhr,et al.  Daffodil: Distributed Agents for User-Friendly Access of Digital Libraries , 2000, ECDL.

[17]  Birger Larsen,et al.  The Interactive Track at INEX 2004 , 2004, INEX.

[18]  Andrew Trotman,et al.  Comparative Evaluation of Focused Retrieval , 2010, Lecture Notes in Computer Science.

[19]  Gabriella Kazai,et al.  A report on the first year of the INitiative for the Evaluation of XML retrieval , 2003, J. Assoc. Inf. Sci. Technol..

[20]  Andrew Trotman,et al.  Advances in Focused Retrieval, 7th International Workshop of the Initiative for the Evaluation of XML Retrieval, INEX 2008, Dagstuhl Castle, Germany, December 15-18, 2008. Revised and Selected Papers , 2009, INEX.

[21]  Norbert Fuhr,et al.  HyREX: hyper-media retrieval engine for XML , 2002, SIGIR '02.

[22]  Gabriella Kazai,et al.  Advances in XML Information Retrieval and Evaluation, 4th International Workshop of the Initiative for the Evaluation of XML Retrieval, INEX 2005, Dagstuhl Castle, Germany, November 28-30, 2005, Revised Selected Papers , 2006, INEX.

[23]  de Arjen Vries,et al.  Relevant contextual features in XML retrieval, in IIiX , 2006 .

[24]  Stuart Macdonald,et al.  User Engagement in Research Data Curation , 2009, ECDL.

[25]  J. Pehcevski.  Relevance in XML Retrieval : The User Perspective , 2006 .

[26]  Khairun Nisa Fachry,et al.  Overview of the INEX 2008 Interactive Track , 2009, INEX.

[27]  Andrew Trotman,et al.  Focused Access to XML Documents, 6th International Workshop of the Initiative for the Evaluation of XML Retrieval, INEX 2007, Dagstuhl Castle, Germany, December 17-19, 2007. Selected Papers , 2008, INEX.

[28]  Birger Larsen,et al.  Is XML retrieval meaningful to users?: searcher preferences for full documents vs. elements , 2006, SIGIR.

[29]  Birger Larsen,et al.  The Interactive Track at INEX 2005 , 2005, INEX.

[30]  Elaine Toms,et al.  Task Effects on Interactive Search: The Query Factor , 2008, INEX.

[31]  Ludovic Denoyer,et al.  The Wikipedia XML corpus , 2006, SIGF.

[32]  Nils Pharo,et al.  The effect of task type on preferred element types in an XML-based retrieval system , 2011, J. Assoc. Inf. Sci. Technol..

[33]  Heesop Kim,et al.  Users Interaction with the Hierarchically Structured Presentation in XML Document Retrieval , 2005, INEX.

[34]  Birger Larsen,et al.  The Interactive Track at INEX 2006 , 2006, INEX.