A framework for the theoretical evaluation of XML retrieval

We present a theoretical framework to evaluate XML retrieval. XML retrieval deals with retrieving those document components—the XML elements—that specifically answer a query. In this article, theoretical evaluation is concerned with the formal representation of qualitative properties of retrieval models. It complements experimental methods by showing the properties of the underlying reasoning assumptions that decide when a document is about a query. We define a theoretical methodology based on the idea of “aboutness” and apply it to current XML retrieval models. This allows comparing and analyzing the reasoning behavior of XML retrieval models experimented within the INEX evaluation campaigns. For each model we derive functional and qualitative properties that qualify its formal behavior. We then use these properties to explain experimental results obtained with some of the XML retrieval models. © 2012 Wiley Periodicals, Inc.

[1]  Yves Chiaramella,et al.  Information Retrieval and Structured Documents , 2000, ESSIR.

[2]  C. J. van Rijsbergen,et al.  Towards an information logic , 1989, SIGIR '89.

[3]  M. Weber.  Methodology of Social Sciences , 1949 .

[4]  Mounia Lalmas,et al.  Specificity aboutness in XML retrieval , 2010, Information Retrieval.

[5]  Brian C. O'Connor,et al.  Nodes of topicality: Modeling user notions of on topic documents , 2003, J. Assoc. Inf. Sci. Technol..

[6]  W. J. Hutchins,et al.  ON THE PROBLEM OF 'ABOUTNESS' IN DOCUMENT ANALYSIS , 1977 .

[7]  Jaap Kamps,et al.  The Effect of Structured Queries and Selective Indexing on XML Retrieval , 2005, INEX.

[8]  Gabriella Kazai,et al.  Advances in XML Information Retrieval and Evaluation: 4th International Workshop of the Initiative for the Evaluation of XML Retrieval, INEX 2005, Dagstuhl ... Papers (Lecture Notes in Computer Science) , 2006 .

[9]  Marc Najork,et al.  Computing Information Retrieval Performance Measures Efficiently in the Presence of Tied Scores , 2008, ECIR.

[10]  Kam-Fai Wong,et al.  Application of aboutness to functional benchmarking in information retrieval , 2001, TOIS.

[11]  C. J. van Rijsbergen,et al.  The geometry of information retrieval , 2004 .

[12]  John Tait,et al.  Current Challenges in Patent Information Retrieval , 2011, The Information Retrieval Series.

[13]  Gabriella Kazai,et al.  INEX 2005 Evaluation Measures , 2005, INEX.

[14]  Tao Tao,et al.  A formal study of information retrieval heuristics , 2004, SIGIR '04.

[15]  Michael Jackman,et al.  Conceptual graphs , 1988 .

[16]  Tobias Blanke,et al.  Using Situation Theory to evaluate XML Retrieval , 2012, Dissertationen zu Datenbanken und Informationssystemen.

[17]  Peter Bruza,et al.  Investigating aboutness axioms using information fields , 1994, SIGIR '94.

[18]  ChengXiang Zhai,et al.  An exploration of axiomatic approaches to information retrieval , 2005, SIGIR '05.

[19]  Lillian Lee,et al.  Opinion Mining and Sentiment Analysis , 2008, Found. Trends Inf. Retr..

[20]  E. Shils,et al.  The Methodology Of The Social Sciences , 1949 .

[21]  Shlomo Geva,et al.  GPX - Gardens Point XML IR at INEX 2006 , 2006, INEX.

[22]  C. J. V. Rijsbergen,et al.  Information calculus for information retrieval , 1996 .

[23]  Kam-Fai Wong,et al.  Aboutness from a commonsense perspective , 2000, J. Am. Soc. Inf. Sci..

[24]  Kam-Fai Wong,et al.  Aboutness from a commonsense perspective , 2000 .

[25]  William S. Cooper,et al.  A definition of relevance for information retrieval , 1971, Inf. Storage Retr..

[26]  N. Cocchiarella,et al.  Situations and Attitudes. , 1986 .

[27]  Meredith Ringel Morris,et al.  Collaborative Web Search: Who, What, Where, When, and Why , 2009, Collaborative Web Search: Who, What, Where, When, and Why.

[28]  Theo Huibers,et al.  An axiomatic theory for information retrieval , 1996 .

[29]  Peter Mika,et al.  Ad-hoc object retrieval in the web of data , 2010, WWW '10.

[30]  Tim Berners-Lee,et al.  Linked data , 2020, Semantic Web for the Working Ontologist.

[31]  Ben Carterette,et al.  Overview of Information Retrieval Evaluation , 2011, Current Challenges in Patent Information Retrieval.

[32]  Sarit Kraus,et al.  Nonmonotonic Reasoning, Preferential Models and Cumulative Logics , 1990, Artif. Intell..

[33]  Iadh Ounis,et al.  Conceptual Graph Aboutness , 1996, ICCS.

[34]  Birger Hjørland,et al.  Towards a theory of aboutness, subject, topicality, theme, domain, field, content ... and relevance , 2001, J. Assoc. Inf. Sci. Technol..

[35]  Wei Zhang,et al.  Opinion retrieval from blogs , 2007, CIKM '07.

[36]  John F. Sowa,et al.  Handbook of Knowledge Representation Edited Conceptual Graphs 5.1 from Existential Graphs to Conceptual Graphs , 2022 .

[37]  Mounia Lalmas,et al.  Theoretical benchmarks of XML retrieval , 2006, SIGIR.

[38]  Gabriella Kazai,et al.  Evaluating the effectiveness of content-oriented XML retrieval , 2003 .

[39]  Gabriella Kazai,et al.  Advances in XML Information Retrieval and Evaluation, 4th International Workshop of the Initiative for the Evaluation of XML Retrieval, INEX 2005, Dagstuhl Castle, Germany, November 28-30, 2005, Revised Selected Papers , 2006, INEX.

[40]  Tom Heath,et al.  Linked Data: Evolving the Web into a Global Data Space , 2011, Linked Data.