Preliminary experiments using subjective logic for the polyrepresentation of information needs

According to the principle of polyrepresentation, retrieval accuracy may improve through the combination of multiple and diverse information object representations about e.g. the context of the user, the information sought, or the retrieval system [9, 10]. Recently, the principle of polyrepresentation was mathematically expressed using subjective logic [12], where the potential suitability of each representation for improving retrieval performance was formalised through degrees of belief and uncertainty [15]. No experimental evidence or practical application has so far validated this model. We extend the work of Lioma et al. (2010) [15], by providing a practical application and analysis of the model. We show how to map the abstract notions of belief and uncertainty to real-life evidence drawn from a retrieval dataset. We also show how to estimate two different types of polyrepresentation assuming either (a) independence or (b) dependence between the information objects that are combined. We focus on the polyrepresentation of different types of context relating to user information needs (i.e. work task, user background knowledge, ideal answer) and show that the subjective logic model can predict their optimal combination prior and independently to the retrieval process.

[1]  Toine Bogers,et al.  An Exploration of Retrieval-Enhancing Methods for Integrated Search in a Digital Library , 2012 .

[2]  Glenn Shafer,et al.  A Mathematical Theory of Evidence , 2020, A Mathematical Theory of Evidence.

[3]  Miles Efron,et al.  Query polyrepresentation for ranking retrieval systems without relevance judgments , 2010 .

[4]  Ann Blandford,et al.  A polyrepresentational approach to interactive query expansion , 2009, JCDL '09.

[5]  Philippe Smets,et al.  What is Dempster-Shafer's model? , 1994 .

[6]  Paul B. Kantor,et al.  A study of information seeking and retrieving. III. Searchers, searches, and overlap , 1988, J. Am. Soc. Inf. Sci..

[7]  Paul B. Kantor,et al.  A Study of Information Seeking and Retrieving. III. Searchers, Searches, and Overlap* , 1988 .

[8]  Paul B. Kantor,et al.  A study of information seeking and retrieving. I. background and methodology , 1988 .

[9]  Nicholas J. Belkin,et al.  The effect multiple query representations on information retrieval system performance , 1993, SIGIR.

[10]  Marie-Francine Moens,et al.  A Belief Model of Query Difficulty That Uses Subjective Logic , 2009, ICTIR.

[11]  Ellen M. Voorhees,et al.  Retrieval evaluation with incomplete information , 2004, SIGIR '04.

[12]  Jaana Kekäläinen,et al.  Cumulated gain-based evaluation of IR techniques , 2002, TOIS.

[13]  Audun Jøsang,et al.  A Logic for Uncertain Probabilities , 2001, Int. J. Uncertain. Fuzziness Knowl. Based Syst..

[14]  Miles Efron,et al.  Query polyrepresentation for ranking retrieval systems without relevance judgments , 2010, J. Assoc. Inf. Sci. Technol..

[15]  Ellen M. Voorhees,et al.  The TREC-8 Question Answering Track Evaluation , 2000, TREC.

[16]  Arthur P. Dempster,et al.  A Generalization of Bayesian Inference , 1968, Classic Works of the Dempster-Shafer Theory of Belief Functions.

[17]  John D. Lafferty,et al.  Two-stage language models for information retrieval , 2002, SIGIR '02.

[18]  C. J. van Rijsbergen,et al.  Supporting polyrepresentation in a quantum-inspired geometrical retrieval framework , 2010, IIiX.

[19]  Christina Lioma,et al.  A subjective logic formalisation of the principle of polyrepresentation for information needs , 2010, IIiX.

[20]  W. Bruce Croft,et al.  Evaluation of an inference network-based retrieval model , 1991, TOIS.

[21]  Yiming Yang,et al.  RCV1: A New Benchmark Collection for Text Categorization Research , 2004, J. Mach. Learn. Res..

[22]  Peter Ingwersen,et al.  The Turn - Integration of Information Seeking and Retrieval in Context , 2005, The Kluwer International Series on Information Retrieval.

[23]  Peter Ingwersen,et al.  Cognitive Perspectives of Information Retrieval Interaction: Elements of a Cognitive IR Theory , 1996, J. Documentation.

[24]  Peter Ingwersen,et al.  Developing a Test Collection for the Evaluation of Integrated Search , 2010, ECIR.

[25]  Martin F. Porter,et al.  An algorithm for suffix stripping , 1997, Program.