Towards a probabilistic model for supporting collaborative information access

In information retrieval research, models and systems traditionally assume that a single person is querying and reviewing the results. However, several empirical studies of professional practice identified collaboration during IR as everyday work patterns in order to solve a shared information need and to benefit from the diverse expertise and experience of the team members. Moreover, most IR systems that are employed in professional work routines are designed for individual use and prototype collaborative systems are too limited to support use in todays work practice. To bridge this gap, this papers develops and formalizes a decision theoretic approach towards supporting a team of people that explicitly set out together to resolve a shared information need. We develop a formal cost model for collaborative IR that considers the trade-off between estimated relevance of a document as well as estimated document redundancy. From this cost model, we use a decision theoretic approach to derive the notion of activity suggestions, that is, a formal optimum criterion that describes optimum collaboration strategies in IR as the solution of an integer linear program. Those collaboration strategies are suggested to team members with the aim to facilitate the collaborative performance of information retrieval tasks. We demonstrate the application of our model by means of search result division in two collaborative search tasks. In the conducted experiments, we study the effects of different domain knowledge and resulting relevance assessments of team members in four different conditions. The gathered results indicate that our approach can improve the retrieval effectiveness of teams in recall-oriented tasks.

[1]  Chirag Shah,et al.  Algorithmic mediation for collaborative exploratory search , 2008, SIGIR '08.

[2]  Stephen E. Robertson,et al.  Simple BM25 extension to multiple weighted fields , 2004, CIKM '04.

[3]  Jun Wang,et al.  Portfolio theory of information retrieval , 2009, SIGIR.

[4]  Jonathon N. Cummings Work Groups, Structural Diversity, and Knowledge Sharing in a Global Organization , 2004, Manag. Sci..

[5]  Susan T. Dumais,et al.  Collaborative information retrieval (CIR) , 2000, CHI Extended Abstracts.

[6]  Raya Fidel,et al.  Collaborative Information Retrieval. , 1999 .

[7]  S. Robertson The probability ranking principle in IR , 1997 .

[8]  John Tait,et al.  CLEF-IP 2009: Retrieval Experiments in the Intellectual Property Domain , 2009, CLEF.

[9]  Ryen W. White,et al.  Characterizing the influence of domain expertise on web search behavior , 2009, WSDM '09.

[10]  Maribeth Back,et al.  A Taxonomy of Collaboration in Online Information Seeking , 2009, ArXiv.

[11]  W. Bruce Croft,et al.  Automatic query generation for patent search , 2009, CIKM.

[12]  Norbert Fuhr,et al.  Probabilistic Models in Information Retrieval , 1992, Comput. J..

[13]  Meredith Ringel Morris,et al.  Collaborative search revisited , 2013, CSCW.

[14]  Ann Blandford,et al.  Social and interactional practices for disseminating current awareness information in an organisational setting , 2010, Inf. Process. Manag..

[15]  Chirag Shah,et al.  Role-based results redistribution for collaborative information retrieval , 2010, Inf. Process. Manag..

[16]  Joemon M. Jose,et al.  Revisiting IR Techniques for Collaborative Search Strategies , 2009, ECIR.

[17]  Martin Halvey,et al.  Is relevance hard work?: evaluating the effort of making relevant assessments , 2013, SIGIR.

[18]  A. M. Pejtersen,et al.  Information seeking and sharing in design teams , 2003, GROUP.

[19]  José A. Pino,et al.  A first step to formally evaluate collaborative work , 1997, GROUP.

[20]  Peter Bailey,et al.  Relevance assessment: are judges exchangeable and does it matter , 2008, SIGIR '08.

[21]  Chris Buckley,et al.  OHSUMED: an interactive retrieval evaluation and new large test collection for research , 1994, SIGIR '94.

[22]  Matthias Hemmje,et al.  Catching the User - Logging the Information Retrieval Dialogue , 2009, UIIR@SIGIR.

[23]  Tefko Saracevic Relevance: A review of the literature and a framework for thinking on the notion in information science. Part III: Behavior and effects of relevance , 2007 .

[24]  Eric Horvitz,et al.  SearchTogether: an interface for collaborative web search , 2007, UIST.

[25]  Amanda Spink,et al.  Real life, real users, and real needs: a study and analysis of user queries on the web , 2000, Inf. Process. Manag..

[26]  Alan F. Smeaton,et al.  Division of labour and sharing of knowledge for synchronous collaborative information retrieval , 2010, Inf. Process. Manag..

[27]  Leif Azzopardi Query side evaluation: an empirical analysis of effectiveness and effort , 2009, SIGIR.

[28]  Preben Hansen,et al.  Collaborative Information Retrieval in an information-intensive domain , 2005, Inf. Process. Manag..

[29]  C. J. van Rijsbergen,et al.  Has portfolio theory got any principles? , 2010, SIGIR.

[30]  Greg Linden,et al.  Amazon . com Recommendations Item-to-Item Collaborative Filtering , 2001 .

[31]  Avi Arampatzis,et al.  A study of query length , 2008, SIGIR '08.

[32]  Wahiba Bahsoun,et al.  A Collaborative Document Ranking Model for a Multi-faceted Search , 2013, AIRS.

[33]  Wim Vanderbauwhede,et al.  A survey of patent users: an analysis of tasks, behavior, search functionality and system requirements , 2010, IIiX.

[34]  Maximilian Eibl,et al.  Does Patent IR Profit from Linguistics or Maximum Query Length? , 2011, CLEF.

[35]  G. Marchionini,et al.  Awareness in collaborative information seeking , 2010, J. Assoc. Inf. Sci. Technol..

[36]  Tefko Saracevic,et al.  Relevance : A Review of the Literature and a Framework for Thinking on the Notion in Information Science . Part III : Behavior and Effects of Relevance , 1976 .

[37]  Chirag Shah,et al.  User-driven system-mediated collaborative information retrieval , 2014, SIGIR.

[38]  Wei Chu,et al.  Modeling the impact of short- and long-term behavior on search personalization , 2012, SIGIR '12.

[39]  W. Bruce Croft,et al.  Automatic boolean query suggestion for professional search , 2011, SIGIR.

[40]  Norbert Fuhr,et al.  A decision-theoretic approach to database selection in networked IR , 1999, TOIS.

[41]  Norbert Fuhr,et al.  A probability ranking principle for interactive information retrieval , 2008, Information Retrieval.

[42]  D. Warner North,et al.  A Tutorial Introduction to Decision Theory , 1968, IEEE Trans. Syst. Sci. Cybern..

[43]  Guido Zuccon,et al.  Using the Quantum Probability Ranking Principle to Rank Interdependent Documents , 2010, ECIR.

[44]  Shuguang Han,et al.  Modeling search processes using hidden states in collaborative exploratory web search , 2014, CSCW.

[45]  David M. Nichols,et al.  Browsing is a collaborative process , 1997, Inf. Process. Manag..

[46]  Madhu C. Reddy,et al.  Collaborative information seeking: A field study of a multidisciplinary patient care team , 2008, Inf. Process. Manag..

[47]  Christina Lioma,et al.  Preliminary study into query translation for patent retrieval , 2010, PaIR '10.

[48]  Matthias Hemmje,et al.  Supporting Collaborative Information Seeking and Searching in Distributed Environments , 2013, LWA.

[49]  M. de Rijke,et al.  Building simulated queries for known-item topics: an analysis using six european languages , 2007, SIGIR.

[50]  John Tait An Introduction to Professional Search , 2014, Professional Search in the Modern World.