Two ' s company, but three ' s no crowd: Evaluating exploratory web search for individuals and teams

Purpose – The purpose of this paper is to investigate when and how people working in collaboration could be benefitted by an exploratory search task, specifically focussing on team size and its effect on the outcomes of such a task. Design/methodology/approach – The paper investigates the effects of team sizes on exploratory search tasks using a lab study involving 68 participants – 12 individuals, ten dyads, and 12 triads. In order to assess various factors during their exploratory search sessions, an evaluation framework is synthesized using relevant literature. The framework consists of measures for five groups of quantities relevant to exploratory search: information exposure, information relevancy, information search, performance, and learning. Findings – The analyses on the user study data using the proposed framework reveals that while individuals working alone cover more information than those working in teams, the teams (dyads and triads) are able to achieve better information coverage and search...

[1]  Nicholas J. Belkin,et al.  Braque: Design of an Interface to Support User Interaction in Information Retrieval , 1993, Inf. Process. Manag..

[2]  Ryen W. White,et al.  Supporting exploratory search , 2006 .

[3]  Raya Fidel,et al.  A multidimensional approach to the study of human-information interaction: A case study of collaborative information retrieval , 2004, J. Assoc. Inf. Sci. Technol..

[4]  Bryce Allen,et al.  Information needs: a person-in-situation approach , 1997 .

[5]  Gary Marchionini,et al.  Exploratory search , 2006, Commun. ACM.

[6]  Marcia J. Bates,et al.  Information search tactics , 1979, J. Am. Soc. Inf. Sci..

[7]  Jette Hyldegård,et al.  Collaborative information behaviour--exploring Kuhlthau's Information Search Process model in a group-based educational setting , 2006, Inf. Process. Manag..

[8]  G. Marchionini,et al.  Correction to Shah, C. and Marchionini, G. (2010). Awareness in Collaborative Information Seeking. Journal of the American Society for Information Science and Technology, 61 (10), 1970-1986 , 2010 .

[9]  Amanda Spink,et al.  A user-centered approach to evaluating human interaction with Web search engines: an exploratory study , 2002, Inf. Process. Manag..

[10]  Gary Marchionini,et al.  Report on ACM SIGIR 2006 workshop on evaluating exploratory search systems , 2006, SIGF.

[11]  Roberto I. González-Ibáñez,et al.  Evaluating the synergic effect of collaboration in information seeking , 2011, SIGIR.

[12]  Roberto I. González-Ibáñez,et al.  Spatial context in collaborative information seeking , 2012, J. Inf. Sci..

[13]  Roberto I. González-Ibáñez,et al.  Time and space in collaborative information seeking: The clash of effectiveness and uniqueness , 2012, ASIST.

[14]  Barry Smyth,et al.  A Live-User Evaluation of Collaborative Web Search , 2005, IJCAI.

[15]  Meredith Ringel Morris.  Collaborating Alone and Together: Investigating Persistent and Multi-User Web Search Activities , 2007 .

[16]  Louise T. Su.  Evaluation Measures for Interactive Information Retrieval , 1992, Inf. Process. Manag..

[17]  Chirag Shah,et al.  User-driven system-mediated collaborative information retrieval , 2014, SIGIR.

[18]  Vladimir Estivill-Castro,et al.  Usability of real-time unconstrained WWW-co-browsing for educational settings , 2005, The 2005 IEEE/WIC/ACM International Conference on Web Intelligence (WI'05).

[19]  T. D. Wilson,et al.  On user studies and information needs , 2006, J. Documentation.

[20]  Bernard J. Jansen,et al.  A model for understanding collaborative information behavior in context: A study of two healthcare teams , 2008, Inf. Process. Manag..

[21]  Chirag Shah,et al.  Role-based results redistribution for collaborative information retrieval , 2010, Inf. Process. Manag..

[22]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[23]  Steve Fox,et al.  Evaluating implicit measures to improve web search , 2005, TOIS.

[24]  G. Marchionini,et al.  Awareness in collaborative information seeking , 2010, J. Assoc. Inf. Sci. Technol..

[25]  Ryen W. White,et al.  Assessing the scenic route: measuring the value of search trails in web logs , 2010, SIGIR.

[26]  Preben Hansen,et al.  Collaborative Information Retrieval in an information-intensive domain , 2005, Inf. Process. Manag..

[27]  Chirag Shah,et al.  Coagmento: A system for supporting collaborative information seeking , 2011, ASIST.

[28]  David Bawden,et al.  Information systems and the stimulation of creativity , 1986, J. Inf. Sci..

[29]  Ryen W. White,et al.  Pseudo-collaboration as a method to perform selective algorithmic mediation in collaborative IR systems , 2012, ASIST.

[30]  Roberto I. González-Ibáñez,et al.  Let's search together, but not too close! An analysis of communication and performance in collaborative information seeking , 2013, Inf. Process. Manag..

[31]  Gary Marchionini,et al.  Information Seeking in Electronic Environments , 1995 .

[32]  Peter J. Denning,et al.  Communications of the acm , 1989 .

[33]  Carol Collier Kuhlthau.  Inside the Search Process: Information Seeking from the User's Perspective. , 1991 .

[34]  Ryen W. White,et al.  Exploratory Search: Beyond the Query-Response Paradigm , 2009, Exploratory Search: Beyond the Query-Response Paradigm.

[35]  Tefko Saracevic,et al.  Evaluation of evaluation in information retrieval , 1995, SIGIR '95.

[36]  José A. Pino,et al.  A first step to formally evaluate collaborative work , 1997, GROUP.

[37]  T. Saracevic,et al.  Relevance: A review of the literature and a framework for thinking on the notion in information science. Part II: nature and manifestations of relevance , 2007, J. Assoc. Inf. Sci. Technol..

[38]  John M. Budd,et al.  Relevance: Language, Semantics, Philosophy , 2004, Libr. Trends.

[39]  D. Sonnenwald.  Perspectives of human information behaviour: contexts, situations, social networks and information horizons , 1999 .

[40]  Chirag Shah,et al.  Collaborative information seeking , 2014, The Information Retrieval Series.

[41]  Peter J. Denning,et al.  Getting to "we" , 2008, CACM.

[42]  Ryen W. White,et al.  Supporting Exploratory Search, Introduction, Special Issue, Communications of the ACM , 2006 .

[43]  Sanna Talja,et al.  Information sharing in academic communities: Types and levels of collaboration in information seeking and use. , 2002 .

[44]  Chirag Shah,et al.  Algorithmic mediation for collaborative exploratory search , 2008, SIGIR '08.

[45]  Diane H. Sonnenwald,et al.  Information behavior in dynamic group work contexts: interwoven situational awareness, dense social networks and contested collaboration in command and control , 2000, Inf. Process. Manag..

[46]  Andrei Broder,et al.  A taxonomy of web search , 2002, SIGF.

[47]  Peter Ingwersen,et al.  The Application of Work Tasks in Connection with the Evaluation of Interactive Information Retrieval Systems: Empirical Results , 1999, MIRA.

[48]  Jonathan Foster,et al.  Collaborative information seeking and retrieval , 2006, Annu. Rev. Inf. Sci. Technol..

[49]  Robert G. Capra,et al.  Designing exploratory search tasks for user studies of information seeking support systems , 2009, JCDL '09.

[50]  Chirag Shah,et al.  Evaluating collaborative information seeking – synthesis, suggestions, and structure , 2014, J. Inf. Sci..

[51]  Luanne Freund,et al.  Assigning search tasks designed to elicit exploratory search behaviors , 2012, HCIR '12.

[52]  Nigel Ford,et al.  Serendipity and information seeking: an empirical study , 2003, J. Documentation.

[53]  Daniel E. Rose,et al.  Understanding user goals in web search , 2004, WWW '04.

[54]  Andreas Paepcke,et al.  TeamSearch: comparing techniques for co-present collaborative search of digital media , 2006, First IEEE International Workshop on Horizontal Interactive Human-Computer Systems (TABLETOP '06).

[55]  Meredith Ringel Morris,et al.  A survey of collaborative web search practices , 2008, CHI.

[56]  Monica M. C. Schraefel,et al.  Evaluating Collaborative Search Interfaces with Information Seeking Theory , 2009, ArXiv.

[57]  Danushka Bollegala,et al.  Collaborative exploratory search in real-world context , 2011, CIKM '11.

[58]  Amanda Spink,et al.  Determining the informational, navigational, and transactional intent of Web queries , 2008, Inf. Process. Manag..