Personalizing information retrieval for multi‐session tasks: Examining the roles of task stage, task type, and topic knowledge on the interpretation of dwell time as an indicator of document usefulness

Personalization of information retrieval tailors search towards individual users to meet their particular information needs by taking into account information about users and their contexts, often through implicit sources of evidence such as user behaviors. This study looks at users' dwelling behavior on documents and several contextual factors: the stage of users' work tasks, task type, and users' knowledge of task topics, to explore whether or not taking account contextual factors could help infer document usefulness from dwell time. A controlled laboratory experiment was conducted with 24 participants, each coming 3 times to work on 3 subtasks in a general work task. The results show that task stage could help interpret certain types of dwell time as reliable indicators of document usefulness in certain task types, as was topic knowledge, and the latter played a more significant role when both were available. This study contributes to a better understanding of how dwell time can be used as implicit evidence of document usefulness, as well as how contextual factors can help interpret dwell time as an indicator of usefulness. These findings have both theoretical and practical implications for using behaviors and contextual factors in the development of personalization systems.

[1]  Robert S. Taylor Question-Negotiation and Information Seeking in Libraries , 1968, Coll. Res. Libr..

[2]  Peter Ingwersen,et al.  Information retrieval in context: IRiX , 2005, SIGF.

[3]  Barbara M. Wildemuth,et al.  The effects of domain knowledge on search tactic formulation , 2004, J. Assoc. Inf. Sci. Technol..

[4]  Rong Tang,et al.  Towards the Identification of the Optimal Number of Relevance Categories , 1999, J. Am. Soc. Inf. Sci..

[5]  Carol Collier Kuhlthau,et al.  Inside the search process: Information seeking from the user's perspective , 1991, J. Am. Soc. Inf. Sci..

[6]  Bryce Allen,et al.  Information needs: a person-in-situation approach , 1997 .

[7]  Omid Madani,et al.  Biasing web search results for topic familiarity , 2005, CIKM '05.

[8]  Brenda Dervin,et al.  Given a context by any other name: methodological tools for taming the unruly beast , 1997 .

[9]  Luanne Freund,et al.  Exploiting task-document relations in support of information retrieval in the workplace , 2008, SIGF.

[10]  Katriina Byström,et al.  Information and information sources in tasks of varying complexity , 2002, J. Assoc. Inf. Sci. Technol..

[11]  Reijo Savolainen Information source horizons and source preferences of environmental activists: A social phenomenological approach , 2007, J. Assoc. Inf. Sci. Technol..

[12]  Joemon M. Jose,et al.  How users assess Web pages for information seeking , 2005, J. Assoc. Inf. Sci. Technol..

[13]  Diane Kelly,et al.  The effects of topic familiarity on information search behavior , 2002, JCDL '02.

[14]  Jeonghyun Kim,et al.  Task as a predictable indicator for information seeking behavior on the Web , 2007 .

[15]  Sanna Talja,et al.  The production of context in information seeking research: a metatheoretical view , 1999, Inf. Process. Manag..

[16]  Liwen Qiu,et al.  Analytical Searching vs. Browsing in Hypertext Information Retrieval Systems. , 1993 .

[17]  Nicholas J. Belkin,et al.  Interaction with Texts: Information Retrieval as Information-Seeking Behavior , 1993, Information Retrieval.

[18]  Kalervo Järvelin,et al.  Task Complexity Affects Information Seeking and Use , 1995, Inf. Process. Manag..

[19]  Nicholas J. Belkin,et al.  Some(what) grand challenges for information retrieval , 2008, SIGF.

[20]  G. Kelly A theory of personality : the psychology of personal constructs , 1963 .

[21]  Nicholas J. Belkin,et al.  A Model for Evaluation of Interactive Information Retrieval , 2009 .

[22]  Pertti Vakkari,et al.  Task complexity, problem structure and information actions - Integrating studies on information seeking and retrieval , 1999, Inf. Process. Manag..

[23]  Nicholas J. Belkin,et al.  Reading time, scrolling and interaction: exploring implicit sources of user preferences for relevance feedback , 2001, Annual International ACM SIGIR Conference on Research and Development in Information Retrieval.

[24]  Nicholas J. Belkin,et al.  Relationships between categories of relevance criteria and stage in task completion , 2007, Inf. Process. Manag..

[25]  Henrik Madsen,et al.  Introduction to General and Generalized Linear Models , 2010 .

[26]  Colleen Cool The Concept of Situation in Information Science. , 2001 .

[27]  Ryen W. White,et al.  Characterizing the influence of domain expertise on web search behavior , 2009, WSDM '09.

[28]  Diane Kelly Measuring online information seeking context, Part 1: Background and method , 2006 .

[29]  N. Roberts,et al.  Value-added processes in information systems , 1986 .

[30]  Jacek Gwizdka,et al.  Analysis and evaluation of query reformulations in different task types , 2010, ASIST.

[31]  Pertti Vakkari,et al.  Changes in relevance criteria and problem stages in task performance , 2000, J. Documentation.

[32]  Wei Chu,et al.  Learning to extract cross-session search tasks , 2013, WWW.

[33]  Nicholas J. Belkin,et al.  Rutgers' HARD Track Experiences at TREC 2004 , 2004, TREC.

[34]  Nicholas J. Belkin,et al.  A faceted approach to conceptualizing tasks in information seeking , 2008, Inf. Process. Manag..

[35]  Reijo Savolainen Everyday life information seeking: Approaching information seeking in the context of “way of life” , 1995 .

[36]  Robert S. Taylor,et al.  Value-Added Processes in Information Systems , 1987 .

[37]  Diane Kelly,et al.  Measuring online information seeking context, Part 1: Background and method , 2006, J. Assoc. Inf. Sci. Technol..

[38]  Peter Ingwersen,et al.  The Turn - Integration of Information Seeking and Retrieval in Context , 2005, The Kluwer International Series on Information Retrieval.

[39]  Nicholas J. Belkin,et al.  Examining the effects of task topic familiarity on searchers' behaviors in different task types , 2013, ASIST.

[40]  Diane Kelly,et al.  Measuring online information seeking context, Part 2: Findings and discussion , 2006, J. Assoc. Inf. Sci. Technol..

[41]  Pertti Vakkari,et al.  A theory of the task-based information retrieval process: a summary and generalisation of a longitudinal study , 2001, J. Documentation.

[42]  Ryen W. White,et al.  A study on the effects of personalization and task information on implicit feedback performance , 2006, CIKM '06.

[43]  Ian Ruthven,et al.  Information Retrieval in Context , 2011, Advanced Topics in Information Retrieval.

[44]  Denise E. Agosto,et al.  People, places, and questions : An investigation of the everyday life information-seeking behaviors of urban young adults , 2005 .

[45]  Gary Marchionini,et al.  Information-seeking strategies of novices using a full-text electronic encyclopedia , 1989, JASIS.

[46]  Nicholas J. Belkin,et al.  A user modeling system for personalized interaction and tailored retrieval in interactive IR , 2005, ASIST.

[47]  Nicholas J. Belkin,et al.  Display time as implicit feedback: understanding task effects , 2004, SIGIR '04.

[48]  Jacek Gwizdka,et al.  Search behaviors in different task types , 2010, JCDL '10.

[49]  Ryen W. White,et al.  A study of factors affecting the utility of implicit relevance feedback , 2005, SIGIR '05.

[50]  Michael A. Shepherd,et al.  A Field Study Characterizing Web-based Information Seeking Tasks , 2022 .

[51]  Nicholas J. Belkin,et al.  An exploration of the relationships between work task and interactive information search behavior , 2010, J. Assoc. Inf. Sci. Technol..

[52]  Yoichi Shinoda,et al.  Information filtering based on user behavior analysis and best match text retrieval , 1994, SIGIR '94.

[53]  Michael J. Cole,et al.  Task Topic Knowledge vs. Background Domain Knowledge: Impact of Two Types of Knowledge on User Search Performance , 2013, WorldCIST.

[54]  Bryce Allen,et al.  Topic Knowledge and Online Catalog Search Formulation , 1991, The Library Quarterly.

[55]  JärvelinKalervo,et al.  Task complexity affects information seeking and use , 1995 .

[56]  Elizabeth D. Liddy,et al.  The effects of expertise and feedback on search term selection and subsequent learning , 2005, J. Assoc. Inf. Sci. Technol..

[57]  Shin-jeng Lin Modeling and supporting multiple information seeking episodes over the Web , 2002 .

[58]  Pia Borlund,et al.  Experimental components for the evaluation of interactive information retrieval systems , 2000, J. Documentation.