Seeking and implementing automated assistance during the search process

Searchers seldom make use of the advanced searching features that could improve the quality of the search process because they do not know these features exist, do not understand how to use them, or do not believe they are effective or efficient. Information retrieval systems offering automated assistance could greatly improve search effectiveness by suggesting or implementing assistance automatically. A critical issue in designing such systems is determining when the system should intervene in the search process. In this paper, we report the results of an empirical study analyzing when during the search process users seek automated searching assistance from the system and when they implement the assistance. We designed a fully functional, automated assistance application and conducted a study with 30 subjects interacting with the system. The study used a 2G TREC document collection and TREC topics. Approximately 50% of the subjects sought assistance, and over 80% of those implemented that assistance. Results from the evaluation indicate that users are willing to accept automated assistance during the search process, especially after viewing results and locating relevant documents. We discuss implications for interactive information retrieval system design and directions for future research.

[1]  Donna Harman,et al.  Information Processing and Management , 2022 .

[2]  K. A. Ericsson,et al.  Protocol Analysis: Verbal Reports as Data , 1984 .

[3]  Stephen Robertson,et al.  THEORIES AND MODELS IN INFORMATION RETRIEVAL , 1977 .

[4]  Michael P. Oakes,et al.  Automated Assistance in the Formulation of Search Statements for Bibliographic Databases , 1998, Inf. Process. Manag..

[5]  Peter Ingwersen,et al.  Information Retrieval Interaction , 1992 .

[6]  Christiane Fellbaum,et al.  Book Reviews: WordNet: An Electronic Lexical Database , 1999, CL.

[7]  Brian Detlor,et al.  Information Seeking on the Web: An Integrated Model of Browsing and Searching , 2000, First Monday.

[8]  Hsinchun Chen,et al.  Cognitive process as a basis for intelligent retrieval systems design , 1991, Inf. Process. Manag..

[9]  Amanda Spink,et al.  Failure analysis in query construction: data and analysis from a large sample of Web queries , 1998, DL '98.

[10]  Nicholas J. Belkin,et al.  Ask for Information Retrieval: Part I. Background and Theory , 1997, J. Documentation.

[11]  Penelope M. Sanderson,et al.  Exploratory Sequential Data Analysis: Foundations , 1994, Hum. Comput. Interact..

[12]  Gary Marchionini,et al.  Information-seeking strategies of novices using a full-text electronic encyclopedia , 1989, JASIS.

[13]  Tom Peters,et al.  The history and development of transaction log analysis , 1993 .

[14]  Peter Willett,et al.  Readings in information retrieval , 1997 .

[15]  Donna K. Harman,et al.  Relevance feedback revisited , 1992, SIGIR '92.

[16]  Hilary Johnson,et al.  Explanation facilities and interactive systems , 1993, IUI '93.

[17]  Tom Johnston,et al.  MacSHAPA and the enterprise of exploratory sequential data analysis (ESDA) , 1994, Int. J. Hum. Comput. Stud..

[18]  Tefko Saracevic,et al.  Modeling Interaction in Information Retrieval (IR): A Review and Proposal. , 1996 .

[19]  Michael McGill,et al.  Introduction to Modern Information Retrieval , 1983 .

[20]  Xiannong Meng,et al.  FEATURES: Real-time adaptive feature and document learning for web search , 2001, J. Assoc. Inf. Sci. Technol..

[21]  Gary Marchionini,et al.  Information Seeking in Electronic Environments , 1995 .

[22]  C. J. van Rijsbergen,et al.  Finding Out About: A Cognitive Perspective on Search Engine Technology and the WWW , 2001 .

[23]  Nicholas J. Belkin,et al.  On the nature and fuction of explanation in intelligent information retrieval , 1988, SIGIR '88.

[24]  Nicholas J. Belkin,et al.  Reading time, scrolling and interaction: exploring implicit sources of user preferences for relevance feedback , 2001, Annual International ACM SIGIR Conference on Research and Development in Information Retrieval.

[25]  W. Bruce Croft,et al.  Predicting query performance , 2002, SIGIR '02.

[26]  W. S. Cooper.  Expected search length: A single measure of retrieval effectiveness based on the weak ordering action of retrieval systems , 1968 .

[27]  Bernard J. Jansen,et al.  Assisting the searcher: utilizing software agents for Web search systems , 2004, Internet Res..

[28]  Rhonda N. Hunter.  Successes and Failures of Patrons Searching the Online Catalog at a Large Academic Library: A Transaction Log Analysis. , 1991 .

[29]  Ayse Göker.  Capturing information need by learning user context , 1999 .

[30]  Martha M. Yee.  System design and cataloging meet the user: User interfaces to online public access catalogs , 1991 .

[31]  Charles T. Meadow,et al.  OAKDEC, a program for studying the effects on users of a procedural expert system for database searching , 1988, Inf. Process. Manag..

[32]  Charles T. Meadow,et al.  A computer intermediary for interactive database searching. II. Evaluation , 1982, J. Am. Soc. Inf. Sci..

[33]  Henry Lieberman,et al.  Letizia: An Agent That Assists Web Browsing , 1995, IJCAI.

[34]  Nicholas J. Belkin,et al.  Iterative exploration, design and evaluation of support for query reformulation in interactive information retrieval , 2001, Inf. Process. Manag..

[35]  John Riedl,et al.  Explaining collaborative filtering recommendations , 2000, CSCW '00.

[36]  Marcia J. Bates,et al.  The design of browsing and berrypicking techniques for the online search interface , 1989 .

[37]  Ian H. Witten,et al.  Managing Gigabytes: Compressing and Indexing Documents and Images , 1999 .

[38]  Marcia J. Bates,et al.  Where should the person stop and the information search interface start? , 1990, Inf. Process. Manag..

[39]  Marcia J. Bates,et al.  Information search tactics , 1979, J. Am. Soc. Inf. Sci..

[40]  Charles T. Meadow,et al.  A Computer Intermediary for Interactive Database Searching. I. Design , 2007, J. Am. Soc. Inf. Sci..

[41]  Robert N. Oddy,et al.  Pthomas: An adaptive information retrieval system on the connection machine , 1991, Inf. Process. Manag..

[42]  Tefko Saracevic,et al.  RELEVANCE: A review of and a framework for the thinking on the notion in information science , 1997, J. Am. Soc. Inf. Sci..

[43]  Amanda Spink,et al.  Exploration into stages in the information search process in online information retrieval: communication between users and intermediaries , 1992 .

[44]  Amanda Spink,et al.  From E-Sex to E-Commerce: Web Search Changes , 2002, Computer.

[45]  Micheline Beaulieu.  Interaction in information searching and retrieval , 2000, J. Documentation.

[46]  Nicholas J. Belkin,et al.  A case for interaction: a study of interactive information retrieval behavior and effectiveness , 1996, CHI.

[47]  Licia Calvi,et al.  AHA : a generic adaptive hypermedia system , 1998 .

[48]  Giovanni Guida,et al.  User modeling in intelligent information retrieval , 1987, Inf. Process. Manag..

[49]  Christine L. Borgman,et al.  Rethinking Online Monitoring Methods for Information Retrieval Systems: From Search Product to Search Process , 1996, J. Am. Soc. Inf. Sci..

[50]  Peter Bruza,et al.  Interactive Internet search: keyword, directory and query reformulation mechanisms compared , 2000, SIGIR '00.

[51]  Susan Gauch,et al.  An expert system for automatic query reformation , 1993 .

[52]  Marcia J. Bates,et al.  A Profile of End-User Searching Behavior by Humanities Scholars: The Getty Online Searching Project Report No. 2 , 1993, J. Am. Soc. Inf. Sci..

[53]  Marcia J. Bates,et al.  A profile of end‐user searching behavior by humanities scholars: The Getty Online Searching Project Report No. 2 , 1993 .

[54]  C. Lee Giles,et al.  Digital Libraries and Autonomous Citation Indexing , 1999, Computer.

[55]  Licia Calvi,et al.  2L670: a flexible adaptive hypertext courseware system , 1998, HYPERTEXT '98.

[56]  Stuart E. Middleton,et al.  Capturing knowledge of user preferences: ontologies in recommender systems , 2001, K-CAP '01.

[57]  Amanda Spink,et al.  Real life, real users, and real needs: a study and analysis of user queries on the web , 2000, Inf. Process. Manag..

[58]  Douglas W. Oard,et al.  Modeling Information Content Using Observable Behavior , 2001 .

[59]  Stefano Mizzaro,et al.  Strategic help in user interfaces for information retrieval , 2002, J. Assoc. Inf. Sci. Technol..

[60]  Daqing He,et al.  Combining evidence for automatic Web session identification , 2002, Inf. Process. Manag..

[61]  S. Mizzaro.  Intelligent Interfaces for Information Retrieval A Review , 1996 .

[62]  Pattie Maes,et al.  Agents that reduce work and information overload , 1994, CACM.

[63]  Krishna Bharat,et al.  The Krakatoa Chronicle: An Interactive Personalized Newspaper on the Web , 1995, World Wide Web J..

[64]  Nicholas J. Belkin,et al.  Cases, scripts, and information-seeking strategies: On the design of interactive information retrieval systems , 1995 .

[65]  Rila Mandala.  Improving automatic query expansion by using heterogeneous thesauri , 2001 .

[66]  Ay Se,et al.  Capturing Information Need by Learning User Context , 1999 .

[67]  Ian Ruthven,et al.  Re-examining the potential effectiveness of interactive query expansion , 2003, SIGIR.

[68]  Peter Willett,et al.  Overall introduction , 1997 .

[69]  Bernard J. Jansen,et al.  The impact of automated assistance on the information retrieval process , 2003, CHI Extended Abstracts.

[70]  W. Bruce Croft,et al.  I 3 R: a new approach to the design of document retrieval systems , 1987 .