A review of factors influencing user satisfaction in information retrieval

This paper investigates factors influencing user satisfaction in information retrieval. It is evident from this study that user satisfaction is a subjective variable which can be influenced by several factors such as system effectiveness, user effectiveness, user effort and user characteristics and expectations. Therefore, information retrieval evaluators should consider all these factors in obtaining user satisfaction and in using it as a criterion of system effectiveness. Previous studies have conflicting conclusion on the relationship between user satisfaction and system effectiveness, this study has substantiated this relationship and supports using user satisfaction as a criterion of system effectiveness.

[1]  Tetsuya Sakai,et al.  The Relationship between Answer Ranking and User Satisfaction in a Question Answering System , 2005, NTCIR.

[2]  Beth Sandore.  Online Searching: What Measure Satisfaction?. , 1990 .

[3]  Louise T. Su.  A comprehensive and systematic model of user evaluation of Web search engines: II. An evaluation by undergraduates , 2003, J. Assoc. Inf. Sci. Technol..

[4]  William S. Cooper,et al.  On selecting a measure of retrieval effectiveness part II. Implementation of the philosophy , 1973, J. Am. Soc. Inf. Sci..

[5]  E. Hufnagel.  User satisfaction-are we really measuring system effectiveness , 1990, Twenty-Third Annual Hawaii International Conference on System Sciences.

[6]  Frances C. Johnson,et al.  Task dimensions of user evaluations of information retrieval systems , 2003, Inf. Res..

[7]  Charles R. Hildreth,et al.  Accounting for users' inflated assessments of on-line catalogue search performance and usefulness: an experimental study , 2001, Inf. Res..

[8]  Louise T. Su.  An investigation to find appropriate measures for evaluating interactive information retrieval , 1991 .

[9]  Andrew Turpin,et al.  Do batch and user evaluations give the same results? , 2000, SIGIR '00.

[10]  Judith A. Tessier.  New Measures of User Satisfaction With Computer Based Literature Searches. , 1977 .

[11]  Chris Buckley,et al.  OHSUMED: an interactive retrieval evaluation and new large test collection for research , 1994, SIGIR '94.

[12]  Jaana Kekäläinen,et al.  IR evaluation methods for retrieving highly relevant documents , 2000, SIGIR Forum.

[13]  Harry Bruce,et al.  User satisfaction with information seeking on the Internet , 1998 .

[14]  M. Butler.  Information Retrieval Systems Characteristics, Testing, and Evaluation , 1970 .

[15]  Amy W. Gatian.  Is user satisfaction a valid measure of system effectiveness? , 1994, Inf. Manag..

[16]  Nicholas J. Belkin,et al.  Interaction in information systems : a review of research from document retrieval to knowledge-based systems , 1985 .

[17]  Stephen P. Harter,et al.  Evaluation of information retrieval systems : Approaches, issues, and methods , 1997 .

[18]  David Hawking,et al.  Evaluation by comparing result sets in context , 2006, CIKM '06.

[19]  William R. Hersh,et al.  A task-oriented approach to information retrieval evaluation , 1996 .

[20]  Stephen J. Westerman,et al.  Individual differences in human-computer interaction , 1993 .

[21]  Andrew Turpin,et al.  Why batch and user evaluations do not give the same results , 2001, SIGIR '01.

[22]  William R. Hersh,et al.  An Evaluation of Interactive Boolean and Natural Language Searching with an Online Medical Textbook , 1995, J. Am. Soc. Inf. Sci..

[23]  Mark Sanderson,et al.  The good and the bad system: does the test collection predict users' effectiveness? , 2008, SIGIR '08.

[24]  Frances C. Johnson,et al.  User satisfaction as a measure of system performance , 2007, J. Libr. Inf. Sci..

[25]  Dagobert Soergel,et al.  Is user satisfaction a hobgoblin? , 1976, J. Am. Soc. Inf. Sci..

[26]  F. W. Lancaster,et al.  MEDLARS: Report on the Evaluation of Its Operating Efficiency. , 1997 .

[27]  F. Wilfrid Lancaster,et al.  Evaluation within the environment of an operating information service , 2008 .

[28]  Louise T. Su.  Value of Search Results as a Whole as the Best Single Measure of Information Retrieval Performance , 1998, Inf. Process. Manag..

[29]  Louise T. Su.  Evaluation Measures for Interactive Information Retrieval , 1992, Inf. Process. Manag..

[30]  W. S. Cooper.  Expected search length: A single measure of retrieval effectiveness based on the weak ordering action of retrieval systems , 1968 .

[31]  Effie Lai-Chong Law,et al.  User Effect in Evaluating Personalized Information Retrieval Systems , 2006, EC-TEL.

[32]  Paul B. Kantor,et al.  A study of information seeking and retrieving. II. Users, questions, and effectiveness , 1988, J. Am. Soc. Inf. Sci..

[33]  Myke Gluck,et al.  Exploring the Relationship between User Satisfaction and Relevance in Information Systems , 1996, Inf. Process. Manag..

[34]  Ramona J. Steffey,et al.  Evaluating User Success and Satisfaction with CD-ROM. , 1989 .

[35]  Gary Marchionini,et al.  Information Seeking in Electronic Environments , 1995 .

[36]  Erik Frekjmr,et al.  Measuring Usability: Are Effectiveness, Efficiency, and Satisfaction Really Correlated? , 2000 .

[37]  Scott B. Huffman,et al.  How well does result relevance predict session satisfaction? , 2007, SIGIR.

[38]  Robert R. Korfhage,et al.  Information Storage and Retrieval , 1963 .