A review of factors influencing user satisfaction in information retrieval

The authors investigate factors influencing user satisfaction in information retrieval. It is evident from this study that user satisfaction is a subjective variable, which can be influenced by several factors such as system effectiveness, user effectiveness, user effort, and user characteristics and expectations. Therefore, information retrieval evaluators should consider all these factors in obtaining user satisfaction and in using it as a criterion of system effectiveness. Previous studies have conflicting conclusions on the relationship between user satisfaction and system effectiveness; this study has substantiated these findings and supports using user satisfaction as a criterion of system effectiveness. © 2010 Wiley Periodicals, Inc.

[1]  Andrew Turpin,et al.  Do batch and user evaluations give the same results? , 2000, SIGIR '00.

[2]  William S. Cooper,et al.  On selecting a measure of retrieval effectiveness part II. Implementation of the philosophy , 1973, J. Am. Soc. Inf. Sci..

[3]  E. Hufnagel.  User satisfaction-are we really measuring system effectiveness , 1990, Twenty-Third Annual Hawaii International Conference on System Sciences.

[4]  Frances C. Johnson,et al.  User satisfaction as a measure of system performance , 2007, J. Libr. Inf. Sci..

[5]  Harm J. A. Biemans,et al.  Differences between novice and experienced users in searching information on the World Wide Web , 2000 .

[6]  Falk Scholer,et al.  User performance versus precision measures for simple search tasks , 2006, SIGIR.

[7]  Louise T. Su.  Value of Search Results as a Whole as the Best Single Measure of Information Retrieval Performance , 1998, Inf. Process. Manag..

[8]  Kasper Hornbæk,et al.  Measuring usability: are effectiveness, efficiency, and satisfaction really correlated? , 2000, CHI.

[9]  T. Landauer,et al.  Handbook of Human-Computer Interaction , 1997 .

[10]  Tetsuya Sakai,et al.  The Relationship between Answer Ranking and User Satisfaction in a Question Answering System , 2005, NTCIR.

[11]  Jakob Nielsen,et al.  Chapter 4 – The Usability Engineering Lifecycle , 1993 .

[12]  Ramona J. Steffey,et al.  Evaluating User Success and Satisfaction with CD-ROM. , 1989 .

[13]  Gary Marchionini,et al.  Information Seeking in Electronic Environments , 1995 .

[14]  Thomas D. Wilson,et al.  Human Information Behavior , 2000, Informing Sci. Int. J. an Emerg. Transdiscipl..

[15]  Andrew Turpin,et al.  Why batch and user evaluations do not give the same results , 2001, SIGIR '01.

[16]  Louise T. Su.  A comprehensive and systematic model of user evaluation of Web search engines: II. An evaluation by undergraduates , 2003, J. Assoc. Inf. Sci. Technol..

[17]  Charles R. Hildreth,et al.  Accounting for users' inflated assessments of on-line catalogue search performance and usefulness: an experimental study , 2001, Inf. Res..

[18]  Louise T. Su.  An investigation to find appropriate measures for evaluating interactive information retrieval , 1991 .

[19]  W. S. Cooper.  Expected search length: A single measure of retrieval effectiveness based on the weak ordering action of retrieval systems , 1968 .

[20]  F. W. Lancaster,et al.  Information retrieval systems; characteristics, testing, and evaluation , 1968 .

[21]  James Allan,et al.  When will information retrieval be "good enough"? , 2005, SIGIR '05.

[22]  Frances C. Johnson,et al.  Task dimensions of user evaluations of information retrieval systems , 2003, Inf. Res..

[23]  Judith A. Tessier.  New Measures of User Satisfaction With Computer Based Literature Searches. , 1977 .

[24]  William R. Hersh,et al.  A task-oriented approach to information retrieval evaluation , 1996 .

[25]  Effie Lai-Chong Law,et al.  User Effect in Evaluating Personalized Information Retrieval Systems , 2006, EC-TEL.

[26]  Mark Sanderson,et al.  The good and the bad system: does the test collection predict users' effectiveness? , 2008, SIGIR '08.

[27]  Myke Gluck,et al.  Exploring the Relationship between User Satisfaction and Relevance in Information Systems , 1996, Inf. Process. Manag..

[28]  Nicholas J. Belkin,et al.  Interaction in information systems : a review of research from document retrieval to knowledge-based systems , 1985 .

[29]  Louise T. Su.  Evaluation Measures for Interactive Information Retrieval , 1992, Inf. Process. Manag..

[30]  William R. Hersh,et al.  An evaluation of interactive Boolean and natural language searching with an online medical textbook , 1995 .

[31]  Paul B. Kantor,et al.  A study of information seeking and retrieving. II. Users, questions, and effectiveness , 1988 .

[32]  Stephen P. Harter,et al.  Evaluation of information retrieval systems : Approaches, issues, and methods , 1997 .

[33]  David Hawking,et al.  Evaluation by comparing result sets in context , 2006, CIKM '06.

[34]  Louise T. Su.  The Relevance of Recall and Precision in User Evaluation , 1994, J. Am. Soc. Inf. Sci..

[35]  Amy W. Gatian.  Is user satisfaction a valid measure of system effectiveness? , 1994, Inf. Manag..

[36]  Dagobert Soergel,et al.  Is user satisfaction a hobgoblin? , 1976, J. Am. Soc. Inf. Sci..

[37]  Beth Sandore.  Online Searching: What Measure Satisfaction?. , 1990 .

[38]  Scott B. Huffman,et al.  How well does result relevance predict session satisfaction? , 2007, SIGIR.

[39]  F. W. Lancaster.  Searching Databases on CD-ROM: Comparison of the Results of End-User Searching with Results from Two Modes of Searching by Skilled Intermediaries. , 1994 .

[40]  William R. Hersh,et al.  Towards new measures of information retrieval evaluation , 1995, SIGIR '95.

[41]  F. W. Lancaster,et al.  MEDLARS: Report on the Evaluation of Its Operating Efficiency. , 1997 .

[42]  Harry Bruce,et al.  User satisfaction with information seeking on the Internet , 1998 .

[43]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[44]  Dennis E. Egan,et al.  Individual Differences In Human-Computer Interaction , 1988 .

[45]  Chris Buckley,et al.  OHSUMED: an interactive retrieval evaluation and new large test collection for research , 1994, SIGIR '94.