Query expansion behavior within a thesaurus-enhanced search environment: A user-centered evaluation

The study reported here investigated the query expansion behavior of end-users interacting with a thesaurus-enhanced search system on the Web. Two groups, namely academic staff and postgraduate students, were recruited into this study. Data were collected from 90 searches performed by 30 users using the OVID interface to the CAB abstracts database. Data-gathering techniques included questionnaires, screen capturing software, and interviews. The results presented here relate to issues of search-topic and search-term characteristics, number and types of expanded queries, usefulness of thesaurus terms, and behavioral differences between academic staff and postgraduate students in their interaction. The key conclusions drawn were that (a) academic staff chose more narrow and synonymous terms than did postgraduate students, who generally selected broader and related terms; (b) topic complexity affected users' interaction with the thesaurus in that complex topics required more query expansion and search term selection; (c) users' prior topic-search experience appeared to have a significant effect on their selection and evaluation of thesaurus terms; (d) in 50p of the searches where additional terms were suggested from the thesaurus, users stated that they had not been aware of the terms at the beginning of the search; this observation was particularly noticeable in the case of postgraduate students. © 2006 Wiley Periodicals, Inc.

[1]  Amanda Spink,et al.  Term Relevance Feedback and Mediated Database Searching: Implications for Information Retrieval Practice and Systems Design , 1995, Inf. Process. Manag..

[2]  Efthimis N. Efthimiadis,et al.  A user-centred evaluation of ranking algorithms for interactive query expansion , 1993, SIGIR.

[3]  Pauline Atherton,et al.  An Analysis of Controlled Vocabulary and Free Text Search Statements in Online Searches , 1980 .

[4]  Jane Greenberg Optimal query expansion (QE) processing methods with semantically encoded structured thesauri terminology , 2001 .

[5]  Stefano Mizzaro,et al.  Evaluating user interfaces to information retrieval systems: a case study on user support , 1996, SIGIR '96.

[6]  Mark Magennis,et al.  The potential and actual effectiveness of interactive query expansion , 1997, SIGIR '97.

[7]  Nicholas J. Belkin,et al.  New tools and old habits : the interactive searching behavior of expert online searchers using INQUERY , 1994 .

[8]  Shirley Anne Cousins,et al.  Enhancing subject Access to OPACs: Controlled vocabulary vs Natural Language , 1992, J. Documentation.

[9]  Helen Howard,et al.  Measures that discriminate among online searchers with different training and experience , 1982 .

[10]  Pauline A. Cochrane,et al.  A Hypertextual Interface for a Searcher's Thesaurus , 1995, Digital library.

[11]  Manikya Rao Muddamalle Natural language versus controlled vocabulary in information retrieval: a case study in soil mechanics , 1998 .

[12]  Jaana Kristensen,et al.  Expanding End-Users' Query Statements for Free Text Searching with a Search-Aid Thesaurus , 1993, Inf. Process. Manag..

[13]  Peter Ingwersen,et al.  Cognitive Perspectives of Information Retrieval Interaction: Elements of a Cognitive IR Theory , 1996, J. Documentation.

[14]  JonesSusan,et al.  Interactive thesaurus navigation , 1995 .

[15]  Efthimis N. Efthimiadis,et al.  Interactive query expansion: A user-based evaluation in a relevance feedback environment , 2000, J. Am. Soc. Inf. Sci..

[16]  Ellen M. Voorhees,et al.  Query expansion using lexical-semantic relations , 1994, SIGIR '94.

[17]  Ali Shiri,et al.  Usability and user perceptions of a thesaurus-enhanced search interface , 2005, J. Documentation.

[18]  Jennifer E. Rowley,et al.  The controlled versus natural indexing languages debate revisited: a perspective on information retrieval practice and research , 1994, J. Inf. Sci..

[19]  Raya Fidel Who Needs Controlled Vocabulary , 1992 .

[20]  Ernest Perez Text Enhancement: Controlled Vocabulary vs. Free Text. , 1982 .

[21]  Marti A. Hearst,et al.  Cat-a-Cone: an interactive interface for specifying searches and viewing retrieval results using a large category hierarchy , 1997, SIGIR '97.

[22]  Da Evans,et al.  RIAO 94 CONFERENCE PROCEEDINGS - INTELLIGENT MULTIMEDIA INFORMATION RETRIEVAL SYSTEMS AND MANAGEMENT , 1994 .

[23]  Carol H. Fenichel,et al.  Online searching: Measures that discriminate among users with different types of experiences , 1981, J. Am. Soc. Inf. Sci..

[24]  Tefko Saracevic,et al.  Evaluation of evaluation in information retrieval , 1995, SIGIR '95.

[25]  Geoffrey P. Ellis,et al.  HIBROWSE for bibliographic databases , 1994, J. Inf. Sci..

[26]  Ali Shiri,et al.  Thesauri on the Web: current developments and trends , 2000, Online Inf. Rev..

[27]  Tobun Dorbin Ng,et al.  Alleviating search uncertainty through concept associations: automatic indexing, co-occurrence analysis, and parallel computing , 1998 .

[28]  Gobinda G. Chowdhury,et al.  Incorporating the results of co-word analyses to increase search variety for information retrieval , 2000, J. Inf. Sci..

[29]  Nicholas J. Belkin,et al.  Braque: Design of an Interface to Support User Interaction in Information Retrieval , 1993, Inf. Process. Manag..

[30]  Ali Shiri,et al.  End-user interaction with thesauri: an evaluation of cognitive overlap in search term selection , 2004 .

[31]  Paul B. Kantor,et al.  A study of information seeking and retrieving. I. background and methodology , 1988 .

[32]  James D. Anderson,et al.  Building End-User Thesauri from Full-Text , 1991 .

[33]  Edie Rasmussen,et al.  Evaluating interactive systems in TREC , 1996 .

[34]  Tefko Saracevic,et al.  The Stratified Model of Information Retrieval Interaction: Extension and Applications , 1997 .

[35]  Gary Marchionini Information-seeking strategies of novices using a full-text electronic encyclopedia , 1989 .

[36]  P. Solomon Children's information retrieval behavior: A case analysis of an OPAC , 1993 .

[37]  Anne B. Piternick Searching vocabularies: a developing category of online search tools , 1984 .

[38]  Hsinchun Chen,et al.  Automatic Thesaurus Generation for an Electronic Community System , 1995, J. Am. Soc. Inf. Sci..

[39]  Alistair Sutcliffe,et al.  Empirical studies of end-user information searching , 2000 .

[40]  W. David Penniman,et al.  Monitoring and evaluation of on-line information system usage , 1980, Inf. Process. Manag..

[41]  Crawford Revie,et al.  Thesaurus-enhanced search interfaces , 2002, J. Inf. Sci..

[42]  Hsinchun Chen,et al.  A concept space approach to addressing the vocabulary problem in scientific information retrieval: an experiment on the worm community system , 1997 .

[43]  C. P. R. Dubois,et al.  Free text vs. controlled vocabulary; a reassessment , 1987 .

[44]  Paul B. Kantor,et al.  A study of information seeking and retrieving. II. Users, questions, and effectiveness , 1988 .

[45]  Marcia J. Bates,et al.  Indexing and Access for Digital Libraries and the Internet: Human, Database, and Domain Factors , 1998, J. Am. Soc. Inf. Sci..

[46]  Pertti Vakkari,et al.  Cognition and changes of search terms and tactics during task performance: A longitudinal case study , 2000, RIAO.

[47]  Raya Fidel,et al.  Searchers' selection of search keys: II. Controlled vocabulary or free‐text searching , 1991 .

[48]  Hsinchun Chen,et al.  An algorithmic approach to concept exploration in a large knowledge network (automatic thesaurus consultation): symbolic branch-and-bound search vs. connectionist Hopfield net activation , 1995 .

[49]  Ali Shiri End-user interaction with thesaurus-enhanced search interfaces: an evaluation of search term selection for query expansion , 2004, SIGF.

[50]  Micheline Beaulieu,et al.  Experiments on interfaces to support query expansion , 1997, J. Documentation.

[51]  Hans-Peter Frei,et al.  Concept based query expansion , 1993, SIGIR.

[52]  Jane Greenberg,et al.  Automatic query expansion via lexical-semantic relationships , 2001, J. Assoc. Inf. Sci. Technol..

[53]  Hsinchun Chen,et al.  Interactive term suggestion for users of digital libraries: using subject thesauri and co-occurrence lists for information retrieval , 1996, DL '96.

[54]  Hinrich Schütze,et al.  A Cooccurrence-Based Thesaurus and Two Applications to Information Retrieval , 1994, Inf. Process. Manag..

[55]  Xia Lin Visual MeSH , 1999, SIGIR '99.

[56]  Hsinchun Chen,et al.  Cognitive process as a basis for intelligent retrieval systems design , 1991, Inf. Process. Manag..

[57]  Laura B. Cohen,et al.  A Natural Language Thesaurus for the Humanities: The Need for a Database Search Aid , 1998, The Library Quarterly.

[58]  Micheline Hancock-Beaulieu,et al.  An Evaluation of Interactive Query Expansion in an Online Library Catalogue with a Graphical User Interface , 1995, J. Documentation.

[59]  Maristella Agosti,et al.  A Hypertext Environment for Interacting with Large Textual Databases , 1992, Inf. Process. Manag..

[60]  Amanda Spink,et al.  A user-centered approach to evaluating human interaction with Web search engines: an exploratory study , 2002, Inf. Process. Manag..

[61]  Marcia J. Bates,et al.  Subject access in online catalogs: A design model , 1986 .

[62]  Takenobu Tokunaga,et al.  Combining multiple evidence from different types of thesaurus for query expansion , 1999, SIGIR '99.

[63]  Stephen E. Robertson,et al.  On the Evaluation of IR Systems , 1992, Inf. Process. Manag..

[64]  Nicholas J. Belkin,et al.  Cases, scripts, and information-seeking strategies: On the design of interactive information retrieval systems , 1995 .

[65]  Elaine Svenonius Unanswered questions in the design of controlled vocabularies , 1986 .

[66]  Jaana Kekäläinen,et al.  The impact of query structure and query expansion on retrieval performance , 1998, SIGIR '98.

[67]  William Sugar User-Centered Perspective of Information Retrieval Research and Analysis Methods. , 1995 .

[68]  Stephen P. Harter,et al.  Evaluation of information retrieval systems : Approaches, issues, and methods , 1997 .

[69]  Soyeon Park Usability, user preferences, effectiveness, and user behaviors when searching individual and integrated full-text databases: implications for digital libraries , 2000 .

[70]  Amanda Spink,et al.  Interaction in information retrieval: selection and effectiveness of search terms , 1997 .

[71]  Ingrid Hsieh-Yee,et al.  Effects of Search Experience and Subject Knowledge on the Search Tactics of Novice and Experienced Searchers. , 1993 .

[72]  Micheline Hancock-Beaulieu,et al.  Interactive thesaurus navigation: intelligence rules ok? , 1995 .

[73]  Gobinda G. Chowdhury,et al.  Assessing the impact of user interaction with thesaural knowledge structures: a quantitative analysis framework , 2002 .

[74]  Bernard J. Jansen,et al.  The effect of query complexity on Web searching results , 2000, Inf. Res..

[75]  Efthimis N. Efthimiadis,et al.  User Choices: A new Yardstick for the Evaluation of Ranking Algorithms for Interactive Query Expansion , 1995, Inf. Process. Manag..

[76]  Stephen E. Robertson,et al.  Research and evaluation in information retrieval , 1997, J. Documentation.

[77]  J. Kristensen,et al.  The effectiveness of a searching thesaurus in free-text searching in a full-text database , 1990 .

[78]  Pertti Vakkari,et al.  Changes of search terms and tactics while writing a research proposal: A longitudinal case study , 2003, Inf. Process. Manag..

[79]  Charles T. Meadow,et al.  A study of user performance and attitudes with information retrieval interfaces , 1995 .