Effects of domain knowledge on reference search with the PubMed database: An experimental study

Many researchers in medical and life sciences commonly use the PubMed online search engine () to access the MEDLINE bibliographic database. The researchers' strategies were investigated as a function of their knowledge of the content area. Sixteen life science researchers with no experience in neuroscience and 16 neuroscience researchers of matched professional experience performed five bibliographic search tasks about neuroscience topics. Objective measures and concomitant verbal protocols were used to assess behavior and performance. Whatever their knowledge of PubMed, neuroscientists could find adequate references within the allotted time period. Despite their lack of knowledge in neuroscience, life scientists could select adequate references with the same efficiency. However, differences were observed in the way neuroscientists and life scientists proceeded. For instance, life scientists took more time to read the task instructions and opened more abstracts while selecting their answers. These data suggest that regular use of online databases combined with graduate-level expertise in a broad scientific field like biology can compensate for the absence of knowledge in the specific domain in which references are sought. The large inter-individual variability in performance within both groups implies that beyond domain knowledge, individual cognitive abilities are the main determinants of bibliographic search performance. © 2009 Wiley Periodicals, Inc.

[1]  Kyung-Sun Kim,et al.  Cognitive style and on-line database search experience as predictors of Web search performance , 2000, J. Am. Soc. Inf. Sci..

[2]  W. Hersh,et al.  Factors associated with successful answering of clinical questions using an information retrieval system. , 2002, Bulletin of the Medical Library Association.

[3]  Klaus Nordhausen,et al.  Modeling successful performance in Web searching , 2006 .

[4]  Christine L. Borgman.  The User's Mental Model of an Information Retrieval System: An Experiment on a Prototype Online Catalog , 1986, Int. J. Man Mach. Stud..

[5]  Charles T. Scialfa,et al.  Age Differences in Search of Web Pages: The Effects of Link Size, Link Number, and Clutter , 2004, Hum. Factors.

[6]  P. McLeod,et al.  Inflexibility of experts—Reality or myth? Quantifying the Einstellung effect in chess masters , 2008, Cognitive Psychology.

[7]  Amanda Spink,et al.  Information-seeking and mediated searching. Part 1. Theoretical framework and research design , 2002, J. Assoc. Inf. Sci. Technol..

[8]  Charles P. Friedman,et al.  Medical students' personal knowledge, searching proficiency, and database use in problem solving , 1995 .

[9]  M. A. Britt,et al.  Studying and Using Multiple Documents in History: Effects of Discipline Expertise , 1997 .

[10]  Emanuel Maidenberg.  Expertise in online information retrieval , 1992 .

[11]  Paul B. Kantor,et al.  A study of information seeking and retrieving. II. Users, questions, and effectiveness , 1988 .

[12]  Bryce Allen,et al.  Topic Knowledge and Online Catalog Search Formulation , 1991, The Library Quarterly.

[13]  R. Buckner.  Memory and Executive Function in Aging and AD Multiple Factors that Cause Decline and Reserve Factors that Compensate , 2004, Neuron.

[14]  Thura Mack,et al.  Designing for Experts: How Scholars Approach an Academic Library Web Site , 2004 .

[15]  Howard D. White,et al.  Combining bibliometrics, information retrieval, and relevance theory, Part 2: Some implications for information science , 2007, J. Assoc. Inf. Sci. Technol..

[16]  L. Taconnat,et al.  The effect of aging in recollective experience: The processing speed and executive functioning hypothesis , 2007, Consciousness and Cognition.

[17]  Thorvald Haerem,et al.  The influence of degree of expertise and objective task complexity on perceived task complexity and performance. , 2007, The Journal of applied psychology.

[18]  Gary Marchionini,et al.  Information Seeking in Electronic Environments , 1995 .

[19]  Susan Wiedenbeck,et al.  PATTERNS OF INFORMATION SEEKING ON THE WEB: A QUALITATIVE STUDY OF DOMAIN EXPERTISE AND WEB EXPERTISE , 2003 .

[20]  J. Vasterling,et al.  Memory of psychodiagnostic information: biases and effects of expertise. , 2001, The American journal of psychology.

[21]  Boryung Ju.  Does domain knowledge matter: Mapping users' expertise to their information interactions , 2007 .

[22]  Elmer V. Bernstam,et al.  A day in the life of PubMed: analysis of a typical day's query log. , 2007, Journal of the American Medical Informatics Association : JAMIA.

[23]  Thomas L. Jacobson,et al.  Computer, system, and subject knowledge in novice searching of a full-text, multifile database , 1992 .

[24]  Jean-François Rouet.  What was I looking for? The influence of task specificity and prior knowledge on students' search strategies in hypertext , 2003, Interact. Comput..

[25]  Amanda Spink,et al.  Human information behavior: Integrating diverse approaches and information use , 2006 .

[26]  William R. Hersh.  Relevance and retrieval evaluation: perspectives from medicine , 1994 .

[27]  Karen Markey.  Twenty-five years of end-user searching, Part 1: Research findings , 2007 .

[28]  M L Pao,et al.  Factors affecting students' use of MEDLINE. , 1993, Computers and biomedical research, an international journal.

[29]  Charles T. Scialfa,et al.  Predictors of Web Navigation Performance in a Life Span Sample of Adults , 2005, Hum. Factors.

[30]  Gary Marchionini.  Information Seeking in Full-Text End-User-Oriented Search Systems: The Roles of Domain and Search Expertise , 1993 .

[31]  Dennis E. Egan,et al.  Individual Differences In Human-Computer Interaction , 1988 .

[32]  Amanda Spink,et al.  Information seeking and mediated searching. Part 4. Cognitive styles in information seeking , 2002, J. Assoc. Inf. Sci. Technol..

[33]  S. P. Harter.  Psychological relevance and information science , 1992 .

[34]  Wai-Tat Fu,et al.  Soft constraints in interactive behavior: the case of ignoring perfect knowledge in-the-world for imperfect knowledge in-the-head , 2004, Cogn. Sci..

[35]  Wayne D. Gray,et al.  Suboptimal tradeoffs in information seeking , 2006, Cognitive Psychology.

[36]  S. D. De Groote,et al.  Measuring use patterns of online journals and databases. , 2003, Journal of the Medical Library Association : JMLA.

[37]  Stephen B. Johnson,et al.  A multi-level model of information seeking in the clinical domain , 2008, J. Biomed. Informatics.

[38]  Marcia J. Bates,et al.  Separating the knowledge layers: Cognitive analysis of search knowledge through hierarchical goal decompositions , 2005, ASIST.

[39]  W. Hersh,et al.  Use of a multi-application computer workstation in a clinical setting. , 1994, Bulletin of the Medical Library Association.

[40]  Menno D. T. de Jong,et al.  Employing think-aloud protocols and constructive interaction to test the usability of online library catalogues: a methodological comparison , 2004, Interact. Comput..

[41]  D. R. Swanson.  Historical note: information retrieval and the future of an illusion , 1997 .

[42]  M. Chi,et al.  Problem-Solving Ability. , 1985 .

[43]  Steven W. Brown,et al.  The effects and interaction of spatial visualization and domain expertise on information seeking , 2004, Comput. Hum. Behav..

[44]  Charles P. Friedman,et al.  Medical errors as a result of specialization , 2003, J. Biomed. Informatics.

[45]  Pertti Vakkari,et al.  The anticipated and assessed contribution of information types in references retrieved for preparing a research proposal , 2005, J. Assoc. Inf. Sci. Technol..

[46]  Vimla L. Patel,et al.  Emerging paradigms of cognition in medical decision-making , 2002, J. Biomed. Informatics.

[47]  Judit Bar-Ilan,et al.  Preference for electronic format of scientific journals—A case study of the Science Library users at the Hebrew University , 2005 .

[48]  Christine Ros,et al.  The use of online electronic information resources in scientific research: The case of neuroscience , 2007 .

[49]  W R Hersh,et al.  How well do physicians use electronic information retrieval systems? A framework for investigation and systematic review. , 1998, JAMA.

[50]  Ingrid Hsieh-Yee,et al.  Effects of Search Experience and Subject Knowledge on the Search Tactics of Novice and Experienced Searchers. , 1993 .

[51]  Karen Macpherson,et al.  An information processing model of undergraduate electronic database information retrieval , 2004, J. Assoc. Inf. Sci. Technol..

[52]  D. Tabatabai,et al.  How experts and novices search the Web , 2005 .

[53]  Anita Ondrusek,et al.  The attributes of research on end-user online searching behavior: A retrospective review and analysis , 2004 .

[54]  Randall W. Engle,et al.  Effects of Domain Knowledge, Working Memory Capacity, and Age on Cognitive Performance: An Investigation of the Knowledge-Is-Power Hypothesis , 2000, Cognitive Psychology.

[55]  Jean-François Rouet,et al.  The Skills of Document Use: From Text Comprehension to Web-Based Learning , 2006 .

[56]  Yan Zhang,et al.  Patterns of Journal Use by Scientists through Three Evolutionary Phases , 2003, D Lib Mag..

[57]  D. Sperber,et al.  Relevance: Communication and Cognition , 1989 .

[58]  Elizabeth D. Liddy,et al.  The effects of expertise and feedback on search term selection and subsequent learning , 2005, J. Assoc. Inf. Sci. Technol..

[59]  K. Lonka,et al.  Conceptions of Learning and Knowledge: Does Training Make a Difference? , 1996 .

[60]  M.J.W. Stokmans,et al.  End‐users searching the online catalogue: the influence of domain and system knowledge on search patterns , 1994 .

[61]  Christine L. Borgman.  The user's mental model of an information retrieval system: an experiment on a prototype online catalog , 1999, Int. J. Hum. Comput. Stud..

[62]  David Miller,et al.  The role of individual differences in Internet searching: an empirical study , 2001 .

[63]  José J. Cañas,et al.  Are expert users always better searchers? Interaction of expertise and semantic grouping in hypertext search tasks , 2005, Behav. Inf. Technol..

[64]  Christoph Hölscher,et al.  Web search behavior of Internet experts and newbies , 2000, Comput. Networks.

[65]  J. Wiley.  Expertise as mental set: The effects of domain knowledge in creative problem solving , 1998, Memory & cognition.

[66]  K. A. Ericsson,et al.  Expert and exceptional performance: evidence of maximal adaptation to task constraints. , 1996, Annual review of psychology.

[67]  Deborah Shaw,et al.  Bibliographic Database Searching by Graduate Students in Language and Literature: Search Strategies, System Interfaces, and Relevance Judgments. , 1995 .

[68]  Swapnesh C. Patel,et al.  Effectiveness of expert semantic knowledge as a navigational aid within hypertext , 1998, Behav. Inf. Technol..

[69]  Dennis J. Devine,et al.  Domain-Specific Knowledge and Task Characteristics in Decision Making , 1995 .

[70]  D Fitzgerald,et al.  How good are clinical MEDLINE searches? A comparative study of clinical end-user and librarian searches. , 1990, Computers and biomedical research, an international journal.

[71]  Marcia J. Bates,et al.  The Getty End-User Online Searching Project in the Humanities: Report No. 6: Overview and Conclusions , 1996 .