Doctors' online information needs, cognitive search strategies, and judgments of information quality and cognitive authority: How predictive judgments introduce bias into cognitive search models

Literature examining information judgments and Internet search behaviors notes a number of major research gaps, including how users actually make these judgments outside of experiments or researcher-defined tasks, and how search behavior is impacted by a user's judgment of online information. Using the medical setting, where doctors face real consequences in applying the information found, we examine how information judgments employed by doctors to mitigate risk impact their cognitive search. Diaries encompassing 444 real clinical information search incidents, combined with semistructured interviews across 35 doctors, were analyzed via thematic analysis. Results show that doctors, though aware of the need for information quality and cognitive authority, rarely make evaluative judgments. This is explained by navigational bias in information searches and via predictive judgments that favor known sites where doctors perceive levels of information quality and cognitive authority. Doctors' mental models of the Internet sites and Web experience relevant to the task type enable these predictive judgments. These results suggest a model connecting online cognitive search and information judgment literatures. Moreover, this implies a need to understand cognitive search through longitudinal- or learning-based views for repeated search tasks, and adaptations to medical practitioner training and tools for online search. © 2010 Wiley Periodicals, Inc.

[1]  Peiling Wang,et al.  A Cognitive Model of Document Use During a Research Project. Study II. Decisions at the Reading and Citing Stages , 1999, J. Am. Soc. Inf. Sci..

[2]  Miriam J. Metzger,et al.  College student Web use, perceptions of information credibility, and verification behavior , 2003, Comput. Educ..

[3]  Brian Detlor,et al.  Information Seeking on the Web: An Integrated Model of Browsing and Searching , 2000, First Monday.

[4]  Wai-Tat Fu,et al.  SNIF-ACT: A Cognitive Model of User Navigation on the World Wide Web , 2007, Hum. Comput. Interact..

[5]  Ziming Liu,et al.  Evaluating the credibility of scholarly information on the web: A cross cultural study , 2005 .

[6]  Gaby Haddow,et al.  Focusing on health information: how to assess information quality on the internet , 2003 .

[7]  Marc Meola,et al.  Chucking the Checklist: A Contextual Approach to Teaching Undergraduates Web-Site Evaluation , 2004 .

[8]  Johanna I. Westbrook,et al.  Do online information retrieval systems help experienced clinicians answer clinical questions? , 2005, Journal of the American Medical Informatics Association : JAMIA.

[9]  James E. Pitkow,et al.  Characterizing Browsing Strategies in the World-Wide Web , 1995, Comput. Networks ISDN Syst..

[10]  D. R. Danielson,et al.  How do users evaluate the credibility of Web sites?: a study with over 2,500 participants , 2003, DUX '03.

[11]  N. Bolger,et al.  Diary methods: capturing life as it is lived. , 2003, Annual review of psychology.

[12]  Nigel King,et al.  Using templates in the thematic analysis of text , 2004 .

[13]  Caroline Haythornthwaite,et al.  New horizons for e-learning in medical education: ecological and Web 2.0 perspectives , 2007, Medical teacher.

[14]  Christian Köhler,et al.  How do consumers search for and appraise health information on the world wide web? Qualitative study using focus groups, usability tests, and in-depth interviews , 2002, BMJ : British Medical Journal.

[15]  Gunther Eysenbach,et al.  Consumer health informatics , 2000, BMJ : British Medical Journal.

[16]  David Miller,et al.  Web search strategies and human individual differences: A combined analysis , 2005, J. Assoc. Inf. Sci. Technol..

[17]  Clement T. Yu,et al.  Personalized Web search for improving retrieval effectiveness , 2004, IEEE Transactions on Knowledge and Data Engineering.

[18]  Ying Zhang,et al.  Time series analysis of a Web search engine transaction log , 2009, Inf. Process. Manag..

[19]  Frank Piontek,et al.  DSS in Healthcare: Advances and Opportunities , 2008 .

[20]  Barbara H Wixom,et al.  A Theoretical Integration of User Satisfaction and Technology Acceptance , 2005, Inf. Syst. Res..

[21]  Yvonne Rogers,et al.  Cognitive strategies in web searching. , 1999 .

[22]  Andrew Thatcher,et al.  Web search strategies: The influence of Web experience and task type , 2008, Inf. Process. Manag..

[23]  Amanda Spink,et al.  Information seeking and mediated searching. Part 4. Cognitive styles in information seeking , 2002, J. Assoc. Inf. Sci. Technol..

[24]  Jane E Lacovara When searching for the evidence, stop using Wikipedia! , 2008, Medsurg nursing : official journal of the Academy of Medical-Surgical Nurses.

[25]  Amanda Spink,et al.  Determining the informational, navigational, and transactional intent of Web queries , 2008, Inf. Process. Manag..

[26]  Wallapak Tavanapong,et al.  Developing a Framework for Assessing Information Quality on the World Wide Web , 2005, Informing Sci. Int. J. an Emerg. Transdiscipl..

[27]  Maged N Kamel Boulos,et al.  The emerging Web 2.0 social software: an enabling suite of sociable technologies in health and health care education. , 2007, Health information and libraries journal.

[28]  Dean Giustini,et al.  How Web 2.0 is changing medicine , 2006, BMJ : British Medical Journal.

[29]  M. McKnight,et al.  Health Care Providers' Information Seeking , 2000, Medical reference services quarterly.

[30]  Efraim Turban,et al.  Are intelligent e-commerce agents partners or predators? , 2002, CACM.

[31]  N. Bennett,et al.  Physicians' Internet information‐seeking behaviors , 2004, The Journal of continuing education in the health professions.

[32]  Miriam J. Metzger Making sense of credibility on the Web: Models for evaluating online information and recommendations for future research , 2007 .

[33]  Ephraim R. McLean,et al.  Information Systems Success: The Quest for the Dependent Variable , 1992, Inf. Syst. Res..

[34]  Marina Papastergiou,et al.  Students’ Mental Models of the Internet and Their Didactical Exploitation in Informatics Education , 2005, Education and Information Technologies.

[35]  A. Rangaswamy,et al.  A Fuzzy Set Model of Search and Consideration with an Application to an Online Market , 2003 .

[36]  Jonathan Wareham,et al.  Junior physician's use of Web 2.0 for information seeking and medical education: A qualitative study , 2009, Int. J. Medical Informatics.

[37]  Nicholas J. Belkin,et al.  Understanding Judgment of Information Quality and Cognitive Authority in the WWW , 1998 .

[38]  Ruth M. Rettie,et al.  Net Generation Culture , 2002, J. Electron. Commer. Res..

[39]  Pamela J. McKenzie Justifying Cognitive Authority Decisions: Discursive Strategies of Information Seekers , 2003, Library quarterly.

[40]  Soo Young Rieh,et al.  Credibility: A multidisciplinary framework , 2007 .

[41]  Glenn J. Browne,et al.  Stopping rule use during information search in design problems , 2004 .

[42]  Fred D. Davis Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology , 1989, MIS Q..

[43]  Nigel Ford,et al.  Web search strategies and human individual differences: Cognitive and demographic factors, Internet attitudes, and approaches: Research Articles , 2005 .

[44]  Vivian Cothey,et al.  A longitudinal study of World Wide Web users' information-searching behavior , 2002, J. Assoc. Inf. Sci. Technol..

[45]  T. Park The Nature of Relevance in Information Retrieval: An Empirical Study , 1993, The Library Quarterly.

[46]  Traci Hong,et al.  The influence of structural and message features on Web site credibility , 2006 .

[47]  Moyez Jiwa,et al.  Does general practice Google? , 2008, Australian family physician.

[48]  Joemon M. Jose,et al.  How users assess Web pages for information seeking , 2005, J. Assoc. Inf. Sci. Technol..

[49]  J. Rowley Product search in e‐shopping: a review and research propositions , 2000 .

[50]  Mark Taubert Use of Google as a diagnostic aid , 2006, BMJ : British Medical Journal.

[51]  Andrew Thatcher,et al.  Information-seeking behaviours and cognitive search strategies in different search tasks on the WWW , 2006 .

[52]  Kyung-Sun Kim,et al.  Implications of User Characteristics in Information Seeking on the World Wide Web , 2001, Int. J. Hum. Comput. Interact..

[53]  Linda L. Casebeer,et al.  Family physicians' information seeking behaviors: A survey comparison with other specialties , 2005, BMC Medical Informatics Decis. Mak..

[54]  Richard E. Boyatzis,et al.  Transforming Qualitative Information: Thematic Analysis and Code Development , 1998 .

[55]  Peter S. Fader,et al.  On the Depth and Dynamics of Online Search Behavior , 2004, Manag. Sci..

[56]  Daniel E. Rose,et al.  Understanding user goals in web search , 2004, WWW '04.

[57]  Jenny K. Holder,et al.  A visit to the information mall: Web searching behavior of high school students , 1999 .

[58]  V. Podichetty,et al.  Assessment of internet use and effects among healthcare professionals: a cross sectional survey , 2006, Postgraduate Medical Journal.

[59]  Amanda Spink,et al.  How are we searching the World Wide Web? A comparison of nine search engine transaction logs , 2006, Inf. Process. Manag..

[60]  Kristi L. Koenig,et al.  Diagnostic Dilemma? Just Google It? , 2007 .

[61]  J. Powell,et al.  Empirical studies assessing the quality of health information for consumers on the world wide web: a systematic review. , 2002, JAMA.

[62]  P. R. H. Hendriks,et al.  Specifying software quality with the extended ISO model , 1996, Software Quality Journal.

[63]  Bernard J. Jansen,et al.  Using the taxonomy of cognitive learning to model online searching , 2009, Inf. Process. Manag..

[64]  Peter Brophy,et al.  Student Searching Behavior and the Web: Use of Academic Resources and Google , 2005, Libr. Trends.

[65]  B. J. Fogg,et al.  What makes Web sites credible?: a report on a large quantitative study , 2001, CHI.

[66]  Olivia R. Liu Sheng,et al.  Interest-based personalized search , 2007, TOIS.

[67]  Saul Greenberg,et al.  How people revisit web pages: empirical findings and implications for the design of history systems , 1997, Int. J. Hum. Comput. Stud..

[68]  Mark Sanderson,et al.  Are Web-based informational queries changing? , 2009 .

[69]  Jane E. Klobas,et al.  Beyond information quality: fitness for purpose and electronic information resource use , 1995, J. Inf. Sci..

[70]  Todd H Wagner,et al.  Who searches the internet for health information? , 2006, Health services research.

[71]  Ken Masters,et al.  For what purpose and reasons do doctors use the Internet: A systematic review , 2008, Int. J. Medical Informatics.

[72]  Peiling Wang,et al.  A cognitive model of document use during a research project. Study I. document selection , 1998 .

[73]  G Eysenbach,et al.  Information in practice Towards quality management of medical information on the internet : evaluation , labelling , and filtering of information , 1998 .

[74]  Fatemeh Zahedi,et al.  The Measurement of Web-Customer Satisfaction: An Expectation and Disconfirmation Approach , 2002, Inf. Syst. Res..

[75]  Debra J. Slone The influence of mental models and goals on search patterns during Web interaction , 2002, J. Assoc. Inf. Sci. Technol..

[76]  Peter Bruza,et al.  Web searching: A process-oriented experimental study of three interactive search paradigms , 2002, J. Assoc. Inf. Sci. Technol..

[77]  Gary Marchionini,et al.  Information Seeking in Electronic Environments , 1995 .

[78]  J. Wareham,et al.  Health 2.0 and Medicine 2.0: Tensions and Controversies in the Field , 2008, Journal of medical Internet research.

[79]  Diane M. Strong,et al.  Information quality benchmarks: product and service performance , 2002, CACM.

[80]  Christoph Hölscher,et al.  Web search behavior of Internet experts and newbies , 2000, Comput. Networks.

[81]  Carol Tenopir,et al.  Users' interaction with World Wide Web resources: an exploratory study using a holistic approach , 2000, Inf. Process. Manag..

[82]  Gregory K. W. K. Chung,et al.  Children's Internet Searching on Complex Problems: Performance and Process Analyses , 1998, J. Am. Soc. Inf. Sci..

[83]  Eszter Hargittai,et al.  Beyond logs and surveys: In-depth measures of people's web use skills , 2002, J. Assoc. Inf. Sci. Technol..

[84]  Dae Ryun Chang,et al.  A model of consumer information search and online network externalities , 2002 .

[85]  Michael D. Byrne,et al.  The tangled Web we wove: a taskonomy of WWW use , 1999, CHI '99.

[86]  J. Sandars,et al.  Web 2.0 technologies for undergraduate and postgraduate medical education: an online survey , 2007, Postgraduate Medical Journal.

[87]  Brad Cahoon,et al.  Teaching and Learning Internet Skills , 1998 .

[88]  Colin Camerer Behavioral Game Theory: Experiments in Strategic Interaction , 2003 .

[89]  John Eng,et al.  A comparison of world wide web resources for identifying medical information. , 2008, Academic radiology.

[90]  H. Jaillet Web metrics: measuring patterns in online shopping , 2003 .

[91]  Robert L. Cromwell,et al.  Evaluating Internet resources: Identity, affiliation, and cognitive authority in a networked world , 2001, J. Assoc. Inf. Sci. Technol..

[92]  Viswanath Venkatesh,et al.  Assimilation of Interorganizational Business Process Standards , 2007, Inf. Syst. Res..

[93]  Soo Young Rieh,et al.  Interaction on the Web: Scholars' Judgment of Information Quality and Cognitive Authority , 2000 .

[94]  Cynthia LeRouge,et al.  Websites Most Frequently Used by Physician for Gathering Medical Information , 2006, AMIA.

[95]  Yan Zhang Undergraduate students' mental models of the Web as an information retrieval system , 2008 .

[96]  Jacob Cohen,et al.  The Equivalence of Weighted Kappa and the Intraclass Correlation Coefficient as Measures of Reliability , 1973 .

[97]  Alison Brettle,et al.  Evaluating information skills training in health libraries: a systematic review. , 2007, Health information and libraries journal.

[98]  R. McLean,et al.  The effect of Web 2.0 on the future of medical practice and education: Darwikinian evolution or folksonomic revolution? , 2007, The Medical journal of Australia.

[99]  Soo Young Rieh Judgement of information quality and cognitive authority in the Web , 2002 .

[100]  Jacquelyn A. Burkell,et al.  Believe it or not: Factors influencing credibility on the Web , 2002, J. Assoc. Inf. Sci. Technol..

[101]  Diane M. Strong,et al.  Beyond Accuracy: What Data Quality Means to Data Consumers , 1996, J. Manag. Inf. Syst..

[102]  B. J. Fogg,et al.  The elements of computer credibility , 1999, CHI '99.

[103]  Bryce Allen,et al.  Cognitive and task influences on Web searching behavior , 2002, J. Assoc. Inf. Sci. Technol..

[104]  B. J. Fogg,et al.  Persuasive technologie301398 , 1999, CACM.

[105]  David Miller,et al.  Web search strategies and human individual differences: A combined analysis: Research Articles , 2005 .

[106]  M. Easterby-Smith,et al.  Management Research: An Introduction , 1991 .

[107]  Chris Hodkinson,et al.  Understanding Web Information Search Behavior: An Exploratory Model , 2003, J. Organ. End User Comput..

[108]  Francisco Muñoz-Leiva,et al.  Web Acceptance Model (WAM): Moderating effects of user experience , 2007, Inf. Manag..

[109]  Gary Marchionini,et al.  Finding facts vs. browsing knowledge in hypertext systems , 1988, Computer.

[110]  David Miller,et al.  Web search strategies and human individual differences: Cognitive and demographic factors, Internet attitudes, and approaches , 2005, J. Assoc. Inf. Sci. Technol..

[111]  D. Scott Brandt Evaluating information on the Internet , 1996 .

[112]  Amy L. Ostrom,et al.  The Internet as information minefield: An analysis of the source and content of brand information yielded by net searches , 2003 .

[113]  Hangwi Tang,et al.  Googling for a diagnosis—use of Google as a diagnostic aid: internet based study , 2006, BMJ : British Medical Journal.

[114]  Ethelene Whitmire,et al.  The relationship between undergraduates' epistemological beliefs, reflective judgment, and their information-seeking behavior , 2004, Inf. Process. Manag..

[115]  Thorsten Joachims,et al.  In Google We Trust: Users' Decisions on Rank, Position, and Relevance , 2007, J. Comput. Mediat. Commun..

[116]  Matthew E Falagas,et al.  Comparison of PubMed, Scopus, Web of Science, and Google Scholar: strengths and weaknesses , 2007, FASEB journal : official publication of the Federation of American Societies for Experimental Biology.

[117]  Thomas J. Johnson,et al.  Cruising is Believing?: Comparing Internet and Traditional Sources on Media Credibility Measures , 1998 .

[118]  P. Pirolli Information Foraging Theory: Adaptive Interaction with Information , 2007 .

[119]  Glenn J. Browne,et al.  Cognitive Stopping Rules for Terminating Information Search in Online Tasks , 2007, MIS Q..

[120]  A. W. Lazonder Exploring novice users' training needs in searching information on the WWW , 2000, J. Comput. Assist. Learn..