The effects of credibility cues on the selection of search engine results

Web search engines act as gatekeepers when people search for information online. Research has shown that search engine users seem to trust the search engines' ranking uncritically and mostly select top‐ranked results. This study further examines search engine users' selection behavior. Drawing from the credibility and information research literature, we test whether the presence or absence of certain credibility cues influences the selection probability of search engine results. In an observational study, participants (N = 247) completed two information research tasks on preset search engine results pages, on which three credibility cues (source reputation, message neutrality, and social recommendations) as well as the search result ranking were systematically varied. The results of our study confirm the significance of the ranking. Of the three credibility cues, only reputation had an additional effect on selection probabilities. Personal characteristics (prior knowledge about the researched issues, search engine usage patterns, etc.) did not influence the preference for search results linked with certain credibility cues. These findings are discussed in light of situational and contextual characteristics (e.g., involvement, low‐cost scenarios).

[1]  Kari-Jouko Räihä,et al.  All the News That's Fit to Read: Finding and Recommending News Online , 2011, INTERACT.

[2]  Susan T. Dumais,et al.  Learning user interaction models for predicting web search result preferences , 2006, SIGIR.

[3]  Esther Thorson,et al.  The Journalist Behind the News: Credibility of Straight, Collaborative, Opinionated, and Blogged “News” , 2010 .

[4]  Jennifer Greer.  Evaluating the Credibility of Online Information: A Test of Source and Advertising Influence , 2003 .

[5]  A Sunmade,et al.  Elaboration likelihood model , 2008 .

[6]  Lindsay Fullerton,et al.  Trust Online: Young Adults' Evaluation of Web Content , 2010 .

[7]  C. Gaziano,et al.  Measuring the Concept of Credibility , 1986 .

[8]  Katsumi Tanaka,et al.  Enhancing credibility judgment of web search results , 2011, CHI.

[9]  Nicole C. Krämer,et al.  Selecting Science Information in Web 2.0: How Source Cues, Message Sidedness, and Need for Cognition Influence Users' Exposure to Blog Posts , 2012, J. Comput. Mediat. Commun..

[10]  Dirk Lewandowski.  Credibility in Web Search Engines , 2012, ArXiv.

[11]  Peter Pirolli,et al.  Information Foraging , 2009, Encyclopedia of Database Systems.

[12]  Silvia Knobloch-Westerwick,et al.  News cues: Information scent and cognitive heuristics: Research Articles , 2007 .

[13]  Miriam J. Metzger,et al.  Credibility for the 21st Century: Integrating Perspectives on Source, Message, and Media Credibility in the Contemporary Media Environment , 2003 .

[14]  B. J. Fogg,et al.  Credibility and computing technology , 1999, CACM.

[15]  Werner Wirth,et al.  Navigating the Internet , 2004 .

[16]  Miriam J. Metzger,et al.  Social and Heuristic Approaches to Credibility Evaluation Online , 2010 .

[17]  Mike Schmierbach,et al.  A Little Bird Told Me, So I Didn't Believe It: Twitter, Credibility, and Issue Perceptions , 2012 .

[18]  S. Shyam Sundar,et al.  News cues: Information scent and cognitive heuristics , 2007, J. Assoc. Inf. Sci. Technol..

[19]  Peter Pirolli,et al.  Rational Analyses of Information Foraging on the Web , 2005, Cogn. Sci..

[20]  Qian Xu,et al.  Social Recommendation, Source Credibility, and Recency , 2013 .

[21]  Ling Xia,et al.  Eye tracking and online search: Lessons learned and challenges ahead , 2008, J. Assoc. Inf. Sci. Technol..

[22]  S. Shyam Sundar,et al.  Effects of Online Health Sources on Credibility and Behavioral Intentions , 2010, Commun. Res..

[23]  Benjamin Edelman,et al.  Adverse selection in online "trust" certifications and search results , 2011, Electron. Commer. Res. Appl..

[24]  Ryen W. White,et al.  Characterizing the influence of domain expertise on web search behavior , 2009, WSDM '09.

[25]  Miriam J. Metzger,et al.  behaviors on the perceived credibility of web-based information The role of site features, user attributes, and information verification , 2007 .

[26]  B. J. Fogg,et al.  What makes Web sites credible?: a report on a large quantitative study , 2001, CHI.

[27]  Jacquelyn A. Burkell,et al.  Believe it or not: Factors influencing credibility on the Web , 2002, J. Assoc. Inf. Sci. Technol..

[28]  Mohan J. Dutta-Bergman.  The Impact of Completeness and Web Use Motivation on the Credibility of e‐Health Information , 2004 .

[29]  Wei Zhang,et al.  Knowledge-Based Trust: Estimating the Trustworthiness of Web Sources , 2015, Proc. VLDB Endow..

[30]  Ofer Arazy,et al.  On the measurability of information quality , 2011, J. Assoc. Inf. Sci. Technol..

[31]  Satoshi Nakamura,et al.  Trustworthiness Analysis of Web Search Results , 2007, ECDL.

[32]  Amanda Spink,et al.  How are we searching the World Wide Web? A comparison of nine search engine transaction logs , 2006, Inf. Process. Manag..

[33]  Andrei Broder,et al.  A taxonomy of web search , 2002, SIGF.

[34]  Soo Young Rieh.  Judgement of information quality and cognitive authority in the Web , 2002 .

[35]  Thomas J. Johnson,et al.  Still Cruising and Believing? An Analysis of Online Credibility Across Three Presidential Campaigns , 2010 .

[36]  Miriam J. Metzger.  Making sense of credibility on the Web: Models for evaluating online information and recommendations for future research , 2007, J. Assoc. Inf. Sci. Technol..

[37]  Dirk Lewandowski,et al.  What Users See - Structures in Search Engine Results Pages , 2009, Inf. Sci..

[38]  Meredith Ringel Morris,et al.  Augmenting web pages and search results to support credibility assessment , 2011, CHI.

[39]  Werner Wirth,et al.  Selektion und Rezeption im WWW: Eine Typologie , 1999 .

[40]  Soo Young Rieh,et al.  Credibility: A multidisciplinary framework , 2007, Annu. Rev. Inf. Sci. Technol..

[41]  Wonchan Choi,et al.  Web credibility assessment: Conceptualization, operationalization, variability, and models , 2015, J. Assoc. Inf. Sci. Technol..

[42]  Werner Wirth,et al.  Success in online searches: Differences between evaluation and finding tasks , 2016, J. Assoc. Inf. Sci. Technol..

[43]  Thomas J. Johnson,et al.  Wag the Blog: How Reliance on Traditional Media and the Internet Influence Credibility Perceptions of Weblogs Among Blog Users , 2004 .

[44]  Soo Young Rieh.  Judgment of information quality and cognitive authority in the Web , 2002, J. Assoc. Inf. Sci. Technol..

[45]  Ian Ruthven,et al.  An eye-tracking approach to the analysis of relevance judgments on the Web: The case of Google search engine , 2012, J. Assoc. Inf. Sci. Technol..

[46]  Monic Sun,et al.  How Does the Variance of Product Ratings Matter? , 2010, Manag. Sci..

[47]  H. Kelley,et al.  Communication and Persuasion: Psychological Studies of Opinion Change , 1982 .

[48]  Kirsten A. Johnson,et al.  Enhancing Perceived Credibility of Citizen Journalism Web Sites , 2009 .

[49]  Ingoo Han,et al.  The Effect of On-Line Consumer Reviews on Consumer Purchasing Intention: The Moderating Role of Involvement , 2007, Int. J. Electron. Commer..

[50]  J. Mccroskey,et al.  Goodwill: A reexamination of the construct and its measurement , 1999 .

[51]  Kakuzo Iwamura,et al.  λ credibility , 2009 .

[52]  Thorsten Joachims,et al.  In Google We Trust: Users' Decisions on Rank, Position, and Relevance , 2007, J. Comput. Mediat. Commun..

[53]  Axel Westerwick,et al.  Effects of Sponsorship, Web Site Design, and Google Ranking on the Credibility of Online Information , 2013, J. Comput. Mediat. Commun..

[54]  Spiro K. Kiousis,et al.  Public Trust or Mistrust? Perceptions of Media Credibility in the Information Age , 2001 .

[55]  Ryen W. White,et al.  Large-scale analysis of individual and task differences in search result page examination strategies , 2012, WSDM '12.

[56]  Dirk Lewandowski,et al.  Challenges for Search Engine Retrieval Effectiveness Evaluations: Universal Search, User Intents, and Results Presentation , 2013, Quality Issues in the Management of Web Information.

[57]  Michael Stefanone,et al.  Exploring Online News Credibility: The Relative Influence of Traditional and Technological Factors , 2012, J. Comput. Mediat. Commun..

[58]  Marcel Machill,et al.  Search-engine research: a European-American overview and systematization of an interdisciplinary and international research field , 2008 .