The role of reading skills in the evaluation of online information gathered from search engine environments

Abstract A critical evaluation of results to find useful information is essential when doing a web search. In this study, we investigated the evaluation skills of secondary school students, based on their behavior in selecting links from a search engine result page (SERP). To clarify the role of reading when evaluating online information, we assessed students' individual reading skills on word, sentence, and text level. Data from 416 15-year-old students participating in a computer based German add-on study to the Programme for International Student Assessment (PISA) in 2012 were investigated. Using generalized linear mixed models (GLMMs), it was found that reading skills affected the ability to evaluate online information. These effects were influenced by the distinctiveness of information in relevance and students' navigation to subsequent SERPs or websites. The results are interpreted to show that skilled readers are able to allocate their cognitive resources more efficiently than less skilled readers when evaluating online information. Implications are discussed in terms of underlying cognitive processes when making web search decisions.

[1]  Miriam J. Metzger.  Making sense of credibility on the Web: Models for evaluating online information and recommendations for future research , 2007, J. Assoc. Inf. Sci. Technol..

[2]  D. Bates,et al.  Fitting Linear Mixed-Effects Models Using lme4 , 2014, 1406.5823.

[3]  Inmaculada Fajardo,et al.  Scanning and deep processing of information in hypertext: an eye tracking and cued retrospective think-aloud study , 2017, J. Comput. Assist. Learn..

[4]  Jeffrey J. Walczyk.  The Interplay Between Automatic and Control Processes in Reading , 2000 .

[5]  Thorsten Joachims,et al.  In Google We Trust: Users' Decisions on Rank, Position, and Relevance , 2007, J. Comput. Mediat. Commun..

[6]  Tobias Richter,et al.  Prozessbezogene Diagnostik von Lesefähigkeiten bei Grundschulkindern , 2012 .

[7]  Soo Young Rieh.  Judgement of information quality and cognitive authority in the Web , 2002 .

[8]  Sascha Schroeder,et al.  What Readers Have and Do: Effects of Students' Verbal Ability and Reading Time Components on Comprehension with and without Text Availability , 2011 .

[9]  G. Masters.  The Partial Credit Model , 2016 .

[10]  L. Salmerón,et al.  Effect of delay on search decisions in a task-oriented reading environment , 2017 .

[11]  Yvonne Kammerer,et al.  Measuring spontaneous and instructed evaluation processes during Web search: Integrating concurrent thinking-aloud protocols and eye-tracking data , 2011 .

[12]  Jérôme Dinet,et al.  The influence of surface and deep cues on primary and secondary school students' assessment of relevance in Web menus , 2011 .

[13]  Peter W. Foltz,et al.  Comprehension, Coherence, and Strategies in Hypertext and Linear Text , 2012 .

[14]  J. Sabatini.  Understanding the Basic Reading Skills of U.S. Adults: Reading Components in the PIAAC Literacy Survey. , 2015 .

[15]  Miriam J. Metzger,et al.  behaviors on the perceived credibility of web-based information The role of site features, user attributes, and information verification , 2007 .

[16]  Johannes Naumann,et al.  Effects of linear reading, basic computer skills, evaluating online information, and navigation on reading digital text , 2016, Comput. Hum. Behav..

[17]  Sriram Raghavan,et al.  Searching the Web , 2001, ACM Trans. Internet Techn..

[18]  Saskia Brand-Gruwel,et al.  Information-problem solving: A review of problems students encounter and instructional solutions , 2008, Comput. Hum. Behav..

[19]  Yvonne Kammerer,et al.  Searching the Web for conflicting topics: Page and user factors , 2013, Comput. Hum. Behav..

[20]  Matthew T. McCrudden,et al.  Relevance and Goal-Focusing in Text Processing , 2007 .

[21]  K. Lawless,et al.  Understanding Research on Navigation in Complex Digital Environments , 2007 .

[22]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[23]  Alexander Robitzsch,et al.  Test Analysis Modules , 2015 .

[24]  Angelo Artale,et al.  A Descriptive Model , 1997 .

[25]  Alexander van Deursen,et al.  Using the Internet: Skill related problems in users' online behavior , 2009, Interact. Comput..

[26]  Byeong-Young Cho,et al.  Competent Adolescent Readers’ Use of Internet Reading Strategies: A Think-Aloud Study , 2014 .

[27]  CHIA-CHING LIN,et al.  A Navigation Flow Map Method of Representing Students' Searching Behaviors and Strategies on the Web, with Relation to Searching Outcomes , 2007, Cyberpsychology Behav. Soc. Netw..

[28]  Saskia Brand-Gruwel,et al.  A descriptive model of information problem solving while using internet , 2009, Comput. Educ..

[29]  Gerd Gigerenzer,et al.  Heuristic decision making. , 2011, Annual review of psychology.

[30]  Marilyn Hughes Blackmon.  Information scent determines attention allocation and link selection among multiple information patches on a webpage , 2012, Behav. Inf. Technol..

[31]  Michael J Cortese,et al.  Visual word recognition of single-syllable words. , 2004, Journal of experimental psychology. General.

[32]  Roger Ratcliff,et al.  A diffusion model account of the lexical decision task. , 2004, Psychological review.

[33]  Saskia Brand-Gruwel,et al.  How students evaluate information and sources when searching the World Wide Web for information , 2009, Comput. Educ..

[34]  Soo Young Rieh,et al.  Developing a unifying framework of credibility assessment: Construct, heuristics, and interaction in context , 2008, Inf. Process. Manag..

[35]  Miriam J. Metzger,et al.  Social and Heuristic Approaches to Credibility Evaluation Online , 2010 .

[36]  José J. Cañas,et al.  Reading Strategies and Hypertext 1 Running head : READING STRATEGIES AND HYPERTEXT COMPREHENSION Reading Strategies and Hypertext Comprehension , 2004 .

[37]  Soo Young Rieh,et al.  Amount of invested mental effort (AIME) in online searching , 2012, Inf. Process. Manag..

[38]  Julie Coiro,et al.  Exploring the Online Reading Comprehension Strategies Used by Sixth-Grade Skilled Readers to Search for and Locate Information on the Internet. , 2007 .

[39]  Jarkko Kari,et al.  User-defined relevance criteria in web searching , 2006, J. Documentation.

[40]  Eduardo Vidal-Abarca,et al.  Selecting information to answer questions: Strategic individual differences when searching texts , 2011 .

[41]  Susan E. Israel,et al.  Identifying and Describing Constructively Responsive Comprehension Strategies in New and Traditional Forms of Reading , 2014 .

[42]  Werner Wirth,et al.  Heuristic and Systematic Use of Search Engines , 2007, J. Comput. Mediat. Commun..

[43]  Andreas Voss,et al.  Fast-dm: A free program for efficient diffusion model analysis , 2007, Behavior research methods.

[44]  Ion Juvina,et al.  Modeling Semantic and Structural Knowledge in Web Navigation , 2008 .

[45]  Saskia Brand-Gruwel,et al.  Solving information-based problems: Evaluating sources and information , 2011 .

[46]  Yvonne Kammerer,et al.  The Role of Search Result Position and Source Trustworthiness in the Selection of Web Search Results When Using a List or a Grid Interface , 2014, Int. J. Hum. Comput. Interact..

[47]  A. Voss,et al.  Interpreting the parameters of the diffusion model: An empirical validation , 2004, Memory & cognition.

[48]  Charles A. Perfetti,et al.  Word Knowledge in a Theory of Reading Comprehension , 2014 .

[49]  Charles A. Perfetti,et al.  Reading Ability: Lexical Quality to Comprehension , 2007 .

[50]  Jean-François Rouet,et al.  The Skills of Document Use: From Text Comprehension to Web-Based Learning , 2006 .

[51]  Jo-Anne LeFevre,et al.  Cognitive load in hypertext reading: A review , 2007, Comput. Hum. Behav..

[52]  Jonathan Evans.  Dual-processing accounts of reasoning, judgment, and social cognition. , 2008, Annual review of psychology.

[53]  Matthijs L. Noordzij,et al.  Topic familiarity and information skills in online credibility evaluation , 2013, J. Assoc. Inf. Sci. Technol..

[54]  Nell K. Duke,et al.  Strategies for Internet Reading with Different Reading Purposes: A Descriptive Study of Twelve Good Internet Readers , 2008 .

[55]  Aristidis Protopsaltis,et al.  Reading Strategies in Hypertexts and Factors Influencing Hyperlink Selection , 2008 .

[56]  S. Jay Samuels,et al.  THE IMPORTANCE OF AUTOMATICITY FOR DEVELOPING EXPERTISE IN READING , 1997 .

[57]  Aristidis Protopsaltis,et al.  Reading Strategies in Hypertexts and Factors Influencing Link Selection , 2006 .

[58]  T. A. Warm.  Weighted likelihood estimation of ability in item response theory , 1989 .

[59]  Soo Young Rieh.  Judgment of information quality and cognitive authority in the Web , 2002, J. Assoc. Inf. Sci. Technol..

[60]  D. Bates,et al.  fitting linear mixed effects models using lme 4 arxiv , 2014 .

[61]  Walter Kintsch,et al.  Comprehension: A Paradigm for Cognition , 1998 .

[62]  Erin M. Freed,et al.  Modeling Reader and Text Interactions During Narrative Comprehension: A Test of the Lexical Quality Hypothesis , 2013, Discourse processes.

[63]  Abe D. Hofman,et al.  The estimation of item response models with the lmer function from the lme4 package in R , 2011 .

[64]  Eduardo Vidal-Abarca,et al.  Individual Differences for Self-Regulating Task-Oriented Reading Activities. , 2010 .