Understanding the effects of task and topical knowledge in the evaluation of websites as information patch

Purpose The purpose of this paper is to examine the role of task and user’s topic familiarity in the evaluation of information patch (websites). Design/methodology/approach An experimental study was conducted in a computer laboratory to examine users’ information seeking and foraging behaviour. In total, 160 university students participated in the research. Two types of task instructions, specifically defined and non-specifically defined (general) task types were administered. Mixed methods approach involving both quantitative and qualitative thematic coding were adopted, from the data of the questionnaire surveys and post-experiment interviews. Findings In the context of task attributes, users who conducted information seeking task with specifically defined instructions, as compared to the non-specifically defined instructions, demonstrated stricter credibility evaluations. Evidence demonstrated the link between topical knowledge and credibility perception. Users with topical knowledge applied critical credibility assessments than users without topical knowledge. Furthermore, the evidential results supported that the level of difficulty and knowledge of the topic or subject matter associated with users’ credibility evaluations. Users who have lesser or no subject knowledge and who experienced difficulty in the information search tended to be less diagnostic in their appraisal of the information patch (website or webpages). Users equipped with topical knowledge and who encountered less difficulty in the search, exhibited higher expectation and evaluative criteria of the information patch. Research limitations/implications The constraints of time in the lab experiment, carried out in the presence of and under the observation of the researcher, may affect users’ information seeking behaviour. It would be beneficial to consider users’ information search gratifications and motivations in studying information evaluations and foraging patterns. There is scope to investigate users’ proficiency such as expert or novice, and individual learning styles in assessing information credibility. Practical implications Past studies on information evaluation, specifically credibility is often associated with users’ characteristics, source, or contents. This study sheds light on the context of task type, task difficulty and topical knowledge in affecting users’ information judgement. Originality/value One of the scarce studies in relating task orientation, task difficulty and topical knowledge to information evaluations.

[1]  Preben Hansen,et al.  Conceptual Framework for Task in Information Studies. , 2005 .

[2]  Soo Young Rieh,et al.  Credibility: A multidisciplinary framework , 2007, Annu. Rev. Inf. Sci. Technol..

[3]  Preben Hansen,et al.  Conceptual framework for tasks in information studies , 2005, J. Assoc. Inf. Sci. Technol..

[4]  Paul B. Kantor,et al.  A study of information seeking and retrieving. II. Users, questions, and effectiveness , 1988, J. Am. Soc. Inf. Sci..

[5]  Carol Collier Kuhlthau Inside the Search Process: Information Seeking from the User's Perspective. , 1991 .

[6]  Michael D. Byrne,et al.  Effects of scent and breadth on use of site-specific search on e-commerce Web sites , 2003, TCHI.

[7]  Clifford A. Lynch,et al.  When documents deceive: Trust and provenance as new factors for information retrieval in a tangled web , 2001, J. Assoc. Inf. Sci. Technol..

[8]  Bryce Allen,et al.  Cognitive and task influences on Web searching behavior , 2002, J. Assoc. Inf. Sci. Technol..

[9]  A. A. Lumsdaine,et al.  Experiments On Mass Communication , 1949 .

[10]  Christopher Olston,et al.  ScentTrails: Integrating browsing and searching on the Web , 2003, TCHI.

[11]  Bryce Allen,et al.  Topic Knowledge and Online Catalog Search Formulation , 1991, The Library Quarterly.

[12]  Rathindra Sarathy,et al.  An experimental investigation of the influence of website emotional design features on trust in unfamiliar online vendors , 2017, Comput. Hum. Behav..

[13]  Terry Ryan,et al.  The Credibility of Online Information , 2003, AMCIS.

[14]  Christine L. Borgman,et al.  The user's mental model of an information retrieval system , 1999, SIGIR '85.

[15]  Pertti Vakkari,et al.  Task-based information searching , 2005, Annu. Rev. Inf. Sci. Technol..

[16]  S. Shyam Sundar,et al.  News cues: Information scent and cognitive heuristics , 2007, J. Assoc. Inf. Sci. Technol..

[17]  B. J. Fogg,et al.  The elements of computer credibility , 1999, CHI '99.

[18]  Bryce Allen,et al.  Information needs: a person-in-situation approach , 1997 .

[19]  Janice Redish,et al.  User and task analysis for interface design , 1998 .

[20]  Kalervo Järvelin,et al.  Information interaction in molecular medicine: integrated use of multiple channels , 2010, IIiX.

[21]  Iwan G. J. H. Wopereis,et al.  Differences between novice and experienced users in searching information on the World Wide Web , 2000, J. Am. Soc. Inf. Sci..

[22]  Joseph S. Dumas,et al.  Comparison of three one-question, post-task usability questionnaires , 2009, CHI.

[23]  J. Nunnally,et al.  Psychometric Theory: NY. , 1978 .

[24]  James D. Ivory,et al.  Enhanced Information Scent, Selective Discounting, or Consummate Breakdown: The Psychological Effects of Web-Based Search Results , 2009 .

[25]  A. Huberman,et al.  Qualitative Data Analysis: A Methods Sourcebook , 1994 .

[26]  M. Slater,et al.  Value-Affirmative and Value-Protective Processing of Alcohol Education Messages That Include Statistical Evidence or Anecdotes , 1996 .

[27]  Charles L. Viles,et al.  User effort in query construction and interface selection , 2000, DL '00.

[28]  Jacquelyn A. Burkell,et al.  Believe it or not: Factors influencing credibility on the Web , 2002, J. Assoc. Inf. Sci. Technol..

[29]  J. E. Russo,et al.  An Eye-Fixation Analysis of Choice Processes for Consumer Nondurables , 1994 .

[30]  Jonathan Kies,et al.  User and task analysis for interface design , 1998 .

[31]  Craig Locatis,et al.  Effects of link annotations on search performance in layered and unlayered hierarchically organized information spaces , 2001, J. Assoc. Inf. Sci. Technol..

[32]  Ziming Liu,et al.  Perceptions of credibility of scholarly information on the web , 2004, Inf. Process. Manag..

[33]  Diane Kelly,et al.  The effects of topic familiarity on information search behavior , 2002, JCDL '02.

[34]  D. Campbell Task Complexity: A Review and Analysis , 1988 .

[35]  Jonathan Wareham,et al.  Doctors' online information needs, cognitive search strategies, and judgments of information quality and cognitive authority: How predictive judgments introduce bias into cognitive search models , 2010 .

[36]  Sanna Kumpulainen,et al.  Task-based information access in molecular medicine: task performance, barriers, and searching within a heterogeneous information environment , 2013 .

[37]  P. Pirolli Information Foraging Theory: Adaptive Interaction with Information , 2007 .

[38]  R. Kolbe,et al.  Content-Analysis Research: An Examination of Applications with Directives for Improving Research Reliability and Objectivity , 1991 .

[39]  Thomas D. Wilson,et al.  Human Information Behavior , 2000, Informing Sci. Int. J. an Emerg. Transdiscipl..

[40]  Hanhwe Kim,et al.  Spatial metaphors and disorientation in hypertext browsing , 1995, Behav. Inf. Technol..

[41]  Nicholas J. Belkin,et al.  Examining the effects of task topic familiarity on searchers' behaviors in different task types , 2013, ASIST.

[42]  Soohyung Joo,et al.  Factors affecting the selection of search tactics: Tasks, knowledge, process, and systems , 2012, Inf. Process. Manag..

[43]  R. Abelson Script processing in attitude formation and decision making. , 1976 .

[44]  Pertti Vakkari,et al.  Subject knowledge improves interactive query expansion assisted by a thesaurus , 2004, J. Documentation.

[45]  Soo Young Rieh Judgement of information quality and cognitive authority in the Web , 2002 .

[46]  M. S. Saleem Basha,et al.  Advancement in Information Foraging Theory , 2012 .

[47]  E. Stotland,et al.  Social psychology : a cognitive approach , 1972 .

[48]  Malcolm Otter,et al.  Lost in hyperspace: metrics and mental models , 2000, Interact. Comput..

[49]  Marilyn Hughes Blackmon Information scent determines attention allocation and link selection among multiple information patches on a webpage , 2012, Behav. Inf. Technol..

[50]  G Gigerenzer,et al.  Reasoning the fast and frugal way: models of bounded rationality. , 1996, Psychological review.

[51]  M. McHugh Interrater reliability: the kappa statistic , 2012, Biochemia medica.

[52]  Nicholas J. Belkin,et al.  A faceted approach to conceptualizing tasks in information seeking , 2008, Inf. Process. Manag..

[53]  W. Newton Suter,et al.  Introduction to Educational Research: A Critical Thinking Approach , 2005 .

[54]  Dan Albertson Analyzing user interaction with the ViewFinder video retrieval system , 2010 .

[55]  Diane Kelly Measuring online information seeking context, Part 1: Background and method , 2006 .

[56]  A. W. Lazonder Exploring novice users' training needs in searching information on the WWW , 2000, J. Comput. Assist. Learn..

[57]  Yvonne Rogers,et al.  Cognitive strategies in web searching. , 1999 .

[58]  Yan Zhang The influence of mental models on undergraduate students' searching behavior on the Web , 2008, Inf. Process. Manag..

[59]  Andrew Thatcher,et al.  Web search strategies: The influence of Web experience and task type , 2008, Inf. Process. Manag..

[60]  B. Padmanabhan,et al.  Goal attainment on long tail web sites: An information foraging approach , 2013, Decis. Support Syst..

[61]  J. Nunnally Psychometric Theory (2nd ed), New York: McGraw-Hill. , 1978 .

[62]  Carol Collier Kuhlthau,et al.  A Principle of Uncertainty for Information seeking , 1993, J. Documentation.

[63]  Jingjing Liu,et al.  Why do users feel search task difficult? , 2013, ASIST.

[64]  Nils Pharo,et al.  The SST Method Schema: a Tool for Analysing Work Task-Based Web Information Search Processes , 2002 .

[65]  Elaine Toms,et al.  Untangling search task complexity and difficulty in the context of interactive information retrieval studies , 2014, J. Documentation.

[66]  Edward Cutrell,et al.  What are you looking for?: an eye-tracking study of information usage in web search , 2007, CHI.

[67]  Thomas P. Moran,et al.  Mental models and problem solving in using a calculator , 1983, CHI '83.

[68]  Madjid Ihadjadene,et al.  The effect of individual differences on searching the web , 2003, ASIST.

[69]  Christine L. Borgman The user's mental model of an information retrieval system: an experiment on a prototype online catalog , 1999, Int. J. Hum. Comput. Stud..

[70]  Diane Kelly Measuring online information seeking context, Part 2: Findings and discussion , 2006 .

[71]  Ali Shiri,et al.  The effects of topic complexity and familiarity on cognitive and physical moves in a thesaurus-enhanced search environment , 2003, J. Inf. Sci..

[72]  Alison A. Plessinger,et al.  Exploring Receivers' Criteria for Perception of Print and Online News , 1999 .

[73]  Ali Shiri Topic familiarity and its effects on term selection and browsing in a thesaurus-enhanced search environment , 2005 .

[74]  Reijo Savolainen,et al.  Elaborating the motivational attributes of information need and uncertainty , 2012, Inf. Res..

[75]  Kakuzo Iwamura,et al.  λ credibility , 2009 .

[76]  Jacek Gwizdka,et al.  Predicting task difficulty for different task types , 2010, ASIST.

[77]  Pertti Vakkari,et al.  Students' cognition and information searching while preparing a research proposal , 2002 .

[78]  Kimberly A. Neuendorf,et al.  The Content Analysis Guidebook , 2001 .

[79]  Charles L. Bernier,et al.  Second-hand knowledge. An inquiry into cognitive authority. Patrick Wilson. Greenwood Press, 1983 , 1984, J. Am. Soc. Inf. Sci..

[80]  Nicole C. Krämer,et al.  Selecting Science Information in Web 2.0: How Source Cues, Message Sidedness, and Need for Cognition Influence Users' Exposure to Blog Posts , 2012, J. Comput. Mediat. Commun..

[81]  Aline Chevalier,et al.  Query strategies during information searching: Effects of prior domain knowledge and complexity of the information problems to be solved , 2015, Inf. Process. Manag..

[82]  Cheryl Campanella Bracken,et al.  Practical Resources for Assessing and Reporting Intercoder Reliability in Content Analysis Research Projects , 2005 .

[83]  Soohyung Joo,et al.  Effects of topic familiarity and search skills on query reformulation behavior , 2013, ASIST.

[84]  Soo Young Rieh,et al.  Developing a unifying framework of credibility assessment: Construct, heuristics, and interaction in context , 2008, Inf. Process. Manag..

[85]  James E. Pitkow,et al.  Characterizing Browsing Strategies in the World-Wide Web , 1995, Comput. Networks ISDN Syst..

[86]  Barbara M. Wildemuth,et al.  The effects of domain knowledge on search tactic formulation , 2004, J. Assoc. Inf. Sci. Technol..

[87]  Tom Gedeon,et al.  Risk and Ambiguity in Information , 2016 .

[88]  Craig Locatis,et al.  Searching Through Cyberspace: The Effects of Link Cues and Correspondence on Information Retrieval from Hypertext on the World Wide Web , 1998, J. Am. Soc. Inf. Sci..

[89]  Johnny Saldaña,et al.  The Coding Manual for Qualitative Researchers , 2009 .

[90]  Robert W. Reeder,et al.  Information scent as a driver of Web behavior graphs: results of a protocol analysis method for Web usability , 2001, CHI.

[91]  D. R. Danielson,et al.  How do users evaluate the credibility of Web sites?: a study with over 2,500 participants , 2003, DUX '03.

[92]  Peter Pirolli,et al.  Information Foraging , 2009, Encyclopedia of Database Systems.

[93]  Matthijs L. Noordzij,et al.  Topic familiarity and information skills in online credibility evaluation , 2013, J. Assoc. Inf. Sci. Technol..

[94]  Zhizhong Li,et al.  Task complexity: A review and conceptualization framework , 2012 .

[95]  David C. Yen,et al.  The interaction effects of familiarity, breadth and media usage on web browsing experience , 2011, Comput. Hum. Behav..

[96]  Shan-Ju L. Chang,et al.  Taking account of User Tasks, Goals and Behavior for the Design of Online Public Access Catalogs , 1990 .

[97]  Wai-Tat Fu,et al.  SNIF-ACT: A Cognitive Model of User Navigation on the World Wide Web , 2007, Hum. Comput. Interact..

[98]  Kalervo Järvelin,et al.  Task complexity affects information seeking and use , 1995 .

[99]  Miriam J. Metzger Making sense of credibility on the Web: Models for evaluating online information and recommendations for future research , 2007 .

[100]  Jacob Cohen A Coefficient of Agreement for Nominal Scales , 1960 .

[101]  Elaine Toms,et al.  How is a search system used in work task completion? , 2013, J. Inf. Sci..

[102]  Preben Hansen User Interface Design for IR Interaction: A Task-Oriented Approach , 1999, CoLIS.

[103]  Jarkko Kari,et al.  Conceptions of the Internet in Everyday Life Information Seeking , 2004, J. Inf. Sci..

[104]  B MilesMatthew,et al.  Qualitative Data Analysis , 2018, Approaches and Processes of Social Science Research.

[105]  Pertti Vakkari,et al.  Changes in relevance criteria and problem stages in task performance , 2000, J. Documentation.

[106]  E. A. Locke,et al.  Goal setting and task performance: 1969–1980. , 1981 .

[107]  Ion Juvina,et al.  Individual Differences and Behavioral Aspects Involved in Modeling Web Navigation , 2004, User Interfaces for All.

[108]  Stella D. Tomasi,et al.  Using cues to forage for information on the Web , 2014, J. Syst. Inf. Technol..

[109]  Pertti Vakkari,et al.  Subject Knowledge, Source of Terms, and Term Selection in Query Expansion: An Analytical Study , 2002, ECIR.