Judgement of information quality and cognitive authority in the Web

In the Web, making judgments of information quality and authority is a difficult task for most users because overall, there is no quality control mechanism. This study examines the problem of the judgment of information quality and cognitive authority by observing people's searching behavior in the Web. Its purpose is to understand the various factors that influence people's judgment of quality and authority in the Web, and the effects of those judgments on selection behaviors. Fifteen scholars from diverse disciplines participated, and data were collected combining verbal protocols during the searches, search logs, and postsearch interviews. It was found that the subjects made two distinct kinds of judgment: predictive judgment, and evaluative judgment. The factors influencing each judgment of quality and authority were identified in terms of characteristics of information objects, characteristics of sources, knowledge, situation, ranking in search output, and general assumption. Implications for Web design that will effectively support people's judgments of quality and authority are also discussed.

[1]  Nicholas J. Belkin,et al.  Interaction with Texts: Information Retrieval as Information-Seeking Behavior , 1993, Information Retrieval.

[2]  Carol Tenopir,et al.  Users' interaction with World Wide Web resources: an exploratory study using a holistic approach , 2000, Inf. Process. Manag..

[3]  Carol L. Barry,et al.  Users' Criteria for Relevance Evaluation: A Cross-situational Comparison , 1998, Inf. Process. Manag..

[4]  Soo Young Rieh,et al.  Interaction on the Web: Scholars' Judgment of Information Quality and Cognitive Authority , 2000 .

[5]  A. Strauss,et al.  Grounded theory methodology: An overview. , 1994 .

[6]  Robert S. Taylor,et al.  Value-Added Processes in Information Systems , 1987 .

[7]  Peiling Wang,et al.  A cognitive model of document use during a research project. Study I. document selection , 1998 .

[8]  T. Park.  The Nature of Relevance in Information Retrieval: An Empirical Study , 1993, The Library Quarterly.

[9]  Peiling Wang,et al.  A Cognitive Model of Document Use During a Research Project. Study II. Decisions at the Reading and Citing Stages , 1999, J. Am. Soc. Inf. Sci..

[10]  N. Roberts,et al.  Second-hand knowledge: An inquiry into cognitive authority: P. Wilson. London: Greenwood Press, 1983. 210 pp. ISBN 0 313 23763 8. £28.95. , 1985 .

[11]  Carol L. Barry.  User-defined relevance criteria: an exploratory study , 1994 .

[12]  Martin J. Eppler.  Managing Information Quality , 2003 .

[13]  O. Holsti.  Content Analysis for the Social Sciences and Humanities , 1969 .

[14]  Jane E. Klobas,et al.  Beyond information quality: fitness for purpose and electronic information resource use , 1995, J. Inf. Sci..

[15]  Alison Cooke,et al.  Neal-Schuman Authoritative Guide to Evaluating Information on the Internet , 1999 .

[16]  Loren G. Terveen,et al.  Does “authority” mean quality? predicting expert quality ratings of Web documents , 2000, SIGIR '00.

[17]  Joseph Janes,et al.  Networked information retrieval and organization: issues and questions , 1996 .

[18]  Nicholas J. Belkin,et al.  Characteristics of Texts Affecting Relevance Judgments , 1993 .

[19]  Amanda Spink,et al.  Regions and levels: Measuring and mapping users' relevance judgments , 2001, J. Assoc. Inf. Sci. Technol..

[20]  Nicholas J. Belkin,et al.  Understanding Judgment of Information Quality and Cognitive Authority in the WWW , 1998 .

[21]  Linda Schamber,et al.  Users' Criteria for Evaluation in a Multimedia Environment. , 1991 .

[22]  Klaus Krippendorff,et al.  Content Analysis: An Introduction to Its Methodology , 1980 .

[23]  Susan Gauch,et al.  Incorporating quality metrics in centralized/distributed information retrieval on the World Wide Web , 2000, SIGIR '00.

[24]  Tefko Saracevic,et al.  The Stratified Model of Information Retrieval Interaction: Extension and Applications , 1997 .

[25]  Jan Alexander,et al.  Teaching critical evaluation skills for World Wide Web resources , 1996 .

[26]  R. Hogarth.  Judgement and choice: The psychology of decision , 1982 .

[27]  N. Roberts,et al.  Value-added processes in information systems , 1986 .

[28]  Amanda Spink,et al.  Real life, real users, and real needs: a study and analysis of user queries on the web , 2000, Inf. Process. Manag..

[29]  Alastair G. Smith,et al.  Testing the Surf: Criteria for Evaluating Internet Information Resources , 1997 .

[30]  H. Rachlin.  Judgment, Decision, and Choice: A Cognitive/Behavioral Synthesis , 1989 .

[31]  Nicholas J. Belkin,et al.  Intelligent Information Retrieval: Whose Intelligence? , 1996, ISI.

[32]  Miyoung Jeong.  Measurement of information quality on lodging websites : an experimental study with eight hypothetical lodging websites , 1999 .

[33]  Jon Kleinberg,et al.  Authoritative sources in a hyperlinked environment , 1999, SODA '98.

[34]  Robert L. Cromwell,et al.  Evaluating Internet resources: Identity, affiliation, and cognitive authority in a networked world , 2001, J. Assoc. Inf. Sci. Technol..

[35]  Paul B. Kantor,et al.  Studying the Value of Library and Information Services. Part I: Establishing a Theoretical Framework. , 1997 .

[36]  Judy Bateman.  Modeling the Importance of End-User Relevance Criteria. , 1999 .

[37]  William R. Hersh,et al.  Filtering Web pages for quality indicators: an empirical approach to finding high quality consumer health information on the World Wide Web , 1999, AMIA.

[38]  Jenny K. Holder,et al.  A visit to the information mall: Web searching behavior of high school students , 1999 .

[39]  N. Denzin,et al.  Handbook of Qualitative Research , 1994 .

[40]  Amanda Cain,et al.  Authoritative Guide to Evaluating Information on the Internet , 2000 .