Searchers' relevance judgments and criteria in evaluating web pages in a learning style perspective

The paper presents the results of a case study of searcher's relevance criteria used for assessments of Web pages in a perspective of learning style. 15 test persons participated in the experiments based on two simulated work tasks that provided cover stories to trigger their information needs. Two learning styles were examined: Global and Sequential learners. The study applied eye-tracking for the observation of relevance hot spots on Web pages, learning style index analysis and post-search interviews to gain more in-depth information on relevance behavior. Findings reveal that with respect to use of graded relevance scores and number of relevance criteria applied per task and test person there are no significant difference between the different styles. Although there differences are detected in the use of relevance criteria between Global and Sequential learners during assessments, they are statistically insignificant. When interviewed in retrospective the resulting profiles tend to become even similar across learning styles but a shift occurs from instant assessments with content features of web pages replacing topicality judgments as predominant relevance criteria.

[1]  James D. Klein,et al.  The effects of search tool type and cognitive style on performance during hypermedia database searches , 1996 .

[2]  Carol L. Barry,et al.  Users' Criteria for Relevance Evaluation: A Cross-situational Comparison , 1998, Inf. Process. Manag..

[3]  Samuel Kaski,et al.  Combining eye movements and collaborative filtering for proactive information retrieval , 2005, SIGIR '05.

[4]  Thorsten Joachims,et al.  Accurately Interpreting Clickthrough Data as Implicit Feedback , 2017 .

[5]  J. Linford.  A concept of relevance , 1965 .

[6]  Kalervo Järvelin,et al.  Task complexity affects information seeking and use , 1995 .

[7]  Kyung-Sun Kim,et al.  Cognitive style and on-line database search experience as predictors of Web search performance , 2000, J. Am. Soc. Inf. Sci..

[8]  Karen Valley,et al.  Learning styles and courseware design , 1997 .

[9]  R. Felder,et al.  Applications, Reliability and Validity of the Index of Learning Styles* , 2005 .

[10]  Pia Borlund,et al.  The IIR evaluation model: a framework for evaluation of interactive information retrieval systems , 2003, Inf. Res..

[11]  Pertti Vakkari,et al.  Changes in Search Tactics and Relevance Judgements when Preparing a Research Proposal A Summary of the Findings of a Longitudinal Study , 2001, Information Retrieval.

[12]  Peter Ingwersen,et al.  The Turn - Integration of Information Seeking and Retrieval in Context , 2005, The Kluwer International Series on Information Retrieval.

[13]  Carol L. Barry.  User-defined relevance criteria: an exploratory study , 1994 .

[14]  Samuel Kaski,et al.  Can Relevance be Inferred from Eye Movements in Information Retrieval , 2003 .

[15]  S. Messick.  The nature of cognitive styles: Problems and promise in educational practice , 1984 .

[16]  David Miller,et al.  The role of individual differences in Internet searching: an empirical study , 2001 .

[17]  Ellen M. Voorhees.  Variations in relevance judgments and the measurement of retrieval effectiveness , 2000, Inf. Process. Manag..

[18]  Kyung-Sun Kim,et al.  Cognitive style and on-line database search experience as predictors of Web search performance , 2000, J. Am. Soc. Inf. Sci..

[19]  Michael B. Eisenberg,et al.  A re-examination of relevance: toward a dynamic, situational definition , 1990, Inf. Process. Manag..

[20]  George Ghinea,et al.  The impact of cognitive styles on perceptual distributed multimedia quality , 2003, Br. J. Educ. Technol..

[21]  W. M. Reed,et al.  The relationship between the learning strategies and learning styles in a hypermedia environment , 1994 .

[22]  Enrique Frías-Martínez,et al.  Automatic cognitive style identification of digital library users for personalization , 2007, J. Assoc. Inf. Sci. Technol..

[23]  Peter Ingwersen,et al.  Dimensions of relevance , 2000, Inf. Process. Manag..

[24]  Peter Ingwersen,et al.  The development of a method for the evaluation of interactive information retrieval systems , 1997, J. Documentation.

[25]  Pia Borlund,et al.  Experimental components for the evaluation of interactive information retrieval systems , 2000, J. Documentation.

[26]  Pia Borlund,et al.  The concept of relevance in IR , 2003, J. Assoc. Inf. Sci. Technol..

[27]  David Ellis,et al.  Hypertext and learning styles , 1993 .

[28]  R. Riding,et al.  Cognitive Styles and Learning Strategies , 2013 .

[29]  N. Ford,et al.  Gender differences in Internet perceptions and use , 1996 .

[30]  Mohammad Issack Santally,et al.  A Learning Object Approach to Personalized Web-based Instruction , 2005 .

[31]  Joemon M. Jose,et al.  Searchers' criteria For assessing web pages , 2003, SIGIR '03.