Semantic models and corpora choice when using Semantic Fields to predict eye movement on web pages

Abstract Ten models are compared in their ability to predict eye-tracking data that was collected from 49 participants' goal-oriented search tasks on a total of 1809 Web pages. Forming the basis of six of these models, three semantic models and two corpus types are compared as components for the Semantic Fields model ( Stone and Dennis, 2007 ) that estimates the semantic salience of different areas displayed on Web pages. Latent Semantic Analysis, Sparse Nonnegative Matrix Factorization, and Vectorspace were used to generate similarity comparisons of goal and Web page text in the semantic component of the Semantic Fields model. Overall, Vectorspace was the best performing semantic model in this study. Two types of corpora or knowledge-bases were used to inform the semantic models, the well known TASA corpus and other corpora that were constructed from the Wikipedia encyclopedia. In all cases the Wikipedia corpora outperformed the TASA corpora. A non-corpus-based Semantic Fields model that incorporated word overlap performed more poorly at these tasks. Three baseline models were also included as a point of comparison to evaluate the effectiveness of the Semantic Fields models. In all cases the corpus-based Semantic Fields models outperformed the baseline models when predicting the participants' eye-tracking data. Both final destination pages and pupil data (dilation) indicated that participants' were actively performing goal-oriented search tasks.

[1]  J. A. Klein,et al.  Pupillary responses during mental activities , 1968 .

[2]  Pete Faraday,et al.  Visually Critiquing Web Pages , 1999, Eurographics Multimedia Workshop.

[3]  Paul van Schaik,et al.  The effect of text and background colour on visual search of Web pages , 2002 .

[4]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[5]  Wai-Tat Fu,et al.  SNIF-ACT: A Cognitive Model of User Navigation on the World Wide Web , 2007, Hum. Comput. Interact..

[6]  J. Giles.  Internet encyclopaedias go head to head , 2005, Nature.

[7]  Jens Riegelsberger,et al.  Could I have the Menu Please? An Eye Tracking Study of Design Conventions , 2004 .

[8]  Bing Pan,et al.  The determinants of web page viewing behavior: an eye-tracking study , 2004, ETRA.

[9]  Michael D. Lee,et al.  Pupil Size and Mental Load , 2004 .

[10]  Tamir Hazan,et al.  Non-negative tensor factorization with applications to statistics and computer vision , 2005, ICML.

[11]  Carolyn Snyder,et al.  Web sites that work: designing with your eyes open , 1998, CHI EA '99.

[12]  Gerard Salton,et al.  A vector space model for automatic indexing , 1975, CACM.

[13]  Curt Burgess,et al.  Modelling Parsing Constraints with High-dimensional Context Space , 1997 .

[14]  Peter Pirolli,et al.  Modeling Information Scent: A Comparison of LSA, PMI and GLSA Similarity Measures on Common Tests and Corpora , 2007, RIAO.

[15]  Muneo Kitajima,et al.  The influence of web browsing experience on web-viewing behavior , 2006, ETRA '06.

[16]  Craig S. Miller,et al.  Modeling Information Navigation: Implications for Information Architecture , 2004, Hum. Comput. Interact..

[17]  Peter W. Foltz,et al.  The intelligent essay assessor: Applications to educational technology , 1999 .

[18]  Muneo Kitajima,et al.  Comparison of eye movements in searching for easy-to-find and hard-to-find information in a hierarchically organized information structure , 2008, ETRA '08.

[19]  Paul van Schaik,et al.  The effect of spatial layout of and link colour in web pages on performance in a visual search task and an interactive search task , 2003, Int. J. Hum. Comput. Stud..

[20]  Philippe A. Palanque,et al.  People and Computers XVII — Designing for Society , 2004, Springer London.

[21]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[22]  Gilbert Cockton,et al.  People and Computers XIV — Usability or Else! , 2000, Springer London.

[23]  Maria Klara Wolters,et al.  Leveraging large data sets for user requirements analysis , 2011, ASSETS.

[24]  Arthur C. Graesser,et al.  Using Latent Semantic Analysis to Evaluate the Contributions of Students in AutoTutor , 2000, Interact. Learn. Environ..

[25]  Simon Dennis,et al.  Using LSA Semantic Fields to Predict Eye Movement on Web Pages , 2007 .

[26]  Xin Liu,et al.  Document clustering based on non-negative matrix factorization , 2003, SIGIR.

[27]  Peter Faraday.  Attending to web pages , 2001, CHI Extended Abstracts.

[28]  Philip Resnik,et al.  Semantic Similarity in a Taxonomy: An Information-Based Measure and its Application to Problems of Ambiguity in Natural Language , 1999, J. Artif. Intell. Res..

[29]  Marilyn Hughes Blackmon,et al.  Tool for accurately predicting website navigation problems, non-problems, problem severity, and effectiveness of repairs , 2005, CHI.

[30]  Rebecca Grier,et al.  VISUAL ATTENTION AND WEB DESIGN , 2004 .

[31]  Marilyn Hughes Blackmon,et al.  Repairing usability problems identified by the cognitive walkthrough for the web , 2003, CHI '03.

[32]  Peter Pirolli,et al.  Computational models of information scent-following in a very large browsable text collection , 1997, CHI.

[33]  Duncan P. Brumby,et al.  Interdependence and Past Experience in Menu Choice Assessment , 2003 .

[34]  Ed H. Chi,et al.  Using information scent to model user information needs and actions and the Web , 2001, CHI.

[35]  Thomas L. Griffiths,et al.  A probabilistic approach to semantic representation , 2019, Proceedings of the Twenty-Fourth Annual Conference of the Cognitive Science Society.

[36]  K. Rayner.  Eye movements in reading and information processing: 20 years of research. , 1998, Psychological bulletin.

[37]  Peter J. Kwantes,et al.  Comparing Methods for Single Paragraph Similarity Analysis , 2011, Top. Cogn. Sci..

[38]  J. Beatty,et al.  Pupillary responses during information processing vary with Scholastic Aptitude Test scores. , 1979, Science.

[39]  Evgeniy Gabrilovich,et al.  Computing Semantic Relatedness Using Wikipedia-based Explicit Semantic Analysis , 2007, IJCAI.

[40]  Johanna D. Moore,et al.  Proceedings of the 28th Annual Conference of the Cognitive Science Society , 2005 .

[41]  Eamonn O'Neill,et al.  People and computers XVII - design for society: proceedings of HCI 2003 , 2003 .

[42]  Jonathan Ling,et al.  The effects of link format and screen location on visual search of web pages , 2004, Ergonomics.

[43]  Minoru Nakayama,et al.  The act of task difficulty and eye-movement frequency for the 'Oculo-motor indices' , 2002, ETRA.

[44]  Richard M. Young,et al.  A Rational Model of the Effect of Information Scent on the Exploration of Menus , 2004, ICCM.

[45]  Arthur C. Graesser,et al.  NLS: A Non-Latent Similarity Algorithm , 2004 .

[46]  Shu-Chieh Wu,et al.  Preliminary evidence for top-down and bottom-up processes in web search navigation , 2007, CHI Extended Abstracts.

[47]  Anthony J. Hornof,et al.  A comparison of LSA, wordNet and PMI-IR for predicting user click behavior , 2005, CHI.

[48]  Marilyn Tremaine.  CHI '01 Extended Abstracts on Human Factors in Computing Systems , 2001, CHI Extended Abstracts.

[49]  Andrew Howes,et al.  Good Enough But I'll Just Check: Web-page Search as Attentional Refocusing , 2004, ICCM.

[50]  Marilyn Hughes Blackmon,et al.  Cognitive walkthrough for the web , 2002, CHI.

[51]  F. Boersma,et al.  Effects of arithmetic problem difficulty on pupillary dilation in normals and educable retardates. , 1970, Journal of experimental child psychology.

[52]  Wai-Tat Fu,et al.  SNIF-ACT: A Model of Information Foraging on the World Wide Web , 2003, User Modeling.

[53]  Miki Namatame,et al.  Suitable representations of hyperlinks for deaf persons: an eye-tracking study , 2008, Assets '08.

[54]  S. Steinhauer,et al.  Cognitive modulation of midbrain function: task-induced reduction of the pupillary light reflex. , 2000, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[55]  J. Cacioppo,et al.  Handbook Of Psychophysiology , 2019 .

[56]  E. Hess,et al.  Pupil Size in Relation to Mental Activity during Simple Problem-Solving , 1964, Science.

[57]  K. Dieussaert,et al.  Proceedings of the 26th annual conference of the cognitive science society , 2004 .

[58]  D. Kirsh,et al.  Proceedings of the 25th annual conference of the Cognitive Science Society , 2003 .

[59]  Niels Taatgen,et al.  Proceedings of the 12th International Conference on Cognitive Modeling , 2004, ICCM 2013.

[60]  Walter Gerbino,et al.  Navigating Within a Web Site: the WebStep Model , 2004, ICCM.

[61]  George A. Miller,et al.  WordNet: A Lexical Database for English , 1995, HLT.

[62]  Julie Chen,et al.  The bloodhound project: automating discovery of web usability issues using the InfoScentπ simulator , 2003, CHI '03.

[63]  Danielle S. McNamara,et al.  Handbook of latent semantic analysis , 2007 .

[64]  P. Kwantes.  Using context to build semantics , 2005, Psychonomic bulletin & review.

[65]  Paul M. Fitts,et al.  Eye movements of aircraft pilots during instrument-landing approaches. , 1950 .

[66]  Susan T. Dumais,et al.  Improving the retrieval of information from external sources , 1991 .

[67]  van SchaikPaul,et al.  The effect of spatial layout of and link colour in web pages on performance in a visual search task and an interactive search task , 2003 .

[68]  Andrew T. Duchowski,et al.  Proceedings of the 2006 symposium on Eye tracking research & applications , 2000 .

[69]  Simon Dennis,et al.  How to Use the LSA Web Site , 2007 .