An integrated model of eye movements and visual encoding

Recent computational models of cognition have made good progress in accounting for the visual processes needed to encode external stimuli. However, these models typically incorporate simplified models of visual processing that assume a constant encoding time for all visual objects and do not distinguish between eye movements and shifts of attention. This paper presents a domain-independent computational model, EMMA, that provides a more rigorous account of eye movements and visual encoding and their interaction with a cognitive processor. The visual-encoding component of the model describes the effects of frequency and foveal eccentricity when encoding visual objects as internal representations. The eye-movement component describes the temporal and spatial characteristics of eye movements as they arise from shifts of visual attention. When integrated with a cognitive model, EMMA generates quantitative predictions concerning when and where the eyes move, thus serving to relate higher-level cognitive processes and attention shifts with lower-level eye-movement behavior. The paper evaluates EMMA in three illustrative domains - equation solving, reading, and visual search - and demonstrates how the model accounts for aspects of behavior that simpler models of cognitive and visual processing fail to explain.

[1]  Robert W. Kentridge,et al.  Eye movement research : mechanisms, processes and applications , 1995 .

[2]  W. Becker,et al.  An analysis of the saccadic system by means of double step stimuli , 1979, Vision Research.

[3]  J. Henderson.  Visual Attention and Eye Movement Control During Reading and Picture Viewing , 1992 .

[4]  Brian D. Ehret.  Learning where to look , 1999, CHI EA '99.

[5]  R. Reilly,et al.  Eye movement control during reading: A simulation of some word-targeting strategies , 1998, Vision Research.

[6]  D E Kieras,et al.  A computational theory of executive cognitive processes and multiple-task performance: Part 1. Basic mechanisms. , 1997, Psychological review.

[7]  L. Stark,et al.  Scanpaths in saccadic eye movements while viewing and recognizing patterns. , 1971, Vision research.

[8]  G. McConkie,et al.  The span of the effective stimulus during a fixation in reading , 1975 .

[9]  Allen Newell,et al.  SOAR: An Architecture for General Intelligence , 1987, Artif. Intell..

[10]  Anthony J. Hornof,et al.  Cognitive modeling reveals menu search in both random and systematic , 1997, CHI.

[11]  M. Just,et al.  From the SelectedWorks of Marcel Adam Just 1992 A capacity theory of comprehension : Individual differences in working memory , 2017 .

[12]  John R. Anderson,et al.  Automated Eye-Movement Protocol Analysis , 2001, Hum. Comput. Interact..

[13]  D. E. Irwin,et al.  Lexical Processing during Saccadic Eye Movements , 1998, Cognitive Psychology.

[14]  P. Viviani.  Eye movements in visual search: cognitive, perceptual and motor control aspects. , 1990, Reviews of oculomotor research.

[15]  P Suppes,et al.  Eye-movement models for arithmetic and reading performance. , 1990, Reviews of oculomotor research.

[16]  K. Rayner.  Eye movements in reading and information processing: 20 years of research. , 1998, Psychological bulletin.

[17]  J. O'Regan.  Optimal Viewing Position in Words and the Strategy-Tactics Theory of Eye Movements in Reading , 1992 .

[18]  Patrick Suppes,et al.  A procedural theory of eye movements in doing arithmetic , 1983 .

[19]  John R. Anderson,et al.  Mapping eye movements to cognitive processes , 1999 .

[20]  Erik D. Reichle,et al.  Chapter 11 – Eye Movement Control in Reading: An Overview and Model , 1998 .

[21]  A. Newell.  Unified Theories of Cognition , 1990 .

[22]  Burkhart Fischer,et al.  Saccadic Reaction Time: Implications for Reading, Dyslexia, and Visual Cognition , 1992 .

[23]  J. O'Regan.  Eye movements and reading. , 1990, Reviews of oculomotor research.

[24]  F. C. Volkmann.  Human visual suppression , 1986, Vision Research.

[25]  John R. Anderson,et al.  ACT-R: A Theory of Higher Level Cognition and Its Relation to Visual Attention , 1997, Hum. Comput. Interact..

[26]  M A Just,et al.  A theory of reading: from eye fixations to comprehension. , 1980, Psychological review.

[27]  G. McConkie,et al.  Asymmetry of the perceptual span in reading , 1976 .

[28]  Gerald L. Lohse,et al.  A Cognitive Model for Understanding Graphical Perception , 1993, Hum. Comput. Interact..

[29]  Brian D. Ehret,et al.  Learning where to look: the acquisition of location knowledge in display-based interaction , 1999 .

[30]  D Wyman,et al.  The oculomotor error signal in the fovea. , 1972, Vision research.

[31]  G. Underwood.  Eye guidance in reading and scene perception , 1998 .

[32]  G. Legge,et al.  Mr. Chips: An ideal-observer model of reading , 1997 .

[33]  K. Rayner,et al.  Comparing naming, lexical decision, and eye fixation times: Word frequency effects and individual differences , 1998, Memory & cognition.

[34]  Erik Nilsen,et al.  PERCEPTUAL-MOTOR CONTROL IN HUMAN-COMPUTER INTERACTION , 1991, SGCH.

[35]  K. Rayner.  Eye movements and visual cognition : scene perception and reading , 1992 .

[36]  Eileen Kowler.  Eye movements and their role in visual and cognitive processes. , 1990, Reviews of oculomotor research.

[37]  E. Matin.  Saccadic suppression: a review and an analysis. , 1974, Psychological bulletin.

[38]  Erik D. Reichle,et al.  Toward a model of eye movement control in reading. , 1998, Psychological review.

[39]  Allen Newell,et al.  The psychology of human-computer interaction , 1983 .

[40]  R. Burchfield.  Frequency Analysis of English Usage: Lexicon and Grammar. By W. Nelson Francis and Henry Kučera with the assistance of Andrew W. Mackie. Boston: Houghton Mifflin. 1982. x + 561 , 1985 .

[41]  P. Hayes,et al.  Analogy , 2000, Encyclopedic Dictionary of Archaeology.

[42]  Eileen Kowler.  The role of visual and cognitive processes in the control of eye movement. , 1990, Reviews of oculomotor research.

[43]  Keith Rayner,et al.  Eye movements and identifying words in parafoveal vision , 1981 .

[44]  C. Lebiere,et al.  The Atomic Components of Thought , 1998 .

[45]  Marcel Adam Just,et al.  A Model of the Time Course and Content of Reading , 1982, Cogn. Sci..

[46]  John R. Anderson,et al.  Integrating analogical mapping and general problem solving : the path-mapping theory , 2001 .

[47]  P. Bach-y-Rita,et al.  The Control of Eye Movements , 1971 .

[48]  David E. Kieras,et al.  New methods in reading comprehension research , 1986 .

[49]  John R. Anderson,et al.  Tracing Eye Movement Protocols with Cognitive Process Models , 1998 .

[50]  John E. Laird,et al.  Modeling dual-task performance improvement: Casting executive process knowledge acquisition as strategy refinement , 1998 .

[51]  R. E. Morrison,et al.  Manipulation of stimulus onset delay in reading: evidence for parallel programming of saccades. , 1984, Journal of experimental psychology. Human perception and performance.

[52]  K. Rayner.  Eye Movements and Cognitive Processes in Reading, Visual Search, and Scene Perception , 1995 .

[53]  N. P. Bichot,et al.  Visuospatial attention: Beyond a spotlight model , 1999, Psychonomic bulletin & review.

[54]  D. Henson.  Corrective saccades: Effects of altering visual feedback , 1978, Vision Research.

[55]  Albert F. Fuchs,et al.  THE SACCADIC SYSTEM , 1971 .

[56]  A. Fuchs,et al.  Further properties of the human saccadic system: eye movements and correction saccades with and without visual fixation points. , 1969, Vision research.

[57]  Keith Rayner.  Eye Movements in Reading , 2001 .

[58]  Erik D. Reichle,et al.  Eye movement control in reading: accounting for initial fixation locations and refixations within the E-Z Reader model , 1999, Vision Research.

[59]  Dario D. Salvucci.  A Model of Eye Movements and Visual Attention , 2000 .

[60]  John R. Anderson,et al.  Eye tracking the visual search of click-down menus , 1999, CHI '99.

[61]  K. Rayner.  Eye movements in reading and information processing. , 1978, Psychological bulletin.

[62]  W. Nelson Francis,et al.  FREQUENCY ANALYSIS OF ENGLISH USAGE: LEXICON AND GRAMMAR , 1983 .