Visual memory and motor planning in a natural task.

This paper investigates the temporal dependencies of natural vision by measuring eye and hand movements while subjects made a sandwich. The phenomenon of change blindness suggests these temporal dependencies might be limited. Our observations are largely consistent with this, suggesting that much natural vision can be accomplished with "just-in-time" representations. However, we also observe several aspects of performance that point to the need for some representation of the spatial structure of the scene that is built up over different fixations. Patterns of eye-hand coordination and fixation sequences suggest the need for planning and coordinating movements over a period of a few seconds. This planning must be in a coordinate frame that is independent of eye position, and thus requires a representation of the spatial structure in a scene that is built up over different fixations.

[1]  A. Fuchs,et al.  Further properties of the human saccadic system: eye movements and correction saccades with and without visual fixation points. , 1969, Vision research.

[2]  W. Becker,et al.  An analysis of the saccadic system by means of double step stimuli , 1979, Vision Research.

[3]  Joel M. Miller,et al.  Information used by the perceptual and oculomotor systems regarding the amplitude of saccadic and pursuit eye movements , 1980, Vision Research.

[4]  O'Regan Jk,et al.  Integrating visual information from successive fixations: does trans-saccadic fusion exist? , 1983 .

[5]  J. O'Regan,et al.  Integrating visual information from successive fixations:Does trans-saccadic fusion exist? , 1983, Vision Research.

[6]  K. Rayner,et al.  Is visual information integrated across saccades? , 1983, Perception & psychophysics.

[7]  S. Ullman.  Visual routines , 1984, Cognition.

[8]  Julian Hochberg,et al.  Representation of motion and space in video and cinematic displays , 1986 .

[9]  L. Kaufman,et al.  Handbook of perception and human performance , 1986 .

[10]  B. Breitmeyer,et al.  Mechanisms of visual attention revealed by saccadic eye movements , 1987, Neuropsychologia.

[11]  Carolina M. Zingale,et al.  Planning sequences of saccades , 1987, Vision Research.

[12]  Z. Pylyshyn.  The role of location indexes in spatial perception: A sketch of the FINST spatial-index model , 1989, Cognition.

[13]  Eileen Kowler.  Eye movements and their role in visual and cognitive processes. , 1990, Reviews of oculomotor research.

[14]  Eileen Kowler.  The role of visual and cognitive processes in the control of eye movement. , 1990, Reviews of oculomotor research.

[15]  David E. Irwin,et al.  Visual memory and the perception of a stable visual environment , 1990, Perception & psychophysics.

[16]  D. Meyer,et al.  Eye-hand coordination: oculomotor control in rapid aimed limb movements. , 1990, Journal of experimental psychology. Human perception and performance.

[17]  David E. Irwin.  Information integration across saccadic eye movements , 1991, Cognitive Psychology.

[18]  Richard A. Andersen,et al.  Sensorimotor transformation during eye movements to remembered visual targets , 1991, Vision Research.

[19]  D. E. Irwin.  Memory for position and identity across eye movements. , 1992 .

[20]  J. C. Johnston,et al.  Involuntary covert orienting is contingent on attentional control settings. , 1992, Journal of experimental psychology. Human perception and performance.

[21]  J. Henderson.  Visual Attention and Eye Movement Control During Reading and Picture Viewing , 1992 .

[22]  K. Rayner.  Eye movements and visual cognition : scene perception and reading , 1992 .

[23]  Mary Hayhoe,et al.  Spatial Memory and Integration Across Saccadic Eye Movements , 1992 .

[24]  J. O'Regan,et al.  Solving the "real" mysteries of visual perception: the world as an outside memory. , 1992, Canadian journal of psychology.

[25]  K. Nakayama,et al.  Priming of pop-out: I. Role of features , 1994, Memory & cognition.

[26]  David N. Lee,et al.  Where we look when we steer , 1994, Nature.

[27]  H. Collewijn,et al.  The function of visual search and memory in sequential looking tasks , 1995, Vision Research.

[28]  R. Wurtz,et al.  Saccade-related activity in monkey superior colliculus. II. Spread of activity during saccades. , 1995, Journal of neurophysiology.

[29]  D. Ballard,et al.  Memory Representations in Natural Tasks , 1995, Journal of Cognitive Neuroscience.

[30]  Christopher B. Currie,et al.  Visual stability across saccades while viewing complex pictures. , 1995, Journal of experimental psychology. Human perception and performance.

[31]  D. Sparks,et al.  Combined eye-head gaze shifts produced by electrical stimulation of the superior colliculus in rhesus monkeys. , 1996, Journal of neurophysiology.

[32]  D. E. Irwin.  Integrating Information Across Saccadic Eye Movements , 1996 .

[33]  D. Simons.  In Sight, Out of Mind: When Object Representations Fail , 1996 .

[34]  M F Land,et al.  The knowledge base of the oculomotor system. , 1997, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[35]  Rajesh P. N. Rao,et al.  PSYCHOLOGICAL SCIENCE Research Article EYE MOVEMENTS REVEAL THE SPATIOTEMPORAL DYNAMICS OE VISUAL SEARCH , 2022 .

[36]  Ronald A. Rensink,et al.  TO SEE OR NOT TO SEE: The Need for Attention to Perceive Changes in Scenes , 1997 .

[37]  D. Simons,et al.  Failure to detect changes to attended objects in motion pictures , 1997 .

[38]  D. Ballard,et al.  Task constraints in visual working memory , 1997, Vision Research.

[39]  B. Gibson,et al.  Surprise! An Unexpected Color Singleton Does Not Capture Attention in Visual Search , 1998 .

[40]  David E. Irwin Robert D. Gordon.  Eye Movements, Attention and Trans-saccadic Memory , 1998 .

[41]  C. Colby.  Action-Oriented Spatial Reference Frames in Cortex , 1998, Neuron.

[42]  D. E. Irwin,et al.  Our Eyes do Not Always Go Where we Want Them to Go: Capture of the Eyes by New Objects , 1998 .

[43]  M. Chun,et al.  Contextual Cueing: Implicit Learning and Memory of Visual Context Guides Spatial Attention , 1998, Cognitive Psychology.

[44]  M. Land,et al.  The Roles of Vision and Eye Movements in the Control of Activities of Daily Living , 1998, Perception.

[45]  J. Henderson,et al.  The Role of Fixation Position in Detecting Scene Changes Across Saccades , 1999 .

[46]  A P Batista,et al.  Reach plans in eye-centered coordinates. , 1999, Science.

[47]  Robert M. McPeek,et al.  Saccades require focal attention and are facilitated by a short-term memory system , 1999, Vision Research.

[48]  M. Chun,et al.  Top-Down Attentional Guidance Based on Implicit Learning of Visual Covariation , 1999 .

[49]  N. P. Bichot,et al.  Effects of similarity and history on neural mechanisms of visual selection , 1999, Nature Neuroscience.

[50]  Ronald A. Rensink,et al.  Change-blindness as a result of ‘mudsplashes’ , 1999, Nature.

[51]  David E. Irwin,et al.  Influence of attentional capture on oculomotor control. , 1999, Journal of experimental psychology. Human perception and performance.

[52]  M. Hayhoe.  Vision Using Routines: A Functional Account of Vision , 2000 .

[53]  G. Wallis,et al.  What's Scene and Not Seen: Influences of Movement and Task Upon What We See , 2000 .

[54]  Michael F. Land,et al.  From eye movements to actions: how batsmen hit the ball , 2000, Nature Neuroscience.

[55]  K. Nakayama,et al.  On the Functional Role of Implicit Visual Memory for the Adaptive Deployment of Attention Across Scenes , 2000 .

[56]  M. Chun,et al.  Organization of visual short-term memory. , 2000, Journal of experimental psychology. Learning, memory, and cognition.

[57]  D. E. Irwin,et al.  Attentional and oculomotor capture by onset, luminance and color singletons , 2000, Vision Research.

[58]  Ronald A. Rensink,et al.  Picture Changes During Blinks: Looking Without Seeing and Seeing Without Looking , 2000 .

[59]  Jeff B. Pelz,et al.  Portable eyetracking: a study of natural eye movements , 2000, Electronic Imaging.

[60]  C. Koch,et al.  A saliency-based search mechanism for overt and covert shifts of visual attention , 2000, Vision Research.

[61]  Robert M. McPeek,et al.  Concurrent processing of saccades in visual search , 2000, Vision Research.

[62]  K. Verfaillie,et al.  Transsaccadic Coding of Object Position: Effects of Saccadic Status and Allocentric Reference Frame , 2001 .

[63]  Eileen Kowler,et al.  Visual scene memory and the guidance of saccadic eye movements , 2001, Vision Research.

[64]  K. Verfaillie,et al.  Selective transsaccadic coding of object and event-diagnostic information , 2001 .

[65]  J. Pelz,et al.  Oculomotor behavior and perceptual strategies in complex tasks , 2001, Vision Research.

[66]  R. Johansson,et al.  Eye–Hand Coordination in Object Manipulation , 2001, The Journal of Neuroscience.

[67]  E. Keller,et al.  Short-term priming, concurrent processing, and saccade curvature during a target selection task in the monkey , 2001, Vision Research.

[68]  Robert M. McPeek,et al.  Superior colliculus activity related to concurrent processing of saccade goals in a visual search task. , 2002, Journal of neurophysiology.

[69]  A. Mack.  Inattentional Blindness , 2003 .

[70]  B. Fischer,et al.  Human express saccades: extremely short reaction times of goal directed eye movements , 2004, Experimental Brain Research.

[71]  Michael S. Ambinder,et al.  Change blindness , 1997, Trends in Cognitive Sciences.