The role of memory in guiding attention during natural vision.

What is the time frame in which perceptual memory guides attention? Current estimates range from a few hundred milliseconds to several seconds, minutes, or even days. Here, we answer this question by establishing the time course of attentional selection in realistic viewing conditions. First, we transformed continuous video clips into MTV-style video clips by stringing together continuous clip segments using abrupt transitions (jump cuts). We then asked participants to visually explore either continuous or MTV-style clips and recorded their saccades as objective behavioral indicators of attentional selections. The utilization of perceptual memory was estimated across viewing conditions and over time by quantifying the agreement between human attentional selections and predictions made by a neurally grounded computational model. In the critical condition, jump cuts led to sharp declines in the impact of perceptual memory on attentional selection, followed by monotonic increases in memory utilization across seven consecutive saccades and 2.5 s. These results demonstrate that perceptual memory traces play an important role in guiding attention across several saccades during natural vision. We propose novel hypotheses and experiments using hybrid natural-artificial stimuli to further elucidate neurocomputational mechanisms of attentional selection.

[1]  C. Koch,et al.  A saliency-based search mechanism for overt and covert shifts of visual attention , 2000, Vision Research.

[2]  J. O'Regan,et al.  Solving the "real" mysteries of visual perception: the world as an outside memory. , 1992, Canadian journal of psychology.

[3]  M F Land,et al.  The knowledge base of the oculomotor system. , 1997, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[4]  Joseph D. Anderson.  The reality of illusion : an ecological approach to cognitive film theory , 1996 .

[5]  Gidon Felsen,et al.  A natural approach to studying vision , 2005, Nature Neuroscience.

[6]  J. Gibson.  The Ecological Approach to Visual Perception , 1979 .

[7]  S Ullman,et al.  Shifts in selective visual attention: towards the underlying neural circuitry. , 1985, Human neurobiology.

[8]  Antonio Torralba,et al.  Top-down control of visual attention in object detection , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[9]  H. Jones,et al.  Visual cortical mechanisms detecting focal orientation discontinuities , 1995, Nature.

[10]  S. Yantis,et al.  Uniqueness of abrupt visual onset in capturing attention , 1988, Perception & psychophysics.

[11]  P. Viviani.  Eye movements in visual search: cognitive, perceptual and motor control aspects. , 1990, Reviews of oculomotor research.

[12]  Asha Iyer,et al.  Components of bottom-up gaze allocation in natural images , 2005, Vision Research.

[13]  David N. Lee,et al.  Where we look when we steer , 1994, Nature.

[14]  F. Volkmar,et al.  Visual fixation patterns during viewing of naturalistic social situations as predictors of social competence in individuals with autism. , 2002, Archives of general psychiatry.

[15]  R. Malach,et al.  Intersubject Synchronization of Cortical Activity During Natural Vision , 2004, Science.

[16]  Miguel P Eckstein,et al.  The time course of visual information accrual guiding eye movement decisions. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[17]  Derrick J. Parkhurst,et al.  Modeling the role of salience in the allocation of overt visual attention , 2002, Vision Research.

[18]  Mary M Hayhoe,et al.  Visual memory and motor planning in a natural task. , 2003, Journal of vision.

[19]  R. Walker,et al.  A model of saccade generation based on parallel processing and competitive inhibition , 1999, Behavioral and Brain Sciences.

[20]  P. Tse,et al.  Time and the Brain: How Subjective Time Relates to Neural Time , 2005 .

[21]  Terrence J. Sejnowski,et al.  Network model of shape-from-shading: neural function arises from both receptive and projective fields , 1988, Nature.

[22]  J. Henderson,et al.  Accurate visual memory for previously attended objects in natural scenes , 2002 .

[23]  K. Nakayama,et al.  Single visual neurons code opposing motion independent of direction. , 1983, Science.

[24]  J. Cutting.  The Reality of Illusion: An Ecological Approach to Cognitive Film Theory , 1999 .

[25]  Iain D. Gilchrist,et al.  Visual correlates of fixation selection: effects of scale and time , 2005, Vision Research.

[26]  J. Henderson.  Human gaze control during real-world scene perception , 2003, Trends in Cognitive Sciences.

[27]  Julian Hochberg,et al.  Representation of motion and space in video and cinematic displays , 1986 .

[28]  Derrick J. Parkhurst,et al.  Scene content selected by active vision. , 2003, Spatial vision.

[29]  J. Atkinson,et al.  Neurobiological models of normal and abnormal visual development , 2005 .

[30]  D. S. Wooding,et al.  Fixation Patterns Made during Brief Examination of Two-Dimensional Images , 1997, Perception.

[31]  David Melcher,et al.  Persistence of visual memory for scenes , 2001, Nature.

[32]  Nicole C. Rust,et al.  In praise of artifice , 2005, Nature Neuroscience.

[33]  Michael F. Land,et al.  From eye movements to actions: how batsmen hit the ball , 2000, Nature Neuroscience.

[34]  J. Wolfe,et al.  Attention is fast but volition is slow , 2000, Nature.

[35]  A. L. I︠A︡rbus.  Eye Movements and Vision , 1967 .

[36]  K. Nakayama,et al.  Priming of pop-out: II. The role of position , 1996, Perception & psychophysics.

[37]  Antonio Torralba,et al.  Modeling global scene factors in attention. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[38]  A. L. Yarbus,et al.  Eye Movements and Vision , 1967, Springer US.

[39]  G. Hauske,et al.  Object and scene analysis by saccadic eye-movements: an investigation with higher-order statistics. , 2000, Spatial vision.

[40]  Kenneth R. Boff,et al.  Sensory processes and perception , 1986 .

[41]  L. Kaufman,et al.  Handbook of perception and human performance , 1986 .

[42]  D. Spalding.  The Principles of Psychology , 1873, Nature.

[43]  Helga C. Arsenio,et al.  Panoramic search: the interaction of memory and vision in search through a familiar scene. , 2004, Journal of experimental psychology. Human perception and performance.

[44]  M. Kenward,et al.  An Introduction to the Bootstrap , 2007 .

[45]  E. Reed.  The Ecological Approach to Visual Perception , 1989 .

[46]  Wilson S. Geisler,et al.  Optimal eye movement strategies in visual search , 2005, Nature.

[47]  Wilson S. Geisler,et al.  Optimal eye movement strategies in visual search , 2005, Nature.

[48]  P Reinagel,et al.  Natural scene statistics at the centre of gaze. , 1999, Network.

[49]  J. Henderson,et al.  High-level scene perception. , 1999, Annual review of psychology.

[50]  Eileen Kowler,et al.  Visual scene memory and the guidance of saccadic eye movements , 2001, Vision Research.

[51]  M. Land.  Motion and vision: why animals move their eyes , 1999, Journal of Comparative Physiology A.

[52]  M. Chun,et al.  Contextual Cueing: Implicit Learning and Memory of Visual Context Guides Spatial Attention , 1998, Cognitive Psychology.

[53]  W. N. Schoenfeld,et al.  Principles of Psychology , 2007 .

[54]  R. Hernández-Peón,et al.  Modification of electric activity in cochlear nucleus during attention in unanesthetized cats. , 1956, Science.

[55]  M. Hayhoe,et al.  What controls attention in natural environments? , 2001, Vision Research.

[56]  HighWire Press.  Philosophical Transactions of the Royal Society of London , 1781, The London Medical Journal.

[57]  Todd S. Horowitz,et al.  Visual search has no memory , 1998, Nature.

[58]  M. Murray,et al.  EEG source imaging , 2004, Clinical Neurophysiology.

[59]  K. Nakayama,et al.  On the Functional Role of Implicit Visual Memory for the Adaptive Deployment of Attention Across Scenes , 2000 .

[60]  A. Treisman,et al.  A feature-integration theory of attention , 1980, Cognitive Psychology.

[61]  Klaus Bartl,et al.  A pivotable head mounted camera system that is aligned by three-dimensional eye movements , 2006, ETRA.

[62]  D. Simons,et al.  Failure to detect changes to attended objects in motion pictures , 1997 .

[63]  L. Itti.  Author address: , 1999 .

[64]  K. Nakayama,et al.  Priming of popout: II. Role of position , 1996 .

[65]  M. Goldberg,et al.  The representation of visual salience in monkey parietal cortex , 1998, Nature.

[66]  David Bordwell,et al.  Film History: An Introduction , 1994 .

[67]  U Polat,et al.  Spatial interactions in human vision: from near to far via experience-dependent cascades of connections. , 1994, Proceedings of the National Academy of Sciences of the United States of America.

[68]  David J. Field,et al.  How Close Are We to Understanding V1? , 2005, Neural Computation.

[69]  K. Nakayama,et al.  Priming of pop-out: I. Role of features , 1994, Memory & cognition.

[70]  Ronald A. Rensink.  Seeing, sensing, and scrutinizing , 2000, Vision Research.