Visual working memory modulates low-level saccade target selection: evidence from rapidly generated saccades in the global effect paradigm.

In three experiments, we examined the influence of visual working memory (VWM) on the metrics of saccade landing position in a global effect paradigm. Participants executed a saccade to the more eccentric object in an object pair appearing on the horizontal midline, to the left or right of central fixation. While completing the saccade task, participants maintained a color in VWM for an unrelated memory task. Either the color of the saccade target matched the memory color (target match), the color of the distractor matched the memory color (distractor match), or the colors of neither object matched the memory color (no match). In the no-match condition, saccades tended to land at the midpoint between the two objects: the global, or averaging, effect. However, when one of the two objects matched VWM, the distribution of landing position shifted toward the matching object, both for target match and for distractor match. VWM modulation of landing position was observed even for the fastest quartile of saccades, with a mean latency as low as 112 ms. Effects of VWM on such rapidly generated saccades, with latencies in the express-saccade range, indicate that VWM interacts with the initial sweep of visual sensory processing, modulating perceptual input to oculomotor systems and thereby biasing oculomotor selection. As a result, differences in memory match produce effects on landing position similar to the effects generated by differences in physical salience.

[1]  A. L. I︠A︡rbus Eye Movements and Vision , 1967 .

[2]  A. L. Yarbus,et al.  Eye Movements and Vision , 1967, Springer US.

[3]  S. Coren,et al.  Effect of Non-Target Stimuli upon Length of Voluntary Saccades , 1972, Perceptual and motor skills.

[4]  J. Findlay Global visual processing for saccadic eye movements , 1982, Vision Research.

[5]  F. Ottes,et al.  Metrics of saccade responses to visual double stimuli: Two different modes , 1984, Vision Research.

[6]  J. Stern Theoretical and applied aspects of eye movement research A. G. Gale and F. Johnson, (Elsevier Science Publishers B.V., Amsterdam, 1984) pp. xiii + 565, Dfl. 185 , 1985, Biological Psychology.

[7]  F. Ottes,et al.  Latency dependence of colour-based target vs nontarget discrimination by the saccadic system , 1985, Vision Research.

[8]  J. Duncan,et al.  Visual search and stimulus similarity. , 1989, Psychological review.

[9]  C. Bundesen A theory of visual attention. , 1990, Psychological review.

[10]  J. Theeuwes Exogenous and endogenous control of attention: The effect of visual onsets and offsets , 1991, Perception & psychophysics.

[11]  J. C. Johnston,et al.  Involuntary covert orienting is contingent on attentional control settings. , 1992, Journal of experimental psychology. Human perception and performance.

[12]  John Duncan,et al.  A neural basis for visual search in inferior temporal cortex , 1993, Nature.

[13]  J. Wolfe,et al.  Guided Search 2.0 A revised model of visual search , 1994, Psychonomic bulletin & review.

[14]  R. Desimone,et al.  Neural mechanisms of selective visual attention. , 1995, Annual review of neuroscience.

[15]  R. Wurtz,et al.  Saccade-related activity in monkey superior colliculus. I. Characteristics of burst and buildup cells. , 1995, Journal of neurophysiology.

[16]  S. Luck,et al.  Bridging the Gap between Monkey Neurophysiology and Human Perception: An Ambiguity Resolution Theory of Visual Selective Attention , 1997, Cognitive Psychology.

[17]  R. Desimone,et al.  Responses of Neurons in Inferior Temporal Cortex during Memory- Guided Visual Search , 1998 .

[18]  M. Chun,et al.  Contextual Cueing: Implicit Learning and Memory of Visual Context Guides Spatial Attention , 1998, Cognitive Psychology.

[19]  J. Henderson,et al.  The effects of semantic consistency on eye movements during complex scene viewing , 1999 .

[20]  P. H. Schiller,et al.  Express averaging saccades in monkeys , 1999, Vision Research.

[21]  C. Koch,et al.  A saliency-based search mechanism for overt and covert shifts of visual attention , 2000, Vision Research.

[22]  Leslie G. Ungerleider,et al.  Mechanisms of visual attention in the human cortex. , 2000, Annual review of neuroscience.

[23]  J Duncan,et al.  Responses of neurons in macaque area V4 during memory-guided visual search. , 2001, Cerebral cortex.

[24]  R. Klein,et al.  A Model of Saccade Initiation Based on the Competitive Integration of Exogenous and Endogenous Signals in the Superior Colliculus , 2001, Journal of Cognitive Neuroscience.

[25]  M. Hayhoe,et al.  In what ways do eye movements contribute to everyday activities? , 2001, Vision Research.

[26]  J. Theeuwes,et al.  Programming of endogenous and exogenous saccades: evidence for a competitive integration model. , 2002, Journal of experimental psychology. Human perception and performance.

[27]  Casimir J. H. Ludwig,et al.  Stimulus-driven and goal-driven control over visual selection. , 2002, Journal of experimental psychology. Human perception and performance.

[28]  B. Fischer,et al.  Human express saccades: extremely short reaction times of goal directed eye movements , 2004, Experimental Brain Research.

[29]  Gregor Schöner,et al.  Saccadic motor planning by integrating visual information and pre-information on neural dynamic fields , 1995, Biological Cybernetics.

[30]  Jeffrey D Schall,et al.  On the role of frontal eye field in guiding attention and saccades , 2004, Vision Research.

[31]  J. Theeuwes,et al.  The role of stimulus-driven and goal-driven control in saccadic visual selection. , 2004, Journal of experimental psychology. Human perception and performance.

[32]  Naomi M. Kenner,et al.  How fast can you change your mind? The speed of top-down guidance in visual search , 2004, Vision Research.

[33]  C. Bundesen,et al.  A neural theory of visual attention: bridging cognition and neurophysiology. , 2005, Psychological review.

[34]  Yuhong Jiang,et al.  Setting up the target template in visual search. , 2005, Journal of vision.

[35]  Robert Desimone,et al.  Parallel and Serial Neural Mechanisms for Visual Search in Macaque Area V4 , 2005, Science.

[36]  G. Humphreys,et al.  Early, involuntary top-down guidance of attention from working memory. , 2005, Journal of experimental psychology. Human perception and performance.

[37]  J. Theeuwes,et al.  Feature-based memory-driven attentional capture: visual working memory content affects visual attention. , 2006, Journal of experimental psychology. Human perception and performance.

[38]  James R. Brockmole,et al.  Contextual cueing in naturalistic scenes: Global and local contexts. , 2006, Journal of experimental psychology. Learning, memory, and cognition.

[39]  Dietmar Heinke,et al.  Working memory can guide pop-out search , 2006, Vision Research.

[40]  Gregor Schöner,et al.  The time course of saccadic decision making: Dynamic field theory , 2006, Neural Networks.

[41]  P. Roelfsema,et al.  The effect of items in working memory on the deployment of attention and the eyes during visual search. , 2006, Journal of experimental psychology. Human perception and performance.

[42]  Antonio Torralba,et al.  Contextual guidance of eye movements and attention in real-world scenes: the role of global features in object search. , 2006, Psychological review.

[43]  Gregory J. Zelinsky,et al.  Scene context guides eye movements during visual search , 2006, Vision Research.

[44]  G. Woodman,et al.  Do the contents of visual working memory automatically influence attentional selection during visual search? , 2007, Journal of experimental psychology. Human perception and performance.

[45]  J. Henderson,et al.  Initial scene representations facilitate eye movement guidance in visual search. , 2007, Journal of experimental psychology. Human perception and performance.

[46]  Ashleigh M. Richard,et al.  Understanding the function of visual short-term memory: transsaccadic memory, object correspondence, and gaze correction. , 2008, Journal of experimental psychology. General.

[47]  Richard D. Morey,et al.  Confidence Intervals from Normalized Data: A correction to Cousineau (2005) , 2008 .

[48]  Vani Pariyadath,et al.  Is subjective duration a signature of coding efficiency? , 2009, Philosophical Transactions of the Royal Society B: Biological Sciences.

[49]  S. Luck,et al.  Feature-based attention modulates feedforward visual processing , 2009, Nature Neuroscience.

[50]  Hany Farid,et al.  The specificity of the search template. , 2009, Journal of vision.

[51]  A. Hollingworth Two forms of scene memory guide visual search: Memory for scene context and memory for the binding of target object to scene location , 2009 .

[52]  Gregory J. Zelinsky,et al.  Visual search is guided to categorically-defined targets , 2009, Vision Research.

[53]  Christian N L Olivers,et al.  What drives memory-driven attentional capture? The effects of memory type, display type, and search type. , 2009, Journal of experimental psychology. Human perception and performance.

[54]  George L. Malcolm,et al.  The effects of target template specificity on visual search in real-world scenes: evidence from eye movements. , 2009, Journal of vision.

[55]  G. Woodman,et al.  The comparison of visual working memory representations with perceptual inputs. , 2009, Journal of experimental psychology. Human perception and performance.

[56]  Jeffrey S. Johnson,et al.  A Dynamic Neural Field Model of Visual Working Memory and Change Detection , 2009, Psychological science.

[57]  G. Humphreys,et al.  Automatic selection of irrelevant object features through working memory: evidence for top-down attentional capture. , 2009, Experimental Psychology.

[58]  Min-Shik Kim,et al.  Do the contents of working memory capture attention? Yes, but cognitive control matters. , 2009, Journal of experimental psychology. Human perception and performance.

[59]  S. Luck,et al.  The role of visual working memory (VWM) in the control of gaze during visual search , 2009, Attention, perception & psychophysics.

[60]  Jan Theeuwes,et al.  A competitive integration model of exogenous and endogenous eye movements , 2010, Biological Cybernetics.

[61]  George L. Malcolm,et al.  Combining top-down processes to guide eye movements during real-world scene search. , 2010, Journal of vision.

[62]  Christopher Kennard,et al.  Early oculomotor capture by new onsets driven by the contents of working memory , 2010, Vision Research.

[63]  John P. Spencer,et al.  Dynamic interactions between visual working memory and saccade planning , 2010 .

[64]  Robert Desimone,et al.  Feature-Based Attention in the Frontal Eye Field and Area V4 during Visual Search , 2011, Neuron.

[65]  Randolph Blake,et al.  Visual working memory contaminates perception , 2011, Psychonomic bulletin & review.

[66]  C. Olivers Long-term visual associations affect attentional guidance. , 2011, Acta psychologica.

[67]  Jason T. Arita,et al.  Direct Electrophysiological Measurement of Attentional Templates in Visual Working Memory , 2011, Psychological science.

[68]  Geoffrey F. Woodman,et al.  Attentional Templates in Visual Working Memory , 2011, The Journal of Neuroscience.

[69]  Andrew Hollingworth,et al.  Guidance of visual search by memory and knowledge. , 2012, Nebraska Symposium on Motivation. Nebraska Symposium on Motivation.

[70]  Stephen J. Gotts,et al.  Cell-Type-Specific Synchronization of Neural Activity in FEF with V4 during Attention , 2012, Neuron.

[71]  Thomas P. Trappenberg,et al.  Spatial Interactions in the Superior Colliculus Predict Saccade Behavior in a Neural Field Model , 2012, Journal of Cognitive Neuroscience.

[72]  A. Hollingworth Task specificity and the influence of memory on visual search: comment on Võ and Wolfe (2012). , 2012, Journal of experimental psychology. Human perception and performance.

[73]  Yi Pan,et al.  Working memory modulates the perception of time , 2011, Psychonomic Bulletin & Review.

[74]  Jochen Triesch,et al.  What’s “up”? Working memory contents can bias orientation processing , 2013, Vision Research.

[75]  J. Wolfe,et al.  The interplay of episodic and semantic memory in guiding repeated search in scenes , 2013, Cognition.

[76]  K. Gegenfurtner,et al.  Visual Working Memory Contents Bias Ambiguous Structure from Motion Perception , 2013, PloS one.

[77]  Andrew Hollingworth,et al.  Visual Working Memory Modulates Rapid Eye Movements to Simple Onset Targets , 2013, Psychological science.

[78]  Andrew Hollingworth,et al.  The relationship between visual working memory and attention: retention of precise colour information in the absence of effects on perceptual selection , 2013, Philosophical Transactions of the Royal Society B: Biological Sciences.

[79]  Stefan Van der Stigchel,et al.  Information Matching the Content of Visual Working Memory Is Prioritized for Conscious Access , 2013, Psychological science.

[80]  J. Silvis,et al.  How memory mechanisms are a key component in the guidance of our eye movements: Evidence from the global effect , 2013, Psychonomic Bulletin & Review.