The cost of making an eye movement: A direct link between visual working memory and saccade execution.

To facilitate visual continuity across eye movements, the visual system must presaccadically acquire information about the future foveal image. Previous studies have indicated that visual working memory (VWM) affects saccade execution. However, the reverse relation, the effect of saccade execution on VWM load is less clear. To investigate the causal link between saccade execution and VWM, we combined a VWM task and a saccade task. Participants were instructed to remember one, two, or three shapes and performed either a No Saccade-, a Single Saccade- or a Dual (corrective) Saccade-task. The results indicate that items stored in VWM are reported less accurately if a single saccade-or a dual saccade-task is performed next to retaining items in VWM. Importantly, the loss of response accuracy for items retained in VWM by performing a saccade was similar to committing an extra item to VWM. In a second experiment, we observed no cost of executing a saccade for auditory working memory performance, indicating that executing a saccade exclusively taxes the VWM system. Our results suggest that the visual system presaccadically stores the upcoming retinal image, which has a similar VWM load as committing one extra item to memory and interferes with stored VWM content. After the saccade, the visual system can retrieve this item from VWM to evaluate saccade accuracy. Our results support the idea that VWM is a system which is directly linked to saccade execution and promotes visual continuity across saccades.

[1]  S. Luck,et al.  The role of visual working memory (VWM) in the control of gaze during visual search , 2009, Attention, perception & psychophysics.

[2]  D. Melcher,et al.  The Role of Attentional Priority and Saliency in Determining Capacity Limits in Enumeration and Visual Working Memory , 2011, PloS one.

[3]  H. Deubel,et al.  Visual attention and saccadic eye movements: Evidence for obligatory and selective spatial coupling , 1995 .

[4]  Jeffrey S. Johnson,et al.  The role of attention in the maintenance of feature bindings in visual short-term memory. , 2008, Journal of experimental psychology. Human perception and performance.

[5]  D. E. Irwin,et al.  Integration and accumulation of information across saccadic eye movements. , 1996 .

[6]  P. Dixon Models of accuracy in repeated-measures designs , 2008 .

[7]  Arvid Herwig,et al.  Transsaccadic integration and perceptual continuity. , 2015, Journal of vision.

[8]  Stefan Van der Stigchel,et al.  Object files across eye movements: Previous fixations affect the latencies of corrective saccades , 2016, Attention, Perception, & Psychophysics.

[9]  Katharina Weiss,et al.  When circles become triangular: how transsaccadic predictions shape the perception of shape , 2015, Annals of the New York Academy of Sciences.

[10]  Brian S. Schnitzer,et al.  Eye movements and attention: The role of pre-saccadic shifts of attention in perception, memory and the control of saccades , 2012, Vision Research.

[11]  W. Schneider,et al.  Predicting object features across saccades: evidence from object recognition and visual search. , 2014, Journal of experimental psychology. General.

[12]  Daniel M. Wolpert,et al.  Forward Models for Physiological Motor Control , 1996, Neural Networks.

[13]  E. Stuyven,et al.  The effect of cognitive load on saccadic eye movements. , 2000, Acta psychologica.

[14]  R H Logie,et al.  Interference with visual short-term memory. , 1990, Acta psychologica.

[15]  D P Munoz,et al.  Saccadic reaction time in the monkey: advanced preparation of oculomotor programs is primarily responsible for express saccade occurrence. , 1996, Journal of neurophysiology.

[16]  Matteo Valsecchi,et al.  Dynamic Re-calibration of Perceived Size in Fovea and Periphery through Predictable Size Changes , 2016, Current Biology.

[17]  H. Deubel,et al.  Saccade target selection and object recognition: Evidence for a common attentional mechanism , 1996, Vision Research.

[18]  A. Postma,et al.  Multisensory Perception and the Coding of Space , 2016 .

[19]  Christopher L. Asplund,et al.  What are the units of storage in visual working memory? , 2010, Journal of vision.

[20]  Edward F. Ester,et al.  Parietal and Frontal Cortex Encode Stimulus-Specific Mnemonic Representations during Visual Working Memory , 2015, Neuron.

[21]  A. Watson,et al.  Quest: A Bayesian adaptive psychometric method , 1983, Perception & psychophysics.

[22]  D. Zee,et al.  Revisiting corrective saccades: Role of visual feedback , 2013, Vision Research.

[23]  W. Ma,et al.  Changing concepts of working memory , 2014, Nature Neuroscience.

[24]  J. Vroomen,et al.  Intersensory binding across space and time: A tutorial review , 2013, Attention, Perception, & Psychophysics.

[25]  D. Rubin,et al.  Inference from Iterative Simulation Using Multiple Sequences , 1992 .

[26]  D. Burr,et al.  Selective suppression of the magnocellular visual pathway during saccadic eye movements , 1994, Nature.

[27]  Markus Lappe,et al.  Adaptation of Saccades and Perceived Size after Trans-Saccadic Changes of Object Size , 2015, The Journal of Neuroscience.

[28]  Martin Rolfs,et al.  Saccadic Eye Movements Impose a Natural Bottleneck on Visual Short-Term Memory , 2017, Journal of experimental psychology. Learning, memory, and cognition.

[29]  Arvid Herwig,et al.  Associating peripheral and foveal visual input across saccades: a default mode of the human visual system? , 2014, Journal of vision.

[30]  L. M. Ward,et al.  Spatial relevance determines facilitatory and inhibitory effects of auditory covert spatial orienting , 1999 .

[31]  Ross Ihaka,et al.  Gentleman R: R: A language for data analysis and graphics , 1996 .

[32]  Arvid Herwig,et al.  Breaking Object Correspondence Across Saccadic Eye Movements Deteriorates Object Recognition , 2015, Front. Syst. Neurosci..

[33]  Eero P. Simoncelli,et al.  Near-optimal integration of orientation information across saccades. , 2015, Journal of vision.

[34]  R. Klein,et al.  Contribution of the Primate Superior Colliculus to Inhibition of Return , 2002, Journal of Cognitive Neuroscience.

[35]  Alexander C. Schütz,et al.  Trans-saccadic integration of peripheral and foveal feature information is close to optimal. , 2015, Journal of vision.

[36]  Jarrod D. Hadfield,et al.  MCMC methods for multi-response generalized linear mixed models , 2010 .

[37]  Leanne Boucher,et al.  Visual–spatial attention aids the maintenance of object representations in visual working memory , 2013, Memory & cognition.

[38]  B. Fischer,et al.  Saccadic eye movements after extremely short reaction times in the monkey , 1983, Brain Research.

[39]  M. Ernst,et al.  Humans integrate visual and haptic information in a statistically optimal fashion , 2002, Nature.

[40]  Ming-Hui Chen,et al.  Monte Carlo Estimation of Bayesian Credible and HPD Intervals , 1999 .

[41]  P. Bertelson,et al.  Cross-modal bias and perceptual fusion with auditory-visual spatial discordance , 1981, Perception & psychophysics.

[42]  B. Bridgeman,et al.  Postsaccadic target blanking prevents saccadic suppression of image displacement , 1996, Vision Research.

[43]  Andrew Gelman,et al.  General methods for monitoring convergence of iterative simulations , 1998 .

[44]  E. Wagenmakers,et al.  Bayesian hypothesis testing for psychologists: A tutorial on the Savage–Dickey method , 2010, Cognitive Psychology.

[45]  Sebastiaan Mathôt,et al.  PyGaze: An open-source, cross-platform toolbox for minimal-effort programming of eyetracking experiments , 2014, Behavior research methods.

[46]  Richard D. Morey,et al.  Baysefactor: Computation of Bayes Factors for Common Designs , 2018 .

[47]  H. Deubel The time course of presaccadic attention shifts , 2008, Psychological research.

[48]  B. Bridgeman A review of the role of efference copy in sensory and oculomotor control systems , 1995, Annals of Biomedical Engineering.

[49]  Heiner Deubel,et al.  Oculomotor selection underlies feature retention in visual working memory. , 2016, Journal of neurophysiology.

[50]  F. D. de Lange,et al.  Prior Expectations Bias Sensory Representations in Visual Cortex , 2013, The Journal of Neuroscience.

[51]  Tom Lodewyckx,et al.  A tutorial on Bayes factor estimation with the product space method , 2011 .

[52]  Gregory C. DeAngelis,et al.  Bridging the gap between theories of sensory cue integration and the physiology of multisensory neurons , 2013, Nature Reviews Neuroscience.

[53]  Ashleigh M. Richard,et al.  Understanding the function of visual short-term memory: transsaccadic memory, object correspondence, and gaze correction. , 2008, Journal of experimental psychology. General.

[54]  D. E. Irwin Memory for position and identity across eye movements. , 1992 .

[55]  M. D’Esposito Working memory. , 2008, Handbook of clinical neurology.

[56]  Werner X. Schneider,et al.  Selective visual processing across competition episodes: a theory of task-driven visual attention and working memory , 2013, Philosophical Transactions of the Royal Society B: Biological Sciences.

[57]  A. Treisman,et al.  A feature-integration theory of attention , 1980, Cognitive Psychology.

[58]  Paul M Bays,et al.  Dynamic Shifts of Limited Working Memory Resources in Human Vision , 2008, Science.

[59]  Robert H Wurtz,et al.  Saccadic Corollary Discharge Underlies Stable Visual Perception , 2016, The Journal of Neuroscience.

[60]  Andrew Hollingworth,et al.  The relationship between visual attention and visual working memory encoding: A dissociation between covert and overt orienting. , 2016, Journal of experimental psychology. Human perception and performance.

[61]  D. Munoz,et al.  Neuronal Activity in Monkey Superior Colliculus Related to the Initiation of Saccadic Eye Movements , 1997, The Journal of Neuroscience.

[62]  David E. Irwin Robert D. Gordon Eye Movements, Attention and Trans-saccadic Memory , 1998 .

[63]  D. Melcher Visual stability , 2011, Philosophical Transactions of the Royal Society B: Biological Sciences.

[64]  Adrian E. Raftery,et al.  Bayesian Model Averaging , 1998 .

[65]  Werner X Schneider,et al.  Breaking object correspondence across saccades impairs object recognition: The role of color and luminance. , 2016, Journal of vision.

[66]  L. Marshall,et al.  Evidence for Optimal Integration of Visual Feature Representations across Saccades , 2015, The Journal of Neuroscience.

[67]  Paul M Bays,et al.  The precision of visual working memory is set by allocation of a shared resource. , 2009, Journal of vision.

[68]  L. Wasserman,et al.  Computing Bayes Factors Using a Generalization of the Savage-Dickey Density Ratio , 1995 .

[69]  B. Webb Neural mechanisms for prediction: do insects have forward models? , 2004, Trends in Neurosciences.

[70]  David E. Irwin Information integration across saccadic eye movements , 1991, Cognitive Psychology.

[71]  Adrian E. Raftery,et al.  Bayesian model averaging: development of an improved multi-class, gene selection and classification tool for microarray data , 2005, Bioinform..

[72]  Jeffrey N. Rouder,et al.  Computation of Bayes Factors for Common Designs , 2015 .

[73]  Ashleigh M. Richard,et al.  Establishing object correspondence across eye movements: Flexible use of spatiotemporal and surface feature information , 2008, Cognition.

[74]  E. Matin Saccadic suppression: a review and an analysis. , 1974, Psychological bulletin.