Bold moves: Inevitable saccadic selection in visual short-term memory

Selection for visual short-term memory (vstm) provides a basis for many cognitive functions. Saccadic eye movements sway this selection in favor of stimuli previously seen at locations congruent with their target. In three experiments, we provide converging evidence that this saccadic selection is implemented as a fundamental, inevitable selection process, rather than a top-down strategy. In particular, benefits for congruent over incongruent items were largely constant across set sizes ranging from two to eight items (Experiment 1), showing that saccadic selection imposes priorities on vstm irrespective of memory load and is effective even when only few representations need to be maintained. Moreover, a decrement in performance for incongruent items occurred reliably, whether the congruent location contained a task-relevant item or an irrelevant noise patch (Experiment 2). Finally, saccadic selection was immune to a strong manipulation of the observer's attentional priorities (Experiment 3). Given the prevalence of saccades in natural vision, our results demonstrate a fundamental and ecologically relevant selection mechanism for vstm: Saccades systematically eliminate information seen at non-target locations, while information at the saccade target remains available to recall. This simple heuristic is effective in the absence of informative cues and may incapacitate voluntary selection mechanisms that are incongruent with ongoing movement plans.

[1]  Richard D. Morey,et al.  Baysefactor: Computation of Bayes Factors for Common Designs , 2018 .

[2]  George Sperling,et al.  The information available in brief visual presentations. , 1960 .

[3]  Albert Postma,et al.  The cost of making an eye movement: A direct link between visual working memory and saccade execution. , 2017, Journal of vision.

[4]  W. A. Phillips On the distinction between sensory storage and short-term visual memory , 1974 .

[5]  Andrew Hollingworth,et al.  The relationship between visual attention and visual working memory encoding: A dissociation between covert and overt orienting. , 2016, Journal of experimental psychology. Human perception and performance.

[6]  A. Nobre,et al.  Orienting Attention to Locations in Internal Representations , 2003, Journal of Cognitive Neuroscience.

[7]  Martin Rolfs,et al.  Saccadic selection of stabilized items in visuospatial working memory , 2018, Consciousness and Cognition.

[8]  Mark W. Becker,et al.  The Role of Iconic Memory in Change-Detection Tasks , 2000, Perception.

[9]  Ralf Engbert,et al.  Microsaccades are triggered by low retinal image slip. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[10]  Martin Rolfs,et al.  Saccadic Eye Movements Impose a Natural Bottleneck on Visual Short-Term Memory , 2017, Journal of experimental psychology. Learning, memory, and cognition.

[11]  V. Lamme,et al.  Forgotten but not gone: Retro-cue costs and benefits in a double-cueing paradigm suggest multiple states in visual short-term memory. , 2015, Journal of experimental psychology. Learning, memory, and cognition.

[12]  Paul M Bays,et al.  Dynamic Shifts of Limited Working Memory Resources in Human Vision , 2008, Science.

[13]  Klaus Oberauer,et al.  Focused attention improves working memory: implications for flexible-resource and discrete-capacity models , 2014, Attention, perception & psychophysics.

[14]  J. Myerson,et al.  The effects of eye and limb movements on working memory , 2001, Memory.

[15]  Edward K. Vogel,et al.  The capacity of visual working memory for features and conjunctions , 1997, Nature.

[16]  Thom Baguley,et al.  Calculating and graphing within-subject confidence intervals for ANOVA , 2012, Behavior research methods.

[17]  H Pashler,et al.  Familiarity and visual change detection , 1988, Perception & psychophysics.

[18]  A. Schubö,et al.  Separate and combined effects of action relevance and motivational value on visual working memory. , 2018, Journal of vision.

[19]  R. Kliegl,et al.  Secondary (micro-)saccades: The influence of primary saccade end point and target eccentricity on the process of postsaccadic fixation , 2011, Vision Research.

[20]  Leanne Boucher,et al.  Visual–spatial attention aids the maintenance of object representations in visual working memory , 2013, Memory & cognition.

[21]  M. Eden,et al.  Note on Short Term Storage of Information in Vision , 1964, Perceptual and motor skills.

[22]  Yuhong Jiang,et al.  Distributing versus focusing attention in visual short-term memory , 2007, Psychonomic bulletin & review.

[23]  Heiner Deubel,et al.  Oculomotor selection underlies feature retention in visual working memory. , 2016, Journal of neurophysiology.

[24]  Anna Christina Nobre,et al.  Orienting attention to locations in mental representations , 2012, Attention, perception & psychophysics.

[25]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[26]  S. Luck,et al.  Attention effects during visual short-term memory maintenance: Protection or prioritization? , 2007 .

[27]  Denis G. Pelli,et al.  ECVP '07 Abstracts , 2007, Perception.

[28]  Martin Rolfs,et al.  Selective enhancement of orientation tuning before saccades. , 2017, Journal of vision.

[29]  H. Deubel,et al.  Saccade target selection and object recognition: Evidence for a common attentional mechanism , 1996, Vision Research.

[30]  R. Gregory The Most Expensive Painting in the World , 2007, Perception.

[31]  D. Pearson,et al.  Oculomotor Control and the Maintenance of Spatially and Temporally Distributed Events in Visuo-Spatial Working Memory , 2003, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[32]  R. Kliegl,et al.  Revealing the time course of signals influencing the generation of secondary saccades using Aalen’s additive hazards model , 2016, Vision Research.

[33]  Frans W Cornelissen,et al.  The Eyelink Toolbox: Eye tracking with MATLAB and the Psychophysics Toolbox , 2002, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[34]  Gordon D. Love,et al.  Creating correct blur and its effect on accommodation , 2018, Journal of vision.

[35]  Alessandra S. Souza,et al.  In search of the focus of attention in working memory: 13 years of the retro-cue effect , 2016, Attention, perception & psychophysics.

[36]  P. Tiňo,et al.  Learning predictive statistics from temporal sequences: Dynamics and strategies , 2017, Journal of vision.

[37]  N. Cowan The magical number 4 in short-term memory: A reconsideration of mental storage capacity , 2001, Behavioral and Brain Sciences.

[38]  Heiner Deubel,et al.  Independent Effects of Eye and Hand Movements on Visual Working Memory , 2018, Front. Syst. Neurosci..

[39]  Anna C. Nobre,et al.  Spatial Attention can Bias Search in Visual Short-Term Memory , 2007, Frontiers in human neuroscience.

[40]  J. Weesie,et al.  Integration of visual and inertial cues in perceived heading of self-motion. , 2010, Journal of vision.

[41]  Jöran Lepsien,et al.  Attentional orienting to mnemonic representations: Reduction of load-sensitive maintenance-related activity in the intraparietal sulcus , 2012, Neuropsychologia.

[42]  Richard D. Morey,et al.  Confidence Intervals from Normalized Data: A correction to Cousineau (2005) , 2008 .

[43]  E. Vogel,et al.  Interactions between attention and working memory , 2006, Neuroscience.

[44]  Joel Myerson,et al.  Interference with spatial working memory: An eye movement is more than a shift of attention , 2004, Psychonomic bulletin & review.

[45]  G. Sperling,et al.  Information transfer in iconic memory experiments. , 1993, Journal of experimental psychology. Human perception and performance.

[46]  Jiangang Lu,et al.  Saccades elicit obligatory allocation of visual working memory , 2010, Memory & cognition.

[47]  Caroline Van Eccelpoel,et al.  The visual analog: evidence for a preattentive representation across saccades. , 2010, Journal of vision.

[48]  J. Theeuwes,et al.  The time course of protecting a visual memory representation from perceptual interference , 2015, Front. Hum. Neurosci..

[49]  Martin Rolfs,et al.  Setting and changing feature priorities in visual short-term memory , 2017, Psychonomic bulletin & review.

[50]  J. Crawford,et al.  Action relevance induces an attentional weighting of representations in visual working memory , 2017, Memory & cognition.

[51]  D G Pelli,et al.  The VideoToolbox software for visual psychophysics: transforming numbers into movies. , 1997, Spatial vision.

[52]  Paul M Bays,et al.  The precision of visual working memory is set by allocation of a shared resource. , 2009, Journal of vision.

[53]  G. Woodman,et al.  Voluntary and automatic attentional control of visual working memory , 2002 .

[54]  C. Blakemore,et al.  Vision: The iconic bottleneck and the tenuous link between early visual processing and perception , 1990 .

[55]  Michael X. Cohen,et al.  Neural Correlates of Visual Short-term Memory Dissociate between Fragile and Working Memory Representations , 2015, Journal of Cognitive Neuroscience.

[56]  M. Carrasco,et al.  Rapid Simultaneous Enhancement of Visual Sensitivity and Perceived Contrast during Saccade Preparation , 2012, The Journal of Neuroscience.

[57]  M. Carrasco Visual attention: The past 25 years , 2011, Vision Research.

[58]  Amanda L. Gilchrist,et al.  Retrospective cues based on object features improve visual working memory performance in older adults , 2016, Neuropsychology, development, and cognition. Section B, Aging, neuropsychology and cognition.

[59]  Mark G. Stokes,et al.  Attention Modulates Maintenance of Representations in Visual Short-term Memory , 2012, Journal of Cognitive Neuroscience.

[60]  B. Dosher,et al.  The role of attention in the programming of saccades , 1995, Vision Research.

[61]  Markus Janczyk,et al.  The (Un)Clear Effects of Invalid Retro-Cues , 2016, Front. Psychol..

[62]  Martin Rolfs,et al.  Attention in Active Vision: A Perspective on Perceptual Continuity Across Saccades , 2015, Perception.

[63]  R. Marois,et al.  Capacity limits of information processing in the brain , 2005, Trends in Cognitive Sciences.

[64]  A. Hollingworth,et al.  Visuospatial Working Memory as a Fundamental Component of the Eye Movement System , 2018, Current directions in psychological science.

[65]  Victor A. F. Lamme,et al.  Are There Multiple Visual Short-Term Memory Stores? , 2008, PloS one.

[66]  Ashleigh M. Richard,et al.  Understanding the function of visual short-term memory: transsaccadic memory, object correspondence, and gaze correction. , 2008, Journal of experimental psychology. General.

[67]  A. Baddeley,et al.  The selective disruption of spatial working memory by eye movements , 2006, Quarterly journal of experimental psychology.

[68]  Rachel S. Sussman,et al.  Orienting attention in visual working memory reduces interference from memory probes. , 2008, Journal of experimental psychology. Learning, memory, and cognition.

[69]  S. Luck,et al.  Discrete fixed-resolution representations in visual working memory , 2008, Nature.