Low-Level Visual Information Is Maintained across Saccades, Allowing for a Postsaccadic Handoff between Visual Areas

Experience seems continuous and detailed despite saccadic eye movements changing retinal input several times per second. There is debate whether neural signals related to updating across saccades contain information about stimulus features, or only location pointers without visual details. We investigated the time course of low-level visual information processing across saccades by decoding the spatial frequency of a stationary stimulus that changed from one visual hemifield to the other because of a horizontal saccadic eye movement. We recorded magnetoencephalography while human subjects (both sexes) monitored the orientation of a grating stimulus, making spatial frequency task irrelevant. Separate trials, in which subjects maintained fixation, were used to train a classifier, whose performance was then tested on saccade trials. Decoding performance showed that spatial frequency information of the presaccadic stimulus remained present for ∼200 ms after the saccade, transcending retinotopic specificity. Postsaccadic information ramped up rapidly after saccade offset. There was an overlap of over 100 ms during which decoding was significant from both presaccadic and postsaccadic processing areas. This suggests that the apparent richness of perception across saccades may be supported by the continuous availability of low-level information with a “soft handoff” of information during the initial processing sweep of the new fixation. SIGNIFICANCE STATEMENT Saccades create frequent discontinuities in visual input, yet perception appears stable and continuous. How is this discontinuous input processed resulting in visual stability? Previous studies have focused on presaccadic remapping. Here we examined the time course of processing of low-level visual information (spatial frequency) across saccades with magnetoencephalography. The results suggest that spatial frequency information is not predictively remapped but also is not discarded. Instead, they suggest a soft handoff over time between different visual areas, making this information continuously available across the saccade. Information about the presaccadic stimulus remains available, while the information about the postsaccadic stimulus has also become available. The simultaneous availability of both the presaccadic and postsaccadic information could enable rich and continuous perception across saccades.

[1]  Valentin Wyart,et al.  The Human Brain Encodes a Chronicle of Visual Events at Each Instant of Time Through the Multiplexing of Traveling Waves , 2019, The Journal of Neuroscience.

[2]  G. McConkie,et al.  Integrating information across eye movements , 1980, Cognitive Psychology.

[3]  Stephen M. Smith,et al.  Threshold-free cluster enhancement: Addressing problems of smoothing, threshold dependence and localisation in cluster inference , 2009, NeuroImage.

[4]  Jeffrey N. Rouder,et al.  Default Bayes factors for ANOVA designs , 2012 .

[5]  Alessio Fracasso,et al.  Time course of spatiotopic updating across saccades , 2019, Proceedings of the National Academy of Sciences.

[6]  Paul B Hibbard,et al.  Consciousness of the first order in blindsight , 2010, Proceedings of the National Academy of Sciences.

[7]  Tijl Grootswagers,et al.  The representational dynamics of visual objects in rapid serial visual processing streams , 2018, NeuroImage.

[8]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[9]  Anders M. Dale,et al.  Generalized Laminar Population Analysis (gLPA) for Interpretation of Multielectrode Data from Cortex , 2016, Front. Neuroinform..

[10]  Daniel Guitton,et al.  Two distinct types of remapping in primate cortical area V4 , 2016, Nature Communications.

[11]  R. M. Siegel,et al.  Encoding of spatial location by posterior parietal neurons. , 1985, Science.

[12]  T. Sejnowski,et al.  Spatial Transformations in the Parietal Cortex Using Basis Functions , 1997, Journal of Cognitive Neuroscience.

[13]  Christopher C. Pack,et al.  Context dependence of receptive field remapping in superior colliculus. , 2011, Journal of neurophysiology.

[14]  C. Cierpka,et al.  Particle imaging techniques for volumetric three-component (3D3C) velocity measurements in microfluidics , 2011, Journal of Visualization.

[15]  Julie D. Golomb.  Remapping locations and features across saccades: a dual-spotlight theory of attentional updating. , 2019, Current opinion in psychology.

[16]  Stefan Treue,et al.  An Attention-Sensitive Memory Trace in Macaque MT Following Saccadic Eye Movements , 2016, PLoS biology.

[17]  R. Gregory.  The Most Expensive Painting in the World , 2007, Perception.

[18]  Johan Wagemans,et al.  Transsaccadic identification of highly similar artificial shapes. , 2009, Journal of vision.

[19]  Maria Concetta Morrone,et al.  Nonretinotopic visual processing in the brain , 2015, Visual Neuroscience.

[20]  Adam P. Morris,et al.  A Stable Visual World in Primate Primary Visual Cortex , 2019, Current Biology.

[21]  Michael A. Cohen,et al.  What is the Bandwidth of Perceptual Experience? , 2016, Trends in Cognitive Sciences.

[22]  S. Taulu,et al.  Applications of the signal space separation method , 2005, IEEE Transactions on Signal Processing.

[23]  A. Pegna,et al.  Affective blindsight relies on low spatial frequencies , 2017, Neuropsychologia.

[24]  Floris P. de Lange,et al.  Visual working memory representations in visual and parietal cortex do not remap after eye movements , 2019, bioRxiv.

[25]  Nancy Kanwisher,et al.  No Evidence for Automatic Remapping of Stimulus Features or Location Found with fMRI , 2016, Front. Syst. Neurosci..

[26]  Lawrence Weiskrantz,et al.  Spatial channels of visual processing in cortical blindness , 2003, The European journal of neuroscience.

[27]  Alfonso Caramazza,et al.  Continuous perception of motion and shape across saccadic eye movements. , 2010, Journal of vision.

[28]  Denis G. Pelli,et al.  ECVP '07 Abstracts , 2007, Perception.

[29]  Dimitrios Pantazis,et al.  Can visual information encoded in cortical columns be decoded from magnetoencephalography data in humans? , 2015, NeuroImage.

[30]  Frans W Cornelissen,et al.  The Eyelink Toolbox: Eye tracking with MATLAB and the Psychophysics Toolbox , 2002, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[31]  E. Halgren,et al.  Top-down facilitation of visual recognition. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[32]  Eckart Zimmermann,et al.  Spatiotopic updating of visual feature information. , 2017, Journal of vision.

[33]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[34]  G. Rhodes,et al.  How is facial expression coded? , 2015, Journal of vision.

[35]  W. E. Collins,et al.  Integrating pictorial information across eye movements. , 1984, Journal of experimental psychology. General.

[36]  L. Marshall,et al.  Evidence for Optimal Integration of Visual Feature Representations across Saccades , 2015, The Journal of Neuroscience.

[37]  M. Morrone,et al.  Vision During Saccadic Eye Movements. , 2018, Annual review of vision science.

[38]  Kae Nakamura,et al.  Updating of the visual representation in monkey striate and extrastriate cortex during saccades , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[39]  D G Pelli,et al.  The VideoToolbox software for visual psychophysics: transforming numbers into movies. , 1997, Spatial vision.

[40]  Edward K. Vogel,et al.  A Soft Handoff of Attention between Cerebral Hemispheres , 2014, Current Biology.

[41]  James W Bisley,et al.  Remapping, Spatial Stability, and Temporal Continuity: From the Pre-Saccadic to Postsaccadic Representation of Visual Space in LIP. , 2016, Cerebral cortex.

[42]  J. Henderson,et al.  High-level scene perception. , 1999, Annual review of psychology.

[43]  D. E. Irwin.  Memory for position and identity across eye movements. , 1992 .

[44]  S. Dehaene,et al.  Characterizing the dynamics of mental representations: the temporal generalization method , 2014, Trends in Cognitive Sciences.

[45]  J Douglas Crawford,et al.  Cortical mechanisms for trans-saccadic memory and integration of multiple object features , 2011, Philosophical Transactions of the Royal Society B: Biological Sciences.

[46]  Marcus Nyström,et al.  An adaptive algorithm for fixation, saccade, and glissade detection in eyetracking data , 2010, Behavior research methods.

[47]  Patrick Cavanagh,et al.  Feature-based attention across saccades: Pop-out in color search is spatiotopic , 2018, Attention, perception & psychophysics.

[48]  S. Taulu,et al.  Suppression of Interference and Artifacts by the Signal Space Separation Method , 2003, Brain Topography.

[49]  Justin M. Ales,et al.  The steady-state visual evoked potential in vision research: A review. , 2015, Journal of vision.

[50]  J. Bisley,et al.  Psychophysical evidence for spatiotopic processing in area MT in a short-term memory for motion task. , 2009, Journal of neurophysiology.

[51]  Chih-Yang Chen,et al.  Spatial frequency sensitivity in macaque midbrain , 2018, Nature Communications.

[52]  B. Wandell,et al.  Visual Field Maps in Human Cortex , 2007, Neuron.

[53]  David E. Irwin,et al.  Evidence against visual integration across saccadic eye movements , 1983, Perception & psychophysics.

[54]  J. Douglas Crawford,et al.  Trans-saccadic interactions in human parietal and occipital cortex during the retention and comparison of object orientation , 2016, Cortex.

[55]  S. Taulu,et al.  Presentation of electromagnetic multichannel data: The signal space separation method , 2005 .

[56]  P. Cavanagh,et al.  Spatial constancy of attention across eye movements is mediated by the presence of visual objects , 2015, Attention, Perception, & Psychophysics.

[57]  S. Gori,et al.  How the visual aspects can be crucial in reading acquisition? The intriguing case of crowding and developmental dyslexia. , 2015, Journal of vision.

[58]  M. A. Goodale,et al.  What is the best fixation target? The effect of target shape on stability of fixational eye movements , 2013, Vision Research.

[59]  Thomas Wachtler,et al.  Perceptual evidence for saccadic updating of color stimuli. , 2008, Journal of vision.

[60]  Christopher B. Currie,et al.  Visual stability across saccades while viewing complex pictures. , 1995, Journal of experimental psychology. Human perception and performance.

[61]  Patrick Cavanagh,et al.  Decoding Trans-Saccadic Memory , 2017, The Journal of Neuroscience.

[62]  Ilona M. Bloem,et al.  Attentional modulation interacts with orientation anisotropies in contrast perception. , 2017, Journal of vision.

[63]  K. Hoffmann,et al.  Neural Dynamics of Saccadic Suppression , 2009, Journal of Neuroscience.

[64]  Alexandria C. Marino,et al.  Saccades Trigger Predictive Updating of Attentional Topography in Area V4 , 2018, Neuron.

[65]  Carol L Colby,et al.  Shape selectivity and remapping in dorsal stream visual area LIP. , 2014, Journal of neurophysiology.

[66]  M. Sommer,et al.  Visual continuity across saccades is influenced by expectations. , 2016, Journal of vision.

[67]  P. Cavanagh,et al.  Visual stability based on remapping of attention pointers , 2010, Trends in Cognitive Sciences.

[68]  H. Spekreijse,et al.  Correlates of transsaccadic integration in the primary visual cortex of the monkey. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[69]  Scott L. Fairhall,et al.  Spatiotopic updating across saccades revealed by spatially-specific fMRI adaptation , 2017, NeuroImage.

[70]  Sang Chul Chong,et al.  The background is remapped across saccades , 2013, Experimental Brain Research.

[71]  J. Douglas Crawford,et al.  Visual memory capacity in transsaccadic integration , 2007, Experimental Brain Research.

[72]  James V. Haxby,et al.  CoSMoMVPA: Multi-Modal Multivariate Pattern Analysis of Neuroimaging Data in Matlab/GNU Octave , 2016, bioRxiv.

[73]  D. Melcher.  Spatiotopic Transfer of Visual-Form Adaptation across Saccadic Eye Movements , 2005, Current Biology.

[74]  David Melcher,et al.  The peripheral preview effect with faces: Combined EEG and eye-tracking suggests multiple stages of trans-saccadic predictive and non-predictive processing , 2019, NeuroImage.

[75]  Tirin Moore,et al.  Dissonant Representations of Visual Space in Prefrontal Cortex during Eye Movements , 2018, Cell reports.

[76]  Eero P. Simoncelli,et al.  Near-optimal integration of orientation information across saccades. , 2015, Journal of vision.

[77]  J R Duhamel,et al.  The updating of the representation of visual space in parietal cortex by intended eye movements. , 1992, Science.

[78]  Alexander C. Schütz,et al.  Trans-saccadic integration of peripheral and foveal feature information is close to optimal. , 2015, Journal of vision.

[79]  C. Colby,et al.  Trans-saccadic perception , 2008, Trends in Cognitive Sciences.

[80]  P. H. Schiller,et al.  Spatial frequency and orientation tuning dynamics in area V1 , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[81]  Nancy Kanwisher,et al.  Feature-Binding Errors After Eye Movements and Shifts of Attention , 2014, Psychological science.

[82]  Urte Roeber,et al.  Lengthy suppression from similar stimuli during rapid serial visual presentation. , 2011, Journal of vision.

[83]  Pavan Ramkumar,et al.  Feature-Specific Information Processing Precedes Concerted Activation in Human Visual Cortex , 2013, The Journal of Neuroscience.

[84]  Julie D. Golomb,et al.  Attentional Facilitation throughout Human Visual Cortex Lingers in Retinotopic Coordinates after Eye Movements , 2010, The Journal of Neuroscience.

[85]  R. Oostenveld,et al.  Nonparametric statistical testing of EEG- and MEG-data , 2007, Journal of Neuroscience Methods.

[86]  D. E. Irwin,et al.  Causal Inference for Spatial Constancy across Saccades , 2016, PLoS Comput. Biol..