Decoding Trans-Saccadic Memory

We examine whether peripheral information at a planned saccade target affects immediate postsaccadic processing at the fovea on saccade landing. Current neuroimaging research suggests that presaccadic stimulation has a late effect on postsaccadic processing, in contrast to the early effect seen in behavioral studies. Human participants (both male and female) were instructed to saccade toward a face or a house that, on different trials, remained the same, changed, or disappeared during the saccade. We used a multivariate pattern analysis of electroencephalography data to decode face versus house processing directly after the saccade. The classifier was trained on separate trials without a saccade, where a house or face was presented at the fovea. When the saccade target remained the same across the saccade, we could reliably decode the target 123 ms after saccade offset. In contrast, when the target was changed during the saccade, the new target was decoded at a later time-point, 151 ms after saccade offset. The “same” condition advantage suggests that congruent presaccadic information facilitates processing of the postsaccadic stimulus compared with incongruent information. Finally, the saccade target could be decoded above chance even when it had been removed during the saccade, albeit with a slower time course (162 ms) and poorer signal strength. These findings indicate that information about the (peripheral) presaccadic stimulus is transferred across the saccade so that it becomes quickly available and influences processing at its expected new retinal position (the fovea). SIGNIFICANCE STATEMENT Here we provide neural evidence for early information transfer across saccades. Specifically, we examined the effect of presaccadic sensory information on the initial neuronal processing of a postsaccadic stimuli. Using electroencephalography and multivariate pattern analysis, we found the following: (1) that the identity of the presaccadic stimulus modulated the postsaccadic latency of stimulus relevant information; and (2) that a saccadic neural marker for a saccade target stimulus could be detected even when the stimulus had been removed during saccade. These results demonstrate that information about the peripheral presaccadic stimulus was transferred across the saccade and influenced processing at a new retinal position (the fovea) directly after the saccade landed.

[1]  Urte Roeber,et al.  Lengthy suppression from similar stimuli during rapid serial visual presentation. , 2011, Journal of vision.

[2]  H. Deubel,et al.  Visual attention and saccadic eye movements: Evidence for obligatory and selective spatial coupling , 1995 .

[3]  T. Carlson,et al.  High temporal resolution decoding of object position and category. , 2011, Journal of vision.

[4]  M D Anes,et al.  Roles of object-file review and type priming in visual identification within and across eye fixations. , 1994, Journal of experimental psychology. Human perception and performance.

[5]  J. Shaffer Multiple Hypothesis Testing , 1995 .

[6]  C. Cierpka,et al.  Particle imaging techniques for volumetric three-component (3D3C) velocity measurements in microfluidics , 2011, Journal of Visualization.

[7]  Sabine Kastner,et al.  Topographic maps in human frontal cortex revealed in memory-guided saccade and spatial working-memory tasks. , 2007, Journal of neurophysiology.

[8]  Eero P. Simoncelli,et al.  Near-optimal integration of orientation information across saccades. , 2015, Journal of vision.

[9]  P. Cavanagh,et al.  Predictive remapping of attention across eye movements , 2011, Nature Neuroscience.

[10]  Ronald A. Rensink,et al.  Change-blindness as a result of ‘mudsplashes’ , 1999, Nature.

[11]  Justin M. Ales,et al.  The steady-state visual evoked potential in vision research: A review. , 2015, Journal of vision.

[12]  Patrick Cavanagh,et al.  Spatiotopic apparent motion reveals local variations in space constancy. , 2011, Journal of vision.

[13]  D. Melcher Selective attention and the active remapping of object features in trans-saccadic perception , 2009, Vision Research.

[14]  Eckart Zimmermann,et al.  Spatiotopic Adaptation in Visual Areas , 2016, The Journal of Neuroscience.

[15]  E. Maris,et al.  Prior Expectation Mediates Neural Adaptation to Repeated Sounds in the Auditory Cortex: An MEG Study , 2011, The Journal of Neuroscience.

[16]  B. Bridgeman,et al.  Postsaccadic target blanking prevents saccadic suppression of image displacement , 1996, Vision Research.

[17]  David Whitney,et al.  Saccadic remapping of object-selective information , 2014, Attention, perception & psychophysics.

[18]  Patrick Cavanagh,et al.  Transsaccadic perceptual fusion. , 2017, Journal of vision.

[19]  Scott L. Fairhall,et al.  Spatiotopic updating across saccades revealed by spatially-specific fMRI adaptation , 2017, NeuroImage.

[20]  J. Crawford,et al.  Transsaccadic integration of visual features in a line intersection task , 2006, Experimental Brain Research.

[21]  Thomas Wachtler,et al.  Perceptual evidence for saccadic updating of color stimuli. , 2008, Journal of vision.

[22]  J. Grimes On the failure to detect changes in scenes across saccades. , 1996 .

[23]  Alexander C. Schütz,et al.  Eye movements and perception: a selective review. , 2011, Journal of vision.

[24]  Alfonso Caramazza,et al.  Continuous perception of motion and shape across saccadic eye movements. , 2010, Journal of vision.

[25]  Lars Muckli,et al.  Decoding Sound and Imagery Content in Early Visual Cortex , 2014, Current Biology.

[26]  Sébastien M. Crouzet,et al.  Fast saccades toward faces: face detection in just 100 ms. , 2010, Journal of vision.

[27]  D. Ballard,et al.  Task constraints in visual working memory , 1997, Vision Research.

[28]  W. Wolf,et al.  Interaction of pre- and postsaccadic patterns having the same coordinates in space , 1980, Vision Research.

[29]  G. McConkie,et al.  Identifying the Span of the Effective Stimulus in Reading. Final Report. , 1974 .

[30]  P. Cavanagh,et al.  Visual stability based on remapping of attention pointers , 2010, Trends in Cognitive Sciences.

[31]  Alexander C. Schütz,et al.  Trans-saccadic integration of peripheral and foveal feature information is close to optimal. , 2015, Journal of vision.

[32]  Patrick Cavanagh,et al.  Allocation of attention across saccades. , 2013, Journal of neurophysiology.

[33]  G. Rhodes,et al.  How is facial expression coded? , 2015, Journal of vision.

[34]  Janneke F. M. Jehee,et al.  Less Is More: Expectation Sharpens Representations in the Primary Visual Cortex , 2012, Neuron.

[35]  David E. Irwin,et al.  Failure to integrate information from successive fixations. , 1983, Science.

[36]  Trafton Drew,et al.  Remapping attention in multiple object tracking , 2011, Vision Research.

[37]  Peter J Bex,et al.  Integrating Retinotopic Features in Spatiotopic Coordinates , 2014, The Journal of Neuroscience.

[38]  S. Gori,et al.  How the visual aspects can be crucial in reading acquisition? The intriguing case of crowding and developmental dyslexia. , 2015, Journal of vision.

[39]  Rajesh P. N. Rao,et al.  Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. , 1999 .

[40]  T. Picton,et al.  The N1 wave of the human electric and magnetic response to sound: a review and an analysis of the component structure. , 1987, Psychophysiology.

[41]  Joel Z. Leibo,et al.  The dynamics of invariant object recognition in the human visual system. , 2014, Journal of neurophysiology.

[42]  Ronald A. Rensink,et al.  Change blindness: past, present, and future , 2005, Trends in Cognitive Sciences.

[43]  R. McPeek,et al.  The eye dominates in guiding attention during simultaneous eye and hand movements. , 2011, Journal of vision.

[44]  Lars Muckli,et al.  Predictive feedback to V1 dynamically updates with sensory input , 2017, Scientific Reports.

[45]  Harry Singer,et al.  Theoretical models and processes of reading , 1976 .

[46]  Elizabeth R Schotter,et al.  Parallel object activation and attentional gating of information: evidence from eye movements in the multiple object naming paradigm. , 2013, Journal of experimental psychology. Learning, memory, and cognition.

[47]  Ashleigh M. Richard,et al.  Understanding the function of visual short-term memory: transsaccadic memory, object correspondence, and gaze correction. , 2008, Journal of experimental psychology. General.

[48]  Mariano Sigman,et al.  Looking for a face in the crowd: Fixation-related potentials in an eye-movement visual search task , 2014, NeuroImage.

[49]  J. Douglas Crawford,et al.  Trans-saccadic interactions in human parietal and occipital cortex during the retention and comparison of object orientation , 2016, Cortex.

[50]  Sébastien M. Crouzet,et al.  Taste Quality Decoding Parallels Taste Sensations , 2015, Current Biology.

[51]  Leila Reddy,et al.  Multivoxel Object Representations in Adult Human Visual Cortex Are Flexible: An Associative Learning Study , 2016, Journal of Cognitive Neuroscience.

[52]  Y. Nakayama,et al.  The history of JOV of 10 years , 2008, J. Vis..

[53]  R. Radach,et al.  Transsaccadic memory for visual object detail , 2003 .

[54]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[55]  J. Richards Cognitive neuroscience of attention : a developmental perspective , 1998 .

[56]  David E. Irwin,et al.  Integrating visual information from successive fixations. , 1982, Science.

[57]  Maria Concetta Morrone,et al.  Spatiotopic neural representations develop slowly across saccades , 2013, Current Biology.

[58]  M. Goldberg,et al.  The representation of visual salience in monkey parietal cortex , 1998, Nature.

[59]  I. Motoyoshi,et al.  Adaptation-induced blindness to sluggish stimuli. , 2010, Journal of vision.

[60]  C. Colby,et al.  Trans-saccadic perception , 2008, Trends in Cognitive Sciences.

[61]  A. Trehub,et al.  Neuronal models for cognitive processes: networks for learning, perception and imagination. , 1977, Journal of theoretical biology.

[62]  Reinhold Kliegl,et al.  Trans-saccadic parafoveal preview benefits in fluent reading: A study with fixation-related brain potentials , 2012, NeuroImage.

[63]  J R Duhamel,et al.  The updating of the representation of visual space in parietal cortex by intended eye movements. , 1992, Science.

[64]  Lars Muckli,et al.  Transfer of Predictive Signals Across Saccades , 2012, Front. Psychology.

[65]  Lars Riecke,et al.  Parietal and superior frontal visuospatial maps activated by pointing and saccades , 2007, NeuroImage.

[66]  O. Dimigen,et al.  Dissociating parafoveal preview benefit and parafovea-on-fovea effects during reading: A combined eye tracking and EEG study. , 2016, Psychophysiology.

[67]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[68]  Susan G. Wardle,et al.  Decoding Dynamic Brain Patterns from Evoked Responses: A Tutorial on Multivariate Pattern Analysis Applied to Time Series Neuroimaging Data , 2016, Journal of Cognitive Neuroscience.

[69]  Peter H. Schiller,et al.  Neural Control of Visually Guided Eye Movements , 2012 .

[70]  Muriel Boucart,et al.  Finding faces, animals, and vehicles in far peripheral vision. , 2016, Journal of vision.

[71]  Werner Sommer,et al.  Lateralization of posterior alpha EEG reflects the distribution of spatial attention during saccadic reading. , 2017, Psychophysiology.

[72]  M. Sereno,et al.  Mapping of Contralateral Space in Retinotopic Coordinates by a Parietal Cortical Area in Humans , 2001, Science.

[73]  K. Rayner,et al.  Eye movements and word skipping during reading: effects of word length and predictability. , 2011, Journal of experimental psychology. Human perception and performance.

[74]  Maxi San Miguel,et al.  Joint effect of ageing and multilayer structure prevents ordering in the voter model , 2017, Scientific Reports.

[75]  Keith Rayner,et al.  Transsaccadic processing: stability, integration, and the potential role of remapping , 2015, Attention, perception & psychophysics.

[76]  Caspar M. Schwiedrzik,et al.  Stimulus Predictability Reduces Responses in Primary Visual Cortex , 2010, The Journal of Neuroscience.