Overlapping neural representations for the position of visible and imagined objects

Humans can covertly track the position of an object, even if the object is temporarily occluded. What are the neural mechanisms underlying our capacity to track moving objects when there is no physical stimulus for the brain to track? One possibility is that the brain “fills-in” information about imagined objects using internally generated representations similar to those generated by feed-forward perceptual mechanisms. Alternatively, the brain might deploy a higher order mechanism, for example using an object tracking model that integrates visual signals and motion dynamics (Kwon et al., 2015). In the present study, we used electroencephalography (EEG) and time-resolved multivariate pattern analyses to investigate the spatial processing of visible and imagined objects. Participants tracked an object that moved in discrete steps around fixation, occupying six consecutive locations. They were asked to imagine that the object continued on the same trajectory after it disappeared and move their attention to the corresponding positions. Time-resolved decoding of EEG data revealed that the location of the visible stimuli could be decoded shortly after image onset, consistent with early retinotopic visual processes. For processing of unseen/imagined positions, the patterns of neural activity resembled stimulus-driven mid-level visual processes, but were detected earlier than perceptual mechanisms, implicating an anticipatory and more variable tracking mechanism. Encoding models revealed that spatial representations were much weaker for imagined than visible stimuli. Monitoring the position of imagined objects thus utilises similar perceptual and attentional processes as monitoring objects that are actually present, but with different temporal dynamics. These results indicate that internally generated representations rely on top-down processes, and their timing is influenced by the predictability of the stimulus. All data and analysis code for this study are available at https://osf.io/8v47t/.

[1]  Chris I. Baker,et al.  Deconstructing multivariate decoding for the study of brain function , 2017, NeuroImage.

[2]  G. V. Simpson,et al.  Anticipatory Biasing of Visuospatial Attention Indexed by Retinotopically Specific α-Bank Electroencephalography Increases over Occipital Cortex , 2000, The Journal of Neuroscience.

[3]  Robert Oostenveld,et al.  The five percent electrode system for high-resolution EEG and ERP measurements , 2001, Clinical Neurophysiology.

[4]  T. Carlson,et al.  High temporal resolution decoding of object position and category. , 2011, Journal of vision.

[5]  Floris P. de Lange,et al.  Differential temporal dynamics during visual imagery and perception , 2017, bioRxiv.

[6]  Karl J. Friston,et al.  Where bottom-up meets top-down: neuronal interactions during perception and imagery. , 2004, Cerebral cortex.

[7]  Giulio Tononi,et al.  Reversal of cortical information flow during visual imagery as compared to visual perception , 2014, NeuroImage.

[8]  Luca Ambrogioni,et al.  Neural dynamics of perceptual inference and its reversal during imagery , 2019, bioRxiv.

[9]  David W. Sutterer,et al.  The topography of alpha-band activity tracks the content of spatial working memory. , 2016, Journal of neurophysiology.

[10]  M. Posner,et al.  Orienting of Attention* , 1980, The Quarterly journal of experimental psychology.

[11]  O. Mimura.  [Eye movements]. , 1992, Nippon Ganka Gakkai zasshi.

[12]  D Le Bihan,et al.  Activation of human primary visual cortex during visual recall: a magnetic resonance imaging study. , 1993, Proceedings of the National Academy of Sciences of the United States of America.

[13]  Denis G. Pelli,et al.  ECVP '07 Abstracts , 2007, Perception.

[14]  B. Fischer,et al.  Human express saccades: extremely short reaction times of goal directed eye movements , 2004, Experimental Brain Research.

[15]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[16]  A. Dale,et al.  Functional analysis of primary visual cortex (V1) in humans. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[17]  Thomas Serre,et al.  Reading the mind's eye: Decoding category information during mental imagery , 2010, NeuroImage.

[18]  G. Mangun.  Neural mechanisms of visual selective attention. , 1995, Psychophysiology.

[19]  Radoslaw Martin Cichy,et al.  Visual Imagery and Perception Share Neural Representations in the Alpha Frequency Band , 2020, Current Biology.

[20]  S. Kosslyn,et al.  Visual Mental Imagery Activates Topographically Organized Visual Cortex: PET Investigations , 1993, Journal of Cognitive Neuroscience.

[21]  Matthew F. Tang,et al.  Neural dynamics of the attentional blink revealed by encoding orientation selectivity during rapid visual presentation , 2019, Nature Communications.

[22]  Thomas A. Carlson,et al.  Spatial and feature-selective attention have distinct effects on population-level tuning , 2019, bioRxiv.

[23]  J. Serences,et al.  Near-Real-Time Feature-Selective Modulations in Human Cortex , 2013, Current Biology.

[24]  Hinze Hogendoorn,et al.  Predictions drive neural representations of visual events ahead of incoming sensory information , 2020, Proceedings of the National Academy of Sciences.

[25]  Cooper A. Smout,et al.  Prediction Error and Repetition Suppression Have Distinct Effects on Neural Representations of Visual Information , 2017 .

[26]  M. Lee,et al.  Statistical Evidence in Experimental Psychology , 2011, Perspectives on psychological science : a journal of the Association for Psychological Science.

[27]  Marcel A. J. van Gerven,et al.  Eye movements explain decodability during perception and cued attention in MEG , 2019, NeuroImage.

[28]  Jesse L. Breedlove,et al.  Generative Feedback Explains Distinct Brain Activity Codes for Seen and Mental Images , 2020, Current Biology.

[29]  P. Berg,et al.  Ocular artifacts in EEG and event-related potentials I: Scalp topography , 2005, Brain Topography.

[30]  John T. Serences,et al.  Using Human Neuroimaging to Examine Top-down Modulation of Visual Perception , 2015 .

[31]  L. M. M.-T..  Theory of Probability , 1929, Nature.

[32]  Jeffrey N. Rouder,et al.  Bayesian t tests for accepting and rejecting the null hypothesis , 2009, Psychonomic bulletin & review.

[33]  E. Wagenmakers,et al.  A default Bayesian hypothesis test for correlations and partial correlations , 2012, Psychonomic bulletin & review.

[34]  I. Toni,et al.  Shared Representations for Working Memory and Mental Imagery in Early Visual Cortex , 2013, Current Biology.

[35]  E. Wagenmakers.  A practical solution to the pervasive problems ofp values , 2007, Psychonomic bulletin & review.

[36]  Tijl Grootswagers,et al.  The influence of image masking on object representations during rapid serial visual presentation , 2019 .

[37]  J. Serences,et al.  Optimal Deployment of Attentional Gain during Fine Discriminations , 2012, The Journal of Neuroscience.

[38]  S. Dehaene,et al.  Characterizing the dynamics of mental representations: the temporal generalization method , 2014, Trends in Cognitive Sciences.

[39]  P. Cavanagh,et al.  The Spatial Resolution of Visual Attention , 2001, Cognitive Psychology.

[40]  Joel Pearson,et al.  The human imagination: the cognitive neuroscience of visual mental imagery , 2019, Nature Reviews Neuroscience.

[41]  Tijl Grootswagers,et al.  The representational dynamics of visual objects in rapid serial visual processing streams , 2018 .

[42]  Susan G. Wardle,et al.  Decoding Dynamic Brain Patterns from Evoked Responses: A Tutorial on Multivariate Pattern Analysis Applied to Time Series Neuroimaging Data , 2016, Journal of Cognitive Neuroscience.

[43]  Pim Mostert,et al.  Differential temporal dynamics during visual imagery and perception , 2018, eLife.

[44]  Leslie G. Ungerleider,et al.  Visual Imagery of Famous Faces: Effects of Memory and Attention Revealed by fMRI , 2002, NeuroImage.

[45]  E. Halgren,et al.  Source estimates for MEG/EEG visual evoked responses constrained by multiple, retinotopically‐mapped stimulus locations , 2009, Human brain mapping.

[46]  Duje Tadin,et al.  Unifying account of visual motion and position perception , 2015, Proceedings of the National Academy of Sciences.

[47]  Karl J. Friston,et al.  Distinct Top-down and Bottom-up Brain Connectivity During Visual Perception and Imagery , 2017, Scientific Reports.

[48]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[49]  D. Heeger,et al.  Decoding and Reconstructing Color from Responses in Human Visual Cortex , 2009, The Journal of Neuroscience.

[50]  Cooper A. Smout,et al.  Prediction error and repetition suppression have distinct effects on neural representations of visual information , 2017, bioRxiv.

[51]  Marina Schmid,et al.  An Introduction To The Event Related Potential Technique , 2016 .

[52]  Anthony N. Burkitt,et al.  Predictive coding of visual object position ahead of moving objects revealed by time-resolved EEG decoding , 2018, NeuroImage.

[53]  James V. Haxby,et al.  CoSMoMVPA: Multi-Modal Multivariate Pattern Analysis of Neuroimaging Data in Matlab/GNU Octave , 2016, bioRxiv.

[54]  A. Zellner,et al.  Posterior odds ratios for selected regression hypotheses , 1980 .

[55]  John T. Serences,et al.  Attention modulates spatial priority maps in the human occipital, parietal and frontal cortices , 2013, Nature Neuroscience.

[56]  Antigona Martínez,et al.  Source analysis of event-related cortical activity during visuo-spatial attention. , 2003, Cerebral cortex.

[57]  Chris I. Baker,et al.  Disentangling visual imagery and perception of real-world objects , 2012, NeuroImage.

[58]  D G Pelli,et al.  The VideoToolbox software for visual psychophysics: transforming numbers into movies. , 1997, Spatial vision.

[59]  Bence Nanay,et al.  Perception and imagination: amodal perception as mental imagery , 2010 .

[60]  E. Wagenmakers,et al.  An Introduction to Model-Based Cognitive Neuroscience , 2015, Springer New York.

[61]  Cooper A. Smout,et al.  Attention promotes the neural encoding of prediction errors , 2019, bioRxiv.

[62]  Z. Dienes.  Bayesian Versus Orthodox Statistics: Which Side Are You On? , 2011, Perspectives on psychological science : a journal of the Association for Psychological Science.