Now you see it, now you don't: Overlapping neural representations for the position of visible and invisible objects

Humans can covertly track the position of an object, even if the object is temporarily occluded. What are the neural mechanisms underlying our capacity to track moving objects when there is no physical stimulus for the brain to track? One possibility is that the brain "fills-in" information about imagined objects using internally generated representations similar to those generated by feed-forward perceptual mechanisms. Alternatively, the brain might deploy a higher order mechanism, for example using an object tracking model that integrates visual signals and motion dynamics (Kwon et al., 2015). In the present study, we used electroencephalography (EEG) and time-resolved multivariate pattern analyses to investigate the spatial processing of visible and imagined objects. Participants tracked an object that moved in discrete steps around fixation, occupying six consecutive locations. They were asked to imagine that the object continued on the same trajectory after it disappeared and move their attention to the corresponding positions. Time-resolved decoding of EEG data revealed that the location of the visible stimuli could be decoded shortly after image onset, consistent with early retinotopic visual processes. For processing of unseen/imagined positions, the patterns of neural activity resembled stimulus-driven mid-level visual processes, but were detected earlier than perceptual mechanisms, implicating an anticipatory and more variable tracking mechanism. Encoding models revealed that spatial representations were much less precise for imagined than visible stimuli. Monitoring the position of imagined objects thus utilises similar perceptual processes as processing objects that are actually present, but with different temporal dynamics. These results indicate that internally generated representations rely on top-down processes, and their timing is influenced by the predictability of the stimulus. All data and analysis code for this study are available at https://osf.io/8v47t/.

[1]  Antigona Martínez,et al.  Source analysis of event-related cortical activity during visuo-spatial attention. , 2003, Cerebral cortex.

[2]  B. Fischer,et al.  Human express saccades: extremely short reaction times of goal directed eye movements , 2004, Experimental Brain Research.

[3]  G. Mangun Neural mechanisms of visual selective attention. , 1995, Psychophysiology.

[4]  E. Wagenmakers A practical solution to the pervasive problems ofp values , 2007, Psychonomic bulletin & review.

[5]  Tijl Grootswagers,et al.  The influence of image masking on object representations during rapid serial visual presentation , 2019 .

[6]  Chris I. Baker,et al.  Disentangling visual imagery and perception of real-world objects , 2012, NeuroImage.

[7]  D G Pelli,et al.  The VideoToolbox software for visual psychophysics: transforming numbers into movies. , 1997, Spatial vision.

[8]  P. Cavanagh,et al.  The Spatial Resolution of Visual Attention , 2001, Cognitive Psychology.

[9]  Tijl Grootswagers,et al.  The representational dynamics of visual objects in rapid serial visual processing streams , 2018, NeuroImage.

[10]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[11]  Susan G. Wardle,et al.  Decoding Dynamic Brain Patterns from Evoked Responses: A Tutorial on Multivariate Pattern Analysis Applied to Time Series Neuroimaging Data , 2016, Journal of Cognitive Neuroscience.

[12]  Joel Pearson,et al.  The human imagination: the cognitive neuroscience of visual mental imagery , 2019, Nature Reviews Neuroscience.

[13]  Duje Tadin,et al.  Unifying account of visual motion and position perception , 2015, Proceedings of the National Academy of Sciences.

[14]  I. Toni,et al.  Shared Representations for Working Memory and Mental Imagery in Early Visual Cortex , 2013, Current Biology.

[15]  D. Heeger,et al.  Decoding and Reconstructing Color from Responses in Human Visual Cortex , 2009, The Journal of Neuroscience.

[16]  S. Kosslyn,et al.  Visual Mental Imagery Activates Topographically Organized Visual Cortex: PET Investigations , 1993, Journal of Cognitive Neuroscience.

[17]  E. Wagenmakers,et al.  A default Bayesian hypothesis test for correlations and partial correlations , 2012, Psychonomic bulletin & review.

[18]  Thomas A. Carlson,et al.  Spatial and feature-selective attention have distinct effects on population-level tuning , 2019, bioRxiv.

[19]  Hinze Hogendoorn,et al.  Predictions drive neural representations of visual events ahead of incoming sensory information , 2020, Proceedings of the National Academy of Sciences.

[20]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[21]  J. Serences,et al.  Optimal Deployment of Attentional Gain during Fine Discriminations , 2012, The Journal of Neuroscience.

[22]  Luca Ambrogioni,et al.  Neural dynamics of perceptual inference and its reversal during imagery , 2019, bioRxiv.

[23]  Bence Nanay,et al.  Perception and imagination: amodal perception as mental imagery , 2010 .

[24]  Cooper A. Smout,et al.  Prediction Error and Repetition Suppression Have Distinct Effects on Neural Representations of Visual Information , 2017 .

[25]  D Le Bihan,et al.  Activation of human primary visual cortex during visual recall: a magnetic resonance imaging study. , 1993, Proceedings of the National Academy of Sciences of the United States of America.

[26]  Cooper A. Smout,et al.  Attention promotes the neural encoding of prediction errors , 2019, bioRxiv.

[27]  Leslie G. Ungerleider,et al.  Visual Imagery of Famous Faces: Effects of Memory and Attention Revealed by fMRI , 2002, NeuroImage.

[28]  Alexander C. Schütz,et al.  Eye movements and perception: a selective review. , 2011, Journal of vision.

[29]  Karl J. Friston,et al.  Distinct Top-down and Bottom-up Brain Connectivity During Visual Perception and Imagery , 2017, Scientific Reports.

[30]  S. Luck An Introduction to the Event-Related Potential Technique , 2005 .

[31]  Anthony N. Burkitt,et al.  Predictive coding of visual object position ahead of moving objects revealed by time-resolved EEG decoding , 2018, NeuroImage.

[32]  James V. Haxby,et al.  CoSMoMVPA: Multi-Modal Multivariate Pattern Analysis of Neuroimaging Data in Matlab/GNU Octave , 2016, bioRxiv.

[33]  A. Zellner,et al.  Posterior odds ratios for selected regression hypotheses , 1980 .

[34]  Jesse L. Breedlove,et al.  Generative Feedback Explains Distinct Brain Activity Codes for Seen and Mental Images , 2020, Current Biology.

[35]  John T. Serences,et al.  Attention modulates spatial priority maps in the human occipital, parietal and frontal cortices , 2013, Nature Neuroscience.

[36]  R. Gregory The Most Expensive Painting in the World , 2007, Perception.

[37]  Z. Dienes Bayesian Versus Orthodox Statistics: Which Side Are You On? , 2011, Perspectives on psychological science : a journal of the Association for Psychological Science.

[38]  Matthew F. Tang,et al.  Neural dynamics of the attentional blink revealed by encoding orientation selectivity during rapid visual presentation , 2020, Nature Communications.

[39]  Jeffrey N. Rouder,et al.  Bayesian t tests for accepting and rejecting the null hypothesis , 2009, Psychonomic bulletin & review.

[40]  M. Lee,et al.  Statistical Evidence in Experimental Psychology , 2011, Perspectives on psychological science : a journal of the Association for Psychological Science.

[41]  Marcel A. J. van Gerven,et al.  Eye movements explain decodability during perception and cued attention in MEG , 2019, NeuroImage.

[42]  P. Berg,et al.  Ocular artifacts in EEG and event-related potentials I: Scalp topography , 2005, Brain Topography.

[43]  John T. Serences,et al.  Using Human Neuroimaging to Examine Top-down Modulation of Visual Perception , 2015 .

[44]  E. Halgren,et al.  Source estimates for MEG/EEG visual evoked responses constrained by multiple, retinotopically‐mapped stimulus locations , 2009, Human brain mapping.

[45]  Chris I. Baker,et al.  Deconstructing multivariate decoding for the study of brain function , 2017, NeuroImage.

[46]  G. V. Simpson,et al.  Anticipatory Biasing of Visuospatial Attention Indexed by Retinotopically Specific α-Bank Electroencephalography Increases over Occipital Cortex , 2000, The Journal of Neuroscience.

[47]  Karl J. Friston,et al.  Where bottom-up meets top-down: neuronal interactions during perception and imagery. , 2004, Cerebral cortex.

[48]  M. Posner,et al.  Orienting of Attention* , 1980, The Quarterly journal of experimental psychology.

[49]  Robert Oostenveld,et al.  The five percent electrode system for high-resolution EEG and ERP measurements , 2001, Clinical Neurophysiology.

[50]  Giulio Tononi,et al.  Reversal of cortical information flow during visual imagery as compared to visual perception , 2014, NeuroImage.

[51]  David W. Sutterer,et al.  The topography of alpha-band activity tracks the content of spatial working memory. , 2016, Journal of neurophysiology.

[52]  Thomas Serre,et al.  Reading the mind's eye: Decoding category information during mental imagery , 2010, NeuroImage.

[53]  Radoslaw Martin Cichy,et al.  Visual Imagery and Perception Share Neural Representations in the Alpha Frequency Band , 2020, Current Biology.

[54]  A. Dale,et al.  Functional analysis of primary visual cortex (V1) in humans. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[55]  J. Serences,et al.  Near-Real-Time Feature-Selective Modulations in Human Cortex , 2013, Current Biology.

[56]  J. L. de la Pompa,et al.  A novel source of arterial valve cells linked to bicuspid aortic valve without raphe in mice , 2018, eLife.

[57]  S. Dehaene,et al.  Characterizing the dynamics of mental representations: the temporal generalization method , 2014, Trends in Cognitive Sciences.