Differential temporal dynamics during visual imagery and perception

Visual perception and imagery rely on similar representations in the visual cortex. During perception, visual activity is characterized by distinct processing stages, but the temporal dynamics underlying imagery remain unclear. Here, we investigated the dynamics of visual imagery in human participants using magnetoencephalography. Firstly, we show that, compared to perception, imagery decoding becomes significant later and representations at the start of imagery already overlap with later time points. This suggests that during imagery, the entire visual representation is activated at once or that there are large differences in the timing of imagery between trials. Secondly, we found consistent overlap between imagery and perceptual processing around 160 ms and from 300 ms after stimulus onset. This indicates that the N170 gets reactivated during imagery and that imagery does not rely on early perceptual representations. Together, these results provide important insights for our understanding of the neural mechanisms of visual imagery.

[1]  Thomas Serre,et al.  The Neural Dynamics of Face Detection in the Wild Revealed by MVPA , 2014, The Journal of Neuroscience.

[2]  Jan-Mathijs Schoffelen,et al.  Stimulus familiarity and expectation jointly modulate neural activity in the visual ventral stream , 2017 .

[3]  Wolf Singer,et al.  Smaller Primary Visual Cortex Is Associated with Stronger, but Less Precise Mental Imagery. , 2016, Cerebral cortex.

[4]  Karl J. Friston,et al.  Where bottom-up meets top-down: neuronal interactions during perception and imagery. , 2004, Cerebral cortex.

[5]  Robert Oostenveld,et al.  FieldTrip: Open Source Software for Advanced Analysis of MEG, EEG, and Invasive Electrophysiological Data , 2010, Comput. Intell. Neurosci..

[6]  P. Montague,et al.  Vividness of mental imagery: Individual variability can be measured objectively , 2007, Vision Research.

[7]  Stefan Haufe,et al.  On the interpretation of weight vectors of linear models in multivariate neuroimaging , 2014, NeuroImage.

[8]  D. F. Marks,et al.  Visual imagery differences in the recall of pictures. , 1973, British journal of psychology.

[9]  S. Dehaene,et al.  Characterizing the dynamics of mental representations: the temporal generalization method , 2014, Trends in Cognitive Sciences.

[10]  Marcel A J van Gerven,et al.  Vividness of Visual Imagery Depends on the Neural Overlap with Perception in Visual Areas , 2017, The Journal of Neuroscience.

[11]  Robert Oostenveld,et al.  Online and offline tools for head movement compensation in MEG , 2013, NeuroImage.

[12]  S. Hochstein,et al.  The reverse hierarchy theory of visual perceptual learning , 2004, Trends in Cognitive Sciences.

[13]  Moritz Grosse-Wentrup,et al.  Multisubject Learning for Common Spatial Patterns in Motor-Imagery BCI , 2011, Comput. Intell. Neurosci..

[14]  Pieter R. Roelfsema,et al.  Object-based attention in the primary visual cortex of the macaque monkey , 1998, Nature.

[15]  Janneke F. M. Jehee,et al.  Less Is More: Expectation Sharpens Representations in the Primary Visual Cortex , 2012, Neuron.

[16]  John H. R. Maunsell,et al.  Visual processing in monkey extrastriate cortex. , 1987, Annual review of neuroscience.

[17]  David A. Tovar,et al.  Representational dynamics of object vision: the first 1000 ms. , 2013, Journal of vision.

[18]  Joel Z. Leibo,et al.  The dynamics of invariant object recognition in the human visual system. , 2014, Journal of neurophysiology.

[19]  Giorgio Ganis,et al.  Visual mental imagery and perception produce opposite adaptation effects on early brain potentials , 2008, NeuroImage.

[20]  Marcel van Gerven,et al.  Convolutional neural network-based encoding and decoding of visual object recognition in space and time , 2017, NeuroImage.

[21]  T. Hendler,et al.  A hierarchical axis of object processing stages in the human visual cortex. , 2001, Cerebral cortex.

[22]  Leslie G. Ungerleider,et al.  Distributed Neural Systems for the Generation of Visual Images , 2000, Neuron.

[23]  S. Kosslyn,et al.  Neural foundations of imagery , 2001, Nature Reviews Neuroscience.

[24]  Anke Marit Albers,et al.  Eye movement-related confounds in neural decoding of visual working memory representations , 2017 .

[25]  W. Drongelen,et al.  Localization of brain electrical activity via linearly constrained minimum variance spatial filtering , 1997, IEEE Transactions on Biomedical Engineering.

[26]  F. Tong,et al.  Decoding reveals the contents of visual working memory in early visual areas , 2009, Nature.

[27]  D. Hubel,et al.  Receptive fields and functional architecture of monkey striate cortex , 1968, The Journal of physiology.

[28]  Thomas Serre,et al.  A feedforward architecture accounts for rapid categorization , 2007, Proceedings of the National Academy of Sciences.

[29]  Tomoyasu Horikawa,et al.  Generic decoding of seen and imagined objects using hierarchical visual features , 2015, Nature Communications.

[30]  D C Van Essen,et al.  Information processing in the primate visual system: an integrated systems perspective. , 1992, Science.

[31]  Leslie G. Ungerleider,et al.  Visual Imagery of Famous Faces: Effects of Memory and Attention Revealed by fMRI , 2002, NeuroImage.

[32]  Denis Fize,et al.  Speed of processing in the human visual system , 1996, Nature.

[33]  Ole Jensen,et al.  Spatiotemporal Dynamics of Cortical Representations during and after Stimulus Presentation , 2016, Front. Syst. Neurosci..

[34]  I. Toni,et al.  Shared Representations for Working Memory and Mental Imagery in Early Visual Cortex , 2013, Current Biology.

[35]  Thomas Serre,et al.  Reading the mind's eye: Decoding category information during mental imagery , 2010, NeuroImage.

[36]  Anders M. Dale,et al.  Automatic parcellation of human cortical gyri and sulci using standard anatomical nomenclature , 2010, NeuroImage.

[37]  H. Kennedy,et al.  Visual Areas Exert Feedforward and Feedback Influences through Distinct Frequency Channels , 2014, Neuron.

[38]  Floris P. de Lange,et al.  Dissociating sensory from decision processes in human perceptual decision making , 2015, Scientific Reports.

[39]  E. Halgren,et al.  Cognitive response profile of the human fusiform face area as determined by MEG. , 2000, Cerebral cortex.

[40]  J. Duhamel,et al.  Differential Dynamics of Spatial Attention, Position, and Color Coding within the Parietofrontal Network , 2015, The Journal of Neuroscience.

[41]  S. Kosslyn,et al.  Brain areas underlying visual mental imagery and visual perception: an fMRI study. , 2004, Brain research. Cognitive brain research.

[42]  Paul E. Downing,et al.  Viewpoint-Specific Scene Representations in Human Parahippocampal Cortex , 2003, Neuron.

[43]  T. Allison,et al.  Electrophysiological Studies of Face Perception in Humans , 1996, Journal of Cognitive Neuroscience.

[44]  Karl J. Friston,et al.  Distinct Top-down and Bottom-up Brain Connectivity During Visual Perception and Imagery , 2017, Scientific Reports.

[45]  S. Hochstein,et al.  View from the Top Hierarchies and Reverse Hierarchies in the Visual System , 2002, Neuron.

[46]  Marius V Peelen,et al.  The Neural Dynamics of Attentional Selection in Natural Scenes , 2016, The Journal of Neuroscience.

[47]  T. Carlson,et al.  High temporal resolution decoding of object position and category. , 2011, Journal of vision.

[48]  N. Kanwisher,et al.  The Fusiform Face Area: A Module in Human Extrastriate Cortex Specialized for Face Perception , 1997, The Journal of Neuroscience.

[49]  R Vogels,et al.  Coding of stimulus invariances by inferior temporal neurons. , 1996, Progress in brain research.

[50]  Gabriel Curio,et al.  MEG/EEG sources of the 170-ms response to faces are co-localized in the fusiform gyrus , 2007, NeuroImage.

[51]  Stanislas Dehaene,et al.  Two Distinct Dynamic Modes Subtend the Detection of Unexpected Sounds , 2014, PloS one.

[52]  S. Kosslyn,et al.  When is early visual cortex activated during visual mental imagery? , 2003, Psychological bulletin.

[53]  R. Oostenveld,et al.  Nonparametric statistical testing of EEG- and MEG-data , 2007, Journal of Neuroscience Methods.

[54]  Karl J. Friston,et al.  Canonical Microcircuits for Predictive Coding , 2012, Neuron.

[55]  Karl J. Friston,et al.  MEG and EEG data fusion: Simultaneous localisation of face-evoked responses , 2009, NeuroImage.

[56]  Chris I. Baker,et al.  Disentangling visual imagery and perception of real-world objects , 2012, NeuroImage.

[57]  M. Koivisto,et al.  Recurrent Processing in V1/V2 Contributes to Categorization of Natural Scenes , 2011, The Journal of Neuroscience.