Anticipatory reinstatement of expected perceptual events during visual sequence learning

Being able to predict future events in learned sequences is a fundamental cognitive ability. Successful behavior requires the brain to not only anticipate an upcoming event, but to also continue to keep track of the sequence in case of eventual disruptions, (e.g., when a predicted event does not occur). However, the precise neural mechanisms supporting such processes remain unknown. Here, using multivariate pattern classification based on electroencephalography (EEG) activity and time-frequency amplitude, we show that the visual system represents upcoming expected stimuli during a sequence-learning task. Stimulus-evoked neural representations were reinstated prior to expected stimulus onset, and when an anticipated stimulus was unexpectedly withheld, suggesting proactive reinstatement of sensory templates. Importantly, stimulus representation of the absent stimulus co-occurred with an emerging representation of the following stimulus in the sequence, showing that the brain actively maintained sequence order even when the sequence was perturbed. Finally, selective activity was evident in the alpha-beta band (9-20 Hz) amplitude topographies, confirming the role of alpha-beta oscillations in carrying information about the nature of sensory expectations. These results show that the brain dynamically implements anticipatory mechanisms that reinstate sensory representations, and that allow us to make predictions about events further in the future.

[1]  Luc H. Arnal,et al.  Delta-Beta Coupled Oscillations Underlie Temporal Prediction Accuracy. , 2015, Cerebral cortex.

[2]  David A. Tovar,et al.  Representational dynamics of object vision: the first 1000 ms. , 2013, Journal of vision.

[3]  Keiji Tanaka,et al.  Matching Categorical Object Representations in Inferior Temporal Cortex of Man and Monkey , 2008, Neuron.

[4]  F. D. Lange,et al.  How Do Expectations Shape Perception? , 2018, Trends in Cognitive Sciences.

[5]  Karl J. Friston,et al.  Canonical Microcircuits for Predictive Coding , 2012, Neuron.

[6]  Tjerk P. Straatsma,et al.  NWChem: A comprehensive and scalable open-source solution for large scale molecular simulations , 2010, Comput. Phys. Commun..

[7]  Stephen M. Smith,et al.  Threshold-free cluster enhancement: Addressing problems of smoothing, threshold dependence and localisation in cluster inference , 2009, NeuroImage.

[8]  Tandra Ghose,et al.  Generalization between canonical and non-canonical views in object recognition. , 2013, Journal of vision.

[9]  Aaron R. Seitz,et al.  Rapidly learned stimulus expectations alter perception of motion. , 2010, Journal of vision.

[10]  Jan-Mathijs Schoffelen,et al.  Temporal Expectation and Attention Jointly Modulate Auditory Oscillatory Activity in the Beta Band , 2015, PloS one.

[11]  D. Melcher,et al.  Frequency modulation of neural oscillations according to visual task demands , 2018, Proceedings of the National Academy of Sciences.

[12]  A. Engel,et al.  Beta-band oscillations—signalling the status quo? , 2010, Current Opinion in Neurobiology.

[13]  Floris P de Lange,et al.  Prior expectations induce prestimulus sensory templates , 2017, Proceedings of the National Academy of Sciences.

[14]  C. Ranganath,et al.  Theta Oscillations Promote Temporal Sequence Learning , 2018, Neurobiology of Learning and Memory.

[15]  Lauren V. Kustner,et al.  Shaping of Object Representations in the Human Medial Temporal Lobe Based on Temporal Regularities , 2012, Current Biology.

[16]  Martin Luessi,et al.  MNE software for processing MEG and EEG data , 2014, NeuroImage.

[17]  Biao Han,et al.  The rhythms of predictive coding? Pre-stimulus phase modulates the influence of shape perception on luminance judgments , 2016, Scientific Reports.

[18]  Saskia Haegens,et al.  Beyond the Status Quo: A Role for Beta Oscillations in Endogenous Content (Re)Activation , 2017, eNeuro.

[19]  Dejan Draschkow,et al.  Cluster-based permutation tests of MEG/EEG data do not establish significance of effect latency or location. , 2019, Psychophysiology.

[20]  Leila Reddy,et al.  Multivoxel Object Representations in Adult Human Visual Cortex Are Flexible: An Associative Learning Study , 2016, Journal of Cognitive Neuroscience.

[21]  Stefan Haufe,et al.  On the interpretation of weight vectors of linear models in multivariate neuroimaging , 2014, NeuroImage.

[22]  Thomas Serre,et al.  Reading the mind's eye: Decoding category information during mental imagery , 2010, NeuroImage.

[23]  J. Kaiser,et al.  Attention samples objects held in working memory at a theta rhythm , 2018, bioRxiv.

[24]  Radoslaw Martin Cichy,et al.  Visual Imagery and Perception Share Neural Representations in the Alpha Frequency Band , 2020, Current Biology.

[25]  F. D. de Lange,et al.  Prior Expectations Bias Sensory Representations in Visual Cortex , 2013, The Journal of Neuroscience.

[26]  Christopher Summerfield,et al.  Dissociable prior influences of signal probability and relevance on visual contrast sensitivity , 2011 .

[27]  H. Kennedy,et al.  Alpha-Beta and Gamma Rhythms Subserve Feedback and Feedforward Influences among Human Visual Cortical Areas , 2016, Neuron.

[28]  Aude Oliva,et al.  Estimating perception of scene layout properties from global image features. , 2011, Journal of vision.

[29]  A. Nobre,et al.  Temporal Expectation Improves the Quality of Sensory Information , 2012, The Journal of Neuroscience.

[30]  A. Nobre,et al.  The hazards of time , 2007, Current Opinion in Neurobiology.

[31]  S. Dehaene,et al.  Characterizing the dynamics of mental representations: the temporal generalization method , 2014, Trends in Cognitive Sciences.

[32]  S. Kouider,et al.  Spontaneous Activity Patterns in Primary Visual Cortex Predispose to Visual Hallucinations , 2015, The Journal of Neuroscience.

[33]  Floris P. de Lange,et al.  Prior Expectations Evoke Stimulus Templates in the Primary Visual Cortex , 2014, Journal of Cognitive Neuroscience.

[34]  J. Kaiser,et al.  Object-based attention prioritizes working memory contents at a theta rhythm. , 2020, Journal of experimental psychology. General.

[35]  Martin Luessi,et al.  MEG and EEG data analysis with MNE-Python , 2013, Front. Neuroinform..

[36]  Karl J. Friston,et al.  The problem of low variance voxels in statistical parametric mapping; a new hat avoids a ‘haircut’ , 2012, NeuroImage.

[37]  L. Reddy,et al.  Learning of anticipatory responses in single neurons of the human medial temporal lobe , 2015, Nature Communications.

[38]  Lila Davachi,et al.  How the hippocampus preserves order: the role of prediction and context , 2015, Trends in Cognitive Sciences.

[39]  W. Penny,et al.  Theta-Coupled Periodic Replay in Working Memory , 2010, Current Biology.

[40]  R. Oostenveld,et al.  Nonparametric statistical testing of EEG- and MEG-data , 2007, Journal of Neuroscience Methods.

[41]  David J. Schwartzman,et al.  Visual Perceptual Echo Reflects Learning of Regularities in Rapid Luminance Sequences , 2017, The Journal of Neuroscience.

[42]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[43]  Janneke F. M. Jehee,et al.  Less Is More: Expectation Sharpens Representations in the Primary Visual Cortex , 2012, Neuron.

[44]  T. Verguts,et al.  Theta oscillations shift towards optimal frequency for cognitive control , 2020, Nature Human Behaviour.

[45]  Susan G. Wardle,et al.  Decoding Dynamic Brain Patterns from Evoked Responses: A Tutorial on Multivariate Pattern Analysis Applied to Time Series Neuroimaging Data , 2016, Journal of Cognitive Neuroscience.

[46]  Jessy K. Possel,et al.  Theta-phase dependent neuronal coding during sequence learning in human single neurons , 2019, Nature Communications.

[47]  Kingson Man,et al.  Multivariate cross-classification: applying machine learning techniques to characterize abstraction in neural representations , 2015, Front. Hum. Neurosci..

[48]  Adam Johnson,et al.  Neural Ensembles in CA3 Transiently Encode Paths Forward of the Animal at a Decision Point , 2007, The Journal of Neuroscience.

[49]  R. VanRullen,et al.  Alpha oscillations and traveling waves: Signatures of predictive coding? , 2019, PLoS biology.

[50]  Joel Z. Leibo,et al.  The dynamics of invariant object recognition in the human visual system. , 2014, Journal of neurophysiology.

[51]  Craig G. Richter,et al.  Top-Down Beta Enhances Bottom-Up Gamma , 2016, The Journal of Neuroscience.

[52]  Mark G. Stokes,et al.  One Thing Leads to Another: Anticipating Visual Object Identity Based on Associative-Memory Templates , 2020, The Journal of Neuroscience.

[53]  E. Miller,et al.  Frequency-specific hippocampal-prefrontal interactions during associative learning , 2015, Nature Neuroscience.

[54]  Ryota Kanai,et al.  Rhythmic Influence of Top–Down Perceptual Priors in the Phase of Prestimulus Occipital Alpha Oscillations , 2016, Journal of Cognitive Neuroscience.

[55]  Aaron R. Seitz,et al.  Changing expectations about speed alters perceived motion direction , 2011, Current Biology.

[56]  Hinze Hogendoorn,et al.  Predictions drive neural representations of visual events ahead of incoming sensory information , 2020, Proceedings of the National Academy of Sciences.