Neural Integration of Stimulus History Underlies Prediction for Naturalistically Evolving Sequences

Forming valid predictions about the environment is crucial to survival. However, whether humans are able to form valid predictions about natural stimuli based on their temporal statistical regularities remains unknown. Here, we presented subjects with tone sequences with pitch fluctuations that, over time, capture long-range temporal dependence structures prevalent in natural stimuli. We found that subjects were able to exploit such naturalistic statistical regularities to make valid predictions about upcoming items in a sequence. Magnetoencephalography (MEG) recordings revealed that slow, arrhythmic cortical dynamics tracked the evolving pitch sequence over time such that neural activity at a given moment was influenced by the pitch of up to seven previous tones. Importantly, such history integration contained in neural activity predicted the expected pitch of the upcoming tone, providing a concrete computational mechanism for prediction. These results establish humans' ability to make valid predictions based on temporal regularities inherent in naturalistic stimuli and further reveal the neural mechanisms underlying such predictive computation. SIGNIFICANCE STATEMENT A fundamental question in neuroscience is how the brain predicts upcoming events in the environment. To date, this question has primarily been addressed in experiments using relatively simple stimulus sequences. Here, we studied predictive processing in the human brain using auditory tone sequences that exhibit temporal statistical regularities similar to those found in natural stimuli. We observed that humans are able to form valid predictions based on such complex temporal statistical regularities. We further show that neural response to a given tone in the sequence reflects integration over the preceding tone sequence and that this history dependence forms the foundation for prediction. These findings deepen our understanding of how humans form predictions in an ecologically valid environment.

[1]  D. Botteldooren,et al.  1/f Noise in Rural and Urban Soundscapes , 2003 .

[2]  S. Kochen,et al.  Expectation and Attention in Hierarchical Auditory Prediction , 2013, The Journal of Neuroscience.

[3]  R. Voss,et al.  ‘1/fnoise’ in music and speech , 1975, Nature.

[4]  D. Tolhurst,et al.  Amplitude spectra of natural images. , 1992, Ophthalmic & physiological optics : the journal of the British College of Ophthalmic Opticians.

[5]  D. Whitney,et al.  Serial dependence in visual perception , 2011 .

[6]  F. D. de Lange,et al.  Prior Expectations Bias Sensory Representations in Visual Cortex , 2013, The Journal of Neuroscience.

[7]  B. Mandelbrot,et al.  Fractional Brownian Motions, Fractional Noises and Applications , 1968 .

[8]  V. Menon,et al.  Musical rhythm spectra from Bach to Joplin obey a 1/f power law , 2012, Proceedings of the National Academy of Sciences.

[9]  Michael Esterman,et al.  Perceptual expectation evokes category-selective cortical activity. , 2010, Cerebral cortex.

[10]  H. Lau,et al.  Prestimulus Oscillatory Activity over Motor Cortex Reflects Perceptual Expectations , 2013, The Journal of Neuroscience.

[11]  D. W. Robinson,et al.  A re-determination of the equal-loudness relations for pure tones , 1956 .

[12]  S. Dehaene,et al.  Neural signature of the conscious processing of auditory regularities , 2009, Proceedings of the National Academy of Sciences.

[13]  Mark F. Bear,et al.  Learned spatiotemporal sequence recognition and prediction in primary visual cortex , 2014, Nature Neuroscience.

[14]  R. Desimone,et al.  Responses of Macaque Perirhinal Neurons during and after Visual Stimulus Association Learning , 1999, The Journal of Neuroscience.

[15]  T. Albright,et al.  Remembering Visual Motion: Neural Correlates of Associative Plasticity and Motion Recall in Cortical Area MT , 2007, Neuron.

[16]  D. Poeppel,et al.  Phase Patterns of Neuronal Responses Reliably Discriminate Speech in Human Auditory Cortex , 2007, Neuron.

[17]  J. Tecce Contingent negative variation (CNV) and psychological processes in man. , 1972, Psychological bulletin.

[18]  D. Whitney,et al.  Serial Dependence in the Perception of Faces , 2014, Current Biology.

[19]  A. Kleinschmidt,et al.  Temporal Tuning Properties along the Human Ventral Visual Stream , 2012, The Journal of Neuroscience.

[20]  Patrice Abry,et al.  Interplay between functional connectivity and scale-free dynamics in intrinsic fMRI networks , 2014, NeuroImage.

[21]  F. D. Lange,et al.  Opposite Effects of Recent History on Perception and Decision , 2017, Current Biology.

[22]  M. Bar Visual objects in context , 2004, Nature Reviews Neuroscience.

[23]  C. Honey,et al.  Hierarchical process memory: memory as an integral component of information processing , 2015, Trends in Cognitive Sciences.

[24]  Christopher Summerfield,et al.  Dissociable prior influences of signal probability and relevance on visual contrast sensitivity , 2011 .

[25]  Aaron R. Seitz,et al.  Rapidly learned stimulus expectations alter perception of motion. , 2010, Journal of vision.

[26]  H. Kennedy,et al.  A Large-Scale Circuit Mechanism for Hierarchical Dynamical Processing in the Primate Cortex , 2015, Neuron.

[27]  J. Schnupp,et al.  Tuning to Natural Stimulus Dynamics in Primary Auditory Cortex , 2006, Current Biology.

[28]  Adam Gazzaley,et al.  Age-Related Changes in 1/f Neural Electrophysiological Noise , 2015, The Journal of Neuroscience.

[29]  A. Eke,et al.  Fractal characterization of complexity in temporal physiological signals , 2002, Physiological measurement.

[30]  Biyu J. He,et al.  Scale-Free Neural and Physiological Dynamics in Naturalistic Stimuli Processing , 2016, eNeuro.

[31]  C. Olson,et al.  Statistical learning of visual transitions in monkey inferotemporal cortex , 2011, Proceedings of the National Academy of Sciences.

[32]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[33]  Aniruddh D. Patel,et al.  Temporal patterns of human cortical activity reflect tone sequence structure , 2000, Nature.

[34]  P. Abry,et al.  Scale-Free and Multifractal Time Dynamics of fMRI Signals during Rest and Task , 2012, Front. Physio..

[35]  Thomas D Albright,et al.  On the Perception of Probable Things: Neural Substrates of Associative Memory, Imagery, and Perception , 2012, Neuron.

[36]  B Lütkenhöner,et al.  Neuromagnetic evidence for a pitch processing center in Heschl's gyrus. , 2003, Cerebral cortex.

[37]  Patrice Abry,et al.  Fast and exact synthesis of stationary multivariate Gaussian time series using circulant embedding , 2011, Signal Process..

[38]  A. Nobre,et al.  Endogenous modulation of low frequency oscillations by temporal expectations , 2011, Journal of neurophysiology.

[39]  J. Palva,et al.  Very Slow EEG Fluctuations Predict the Dynamics of Stimulus Detection and Oscillation Amplitudes in Humans , 2008, The Journal of Neuroscience.

[40]  John G. Proakis,et al.  Probability, random variables and stochastic processes , 1985, IEEE Trans. Acoust. Speech Signal Process..

[41]  Raymond J. Dolan,et al.  Outlier Responses Reflect Sensitivity to Statistical Structure in the Human Brain , 2013, PLoS Comput. Biol..

[42]  J. Atick,et al.  STATISTICS OF NATURAL TIME-VARYING IMAGES , 1995 .

[43]  C. Summerfield,et al.  Expectation in perceptual decision making: neural and computational mechanisms , 2014, Nature Reviews Neuroscience.

[44]  M. S. Keshner 1/f noise , 1982, Proceedings of the IEEE.

[45]  Phillipp Meister,et al.  Statistical Signal Processing Detection Estimation And Time Series Analysis , 2016 .

[46]  Biyu J. He Scale-free brain activity: past, present, and future , 2014, Trends in Cognitive Sciences.

[47]  Caspar M. Schwiedrzik,et al.  Stimulus Predictability Reduces Responses in Primary Visual Cortex , 2010, The Journal of Neuroscience.

[48]  Robert Oostenveld,et al.  Online and offline tools for head movement compensation in MEG , 2013, NeuroImage.

[49]  Charan Ranganath,et al.  Category expectation modulates baseline and stimulus-evoked activity in human inferotemporal cortex , 2009, Brain Research.

[50]  E. Wagenmakers,et al.  Temporal expectation and information processing: A model-based analysis , 2012, Cognition.

[51]  C. Schroeder,et al.  Low-frequency neuronal oscillations as instruments of sensory selection , 2009, Trends in Neurosciences.

[52]  R N Aslin,et al.  Statistical Learning by 8-Month-Old Infants , 1996, Science.

[53]  D. Heeger,et al.  Slow Cortical Dynamics and the Accumulation of Information over Long Timescales , 2012, Neuron.

[54]  Robert Oostenveld,et al.  FieldTrip: Open Source Software for Advanced Analysis of MEG, EEG, and Invasive Electrophysiological Data , 2010, Comput. Intell. Neurosci..

[55]  R. Oostenveld,et al.  Nonparametric statistical testing of EEG- and MEG-data , 2007, Journal of Neuroscience Methods.

[56]  D. Schacter,et al.  Remembering the past to imagine the future: the prospective brain , 2007, Nature Reviews Neuroscience.

[57]  I. Nelken,et al.  Sensitivity to Complex Statistical Regularities in Rat Auditory Cortex , 2012, Neuron.

[58]  D. Burr,et al.  Vision: Efficient Adaptive Coding , 2014, Current Biology.

[59]  Biyu J. He,et al.  The Temporal Structures and Functional Significance of Scale-free Brain Activity , 2010, Neuron.

[60]  M A Schmuckler,et al.  Auditory perception of fractal contours. , 1993, Journal of experimental psychology. Human perception and performance.

[61]  Timothy D. Hanks,et al.  Elapsed Decision Time Affects the Weighting of Prior Probability in a Perceptual Decision Task , 2011, The Journal of Neuroscience.