Human Inferences about Sequences: A Minimal Transition Probability Model

The brain constantly infers the causes of the inputs it receives and uses these inferences to generate statistical expectations about future observations. Experimental evidence for these expectations and their violations include explicit reports, sequential effects on reaction times, and mismatch or surprise signals recorded in electrophysiology and functional MRI. Here, we explore the hypothesis that the brain acts as a near-optimal inference device that constantly attempts to infer the time-varying matrix of transition probabilities between the stimuli it receives, even when those stimuli are in fact fully unpredictable. This parsimonious Bayesian model, with a single free parameter, accounts for a broad range of findings on surprise signals, sequential effects and the perception of randomness. Notably, it explains the pervasive asymmetry between repetitions and alternations encountered in those studies. Our analysis suggests that a neural machinery for inferring transition probabilities lies at the core of human sequence knowledge.

[1]  Axel Cleeremans,et al.  Dissociating the effects of automatic activation and explicit expectancy on reaction times in a simple associative learning task. , 2006, Journal of experimental psychology. Learning, memory, and cognition.

[2]  Wei Ji Ma,et al.  Bayesian inference with probabilistic population codes , 2006, Nature Neuroscience.

[3]  Eduardo F. Morales,et al.  An Introduction to Reinforcement Learning , 2011 .

[4]  Klaas E. Stephan,et al.  Inferring on the Intentions of Others by Hierarchical Bayesian Learning , 2014, PLoS Comput. Biol..

[5]  Yaakov Kareev,et al.  Positive bias in the perception of covariation. , 1995 .

[6]  Nathaniel D. Daw,et al.  Cortical and Hippocampal Correlates of Deliberation during Model-Based Decisions for Rewards in Humans , 2013, PLoS Comput. Biol..

[7]  Joseph T. McGuire,et al.  Functionally Dissociable Influences on Learning Rate in a Dynamic Environment , 2014, Neuron.

[8]  Erich Schröger,et al.  Human Visual System Automatically Encodes Sequential Regularities of Discrete Events , 2010, Journal of Cognitive Neuroscience.

[9]  Risto Näätänen,et al.  Implicit, Intuitive, and Explicit Knowledge of Abstract Regularities in a Sound Sequence: An Event-related Brain Potential Study , 2006, Journal of Cognitive Neuroscience.

[10]  Henry Rouanet Les modèles stochastiques d'apprentissage : recherches et perspectives , 1967 .

[11]  R. Hyman Stimulus information as a determinant of reaction time. , 1953, Journal of experimental psychology.

[12]  S. Debener,et al.  Trial-by-Trial Fluctuations in the Event-Related Electroencephalogram Reflect Dynamic Changes in the Degree of Surprise , 2008, The Journal of Neuroscience.

[13]  L. Maloney,et al.  Decision-theoretic models of visual perception and action , 2010, Vision Research.

[14]  C. Gallistel,et al.  The Perception of Probability , 2022 .

[15]  Henry Rouanet,et al.  Les modèles stochastiques d'apprentissage , 1969 .

[16]  Zoubin Ghahramani,et al.  Computational principles of movement neuroscience , 2000, Nature Neuroscience.

[17]  Ovidiu Lungu,et al.  Probability detection mechanisms and motor learning , 2004, Experimental Brain Research.

[18]  Eero P. Simoncelli,et al.  Cardinal rules: Visual orientation perception reflects knowledge of environmental statistics , 2011, Nature Neuroscience.

[19]  E. Soetens,et al.  Covert signs of expectancy in serial reaction time tasks revealed by event-related potentials , 1999, Perception & psychophysics.

[20]  E. Soetens,et al.  Expectancy or Automatic Facilitation? Separating Sequential Effects in Two-Choice Reaction Time , 1985 .

[21]  J. Changeux,et al.  A Neuronal Model of Predictive Coding Accounting for the Mismatch Negativity , 2012, The Journal of Neuroscience.

[22]  Florent Meyniel,et al.  The Neural Representation of Sequences: From Transition Probabilities to Algebraic Patterns and Linguistic Trees , 2015, Neuron.

[23]  Karl J. Friston,et al.  Modelling Trial-by-Trial Changes in the Mismatch Negativity , 2013, PLoS Comput. Biol..

[24]  Erich Schröger,et al.  Regularity Extraction and Application in Dynamic Auditory Stimulus Sequences , 2007, Journal of Cognitive Neuroscience.

[25]  Hongbin Wang,et al.  Latent structure in random sequences drives neural learning toward a rational bias , 2015, Proceedings of the National Academy of Sciences.

[26]  N. Kirby Sequential effects in two-choice reaction time: automatic facilitation or subjective expectancy? , 1976, Journal of experimental psychology. Human perception and performance.

[27]  A. Pouget,et al.  Reading population codes: a neural implementation of ideal observers , 1999, Nature Neuroscience.

[28]  P. Berkes,et al.  Statistically Optimal Perception and Learning: from Behavior to Neural Representations , 2022 .

[29]  C. Olson,et al.  Prediction suppression in monkey inferotemporal cortex depends on the conditional probability between images. , 2016, Journal of neurophysiology.

[30]  Robert C. Wilson,et al.  Rational regulation of learning dynamics by pupil–linked arousal systems , 2012, Nature Neuroscience.

[31]  Charles Kemp,et al.  The discovery of structural form , 2008, Proceedings of the National Academy of Sciences.

[32]  Timothy E. J. Behrens,et al.  Learning the value of information in an uncertain world , 2007, Nature Neuroscience.

[33]  Jonathan D. Cohen,et al.  Sequential effects: Superstition or rational behavior? , 2008, NIPS.

[34]  Jonathan D. Cohen,et al.  Mechanisms underlying dependencies of performance on stimulus history in a two-alternative forced-choice task , 2002, Cognitive, affective & behavioral neuroscience.

[35]  David B. Dunson,et al.  Bayesian data analysis, third edition , 2013 .

[36]  S. Dehaene,et al.  Disruption of hierarchical predictive coding during sleep , 2015, Proceedings of the National Academy of Sciences.

[37]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[38]  Christian Büchel,et al.  Unconscious Detection of Implicit Expectancies , 2005, Journal of Cognitive Neuroscience.

[39]  I. Nelken,et al.  Sensitivity to Complex Statistical Regularities in Rat Auditory Cortex , 2012, Neuron.

[40]  Wei Ji Ma,et al.  Neural coding of uncertainty and probability. , 2014, Annual review of neuroscience.

[41]  G. McCarthy,et al.  Perceiving patterns in random series: dynamic processing of sequence in prefrontal cortex , 2002, Nature Neuroscience.

[42]  D. Knill,et al.  The Bayesian brain: the role of uncertainty in neural coding and computation , 2004, Trends in Neurosciences.

[43]  Karl J. Friston The free-energy principle: a unified brain theory? , 2010, Nature Reviews Neuroscience.

[44]  R N Aslin,et al.  Statistical Learning by 8-Month-Old Infants , 1996, Science.

[45]  D. Heeger,et al.  Slow Cortical Dynamics and the Accumulation of Information over Long Timescales , 2012, Neuron.

[46]  Florent Meyniel,et al.  The Sense of Confidence during Probabilistic Learning: A Normative Account , 2015, PLoS Comput. Biol..

[47]  G. Schwarz Estimating the Dimension of a Model , 1978 .

[48]  Karl J. Friston,et al.  A theory of cortical responses , 2005, Philosophical Transactions of the Royal Society B: Biological Sciences.

[49]  Timothy D. Hanks,et al.  Probabilistic Population Codes for Bayesian Decision Making , 2008, Neuron.

[50]  N. Squires,et al.  The effect of stimulus sequence on the waveform of the cortical event-related potential. , 1976, Science.

[51]  W. Chase,et al.  Sequential effects in choice reaction time. , 1969 .

[52]  M. Ernst,et al.  Humans integrate visual and haptic information in a statistically optimal fashion , 2002, Nature.

[53]  Jim M. Monti,et al.  Neural repetition suppression reflects fulfilled perceptual expectations , 2008, Nature Neuroscience.

[54]  Clifford Konold,et al.  Making Sense of Randomness " Implicit Encoding as a Basis for Judgment , 1997 .

[55]  József Fiser,et al.  Spontaneous Cortical Activity Reveals Hallmarks of an Optimal Internal Model of the Environment , 2011, Science.

[56]  Hongbin Wang,et al.  Perception of randomness: On the time of streaks , 2010, Cognitive Psychology.

[57]  E. Todorov Optimality principles in sensorimotor control , 2004, Nature Neuroscience.

[58]  Axel Cleeremans,et al.  Nonconscious Learning From Crowded Sequences , 2014, Psychological science.

[59]  Karl J. Friston Learning and inference in the brain , 2003, Neural Networks.

[60]  Carl R Olson,et al.  Statistical Learning of Serial Visual Transitions by Neurons in Monkey Inferotemporal Cortex , 2014, The Journal of Neuroscience.

[61]  Ulrike Hahn,et al.  Perceptions of randomness: why three heads are better than four. , 2009, Psychological review.

[62]  G. S. Tune RESPONSE PREFERENCES: A REVIEW OF SOME RELEVANT LITERATURE. , 1964, Psychological bulletin.

[63]  M. Tribus,et al.  Probability theory: the logic of science , 2003 .

[64]  S. Dehaene,et al.  Neural signature of the conscious processing of auditory regularities , 2009, Proceedings of the National Academy of Sciences.

[65]  Xiao-Jing Wang,et al.  Probabilistic Decision Making by Slow Reverberation in Cortical Circuits , 2002, Neuron.

[66]  Tai Sing Lee,et al.  Hierarchical Bayesian inference in the visual cortex. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[67]  P. Bertelson Sequential Redundancy and Speed in a Serial Two-Choice Responding Task , 1961 .

[68]  Marius Usher,et al.  The Timescale of Perceptual Evidence Integration Can Be Adapted to the Environment , 2013, Current Biology.

[69]  Rajesh P. N. Rao,et al.  An optimal estimation approach to visual perception and learning , 1999, Vision Research.

[70]  Rajesh P. N. Rao,et al.  Predictive Coding , Cortical Feedback , and Spike-Timing Dependent Plasticity , 2001 .

[71]  E. Maris,et al.  Prior Expectation Mediates Neural Adaptation to Repeated Sounds in the Auditory Cortex: An MEG Study , 2011, The Journal of Neuroscience.

[72]  Pete C. Trimmer,et al.  The evolution of decision rules in complex environments , 2014, Trends in Cognitive Sciences.

[73]  Matt Jones,et al.  Sequential Effects Reflect Parallel Learning of Multiple Environmental Regularities , 2022 .

[74]  Aapo Hyvärinen,et al.  Interpreting Neural Response Variability as Monte Carlo Sampling of the Posterior , 2002, NIPS.

[75]  Tim Fingscheidt,et al.  A Model-Based Approach to Trial-By-Trial P300 Amplitude Fluctuations , 2013, Front. Hum. Neurosci..