Statistical context dictates the relationship between feedback-related EEG signals and learning

Learning should be adjusted according to the surprise associated with observed outcomes but calibrated according to statistical context. For example, when occasional changepoints are expected, surprising outcomes should be weighted heavily to speed learning. In contrast, when uninformative outliers are expected to occur occasionally, surprising outcomes should be less influential. Here we dissociate surprising outcomes from the degree to which they demand learning using a predictive inference task and computational modeling. We show that the P300, a stimulus-locked electrophysiological response previously associated with adjustments in learning behavior, does so conditionally on the source of surprise. Larger P300 signals predicted greater learning in a changing context, but less learning in a context where surprise was indicative of a one-off outlier (oddball). Our results suggest that the P300 provides a surprise signal that is interpreted by downstream learning processes differentially according to statistical context in order to appropriately calibrate learning across complex environments.

[1]  C. Summerfield,et al.  Do humans make good decisions? , 2015, Trends in Cognitive Sciences.

[2]  Anne E. Urai,et al.  Pupil-linked arousal is driven by decision uncertainty and alters serial choice bias , 2017, Nature Communications.

[3]  Robert C. Wilson,et al.  Orbitofrontal Cortex as a Cognitive Map of Task Space , 2014, Neuron.

[4]  Tim Fingscheidt,et al.  A computational analysis of the neural bases of Bayesian inference , 2015, NeuroImage.

[5]  J. Gold,et al.  Arousal-related adjustments of perceptual biases optimize perception in dynamic environments , 2017, Nature Human Behaviour.

[6]  E. Donchin,et al.  Is the P300 component a manifestation of context updating? , 1988, Behavioral and Brain Sciences.

[7]  Joshua I. Gold,et al.  Bayesian Online Learning of the Hazard Rate in Change-Point Problems , 2010, Neural Computation.

[8]  P. Bossaerts,et al.  Neural Mechanisms Behind Identification of Leptokurtic Noise and Adaptive Behavioral Response , 2016, Cerebral cortex.

[9]  C. Summerfield,et al.  Rhythmic Fluctuations in Evidence Accumulation during Decision Making in the Human Brain , 2012, Neuron.

[10]  Jeremy A. Taylor,et al.  Surprise responses in the human brain demonstrate statistical learning under high concurrent cognitive demand , 2016, npj Science of Learning.

[11]  Gary Aston-Jones,et al.  Phasic locus coeruleus activity regulates cortical encoding of salience information , 2018, Proceedings of the National Academy of Sciences.

[12]  C. Mathys,et al.  Hierarchical Prediction Errors in Midbrain and Basal Forebrain during Sensory Learning , 2013, Neuron.

[13]  Thomas E. Nichols,et al.  Nonparametric permutation tests for functional neuroimaging: A primer with examples , 2002, Human brain mapping.

[14]  S. Nieuwenhuis,et al.  The anatomical and functional relationship between the P3 and autonomic components of the orienting response. , 2011, Psychophysiology.

[15]  W. Schultz,et al.  Adaptive Prediction Error Coding in the Human Midbrain and Striatum Facilitates Behavioral Adaptation and Learning Efficiency , 2016, Neuron.

[16]  Berrin Maraşligil,et al.  İnsanlarda Yenilik N2 Yanıtı Hedef Uyaranların Zamansal Sınıflamasını Yansıtır , 2011 .

[17]  Michael J. Frank,et al.  Within and across-trial dynamics of human EEG reveal cooperative interplay between reinforcement learning and working memory , 2017, bioRxiv.

[18]  Jan R. Wessel,et al.  A Neural Mechanism for Surprise-related Interruptions of Visuospatial Working Memory , 2018, Cerebral cortex.

[19]  Anne G E Collins,et al.  Cognitive control over learning: creating, clustering, and generalizing task-set structure. , 2013, Psychological review.

[20]  Ryan P. Adams,et al.  Bayesian Online Changepoint Detection , 2007, 0710.3742.

[21]  Raymond J. Dolan,et al.  Neural signals encoding shifts in beliefs , 2016, NeuroImage.

[22]  C. Mathys,et al.  Hierarchical Prediction Errors in Midbrain and Basal Forebrain during Sensory Learning , 2013, Neuron.

[23]  E. Koechlin,et al.  Reasoning, Learning, and Creativity: Frontal Lobe Function and Human Decision-Making , 2012, PLoS biology.

[24]  Nicolas W. Schuck,et al.  Human Orbitofrontal Cortex Represents a Cognitive Map of State Space , 2016, Neuron.

[25]  Rick A Adams,et al.  Dopaminergic basis for signaling belief updates, but not surprise, and the link to paranoia , 2018, Proceedings of the National Academy of Sciences.

[26]  Joseph T. McGuire,et al.  Dissociable Forms of Uncertainty-Driven Representational Change Across the Human Brain , 2018, The Journal of Neuroscience.

[27]  J. Polich.  Theoretical Overview of P3a and P3b , 2003 .

[28]  Y. Niv,et al.  Learning latent structure: carving nature at its joints , 2010, Current Opinion in Neurobiology.

[29]  Matthew R Nassar,et al.  Age differences in learning emerge from an insufficient representation of uncertainty in older adults , 2016, Nature Communications.

[30]  J. Polich.  Updating P300: An integrative theory of P3a and P3b , 2007, Clinical Neurophysiology.

[31]  Markus Ullsperger,et al.  Real and Fictive Outcomes Are Processed Differently but Converge on a Common Adaptive Mechanism , 2013, Neuron.

[32]  Matthew R. Nassar,et al.  Catecholaminergic Regulation of Learning Rate in a Dynamic Environment , 2016, PLoS Comput. Biol..

[33]  Jan R. Wessel,et al.  On the Globality of Motor Suppression: Unexpected Events and Their Influence on Behavior and Cognition , 2017, Neuron.

[34]  B. Kopp,et al.  Prior probabilities modulate cortical surprise responses: A study of event-related potentials , 2016, Brain and Cognition.

[35]  H. Hoijtink,et al.  P300 amplitude variations, prior probabilities, and likelihoods: A Bayesian ERP study , 2016, Cognitive, affective & behavioral neuroscience.

[36]  S. Nieuwenhuis,et al.  Author response: Dynamic modulation of decision biases by brainstem arousal systems , 2017 .

[37]  Tim Fingscheidt,et al.  A Model-Based Approach to Trial-By-Trial P300 Amplitude Fluctuations , 2013, Front. Hum. Neurosci..

[38]  Robert C. Wilson,et al.  An Approximately Bayesian Delta-Rule Model Explains the Dynamics of Belief Updating in a Changing Environment , 2010, The Journal of Neuroscience.

[39]  Robert C. Wilson,et al.  Rational regulation of learning dynamics by pupil–linked arousal systems , 2012, Nature Neuroscience.

[40]  Timothy E. J. Behrens,et al.  Learning the value of information in an uncertain world , 2007, Nature Neuroscience.

[41]  S. Kelly,et al.  Internal and External Influences on the Rate of Sensory Evidence Accumulation in the Human Brain , 2013, The Journal of Neuroscience.

[42]  Sander Nieuwenhuis,et al.  Noradrenergic and Cholinergic Modulation of Belief Updating , 2018, Journal of Cognitive Neuroscience.

[43]  E. Donchin.  Presidential address, 1980. Surprise!...Surprise? , 1981, Psychophysiology.

[44]  C. Summerfield,et al.  Rhythmic fluctuations in evidence accumulation during decision making in the human brain , 2012 .

[45]  Joseph T. McGuire,et al.  Functionally Dissociable Influences on Learning Rate in a Dynamic Environment , 2014, Neuron.

[46]  Samuel W Cheadle,et al.  Adaptive Gain Control during Human Perceptual Choice , 2014, Neuron.

[47]  S. Kelly,et al.  A supramodal accumulation-to-bound signal that determines perceptual decisions in humans , 2012, Nature Neuroscience.

[48]  Timothy E. J. Behrens,et al.  Dissociable effects of surprise and model update in parietal and anterior cingulate cortex , 2013, Proceedings of the National Academy of Sciences.

[49]  S. Debener,et al.  Trial-by-Trial Fluctuations in the Event-Related Electroencephalogram Reflect Dynamic Changes in the Degree of Surprise , 2008, The Journal of Neuroscience.