Statistical context dictates the relationship between feedback-related EEG signals and learning

Successful decision-making requires learning expectations based on experienced outcomes. This learning should be calibrated according to the surprise associated with an outcome, but also to the statistical context dictating the most likely source of surprise. For example, when occasional changepoints are expected, surprising outcomes should be weighted heavily, demanding increased learning. In contrast, when signal corruption is expected to occur occasionally, surprising outcomes can suggest a corrupt signal that should be ignored by learning systems. Here we dissociate surprising outcomes from the degree to which they demand learning using a predictive inference task and computational modeling. We show that the updating P300, a stimulus-locked electrophysiological response previously associated with adjustments in learning behavior, does so conditionally on the source of surprise. Larger P300 signals predicted greater learning in a changing context, but predicted less learning in a context where surprise was indicative of a one-off outlier (oddball). The conditional predictive relationship between the P300 and learning behavior was persistent even after adjusting for known sources of learning rate variability. Our results suggest that the P300 provides a surprise signal that is interpreted by downstream learning processes differentially according to statistical context in order to appropriately calibrate learning across complex environments.

[1]  S. Kelly,et al.  Internal and External Influences on the Rate of Sensory Evidence Accumulation in the Human Brain , 2013, The Journal of Neuroscience.

[2]  Sander Nieuwenhuis,et al.  Noradrenergic and Cholinergic Modulation of Belief Updating , 2018, Journal of Cognitive Neuroscience.

[3]  Rick A Adams,et al.  Dopaminergic basis for signaling belief updates, but not surprise, and the link to paranoia , 2018, Proceedings of the National Academy of Sciences.

[4]  E. Donchin.  Presidential address, 1980. Surprise!...Surprise? , 1981, Psychophysiology.

[5]  Joseph T. McGuire,et al.  Dissociable Forms of Uncertainty-Driven Representational Change Across the Human Brain , 2018, The Journal of Neuroscience.

[6]  P. Bossaerts,et al.  Neural Mechanisms Behind Identification of Leptokurtic Noise and Adaptive Behavioral Response , 2016, Cerebral cortex.

[7]  W. Schultz,et al.  Adaptive Prediction Error Coding in the Human Midbrain and Striatum Facilitates Behavioral Adaptation and Learning Efficiency , 2016, Neuron.

[8]  Jan R. Wessel,et al.  A Neural Mechanism for Surprise-related Interruptions of Visuospatial Working Memory , 2018, Cerebral cortex.

[9]  Anne G E Collins,et al.  Cognitive control over learning: creating, clustering, and generalizing task-set structure. , 2013, Psychological review.

[10]  Ryan P. Adams,et al.  Bayesian Online Changepoint Detection , 2007, 0710.3742.

[11]  Anne E. Urai,et al.  Pupil-linked arousal is driven by decision uncertainty and alters serial choice bias , 2017, Nature Communications.

[12]  M. McCabe.  Presidential address. , 1970, Irish nurses' journal.

[13]  Thomas E. Nichols,et al.  Nonparametric permutation tests for functional neuroimaging: A primer with examples , 2002, Human brain mapping.

[14]  S. Nieuwenhuis,et al.  The anatomical and functional relationship between the P3 and autonomic components of the orienting response. , 2011, Psychophysiology.

[15]  Matthew R. Nassar,et al.  Catecholaminergic Regulation of Learning Rate in a Dynamic Environment , 2016, PLoS Comput. Biol..

[16]  J. Polich.  Theoretical Overview of P3a and P3b , 2003 .

[17]  T. Sejnowski,et al.  Brain and cognition , 1989 .

[18]  S. Kelly,et al.  A supramodal accumulation-to-bound signal that determines perceptual decisions in humans , 2012, Nature Neuroscience.

[19]  Timothy E. J. Behrens,et al.  Dissociable effects of surprise and model update in parietal and anterior cingulate cortex , 2013, Proceedings of the National Academy of Sciences.

[20]  S. Debener,et al.  Trial-by-Trial Fluctuations in the Event-Related Electroencephalogram Reflect Dynamic Changes in the Degree of Surprise , 2008, The Journal of Neuroscience.

[21]  Raymond J. Dolan,et al.  Neural signals encoding shifts in beliefs , 2016, NeuroImage.

[22]  Robert C. Wilson,et al.  Orbitofrontal Cortex as a Cognitive Map of Task Space , 2014, Neuron.

[23]  Robert C. Wilson,et al.  An Approximately Bayesian Delta-Rule Model Explains the Dynamics of Belief Updating in a Changing Environment , 2010, The Journal of Neuroscience.

[24]  Robert C. Wilson,et al.  Rational regulation of learning dynamics by pupil–linked arousal systems , 2012, Nature Neuroscience.

[25]  Jan R. Wessel,et al.  On the Globality of Motor Suppression: Unexpected Events and Their Influence on Behavior and Cognition , 2017, Neuron.

[26]  Matthew R Nassar,et al.  Age differences in learning emerge from an insufficient representation of uncertainty in older adults , 2016, Nature Communications.

[27]  C. Summerfield,et al.  Rhythmic fluctuations in evidence accumulation during decision making in the human brain , 2012 .

[28]  C. Summerfield,et al.  Do humans make good decisions? , 2015, Trends in Cognitive Sciences.

[29]  Niels A. Kloosterman,et al.  Dynamic modulation of decision biases by brainstem arousal systems , 2017, eLife.

[30]  E. Koechlin,et al.  Reasoning, Learning, and Creativity: Frontal Lobe Function and Human Decision-Making , 2012, PLoS biology.

[31]  Nicolas W. Schuck,et al.  Human Orbitofrontal Cortex Represents a Cognitive Map of State Space , 2016, Neuron.

[32]  Joseph T. McGuire,et al.  Functionally Dissociable Influences on Learning Rate in a Dynamic Environment , 2014, Neuron.

[33]  J. Polich.  Updating P300: An integrative theory of P3a and P3b , 2007, Clinical Neurophysiology.

[34]  E. Donchin,et al.  Is the P300 component a manifestation of context updating? , 1988, Behavioral and Brain Sciences.

[35]  Joshua I. Gold,et al.  Bayesian Online Learning of the Hazard Rate in Change-Point Problems , 2010, Neural Computation.

[36]  C. Summerfield,et al.  Rhythmic Fluctuations in Evidence Accumulation during Decision Making in the Human Brain , 2012, Neuron.

[37]  Gary Aston-Jones,et al.  Phasic locus coeruleus activity regulates cortical encoding of salience information , 2018, Proceedings of the National Academy of Sciences.

[38]  C. Mathys,et al.  Hierarchical Prediction Errors in Midbrain and Basal Forebrain during Sensory Learning , 2013, Neuron.

[39]  Berrin Maraşligil,et al.  İnsanlarda Yenilik N2 Yanıtı Hedef Uyaranların Zamansal Sınıflamasını Yansıtır , 2011 .

[40]  J. Polich.  Updating P 300 : An Integrative Theory of P 3 a and P 3 b , 2009 .

[41]  Y. Niv,et al.  Learning latent structure: carving nature at its joints , 2010, Current Opinion in Neurobiology.

[42]  Markus Ullsperger,et al.  Real and Fictive Outcomes Are Processed Differently but Converge on a Common Adaptive Mechanism , 2013, Neuron.

[43]  Samuel W Cheadle,et al.  Adaptive Gain Control during Human Perceptual Choice , 2014, Neuron.

[44]  Jeremy A. Taylor,et al.  Surprise responses in the human brain demonstrate statistical learning under high concurrent cognitive demand , 2016, npj Science of Learning.

[45]  Michael J Frank,et al.  Within- and across-trial dynamics of human EEG reveal cooperative interplay between reinforcement learning and working memory , 2017, Proceedings of the National Academy of Sciences.

[46]  Tim Fingscheidt,et al.  A computational analysis of the neural bases of Bayesian inference , 2015, NeuroImage.

[47]  J. Gold,et al.  Arousal-related adjustments of perceptual biases optimize perception in dynamic environments , 2017, Nature Human Behaviour.

[48]  Timothy E. J. Behrens,et al.  Learning the value of information in an uncertain world , 2007, Nature Neuroscience.