Feedback-related negativity codes outcome valence, but not outcome expectancy, during reversal learning

Optimal behavior depends on the ability to assess the predictive value of events and to adjust behavior accordingly. Outcome processing can be studied by using its electrophysiological signatures—that is, the feedback-related negativity (FRN) and the P300. A prominent reinforcement-learning model predicts an FRN on negative prediction errors, as well as implying a role for the FRN in learning and the adaptation of behavior. However, these predictions have recently been challenged. Notably, studies so far have used tasks in which the outcomes have been contingent on the response. In these paradigms, the need to adapt behavioral responses is present only for negative, not for positive feedback. The goal of the present study was to investigate the effects of positive as well as negative violations of expectancy on FRN amplitudes, without the usual confound of behavioral adjustments. A reversal-learning task was employed in which outcome value and outcome expectancy were orthogonalized; that is, both positive and negative outcomes were equally unexpected. The results revealed a double dissociation, with effects of valence but not expectancy on the FRN and, conversely, effects of expectancy but not valence on the P300. While FRN amplitudes were largest for negative-outcome trials, irrespective of outcome expectancy, P300 amplitudes were largest for unexpected-outcome trials, irrespective of outcome valence. These FRN effects were interpreted to reflect an evaluation along a good–bad dimension, rather than reflecting a negative prediction error or a role in behavioral adaptation. By contrast, the P300 reflects the updating of information relevant for behavior in a changing context.

[1]  Hong Li,et al.  This ought to be good: brain activity accompanying positive and negative expectations and outcomes. , 2011, Psychophysiology.

[2]  Clay B. Holroyd,et al.  ERP correlates of feedback and reward processing in the presence and absence of response choice. , 2005, Cerebral cortex.

[3]  Ryan K. Jessup,et al.  Error Effects in Anterior Cingulate Cortex Reverse when Error Likelihood Is High , 2010, The Journal of Neuroscience.

[4]  A. Engel,et al.  Trial-by-Trial Coupling of Concurrent Electroencephalogram and Functional Magnetic Resonance Imaging Identifies the Dynamics of Performance Monitoring , 2005, The Journal of Neuroscience.

[5]  R. Simons,et al.  The Neural Consequences of Flip-flopping: the Feedback- Related Negativity and Salience of Reward Prediction the Authors Thank Jenna Dietz and Natalie Shroyer for Their Help with Data Collection. We Also Thank , 2022 .

[6]  M. Walton,et al.  Action sets and decisions in the medial frontal cortex , 2004, Trends in Cognitive Sciences.

[7]  Clay B. Holroyd,et al.  Dorsal anterior cingulate cortex shows fMRI response to internal and external error signals , 2004, Nature Neuroscience.

[8]  Emanuel Donchin,et al.  Context updating and the P300 , 1998, Behavioral and Brain Sciences.

[9]  Michael Falkenstein,et al.  Errors, Conflicts, and the Brain , 2004 .

[10]  Clay B. Holroyd,et al.  Knowing good from bad: differential activation of human cortical areas by positive and negative outcomes , 2005, The European journal of neuroscience.

[11]  Werner Sommer,et al.  The expectancies that govern the P300 amplitude are mostly automatic and unconscious , 1998, Behavioral and Brain Sciences.

[12]  Uta Sailer,et al.  Manipulation of feedback expectancy and valence induces negative and positive reward prediction error signals manifest in event-related brain potentials. , 2011, Psychophysiology.

[13]  Clay B. Holroyd,et al.  The feedback-related negativity reflects the binary evaluation of good versus bad outcomes , 2006, Biological Psychology.

[14]  Clay B. Holroyd,et al.  Brain potentials associated with expected and unexpected good and bad outcomes. , 2005, Psychophysiology.

[15]  Clay B. Holroyd,et al.  Errors in reward prediction are re£ected in the event-related brain potential , 2003 .

[16]  E. Donchin,et al.  On quantifying surprise: the variation of event-related potentials with subjective probability. , 1977, Psychophysiology.

[17]  A. Sanfey,et al.  Independent Coding of Reward Magnitude and Valence in the Human Brain , 2004, The Journal of Neuroscience.

[18]  Clay B. Holroyd,et al.  Reward prediction error signals associated with a modified time estimation task. , 2007, Psychophysiology.

[19]  Rogier B. Mars,et al.  De Bruijn ERA, Mars RB, & Hulstijn W It wasn't me... or was it? How false feedback affects performance , 2004 .

[20]  Michael G. H. Coles,et al.  What if I told you: "You were wrong"? Brain potentials and behavioral adjustments elicited by feedback in a time-estimation task , 2004 .

[21]  Clay B. Holroyd,et al.  The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. , 2002, Psychological review.

[22]  P. Ekman Pictures of Facial Affect , 1976 .

[23]  Cameron S. Carter,et al.  Errors without conflict: Implications for performance monitoring theories of anterior cingulate cortex , 2004, Brain and Cognition.

[24]  Clay B. Holroyd,et al.  Reinforcement-related brain potentials from medial frontal cortex: origins and functional significance , 2004, Neuroscience & Biobehavioral Reviews.

[25]  Roshan Cools,et al.  Feedback-related Negativity Codes Prediction Error but Not Behavioral Adjustment during Probabilistic Reversal Learning , 2011, Journal of Cognitive Neuroscience.

[26]  David Goodman,et al.  Performance Monitoring in the Anterior Cingulate is Not All Error Related: Expectancy Deviation and the Representation of Action-Outcome Associations , 2007, Journal of Cognitive Neuroscience.

[27]  M. D’Esposito,et al.  Reversal learning in Parkinson's disease depends on medication status and outcome valence , 2006, Neuropsychologia.

[28]  Atsushi Sato,et al.  Effects of value and reward magnitude on feedback negativity and P300 , 2005, Neuroreport.

[29]  Reitherman Rw You were wrong , 1995 .

[30]  K. R. Ridderinkhof,et al.  The Role of the Medial Frontal Cortex in Cognitive Control , 2004, Science.

[31]  J. Cacioppo,et al.  Negative information weighs more heavily on the brain: The negativity bias in evaluative categorizations. , 1998 .

[32]  B. Sahakian,et al.  Acute Tryptophan Depletion in Healthy Volunteers Enhances Punishment Prediction but Does not Affect Reward Prediction , 2008, Neuropsychopharmacology.

[33]  Michael X. Cohen,et al.  Behavioral / Systems / Cognitive Reinforcement Learning Signals Predict Future Decisions , 2007 .

[34]  John R. Anderson,et al.  Modulation of the feedback-related negativity by instruction and experience , 2011, Proceedings of the National Academy of Sciences.

[35]  E. Donchin,et al.  Is the P300 component a manifestation of context updating? , 1988, Behavioral and Brain Sciences.

[36]  Maarten A. S. Boksem,et al.  The Importance of Failure: Feedback Related Negativity Predicts Motor Learning Efficiency , 2009, NeuroImage.

[37]  Clay B. Holroyd,et al.  It's worse than you thought: the feedback negativity and violations of reward prediction in gambling tasks. , 2007, Psychophysiology.

[38]  B. Sahakian,et al.  Tryptophan depletion disinhibits punishment but not reward prediction: implications for resilience , 2011, Psychopharmacology.

[39]  Jeff T. Larsen,et al.  Context dependence of the event-related brain potential associated with reward and punishment. , 2004, Psychophysiology.

[40]  Clay B. Holroyd,et al.  Sensitivity of electrophysiological activity from medial frontal cortex to utilitarian and performance feedback. , 2004, Cerebral cortex.

[41]  W. Schultz Getting Formal with Dopamine and Reward , 2002, Neuron.

[42]  Reginald B. Adams,et al.  Emotional expressions forecast approach-avoidance behavior , 2006 .

[43]  I. Daum,et al.  It is less than you expected: The feedback-related negativity reflects violations of reward magnitude expectations , 2010, Neuropsychologia.

[44]  M. Rushworth Intention, Choice, and the Medial Frontal Cortex , 2008, Annals of the New York Academy of Sciences.

[45]  Ivan Toni,et al.  Neural dynamics of error processing in medial frontal cortex , 2005, NeuroImage.

[46]  Michael X. Cohen,et al.  Reward expectation modulates feedback-related negativity and EEG spectra , 2007, NeuroImage.

[47]  Peter Ullsperger,et al.  Dissociable medial frontal negativities from a common monitoring system for self- and externally caused failure of goal achievement , 2009, NeuroImage.