Modulation of the feedback-related negativity by instruction and experience

A great deal of research focuses on how humans and animals learn from trial-and-error interactions with the environment. This research has established the viability of reinforcement learning as a model of behavioral adaptation and neural reward valuation. Error-driven learning is inefficient and dangerous, however. Fortunately, humans learn from nonexperiential sources of information as well. In the present study, we focused on one such form of information, instruction. We recorded event-related potentials as participants performed a probabilistic learning task. In one experiment condition, participants received feedback only about whether their responses were rewarded. In the other condition, they also received instruction about reward probabilities before performing the task. We found that instruction eliminated participants’ reliance on feedback as evidenced by their immediate asymptotic performance in the instruction condition. In striking contrast, the feedback-related negativity, an event-related potential component thought to reflect neural reward prediction error, continued to adapt with experience in both conditions. These results show that, whereas instruction may immediately control behavior, certain neural responses must be learned from experience.

[1]  W. Brown.  Animal Intelligence: Experimental Studies , 1912, Nature.

[2]  Masataka Watanabe,et al.  Prefrontal and cingulate unit activity during timing behavior in the monkey , 1979, Brain Research.

[3]  R. Johnson.  A triarchic model of P300 amplitude. , 1986, Psychophysiology.

[4]  Ray Johnson.  For Distinguished Early Career Contribution to Psychophysiology: Award Address, 1985 , 1986 .

[5]  C. Braun,et al.  Event-Related Brain Potentials Following Incorrect Feedback in a Time-Estimation Task: Evidence for a Generic Neural System for Error Detection , 1997, Journal of Cognitive Neuroscience.

[6]  Richard Gonzalez,et al.  On the Shape of the Probability Weighting Function , 1999, Cognitive Psychology.

[7]  M. Gluck,et al.  Interactive memory systems in the human brain , 2001, Nature.

[8]  Clay B. Holroyd,et al.  The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. , 2002, Psychological review.

[9]  Clay B. Holroyd,et al.  Dorsal anterior cingulate cortex shows fMRI response to internal and external error signals , 2004, Nature Neuroscience.

[10]  M. Walton,et al.  Action sets and decisions in the medial frontal cortex , 2004, Trends in Cognitive Sciences.

[11]  Samuel M. McClure,et al.  Separate Neural Systems Value Immediate and Delayed Monetary Rewards , 2004, Science.

[12]  Clay B. Holroyd,et al.  Reinforcement-related brain potentials from medial frontal cortex: origins and functional significance , 2004, Neuroscience & Biobehavioral Reviews.

[13]  Jeff T. Larsen,et al.  Context dependence of the event-related brain potential associated with reward and punishment. , 2004, Psychophysiology.

[14]  J. O'Doherty,et al.  Reward representations and reward-related learning in the human brain: insights from neuroimaging , 2004, Current Opinion in Neurobiology.

[15]  E. Miller,et al.  Different time courses of learning-related activity in the prefrontal cortex and striatum , 2005, Nature.

[16]  Clay B. Holroyd,et al.  ERP correlates of feedback and reward processing in the presence and absence of response choice. , 2005, Cerebral cortex.

[17]  P. Dayan,et al.  Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control , 2005, Nature Neuroscience.

[18]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[19]  Michael X. Cohen,et al.  Behavioral / Systems / Cognitive Reinforcement Learning Signals Predict Future Decisions , 2007 .

[20]  Clay B. Holroyd,et al.  The feedback correct-related positivity: sensitivity of the event-related brain potential to unexpected positive feedback. , 2008, Psychophysiology.

[21]  Daniel G. Dillon,et al.  Individual differences in reinforcement learning: Behavioral, electrophysiological, and neuroimaging correlates , 2008, NeuroImage.

[22]  M. Frank,et al.  Instructional control of reinforcement learning: A behavioral and neurocomputational investigation , 2009, Brain Research.

[23]  Michael J. Frank,et al.  Single dose of a dopamine agonist impairs reinforcement learning in humans: Evidence from event‐related potentials and computational modeling of striatal‐cortical function , 2009, Human brain mapping.

[24]  R. Baker,et al.  When is an error not a prediction error? An electrophysiological investigation , 2009, Cognitive, affective & behavioral neuroscience.

[25]  Richard Gonzalez,et al.  Computational Models for the Combination of Advice and Individual Learning , 2009, Cogn. Sci..

[26]  M. Delgado,et al.  How instructed knowledge modulates the neural systems of reward learning , 2010, Proceedings of the National Academy of Sciences.

[27]  P. Dayan,et al.  States versus Rewards: Dissociable Neural Prediction Error Signals Underlying Model-Based and Model-Free Reinforcement Learning , 2010, Neuron.

[28]  B. Balleine,et al.  Human and Rodent Homologies in Action Control: Corticostriatal Determinants of Goal-Directed and Habitual Action , 2010, Neuropsychopharmacology.

[29]  John R. Anderson,et al.  Learning from delayed feedback: neural responses in temporal credit assignment , 2011, Cognitive, affective & behavioral neuroscience.

[30]  A. Cooper,et al.  Predictive Reward Signal of Dopamine Neurons , 2011 .

[31]  P. Dayan,et al.  Model-based influences on humans’ choices and striatal prediction errors , 2011, Neuron.

[32]  M. Frank,et al.  Dopaminergic Genes Predict Individual Differences in Susceptibility to Confirmation Bias , 2011, The Journal of Neuroscience.

[33]  M. Rushworth,et al.  Model-based analyses: Promises, pitfalls, and example applications to the study of cognitive control , 2011, Quarterly journal of experimental psychology.