Medial prefrontal cell activity signaling prediction errors of action values

To adapt behavior to a changing environment, one must monitor outcomes of executed actions and adjust subsequent actions accordingly. Involvement of the medial frontal cortex in performance monitoring has been suggested, but little is known about neural processes that link performance monitoring to performance adjustment. Here, we recorded from neurons in the medial prefrontal cortex of monkeys learning arbitrary action-outcome contingencies. Some cells preferentially responded to positive visual feedback stimuli and others to negative feedback stimuli. The magnitude of responses to positive feedback stimuli decreased over the course of behavioral adaptation, in correlation with decreases in the amount of prediction error of action values. Therefore, these responses in medial prefrontal cells may signal the direction and amount of error in prediction of values of executed actions to specify the adjustment in subsequent action selections.

[1]  A. Walker,et al.  A cytoarchitectural study of the prefrontal area of the macaque monkey , 1940 .

[2]  B. Skinner,et al.  Science and human behavior , 1953 .

[3]  A. Maslow.  Motivation and Personality , 1954 .

[4]  R. Woodworth.  Dynamics of behavior , 1958 .

[5]  Gordon W. Allport,et al.  Pattern and growth in personality , 1961 .

[6]  E. Fantino.  Choice and rate of reinforcement. , 1969, Journal of the experimental analysis of behavior.

[7]  Masataka Watanabe,et al.  Prefrontal and cingulate unit activity during timing behavior in the monkey , 1979, Brain Research.

[8]  Menek Goldstein,et al.  The dopaminergic innervation of monkey prefrontal cortex: a tyrosine hydroxylase immunohistochemical study , 1988, Brain Research.

[9]  D. Pandya,et al.  Architecture and intrinsic connections of the prefrontal cortex in the rhesus monkey , 1989, The Journal of comparative neurology.

[10]  M. Inase,et al.  Two movement-related foci in the primate cingulate cortex observed in signal-triggered and self-paced forelimb movements. , 1991, Journal of neurophysiology.

[11]  J. Hohnsbein,et al.  Effects of crossmodal divided attention on late ERP components. II. Error processing in choice reaction tasks. , 1991, Electroencephalography and clinical neurophysiology.

[12]  D. Meyer,et al.  A Neural System for Error Detection and Compensation , 1993 .

[13]  B A Williams,et al.  Conditioned Reinforcement: Experimental and Theoretical Issues , 1994, The Behavior analyst.

[14]  J. Price,et al.  Architectonic subdivision of the orbital and medial prefrontal cortex in the macaque monkey , 1994, The Journal of comparative neurology.

[15]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[16]  C. Braun,et al.  Event-Related Brain Potentials Following Incorrect Feedback in a Time-Estimation Task: Evidence for a Generic Neural System for Error Detection , 1997, Journal of Cognitive Neuroscience.

[17]  M. Botvinick,et al.  Anterior cingulate cortex, error detection, and the online monitoring of performance. , 1998, Science.

[18]  Brian Knutson,et al.  FMRI Visualization of Brain Activity during a Monetary Incentive Delay Task , 2000, NeuroImage.

[19]  R. Knight,et al.  Prefrontal–cingulate interactions in action monitoring , 2000, Nature Neuroscience.

[20]  W. Schultz.  Multiple reward signals in the brain , 2000, Nature Reviews Neuroscience.

[21]  E. Procyk,et al.  Anterior cingulate activity during routine and non-routine sequential behaviors in macaques , 2000, Nature Neuroscience.

[22]  A C Roberts,et al.  The Role of the Primate Amygdala in Conditioned Reinforcement , 2001, The Journal of Neuroscience.

[23]  Sham M. Kakade,et al.  Opponent interactions between serotonin and dopamine , 2002, Neural Networks.

[24]  Clay B. Holroyd,et al.  The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. , 2002, Psychological review.

[25]  B. Richmond,et al.  Anterior Cingulate: Single Neuronal Signals Related to Degree of Reward Expectancy , 2002, Science.

[26]  B. Burle,et al.  Error negativity on correct trials: a reexamination of available data , 2003, Biological Psychology.

[27]  D. V. von Cramon,et al.  Error Monitoring Using External Feedback: Specific Roles of the Habenular Complex, the Reward System, and the Cingulate Motor Area Revealed by Functional Magnetic Resonance Imaging , 2003, The Journal of Neuroscience.

[28]  W. Schultz,et al.  Discrete Coding of Reward Probability and Uncertainty by Dopamine Neurons , 2003, Science.

[29]  Joshua W. Brown,et al.  Performance Monitoring by the Anterior Cingulate Cortex During Saccade Countermanding , 2003, Science.

[30]  Tatsuo K Sato,et al.  Correlated Coding of Motivation and Outcome of Decision by Dopamine Neurons , 2003, The Journal of Neuroscience.

[31]  Clay B. Holroyd,et al.  Errors in reward prediction are re£ected in the event-related brain potential , 2003 .

[32]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[33]  Clay B. Holroyd,et al.  Dorsal anterior cingulate cortex shows fMRI response to internal and external error signals , 2004, Nature Neuroscience.

[34]  M. Walton,et al.  Action sets and decisions in the medial frontal cortex , 2004, Trends in Cognitive Sciences.

[35]  M. Walton,et al.  Interactions between decision making and performance monitoring within prefrontal cortex , 2004, Nature Neuroscience.

[36]  Peter Dayan,et al.  Technical Note: Q-Learning , 2004, Machine Learning.

[37]  Sidney J. Segalowitz,et al.  The effects of uncertainty in error monitoring on associated ERPs , 2004, Brain and Cognition.

[38]  Keiji Tanaka,et al.  The role of the medial prefrontal cortex in achieving goals , 2004, Current Opinion in Neurobiology.

[39]  A. Sanfey,et al.  Independent Coding of Reward Magnitude and Valence in the Human Brain , 2004, The Journal of Neuroscience.

[40]  T. Robbins.  Chemistry of the mind: Neurochemical modulation of prefrontal cortical function , 2005, The Journal of comparative neurology.

[41]  Michael J. Frank,et al.  Error-Related Negativity Predicts Reinforcement Learning and Conflict Biases , 2005, Neuron.

[42]  Wolfram Schultz,et al.  Rewarding properties of visual stimuli , 2005, Experimental Brain Research.

[43]  M. Roesch,et al.  Neuronal activity in macaque SEF and ACC during performance of tasks involving conflict. , 2005, Journal of neurophysiology.

[44]  Cheryl L. Dickter,et al.  Strategic control and medial frontal negativity: beyond errors and response conflict. , 2005, Psychophysiology.

[45]  Sander Nieuwenhuis,et al.  Mediofrontal negativities in the absence of responding. , 2005, Brain research. Cognitive brain research.

[46]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[47]  Ivan Toni,et al.  Neural dynamics of error processing in medial frontal cortex , 2005, NeuroImage.

[48]  E. Procyk,et al.  Anterior cingulate error‐related activity is modulated by predicted reward , 2005, The European journal of neuroscience.

[49]  P. Glimcher,et al.  Midbrain Dopamine Neurons Encode a Quantitative Reward Prediction Error Signal , 2005, Neuron.

[50]  D. Yves von Cramon,et al.  The Role of Intact Frontostriatal Circuits in Error Processing , 2006, Journal of Cognitive Neuroscience.

[51]  K. Doya,et al.  The computational neurobiology of learning and reward , 2006, Current Opinion in Neurobiology.

[52]  R. K. Simpson.  Nature Neuroscience , 2022 .