Medial prefrontal cortex as an action-outcome predictor

The medial prefrontal cortex (mPFC) and especially anterior cingulate cortex is central to higher cognitive function and many clinical disorders, yet its basic function remains in dispute. Various competing theories of mPFC have treated effects of errors, conflict, error likelihood, volatility and reward, using findings from neuroimaging and neurophysiology in humans and monkeys. No single theory has been able to reconcile and account for the variety of findings. Here we show that a simple model based on standard learning rules can simulate and unify an unprecedented range of known effects in mPFC. The model reinterprets many known effects and suggests a new view of mPFC, as a region concerned with learning and predicting the likely outcomes of actions, whether good or bad. Cognitive control at the neural level is then seen as a result of evaluating the probable and actual outcomes of one's actions.

[1]  C. Eriksen,et al.  Effects of noise letters upon the identification of a target letter in a nonsearch task , 1974 .

[2]  J. Pearce,et al.  A model for Pavlovian learning: variations in the effectiveness of conditioned but not of unconditioned stimuli. , 1980, Psychological review.

[3]  Derek W. Johnston,et al.  The relationship between cardiovascular responses in the laboratory and in the field. , 1990, Psychophysiology.

[4]  J. Hohnsbein,et al.  Effects of crossmodal divided attention on late ERP components. II. Error processing in choice reaction tasks. , 1991, Electroencephalography and clinical neurophysiology.

[5]  이기수,et al.  II. , 1992 .

[6]  Richard S. Sutton,et al.  Reinforcement Learning with Replacing Eligibility Traces , 2005, Machine Learning.

[7]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[8]  C. Braun,et al.  Event-Related Brain Potentials Following Incorrect Feedback in a Time-Estimation Task: Evidence for a Generic Neural System for Error Detection , 1997, Journal of Cognitive Neuroscience.

[9]  M. Botvinick,et al.  Anterior cingulate cortex, error detection, and the online monitoring of performance. , 1998, Science.

[10]  J. Tanji,et al.  Role for cingulate motor area cells in voluntary movement selection based on reward. , 1998, Science.

[11]  J Schlag,et al.  Reward-predicting and reward-detecting neuronal activity in the primate supplementary eye field. , 2000, Journal of neurophysiology.

[12]  M. Coles,et al.  Performance monitoring in a confusing world: error-related brain activity, judgments of response accuracy, and types of errors. , 2000, Journal of experimental psychology. Human perception and performance.

[13]  E. Procyk,et al.  Anterior cingulate activity during routine and non-routine sequential behaviors in macaques , 2000, Nature Neuroscience.

[14]  M. Botvinick,et al.  Conflict monitoring and cognitive control. , 2001, Psychological review.

[15]  C. Carter,et al.  Anterior cingulate cortex activity and impaired self-monitoring of performance in patients with schizophrenia: an event-related fMRI study. , 2001, The American journal of psychiatry.

[16]  A. Dale,et al.  Dorsal anterior cingulate cortex: A role in reward-based decision making , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[17]  Carl R. Olson,et al.  Neuronal activity related to rule and conflict in macaque supplementary eye field , 2002, Physiology & Behavior.

[18]  Mitsuo Kawato,et al.  Multiple Model-Based Reinforcement Learning , 2002, Neural Computation.

[19]  Clay B. Holroyd,et al.  The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. , 2002, Psychological review.

[20]  B. Richmond,et al.  Anterior Cingulate: Single Neuronal Signals Related to Degree of Reward Expectancy , 2002, Science.

[21]  Jonathan D. Cohen,et al.  A computational model of anterior cingulate function in speeded response tasks: Effects of frequency, sequence, and conflict , 2002, Cognitive, affective & behavioral neuroscience.

[22]  G. Schoenbaum,et al.  Encoding Predicted Outcome and Acquired Value in Orbitofrontal Cortex during Cue Sampling Depends upon Input from Basolateral Amygdala , 2003, Neuron.

[23]  Keiji Tanaka,et al.  Neuronal Correlates of Goal-Based Motor Selection in the Prefrontal Cortex , 2003, Science.

[24]  Joshua W. Brown,et al.  Performance Monitoring by the Anterior Cingulate Cortex During Saccade Countermanding , 2003, Science.

[25]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[26]  M. Walton,et al.  Interactions between decision making and performance monitoring within prefrontal cortex , 2004, Nature Neuroscience.

[27]  Peter Dayan,et al.  Technical Note: Q-Learning , 2004, Machine Learning.

[28]  Jonathan D. Cohen,et al.  The neural basis of error detection: conflict monitoring and the error-related negativity. , 2004, Psychological review.

[29]  Clay B. Holroyd,et al.  A mechanism for error detection in speeded response time tasks. , 2005, Journal of experimental psychology. General.

[30]  M. Roesch,et al.  Neuronal activity in macaque SEF and ACC during performance of tasks involving conflict. , 2005, Journal of neurophysiology.

[31]  Joshua W. Brown,et al.  Learned Predictions of Error Likelihood in the Anterior Cingulate Cortex , 2005, Science.

[32]  Angela J. Yu,et al.  Uncertainty, Neuromodulation, and Attention , 2005, Neuron.

[33]  E. Procyk,et al.  Anterior cingulate error‐related activity is modulated by predicted reward , 2005, The European journal of neuroscience.

[34]  E. Procyk,et al.  Reward encoding in the monkey anterior cingulate cortex. , 2006, Cerebral cortex.

[35]  Clay B. Holroyd,et al.  Reward prediction error signals associated with a modified time estimation task. , 2007, Psychophysiology.

[36]  David Goodman,et al.  Performance Monitoring in the Anterior Cingulate is Not All Error Related: Expectancy Deviation and the Representation of Action-Outcome Associations , 2007, Journal of Cognitive Neuroscience.

[37]  Keiji Tanaka,et al.  Medial prefrontal cell activity signaling prediction errors of action values , 2007, Nature Neuroscience.

[38]  Joshua W. Brown,et al.  Risk prediction and aversion by anterior cingulate cortex , 2007, Cognitive, affective & behavioral neuroscience.

[39]  Timothy E. J. Behrens,et al.  Learning the value of information in an uncertain world , 2007, Nature Neuroscience.

[40]  E. Procyk,et al.  Expectations, gains, and losses in the anterior cingulate cortex , 2007, Cognitive, affective & behavioral neuroscience.

[41]  T. Braver,et al.  Explaining the many varieties of working memory variation: Dual mechanisms of cognitive control. , 2007 .

[42]  E. Aarts,et al.  Anticipatory activity in anterior cingulate cortex can be independent of conflict and error likelihood. , 2008, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[43]  Timothy E. J. Behrens,et al.  Frontal Cortex Subregions Play Distinct Roles in Choices between Actions and Stimuli , 2008, The Journal of Neuroscience.

[44]  E. Procyk,et al.  Behavioral Shifts and Action Valuation in the Anterior Cingulate Cortex , 2008, Neuron.

[45]  Borís Burle,et al.  Error Negativity Does Not Reflect Conflict: A Reappraisal of Conflict Monitoring and Anterior Cingulate Cortex Activity , 2008, Journal of Cognitive Neuroscience.

[46]  P. Haggard.  Human volition: towards a neuroscience of will , 2008, Nature Reviews Neuroscience.

[47]  S. Nieuwenhuis,et al.  Dissociating Response Conflict and Error Likelihood in Anterior Cingulate Cortex , 2009, The Journal of Neuroscience.

[48]  Joshua W. Brown.  Multiple cognitive control effects of error likelihood and conflict , 2009, Psychological research.

[49]  Joshua W. Brown,et al.  Conflict effects without conflict in anterior cingulate cortex: Multiple response effects and context specific representations , 2009, NeuroImage.

[50]  Jonathan D. Wallis,et al.  Neurons in the Frontal Lobe Encode the Value of Multiple Decision Variables , 2009, Journal of Cognitive Neuroscience.

[51]  Kristen A. Ford,et al.  BOLD fMRI activation for anti-saccades in nonhuman primates , 2009, NeuroImage.

[52]  Timothy Edward John Behrens,et al.  Effort-Based Cost–Benefit Valuation and the Human Brain , 2009, The Journal of Neuroscience.

[53]  Michael W. Cole,et al.  Cingulate cortex: Diverging data from humans and monkeys , 2009, Trends in Neurosciences.

[54]  P. Dayan,et al.  States versus Rewards: Dissociable Neural Prediction Error Signals Underlying Model-Based and Model-Free Reinforcement Learning , 2010, Neuron.

[55]  S. Taylor,et al.  Conditional Differences in Mean Reaction Time Explain Effects of Response Congruency, but not Accuracy, on Posterior Medial Frontal Cortex Activity , 2010, Front. Hum. Neurosci..

[56]  Michael J. Frank,et al.  Faculty Opinions recommendation of States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. , 2010 .

[57]  Ryan K. Jessup,et al.  Error Effects in Anterior Cingulate Cortex Reverse when Error Likelihood Is High , 2010, The Journal of Neuroscience.

[58]  Tobias Teichert,et al.  The dorsal medial frontal cortex is sensitive to time on task, not response conflict or error likelihood , 2011, NeuroImage.

[59]  Sabine Kastner,et al.  Functional heterogeneity of conflict, error, task-switching, and unexpectedness effects within medial prefrontal cortex , 2011, NeuroImage.