Optimal decision making and the anterior cingulate cortex

Learning the value of options in an uncertain environment is central to optimal decision making. The anterior cingulate cortex (ACC) has been implicated in using reinforcement information to control behavior. Here we demonstrate that the ACC's critical role in reinforcement-guided behavior is neither in detecting nor in correcting errors, but in guiding voluntary choices based on the history of actions and outcomes. ACC lesions did not impair the performance of monkeys (Macaca mulatta) immediately after errors, but made them unable to sustain rewarded responses in a reinforcement-guided choice task and to integrate risk and payoff in a dynamic foraging task. These data suggest that the ACC is essential for learning the value of actions.

[1]  Masataka Watanabe,et al.  Prefrontal and cingulate unit activity during timing behavior in the monkey , 1979, Brain Research.

[2]  P. Goldman-Rakic,et al.  Prefrontal connections of medial motor areas in the rhesus monkey , 1993, The Journal of comparative neurology.

[3]  RP Dum,et al.  Topographic organization of corticospinal projections from the frontal lobe: motor areas on the lateral surface of the hemisphere , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[4]  D. Meyer,et al.  A Neural System for Error Detection and Compensation , 1993 .

[5]  RP Dum,et al.  Topographic organization of corticospinal projections from the frontal lobe: motor areas on the medial surface of the hemisphere , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[6]  R. Herrnstein,et al.  The Matching Law Papers in Psychology and Economics , 1997 .

[7]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[8]  R. Morecraft,et al.  Convergence of Limbic Input to the Cingulate Motor Cortex in the Rhesus Monkey , 1998, Brain Research Bulletin.

[9]  J. Tanji,et al.  Role for cingulate motor area cells in voluntary movement selection based on reward. , 1998, Science.

[10]  P S Goldman-Rakic,et al.  Widespread origin of the primate mesofrontal dopamine system. , 1998, Cerebral cortex.

[11]  Richard S. Sutton,et al.  Dimensions of Reinforcement Learning , 1998 .

[12]  A. Toga,et al.  The Rhesus Monkey Brain in Stereotaxic Coordinates , 1999 .

[13]  Michael L. Platt,et al.  Neural correlates of decision variables in parietal cortex , 1999, Nature.

[14]  D. Pandya,et al.  Fiber system linking the mid‐dorsolateral frontal cortex with the retrosplenial/presubicular region in the rhesus monkey , 1999, The Journal of comparative neurology.

[15]  T. Bussey,et al.  Role of prefrontal cortex in a network for arbitrary visuomotor mapping , 2000, Experimental Brain Research.

[16]  A. Roberts,et al.  Inhibitory control and affective processing in the prefrontal cortex: neuropsychological studies in the common marmoset. , 2000, Cerebral cortex.

[17]  M. Botvinick,et al.  Conflict monitoring and cognitive control. , 2001, Psychological review.

[18]  A. Dale,et al.  Dorsal anterior cingulate cortex: A role in reward-based decision making , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[19]  T. Paus.  Primate anterior cingulate cortex: Where motor control, drive and cognition interface , 2001, Nature Reviews Neuroscience.

[20]  W. Schultz.  Getting Formal with Dopamine and Reward , 2002, Neuron.

[21]  Clay B. Holroyd,et al.  The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. , 2002, Psychological review.

[22]  A. Turken,et al.  Dissociation between conflict detection and error monitoring in the human anterior cingulate cortex , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[23]  B. Richmond,et al.  Anterior Cingulate: Single Neuronal Signals Related to Degree of Reward Expectancy , 2002, Science.

[24]  K. A. Hadland,et al.  The anterior cingulate and reward-guided selection of actions. , 2003, Journal of neurophysiology.

[25]  D. V. von Cramon,et al.  Error Monitoring Using External Feedback: Specific Roles of the Habenular Complex, the Reward System, and the Cingulate Motor Area Revealed by Functional Magnetic Resonance Imaging , 2003, The Journal of Neuroscience.

[26]  E. Miller,et al.  Neuronal activity in primate dorsolateral and orbital prefrontal cortex during performance of a reward preference task , 2003, The European journal of neuroscience.

[27]  K. A. Hadland,et al.  The Effect of Cingulate Cortex Lesions on Task Switching and Working Memory , 2003, Journal of Cognitive Neuroscience.

[28]  Keiji Tanaka,et al.  Neuronal Correlates of Goal-Based Motor Selection in the Prefrontal Cortex , 2003, Science.

[29]  Joshua W. Brown,et al.  Performance Monitoring by the Anterior Cingulate Cortex During Saccade Countermanding , 2003, Science.

[30]  Tatsuo K Sato,et al.  Correlated Coding of Motivation and Outcome of Decision by Dopamine Neurons , 2003, The Journal of Neuroscience.

[31]  O. Hikosaka,et al.  Dopamine Neurons Can Represent Context-Dependent Prediction Error , 2004, Neuron.

[32]  T. Robbins,et al.  Inhibition and the right inferior frontal cortex , 2004, Trends in Cognitive Sciences.

[33]  W. Newsome,et al.  Matching Behavior and the Representation of Value in the Parietal Cortex , 2004, Science.

[34]  M. Walton,et al.  Action sets and decisions in the medial frontal cortex , 2004, Trends in Cognitive Sciences.

[35]  D. S. Olton,et al.  Effects of fornix transection and cingulate cortical ablation on spatial memory in rhesus monkeys , 2004, Experimental Brain Research.

[36]  M. Walton,et al.  Interactions between decision making and performance monitoring within prefrontal cortex , 2004, Nature Neuroscience.

[37]  D. Barraclough,et al.  Prefrontal cortex and decision making in a mixed-strategy game , 2004, Nature Neuroscience.

[38]  M. Roesch,et al.  Neuronal Activity Related to Reward Value and Motivation in Primate Frontal Cortex , 2004, Science.

[39]  D. Barraclough,et al.  Reinforcement learning and decision making in monkeys during a competitive game. , 2004, Brain research. Cognitive brain research.

[40]  M. Platt,et al.  Risk-sensitive neurons in macaque posterior cingulate cortex , 2005, Nature Neuroscience.

[41]  Matthew T. Kaufman,et al.  Distributed Neural Representation of Expected Value , 2005, The Journal of Neuroscience.

[42]  M. Roesch,et al.  Neuronal activity in macaque SEF and ACC during performance of tasks involving conflict. , 2005, Journal of neurophysiology.

[43]  K. Doya,et al.  Representation of Action-Specific Reward Values in the Striatum , 2005, Science.

[44]  Jonathan D. Cohen,et al.  An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance. , 2005, Annual review of neuroscience.

[45]  Joshua W. Brown,et al.  Learned Predictions of Error Likelihood in the Anterior Cingulate Cortex , 2005, Science.

[46]  Angela J. Yu,et al.  Uncertainty, Neuromodulation, and Attention , 2005, Neuron.

[47]  Andrew R. Mitz,et al.  Prefrontal Cortex Activity Related to Abstract Response Strategies , 2005, Neuron.

[48]  E. Procyk,et al.  Anterior cingulate error‐related activity is modulated by predicted reward , 2005, The European journal of neuroscience.

[49]  P. Glimcher,et al.  Midbrain Dopamine Neurons Encode a Quantitative Reward Prediction Error Signal , 2005, Neuron.

[50]  P. Glimcher,et al.  JOURNAL OF THE EXPERIMENTAL ANALYSIS OF BEHAVIOR 2005, 84, 555–579 NUMBER 3(NOVEMBER) DYNAMIC RESPONSE-BY-RESPONSE MODELS OF MATCHING BEHAVIOR IN RHESUS MONKEYS , 2022 .

[51]  M. Farah,et al.  Is anterior cingulate cortex necessary for cognitive control? , 2005, Brain : a journal of neurology.

[52]  E. Procyk,et al.  Reward encoding in the monkey anterior cingulate cortex. , 2006, Cerebral cortex.