Choice, uncertainty and value in prefrontal and cingulate cortex

Reinforcement learning models that focus on the striatum and dopamine can predict the choices of animals and people. Representations of reward expectation and of reward prediction errors that are pertinent to decision making, however, are not confined to these regions but are also found in prefrontal and cingulate cortex. Moreover, decisions are not guided solely by the magnitude of the reward that is expected. Uncertainty in the estimate of the reward expectation, the value of information that might be gained by taking a course of action and the cost of an action all influence the manner in which decisions are made through prefrontal and cingulate cortex.

[1]  J. R.,et al.  Quantitative analysis , 1892, Nature.

[2]  R. Rescorla,et al.  A theory of Pavlovian conditioning : Variations in the effectiveness of reinforcement and nonreinforcement , 1972 .

[3]  E. Rolls,et al.  Hunger Modulates the Responses to Gustatory Stimuli of Single Neurons in the Caudolateral Orbitofrontal Cortex of the Macaque Monkey , 1989, The European journal of neuroscience.

[4]  W. Schultz,et al.  Responses of monkey dopamine neurons during learning of behavioral reactions. , 1992, Journal of neurophysiology.

[5]  M. Gabriel,et al.  Neurobiology of Cingulate Cortex and Limbic Thalamus , 1993 .

[6]  B. Vogt,et al.  Connections of the Monkey Cingulate Cortex , 1993 .

[7]  M. George Neurobiology of Cingulate Cortex and Limbic Thalamus , 1994, Neurology.

[8]  E. Rolls,et al.  Gustatory, olfactory, and visual convergence within the primate orbitofrontal cortex , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[9]  P. Strick,et al.  Spinal Cord Terminations of the Medial Wall Motor Areas in Macaque Monkeys , 1996, The Journal of Neuroscience.

[10]  S. Carmichael,et al.  Connectional networks within the orbital and medial prefrontal cortex of macaque monkeys. , 1996, The Journal of comparative neurology.

[11]  Masataka Watanabe Reward expectancy in primate prefrental neurons , 1996, Nature.

[12]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[13]  G. Schoenbaum,et al.  Orbitofrontal cortex and basolateral amygdala encode expected outcomes during learning , 1998, Nature Neuroscience.

[14]  P S Goldman-Rakic,et al.  Widespread origin of the primate mesofrontal dopamine system. , 1998, Cerebral cortex.

[15]  G. Schoenbaum,et al.  Orbitofrontal Cortex and Representation of Incentive Value in Associative Learning , 1999, The Journal of Neuroscience.

[16]  Michael L. Platt,et al.  Neural correlates of decision variables in parietal cortex , 1999, Nature.

[17]  E T Rolls,et al.  Responses to the Sensory Properties of Fat of Neurons in the Primate Orbitofrontal Cortex , 1999, The Journal of Neuroscience.

[18]  W. Schultz,et al.  Relative reward preference in primate orbitofrontal cortex , 1999, Nature.

[19]  S. Wise,et al.  Arbitrary associations between antecedents and actions , 2000, Trends in Neurosciences.

[20]  E. Murray,et al.  Control of Response Selection by Reinforcer Value Requires Interaction of Amygdala and Orbital Prefrontal Cortex , 2000, The Journal of Neuroscience.

[21]  S. Kakade,et al.  Learning and selective attention , 2000, Nature Neuroscience.

[22]  E. Procyk,et al.  Anterior cingulate activity during routine and non-routine sequential behaviors in macaques , 2000, Nature Neuroscience.

[23]  J. O'Doherty,et al.  Neural Responses during Anticipation of a Primary Taste Reward , 2002, Neuron.

[24]  D Akkal,et al.  Comparison of neuronal activity in the rostral supplementary and cingulate motor areas during a task with cognitive and motor demands , 2002, The European journal of neuroscience.

[25]  Clay B. Holroyd,et al.  The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. , 2002, Psychological review.

[26]  M. Walton,et al.  The Role of Rat Medial Frontal Cortex in Effort-Based Decision Making , 2002, The Journal of Neuroscience.

[27]  J. Deakin,et al.  Effects of quinolinic acid-induced lesions of the orbital prefrontal cortex on inter-temporal choice: a quantitative analysis , 2002, Psychopharmacology.

[28]  J. Deakin,et al.  Effects of lesions of the orbitofrontal cortex on sensitivity to delayed and probabilistic reinforcement , 2002, Psychopharmacology.

[29]  B. Richmond,et al.  Anterior Cingulate: Single Neuronal Signals Related to Degree of Reward Expectancy , 2002, Science.

[30]  W. Schultz,et al.  Coding of Predicted Reward Omission by Dopamine Neurons in a Conditioned Inhibition Paradigm , 2003, The Journal of Neuroscience.

[31]  E. Miller,et al.  Neuronal activity in primate dorsolateral and orbital prefrontal cortex during performance of a reward preference task , 2003, The European journal of neuroscience.

[32]  Matthew F S Rushworth,et al.  Functional Specialization within Medial Frontal Cortex of the Anterior Cingulate for Evaluating Effort-Related Decisions , 2003, The Journal of Neuroscience.

[33]  J. Price,et al.  Architectonic subdivision of the human orbital and medial prefrontal cortex , 2003, The Journal of comparative neurology.

[34]  W. Schultz,et al.  Discrete Coding of Reward Probability and Uncertainty by Dopamine Neurons , 2003, Science.

[35]  Keiji Tanaka,et al.  Neuronal Correlates of Goal-Based Motor Selection in the Prefrontal Cortex , 2003, Science.

[36]  Tatsuo K Sato,et al.  Correlated Coding of Motivation and Outcome of Decision by Dopamine Neurons , 2003, The Journal of Neuroscience.

[37]  Saori C. Tanaka,et al.  Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops , 2004, Nature Neuroscience.

[38]  W. Newsome,et al.  Matching Behavior and the Representation of Value in the Parietal Cortex , 2004, Science.

[39]  M. Walton,et al.  Interactions between decision making and performance monitoring within prefrontal cortex , 2004, Nature Neuroscience.

[40]  Karl J. Friston,et al.  Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning , 2004, Science.

[41]  Samuel M. McClure,et al.  Separate Neural Systems Value Immediate and Delayed Monetary Rewards , 2004, Science.

[42]  E. Murray,et al.  Bilateral Orbital Prefrontal Cortex Lesions in Rhesus Monkeys Disrupt Choices Guided by Both Reward Value and Reward Contingency , 2004, The Journal of Neuroscience.

[43]  K. Doya,et al.  A Neural Correlate of Reward-Based Behavioral Learning in Caudate Nucleus: A Functional Magnetic Resonance Imaging Study of a Stochastic Decision Task , 2004, The Journal of Neuroscience.

[44]  Karl J. Friston,et al.  Cortical areas and the selection of movement: a study with positron emission tomography , 1991, Experimental Brain Research.

[45]  A. Engel,et al.  Trial-by-Trial Coupling of Concurrent Electroencephalogram and Functional Magnetic Resonance Imaging Identifies the Dynamics of Performance Monitoring , 2005, The Journal of Neuroscience.

[46]  S. Quartz,et al.  Getting to Know You: Reputation and Trust in a Two-Person Economic Exchange , 2005, Science.

[47]  Matthew T. Kaufman,et al.  Distributed Neural Representation of Expected Value , 2005, The Journal of Neuroscience.

[48]  M. Roesch,et al.  Neuronal activity in primate orbitofrontal cortex reflects the value of time. , 2005, Journal of neurophysiology.

[49]  K. Doya,et al.  Representation of Action-Specific Reward Values in the Striatum , 2005, Science.

[50]  M. Delgado,et al.  Perceptions of moral character modulate the neural systems of reward during the trust game , 2005, Nature Neuroscience.

[51]  W. Schultz,et al.  Adaptive Coding of Reward Value by Dopamine Neurons , 2005, Science.

[52]  P. Dayan,et al.  Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control , 2005, Nature Neuroscience.

[53]  B. Vogt,et al.  Architecture and neurocytology of monkey cingulate gyrus , 2005, The Journal of comparative neurology.

[54]  Ivan Toni,et al.  Neural dynamics of error processing in medial frontal cortex , 2005, NeuroImage.

[55]  E. Procyk,et al.  Anterior cingulate error‐related activity is modulated by predicted reward , 2005, The European journal of neuroscience.

[56]  J. Tanji,et al.  Neurons in the rostral cingulate motor area monitor multiple phases of visuomotor behavior with modest parametric selectivity. , 2005, Journal of neurophysiology.

[57]  P. Glimcher,et al.  Midbrain Dopamine Neurons Encode a Quantitative Reward Prediction Error Signal , 2005, Neuron.

[58]  H. Seung,et al.  JOURNAL OF THE EXPERIMENTAL ANALYSIS OF BEHAVIOR 2005, 84, 581–617 NUMBER 3(NOVEMBER) LINEAR-NONLINEAR-POISSON MODELS OF PRIMATE CHOICE DYNAMICS , 2022 .

[59]  Michael L. Platt,et al.  Monkeys Pay Per View: Adaptive Valuation of Social Images by Rhesus Macaques , 2005, Current Biology.

[60]  J. Tanji,et al.  Representation of immediate and final behavioral goals in the monkey prefrontal cortex during an instructed delay period. , 2005, Cerebral cortex.

[61]  M. Kawato,et al.  Different neural correlates of reward expectation and reward expectation error in the putamen and caudate nucleus during stimulus-action-reward association learning. , 2006, Journal of neurophysiology.

[62]  Timothy E. J. Behrens,et al.  Optimal decision making and the anterior cingulate cortex , 2006, Nature Neuroscience.

[63]  C. Padoa-Schioppa,et al.  Neurons in the orbitofrontal cortex encode economic value , 2006, Nature.

[64]  Aldo Genovesio,et al.  Representation of Future and Previous Spatial Goals by Separate Neural Populations in Prefrontal Cortex , 2006, The Journal of Neuroscience.

[65]  C. Frith,et al.  Meeting of minds: the medial frontal cortex and social cognition , 2006, Nature Reviews Neuroscience.

[66]  Matthew F. S. Rushworth,et al.  Weighing up the benefits of work: Behavioral and neural analyses of effort-related decision making , 2006, Neural Networks.

[67]  W. Schultz,et al.  Influences of Rewarding and Aversive Outcomes on Activity in Macaque Lateral Prefrontal Cortex , 2006, Neuron.

[68]  P. Dayan,et al.  Cortical substrates for exploratory decisions in humans , 2006, Nature.

[69]  Birte U. Forstmann,et al.  Voluntary Selection of Task Sets Revealed by Functional Magnetic Resonance Imaging , 2006, Journal of Cognitive Neuroscience.

[70]  M. Quirk,et al.  Representation of Spatial Goals in Rat Orbitofrontal Cortex , 2006, Neuron.

[71]  M. Walton,et al.  Separate neural pathways process different decision costs , 2006, Nature Neuroscience.

[72]  S. Quartz,et al.  Neural Differentiation of Expected Reward and Risk in Human Subcortical Structures , 2006, Neuron.

[73]  Stephen V. Shepherd,et al.  Social status gates social attention in monkeys , 2006, Current Biology.

[74]  Mitsuo Kawato,et al.  Heterarchical reinforcement-learning model for integration of multiple cortico-striatal loops: fMRI examination in stimulus-action-reward association learning , 2006, Neural Networks.

[75]  J. O'Doherty,et al.  The Role of the Ventromedial Prefrontal Cortex in Abstract State-Based Inference during Decision Making in Humans , 2006, The Journal of Neuroscience.

[76]  Colin F Camerer,et al.  Agent-Specific Responses in the Cingulate Cortex During Economic Exchanges , 2006, Science.

[77]  E. Vaadia,et al.  Midbrain dopamine neurons encode decisions for future action , 2006, Nature Neuroscience.

[78]  M. Roesch,et al.  Encoding of Time-Discounted Rewards in Orbitofrontal Cortex Is Independent of Value Representation , 2006, Neuron.

[79]  S. Ishii,et al.  Resolution of Uncertainty in Prefrontal Cortex , 2006, Neuron.

[80]  M. Walton,et al.  A Role for the Macaque Anterior Cingulate Gyrus in Social Valuation , 2006, Science.

[81]  Daeyeol Lee,et al.  Activity in prefrontal cortex during dynamic selection of action sequences , 2006, Nature Neuroscience.

[82]  W. Schultz Behavioral theories and the neurophysiology of reward. , 2006, Annual review of psychology.

[83]  J. Tanji,et al.  Activity in the Lateral Prefrontal Cortex Reflects Multiple Steps of Future Events in Action Plans , 2006, Neuron.

[84]  E. Procyk,et al.  Reward encoding in the monkey anterior cingulate cortex. , 2006, Cerebral cortex.

[85]  Aaron C. Courville,et al.  Bayesian theories of conditioning in a changing world , 2006, Trends in Cognitive Sciences.

[86]  J. O'Doherty,et al.  Reward Value Coding Distinct From Risk Attitude-Related Uncertainty Coding in Human Reward Systems , 2006, Journal of neurophysiology.

[87]  Samuel M. McClure,et al.  Time Discounting for Primary Rewards , 2007, The Journal of Neuroscience.

[88]  M. Reuter,et al.  Genetically Determined Differences in Learning from Errors , 2007, Science.

[89]  Timothy Edward John Behrens,et al.  Contrasting roles for cingulate and orbitofrontal cortex in decisions and social behaviour , 2007, Trends in Cognitive Sciences.

[90]  Michael X. Cohen,et al.  Behavioral / Systems / Cognitive Reinforcement Learning Signals Predict Future Decisions , 2007 .

[91]  J. O'Doherty,et al.  Orbitofrontal Cortex Encodes Willingness to Pay in Everyday Economic Transactions , 2007, The Journal of Neuroscience.

[92]  David Goodman,et al.  Performance Monitoring in the Anterior Cingulate is Not All Error Related: Expectancy Deviation and the Representation of Action-Outcome Associations , 2007, Journal of Cognitive Neuroscience.

[93]  Keiji Tanaka,et al.  Medial prefrontal cell activity signaling prediction errors of action values , 2007, Nature Neuroscience.

[94]  M. Roesch,et al.  Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards , 2007, Nature Neuroscience.

[95]  B. Balleine,et al.  The Contribution of Orbitofrontal Cortex to Action Selection , 2007, Annals of the New York Academy of Sciences.

[96]  Timothy E. J. Behrens,et al.  Learning the value of information in an uncertain world , 2007, Nature Neuroscience.

[97]  H. Seo,et al.  Temporal Filtering of Reward Signals in the Dorsal Anterior Cingulate Cortex during a Mixed-Strategy Game , 2007, The Journal of Neuroscience.

[98]  G. Loewenstein,et al.  Neural Predictors of Purchases , 2007, Neuron.

[99]  Masataka Watanabe,et al.  Integration of cognitive and motivational context information in the primate prefrontal cortex. , 2007, Cerebral cortex.

[100]  D. Yves von Cramon,et al.  When goals are missed: Dealing with self-generated and externally induced failure , 2007, NeuroImage.

[101]  P. Glimcher,et al.  The neural correlates of subjective value during intertemporal choice , 2007, Nature Neuroscience.

[102]  M. Walton,et al.  Distinct contributions of frontal areas to emotion and social behaviour in the rat , 2007, The European journal of neuroscience.

[103]  C. Padoa-Schioppa,et al.  The representation of economic value in the orbitofrontal cortex is invariant for changes of menu , 2008, Nature Neuroscience.