Neural computations associated with goal-directed choice

In goal-directed decision-making, animals choose between actions that are associated with different reward outcomes (e.g., foods) and with different costs (e.g., effort). Rapid advances have been made over the past few years in our understanding of the computations associated with goal-directed choices, and of how those computations are implemented in the brain. We review some important findings, with an emphasis on computational models, human fMRI, and monkey neurophysiology studies.

[1]  A. Tversky,et al.  Prospect theory: an analysis of decision under risk — Source link , 2007 .

[2]  Geoffrey Schoenbaum,et al.  The role of the orbitofrontal cortex in the pursuit of happiness and more specific rewards , 2008, Nature.

[3]  K. Doya,et al.  Representation of Action-Specific Reward Values in the Striatum , 2005, Science.

[4]  Wei Ji Ma,et al.  Bayesian inference with probabilistic population codes , 2006, Nature Neuroscience.

[5]  B. Balleine,et al.  Goal-directed instrumental action: contingency and incentive learning and their cortical substrates , 1998, Neuropharmacology.

[6]  J. Mink.  THE BASAL GANGLIA: FOCUSED SELECTION AND INHIBITION OF COMPETING MOTOR PROGRAMS , 1996, Progress in Neurobiology.

[7]  B. Balleine,et al.  Multiple Forms of Value Learning and the Function of Dopamine , 2009 .

[8]  Vivian V. Valentin,et al.  Determining the Neural Substrates of Goal-Directed Learning in the Human Brain , 2007, The Journal of Neuroscience.

[9]  P. Dayan,et al.  How Humans Integrate the Prospects of Pain and Reward during Choice , 2009, The Journal of Neuroscience.

[10]  J. Ditterich.  Evidence for time‐variant decision making , 2006, The European journal of neuroscience.

[11]  K. Saleem,et al.  Complementary circuits connecting the orbital and medial prefrontal networks with the temporal, insular, and opercular cortex in the macaque monkey , 2008, The Journal of comparative neurology.

[12]  Xiao-Jing Wang,et al.  Probabilistic Decision Making by Slow Reverberation in Cortical Circuits , 2002, Neuron.

[13]  R. Rescorla,et al.  A theory of Pavlovian conditioning : Variations in the effectiveness of reinforcement and nonreinforcement , 1972 .

[14]  Timothy E. J. Behrens,et al.  Choice, uncertainty and value in prefrontal and cingulate cortex , 2008, Nature Neuroscience.

[15]  Hatim A. Zariwala,et al.  Neural correlates, computation and behavioural impact of decision confidence , 2008, Nature.

[16]  R. Bogacz.  Optimal decision-making theories: linking neurobiology with behaviour , 2007, Trends in Cognitive Sciences.

[17]  Jochen Ditterich,et al.  Perceptual Decisions between Multiple Directions of Visual Motion , 2008, The Journal of Neuroscience.

[18]  A. Barto,et al.  Adaptive Critics and the Basal Ganglia , 1994 .

[19]  Timothy E. J. Behrens,et al.  Learning the value of information in an uncertain world , 2007, Nature Neuroscience.

[20]  Colin Camerer,et al.  Neuroeconomics: decision making and the brain , 2008 .

[21]  M. Shadlen,et al.  Decision-making with multiple alternatives , 2008, Nature Neuroscience.

[22]  B. Balleine,et al.  The role of incentive learning in instrumental outcome revaluation by sensory-specific satiety , 1998 .

[23]  M. Shadlen,et al.  Response of Neurons in the Lateral Intraparietal Area during a Combined Visual Discrimination Reaction Time Task , 2002, The Journal of Neuroscience.

[24]  Peter Dayan,et al.  The role of value systems in decision making. , 2008 .

[25]  J. O'Doherty,et al.  Orbitofrontal Cortex Encodes Willingness to Pay in Everyday Economic Transactions , 2007, The Journal of Neuroscience.

[26]  C. Padoa-Schioppa.  Range-Adapting Representation of Economic Value in the Orbitofrontal Cortex , 2009, The Journal of Neuroscience.

[27]  S. Kennerley,et al.  Evaluating choices by single neurons in the frontal lobe: outcome value encoded across multiple decision variables , 2009, The European journal of neuroscience.

[28]  Lorena R. R. Gianotti,et al.  Disruption of Right Prefrontal Cortex by Low-Frequency Repetitive Transcranial Magnetic Stimulation Induces Risk-Taking Behavior , 2006, The Journal of Neuroscience.

[29]  P. Dayan,et al.  Cortical substrates for exploratory decisions in humans , 2006, Nature.

[30]  Daeyeol Lee,et al.  Prefrontal Coding of Temporally Discounted Values during Intertemporal Choice , 2008, Neuron.

[31]  P. Cisek,et al.  Decisions in Changing Conditions: The Urgency-Gating Model , 2009, The Journal of Neuroscience.

[32]  P. Cisek.  Integrated Neural Processes for Defining Potential Actions and Deciding between Them: A Computational Model , 2006, The Journal of Neuroscience.

[33]  Philip L. Smith,et al.  A comparison of sequential sampling models for two-choice reaction time. , 2004, Psychological review.

[34]  J. Price,et al.  Midline and intralaminar thalamic connections with the orbital and medial prefrontal networks in macaque monkeys , 2007, The Journal of comparative neurology.

[35]  B. Balleine.  Neural bases of food-seeking: Affect, arousal and reward in corticostriatolimbic circuits , 2005, Physiology & Behavior.

[36]  James L. McClelland,et al.  The time course of perceptual choice: the leaky, competing accumulator model. , 2001, Psychological review.

[37]  J. O'Doherty,et al.  Evidence for a Common Representation of Decision Values for Dissimilar Goods in Human Ventromedial Prefrontal Cortex , 2009, The Journal of Neuroscience.

[38]  P. Glimcher,et al.  Title: the Neural Representation of Subjective Value under Risk and Ambiguity 1 2 , 2009 .

[39]  Jonathan D. Wallis,et al.  Neurons in the Frontal Lobe Encode the Value of Multiple Decision Variables , 2009, Journal of Cognitive Neuroscience.

[40]  P. Glimcher,et al.  The neural correlates of subjective value during intertemporal choice , 2007, Nature Neuroscience.

[41]  P. Redgrave,et al.  The basal ganglia: a vertebrate solution to the selection problem? , 1999, Neuroscience.

[42]  J. O'Doherty,et al.  Aversive Goal Values Are Negatively Encoded In the Medial Orbitofrontal Cortex , 2008 .

[43]  J. O'Doherty,et al.  Appetitive and Aversive goal values are encoded in the medial orbitofrontal cortex at the time of decision-making , 2009, NeuroImage.

[44]  Klaus Wunderlich,et al.  Neural computations underlying action-based decision making in the human brain , 2009, Proceedings of the National Academy of Sciences.

[45]  W. F. Prokasy,et al.  Classical conditioning II: Current research and theory. , 1972 .

[46]  Timothy E. J. Behrens,et al.  Frontal Cortex Subregions Play Distinct Roles in Choices between Actions and Stimuli , 2008, The Journal of Neuroscience.

[47]  Philip L. Smith,et al.  Psychology and neurobiology of simple decisions , 2004, Trends in Neurosciences.

[48]  C. Padoa-Schioppa,et al.  Neurons in the orbitofrontal cortex encode economic value , 2006, Nature.

[49]  Wolf Singer,et al.  Better than Conscious? Implications for Performance and Institutional Analysis , 2008 .

[50]  A. Graybiel.  The basal ganglia: learning new tricks and loving it , 2005, Current Opinion in Neurobiology.

[51]  J. O'Doherty,et al.  The Role of the Ventromedial Prefrontal Cortex in Abstract State-Based Inference during Decision Making in Humans , 2006, The Journal of Neuroscience.

[52]  Colin Camerer,et al.  A framework for studying the neurobiology of value-based decision making , 2008, Nature Reviews Neuroscience.

[53]  M. Rushworth,et al.  General Mechanisms for Making Decisions? This Review Comes from a Themed Issue on Cognitive Neuroscience Edited the Representation of Value and Reward Expectations in Frontal Cortex Reward Prediction Errors and Learning Rates Other Types of Prediction Error , 2022 .

[54]  A. Tversky,et al.  Prospect theory: analysis of decision under risk , 1979 .

[55]  Daeyeol Lee,et al.  Order-Dependent Modulation of Directional Signals in the Supplementary and Presupplementary Motor Areas , 2007, The Journal of Neuroscience.

[56]  P. Dayan,et al.  Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control , 2005, Nature Neuroscience.

[57]  J. Price,et al.  Architectonic subdivision of the human orbital and medial prefrontal cortex , 2003, The Journal of comparative neurology.

[58]  C. Padoa-Schioppa,et al.  The representation of economic value in the orbitofrontal cortex is invariant for changes of menu , 2008, Nature Neuroscience.

[59]  Colin Camerer,et al.  Self-control in decision-making involves modulation of the vmPFC valuation system , 2009, NeuroImage.

[60]  Sabrina M. Tom,et al.  The Neural Basis of Loss Aversion in Decision-Making Under Risk , 2007, Science.

[61]  E. Miller,et al.  Neuronal activity in primate dorsolateral and orbital prefrontal cortex during performance of a reward preference task , 2003, The European journal of neuroscience.

[62]  Kevin N. Gurney,et al.  The Basal Ganglia and Cortex Implement Optimal Decision Making Between Alternative Actions , 2007, Neural Computation.

[63]  P. Glimcher,et al.  Value Representations in the Primate Striatum during Matching Behavior , 2008, Neuron.

[64]  Timothy D. Hanks,et al.  Probabilistic Population Codes for Bayesian Decision Making , 2008, Neuron.

[65]  Roger Ratcliff,et al.  The Diffusion Decision Model: Theory and Data for Two-Choice Decision Tasks , 2008, Neural Computation.

[66]  Timothy Edward John Behrens,et al.  Effort-Based Cost–Benefit Valuation and the Human Brain , 2009, The Journal of Neuroscience.

[67]  Peter Dayan,et al.  Non-commercial Research and Educational Use including without Limitation Use in Instruction at Your Institution, Sending It to Specific Colleagues That You Know, and Providing a Copy to Your Institution's Administrator. All Other Uses, Reproduction and Distribution, including without Limitation Comm , 2022 .

[68]  P. Montague,et al.  Theoretical and Empirical Studies of Learning , 2009 .

[69]  J. Price,et al.  The organization of networks within the orbital and medial prefrontal cortex of rats, monkeys and humans. , 2000, Cerebral cortex.

[70]  Jonathan D. Cohen,et al.  The physics of optimal decision making: a formal analysis of models of performance in two-alternative forced-choice tasks. , 2006, Psychological review.

[71]  Xiao-Jing Wang.  Decision Making in Recurrent Neuronal Circuits , 2008, Neuron.

[72]  Dhanistha Panyasak,et al.  Circuits , 1995, Annals of the New York Academy of Sciences.

[73]  Xiao-Jing Wang,et al.  Cortico–basal ganglia circuit mechanism for a decision threshold in reaction time tasks , 2006, Nature Neuroscience.

[74]  Daniel T. Knoepfle,et al.  Value Computations in Ventral Medial Prefrontal Cortex during Charitable Decision Making Incorporate Input from Regions Involved in Social Cognition , 2010, The Journal of Neuroscience.

[75]  Ernst Fehr,et al.  Disrupting the prefrontal cortex diminishes the human ability to build a good reputation , 2009, Proceedings of the National Academy of Sciences.

[76]  M. Roesch,et al.  Neuronal activity in primate orbitofrontal cortex reflects the value of time. , 2005, Journal of neurophysiology.

[77]  Á. Pascual-Leone,et al.  Diminishing Reciprocal Fairness by Disrupting the Right Prefrontal Cortex , 2006, Science.

[78]  Shinsuke Shimojo,et al.  Repetitive transcranial magnetic stimulation over the right dorsolateral prefrontal cortex decreases valuations during food choices , 2009, The European journal of neuroscience.

[79]  Colin Camerer,et al.  Dissociating the Role of the Orbitofrontal Cortex and the Striatum in the Computation of Goal Values and Prediction Errors , 2008, The Journal of Neuroscience.