A framework for studying the neurobiology of value-based decision making

Neuroeconomics is the study of the neurobiological and computational basis of value-based decision making. Its goal is to provide a biologically based account of human behaviour that can be applied in both the natural and the social sciences. This Review proposes a framework to investigate different aspects of the neurobiology of decision making. The framework allows us to bring together recent findings in the field, highlight some of the most important outstanding problems, define a common lexicon that bridges the different disciplines that inform neuroeconomics, and point the way to future applications.

[1]  R J HERRNSTEIN,et al.  Relative and absolute strength of response as a function of frequency of reinforcement. , 1961, Journal of the experimental analysis of behavior.

[2]  W. Mischel,et al.  Instrumental ideation in delay of gratification. , 1974, Child development.

[3]  A. Tversky,et al.  Prospect theory: analysis of decision under risk , 1979 .

[4]  A. Tversky,et al.  Prospect Theory : An Analysis of Decision under Risk Author ( s ) : , 2007 .

[5]  G. Loewenstein.  Anticipation and the Valuation of Delayed Consumption , 1987 .

[6]  J. E. Mazur.  Estimation of indifference points with an adjusting-delay procedure. , 1988, Journal of the experimental analysis of behavior.

[7]  K. H. Britten,et al.  Neuronal correlates of a perceptual decision , 1989, Nature.

[8]  I. Gilboa,et al.  Maxmin Expected Utility with Non-Unique Prior , 1989 .

[9]  Colin F. Camerer,et al.  Recent developments in modelling preferences: Uncertainty and ambiguitiy , 1991 .

[10]  Colin Camerer,et al.  Recent developments in modeling preferences: Uncertainty and ambiguity , 1992 .

[11]  A. Tversky,et al.  Advances in prospect theory: Cumulative representation of uncertainty , 1992 .

[12]  W. Schultz,et al.  Importance of unpredictability for reward responses in primate dopamine neurons. , 1994, Journal of neurophysiology.

[13]  P. Dayan,et al.  A framework for mesencephalic dopamine systems based on predictive Hebbian learning , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[14]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[15]  Ilya Segal,et al.  Solutions manual for Microeconomic theory : Mas-Colell, Whinston and Green , 1997 .

[16]  M. L. Wood,et al.  Functional MRI of pain- and attention-related activations in the human cingulate cortex. , 1997, Journal of neurophysiology.

[17]  J. Hollerman,et al.  Dopamine neurons report an error in the temporal prediction of reward during learning , 1998, Nature Neuroscience.

[18]  J. Hollerman,et al.  Modifications of reward expectation-related neuronal activity during learning in primate striatum. , 1998, Journal of neurophysiology.

[19]  M. Fendt,et al.  The neuroanatomical and neurochemical basis of conditioned fear , 1999, Neuroscience & Biobehavioral Reviews.

[20]  Michael L. Platt,et al.  Neural correlates of decision variables in parietal cortex , 1999, Nature.

[21]  M. Shadlen,et al.  Neural correlates of a decision in the dorsolateral prefrontal cortex of the macaque , 1999, Nature Neuroscience.

[22]  N. Costes,et al.  Haemodynamic brain responses to acute pain in humans: sensory and attentional networks. , 1999, Brain : a journal of neurology.

[23]  E T Rolls,et al.  Sensory-specific satiety-related olfactory activation of the human orbitofrontal cortex. , 2000, Neuroreport.

[24]  E T Rolls,et al.  Sensory‐specific satiety‐related olfactory activation of the human orbitofrontal cortex , 2000, Neuroreport.

[25]  A. Tversky,et al.  Choices, Values, and Frames , 2000 .

[26]  Brian Knutson,et al.  FMRI Visualization of Brain Activity during a Monetary Incentive Delay Task , 2000, NeuroImage.

[27]  L. Nystrom,et al.  Tracking the hemodynamic responses to reward and punishment in the striatum. , 2000, Journal of neurophysiology.

[28]  E. Miller,et al.  An integrative theory of prefrontal cortex function. , 2001, Annual review of neuroscience.

[29]  Gregory G. Brown,et al.  Prefrontal, Parietal, and Temporal Cortex Networks Underlie Decision-Making in the Presence of Uncertainty , 2001, NeuroImage.

[30]  D. Kahneman,et al.  Functional Imaging of Neural Responses to Expectancy and Experience of Monetary Gains and Losses tasks with monetary payoffs , 2001 .

[31]  Peter Dayan,et al.  Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems , 2001 .

[32]  R. Bandler,et al.  Parallel circuits mediating distinct emotional coping reactions to different types of stress , 2001, Neuroscience & Biobehavioral Reviews.

[33]  Brian Knutson,et al.  Dissociation of reward anticipation and outcome with event-related fMRI , 2001, Neuroreport.

[34]  Alan C. Evans,et al.  Changes in brain activity related to eating chocolate: from pleasure to aversion. , 2001, Brain : a journal of neurology.

[35]  J. Gold,et al.  Neural computations that underlie decisions about sensory stimuli , 2001, Trends in Cognitive Sciences.

[36]  R. Zatorre,et al.  Intensely pleasurable responses to music correlate with activity in brain regions implicated in reward and emotion , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[37]  Sham M. Kakade,et al.  Opponent interactions between serotonin and dopamine , 2002, Neural Networks.

[38]  J. Gold,et al.  Banburismus and the Brain Decoding the Relationship between Sensory Stimuli, Decisions, and Reward , 2002, Neuron.

[39]  Henrik Walter,et al.  Cultural objects modulate reward circuitry , 2002, Neuroreport.

[40]  B. Everitt,et al.  Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex , 2002, Neuroscience & Biobehavioral Reviews.

[41]  P. Montague,et al.  Activity in human ventral striatum locked to errors of reward prediction , 2002, Nature Neuroscience.

[42]  G. Loewenstein,et al.  Time Discounting and Time Preference: A Critical Review , 2002 .

[43]  G. Glover,et al.  Dissociated neural representations of intensity and valence in human olfaction , 2003, Nature Neuroscience.

[44]  E. Rolls,et al.  Activation of the human orbitofrontal cortex to a liquid food stimulus is correlated with its subjective pleasantness. , 2003, Cerebral cortex.

[45]  Corianne Rogalsky,et al.  Increased activation in the right insula during risk-taking decision making is related to harm avoidance and neuroticism , 2003, NeuroImage.

[46]  E. Rolls,et al.  Taste‐olfactory convergence, and the representation of the pleasantness of flavour, in the human brain , 2003, The European journal of neuroscience.

[47]  K. Berridge,et al.  Erratum to: “Parsing reward” [Trends Neurosci. 26 (2003) 507–513] , 2003, Trends in Neurosciences.

[48]  E. Rolls,et al.  Human cortical responses to water in the mouth, and the effects of thirst. , 2003, Journal of neurophysiology.

[49]  M. Mesulam,et al.  Dissociation of Neural Representation of Intensity and Affective Valuation in Human Gustation , 2003, Neuron.

[50]  E. Miller,et al.  Neuronal activity in primate dorsolateral and orbital prefrontal cortex during performance of a reward preference task , 2003, The European journal of neuroscience.

[51]  Kent C. Berridge,et al.  Pleasures of the brain , 2003, Brain and Cognition.

[52]  Karl J. Friston,et al.  Temporal Difference Models and Reward-Related Learning in the Human Brain , 2003, Neuron.

[53]  S. Killcross,et al.  Inactivation of the infralimbic prefrontal cortex reinstates goal-directed responding in overtrained rats , 2003, Behavioural Brain Research.

[54]  K. Berridge,et al.  Parsing reward , 2003, Trends in Neurosciences.

[55]  S. Killcross,et al.  Coordination of actions and habits in the medial prefrontal cortex of rats. , 2003, Cerebral cortex.

[56]  Martin P Paulus,et al.  Ventromedial prefrontal cortex activation is critical for preference judgments , 2003, Neuroreport.

[57]  P. Montague,et al.  Dynamic Gain Control of Dopamine Delivery in Freely Moving Animals , 2004, The Journal of Neuroscience.

[58]  S. Siegel,et al.  Neurobiology of Mental Illness , 2004 .

[59]  Saori C. Tanaka,et al.  Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops , 2004, Nature Neuroscience.

[60]  Samuel M. McClure,et al.  Neural Correlates of Behavioral Preference for Culturally Familiar Drinks , 2004, Neuron.

[61]  Margo Wilson,et al.  Do pretty women inspire men to discount the future? , 2004, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[62]  Samuel M. McClure,et al.  Separate Neural Systems Value Immediate and Delayed Monetary Rewards , 2004, Science.

[63]  G. Pagnoni,et al.  Human Striatal Responses to Monetary Reward Depend On Saliency , 2004, Neuron.

[64]  W. Schultz.  Neural coding of basic reward terms of animal learning theory, game theory, microeconomics and behavioural ecology , 2004, Current Opinion in Neurobiology.

[65]  Michael J. Frank,et al.  By Carrot or by Stick: Cognitive Reinforcement Learning in Parkinsonism , 2004, Science.

[66]  P. Holland,et al.  Amygdala–frontal interactions and reward expectancy , 2004, Current Opinion in Neurobiology.

[67]  R. Hertwig,et al.  Decisions from Experience and the Effect of Rare Events in Risky Choice , 2004, Psychological science.

[68]  D. Barraclough,et al.  Prefrontal cortex and decision making in a mixed-strategy game , 2004, Nature Neuroscience.

[69]  Massimo Marinacci,et al.  Differentiating ambiguity and ambiguity attitude , 2004, J. Econ. Theory.

[70]  A. Redish,et al.  Addiction as a Computational Process Gone Awry , 2004, Science.

[71]  Colin Camerer,et al.  Neural Systems Responding to Degrees of Uncertainty in Human Decision-Making , 2005, Science.

[72]  A. Chatterjee.  The Neurobiology of Mental Illness , 2005 .

[73]  Daniel Read,et al.  Four Score and Seven Years from Now: The Date/Delay Effect in Temporal Discounting , 2005, Manag. Sci..

[74]  M. Roesch,et al.  Orbitofrontal Cortex, Associative Learning, and Expectancies , 2005, Neuron.

[75]  W. Schultz,et al.  Adaptive Coding of Reward Value by Dopamine Neurons , 2005, Science.

[76]  P. Dayan,et al.  Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control , 2005, Nature Neuroscience.

[77]  G. McCarthy,et al.  Decisions under Uncertainty: Probabilistic Context Influences Activation of Prefrontal and Parietal Cortices , 2005, The Journal of Neuroscience.

[78]  B. Balleine.  Neural bases of food-seeking: Affect, arousal and reward in corticostriatolimbic circuits , 2005, Physiology & Behavior.

[79]  M. Hauser,et al.  The ecology and evolution of patience in two New World monkeys , 2005, Biology Letters.

[80]  E. Rolls,et al.  Cognitive Modulation of Olfactory Processing , 2005, Neuron.

[81]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[82]  P. Glimcher,et al.  Midbrain Dopamine Neurons Encode a Quantitative Reward Prediction Error Signal , 2005, Neuron.

[83]  H. Seung,et al.  JOURNAL OF THE EXPERIMENTAL ANALYSIS OF BEHAVIOR 2005, 84, 581–617 NUMBER 3(NOVEMBER) LINEAR-NONLINEAR-POISSON MODELS OF PRIMATE CHOICE DYNAMICS , 2022 .

[84]  David S. Leland,et al.  Increased risk-taking decision-making but not altered response to punishment in stimulant-using young adults. , 2005, Drug and alcohol dependence.

[85]  B. Balleine,et al.  Blockade of NMDA receptors in the dorsomedial striatum prevents action–outcome learning in instrumental conditioning , 2005, The European journal of neuroscience.

[86]  G. Pagnoni,et al.  Neurobiological Substrates of Dread , 2006, Science.

[87]  K. Berman,et al.  Cerebral Cortex doi:10.1093/cercor/bhj004 Neural Coding of Distinct Statistical Properties of Reward Information in Humans , 2005 .

[88]  C. Padoa-Schioppa,et al.  Neurons in the orbitofrontal cortex encode economic value , 2006, Nature.

[89]  S. Hyman,et al.  Neural mechanisms of addiction: the role of reward-related learning and memory. , 2006, Annual review of neuroscience.

[90]  Peter K. Schott,et al.  How Basic Are Behavioral Biases? Evidence from Capuchin Monkey Trading Behavior , 2006, Journal of Political Economy.

[91]  H. Yin,et al.  The role of the basal ganglia in habit formation , 2006, Nature Reviews Neuroscience.

[92]  Jonathan D. Cohen,et al.  Imaging valuation models in human choice. , 2006, Annual review of neuroscience.

[93]  D. Kumaran,et al.  Frames, Biases, and Rational Decision-Making in the Human Brain , 2006, Science.

[94]  S. Quartz,et al.  Neural Differentiation of Expected Reward and Risk in Human Subcortical Structures , 2006, Neuron.

[95]  P. Dayan,et al.  Opinion TRENDS in Cognitive Sciences Vol.10 No.8 Full text provided by www.sciencedirect.com A normative perspective on motivation , 2022 .

[96]  Evan M. Gordon,et al.  Neural Signatures of Economic Preferences for Risk and Ambiguity , 2006, Neuron.

[97]  Peter Dayan,et al.  Non-commercial Research and Educational Use including without Limitation Use in Instruction at Your Institution, Sending It to Specific Colleagues That You Know, and Providing a Copy to Your Institution's Administrator. All Other Uses, Reproduction and Distribution, including without Limitation Comm , 2022 .

[98]  M. Bouton.  Learning and Behavior: A Contemporary Synthesis , 2006 .

[99]  Michael J. Frank,et al.  Hold your horses: A dynamic computational role for the subthalamic nucleus in decision making , 2006, Neural Networks.

[100]  Kyle S. Smith,et al.  Hedonic Hot Spots in the Brain , 2006, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[101]  J. Gläscher,et al.  Dissociable Systems for Gain- and Loss-Related Value Predictions and Errors of Prediction in the Human Brain , 2006, The Journal of Neuroscience.

[102]  D. Adams,et al.  Brain mechanisms of aggressive behavior: An updated review , 2006, Neuroscience & Biobehavioral Reviews.

[103]  A. Tversky,et al.  Prospect theory: an analysis of decision under risk — Source link , 2007 .

[104]  R. Malenka,et al.  Synaptic plasticity and addiction , 2007, Nature Reviews Neuroscience.

[105]  J. O'Doherty,et al.  Reward Value Coding Distinct From Risk Attitude-Related Uncertainty Coding in Human Reward Systems , 2006, Journal of neurophysiology.

[106]  P. Glimcher,et al.  Statistics of midbrain dopamine neuron spike trains in the awake primate. , 2007, Journal of neurophysiology.

[107]  Samuel M. McClure,et al.  Time Discounting for Primary Rewards , 2007, The Journal of Neuroscience.

[108]  K. Preuschoff,et al.  Adding Prediction Risk to the Theory of Reward Learning , 2007, Annals of the New York Academy of Sciences.

[109]  J. Wallis.  Orbitofrontal cortex and its contribution to decision-making. , 2007, Annual review of neuroscience.

[110]  Teck-Hua Ho,et al.  Self-tuning experience weighted attraction learning in games , 2007, J. Econ. Theory.

[111]  M. Farah,et al.  Cerebral Cortex doi:10.1093/cercor/bhl176 The Role of Ventromedial Prefrontal Cortex in Decision Making: Judgment under Uncertainty or Judgment Per Se? , 2007 .

[112]  Martin P Paulus,et al.  Decision-Making Dysfunctions in Psychiatry—Altered Homeostatic Processing? , 2007, Science.

[113]  J. O'Doherty,et al.  Orbitofrontal Cortex Encodes Willingness to Pay in Everyday Economic Transactions , 2007, The Journal of Neuroscience.

[114]  Colin Camerer,et al.  Social neuroeconomics: the neural circuitry of social preferences , 2007, Trends in Cognitive Sciences.

[115]  W. Schultz.  Multiple dopamine functions at different time courses. , 2007, Annual review of neuroscience.

[116]  J. Gold,et al.  The neural basis of decision making. , 2007, Annual review of neuroscience.

[117]  E. Phelps,et al.  Social learning of fear , 2007, Nature Neuroscience.

[118]  H. Seo,et al.  Temporal Filtering of Reward Signals in the Dorsal Anterior Cingulate Cortex during a Mixed-Strategy Game , 2007, The Journal of Neuroscience.

[119]  G. Berns,et al.  Intertemporal choice – toward an integrative framework , 2007, Trends in Cognitive Sciences.

[120]  Sabrina M. Tom,et al.  The Neural Basis of Loss Aversion in Decision-Making Under Risk , 2007, Science.

[121]  Adam Johnson,et al.  A Computational Model of Craving and Obsession , 2007, Annals of the New York Academy of Sciences.

[122]  A. Bechara,et al.  PSYCHOLOGICAL SCIENCE Research Article Neural Correlates of Adaptive Decision Making for Risky Gains and Losses , 2022 .

[123]  Thomas E. Hazy,et al.  Towards an executive without a homunculus: computational models of the prefrontal cortex/basal ganglia system , 2007, Philosophical Transactions of the Royal Society B: Biological Sciences.

[124]  Peter Dayan,et al.  Hippocampal Contributions to Control: The Third Way , 2007, NIPS.

[125]  P. Dayan,et al.  Differential Encoding of Losses and Gains in the Human Striatum , 2007, The Journal of Neuroscience.

[126]  Kevin McCabe,et al.  Neural signature of fictive learning signals in a sequential investment task , 2007, Proceedings of the National Academy of Sciences.

[127]  P. Glimcher,et al.  The neural correlates of subjective value during intertemporal choice , 2007, Nature Neuroscience.

[128]  J. O'Doherty,et al.  Marketing actions can modulate neural representations of experienced pleasantness , 2008, Proceedings of the National Academy of Sciences.

[129]  Daeyeol Lee.  Game theory and neural basis of social decision making , 2008, Nature Neuroscience.

[130]  Jerome R. Busemeyer,et al.  Computational Models of Decision Making , 2003 .

[131]  Colin Camerer,et al.  Dissociating the Role of the Orbitofrontal Cortex and the Striatum in the Computation of Goal Values and Prediction Errors , 2008, The Journal of Neuroscience.

[132]  Leslie G. Ungerleider,et al.  The neural systems that mediate human perceptual decision making , 2008, Nature Reviews Neuroscience.

[133]  E. Rolls,et al.  Cerebral Cortex Advance Access published June 22, 2007 Expected Value, Reward Outcome, and Temporal Difference Error Representations in a Probabilistic Decision Task , 2022 .

[134]  S. Quartz,et al.  Human Insula Activation Reflects Risk Prediction Errors As Well As Risk , 2008, The Journal of Neuroscience.

[135]  Colin Camerer,et al.  Neuroeconomics: decision making and the brain , 2008 .

[136]  M. Platt,et al.  Risky business: the neuroeconomics of decision making under uncertainty , 2008, Nature Neuroscience.

[137]  Wolf Singer,et al.  Better than Conscious? Implications for Performance and Institutional Analysis , 2008 .

[138]  M. Andrés.  Learning and behavior: A contemporary synthesis , 2008 .

[139]  廣瀬雄一,et al.  Neuroscience , 2019, Workplace Attachments.