An "as soon as possible" effect in human intertemporal decision making: behavioral evidence and neural mechanisms.

Many decisions involve a trade-off between the quality of an outcome and the time at which that outcome is received. In psychology and behavioral economics, the most widely studied models hypothesize that the values of future gains decline as a roughly hyperbolic function of delay from the present. Recently, it has been proposed that this hyperbolic-like decline in value arises from the interaction of two separate neural systems: one specialized to value immediate rewards and the other specialized to value delayed rewards. Here we report behavioral and functional magnetic resonance imaging results that are inconsistent with both the standard behavioral models of discounting and the hypothesis that separate neural systems value immediate and delayed rewards. Behaviorally, we find that human subjects do not necessarily make the impulsive preference reversals predicted by hyperbolic-like discounting. We also find that blood oxygenation level dependent activity in ventral striatum, medial prefrontal, and posterior cingulate cortex does not track whether an immediate reward was present, as proposed by the separate neural systems hypothesis. Activity in these regions was correlated with the subjective value of both immediate and delayed rewards. Rather than encoding only the relative value of one reward compared with another, these values are represented on a more absolute scale. These data support an alternative behavioral-neural model (which we call "ASAP"), in which subjective value declines hyperbolically relative to the soonest currently available reward and a small number of valuation areas serve as a final common pathway through which these subjective values guide choice.

[1]  P. Samuelson A Note on Measurement of Utility , 1937 .

[2]  R. H. Strotz Myopia and Inconsistency in Dynamic Utility Maximization , 1955 .

[3]  Herbert A. Simon,et al.  Economics and Psychology. , 1963 .

[4]  J. E. Mazur An adjusting procedure for studying delayed reinforcement. , 1987 .

[5]  P. Nelson,et al.  Another Experimental Look at Individual Time Preference , 1992 .

[6]  L. Green,et al.  Temporal discounting and preference reversals in choice between delayed outcomes , 1994, Psychonomic bulletin & review.

[7]  R. Herrnstein,et al.  Preference Reversals Due to Myopic Discounting of Delayed Reward , 1995 .

[8]  David I. Laibson,et al.  Golden Eggs and Hyperbolic Discounting , 1997 .

[9]  Martin Ahlbrecht,et al.  An empirical study on intertemporal decision making under risk , 1997 .

[10]  Colin Camerer,et al.  The Effects of Financial Incentives in Experiments: A Review and Capital-Labor-Production Framework , 1999 .

[11]  M. Coller,et al.  Eliciting Individual Discount Rates , 1999 .

[12]  Michael L. Platt,et al.  Neural correlates of decision variables in parietal cortex , 1999, Nature.

[13]  V. Smith,et al.  Monetary rewards and decision cost in experimental economics , 2000 .

[14]  Peter H. M. P. Roelofsma,et al.  Intransitive intertemporal choice , 2000 .

[15]  J Baron,et al.  Can We Use Human Judgments to Determine the Discount Rate? , 2000, Risk analysis : an official publication of the Society for Risk Analysis.

[16]  W. Bickel,et al.  Toward a behavioral economic understanding of drug dependence: delay discounting processes. , 2001, Addiction.

[17]  D. Kahneman,et al.  Functional Imaging of Neural Responses to Expectancy and Experience of Monetary Gains and Losses tasks with monetary payoffs , 2001 .

[18]  W. Newsome,et al.  Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. , 2001, Journal of neurophysiology.

[19]  D. Read Is Time-Discounting Hyperbolic or Subadditive? , 2001 .

[20]  P. Dayan,et al.  Reward, Motivation, and Reinforcement Learning , 2002, Neuron.

[21]  Charles A. Holt,et al.  Risk Aversion and Incentive Effects , 2002 .

[22]  G. Loewenstein,et al.  Time Discounting and Time Preference: A Critical Review , 2002 .

[23]  A. Rubinstein International Economic Review " Economics and Psychology " ? the Case of Hyperbolic Discounting * , 2003 .

[24]  Yusufcan Masatlioglu,et al.  A General Theory of Time Preferences , 2003 .

[25]  Peter H. M. P. Roelofsma,et al.  Subadditive versus hyperbolic discounting: A comparison of choice and matching , 2003 .

[26]  Saori C. Tanaka,et al.  Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops , 2004, Nature Neuroscience.

[27]  P. Glimcher,et al.  Activity in Posterior Parietal Cortex Is Correlated with the Relative Subjective Desirability of Action , 2004, Neuron.

[28]  W. Newsome,et al.  Matching Behavior and the Representation of Value in the Parietal Cortex , 2004, Science.

[29]  Samuel M. McClure,et al.  Neural Correlates of Behavioral Preference for Culturally Familiar Drinks , 2004, Neuron.

[30]  Samuel M. McClure,et al.  Separate Neural Systems Value Immediate and Delayed Monetary Rewards , 2004, Science.

[31]  L. Green,et al.  A discounting framework for choice with delayed and probabilistic rewards. , 2004, Psychological bulletin.

[32]  M. Platt,et al.  Risk-sensitive neurons in macaque posterior cingulate cortex , 2005, Nature Neuroscience.

[33]  L. Green,et al.  Temporal discounting when the choice is between two delayed rewards. , 2005, Journal of experimental psychology. Learning, memory, and cognition.

[34]  K. Doya,et al.  Representation of Action-Specific Reward Values in the Striatum , 2005, Science.

[35]  P. Dayan,et al.  Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control , 2005, Nature Neuroscience.

[36]  Dirk J. Heslenfeld,et al.  Activity in human reward-sensitive brain areas is strongly context dependent , 2005, NeuroImage.

[37]  C. Padoa-Schioppa,et al.  Neurons in the orbitofrontal cortex encode economic value , 2006, Nature.

[38]  P. Dayan,et al.  Cortical substrates for exploratory decisions in humans , 2006, Nature.

[39]  J. O'Doherty,et al.  The Role of the Ventromedial Prefrontal Cortex in Abstract State-Based Inference during Decision Making in Humans , 2006, The Journal of Neuroscience.

[40]  Marco Casari,et al.  Pre-commitment and flexibility in a time decision experiment , 2006 .

[41]  Daniel Read,et al.  Discounting by Intervals: A Generalized Model of Intertemporal Choice , 2006, Manag. Sci..

[42]  Samuel M. McClure,et al.  Time Discounting for Primary Rewards , 2007, The Journal of Neuroscience.

[43]  J. O'Doherty,et al.  Orbitofrontal Cortex Encodes Willingness to Pay in Everyday Economic Transactions , 2007, The Journal of Neuroscience.

[44]  Sabrina M. Tom,et al.  The Neural Basis of Loss Aversion in Decision-Making Under Risk , 2007, Science.

[45]  P. Glimcher,et al.  The neural correlates of subjective value during intertemporal choice , 2007, Nature Neuroscience.

[46]  Colin Camerer,et al.  A framework for studying the neurobiology of value-based decision making , 2008, Nature Reviews Neuroscience.

[47]  C. Padoa-Schioppa,et al.  The representation of economic value in the orbitofrontal cortex is invariant for changes of menu , 2008, Nature Neuroscience.

[48]  Peter Bossaerts,et al.  Neural correlates of mentalizing-related computations during strategic interactions in humans , 2008, Proceedings of the National Academy of Sciences.

[49]  Colin Camerer,et al.  Dissociating the Role of the Orbitofrontal Cortex and the Striatum in the Computation of Goal Values and Prediction Errors , 2008, The Journal of Neuroscience.

[50]  P. Glimcher,et al.  Value Representations in the Primate Striatum during Matching Behavior , 2008, Neuron.

[51]  M. Husain,et al.  Control over Conflict during Movement Preparation: Role of Posterior Parietal Cortex , 2008, Neuron.

[52]  P. Glimcher Choice: Towards a Standard Back-pocket Model , 2009 .

[53]  Brian Knutson,et al.  Dissociable neural representations of future reward magnitude and delay during temporal discounting , 2009, NeuroImage.

[54]  Colin Camerer,et al.  Self-control in decision-making involves modulation of the vmPFC valuation system , 2009, NeuroImage.

[55]  Karl J. Friston,et al.  Encoding of Marginal Utility across Time in the Human Brain , 2009, The Journal of Neuroscience.