Neural Dissociation of Delay and Uncertainty in Intertemporal Choice

Decision makers often face choices whose consequences unfold over time. To explore the neural basis of such intertemporal choice behavior, we devised a novel two-alternative choice task with probabilistic reward delivery and contrasted two conditions that differed only in whether the outcome was revealed immediately or after some delay. In the immediate condition, we simply varied the reward probability of each option and the outcome was revealed immediately. In the delay condition, the outcome was revealed after a delay during which the reward probability was governed by a constant hazard rate. Functional imaging revealed a set of brain regions, such as the posterior cingulate cortex, parahippocampal gyri, and frontal pole, that exhibited activity uniquely associated with the temporal aspects of the task. This engagement of the so-called “default network” suggests that during intertemporal choice, decision makers simulate the impending delay via a process of prospection.

[1]  J. E. Mazur Choice, delay, probability, and conditioned reinforcement , 1997 .

[2]  George Wu,et al.  Anxiety and Decision Making with Delayed Resolution of Uncertainty , 1999 .

[3]  M. Torrens Co-Planar Stereotaxic Atlas of the Human Brain—3-Dimensional Proportional System: An Approach to Cerebral Imaging, J. Talairach, P. Tournoux. Georg Thieme Verlag, New York (1988), 122 pp., 130 figs. DM 268 , 1990 .

[4]  A. Hariri,et al.  Preference for Immediate over Delayed Rewards Is Associated with Magnitude of Ventral Striatal Activity , 2006, The Journal of Neuroscience.

[5]  K. Szpunar,et al.  Neural substrates of envisioning the future , 2007, Proceedings of the National Academy of Sciences.

[6]  Saori C. Tanaka,et al.  Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops , 2004, Nature Neuroscience.

[7]  E. Böhm-Bawerk Capital and interest , 1890 .

[8]  G. Pagnoni,et al.  Neurobiological Substrates of Dread , 2006, Science.

[9]  Samuel M. McClure,et al.  Time Discounting for Primary Rewards , 2007, The Journal of Neuroscience.

[10]  Rudolf N. Cardinal,et al.  Neural systems implicated in delayed and probabilistic reinforcement , 2006, Neural Networks.

[11]  C. Padoa-Schioppa,et al.  Neurons in the orbitofrontal cortex encode economic value , 2006, Nature.

[12]  K. Kirby,et al.  Bidding on the Future: Evidence Against Normative Discounting of Delayed Rewards , 1997 .

[13]  M. Platt,et al.  Risk-sensitive neurons in macaque posterior cingulate cortex , 2005, Nature Neuroscience.

[14]  G. Ainslie Picoeconomics: The Strategic Interaction of Successive Motivational States within the Person , 1992 .

[15]  J. E. Mazur Conditioned reinforcement and choice with delayed and uncertain primary reinforcers. , 1995, Journal of the experimental analysis of behavior.

[16]  Peter D. Sozou,et al.  On hyperbolic discounting and uncertain hazard rates , 1998, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[17]  Colin Camerer,et al.  Neural Systems Responding to Degrees of Uncertainty in Human Decision-Making , 2005, Science.

[18]  J. E. Mazur OF THE EXPERIMENTAL ANALYSIS OF BEHAVIOR THEORIES OF PROBABILISTIC REINFORCEMENT , 2005 .

[19]  Daeyeol Lee,et al.  Prefrontal Coding of Temporally Discounted Values during Intertemporal Choice , 2008, Neuron.

[20]  Hatim A. Zariwala,et al.  Neural correlates, computation and behavioural impact of decision confidence , 2008, Nature.

[21]  Joanna L. Y. Ho,et al.  Hope: An empirical study of attitude toward the timing of uncertainty resolution , 1994 .

[22]  F. E. Principles of Economics , 1890, Nature.

[23]  Samuel M. McClure,et al.  Separate Neural Systems Value Immediate and Delayed Monetary Rewards , 2004, Science.

[24]  E. Maskin,et al.  Uncertainty and Hyperbolic Discounting , 2005 .

[25]  Masatoshi Itoh,et al.  Thinking of the future and past: the roles of the frontal pole and the medial temporal lobes , 2003, NeuroImage.

[26]  L. Green,et al.  A discounting framework for choice with delayed and probabilistic rewards. , 2004, Psychological bulletin.

[27]  D. Hassabis,et al.  Patients with hippocampal amnesia cannot imagine new experiences , 2007, Proceedings of the National Academy of Sciences.

[28]  Yoram Halevy Diminishing Impatience: Disentangling Time Preference from Uncertain Lifetime , 2004 .

[29]  Jonathan D. Cohen,et al.  Conflict monitoring and anterior cingulate cortex: an update , 2004, Trends in Cognitive Sciences.

[30]  R. Herrnstein,et al.  Preference Reversals Due to Myopic Discounting of Delayed Reward , 1995 .

[31]  M. Roesch,et al.  Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards , 2007, Nature Neuroscience.

[32]  G. Loewenstein,et al.  Time and Decision: Economic and Psychological Perspectives of Intertemporal Choice , 2003 .

[33]  W. Mischel,et al.  Theory and research on the antecedents of self-imposed delay of reward. , 1966, Progress in experimental personality research.

[34]  L Green,et al.  Amount of reward has opposite effects on the discounting of delayed and probabilistic outcomes. , 1999, Journal of experimental psychology. Learning, memory, and cognition.

[35]  G. Loewenstein,et al.  Decision making over time and under uncertainty: a common approach , 1991 .

[36]  Dariush Arai Temporal resolution of uncertainty in risky choices , 1997 .

[37]  G. Ainslie Specious reward: a behavioral theory of impulsiveness and impulse control. , 1975, Psychological bulletin.

[38]  D. Cross,et al.  Subjective probability and delay. , 1991, Journal of the experimental analysis of behavior.

[39]  R. H. Strotz Myopia and Inconsistency in Dynamic Utility Maximization , 1955 .

[40]  G. Loewenstein Anticipation and the Valuation of Delayed Consumption , 1987 .

[41]  J. C. Crowley,et al.  Saccade Reward Signals in Posterior Cingulate Cortex , 2003, Neuron.

[42]  G. Madden,et al.  Exchange delays and impulsive choice in adult humans. , 1994, Journal of the experimental analysis of behavior.

[43]  G. Loewenstein,et al.  Time Discounting and Time Preference: A Critical Review , 2002 .

[44]  G. Berns,et al.  Intertemporal choice – toward an integrative framework , 2007, Trends in Cognitive Sciences.

[45]  R. Buckner,et al.  Self-projection and the brain , 2007, Trends in Cognitive Sciences.

[46]  Alana T. Wong,et al.  Remembering the past and imagining the future: Common and distinct neural substrates during event construction and elaboration , 2007, Neuropsychologia.

[47]  Saori C. Tanaka,et al.  Serotonin Differentially Regulates Short- and Long-Term Prediction of Rewards in the Ventral and Dorsal Striatum , 2007, PloS one.

[48]  H. Damasio,et al.  Damage to the Insula Disrupts Addiction to Cigarette Smoking , 2007, Science.

[49]  Martin P. Paulus,et al.  Time and decision making: differential contribution of the posterior insular cortex and the striatum during a delay discounting task , 2007, Experimental Brain Research.

[50]  T. Robbins,et al.  Impulsive Choice Induced in Rats by Lesions of the Nucleus Accumbens Core , 2001, Science.

[51]  M. Hauser,et al.  The Evolutionary Origins of Human Patience: Temporal Preferences in Chimpanzees, Bonobos, and Human Adults , 2007, Current Biology.

[52]  G. Chapman,et al.  Decision biases in intertemporal choice and choice under uncertainty: Testing a common account , 2006, Memory & cognition.

[53]  C. Pennartz,et al.  Is a bird in the hand worth two in the future? The neuroeconomics of intertemporal decision-making , 2008, Progress in Neurobiology.

[54]  P. Glimcher,et al.  The neural correlates of subjective value during intertemporal choice , 2007, Nature Neuroscience.

[55]  Daniel D. Holt,et al.  Is discounting impulsive? Evidence from temporal and probability discounting in gambling and non-gambling college students , 2003, Behavioural Processes.

[56]  Jonathan D. Cohen,et al.  The Neural Basis of Economic Decision-Making in the Ultimatum Game , 2003, Science.

[57]  M. K. Stevenson,et al.  A Discounting Model for Decisions With Delayed Positive or Negative Outcomes , 1986 .