A Computational Analysis of Aberrant Delay Discounting in Psychiatric Disorders

Impatience for reward is a facet of many psychiatric disorders. We draw attention to a growing literature finding greater discounting of delayed reward, an important aspect of impatience, across a range of psychiatric disorders. We propose these findings are best understood by considering the goals and motivation for discounting future reward. We characterize these as arising from either the opportunity costs of waiting or the uncertainty associated with delayed reward. We link specific instances of higher discounting in psychiatric disorder to heightened subjective estimates of either of these factors. We propose these costs are learned and represented based either on a flexible cognitive model of the world, an accumulation of previous experience, or through evolutionary specification. Any of these can be considered suboptimal for the individual if the resulting behavior results in impairments in personal and social functioning and/or in distress. By considering the neurochemical and neuroanatomical implementation of these processes, we illustrate how this approach can in principle unite social, psychological and biological conceptions of impulsive choice.

[1]  E. Thorndike The law of effect. , 1927 .

[2]  P. Samuelson A Note on Measurement of Utility , 1937 .

[3]  R. H. Strotz Myopia and Inconsistency in Dynamic Utility Maximization , 1955 .

[4]  George C. Williams,et al.  PLEIOTROPY, NATURAL SELECTION, AND THE EVOLUTION OF SENESCENCE , 1957, Science of Aging Knowledge Environment.

[5]  Z. A. Trapeznikova On the Interaction of , 1959 .

[6]  A. Alchian,et al.  Effects of Inflation , 1962, Journal of Political Economy.

[7]  M. Glatt Heroin Addicts , 1964, British medical journal.

[8]  R. Rescorla,et al.  Two-process learning theory: Relationships between Pavlovian conditioning and instrumental learning. , 1967, Psychological review.

[9]  E. Fischer Conditioned Reflexes , 1942, American journal of physical medicine.

[10]  R. Herrnstein,et al.  Choice and delay of reinforcement. , 1967, Journal of the experimental analysis of behavior.

[11]  D. R. Williams,et al.  Auto-maintenance in the pigeon: sustained pecking despite contingent non-reinforcement. , 1969, Journal of the experimental analysis of behavior.

[12]  R. Herrnstein On the law of effect. , 1970, Journal of the experimental analysis of behavior.

[13]  C. B. Ferster A functional analysis of depression. , 1973 .

[14]  G. Ainslie,et al.  Impulse control in pigeons. , 1974, Journal of the experimental analysis of behavior.

[15]  E. Paykel,et al.  Contribution of life events to causation of psychiatric illness , 1978, Psychological Medicine.

[16]  A. Tversky,et al.  Prospect theory: analysis of decision under risk , 1979 .

[17]  R. Thaler,et al.  An Economic Theory of Self-Control , 1977, Journal of Political Economy.

[18]  R. Thaler,et al.  An Economic Theory of Self-Control , 1977, Journal of Political Economy.

[19]  R. Thaler Some empirical evidence on dynamic inconsistency , 1981 .

[20]  J. Christensen-Szalanski Discount Functions and the Measurement of Patients' Values , 1984, Medical decision making : an international journal of the Society for Medical Decision Making.

[21]  M. Rapport,et al.  Hyperactivity and frustration: The influence of control over and size of rewards in delaying gratification , 1986, Journal of abnormal child psychology.

[22]  W. Hershberger An approach through the looking-glass , 1986 .

[23]  J. Leigh Accounting for tastes: correlates of risk and time preferences , 1986 .

[24]  J. E. Mazur An adjusting procedure for studying delayed reinforcement. , 1987 .

[25]  J. C. Perry,et al.  Childhood trauma in borderline personality disorder. , 1989, The American journal of psychiatry.

[26]  W. Mischel,et al.  Delay of gratification in children. , 1989, Science.

[27]  D. Westen,et al.  Childhood sexual and physical abuse in adult patients with borderline personality disorder. , 1990, The American journal of psychiatry.

[28]  E. Sonuga-Barke,et al.  Hyperactivity and delay aversion--I. The effect of delay on choice. , 1992, Journal of child psychology and psychiatry, and allied disciplines.

[29]  T. Swirsky-Sacchetti,et al.  Neuropsychological function in borderline personality disorder. , 1993, Journal of clinical psychology.

[30]  B. Balleine,et al.  Motivational control of goal-directed action , 1994 .

[31]  L. Green,et al.  Temporal discounting and preference reversals in choice between delayed outcomes , 1994, Psychonomic bulletin & review.

[32]  L. Green,et al.  Discounting of delayed rewards: Models of individual choice. , 1995, Journal of the experimental analysis of behavior.

[33]  R. Boakes,et al.  Motivational control after extended instrumental training , 1995 .

[34]  R. Herrnstein,et al.  Preference Reversals Due to Myopic Discounting of Delayed Reward , 1995 .

[35]  K. Kirby,et al.  Modeling Myopic Decisions: Evidence for Hyperbolic Delay-Discounting within Subjects and Amounts , 1995 .

[36]  J. Schweitzer,et al.  Self-control in boys with attention deficit hyperactivity disorder: effects of added stimulation and time. , 1995, Journal of child psychology and psychiatry, and allied disciplines.

[37]  G. Loewenstein Out of control: Visceral influences on behavior , 1996 .

[38]  L Green,et al.  Temporal discounting in choice between delayed rewards: the role of age and income. , 1996, Psychology and aging.

[39]  R. Reich,et al.  Reported pathological childhood experiences associated with the development of borderline personality disorder. , 1997, The American journal of psychiatry.

[40]  David I. Laibson,et al.  Golden Eggs and Hyperbolic Discounting , 1997 .

[41]  W K Bickel,et al.  Impulsive and self-control choices in opioid-dependent patients and non-drug-using control participants: drug and monetary rewards. , 1997, Experimental and clinical psychopharmacology.

[42]  Y. Benyamini,et al.  Self-rated health and mortality: a review of twenty-seven community studies. , 1997, Journal of health and social behavior.

[43]  Effects of Deprivation Level on Humans' Self-Control for Food Reinforcers , 1997, Appetite.

[44]  Wendy Wood,et al.  Habit and intention in everyday life: The multiple processes by which past behavior predicts future behavior. , 1998 .

[45]  Anna Rose Childress,et al.  Conditioning factors in drug abuse: can they explain compulsion? , 1998, Journal of psychopharmacology.

[46]  Peter D. Sozou,et al.  On hyperbolic discounting and uncertain hazard rates , 1998, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[47]  B. Balleine,et al.  Goal-directed instrumental action: contingency and incentive learning and their cortical substrates , 1998, Neuropharmacology.

[48]  Joel Myerson,et al.  Effects of inflation on the subjective value of delayed and probabilistic rewards , 1998 .

[49]  S. Weich,et al.  Poverty, unemployment, and common mental disorders: population based cohort study , 1998, BMJ.

[50]  M. Domjan The principles of learning and behavior, 4th ed. , 1998 .

[51]  D. Read,et al.  Predicting Hunger: The Effects of Appetite and Delay on Choice. , 1998, Organizational behavior and human decision processes.

[52]  W. Bickel,et al.  Heroin addicts have higher discount rates for delayed rewards than non-drug-using controls. , 1999, Journal of experimental psychology. General.

[53]  H. Garretsen,et al.  Alcohol consumption, alcohol-related problems, problem drinking, and socioeconomic status. , 1999, Alcohol and alcoholism.

[54]  J. Bjork,et al.  Laboratory measures of aggression and impulsivity in women with borderline personality disorder , 1999, Psychiatry Research.

[55]  L. Green,et al.  Discounting of delayed rewards across the life span: age differences in individual discounting functions , 1999, Behavioural Processes.

[56]  M Cortsen,et al.  Progressive atrophy of the frontal lobes in first‐episode schizophrenia: interaction with clinical course and neuroleptic treatment , 1999, Acta psychiatrica Scandinavica.

[57]  T. Robbins,et al.  Emotional bias and inhibitory control processes in mania and depression , 1999, Psychological Medicine.

[58]  A. Odum,et al.  Impulsivity and cigarette smoking: delay discounting in current, never, and ex-smokers , 1999, Psychopharmacology.

[59]  J. Metcalfe,et al.  A hot/cool-system analysis of delay of gratification: dynamics of willpower. , 1999, Psychological review.

[60]  N. Petry,et al.  Excessive discounting of delayed rewards in substance abusers with gambling problems. , 1999, Drug and alcohol dependence.

[61]  V. Arango,et al.  A serotonin transporter gene promoter polymorphism (5-HTTLPR) and prefrontal cortical binding in major depression and suicide. , 2000, Archives of general psychiatry.

[62]  A. Odum,et al.  Needle sharing in opioid-dependent outpatients: psychological processes underlying risk. , 2000, Drug and alcohol dependence.

[63]  David S. Touretzky,et al.  Behavioral considerations suggest an average reward TD model of the dopamine system , 2000, Neurocomputing.

[64]  J E Steinmetz,et al.  Decision-making biases, antisocial personality, and early-onset alcoholism. , 2000, Alcoholism, clinical and experimental research.

[65]  R. Barkley,et al.  Executive Functioning, Temporal Discounting, and Sense of Time in Adolescents with Attention Deficit Hyperactivity Disorder (ADHD) and Oppositional Defiant Disorder (ODD) , 2001, Journal of abnormal child psychology.

[66]  E. Barratt,et al.  Psychiatric aspects of impulsivity. , 2001, The American journal of psychiatry.

[67]  N. Petry,et al.  Delay discounting of money and alcohol in actively using alcoholics, currently abstinent alcoholics, and controls , 2001, Psychopharmacology.

[68]  T S Critchfield,et al.  Temporal discounting: basic research and the analysis of socially important behavior. , 2001, Journal of applied behavior analysis.

[69]  D. Read Is Time-Discounting Hyperbolic or Subadditive? , 2001 .

[70]  B. Alsop,et al.  Sensitivity to reward delay in children with attention deficit hyperactivity disorder (ADHD). , 2001, Journal of child psychology and psychiatry, and allied disciplines.

[71]  Patricia Desmond,et al.  Selective bilateral hippocampal volume loss in chronic schizophrenia , 2001, Biological Psychiatry.

[72]  L. Green,et al.  Area under the curve as a measure of discounting. , 2001, Journal of the experimental analysis of behavior.

[73]  J. Oosterlaan,et al.  Psychological mechanisms in hyperactivity: I. Response inhibition deficit, working memory impairment, delay aversion, or something else? , 2001, Journal of child psychology and psychiatry, and allied disciplines.

[74]  Werner Güth,et al.  On the Interaction of Risk and Time Preferences: An Experimental Study , 2001 .

[75]  Peter Dayan,et al.  Expected and Unexpected Uncertainty: ACh and NE in the Neocortex , 2002, NIPS.

[76]  John Cairns,et al.  A comparison of the discounted utility model and hyperbolic discounting models in the case of social and private intertemporal preferences for health , 2002 .

[77]  P. Dayan,et al.  Reward, Motivation, and Reinforcement Learning , 2002, Neuron.

[78]  Charles A. Holt,et al.  Risk Aversion and Incentive Effects , 2002 .

[79]  N. Petry Discounting of delayed rewards in substance abusers: relationship to antisocial personality disorder , 2002, Psychopharmacology.

[80]  Amy L Odum,et al.  Discounting of delayed health gains and losses by current, never- and ex-smokers of cigarettes. , 2002, Nicotine & tobacco research : official journal of the Society for Research on Nicotine and Tobacco.

[81]  George Loewenstein,et al.  Mild opioid deprivation increases the degree that opioid-dependent outpatients discount delayed heroin and money , 2002, Psychopharmacology.

[82]  E. Barratt,et al.  Increased impulsivity in cocaine dependent subjects independent of antisocial personality disorder and aggression. , 2002, Drug and alcohol dependence.

[83]  G. Loewenstein,et al.  Time Discounting and Time Preference: A Critical Review , 2002 .

[84]  A. Rubinstein International Economic Review " Economics and Psychology " ? the Case of Hyperbolic Discounting * , 2003 .

[85]  Karl J. Friston,et al.  Temporal Difference Models and Reward-Related Learning in the Human Brain , 2003, Neuron.

[86]  C. Curtis,et al.  Persistent activity in the prefrontal cortex during working memory , 2003, Trends in Cognitive Sciences.

[87]  S. Kollins,et al.  Delay discounting is associated with substance use in college students. , 2003, Addictive behaviors.

[88]  L. Epstein,et al.  Comparison between two measures of delay discounting in smokers. , 2003, Experimental and clinical psychopharmacology.

[89]  N. Petry Discounting of money, health, and freedom in substance abusers and controls. , 2003, Drug and alcohol dependence.

[90]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[91]  J. Richards,et al.  Delay discounting and probability discounting as related to cigarette smoking status in adults , 2004, Behavioural Processes.

[92]  Yuval Levin Imagining the future. , 2004, New Atlantis.

[93]  Samuel M. McClure,et al.  Separate Neural Systems Value Immediate and Delayed Monetary Rewards , 2004, Science.

[94]  L. Green,et al.  A discounting framework for choice with delayed and probabilistic rewards. , 2004, Psychological bulletin.

[95]  Peter Dayan,et al.  Technical Note: Q-Learning , 2004, Machine Learning.

[96]  M. Walton,et al.  Differential involvement of serotonin and dopamine systems in cost-benefit decisions about delay or effort , 2005, Psychopharmacology.

[97]  Kris N Kirby,et al.  Heroin and cocaine abusers have higher discount rates for delayed rewards than alcoholics or non-drug-using controls. , 2004, Addiction.

[98]  P. Fonagy,et al.  Mentalization-Based Treatment for BPD , 2004, Journal of personality disorders.

[99]  Taiki Takahashi Cortisol levels and time-discounting of monetary gain in humans , 2004, Neuroreport.

[100]  Catherine C. Eckel,et al.  SAVING DECISIONS OF THE WORKING POOR: SHORT- AND LONG-TERM HORIZONS , 2005 .

[101]  Jaap Oosterlaan,et al.  The impact of reinforcement contingencies on AD/HD: a review and theoretical appraisal. , 2005, Clinical psychology review.

[102]  Morten I. Lau,et al.  Eliciting Risk and Time Preferences , 2008 .

[103]  R. Poldrack,et al.  Prospect theory on the brain? Toward a cognitive neuroscience of decision under risk. , 2005, Brain research. Cognitive brain research.

[104]  Karl J. Friston,et al.  Opponent appetitive-aversive neural processes underlie predictive learning of pain relief , 2005, Nature Neuroscience.

[105]  Angela J. Yu,et al.  Uncertainty, Neuromodulation, and Attention , 2005, Neuron.

[106]  P. Dayan,et al.  Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control , 2005, Nature Neuroscience.

[107]  Jerome R. Busemeyer,et al.  Using Cognitive Models to Map Relations Between Neuropsychological Disorders and Human Decision-Making Deficits , 2005, Psychological science.

[108]  Trevor W Robbins,et al.  Interactions between Serotonin and Dopamine in the Control of Impulsive Choice in Rats: Therapeutic Implications for Impulse Control Disorders , 2005, Neuropsychopharmacology.

[109]  安藤 広志,et al.  20世紀の名著名論:David Marr:Vision:a Computational Investigation into the Human Representation and Processing of Visual Information , 2005 .

[110]  G. Chapman Short-term cost for long-term benefit: time preference and cancer control. , 2005, Health psychology : official journal of the Division of Health Psychology, American Psychological Association.

[111]  Wouter Hulstijn,et al.  Behavioural aspects of impulsivity in alcoholics with and without a cluster-B personality disorder. , 2006, Alcohol and alcoholism.

[112]  G. Pagnoni,et al.  Neurobiological Substrates of Dread , 2006, Science.

[113]  W. van den Brink,et al.  Decision-making deficits in alcohol-dependent patients with and without comorbid personality disorder. , 2006, Alcoholism, clinical and experimental research.

[114]  Warren K Bickel,et al.  Discounting of past outcomes. , 2006, Experimental and clinical psychopharmacology.

[115]  H. Rachlin Notes on discounting. , 2006, Journal of the experimental analysis of behavior.

[116]  Sushil Jajodia,et al.  On the anonymity and traceability of peer-to-peer VoIP calls , 2006, IEEE Network.

[117]  Peter Dayan,et al.  Non-commercial Research and Educational Use including without Limitation Use in Instruction at Your Institution, Sending It to Specific Colleagues That You Know, and Providing a Copy to Your Institution's Administrator. All Other Uses, Reproduction and Distribution, including without Limitation Comm , 2022 .

[118]  David T. Neal,et al.  Habits—A Repeat Performance , 2006 .

[119]  J. Stevens Intertemporal Choice , 2013 .

[120]  W. Schultz Behavioral theories and the neurophysiology of reward. , 2006, Annual review of psychology.

[121]  P. Dayan,et al.  Tonic dopamine: opportunity costs and the control of response vigor , 2007, Psychopharmacology.

[122]  A. Tversky,et al.  Prospect theory: an analysis of decision under risk — Source link , 2007 .

[123]  S. Lupien,et al.  The effects of stress and stress hormones on human cognition: Implications for the field of brain and cognition , 2007, Brain and Cognition.

[124]  Samuel M. McClure,et al.  Time Discounting for Primary Rewards , 2007, The Journal of Neuroscience.

[125]  Richard Gray,et al.  Serious mental illness and physical health problems: a discussion paper. , 2007, International journal of nursing studies.

[126]  Toshikazu Hasegawa,et al.  A hyperbolic decay of subjective probability of obtaining delayed rewards , 2007, Behavioral and Brain Functions.

[127]  Benjamin M. Robinson,et al.  Delay discounting in schizophrenia , 2007, Cognitive neuropsychiatry.

[128]  Brady Reynolds,et al.  Adolescent smokers rate delayed rewards as less certain than adolescent nonsmokers. , 2007, Drug and alcohol dependence.

[129]  D. Hassabis,et al.  Patients with hippocampal amnesia cannot imagine new experiences , 2007, Proceedings of the National Academy of Sciences.

[130]  C. Croux,et al.  Depression and socio-economic risk factors: 7-year longitudinal population study , 2007, British Journal of Psychiatry.

[131]  A. Falk,et al.  Are Risk Aversion and Impatience Related to Cognitive Ability? , 2007, SSRN Electronic Journal.

[132]  Matthijs A. A. van der Meer,et al.  Integrating hippocampus and striatum in decision-making , 2007, Current Opinion in Neurobiology.

[133]  Timothy E. J. Behrens,et al.  Learning the value of information in an uncertain world , 2007, Nature Neuroscience.

[134]  W. Bickel,et al.  Behavioral and neuroeconomics of drug addiction: competing neural systems and temporal discounting processes. , 2007, Drug and alcohol dependence.

[135]  Saori C. Tanaka,et al.  Serotonin Differentially Regulates Short- and Long-Term Prediction of Rewards in the Ventral and Dorsal Striatum , 2007, PloS one.

[136]  J. Beck,et al.  Is the delay discounting paradigm useful in understanding social anxiety? , 2007, Behaviour research and therapy.

[137]  Bruce D. McCandliss,et al.  Socioeconomic gradients predict individual differences in neurocognitive abilities. , 2007, Developmental science.

[138]  Paul Christiansen,et al.  Delay discounting and the alcohol Stroop in heavy drinking adolescents. , 2007, Addiction.

[139]  Daniel Read,et al.  Chapter 21. Intertemporal Choice , 2008 .

[140]  G. Loewenstein,et al.  Conflicting motives in evaluations of sequences , 2008 .

[141]  L. Warlop,et al.  Bikinis Instigate Generalized Impatience in Intertemporal Choice , 2008 .

[142]  P. Dayan,et al.  A temporal difference account of avoidance learning , 2008, Network.

[143]  Warren K Bickel,et al.  Cigarette smokers discount past and future rewards symmetrically and more than controls: is discounting a measure of impulsivity? , 2008, Drug and alcohol dependence.

[144]  N. Adler,et al.  U.S. disparities in health: descriptions, causes, and mechanisms. , 2008, Annual review of public health.

[145]  Noah A. Shamosh,et al.  Individual Differences in Delay Discounting , 2008, Psychological science.

[146]  I. Rossow Alcohol consumption and discounting , 2008 .

[147]  P. Dayan,et al.  Decision theory, reinforcement learning, and the brain , 2008, Cognitive, affective & behavioral neuroscience.

[148]  D. Schacter,et al.  Episodic Simulation of Future Events , 2008, Annals of the New York Academy of Sciences.

[149]  Hilleke E. Hulshoff Pol,et al.  Progressive Brain Volume Loss in Schizophrenia Over the Course of the Illness: Evidence of Maturational Abnormalities in Early Adulthood , 2008, Biological Psychiatry.

[150]  Saori C. Tanaka,et al.  Low-Serotonin Levels Increase Delayed Reward Discounting in Humans , 2008, The Journal of Neuroscience.

[151]  R. Dolan,et al.  Emotion, Decision Making, and the Amygdala , 2008, Neuron.

[152]  S. Floresco,et al.  Dopaminergic and Glutamatergic Regulation of Effort- and Delay-Based Decision Making , 2008, Neuropsychopharmacology.

[153]  M. Platt,et al.  Risky business: the neuroeconomics of decision making under uncertainty , 2008, Nature Neuroscience.

[154]  Lisa Farmer,et al.  Family unpredictability, future discounting, and risk taking , 2008 .

[155]  Taiki Takahashi,et al.  Depressive patients are more impulsive and inconsistent in intertemporal choice behavior for monetary gain and loss than healthy subjects--an analysis based on Tsallis' statistics. , 2008, Neuro endocrinology letters.

[156]  B. Balleine,et al.  A specific role for posterior dorsolateral striatum in human habit learning , 2009, The European journal of neuroscience.

[157]  James MacKillop,et al.  Delayed reward discounting predicts treatment response for heavy drinkers receiving smoking cessation treatment. , 2009, Drug and alcohol dependence.

[158]  Howard Rachlin,et al.  Delay, probability, and social discounting in a public goods game. , 2009, Journal of the experimental analysis of behavior.

[159]  M. Walton,et al.  Impulsive choice in hippocampal but not orbitofrontal cortex-lesioned rats on a nonspatial decision-making maze task , 2009, The European journal of neuroscience.

[160]  A. Swann,et al.  Impulsivity in mania , 2009, Current psychiatry reports.

[161]  Laurence Steinberg,et al.  Age differences in future orientation and delay discounting. , 2009, Child development.

[162]  Zeb Kurth-Nelson,et al.  Temporal-Difference Reinforcement Learning with Distributed Representations , 2009, PloS one.

[163]  Caro Lucas,et al.  A Neurocomputational Model for Cocaine Addiction , 2009, Neural Computation.

[164]  B. Reynolds,et al.  Delay discounting as a mediator of the relationship between perceived stress and cigarette smoking status in adolescents , 2009, Behavioural pharmacology.

[165]  P. Dayan Prospective and retrospective temporal difference learning , 2009, Network.

[166]  Philip Asherson,et al.  Are ADHD symptoms associated with delay aversion or choice impulsivity? A general population study. , 2009, Journal of the American Academy of Child and Adolescent Psychiatry.

[167]  E. Sonuga-Barke,et al.  Delay Aversion in Attention Deficit/Hyperactivity Disorder: An empirical investigation of the broader phenotype , 2009, Neuropsychologia.

[168]  Karl J. Friston,et al.  Encoding of Marginal Utility across Time in the Human Brain , 2009, The Journal of Neuroscience.

[169]  J. M. Olson,et al.  Foregoing the labor for the fruits: The effect of just world threat on the desire for immediate monetary rewards. , 2009 .

[170]  B. Reynolds,et al.  Dimensions of impulsive behavior in adolescent smokers and nonsmokers. , 2009, Experimental and clinical psychopharmacology.

[171]  M. Pauly,et al.  Time preference and its relationship with age, health, and survival probability. , 2009, Judgment and decision making.

[172]  G. Evans,et al.  Childhood poverty, chronic stress, and adult working memory , 2009, Proceedings of the National Academy of Sciences.

[173]  N. Chater,et al.  Associations between a one-shot delay discounting measure and age, income, education and real-world impulsive behavior , 2009 .

[174]  What We Can and Cannot Conclude About the Relationship Between Steep Temporal Reward Discounting and Hyperactivity-Impulsivity Symptoms in Attention-Deficit/Hyperactivity Disorder , 2010, Biological Psychiatry.

[175]  Mitul A Mehta,et al.  DAT1 and COMT Effects on Delay Discounting and Trait Impulsivity in Male Adolescents with Attention Deficit/Hyperactivity Disorder and Healthy Controls , 2010, Neuropsychopharmacology.

[176]  R. M. Tobin,et al.  Delay of Gratification , 2010 .

[177]  P. Dayan,et al.  States versus Rewards: Dissociable Neural Prediction Error Signals Underlying Model-Based and Model-Free Reinforcement Learning , 2010, Neuron.

[178]  Joshua I. Gold,et al.  Bayesian Online Learning of the Hazard Rate in Change-Point Problems , 2010, Neural Computation.

[179]  P. Glimcher,et al.  An "as soon as possible" effect in human intertemporal decision making: behavioral evidence and neural mechanisms. , 2010, Journal of neurophysiology.

[180]  R. Dolan,et al.  Dopamine, Time, and Impulsivity in Humans , 2010, The Journal of Neuroscience.

[181]  Marie Johnston,et al.  Unintentional eating. What determines goal-incongruent chocolate consumption? , 2010, Appetite.

[182]  Are Steeper Discounting Rates in Attention-Deficit/Hyperactivity Disorder Specifically Associated with Hyperactivity-Impulsivity Symptoms or Is This a Statistical Artifact? , 2010, Biological Psychiatry.

[183]  S. Lawyer,et al.  Probability and delay discounting of hypothetical sexual outcomes , 2010, Behavioural Processes.

[184]  Jan Peters,et al.  Episodic Future Thinking Reduces Reward Delay Discounting through an Enhancement of Prefrontal-Mediotemporal Interactions , 2010, Neuron.

[185]  A. Scheres,et al.  Temporal Reward Discounting in Attention-Deficit/Hyperactivity Disorder: The Contribution of Symptom Domains, Reward Magnitude, and Session Length , 2010, Biological Psychiatry.

[186]  T. Yamagishi,et al.  Stress hormones predict hyperbolic time-discount rates six months later in adults. , 2011, Neuro endocrinology letters.

[187]  A. Chanen,et al.  Impulsivity in borderline personality disorder: reward-based decision-making and its relationship to emotional distress. , 2010, Journal of personality disorders.

[188]  Lori L. Altshuler,et al.  fMRI abnormalities in dorsolateral prefrontal cortex during a working memory task in manic, euthymic and depressed bipolar subjects , 2010, Psychiatry Research: Neuroimaging.

[189]  Robert D. Dvorak,et al.  Sweet Future , 2010, Psychological science.

[190]  S. Moore,et al.  Delay discounting predicts increase in blood alcohol level in social drinkers , 2010, Psychiatry Research.

[191]  J. MacKillop,et al.  Cigarette demand and delayed reward discounting in nicotine-dependent individuals with schizophrenia and controls: an initial study , 2011, Psychopharmacology.

[192]  Zeb Kurth-Nelson,et al.  Single- and cross-commodity discounting among cocaine addicts: the commodity and its temporal location determine discounting rate , 2011, Psychopharmacology.

[193]  Paul F. Hill,et al.  Remember the Future: Working Memory Training Decreases Delay Discounting Among Stimulant Addicts , 2011, Biological Psychiatry.

[194]  L. Amass,et al.  Delay-discounting among homeless, out-of-treatment, substance-dependent men who have sex with men , 2011, The American journal of drug and alcohol abuse.

[195]  E. Walker,et al.  Diagnostic and Statistical Manual of Mental Disorders , 2013 .

[196]  J. Nigg,et al.  Delay discounting of reward in ADHD: application in young children. , 2011, Journal of child psychology and psychiatry, and allied disciplines.

[197]  James M Gold,et al.  Imagining the future: degraded representations of future rewards and events in schizophrenia. , 2011, Journal of abnormal psychology.

[198]  M. Rieger,et al.  How Time Preferences Differ: Evidence from 53 Countries , 2011, Cultural Finance.

[199]  Karl J. Friston,et al.  A Bayesian Foundation for Individual Learning Under Uncertainty , 2011, Front. Hum. Neurosci..

[200]  B. Sahakian,et al.  Lethal Forethought: Delayed Reward Discounting Differentiates High- and Low-Lethality Suicide Attempts in Old Age , 2011, Biological Psychiatry.

[201]  T. Shavit,et al.  The effect of military service on soldiers’ time preferences — Evidence from Israel , 2011, Judgment and Decision Making.

[202]  L. Hawk,et al.  Impulsivity and risk-taking in borderline personality disorder with and without substance use disorders. , 2011, Personality disorders.

[203]  Michael Moutoussis,et al.  Are computational models of any use to psychiatry? , 2011, Neural Networks.

[204]  A. Odum Delay discounting: Trait variable? , 2011, Behavioural Processes.

[205]  Eduardo F. Morales,et al.  An Introduction to Reinforcement Learning , 2011 .

[206]  P. Dayan,et al.  Model-based influences on humans’ choices and striatal prediction errors , 2011, Neuron.

[207]  A. Young Prospect Theory: An Analysis of Decision Under Risk (Kahneman and Tversky, 1979) , 2011 .

[208]  Y. Niv,et al.  Ventral Striatum and Orbitofrontal Cortex Are Both Required for Model-Based, But Not Model-Free, Reinforcement Learning , 2011, The Journal of Neuroscience.

[209]  Hayden O. Kepley,et al.  Internal Consistency and Discriminant Validity of a Delay-Discounting Task With an Adult Self-Reported ADHD Sample , 2011, Journal of attention disorders.

[210]  C F Reynolds,et al.  The temptation of suicide: striatal gray matter, discounting of delayed rewards, and suicide attempts in late-life depression , 2011, Psychological Medicine.

[211]  Todd L McKerchar,et al.  Delay discounting is associated with treatment response among cocaine-dependent outpatients. , 2011, Experimental and clinical psychopharmacology.

[212]  James G. Murphy,et al.  Associations between depression, distress tolerance, delay discounting, and alcohol-related problems in European American and African American college students. , 2011, Psychology of addictive behaviors : journal of the Society of Psychologists in Addictive Behaviors.

[213]  James W. Diller,et al.  Temporal discounting and heart rate reactivity to stress , 2011, Behavioural Processes.

[214]  Boris S. Gutkin,et al.  A Reinforcement Learning Theory for Homeostatic Regulation , 2011, NIPS.

[215]  J. M. Olson,et al.  Personal relative deprivation, delay discounting, and gambling. , 2011, Journal of personality and social psychology.

[216]  Daniel J Fridberg,et al.  Temporal discounting of rewards in patients with bipolar disorder and schizophrenia. , 2011, Journal of abnormal psychology.

[217]  Peter Dayan,et al.  Bonsai Trees in Your Head: How the Pavlovian System Sculpts Goal-Directed Choices by Pruning Decision Trees , 2012, PLoS Comput. Biol..

[218]  Stephan Meier,et al.  Time Discounting Predicts Creditworthiness , 2012, Psychological science.

[219]  Mikhail N. Koffarnus,et al.  Excessive discounting of delayed reinforcers as a trans-disease process contributing to addiction and other disease-related vulnerabilities: emerging evidence. , 2012, Pharmacology & therapeutics.

[220]  Karolina M. Lempert,et al.  Individual Differences in Delay Discounting Under Acute Stress: The Role of Trait Perceived Stress , 2012, Front. Psychology.

[221]  Daniel E. Geer,et al.  Risk Aversion , 2012, IEEE Secur. Priv..

[222]  R. Bentall,et al.  I Want It Now! Neural Correlates of Hypersensitivity to Immediate Reward in Hypomania , 2012, Biological Psychiatry.

[223]  Karl J. Friston,et al.  Computational psychiatry , 2012, Trends in Cognitive Sciences.

[224]  Sherecce Fields,et al.  Delay discounting by adolescents experimenting with cigarette smoking. , 2012, Addiction.

[225]  P. Dayan Instrumental vigour in punishment and reward , 2012, The European journal of neuroscience.

[226]  Is Deprivation-Related ADHD Different from ADHD Among Children Without Histories of Deprivation? , 2012 .

[227]  Alan J Budney,et al.  Delay discounting predicts adolescent substance abuse treatment outcome. , 2012, Experimental and clinical psychopharmacology.

[228]  Herbert Roeyers,et al.  Temporal discounting of monetary rewards in children and adolescents with ADHD and autism spectrum disorders. , 2012, Developmental science.

[229]  S. Frederick,et al.  Four days later in Cincinnati: longitudinal tests of hyperbolic discounting. , 2012, Acta psychologica.

[230]  R. Dolan,et al.  Dopamine Enhances Model-Based over Model-Free Choice Behavior , 2012, Neuron.

[231]  S. McClure,et al.  Are executive function and impulsivity antipodes? A conceptual reconstruction with special reference to addiction , 2012, Psychopharmacology.

[232]  T. Robbins,et al.  Serotonin Modulates the Effects of Pavlovian Aversive Predictions on Response Vigor , 2012, Neuropsychopharmacology.

[233]  Peter Dayan,et al.  Models of value and choice. , 2012 .

[234]  Robert C. Wilson,et al.  Rational regulation of learning dynamics by pupil–linked arousal systems , 2012, Nature Neuroscience.

[235]  T. George,et al.  Effects of cigarette smoking status on delay discounting in schizophrenia and healthy controls. , 2012, Addictive behaviors.

[236]  Shu-Chen Li,et al.  Of goals and habits: age-related and individual differences in goal-directed decision-making , 2013, Front. Neurosci..

[237]  K. C. Yamada,et al.  The biological effects of acute psychosocial stress on delay discounting , 2013, Psychoneuroendocrinology.

[238]  V. Walsh,et al.  Bi-frontal direct current stimulation affects delay discounting choices , 2013, Cognitive neuroscience.

[239]  Raymond J. Dolan,et al.  The anatomy of choice: active inference and agency , 2013, Front. Hum. Neurosci..

[240]  G207 Is There a Link Between ADHD and Social Deprivation? , 2013, Archives of Disease in Childhood.

[241]  Raymond J. Dolan,et al.  Dread and the Disvalue of Future Pain , 2013, PLoS Comput. Biol..

[242]  R. Elliott,et al.  Temporal discounting in major depressive disorder , 2013, Psychological Medicine.

[243]  W. Bickel,et al.  Alcohol-dependent individuals discount sex at higher rates than controls. , 2013, Drug and alcohol dependence.

[244]  David P Jarmolowicz,et al.  Changing delay discounting in the light of the competing neurobehavioral decision systems theory: a review. , 2013, Journal of the experimental analysis of behavior.

[245]  M. Daly,et al.  Future Discounting by Slum-Dwelling Youth Versus University Students in Rio de Janeiro , 2013 .

[246]  Christopher H. Chatham,et al.  Delaying gratification depends on social trust , 2013, Front. Psychol..

[247]  Christian C Luhmann Discounting of delayed rewards is not hyperbolic. , 2013, Journal of experimental psychology. Learning, memory, and cognition.

[248]  Ernst Fehr,et al.  No Effects of Psychosocial Stress on Intertemporal Choice , 2013, PloS one.

[249]  J. Haushofer,et al.  Time-Dependent Effect of Hydrocortisone Administration on Intertemporal Choice , 2013 .

[250]  Gillian V. Pepper,et al.  Death and the time of your life: experiences of close bereavement are associated with steeper financial future discounting and earlier reproduction , 2013 .

[251]  M. E. Yamamoto,et al.  Do people adjust career choices according to socioeconomic conditions?: an evolutionary analysis of future discounting , 2013 .

[252]  Sendhil Mullainathan,et al.  Poverty Impedes Cognitive Function , 2013, Science.

[253]  R. Weller,et al.  An fMRI investigation of delay discounting in patients with schizophrenia , 2013, Brain and behavior.

[254]  P. Dayan,et al.  Goals and Habits in the Brain , 2013, Neuron.

[255]  Thomas H. B. FitzGerald,et al.  Disruption of Dorsolateral Prefrontal Cortex Decreases Model-Based in Favor of Model-free Control in Humans , 2013, Neuron.

[256]  Karl J. Friston,et al.  Computational psychiatry: the brain as a phantastic organ. , 2014, The lancet. Psychiatry.

[257]  C. Mathys,et al.  Computational approaches to psychiatry , 2014, Current Opinion in Neurobiology.

[258]  B. Reynolds,et al.  Delay discounting by depressed and non-depressed adolescent smokers and non-smokers. , 2014, Drug and alcohol dependence.

[259]  Mikhail N. Koffarnus,et al.  The behavioral- and neuro-economic process of temporal discounting: A candidate behavioral marker of addiction , 2014, Neuropharmacology.

[260]  R. Weller,et al.  Delay discounting and task performance consistency in patients with schizophrenia , 2014, Psychiatry Research.

[261]  W. Bickel,et al.  Discounting of money and sex: effects of commodity and temporal position in stimulant-dependent men and women. , 2014, Addictive behaviors.

[262]  S. Fields,et al.  The relationship between stress and delay discounting: a meta-analytic review , 2014, Behavioural pharmacology.

[263]  A. Marchetti,et al.  Is a bird in the hand worth two in the future? Intertemporal choice, attachment and theory of mind in school-aged children , 2014, Front. Psychol..

[264]  G. Schoenbaum,et al.  Transition from ‘model-based’ to ‘model-free’ behavioral control in addiction: Involvement of the orbitofrontal cortex and dorsolateral striatum , 2014, Neuropharmacology.

[265]  Samuel M. McClure,et al.  Choosing Money over Drugs: The Neural Underpinnings of Difficult Choice in Chronic Cocaine Users , 2014, Journal of addiction.

[266]  Edmund J S Sonuga-Barke Reward: commentary. Temporal discounting in conduct disorder: toward an experience-adaptation hypothesis of the role of psychosocial insecurity. , 2014, Journal of personality disorders.

[267]  M. D. Giudice,et al.  An Evolutionary Life History Framework for Psychopathology , 2014 .

[268]  C. Ramey,et al.  A Competing Neurobehavioral Decision Systems model of SES-related health and behavioral disparities. , 2014, Preventive medicine.

[269]  Giles W. Story,et al.  Does temporal discounting explain unhealthy behavior? A systematic review and reinforcement learning perspective , 2014, Front. Behav. Neurosci..

[270]  Giles W. Story,et al.  The computational psychiatry of reward: broken brains or misguided minds? , 2015, Front. Psychol..

[271]  B. Verplanken,et al.  The strength of habit , 2015, Health psychology review.

[272]  Raymond J. Dolan,et al.  Anticipation and Choice Heuristics in the Dynamic Consumption of Pain Relief , 2015, PLoS Comput. Biol..

[273]  T. Ford,et al.  The Association Between Socioeconomic Disadvantage and Attention Deficit/Hyperactivity Disorder (ADHD): A Systematic Review , 2015, Child Psychiatry & Human Development.

[274]  Shanefrederick,et al.  Time Discounting and Time Preference : A Critical Review , 2022 .