Don’t Let Me Do That! – Models of Precommitment

Precommitment, or taking away a future choice from oneself, is a mechanism for overcoming impulsivity. Here we review recent work suggesting that precommitment can be best explained through a distributed decision-making system with multiple discounting rates. This model makes specific predictions about precommitment behavior and is especially interesting in light of the emerging multiple-systems view of decision-making, in which functional systems with distinct neural substrates use different computational strategies to optimize decisions. Given the growing consensus that impulsivity constitutes a common point of breakdown in decision-making processes, with common neural and computational mechanisms across multiple psychiatric disorders, it is useful to translate precommitment into the common language of temporal difference reinforcement learning that unites many of these behavioral and neural data.

[1]  Janet B W Williams Diagnostic and Statistical Manual of Mental Disorders , 2013 .

[2]  Alex Blaszczynski,et al.  Pre-commitment in gambling: a review of the empirical evidence , 2012 .

[3]  Mikhail N. Koffarnus,et al.  Excessive discounting of delayed reinforcers as a trans-disease process contributing to addiction and other disease-related vulnerabilities: emerging evidence. , 2012, Pharmacology & therapeutics.

[4]  Karl J. Friston,et al.  Computational psychiatry (vol 16, pg 72, 2012) , 2012 .

[5]  Matthijs A. A. van der Meer,et al.  Information Processing in Decision-Making Systems , 2012, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[6]  Guillem R. Esber,et al.  Surprise! Neural correlates of Pearce–Hall and Rescorla–Wagner coexist within the brain , 2012, The European journal of neuroscience.

[7]  J. O'Doherty,et al.  Beyond simple reinforcement learning: the computational neurobiology of reward‐learning and valuation , 2012, The European journal of neuroscience.

[8]  Z. Kurth-Nelson,et al.  A theoretical account of cognitive effects in delay discounting , 2012, The European journal of neuroscience.

[9]  Peter Dayan,et al.  Bonsai Trees in Your Head: How the Pavlovian System Sculpts Goal-Directed Choices by Pruning Decision Trees , 2012, PLoS Comput. Biol..

[10]  P. Dayan,et al.  Mapping value based planning and extensively trained choice in the human brain , 2012, Nature Neuroscience.

[11]  T. Robbins,et al.  Neurocognitive endophenotypes of impulsivity and compulsivity: towards dimensional psychiatry , 2012, Trends in Cognitive Sciences.

[12]  Karl J. Friston,et al.  Computational psychiatry , 2012, Trends in Cognitive Sciences.

[13]  A. David Redish,et al.  Modeling Decision-Making Systems in Addiction , 2012 .

[14]  J. Giannetta,et al.  Does adolescent risk taking imply weak executive function? A prospective study of relations between working memory performance, impulsivity, and risk taking in early adolescence. , 2011, Developmental science.

[15]  N. Petry Contingency Management for Substance Abuse Treatment: A Guide to Implementing This Evidence-Based Practice , 2011 .

[16]  L. Green,et al.  Domain independence and stability in young and older adults’ discounting of delayed rewards , 2011, Behavioural Processes.

[17]  Dylan A. Simon,et al.  Neural Correlates of Forward Planning in a Spatial Decision Task in Humans , 2011, The Journal of Neuroscience.

[18]  P. Dayan,et al.  Model-based influences on humans’ choices and striatal prediction errors , 2011, Neuron.

[19]  M. Frank,et al.  From reinforcement learning models to psychiatric and neurological disorders , 2011, Nature Neuroscience.

[20]  R. Kurzban,et al.  Why Everyone (Else) Is a Hypocrite: Evolution and the Modular Mind , 2011 .

[21]  R. Baumeister,et al.  Willpower: Rediscovering the Greatest Human Strength , 2011 .

[22]  A. Cooper,et al.  Predictive Reward Signal of Dopamine Neurons , 2011 .

[23]  Z. Kurth-Nelson,et al.  A Reinforcement Learning Model of Precommitment in Decision Making , 2010, Front. Behav. Neurosci..

[24]  Makoto Ito,et al.  Evidence for Model-Based Action Planning in a Sequential Finger Movement Task , 2010, Journal of motor behavior.

[25]  William H. Alexander,et al.  Hyperbolically Discounted Temporal Difference Learning , 2010, Neural Computation.

[26]  P. Dayan,et al.  States versus Rewards: Dissociable Neural Prediction Error Signals Underlying Model-Based and Model-Free Reinforcement Learning , 2010, Neuron.

[27]  Stefan M. Gruenert,et al.  Addiction: A Disorder of Choice , 2010 .

[28]  G. Madden,et al.  Impulsivity: The Behavioral and Neurological Science of Discounting , 2010 .

[29]  Zeb Kurth-Nelson,et al.  Temporal-Difference Reinforcement Learning with Distributed Representations , 2009, PloS one.

[30]  P. Glimcher,et al.  The Neurobiology of Decision: Consensus and Controversy , 2009, Neuron.

[31]  W. Bickel,et al.  Buprenorphine medication versus voucher contingencies in promoting abstinence from opioids and cocaine. , 2009, Experimental and clinical psychopharmacology.

[32]  K. Kirby One-year temporal stability of delay-discount rates , 2009, Psychonomic bulletin & review.

[33]  Aldo Rustichini,et al.  Cognitive skills affect economic preferences, strategic behavior, and job attachment , 2009, Proceedings of the National Academy of Sciences.

[34]  Michael J. A. Wohl,et al.  Episodic Cessation of Gambling: A Numerically Aided Phenomenological Assessment of Why Gamblers Stop Playing in a Given Session , 2008 .

[35]  T. Robbins,et al.  Neurobehavioral mechanisms of impulsivity: Fronto-striatal systems and functional neurochemistry , 2008, Pharmacology Biochemistry and Behavior.

[36]  Adam Johnson,et al.  Computing motivation: Incentive salience boosts of drug or appetite states , 2008, Behavioral and Brain Sciences.

[37]  Colin Camerer,et al.  A framework for studying the neurobiology of value-based decision making , 2008, Nature Reviews Neuroscience.

[38]  Noelle M. Nelson,et al.  Making choices impairs subsequent self-control: a limited-resource account of decision making, self-regulation, and active initiative. , 2008, Journal of personality and social psychology.

[39]  Saori C. Tanaka,et al.  Low-Serotonin Levels Increase Delayed Reward Discounting in Humans , 2008, The Journal of Neuroscience.

[40]  G. Loewenstein,et al.  Intangibility in intertemporal choice , 2008, Philosophical Transactions of the Royal Society B: Biological Sciences.

[41]  Colin Camerer,et al.  Neuroeconomics: decision making and the brain , 2008 .

[42]  Saori C. Tanaka,et al.  Serotonin Differentially Regulates Short- and Long-Term Prediction of Rewards in the Ventral and Dorsal Striatum , 2007, PloS one.

[43]  Tomohiro Miyakawa,et al.  Two-month stability of hyperbolic discount rates for delayed monetary gains in abstinent inpatient alcoholics. , 2007, Neuro endocrinology letters.

[44]  Ronald J. Faber,et al.  Spent Resources: Self‐Regulatory Resource Availability Affects Impulse Buying , 2007 .

[45]  Peter Dayan,et al.  Non-commercial Research and Educational Use including without Limitation Use in Instruction at Your Institution, Sending It to Specific Colleagues That You Know, and Providing a Copy to Your Institution's Administrator. All Other Uses, Reproduction and Distribution, including without Limitation Comm , 2022 .

[46]  Taiki Takahashi,et al.  Three-month stability of delay and probability discounting measures. , 2006, Experimental and clinical psychopharmacology.

[47]  P. Dayan,et al.  Opinion TRENDS in Cognitive Sciences Vol.10 No.8 Full text provided by www.sciencedirect.com A normative perspective on motivation , 2022 .

[48]  J. Haidt The happiness hypothesis: Finding modern truth in ancient wisdom. , 2006 .

[49]  P. Dayan,et al.  Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control , 2005, Nature Neuroscience.

[50]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[51]  Samuel M. McClure,et al.  Separate Neural Systems Value Immediate and Delayed Monetary Rewards , 2004, Science.

[52]  Sean R Eddy,et al.  What is dynamic programming? , 2004, Nature Biotechnology.

[53]  George Ainslie,et al.  Building blocks of self-control: increased tolerance for delay with bundled rewards. , 2003, Journal of the experimental analysis of behavior.

[54]  N. Daw,et al.  Reinforcement learning models of the dopamine system and their behavioral implications , 2003 .

[55]  G. Loewenstein,et al.  Time Discounting and Time Preference: A Critical Review , 2002 .

[56]  M. El-Sabaawi Breakdown of Will , 2002 .

[57]  D. Ariely,et al.  Procrastination, Deadlines, and Performance: Self-Control by Precommitment , 2002, Psychological science.

[58]  W. Bickel,et al.  Toward a behavioral economic understanding of drug dependence: delay discounting processes. , 2001, Addiction.

[59]  David S. Touretzky,et al.  Behavioral considerations suggest an average reward TD model of the dopamine system , 2000, Neurocomputing.

[60]  K. Nader,et al.  A Two-Separate-Motivational-Systems Hypothesis of Opioid Addiction , 1998, Pharmacology Biochemistry and Behavior.

[61]  J. E. Mazur Choice, delay, probability, and conditioned reinforcement , 1997 .

[62]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[63]  P. Dayan,et al.  A framework for mesencephalic dopamine systems based on predictive Hebbian learning , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[64]  R. Baumeister,et al.  Losing Control: How and Why People Fail at Self-Regulation , 1994 .

[65]  G. Ainslie Picoeconomics: The Strategic Interaction of Successive Motivational States within the Person , 1992 .

[66]  G. Ainslie,et al.  Impulse control in pigeons. , 1974, Journal of the experimental analysis of behavior.

[67]  H Rachlin,et al.  Commitment, choice and self-control. , 1972, Journal of the experimental analysis of behavior.

[68]  R. H. Strotz Myopia and Inconsistency in Dynamic Utility Maximization , 1955 .