Loss Aversion Correlates With the Propensity to Deploy Model-Based Control

Reward-based decision making is thought to be driven by at least two different types of decision systems: a simple stimulus–response cache-based system which embodies the common-sense notion of “habit,” for which model-free reinforcement learning serves as a computational substrate, and a more deliberate, prospective, model-based planning system. Previous work has shown that loss aversion, a well-studied measure of how much more on average individuals weigh losses relative to gains during decision making, is reduced when participants take all possible decisions and outcomes into account including future ones, relative to when they myopically focus on the current decision. Model-based control offers a putative mechanism for implementing such foresight. Using a well-powered data set (N = 117) in which participants completed two different tasks designed to measure each of the two quantities of interest, and four models of choice data for these tasks, we found consistent evidence of a relationship between loss aversion and model-based control but in the direction opposite to that expected based on previous work: loss aversion had a positive relationship with model-based control. We did not find evidence for a relationship between either decision system and risk aversion, a related aspect of subjective utility.

[1]  P. Bossaerts,et al.  Neurobiological studies of risk assessment: A comparison of expected utility and mean-variance approaches , 2008, Cognitive, affective & behavioral neuroscience.

[2]  J. Roiser,et al.  Enhanced Risk Aversion, But Not Loss Aversion, in Unmedicated Pathological Anxiety , 2017, Biological Psychiatry.

[3]  P. Dayan,et al.  Disorders of compulsivity: a common bias towards learning habits , 2014, Molecular Psychiatry.

[4]  Peter Dayan,et al.  Bonsai Trees in Your Head: How the Pavlovian System Sculpts Goal-Directed Choices by Pruning Decision Trees , 2012, PLoS Comput. Biol..

[5]  Shinsuke Shimojo,et al.  Neural Computations Underlying Arbitration between Model-Based and Model-free Learning , 2013, Neuron.

[6]  Alec Solway,et al.  Goal-directed decision making as probabilistic inference: a computational framework and potential neural correlates. , 2012, Psychological review.

[7]  Nathaniel D. Daw,et al.  Cognitive Control Predicts Use of Model-based Reinforcement Learning , 2014, Journal of Cognitive Neuroscience.

[8]  Alec Solway,et al.  Simulating future value in intertemporal choice , 2017, Scientific Reports.

[9]  Y. Bar-Haim,et al.  Threat-related attentional bias in anxious and nonanxious individuals: a meta-analytic study. , 2007, Psychological bulletin.

[10]  Mahesan Niranjan,et al.  On-line Q-learning using connectionist systems , 1994 .

[11]  H. Berenbaum,et al.  The relation between worrying and concerns: the importance of perceived probability and cost. , 2007, Behaviour research and therapy.

[12]  A. Tversky,et al.  Advances in prospect theory: Cumulative representation of uncertainty , 1992 .

[13]  Thomas H. B. FitzGerald,et al.  Disruption of Dorsolateral Prefrontal Cortex Decreases Model-Based in Favor of Model-free Control in Humans , 2013, Neuron.

[14]  Sandra F. Lackovic,et al.  Determinants of Propranolol’s Selective Effect on Loss Aversion , 2015, Psychological science.

[15]  D. Rubin,et al.  Inference from Iterative Simulation Using Multiple Sequences , 1992 .

[16]  F. W. Irwin.  Purposive Behavior in Animals and Men , 1932, The Psychological Clinic.

[17]  Dylan A. Simon,et al.  Neural Correlates of Forward Planning in a Spatial Decision Task in Humans , 2011, The Journal of Neuroscience.

[18]  N. Daw,et al.  Multiple memory systems as substrates for multiple decision systems , 2015, Neurobiology of Learning and Memory.

[19]  A. Tversky,et al.  Prospect theory: an analysis of decision under risk — Source link , 2007 .

[20]  L. S. Kogan.  Review of Principles of Behavior. , 1943 .

[21]  Peter Sokol-Hessner,et al.  Interoceptive ability predicts aversion to losses , 2015, Cognition & emotion.

[22]  M. Botvinick,et al.  Evidence integration in model-based tree search , 2015, Proceedings of the National Academy of Sciences.

[23]  Amir Dezfouli,et al.  Speed/Accuracy Trade-Off between the Habitual and the Goal-Directed Processes , 2011, PLoS Comput. Biol..

[24]  Andrew Gelman,et al.  The No-U-turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo , 2011, J. Mach. Learn. Res..

[25]  Vivian V. Valentin,et al.  Determining the Neural Substrates of Goal-Directed Learning in the Human Brain , 2007, The Journal of Neuroscience.

[26]  J. Pratt.  RISK AVERSION IN THE SMALL AND IN THE LARGE11This research was supported by the National Science Foundation (grant NSF-G24035). Reproduction in whole or in part is permitted for any purpose of the United States Government. , 1964 .

[27]  P. Dayan,et al.  States versus Rewards: Dissociable Neural Prediction Error Signals Underlying Model-Based and Model-Free Reinforcement Learning , 2010, Neuron.

[28]  Andrew Mathews,et al.  COGNITIVE PROCESSES IN ANXIETY , 1983 .

[29]  Colin F Camerer,et al.  Emotion regulation reduces loss aversion and decreases amygdala responses to losses. , 2013, Social cognitive and affective neuroscience.

[30]  M. Vasey,et al.  A catastrophizing assessment of worrisome thoughts , 1992, Cognitive Therapy and Research.

[31]  Dylan A. Simon,et al.  Model-based choices involve prospective neural activity , 2015, Nature Neuroscience.

[32]  P. Dayan,et al.  Model-based influences on humans’ choices and striatal prediction errors , 2011, Neuron.

[33]  Alice Y. Chiang,et al.  Working-memory capacity protects model-based learning from stress , 2013, Proceedings of the National Academy of Sciences.

[34]  Neil Stewart,et al.  How to Make Loss Aversion Disappear and Reverse: Tests of the Decision by Sampling Origin of Loss Aversion , 2014, Journal of experimental psychology. General.

[35]  Gordon D. A. Brown,et al.  Decision by sampling , 2006, Cognitive Psychology.

[36]  A. Tversky,et al.  Prospect theory: analysis of decision under risk , 1979 .

[37]  P. Dayan,et al.  Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control , 2005, Nature Neuroscience.

[38]  Andreas Glöckner,et al.  Cognitive models of risky choice: Parameter stability and predictive accuracy of prospect theory , 2012, Cognition.

[39]  Colin Camerer,et al.  Thinking like a trader selectively reduces individuals' loss aversion , 2009, Proceedings of the National Academy of Sciences.

[40]  K. Bredemeier,et al.  Perceived threat: exploring its association with worry and its hypothesized antecedents. , 2007, Behaviour research and therapy.

[41]  E. Koster,et al.  Mechanisms of attentional biases towards threat in anxiety disorders: An integrative review. , 2010, Clinical psychology review.

[42]  S. Killcross,et al.  Coordination of actions and habits in the medial prefrontal cortex of rats. , 2003, Cerebral cortex.

[43]  Sabrina M. Tom,et al.  The Neural Basis of Loss Aversion in Decision-Making Under Risk , 2007, Science.