Behavioural and neural characterization of optimistic reinforcement learning

When forming and updating beliefs about future life outcomes, people tend to consider good news and to disregard bad news. This tendency is assumed to support the optimism bias. Whether this learning bias is specific to ‘high-level’ abstract belief update or a particular expression of a more general ‘low-level’ reinforcement learning process is unknown. Here we report evidence in favour of the second hypothesis. In a simple instrumental learning task, participants incorporated better-than-expected outcomes at a higher rate than worse-than-expected ones. In addition, functional imaging indicated that inter-individual difference in the expression of optimistic update corresponds to enhanced prediction error signalling in the reward circuitry. Our results constitute a step towards the understanding of the genesis of optimism bias at the neurocomputational level.

[1]  John Butt,et al.  Candide, or, Optimism , 1919 .

[2]  K. Popper,et al.  The Logic of Scientific Discovery , 1960 .

[3]  Günter Altner,et al.  The Nature of human behaviour , 1976 .

[4]  E. A. Burtt The English Philosophers from Bacon to Mill , 1977 .

[5]  A. Tversky,et al.  Prospect theory: analysis of decision under risk , 1979 .

[6]  N. Weinstein Unrealistic optimism about future life events , 1980 .

[7]  Acknowledgements , 1992, Experimental Gerontology.

[8]  J. Ouellette,et al.  Abandoning Unrealistic Optimism: Performance Estimates and the Temporal Proximity of Self-Relevant Feedback , 1996 .

[9]  M. Schoenbaum,et al.  Do smokers understand the mortality effects of smoking? Evidence from the Health and Retirement Survey. , 1997, American journal of public health.

[10]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[11]  Clare Conway,et al.  Well‐being and the anticipation of future positive experiences: The role of income, social networks, and planning ability , 2005, Cognition & emotion.

[12]  J. O'Doherty,et al.  The Role of the Ventromedial Prefrontal Cortex in Abstract State-Based Inference during Decision Making in Humans , 2006, The Journal of Neuroscience.

[13]  Larry Dossey,et al.  Optimism , 1908, Explore.

[14]  A. Tversky,et al.  Prospect theory: an analysis of decision under risk — Source link , 2007 .

[15]  J. O'Doherty,et al.  Model‐Based fMRI and Its Application to Reward Learning and Decision Making , 2007, Annals of the New York Academy of Sciences.

[16]  Michael J. Frank,et al.  Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning , 2007, Proceedings of the National Academy of Sciences.

[17]  E. Phelps,et al.  Neural mechanisms mediating optimism bias , 2007, Nature.

[18]  Z. Diénès Understanding Psychology as a Science: An Introduction to Scientific and Statistical Inference , 2008 .

[19]  Nick Chater,et al.  Herding in humans , 2009, Trends in Cognitive Sciences.

[20]  Jennifer G. Robinson,et al.  Optimism, Cynical Hostility, and Incident Coronary Heart Disease and Mortality in the Women’s Health Initiative , 2009, Circulation.

[21]  M. Pessiglione,et al.  Brain Hemispheres Selectively Track the Expected Value of Contralateral Options , 2009, The Journal of Neuroscience.

[22]  Erika A. Waters,et al.  Correlates of unrealistic risk beliefs in a nationally representative sample , 2011, Journal of Behavioral Medicine.

[23]  M. Pessiglione,et al.  Reinforcement learning and Gilles de la Tourette syndrome: dissociation of clinical phenotypes and pharmacological treatments. , 2011, Archives of general psychiatry.

[24]  Karl J. Friston,et al.  A Bayesian Foundation for Individual Learning Under Uncertainty , 2011, Front. Hum. Neurosci..

[25]  Dominic D. P. Johnson,et al.  The evolution of overconfidence , 2009, Nature.

[26]  R. Gifford The dragons of inaction: psychological barriers that limit climate change mitigation and adaptation. , 2011, The American psychologist.

[27]  Justin M. Rao,et al.  The Good News-Bad News Effect: Asymmetric Processing of Objective Information about Yourself , 2011 .

[28]  Christoph W Korn,et al.  How unrealistic optimism is maintained in the face of reality , 2011, Nature Neuroscience.

[29]  M. Frank,et al.  Dopaminergic Genes Predict Individual Differences in Susceptibility to Confirmation Bias , 2011, The Journal of Neuroscience.

[30]  Stephen Ginn The Optimism Bias: Why We're Wired to Look on the Bright Side Tali Sharot Constable & Robinson, 2012, £8.99, pb, 272 pp. ISBN: 9781780332635 , 2012 .

[31]  Michael X. Cohen,et al.  Striatum-medial prefrontal cortex connectivity predicts developmental changes in reinforcement learning. , 2012, Cerebral cortex.

[32]  R. Dolan,et al.  How Dopamine Enhances an Optimism Bias in Humans , 2012, Current Biology.

[33]  M. Pessiglione,et al.  Critical Roles for Anterior Insula and Dorsal Striatum in Punishment-Based Avoidance Learning , 2012, Neuron.

[34]  P. Dayan,et al.  Neural Prediction Errors Reveal a Risk-Sensitive Reinforcement-Learning Process in the Human Brain , 2012, The Journal of Neuroscience.

[35]  Jane R. Garrison,et al.  Prediction error in reinforcement learning: A meta-analysis of neuroimaging studies , 2013, Neuroscience & Biobehavioral Reviews.

[36]  Romain D. Cazé,et al.  Adaptive properties of differential learning rates for positive and negative outcomes , 2013, Biological Cybernetics.

[37]  Joseph W. Kable,et al.  The valuation system: A coordinate-based meta-analysis of BOLD fMRI experiments examining neural correlates of subjective value , 2013, NeuroImage.

[38]  Neil D Weinstein,et al.  Taking Stock of Unrealistic Optimism , 2013, Perspectives on psychological science : a journal of the Association for Psychological Science.

[39]  Richard C. Clarke,et al.  Human development of the ability to learn from bad news , 2013, Proceedings of the National Academy of Sciences.

[40]  M. Pessiglione,et al.  Learning To Minimize Efforts versus Maximizing Rewards: Computational Principles and Neural Correlates , 2014, The Journal of Neuroscience.

[41]  R. Dolan,et al.  Losing the rose tinted glasses: neural substrates of unbiased belief updating in depression , 2014, Front. Hum. Neurosci..

[42]  Lionel Rigoux,et al.  VBA: A Probabilistic Treatment of Nonlinear Models for Neurobiological and Behavioural Data , 2014, PLoS Comput. Biol..

[43]  Shannon M. Pruden,et al.  A Meta-analysis of Neuroimaging Studies , 2015 .

[44]  Tali Sharot,et al.  Human Frontal–Subcortical Circuit and Asymmetric Belief Updating , 2015, The Journal of Neuroscience.

[45]  M. Khamassi,et al.  Contextual modulation of value signals in reward and punishment learning , 2015, Nature Communications.

[46]  E. Koechlin,et al.  Executive control and decision-making in the prefrontal cortex , 2015, Current Opinion in Behavioral Sciences.

[47]  J. Daunizeau,et al.  Automatic integration of confidence in the brain valuation signal , 2015, Nature Neuroscience.

[48]  Thomas T. Hills,et al.  Exploration versus exploitation in space, mind, and society , 2015, Trends in Cognitive Sciences.

[49]  Robert C. Wilson,et al.  Reinforcement Learning in Multidimensional Environments Relies on Attention Mechanisms , 2015, The Journal of Neuroscience.

[50]  Erika A. Waters,et al.  A Primer on Unrealistic Optimism , 2015, Current directions in psychological science.

[51]  T. Sharot,et al.  Forming Beliefs: Why Valence Matters , 2016, Trends in Cognitive Sciences.

[52]  The Myth of a Pessimistic View of Optimistic Belief Updating – A Commentary on Shah et al. , 2016 .

[53]  Adam J. L. Harris,et al.  A pessimistic view of optimistic belief updating , 2016, Cognitive Psychology.

[54]  M. Frank,et al.  Computational psychiatry as a bridge from neuroscience to clinical applications , 2016, Nature Neuroscience.

[55]  Kai Vogeley,et al.  The role of the neural reward circuitry in self-referential optimistic belief updates , 2016, NeuroImage.

[56]  M. Lebreton,et al.  Assessing Inter-individual Variability in Brain-behavior Relationship Assessing Inter-individual Variability in Brain-behavior Relationship with Functional Neuroimaging , 2022 .

[57]  Marco K. Wittmann,et al.  Multiple signals in anterior cingulate cortex , 2016, Current Opinion in Neurobiology.

[58]  An Analysis of Decision Under Risk in Rats , 2018 .