Dissecting the links between reward and loss, decision-making, and self-reported affect using a computational approach

Links between affective states and risk-taking are often characterised using summary statistics from serial decision-making tasks. However, our understanding of these links, and the utility of decision-making as a marker of affect, needs to accommodate the fact that ongoing (e.g., within-task) experience of rewarding and punishing decision outcomes may alter future decisions and affective states. To date, the interplay between affect, ongoing reward and punisher experience, and decision-making has received little detailed investigation. Here, we examined the relationships between reward and loss experience, affect, and decision-making in humans using a novel judgement bias task analysed with a novel computational model. We demonstrated the influence of within-task favourability on decision-making, with more risk-averse/‘pessimistic’ decisions following more positive previous outcomes and a greater current average earning rate. Additionally, individuals reporting more negative affect tended to exhibit greater risk-seeking decision-making, and, based on our model, estimated time more poorly. We also found that individuals reported more positive affective valence during periods of the task when prediction errors and offered decision outcomes were more positive. Our results thus provide new evidence that (short-term) within-task rewarding and punishing experiences determine both future decision-making and subjectively experienced affective states.

[1]  P. Dayan,et al.  Reward and punisher experience alter rodent decision-making in a judgement bias task , 2020, Scientific Reports.

[2]  M. Bateson,et al.  Optimism, pessimism and judgement bias in animals: A systematic review and meta-analysis , 2020, Neuroscience & Biobehavioral Reviews.

[3]  M. Mendl,et al.  Animal affect and decision-making , 2020, Neuroscience & Biobehavioral Reviews.

[4]  M. Bateson,et al.  Pharmacological manipulations of judgement bias: A systematic review and meta-analysis , 2019, Neuroscience & Biobehavioral Reviews.

[5]  Tobias U. Hauser,et al.  Endogenous fluctuations in the dopaminergic midbrain drive behavioral choice variability , 2019, Proceedings of the National Academy of Sciences.

[6]  P. Dayan,et al.  Assessing animal affect: an automated and self-initiated judgement bias task based on natural investigative behaviour , 2018, Scientific Reports.

[7]  Bruna Velasques,et al.  The dopaminergic system dynamic in the time perception: a review of the evidence , 2018, The International journal of neuroscience.

[8]  Karl J. Friston,et al.  What is mood? A computational perspective , 2018, Psychological Medicine.

[9]  A. Graybiel,et al.  Chronic Stress Alters Striosome-Circuit Dynamics, Leading to Aberrant Decision-Making , 2017, Cell.

[10]  P. Dayan,et al.  Pavlovian influences on learning differ between rats and mice in a counter-balanced Go/NoGo judgement bias task , 2017, Behavioural Brain Research.

[11]  K. Thomas,et al.  Clinical Depression and Punishment Sensitivity on the BART , 2017, Front. Psychol..

[12]  Ulrik R Beierholm,et al.  Opposing effects of reward and punishment on human vigor , 2017, Scientific Reports.

[13]  P. Dayan,et al.  Cognitive Bias in Ambiguity Judgements: Using Computational Models to Dissect the Effects of Mild Mood Manipulation in Humans , 2016, PloS one.

[14]  B. Kudielka,et al.  Increased Risk Taking in Relation to Chronic Stress in Adults , 2016, Front. Psychol..

[15]  P. Glimcher,et al.  Unexpected but Incidental Positive Outcomes Predict Real-World Gambling , 2016, Psychological science.

[16]  Y. Niv,et al.  Mood as Representation of Momentum , 2016, Trends in Cognitive Sciences.

[17]  Angela J. Yu,et al.  Statistical learning and adaptive decision-making underlie human response time variability in inhibitory control , 2015, Front. Psychol..

[18]  Timothy E. J. Behrens,et al.  Anxious individuals have difficulty learning the causal statistics of aversive environments , 2015, Nature Neuroscience.

[19]  G. Arbanas Diagnostic and Statistical Manual of Mental Disorders (DSM-5) , 2015 .

[20]  P. Dayan,et al.  A computational and neural model of momentary subjective well-being , 2014, Proceedings of the National Academy of Sciences.

[21]  Karl J. Friston,et al.  Computational psychiatry: the brain as a phantastic organ. , 2014, The lancet. Psychiatry.

[22]  Peggy Seriès,et al.  Edinburgh Research Explorer Optimism as a Prior Belief about the Probability of Future Reward , 2022 .

[23]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[24]  Sylvie Droit-Volet,et al.  Time perception, emotions and mood disorders , 2013, Journal of Physiology-Paris.

[25]  O. Arias-Carrión,et al.  Time perception distortion in neuropsychiatric and neurological disorders. , 2013, CNS & neurological disorders drug targets.

[26]  Edmund T. Rolls,et al.  What are Emotional States, and Why Do We Have Them? , 2013 .

[27]  Ulrik R Beierholm,et al.  Dopamine Modulates Reward-Related Vigor , 2013, Neuropsychopharmacology.

[28]  P. Dayan,et al.  Mapping anhedonia onto reinforcement learning: a behavioural meta-analysis , 2013, Biology of Mood & Anxiety Disorders.

[29]  Daniel Nettle,et al.  The Evolutionary Origins of Mood and Its Disorders , 2012, Current Biology.

[30]  Georges J. Han,et al.  Selective neurocognitive impairments in adolescents with major depressive disorder. , 2012, Journal of adolescence.

[31]  Peter Dayan,et al.  Vigor in the Face of Fluctuating Rates of Reward: An Experimental Examination , 2011, Journal of Cognitive Neuroscience.

[32]  K. LaBar,et al.  Unpredictability and Uncertainty in Anxiety: A New Direction for Emotional Timing Research , 2011, Front. Integr. Neurosci..

[33]  James A. R. Marshall,et al.  Decision-making under uncertainty: biases and Bayesians , 2011, Animal Cognition.

[34]  I. Cuthill,et al.  Mood and the speed of decisions about anticipated resources and hazards , 2011 .

[35]  Eduardo F. Morales,et al.  An Introduction to Reinforcement Learning , 2011 .

[36]  M. Mendl,et al.  An integrative and functional framework for the study of animal emotion and mood , 2010, Proceedings of the Royal Society B: Biological Sciences.

[37]  A. Cooper,et al.  Impaired flexible decision-making in Major Depressive Disorder. , 2010, Journal of affective disorders.

[38]  D. Washburn,et al.  Task engagement, cerebral blood flow velocity, and diagnostic monitoring for sustained attention. , 2010, Journal of experimental psychology. Applied.

[39]  Marissa A. Gorlick,et al.  Acute Stress Increases Sex Differences in Risk Seeking in the Balloon Analogue Risk Task , 2009, PloS one.

[40]  Richard M A Parker,et al.  Cognitive bias as an indicator of animal emotion and welfare: Emerging evidence and underlying mechanisms , 2009 .

[41]  A. Kacelnik,et al.  Successive negative contrast in a bird: starlings' behaviour after unpredictable negative changes in food quality , 2009, Animal Behaviour.

[42]  D. Nettle An evolutionary model of low mood states. , 2009, Journal of theoretical biology.

[43]  S. Droit-Volet,et al.  Time perception, depression and sadness , 2009, Behavioural Processes.

[44]  M. Fava,et al.  Reduced hedonic capacity in major depressive disorder: evidence from a probabilistic reward task. , 2008, Journal of psychiatric research.

[45]  I. Tracey,et al.  A common neurobiology for pain and pleasure , 2008, Nature Reviews Neuroscience.

[46]  Roger Ratcliff,et al.  The Diffusion Decision Model: Theory and Data for Two-Choice Decision Tasks , 2008, Neural Computation.

[47]  J. Gold,et al.  The neural basis of decision making. , 2007, Annual review of neuroscience.

[48]  R. Dolan,et al.  The neurobiology of punishment , 2007, Nature Reviews Neuroscience.

[49]  H. Buchanan-Smith,et al.  Effects of predictability on the welfare of captive animals , 2007 .

[50]  A. Schene,et al.  Mood is indirectly related to serotonin, norepinephrine and dopamine levels in humans: a meta-analysis of monoamine depletion studies , 2007, Molecular Psychiatry.

[51]  D. Pizzagalli,et al.  Acute Stress Reduces Reward Responsiveness: Implications for Depression , 2006, Biological Psychiatry.

[52]  Timothy D. Wilson,et al.  Loss Aversion Is an Affective Forecasting Error , 2006, Psychological science.

[53]  Angela J. Yu,et al.  Uncertainty, Neuromodulation, and Attention , 2005, Neuron.

[54]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[55]  M. Mendl,et al.  Animal behaviour: Cognitive bias and affective state , 2004, Nature.

[56]  C. Clifford Perceptual adaptation: motion parallels orientation , 2002, Trends in Cognitive Sciences.

[57]  Gregory L. Stuart,et al.  Evaluation of a behavioral measure of risk taking: the Balloon Analogue Risk Task (BART). , 2002, Journal of experimental psychology. Applied.

[58]  F A Wichmann,et al.  Ning for Helpful Comments and Suggestions. This Paper Benefited Con- Siderably from Conscientious Peer Review, and We Thank Our Reviewers the Psychometric Function: I. Fitting, Sampling, and Goodness of Fit , 2001 .

[59]  Charles S. Carver,et al.  Affect and the Functional Bases of Behavior: On the Dimensional Structure of Affective Experience , 2001 .

[60]  Gerald Matthews,et al.  Individual differences in energetic arousal and sustained attention: a dual-task study , 2001 .

[61]  Matthew R. Tinsley,et al.  Incentive downshifts evoke search repertoires in rats. , 1999, Journal of experimental psychology. Animal behavior processes.

[62]  W. Killgore,et al.  The Affect Grid: A Moderately Valid, Nonspecific Measure of Pleasure and Arousal , 1998, Psychological reports.

[63]  A. Damasio,et al.  Deciding Advantageously Before Knowing the Advantageous Strategy , 1997, Science.

[64]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[65]  Thomas E. Nygren,et al.  The Influence of Positive Affect on the Decision Rule in Risk Situations: Focus on Outcome (and Especially Avoidance of Loss) Rather Than Probability , 1996 .

[66]  D. Diehl,et al.  The role of dopamine in mood disorders. , 1992, Comprehensive psychiatry.

[67]  Shelley E. Taylor,et al.  Asymmetrical effects of positive and negative events: the mobilization-minimization hypothesis. , 1991, Psychological bulletin.

[68]  G. A. Mendelsohn,et al.  Affect grid : A single-item scale of pleasure and arousal , 1989 .

[69]  Thomas E. Nygren,et al.  Influence of positive affect on the subjective utility of gains and losses: it is just not worth the risk. , 1988, Journal of personality and social psychology.

[70]  A. Isen,et al.  The effect of positive feelings on risk taking: When the chips are down. , 1983 .

[71]  A. Tversky,et al.  Prospect Theory : An Analysis of Decision under Risk Author ( s ) : , 2007 .

[72]  A. Tversky,et al.  Prospect theory: analysis of decision under risk , 1979 .

[73]  G. Schwarz Estimating the Dimension of a Model , 1978 .

[74]  J. Gibbon Scalar expectancy theory and Weber's law in animal timing. , 1977 .

[75]  H. Akaike A new look at the statistical model identification , 1974 .

[76]  Edward J. Sondik,et al.  The Optimal Control of Partially Observable Markov Processes over a Finite Horizon , 1973, Oper. Res..

[77]  R. Rescorla,et al.  A theory of Pavlovian conditioning : Variations in the effectiveness of reinforcement and nonreinforcement , 1972 .

[78]  J. Gray,et al.  The psychology of fear and stress , 1971 .

[79]  M. Seligman,et al.  Chronic fear and ulcers in rats as a function of the unpredictability of safety. , 1970, Journal of comparative and physiological psychology.

[80]  D. M. Green,et al.  Signal detection theory and psychophysics , 1966 .

[81]  Karl Johan Åström,et al.  Optimal control of Markov processes with incomplete state information , 1965 .

[82]  R Bellman,et al.  On the Theory of Dynamic Programming. , 1952, Proceedings of the National Academy of Sciences of the United States of America.