How Costs Influence Decision Values for Mixed Outcomes

The things that we hold dearest often require a sacrifice, as epitomized in the maxim “no pain, no gain.” But how is the subjective value of outcomes established when they consist of mixtures of costs and benefits? We describe theoretical models for the integration of costs and benefits into a single value, drawing on both the economic and the empirical literatures, with the goal of rendering them accessible to the neuroscience community. We propose two key assays that go beyond goodness of fit for deciding between the dominant additive model and four varieties of interactive models. First, how they model decisions between costs when reward is not on offer; and second, whether they predict changes in reward sensitivity when costs are added to outcomes, and in what direction. We provide a selective review of relevant neurobiological work from a computational perspective, focusing on those studies that illuminate the underlying valuation mechanisms. Cognitive neuroscience has great potential to decide which of the theoretical models is actually employed by our brains, but empirical work has yet to fully embrace this challenge. We hope that future research improves our understanding of how our brain decides whether mixed outcomes are worthwhile.

[1]  G. Ainslie Specious reward: a behavioral theory of impulsiveness and impulse control. , 1975, Psychological bulletin.

[2]  S. Floresco,et al.  Differential effects on effort discounting induced by inactivations of the nucleus accumbens core or shell. , 2010, Behavioral neuroscience.

[3]  I. Simonson,et al.  Choice Based on Reasons: The Case of Attraction and Compromise Effects , 1989 .

[4]  C. M. Bradshaw,et al.  Effects of quinolinic acid-induced lesions of the nucleus accumbens core on inter-temporal choice: a quantitative analysis , 2007, Psychopharmacology.

[5]  Stan B. Floresco,et al.  Cortico-limbic-striatal circuits subserving different forms of cost-benefit decision making , 2008, Cognitive, affective & behavioral neuroscience.

[6]  P. Glimcher,et al.  The neural correlates of subjective value during intertemporal choice , 2007, Nature Neuroscience.

[7]  Daeyeol Lee,et al.  Heterogeneous Coding of Temporally Discounted Values in the Dorsal and Ventral Striatum during Intertemporal Choice , 2011, Neuron.

[8]  G. Loewenstein Anticipation and the Valuation of Delayed Consumption , 1987 .

[9]  M. Roesch,et al.  Should I Stay or Should I Go? , 2007 .

[10]  G. Berns,et al.  BAD TO WORSE , 1975, The Lancet.

[11]  E. Rolls,et al.  Cerebral Cortex Advance Access published June 22, 2007 Expected Value, Reward Outcome, and Temporal Difference Error Representations in a Probabilistic Decision Task , 2022 .

[12]  Martin P. Paulus,et al.  Time and decision making: differential contribution of the posterior insular cortex and the striatum during a delay discounting task , 2007, Experimental Brain Research.

[13]  Gordon D. A. Brown,et al.  Does the brain calculate value? , 2011, Trends in Cognitive Sciences.

[14]  F. B. Vernadat,et al.  Decisions with Multiple Objectives: Preferences and Value Tradeoffs , 1994 .

[15]  Karl J. Friston,et al.  Encoding of Marginal Utility across Time in the Human Brain , 2009, The Journal of Neuroscience.

[16]  L. Green,et al.  Rate of temporal discounting decreases with amount of reward , 1997, Memory & cognition.

[17]  R. Dolan,et al.  Dopamine, Time, and Impulsivity in Humans , 2010, The Journal of Neuroscience.

[18]  Wolfgang Hauber,et al.  Involvement of the rat anterior cingulate cortex in control of instrumental responses guided by reward expectancy. , 2005, Learning & memory.

[19]  J. Deakin,et al.  Effects of orbital prefrontal cortex dopamine depletion on inter-temporal choice: a quantitative analysis , 2004, Psychopharmacology.

[20]  S. Quartz,et al.  Neural Differentiation of Expected Reward and Risk in Human Subcortical Structures , 2006, Neuron.

[21]  Deborah L Thurston,et al.  Multi-Attribute Utility Analysis of Conflicting Preferences , 2006 .

[22]  K. Yau,et al.  Interoception: the sense of the physiological condition of the body , 2003, Current Opinion in Neurobiology.

[23]  A. Tversky,et al.  The framing of decisions and the psychology of choice. , 1981, Science.

[24]  W. Schultz,et al.  Influence of Reward Delays on Responses of Dopamine Neurons , 2008, The Journal of Neuroscience.

[25]  Dino J. Levy,et al.  The root of all value: a neural common currency for choice , 2012, Current Opinion in Neurobiology.

[26]  Marco Casari,et al.  On Negative Time Preference , 2010 .

[27]  Daniel D. Holt,et al.  Discounting of delayed food rewards in pigeons and rats: is there a magnitude effect? , 2004, Journal of the experimental analysis of behavior.

[28]  S. Floresco,et al.  Amygdala-prefrontal cortical circuitry regulates effort-based decision making. , 2006, Cerebral cortex.

[29]  Sabrina M. Tom,et al.  The Neural Basis of Loss Aversion in Decision-Making Under Risk , 2007, Science.

[30]  B. Skinner,et al.  Principles of Behavior , 1944 .

[31]  D. Prelec,et al.  Negative Time Preference , 1991 .

[32]  W. Schultz,et al.  Discrete Coding of Reward Probability and Uncertainty by Dopamine Neurons , 2003, Science.

[33]  T. Robbins,et al.  Limbic Corticostriatal Systems and Delayed Reinforcement , 2004, Annals of the New York Academy of Sciences.

[34]  M. Roesch,et al.  Impact of Size and Delay on Neural Activity in the Rat Limbic Corticostriatal System , 2011, Front. Neurosci..

[35]  D. Zald,et al.  Dopaminergic Mechanisms of Individual Differences in Human Effort-Based Decision-Making , 2012, The Journal of Neuroscience.

[36]  G. Loewenstein,et al.  Decision making over time and under uncertainty: a common approach , 1991 .

[37]  A. Tversky,et al.  Advances in prospect theory: Cumulative representation of uncertainty , 1992 .

[38]  K. White,et al.  Evidence for a magnitude effect in temporal discounting with pigeons. , 2012, Journal of experimental psychology. Animal behavior processes.

[39]  G. Chapman,et al.  The combined effects of risk and time on choice: Does uncertainty eliminate the immediacy effect? Does delay eliminate the certainty effect? , 2005 .

[40]  C. Büchel,et al.  Neural representations of subjective reward value , 2010, Behavioural Brain Research.

[41]  G. Berns,et al.  Intertemporal choice – toward an integrative framework , 2007, Trends in Cognitive Sciences.

[42]  Henrik Walter,et al.  Prediction error as a linear function of reward probability is coded in human nucleus accumbens , 2006, NeuroImage.

[43]  W. Hauber,et al.  Motivational states influence effort-based decision making in rats: The role of dopamine in the nucleus accumbens , 2012, Cognitive, affective & behavioral neuroscience.

[44]  Pablo Brañas-Garza,et al.  Time discounting and pain anticipation: experimental evidence , 2012 .

[45]  R. L. Keeney,et al.  Decisions with Multiple Objectives: Preferences and Value Trade-Offs , 1977, IEEE Transactions on Systems, Man, and Cybernetics.

[46]  Timothy Edward John Behrens,et al.  Effort-Based Cost–Benefit Valuation and the Human Brain , 2009, The Journal of Neuroscience.

[47]  J. Deakin,et al.  Effects of quinolinic acid-induced lesions of the orbital prefrontal cortex on inter-temporal choice: a quantitative analysis , 2002, Psychopharmacology.

[48]  L. Green,et al.  Dopamine modulates effort-based decision making in rats. , 2009, Behavioral neuroscience.

[49]  Barry Setlow,et al.  Balancing Risk and Reward: A Rat Model of Risky Decision-Making , 2009, Neuropsychopharmacology.

[50]  S. Mobini,et al.  Theory and method in the quantitative analysis of ”impulsive choice” behaviour: implications for psychopharmacology , 1999, Psychopharmacology.

[51]  Matthew F S Rushworth,et al.  Functional Specialization within Medial Frontal Cortex of the Anterior Cingulate for Evaluating Effort-Related Decisions , 2003, The Journal of Neuroscience.

[52]  S. Quartz,et al.  Human Insula Activation Reflects Risk Prediction Errors As Well As Risk , 2008, The Journal of Neuroscience.

[53]  A. Tversky,et al.  Prospect theory: an analysis of decision under risk — Source link , 2007 .

[54]  G. Glover,et al.  Dissociable Intrinsic Connectivity Networks for Salience Processing and Executive Control , 2007, The Journal of Neuroscience.

[55]  Thomas S. Critchfield,et al.  Introduction to the Special Issue: Translational Research on Discounting , 2011 .

[56]  A. Tversky,et al.  Choices, Values, and Frames , 2000 .

[57]  L. S. Kogan Review of Principles of Behavior. , 1943 .

[58]  C. Padoa-Schioppa Neurobiology of economic choice: a good-based model. , 2011, Annual review of neuroscience.

[59]  Joseph T. McGuire,et al.  Effort discounting in human nucleus accumbens , 2009, Cognitive, affective & behavioral neuroscience.

[60]  Brian Knutson,et al.  Anticipation of Increasing Monetary Reward Selectively Recruits Nucleus Accumbens , 2001, The Journal of Neuroscience.

[61]  John D. Salamone,et al.  Nucleus accumbens dopamine depletions in rats affect relative response allocation in a novel cost/benefit procedure , 1994, Pharmacology Biochemistry and Behavior.

[62]  M. Roesch,et al.  Neuronal activity in primate orbitofrontal cortex reflects the value of time. , 2005, Journal of neurophysiology.

[63]  Mathias Pessiglione,et al.  Separate Valuation Subsystems for Delay and Effort Decision Costs , 2010, The Journal of Neuroscience.

[64]  María Paz Espinosa,et al.  Descuento temporal y anticipación del dolor. Evidencia experimental , 2012 .

[65]  J. Pender Discount rates and credit markets" Theory and evidence from rural india , 1996 .

[66]  L. Green,et al.  Temporal discounting and preference reversals in choice between delayed outcomes , 1994, Psychonomic bulletin & review.

[67]  T. Kalenscher,et al.  Single Units in the Pigeon Brain Integrate Reward Amount and Time-to-Reward in an Impulsive Choice Task , 2005, Current Biology.

[68]  J. O'Doherty,et al.  Reward Value Coding Distinct From Risk Attitude-Related Uncertainty Coding in Human Reward Systems , 2006, Journal of neurophysiology.

[69]  Joel Myerson,et al.  Delay discounting in rhesus monkeys: Equivalent discounting of more and less preferred sucrose concentrations , 2012, Learning & behavior.

[70]  R. Thaler Some empirical evidence on dynamic inconsistency , 1981 .

[71]  Joshua L. Jones,et al.  Nucleus accumbens neurons encode predicted and ongoing reward costs in rats , 2011, The European journal of neuroscience.

[72]  Christopher K. Hsee,et al.  Decision and experience: why don't we choose what makes us happy? , 2006, Trends in Cognitive Sciences.

[73]  L. Green,et al.  Modeling the effect of reward amount on probability discounting. , 2011, Journal of the experimental analysis of behavior.

[74]  Colin Camerer,et al.  Explicit neural signals reflecting reward uncertainty , 2008, Philosophical Transactions of the Royal Society B: Biological Sciences.

[75]  Samuel M. McClure,et al.  Separate Neural Systems Value Immediate and Delayed Monetary Rewards , 2004, Science.

[76]  S. Floresco,et al.  Dopaminergic and Glutamatergic Regulation of Effort- and Delay-Based Decision Making , 2008, Neuropsychopharmacology.

[77]  Jeffrey R. Stevens,et al.  Will Travel for Food: Spatial Discounting in Two New World Monkeys , 2005, Current Biology.

[78]  Emily Bell,et al.  Striatal topography of probability and magnitude information for decisions under uncertainty , 2012, NeuroImage.

[79]  Edmund S. Phelps SECOND-BEST NATIONAL SAVING AND GAME-EQUILIBRIUM GROWTH , 1980 .

[80]  M. Walton,et al.  Separate neural pathways process different decision costs , 2006, Nature Neuroscience.

[81]  A. Kacelnik,et al.  To walk or to fly? How birds choose among foraging modes. , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[82]  J. E. Mazur An adjusting procedure for studying delayed reinforcement. , 1987 .

[83]  W. Schultz,et al.  Short-Term Temporal Discounting of Reward Value in Human Ventral Striatum , 2009, Journal of neurophysiology.

[84]  C. Büchel,et al.  Overlapping and Distinct Neural Systems Code for Subjective Value during Intertemporal and Risky Decision Making , 2009, The Journal of Neuroscience.

[85]  Gretchen B. Chapman,et al.  The magnitude effect: Temporal discount rates and restaurant tips , 1998 .

[86]  P. Samuelson A Note on Measurement of Utility , 1937 .

[87]  G. Pagnoni,et al.  Neurobiological Substrates of Dread , 2006, Science.

[88]  Samuel M. McClure,et al.  Time Discounting for Primary Rewards , 2007, The Journal of Neuroscience.

[89]  M. Walton,et al.  Calculating utility: preclinical evidence for cost–benefit analysis by mesolimbic dopamine , 2007, Psychopharmacology.

[90]  R. Thaler,et al.  Anomalies: Risk Aversion , 2001 .

[91]  G. Chapman,et al.  Time preferences for the very long term. , 2001, Acta psychologica.

[92]  G. Loewenstein,et al.  Time Discounting and Time Preference: A Critical Review , 2002 .

[93]  T. Koopmans Stationary Ordinal Utility and Impatience , 1960 .

[94]  D. Kahneman,et al.  Back to Bentham? Explorations of experience utility , 1997 .

[95]  N. Chater,et al.  Choosing to Make an Effort: The Role of Striatum in Signaling Physical Effort of a Chosen Action , 2010, Journal of neurophysiology.

[96]  D. Cross,et al.  Subjective probability and delay. , 1991, Journal of the experimental analysis of behavior.

[97]  J. Salamone,et al.  Effort-related functions of nucleus accumbens dopamine and associated forebrain circuits , 2007, Psychopharmacology.

[98]  G. Berns,et al.  Three studies on the neuroeconomics of decision-making when payoffs are real and negative. , 2008, Advances in health economics and health services research.

[99]  Soyoung Q. Park,et al.  Neurobiology of Value Integration: When Value Impacts Valuation , 2011, The Journal of Neuroscience.

[100]  K. Berman,et al.  Cerebral Cortex doi:10.1093/cercor/bhj004 Neural Coding of Distinct Statistical Properties of Reward Information in Humans , 2005 .

[101]  K. Fernow New York , 1896, American Potato Journal.

[102]  M. Walton,et al.  The Role of Rat Medial Frontal Cortex in Effort-Based Decision Making , 2002, The Journal of Neuroscience.

[103]  Saori C. Tanaka,et al.  Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops , 2004, Nature Neuroscience.

[104]  Colin Camerer,et al.  A framework for studying the neurobiology of value-based decision making , 2008, Nature Reviews Neuroscience.

[106]  Marjorie K. Shelley Outcome signs, question frames and discount rates , 1993 .

[107]  Daniel D. Holt,et al.  Differential effects of amount on temporal and probability discounting of gains and losses , 2006, Memory & cognition.

[108]  R. Herrnstein,et al.  Preference reversal and delayed reinforcement , 1981 .

[109]  R. Dolan,et al.  Dopamine and Effort-Based Decision Making , 2011, Front. Neurosci..

[110]  Brian Knutson,et al.  Dissociable neural representations of future reward magnitude and delay during temporal discounting , 2009, NeuroImage.

[111]  Kenji Doya,et al.  Humans Can Adopt Optimal Discounting Strategy under Real-Time Constraints , 2006, PLoS Comput. Biol..

[112]  Saori C. Tanaka,et al.  Serotonin Differentially Regulates Short- and Long-Term Prediction of Rewards in the Ventral and Dorsal Striatum , 2007, PloS one.

[113]  G. Chapman Temporal discounting and utility for health and money. , 1996, Journal of experimental psychology. Learning, memory, and cognition.

[114]  M. Allais Le comportement de l'homme rationnel devant le risque : critique des postulats et axiomes de l'ecole americaine , 1953 .

[115]  R. Pollak,et al.  SECOND-BEST NATIONAL SAVING AND GAME-EQUILIBRIUM GROWTH , 1980 .

[116]  Colin Camerer,et al.  Neural Response to Reward Anticipation under Risk Is Nonlinear in Probabilities , 2009, The Journal of Neuroscience.

[117]  T. Robbins,et al.  Behavioral models of impulsivity in relation to ADHD: Translation between clinical and preclinical studies , 2006, Clinical psychology review.

[118]  M. Commons The effect of delay and of intervening events on reinforcement value , 2013 .

[119]  David E. Bell,et al.  Regret in Decision Making under Uncertainty , 1982, Oper. Res..

[120]  Daeyeol Lee,et al.  Neural Dissociation of Delay and Uncertainty in Intertemporal Choice , 2008, The Journal of Neuroscience.

[121]  R. Grace,et al.  The Matching Law And Amount-dependent Exponential Discounting As Accounts Of Self-control Choice. , 1999, Journal of the experimental analysis of behavior.

[122]  Paola Manzini,et al.  Choice Over Time , 2007, The Handbook of Rational and Social Choice.

[123]  S. Floresco,et al.  Fundamental Contribution by the Basolateral Amygdala to Different Forms of Decision Making , 2009, The Journal of Neuroscience.

[124]  P. Dayan How to set the switches on this thing , 2012, Current Opinion in Neurobiology.

[125]  Sara Kim,et al.  From Inherent Value to Incentive Value: When and Why Pointless Effort Enhances Consumer Preference , 2011 .

[126]  J. O'Doherty,et al.  Orbitofrontal Cortex Encodes Willingness to Pay in Everyday Economic Transactions , 2007, The Journal of Neuroscience.

[127]  H. Rachlin,et al.  The effect of temporal constraints on the value of money and other commodities , 1993 .

[128]  N. Logothetis,et al.  Neurophysiological investigation of the basis of the fMRI signal , 2001, Nature.

[129]  G. Chapman,et al.  Decision biases in intertemporal choice and choice under uncertainty: Testing a common account , 2006, Memory & cognition.

[130]  D A Redelmeier,et al.  Time Preference in Medical Decision Making and Cost - Effectiveness Analysis , 1993, Medical decision making : an international journal of the Society for Medical Decision Making.

[131]  C. Winstanley,et al.  Sensitivity to Cognitive Effort Mediates Psychostimulant Effects on a Novel Rodent Cost/Benefit Decision-Making Task , 2012, Neuropsychopharmacology.

[132]  Matthew W. Johnson,et al.  Delay discounting in current and never-before cigarette smokers: similarities and differences across commodity, sign, and magnitude. , 2003, Journal of abnormal psychology.

[133]  A. Hariri,et al.  Preference for Immediate over Delayed Rewards Is Associated with Magnitude of Ventral Striatal Activity , 2006, The Journal of Neuroscience.

[134]  Joseph T. McGuire,et al.  Decision making and the avoidance of cognitive demand. , 2010, Journal of experimental psychology. General.

[135]  J. Elster,et al.  Choice Over Time , 1992 .

[136]  S Kheramin,et al.  Role of the orbital prefrontal cortex in choice between delayed and uncertain reinforcers: a quantitative analysis , 2003, Behavioural Processes.

[137]  Ken-ichi Amemori,et al.  Localized Microstimulation of Primate Pregenual Cingulate Cortex Induces Negative Decision-Making , 2012, Nature Neuroscience.

[138]  E. Rowland Theory of Games and Economic Behavior , 1946, Nature.

[139]  H. de Wit,et al.  Determination of discount functions in rats with an adjusting-amount procedure. , 1997, Journal of the experimental analysis of behavior.

[140]  Donald A. Hantula,et al.  A job with a future? Delay discounting, magnitude effects, and domain independence of utility for career decisions. , 2003 .

[141]  Colin Camerer,et al.  Dissociating the Role of the Orbitofrontal Cortex and the Striatum in the Computation of Goal Values and Prediction Errors , 2008, The Journal of Neuroscience.

[142]  Ran Kivetz The Effects of Effort and Intrinsic Motivation on Risky Choice , 2003 .

[143]  Brian Knutson,et al.  Dissociation of reward anticipation and outcome with event-related fMRI , 2001, Neuroreport.

[144]  D. Kahneman,et al.  Functional Imaging of Neural Responses to Expectancy and Experience of Monetary Gains and Losses tasks with monetary payoffs , 2001 .

[145]  M. El-Sabaawi Breakdown of Will , 2002 .

[146]  D. Soman The effect of time delay on multi-attribute choice , 2004 .

[147]  P. Dayan,et al.  How Humans Integrate the Prospects of Pain and Reward during Choice , 2009, The Journal of Neuroscience.

[148]  L. Green,et al.  A discounting framework for choice with delayed and probabilistic rewards. , 2004, Psychological bulletin.