Context-Dependent Probability Estimation and its Neurocomputational Substrates

Many decisions rely on how we evaluate potential outcomes associated with the options under consideration and estimate their corresponding probabilities of occurrence. Outcome valuation is subjective as it requires consulting internal preferences and is sensitive to context. In contrast, probability estimation requires extracting statistics from the environment and therefore imposes unique challenges to the decision maker. Here we show that probability estimation, like outcome valuation, is subject to context effects that bias probability estimates away from other stimuli present in the same context. However, unlike valuation, these context effects appeared to be scaled by estimated uncertainty, which is largest at intermediate probabilities. BOLD imaging showed that patterns of multivoxel activity in dorsal anterior cingulate cortex (dACC) and ventromedial prefrontal cortex (VMPFC) predicted individual differences in context effects on probability estimate. These results establish VMPFC as the neurocomputational substrate shared between valuation and probability estimation and highlight the additional involvement of dACC that can be uniquely attributed to probability estimation. As probability estimation is a required component of computational accounts from sensory inference to higher cognition, the context effects found here may affect a wide array of cognitive computations. Highlights Context impacts subjective estimates on reward probability – Stimuli carrying greater variance are more strongly affected by other stimuli present in the same context This phenomenon can be explained by reference-dependent computations that are gated by reward variance Multivoxel patterns of dACC and VMPFC activity predicts individual differences in context effect on probability estimate

[1]  Mimi Liljeholm,et al.  Neural Correlates of Instrumental Contingency Learning: Differential Effects of Action–Reward Conjunction and Disjunction , 2011, The Journal of Neuroscience.

[2]  T. Belke Stimulus preference and the transitivity of preference , 1992 .

[3]  Richard Gonzalez,et al.  Curvature of the Probability Weighting Function , 1996 .

[4]  A. Pouget,et al.  Probabilistic brains: knowns and unknowns , 2013, Nature Neuroscience.

[5]  M. Rabin,et al.  A Model of Reference-Dependent Preferences , 2006 .

[6]  Colin Camerer,et al.  Neural Response to Reward Anticipation under Risk Is Nonlinear in Probabilities , 2009, The Journal of Neuroscience.

[7]  P. Glimcher,et al.  The Neurobiology of Decision: Consensus and Controversy , 2009, Neuron.

[8]  Joseph W. Kable,et al.  The valuation system: A coordinate-based meta-analysis of BOLD fMRI experiments examining neural correlates of subjective value , 2013, NeuroImage.

[9]  Samuel M. McClure,et al.  Temporal Prediction Errors in a Passive Learning Task Activate Human Striatum , 2003, Neuron.

[10]  C. Plott,et al.  Economic Theory of Choice and the Preference Reversal Phenomenon , 1979 .

[11]  Michael J. Frank,et al.  Faculty Opinions recommendation of States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. , 2010 .

[12]  S. Haber,et al.  Primate cingulostriatal projection: Limbic striatal versus sensorimotor striatal input , 1994, The Journal of comparative neurology.

[13]  N. Chater,et al.  Are Probabilities Overweighted or Underweighted When Rare Outcomes Are Experienced (Rarely)? , 2009, Psychological science.

[14]  J. Haynes,et al.  Predicting Subjective Affective Salience from Cortical Responses to Invisible Object Stimuli. , 2016, Cerebral cortex.

[15]  Henrik Walter,et al.  Prediction error as a linear function of reward probability is coded in human nucleus accumbens , 2006, NeuroImage.

[16]  P. Glimcher,et al.  Reward Value-Based Gain Control: Divisive Normalization in Parietal Cortex , 2011, The Journal of Neuroscience.

[17]  W. Schultz,et al.  Adaptation of Reward Sensitivity in Orbitofrontal Neurons , 2010, The Journal of Neuroscience.

[18]  P. Glimcher,et al.  Free choice shapes normalized value signals in medial orbitofrontal cortex , 2018, Nature Communications.

[19]  M. Rabin,et al.  Reference-Dependent Risk Attitudes , 2007 .

[20]  A. Tversky Intransitivity of preferences. , 1969 .

[21]  P. Slovic,et al.  Reversals of preference between bids and choices in gambling decisions. , 1971 .

[22]  M. Khamassi,et al.  Contextual modulation of value signals in reward and punishment learning , 2015, Nature Communications.

[23]  George I. Christopoulos,et al.  Neuronal Distortions of Reward Probability without Choice , 2008, The Journal of Neuroscience.

[24]  G. Davies,et al.  Knowns and Unknowns , 2003 .

[25]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[26]  M. Lepper,et al.  The Construction of Preference: When Choice Is Demotivating: Can One Desire Too Much of a Good Thing? , 2006 .

[27]  Alex Kacelnik,et al.  Context-dependent utility overrides absolute memory as a determinant of choice , 2009, Proceedings of the National Academy of Sciences.

[28]  R. Elliott,et al.  Medial orbitofrontal cortex codes relative rather than absolute value of financial rewards in humans , 2008, The European journal of neuroscience.

[29]  J. O'Doherty,et al.  Human Neural Learning Depends on Reward Prediction Errors in the Blocking Paradigm , 2005, Journal of neurophysiology.

[30]  J. Oller,et al.  Consensus and controversy , 1984 .

[31]  Jan Zimmermann,et al.  Multiple timescales of normalized value coding underlie adaptive choice behavior , 2018, Nature Communications.

[32]  A. Rangel,et al.  Informatic parcellation of the network involved in the computation of subjective value. , 2014, Social cognitive and affective neuroscience.

[33]  J. Lygeros,et al.  Decision Making I , 2014 .

[34]  Mel W. Khaw,et al.  Normalized value coding explains dynamic adaptation in the human valuation process , 2017, Proceedings of the National Academy of Sciences.

[35]  R. Hertwig,et al.  The description–experience gap in risky choice , 2009, Trends in Cognitive Sciences.

[36]  Timothy E. J. Behrens,et al.  Choice, uncertainty and value in prefrontal and cingulate cortex , 2008, Nature Neuroscience.

[37]  Mark W. Woolrich,et al.  Advances in functional and structural MR image analysis and implementation as FSL , 2004, NeuroImage.

[38]  J. Kable,et al.  BOLD Subjective Value Signals Exhibit Robust Range Adaptation , 2014, The Journal of Neuroscience.

[39]  E. Rowland Theory of Games and Economic Behavior , 1946, Nature.

[40]  C. Gallistel The Replacement of General-Purpose Learning Models with Adaptively Specialized Learning Modules , 2000 .

[41]  Alexander Strashny Asymmetric loss utility: an analysis of decision under risk , 2004 .

[42]  C. Padoa-Schioppa Range-Adapting Representation of Economic Value in the Orbitofrontal Cortex , 2009, The Journal of Neuroscience.

[43]  W. Schultz,et al.  Relative reward preference in primate orbitofrontal cortex , 1999, Nature.

[44]  A. Tversky,et al.  A Belief-Based Account of Decision Under Uncertainty , 1998 .

[45]  A. Tversky,et al.  Context-dependent preferences , 1993 .

[46]  Elizabeth Tricomi,et al.  Value and probability coding in a feedback-based learning task utilizing food rewards. , 2015, Journal of neurophysiology.

[47]  Charles Kemp,et al.  How to Grow a Mind: Statistics, Structure, and Abstraction , 2011, Science.

[48]  R. Poldrack,et al.  Ventral–striatal/nucleus–accumbens sensitivity to prediction errors during classification learning , 2006, Human brain mapping.

[49]  Keith M. Marzilli Ericson,et al.  Expectations as Endowments: Evidence on Reference-Dependent Preferences from Exchange and Valuation Experiments , 2010 .

[50]  Mel W. Khaw,et al.  Normalization is a general neural mechanism for context-dependent decision making , 2013, Proceedings of the National Academy of Sciences.

[51]  A. Tversky,et al.  Prospect theory: an analysis of decision under risk — Source link , 2007 .

[52]  D. Kumaran,et al.  The Neurobiology of Reference-Dependent Value Computation , 2009, NeuroImage.

[53]  P. Dayan,et al.  States versus Rewards: Dissociable Neural Prediction Error Signals Underlying Model-Based and Model-Free Reinforcement Learning , 2010, Neuron.

[54]  Rainer Goebel,et al.  Information-based functional brain mapping. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[55]  David Gill,et al.  Nuffield Centre for Experimental Social Sciences Discussion Paper Series a Structural Analysis of Disappointment Aversion in a Real Effort Competition a Structural Analysis of Disappointment Aversion in a Real Effort Competition , 2022 .

[56]  David Huffman,et al.  Reference Points and Effort Provision , 2009, SSRN Electronic Journal.

[57]  A. Tversky,et al.  Prospect theory: analysis of decision under risk , 1979 .

[58]  A. Tversky,et al.  Advances in prospect theory: Cumulative representation of uncertainty , 1992 .

[59]  L. Maloney,et al.  Economic decision-making compared with an equivalent motor task , 2009, Proceedings of the National Academy of Sciences.

[60]  Timothy E. J. Behrens,et al.  Learning the value of information in an uncertain world , 2007, Nature Neuroscience.

[61]  D G Pelli,et al.  The VideoToolbox software for visual psychophysics: transforming numbers into movies. , 1997, Spatial vision.

[62]  Richard Gonzalez,et al.  On the Shape of the Probability Weighting Function , 1999, Cognitive Psychology.

[63]  Nils Kolling,et al.  Predictive decision making driven by multiple time-linked reward representations in the anterior cingulate cortex , 2016, Nature Communications.

[64]  Marco K. Wittmann,et al.  Multiple signals in anterior cingulate cortex , 2016, Current Opinion in Neurobiology.

[65]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[66]  D. Bernoulli Exposition of a New Theory on the Measurement of Risk , 1954 .

[67]  Colin Camerer,et al.  Dissociating the Role of the Orbitofrontal Cortex and the Striatum in the Computation of Goal Values and Prediction Errors , 2008, The Journal of Neuroscience.

[68]  Adam Binch,et al.  Perception as Bayesian Inference , 2014 .

[69]  Wilma Otten,et al.  Context effects in the measurement of optimism in probability judgment , 1996 .

[70]  J. O'Doherty,et al.  Reward Value Coding Distinct From Risk Attitude-Related Uncertainty Coding in Human Reward Systems , 2006, Journal of neurophysiology.

[71]  Karl J. Friston,et al.  Temporal Difference Models and Reward-Related Learning in the Human Brain , 2003, Neuron.

[72]  George I. Christopoulos,et al.  Neural Correlates of Value, Risk, and Risk Aversion Contributing to Decision Making under Risk , 2009, The Journal of Neuroscience.

[73]  Takashi Kitamura,et al.  The role of engram cells in the systems consolidation of memory , 2018, Nature Reviews Neuroscience.

[74]  Paul Slovic,et al.  Response-induced reversals of preference in gambling: An extended replication in las vegas , 1973 .

[75]  M. Frank,et al.  From reinforcement learning models to psychiatric and neurological disorders , 2011, Nature Neuroscience.

[76]  S. T. Buckland,et al.  An Introduction to the Bootstrap. , 1994 .

[77]  L. Maloney,et al.  The Neural Correlates of Subjective Utility of Monetary Outcome and Probability Weight in Economic and in Motor Decision under Risk , 2011, The Journal of Neuroscience.

[78]  M. Kenward,et al.  An Introduction to the Bootstrap , 2007 .

[79]  Timothy E. J. Behrens,et al.  Optimal decision making and the anterior cingulate cortex , 2006, Nature Neuroscience.

[80]  Christopher P. Puto,et al.  Adding Asymmetrically Dominated Alternatives: Violations of Regularity & the Similarity Hypothesis. , 1981 .

[81]  Peter K. Schott,et al.  How Basic Are Behavioral Biases? Evidence from Capuchin Monkey Trading Behavior , 2006, Journal of Political Economy.

[82]  E. Procyk,et al.  Expectations, gains, and losses in the anterior cingulate cortex , 2007, Cognitive, affective & behavioral neuroscience.