Computing Value from Quality and Quantity in Human Decision-Making

How organisms learn the value of single stimuli through experience is well described. In many decisions, however, value estimates are computed “on the fly” by combining multiple stimulus attributes. The neural basis of this computation is poorly understood. Here we explore a common scenario in which decision-makers must combine information about quality and quantity to determine the best option. Using fMRI, we examined the neural representation of quality, quantity, and their integration into an integrated subjective value signal in humans of both genders. We found that activity within inferior frontal gyrus (IFG) correlated with offer quality, while activity in the intraparietal sulcus (IPS) specifically correlated with offer quantity. Several brain regions, including the anterior cingulate cortex (ACC), were sensitive to an interaction of quality and quantity. However, the ACC was uniquely activated by quality, quantity, and their interaction, suggesting that this region provides a substrate for flexible computation of value from both quality and quantity. Furthermore, ACC signals across subjects correlated with the strength of quality and quantity signals in IFG and IPS, respectively. ACC tracking of subjective value also correlated with choice predictability. Finally, activity in the ACC was elevated for choice trials, suggesting that ACC provides a nexus for the computation of subjective value in multiattribute decision-making. SIGNIFICANCE STATEMENT Would you prefer three apples or two oranges? Many choices we make each day require us to weigh up the quality and quantity of different outcomes. Using fMRI, we show that option quality is selectively represented in the inferior frontal gyrus, while option quantity correlates with areas of the intraparietal sulcus that have previously been associated with numerical processing. We show that information about the two is integrated into a value signal in the anterior cingulate cortex, and the fidelity of this integration predicts choice predictability. Our results demonstrate how on-the-fly value estimates are computed from multiple attributes in human value-based decision-making.

[1]  Karl J. Friston,et al.  Multisubject fMRI Studies and Conjunction Analyses , 1999, NeuroImage.

[2]  Rony Paz,et al.  Theta synchronizes the activity of medial prefrontal neurons during learning. , 2008, Learning & memory.

[3]  G. Loewenstein,et al.  Neural Predictors of Purchases , 2007, Neuron.

[4]  M. Degroot,et al.  Measuring utility by a single-response sequential method. , 1964, Behavioral science.

[5]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[6]  Giorgio Vallortigara,et al.  Arithmetic in newborn chicks , 2009, Proceedings of the Royal Society B: Biological Sciences.

[7]  David J. Freedman,et al.  Representation of the Quantity of Visual Items in the Primate Prefrontal Cortex , 2002, Science.

[8]  Nik Fadzly,et al.  The Numerical Competency of Two Bird Species (Corvus splendens and Acridotheres tristis). , 2014, Tropical life sciences research.

[9]  Daniel L Schacter,et al.  Ventromedial prefrontal cortex supports affective future simulation by integrating distributed knowledge , 2014, Proceedings of the National Academy of Sciences.

[10]  C. Padoa-Schioppa,et al.  The representation of economic value in the orbitofrontal cortex is invariant for changes of menu , 2008, Nature Neuroscience.

[11]  Andreas Nieder,et al.  A parieto-frontal network for visual numerical information in the monkey. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[12]  R. Dolan,et al.  Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans , 2006, Nature.

[13]  G. Schoenbaum,et al.  Lateral orbitofrontal neurons acquire responses to upshifted, downshifted, or blocked cues during unblocking , 2015, eLife.

[14]  Dylan A. Simon,et al.  Model-based choices involve prospective neural activity , 2015, Nature Neuroscience.

[15]  Daniel C. McNamee,et al.  Category-dependent and category-independent goal-value codes in human ventromedial prefrontal cortex , 2013, Nature Neuroscience.

[16]  C. Padoa-Schioppa,et al.  Dialogue on economic choice, learning theory, and neuronal representations , 2015, Current Opinion in Behavioral Sciences.

[17]  M. Rushworth,et al.  Distinct Roles of Three Frontal Cortical Areas in Reward-Guided Behavior , 2011, The Journal of Neuroscience.

[18]  Zeb Kurth-Nelson,et al.  Learning-Induced Plasticity in Medial Prefrontal Cortex Predicts Preference Malleability , 2015, Neuron.

[19]  J. Kalaska,et al.  Neural mechanisms for interacting with a world full of action choices. , 2010, Annual review of neuroscience.

[20]  Mimi Liljeholm,et al.  Neural Correlates of Instrumental Contingency Learning: Differential Effects of Action–Reward Conjunction and Disjunction , 2011, The Journal of Neuroscience.

[21]  Andreas Nieder,et al.  Neuronal Representation of Numerosity Zero in the Primate Parieto-Frontal Number Network , 2016, Current Biology.

[22]  Oliver Speck,et al.  The impact of physiological noise correction on fMRI at 7 T , 2011, NeuroImage.

[23]  H. Seo,et al.  Temporal Filtering of Reward Signals in the Dorsal Anterior Cingulate Cortex during a Mixed-Strategy Game , 2007, The Journal of Neuroscience.

[24]  Marco K. Wittmann,et al.  Multiple signals in anterior cingulate cortex , 2016, Current Opinion in Neurobiology.

[25]  M. Woolrich,et al.  Mechanisms underlying cortical activity during value-guided choice , 2011, Nature Neuroscience.

[26]  D. Yves von Cramon,et al.  The neural implementation of multi-attribute decision making: A parametric fMRI study with human subjects , 2006, NeuroImage.

[27]  Timothy Edward John Behrens,et al.  Segregated Encoding of Reward–Identity and Stimulus–Reward Associations in Human Orbitofrontal Cortex , 2013, The Journal of Neuroscience.

[28]  G. Schoenbaum,et al.  What the orbitofrontal cortex does not do , 2015, Nature Neuroscience.

[29]  B. Richmond,et al.  Neural Mechanisms Underlying Contextual Dependency of Subjective Values: Converging Evidence from Monkeys and Humans , 2015, The Journal of Neuroscience.

[30]  Nils Kolling,et al.  A neural mechanism underlying failure of optimal choice with multiple alternatives , 2014, Nature Neuroscience.

[31]  Geoffrey Schoenbaum,et al.  Neural Estimates of Imagined Outcomes in Basolateral Amygdala Depend on Orbitofrontal Cortex , 2015, The Journal of Neuroscience.

[32]  J. O'Doherty,et al.  The problem with value , 2014, Neuroscience & Biobehavioral Reviews.

[33]  Xiao-Jing Wang.  Decision Making in Recurrent Neuronal Circuits , 2008, Neuron.

[34]  Cathy J. Price,et al.  A review and synthesis of the first 20 years of PET and fMRI studies of heard speech, spoken language and reading , 2012, NeuroImage.

[35]  Timothy E. J. Behrens,et al.  Hierarchical competitions subserving multi-attribute choice , 2014, Nature Neuroscience.

[36]  P. Dayan,et al.  Mapping value based planning and extensively trained choice in the human brain , 2012, Nature Neuroscience.

[37]  K. Grill-Spector,et al.  Repetition and the brain: neural models of stimulus-specific effects , 2006, Trends in Cognitive Sciences.

[38]  P. Glimcher,et al.  Reward Value-Based Gain Control: Divisive Normalization in Parietal Cortex , 2011, The Journal of Neuroscience.

[39]  Thomas H. B. FitzGerald,et al.  The Role of Human Orbitofrontal Cortex in Value Comparison for Incommensurable Objects , 2009, The Journal of Neuroscience.

[40]  Geoffrey Schoenbaum,et al.  Neural Estimates of Imagined Outcomes in the Orbitofrontal Cortex Drive Behavior and Learning , 2013, Neuron.

[41]  Timothy Edward John Behrens,et al.  Two Anatomically and Computationally Distinct Learning Signals Predict Changes to Stimulus-Outcome Associations in Hippocampus , 2016, Neuron.

[42]  Andreas Nieder,et al.  Notation-Independent Representation of Fractions in the Human Parietal Cortex , 2009, The Journal of Neuroscience.

[43]  Timothy E. J. Behrens,et al.  Dissociable effects of surprise and model update in parietal and anterior cingulate cortex , 2013, Proceedings of the National Academy of Sciences.

[44]  B. Hayden,et al.  Neurons in Anterior Cingulate Cortex Multiplex Information about Reward and Action , 2010, The Journal of Neuroscience.

[45]  V. Michel,et al.  An Automatic Valuation System in the Human Brain: Evidence from Functional Neuroimaging , 2009, Neuron.

[46]  P. Glimcher,et al.  Choice from Non-Choice: Predicting Consumer Preferences from Blood Oxygenation Level-Dependent Signals Obtained during Passive Viewing , 2011, The Journal of Neuroscience.

[47]  Philippe Pinel,et al.  Distributed and Overlapping Cerebral Representations of Number, Size, and Luminance during Comparative Judgments , 2004, Neuron.

[48]  Colin Camerer,et al.  Transformation of stimulus value signals into motor commands during simple choice , 2011, Proceedings of the National Academy of Sciences.

[49]  Soyoung Q. Park,et al.  Decoding the Formation of Reward Predictions across Learning , 2011, The Journal of Neuroscience.

[50]  J. O'Doherty,et al.  Orbitofrontal Cortex Encodes Willingness to Pay in Everyday Economic Transactions , 2007, The Journal of Neuroscience.

[51]  Nikolaus Weiskopf,et al.  Optimal EPI parameters for reduction of susceptibility-induced BOLD sensitivity losses: A whole-brain analysis at 3 T and 1.5 T , 2006, NeuroImage.

[52]  Timothy E. J. Behrens,et al.  Neural Mechanisms of Foraging , 2012, Science.

[53]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[54]  Timothy Edward John Behrens,et al.  Ventromedial Prefrontal and Anterior Cingulate Cortex Adopt Choice and Default Reference Frames during Sequential Multi-Alternative Choice , 2013, The Journal of Neuroscience.

[55]  Camillo Padoa-Schioppa,et al.  Neuronal Remapping and Circuit Persistence in Economic Decisions , 2016, Nature Neuroscience.

[56]  R. Dolan,et al.  Confidence in value-based choice , 2012, Nature Neuroscience.

[57]  Timothy E. J. Behrens,et al.  Learning the value of information in an uncertain world , 2007, Nature Neuroscience.

[58]  Z. Kurth-Nelson,et al.  Anterior Cingulate Cortex Instigates Adaptive Switches in Choice by Integrating Immediate and Delayed Components of Value in Ventromedial Prefrontal Cortex , 2014, The Journal of Neuroscience.

[59]  P. Glimcher,et al.  Testing the Reward Prediction Error Hypothesis with an Axiomatic Model , 2010, The Journal of Neuroscience.

[60]  Mark W. Woolrich,et al.  Trial-Type Dependent Frames of Reference for Value Comparison , 2013, PLoS Comput. Biol..

[61]  Philippe Pinel,et al.  Tuning Curves for Approximate Numerosity in the Human Intraparietal Sulcus , 2004, Neuron.

[62]  John M. Pearson,et al.  Fictive Reward Signals in the Anterior Cingulate Cortex , 2009, Science.

[63]  Timothy E. J. Behrens,et al.  Online evaluation of novel choices by simultaneous representation of multiple memories , 2013, Nature Neuroscience.

[64]  Todd A. Hare,et al.  Automatic versus Choice-Dependent Value Representations in the Human Brain , 2015, Neuron.

[65]  Marco K. Wittmann,et al.  Multiple Neural Mechanisms of Decision Making and Their Competition under Changing Risk Pressure , 2014, Neuron.

[66]  R Turner,et al.  Optimisation of the 3D MDEFT sequence for anatomical brain imaging: technical implications at 1.5 and 3 T , 2004, NeuroImage.

[67]  Scott A. Huettel,et al.  Local pattern classification differentiates processes of economic valuation , 2009, NeuroImage.

[68]  W. Schultz,et al.  Dopamine Modulates the Neural Representation of Subjective Value of Food in Hungry Subjects , 2014, The Journal of Neuroscience.

[69]  Timothy Edward John Behrens,et al.  Dissociable contributions of ventromedial prefrontal and posterior parietal cortex to value-guided choice , 2014, NeuroImage.

[70]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[71]  Tommy C. Blanchard,et al.  Reward Value Comparison via Mutual Inhibition in Ventromedial Prefrontal Cortex , 2014, Neuron.

[72]  G. Schoenbaum,et al.  Orbitofrontal neurons infer the value and identity of predicted outcomes , 2014, Nature Communications.

[73]  B. P. Klein,et al.  Topographic Representation of Numerosity in the Human Parietal Cortex , 2013, Science.

[74]  Ian Krajbich,et al.  Visual fixations and the computation and comparison of value in simple choice , 2010, Nature Neuroscience.

[75]  C. Summerfield,et al.  Neural repetition suppression reflects fulfilled perceptual expectations , 2008, Nature Neuroscience.

[76]  C. H. Donahue,et al.  Metaplasticity as a Neural Substrate for Adaptive Learning and Choice under Uncertainty , 2017, Neuron.

[77]  Timothy E. J. Behrens,et al.  Choice, uncertainty and value in prefrontal and cingulate cortex , 2008, Nature Neuroscience.

[78]  P. Dayan,et al.  Goals and Habits in the Brain , 2013, Neuron.

[79]  C. Packer,et al.  Roaring and numerical assessment in contests between groups of female lions, Panthera leo , 1994, Animal Behaviour.

[80]  A. Dale,et al.  Dorsal anterior cingulate cortex: A role in reward-based decision making , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[81]  C. Padoa-Schioppa,et al.  Neurons in the orbitofrontal cortex encode economic value , 2006, Nature.

[82]  Daniel Ansari,et al.  Age-related Changes in the Activation of the Intraparietal Sulcus during Nonsymbolic Magnitude Processing: An Event-related Functional Magnetic Resonance Imaging Study , 2006, Journal of Cognitive Neuroscience.

[83]  P. Montague,et al.  Neural Economics and the Biological Substrates of Valuation , 2002, Neuron.

[84]  M. Rushworth,et al.  Connectivity-Based Parcellation of Human Cingulate Cortex and Its Relation to Functional Specialization , 2009, The Journal of Neuroscience.

[85]  A. Nieder.  The neuronal code for number , 2016, Nature Reviews Neuroscience.

[86]  T. Robbins,et al.  Choosing between Small, Likely Rewards and Large, Unlikely Rewards Activates Inferior and Orbital Prefrontal Cortex , 1999, The Journal of Neuroscience.

[87]  Timothy E. J. Behrens,et al.  Capturing the temporal evolution of choice across prefrontal cortex , 2015, eLife.

[88]  William A. Cunningham,et al.  The Value of Numbers in Economic Rewards , 2014, Psychological science.

[89]  Dhanistha Panyasak,et al.  Circuits , 1995, Annals of the New York Academy of Sciences.

[90]  John Ashburner,et al.  A fast diffeomorphic image registration algorithm , 2007, NeuroImage.

[91]  Robert C. Wilson,et al.  Orbitofrontal Cortex as a Cognitive Map of Task Space , 2014, Neuron.

[92]  Antonio Rangel,et al.  Stimulus Value Signals in Ventromedial PFC Reflect the Integration of Attribute Value Signals Computed in Fusiform Gyrus and Posterior Superior Temporal Gyrus , 2013, The Journal of Neuroscience.

[93]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[94]  P. Tobler,et al.  Identity-specific coding of future rewards in the human orbitofrontal cortex , 2015, Proceedings of the National Academy of Sciences.

[95]  P. Glimcher,et al.  The neural correlates of subjective value during intertemporal choice , 2007, Nature Neuroscience.

[96]  C. Padoa-Schioppa,et al.  Neuronal Encoding of Subjective Value in Dorsal and Ventral Anterior Cingulate Cortex , 2012, The Journal of Neuroscience.

[97]  H. Wilson,et al.  fMRI evidence for the neural representation of faces , 2005, Nature Neuroscience.

[98]  Timothy E. J. Behrens,et al.  Double dissociation of value computations in orbitofrontal and anterior cingulate neurons , 2011, Nature Neuroscience.

[99]  E. Rolls,et al.  Abstract reward and punishment representations in the human orbitofrontal cortex , 2001, Nature Neuroscience.

[100]  S. Dehaene,et al.  A Magnitude Code Common to Numerosities and Number Symbols in Human Intraparietal Cortex , 2007, Neuron.

[101]  Christian C. Ruff,et al.  Neural Oscillations and Synchronization Differentially Support Evidence Accumulation in Perceptual and Value-Based Decision Making , 2014, Neuron.

[102]  Brian Knutson,et al.  Dissociable neural representations of future reward magnitude and delay during temporal discounting , 2009, NeuroImage.

[103]  Helen C. Barron,et al.  Repetition suppression: a means to index neural representations using BOLD? , 2016, Philosophical Transactions of the Royal Society B: Biological Sciences.

[104]  Nicolas W. Schuck,et al.  Human Orbitofrontal Cortex Represents a Cognitive Map of State Space , 2016, Neuron.

[105]  Mihir A. Topiwala,et al.  Synchronized Activity between the Ventral Hippocampus and the Medial Prefrontal Cortex during Anxiety , 2010, Neuron.

[106]  Demis Hassabis,et al.  The construction system of the brain , 2009, Philosophical Transactions of the Royal Society B: Biological Sciences.

[107]  John M. Pearson,et al.  Neuronal basis of sequential foraging decisions in a patchy environment , 2011, Nature Neuroscience.

[108]  C. Padoa-Schioppa.  Neurobiology of economic choice: a good-based model. , 2011, Annual review of neuroscience.