Reward-Related Activity in Ventral Striatum Is Action Contingent and Modulated by Behavioral Relevance

Multiple features of the environment are often imbued with motivational significance, and the relative importance of these can change across contexts. The ability to flexibly adjust evaluative processes so that currently important features of the environment alone drive behavior is critical to adaptive routines. We know relatively little about the neural mechanisms involved, including whether motivationally significant features are obligatorily evaluated or whether current relevance gates access to value-sensitive regions. We addressed these questions using functional magnetic resonance imaging data and a task design where human subjects had to choose whether to accept or reject an offer indicated by visual and auditory stimuli. By manipulating, on a trial-by-trial basis, which stimulus determined the value of the offer, we show choice activity in the ventral striatum solely reflects the value of the currently relevant stimulus, consistent with a model wherein behavioral relevance modulates the impact of sensory stimuli on value processing. Choice outcome signals in this same region covaried positively with wins on accept trials, and negatively with wins on reject trials, consistent with striatal activity at feedback reflecting correctness of response rather than reward processing per se. We conclude that ventral striatum activity during decision making is dynamically modulated by behavioral context, indexed here by task relevance and action selection.

[1]  W. Schultz,et al.  Neuronal activity in monkey ventral striatum related to the expectation of reward , 1992, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[2]  S. Kakade,et al.  Learning and selective attention , 2000, Nature Neuroscience.

[3]  Brian Knutson,et al.  Anticipation of Increasing Monetary Reward Selectively Recruits Nucleus Accumbens , 2001, The Journal of Neuroscience.

[4]  Robert Turner,et al.  Image Distortion Correction in fMRI: A Quantitative Evaluation , 2002, NeuroImage.

[5]  R Turner,et al.  Optimized EPI for fMRI studies of the orbitofrontal cortex , 2003, NeuroImage.

[6]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[7]  Karl J. Friston,et al.  Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning , 2004, Science.

[8]  G. Pagnoni,et al.  Human Striatal Responses to Monetary Reward Depend On Saliency , 2004, Neuron.

[9]  R. Ellis,et al.  Action priming by briefly presented objects. , 2004, Acta psychologica.

[10]  K. R. Ridderinkhof,et al.  The Role of the Medial Frontal Cortex in Cognitive Control , 2004, Science.

[11]  M. Delgado,et al.  Modulation of Caudate Activity by Action Contingency , 2004, Neuron.

[12]  Peter Dayan,et al.  Temporal difference models describe higher-order learning in humans , 2004, Nature.

[13]  F. Tong,et al.  Decoding the visual and subjective contents of the human brain , 2005, Nature Neuroscience.

[14]  Angela J. Yu,et al.  Uncertainty, Neuromodulation, and Attention , 2005, Neuron.

[15]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[16]  J. O'Doherty,et al.  Regret and its avoidance: a neuroimaging study of choice behavior , 2005, Nature Neuroscience.

[17]  Rajesh P. N. Rao,et al.  Bayesian inference and attentional modulation in the visual cortex , 2005, Neuroreport.

[18]  B. Balleine,et al.  Parallel incentive processing: an integrated view of amygdala function , 2006, Trends in Neurosciences.

[19]  Sean M. Polyn,et al.  Beyond mind-reading: multi-voxel pattern analysis of fMRI data , 2006, Trends in Cognitive Sciences.

[20]  John Ashburner,et al.  A fast diffeomorphic image registration algorithm , 2007, NeuroImage.

[21]  Nikolaus Kriegeskorte,et al.  Analyzing for information, not activation, to exploit high-resolution fMRI , 2007, NeuroImage.

[22]  Norbert Kathmann,et al.  Neural correlates of error awareness , 2007, NeuroImage.

[23]  Kevin McCabe,et al.  Neural signature of fictive learning signals in a sequential investment task , 2007, Proceedings of the National Academy of Sciences.

[24]  P. Glimcher,et al.  The neural correlates of subjective value during intertemporal choice , 2007, Nature Neuroscience.

[25]  Colin Camerer,et al.  A framework for studying the neurobiology of value-based decision making , 2008, Nature Reviews Neuroscience.

[26]  R. Dolan,et al.  Reward Facilitates Tactile Judgments and Modulates Hemodynamic Responses in Human Primary Somatosensory Cortex , 2008, The Journal of Neuroscience.

[27]  B. Balleine,et al.  The Neural Mechanisms Underlying the Influence of Pavlovian Cues on Human Decision Making , 2008, The Journal of Neuroscience.

[28]  Colin Camerer,et al.  Dissociating the Role of the Orbitofrontal Cortex and the Striatum in the Computation of Goal Values and Prediction Errors , 2008, The Journal of Neuroscience.

[29]  P. Dayan,et al.  Decision theory, reinforcement learning, and the brain , 2008, Cognitive, affective & behavioral neuroscience.

[30]  P. Dayan,et al.  Human Pavlovian–Instrumental Transfer , 2008, The Journal of Neuroscience.

[31]  R. Dolan,et al.  Subliminal Instrumental Conditioning Demonstrated in the Human Brain , 2008, Neuron.

[32]  Angela J. Yu,et al.  Dynamics of attentional selection under conflict: toward a rational Bayesian account. , 2009, Journal of experimental psychology. Human perception and performance.

[33]  Karl J. Friston,et al.  Reinforcement Learning or Active Inference? , 2009, PloS one.

[34]  Jung Hoon Sul,et al.  Role of Striatum in Updating Values of Chosen Actions , 2009, The Journal of Neuroscience.

[35]  R. Dolan,et al.  Influence of Dopaminergically Mediated Reward on Somatosensory Decision-Making , 2009, PLoS biology.

[36]  Nikolaus Kriegeskorte,et al.  Comparison of multivariate classifiers and response normalizations for pattern-information fMRI , 2010, NeuroImage.

[37]  P. Dayan,et al.  journal homepage: www.elsevier.com/locate/visres Selective Bayes: Attentional load and crowding , 2022 .

[38]  Karl J. Friston,et al.  Attention, Uncertainty, and Free-Energy , 2010, Front. Hum. Neurosci..

[39]  J. Driver,et al.  Rewarding Feedback After Correct Visual Discriminations Has Both General and Specific Influences on Visual Cortex , 2010, Journal of neurophysiology.

[40]  Jonathan D. Cohen,et al.  Learning to selectively attend , 2010 .

[41]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[42]  Oliver Speck,et al.  The impact of physiological noise correction on fMRI at 7 T , 2011, NeuroImage.

[43]  Timothy E. J. Behrens,et al.  Counterfactual Choice and Learning in a Neural Network Centered on Human Lateral Frontopolar Cortex , 2011, PLoS biology.

[44]  Timothy E. J. Behrens,et al.  Dissociable Reward and Timing Signals in Human Midbrain and Ventral Striatum , 2011, Neuron.

[45]  P. Dayan,et al.  Behavioral/systems/cognitive Action Dominates Valence in Anticipatory Representations in the Human Striatum and Dopaminergic Midbrain , 2010 .

[46]  Antonio Rangel,et al.  The Decision Value Computations in the vmPFC and Striatum Use a Relative Value Code That is Guided by Visual Attention , 2011, The Journal of Neuroscience.

[47]  Raymond J. Dolan,et al.  A Role for the Striatum in Regret-related Choice Repetition , 2011, Journal of Cognitive Neuroscience.

[48]  N. Daw,et al.  Signals in Human Striatum Are Appropriate for Policy Update Rather than Value Prediction , 2011, The Journal of Neuroscience.

[49]  Robert C. Wilson,et al.  Inferring Relevance in a Changing World , 2012, Front. Hum. Neurosci..

[50]  Karl J. Friston,et al.  Action-Specific Value Signals in Reward-Related Regions of the Human Brain , 2012, The Journal of Neuroscience.

[51]  Raymond J. Dolan,et al.  Go and no-go learning in reward and punishment: Interactions between affect and effect , 2012, NeuroImage.

[52]  Eleanor A. Maguire,et al.  Decoding information in the human hippocampus: A user's guide , 2012, Neuropsychologia.

[53]  Raymond J. Dolan,et al.  Distinct encoding of risk and value in economic choice between multiple risky options☆ , 2013, NeuroImage.

[54]  Joseph W. Kable,et al.  The valuation system: A coordinate-based meta-analysis of BOLD fMRI experiments examining neural correlates of subjective value , 2013, NeuroImage.

[55]  Raymond J. Dolan,et al.  Characterising reward outcome signals in sensory cortex☆ , 2013, NeuroImage.

[56]  A. Rangel,et al.  Informatic parcellation of the network involved in the computation of subjective value. , 2014, Social cognitive and affective neuroscience.