Separate brain regions code for salience vs. valence during reward prediction in humans

Predicting rewards and avoiding aversive conditions is essential for survival. Recent studies using computational models of reward prediction implicate the ventral striatum in appetitive rewards. Whether the same system mediates an organism's response to aversive conditions is unclear. We examined the question using fMRI blood oxygen level‐dependent measurements while healthy volunteers were conditioned using appetitive and aversive stimuli. The temporal difference learning algorithm was used to estimate reward prediction error. Activations in the ventral striatum were robustly correlated with prediction error, regardless of the valence of the stimuli, suggesting that the ventral striatum processes salience prediction error. In contrast, the orbitofrontal cortex and anterior insula coded for the differential valence of appetitive/aversive stimuli. Given its location at the interface of limbic and motor regions, the ventral striatum may be critical in learning about motivationally salient stimuli, regardless of valence, and using that information to bias selection of actions. Inc. Hum Brain Mapp, 2007. © 2006 Wiley‐Liss, Inc.

[1]  Richard S. Sutton,et al.  Time-Derivative Models of Pavlovian Reinforcement , 1990 .

[2]  Terry M. Peters,et al.  3D statistical neuroanatomical models from 305 MRI volumes , 1993, 1993 IEEE Conference Record Nuclear Science Symposium and Medical Imaging Conference.

[3]  Edmund T. Rolls,et al.  Neuronal responses in the ventral striatum of the behaving macaque , 1993, Behavioural Brain Research.

[4]  P. Fox,et al.  Computational approaches to network analysis in functional brain imaging , 1994 .

[5]  Karl J. Friston,et al.  Statistical parametric maps in functional imaging: A general linear approach , 1994 .

[6]  Karl J. Friston,et al.  Spatial registration and normalization of images , 1995 .

[7]  E. Lynd-Balta,et al.  The orbital and medial prefrontal circuit through the primate basal ganglia , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[8]  W. Schultz,et al.  Preferential activation of midbrain dopamine neurons by appetitive rather than aversive stimuli , 1996, Nature.

[9]  P. Dayan,et al.  A framework for mesencephalic dopamine systems based on predictive Hebbian learning , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[10]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[11]  J. Salamone,et al.  Behavioral functions of nucleus accumbens dopamine: Empirical and conceptual problems with the anhedonia hypothesis , 1997, Neuroscience & Biobehavioral Reviews.

[12]  Karl J. Friston,et al.  Brain Systems Mediating Aversive Conditioning: an Event-Related fMRI Study , 1998, Neuron.

[13]  Joseph E LeDoux,et al.  Human Amygdala Activation during Conditioned Fear Acquisition and Extinction: a Mixed-Trial fMRI Study , 1998, Neuron.

[14]  K. Berridge,et al.  What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? , 1998, Brain Research Reviews.

[15]  P. Garris,et al.  Dissociation of dopamine release in the nucleus accumbens from intracranial self-stimulation , 1999, Nature.

[16]  Karl J. Friston,et al.  Amygdala–Hippocampal Involvement in Human Aversive Trace Conditioning Revealed through Event-Related Functional Magnetic Resonance Imaging , 1999, The Journal of Neuroscience.

[17]  W. Schultz The Reward Signal of Midbrain Dopamine Neurons. , 1999, News in physiological sciences : an international journal of physiology produced jointly by the International Union of Physiological Sciences and the American Physiological Society.

[18]  E. Rolls The orbitofrontal cortex and reward. , 2000, Cerebral cortex.

[19]  Karl J. Friston,et al.  Dissociable Neural Responses in Human Reward Systems , 2000, The Journal of Neuroscience.

[20]  Nikolaus R. McFarland,et al.  Striatonigrostriatal Pathways in Primates Form an Ascending Spiral from the Shell to the Dorsolateral Striatum , 2000, The Journal of Neuroscience.

[21]  L. Nystrom,et al.  Tracking the hemodynamic responses to reward and punishment in the striatum. , 2000, Journal of neurophysiology.

[22]  Samuel M. McClure,et al.  Predictability Modulates Human Brain Response to Reward , 2001, The Journal of Neuroscience.

[23]  D. Kahneman,et al.  Functional Imaging of Neural Responses to Expectancy and Experience of Monetary Gains and Losses tasks with monetary payoffs , 2001 .

[24]  Brian Knutson,et al.  Dissociation of reward anticipation and outcome with event-related fMRI , 2001, Neuroreport.

[25]  W. Schultz Book Review: Reward Signaling by Dopamine Neurons , 2001, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[26]  J. O'Doherty,et al.  Neural Responses during Anticipation of a Primary Taste Reward , 2002, Neuron.

[27]  W. Schultz Getting Formal with Dopamine and Reward , 2002, Neuron.

[28]  J. O'Doherty,et al.  Appetitive and Aversive Olfactory Learning in Humans Studied Using Event-Related Functional Magnetic Resonance Imaging , 2002, The Journal of Neuroscience.

[29]  S. Kapur,et al.  Direct Activation of the Ventral Striatum in Anticipation of Aversive Stimuli , 2003, Neuron.

[30]  Sabrina Ravel,et al.  Responses of Tonically Active Neurons in the Monkey Striatum Discriminate between Motivationally Opposing Stimuli , 2003, The Journal of Neuroscience.

[31]  G. Glover,et al.  Dissociated neural representations of intensity and valence in human olfaction , 2003, Nature Neuroscience.

[32]  E. Rolls,et al.  Different representations of pleasant and unpleasant odours in the human brain , 2003, The European journal of neuroscience.

[33]  Geoffrey Schoenbaum,et al.  Lesions of orbitofrontal cortex and basolateral amygdala complex disrupt acquisition of odor-guided discriminations and reversals. , 2003, Learning & memory.

[34]  M. Mesulam,et al.  Dissociation of Neural Representation of Intensity and Affective Valuation in Human Gustation , 2003, Neuron.

[35]  Samuel M. McClure,et al.  A computational substrate for incentive salience , 2003, Trends in Neurosciences.

[36]  A. Young,et al.  The interpretation of the measurement of nucleus accumbens dopamine by in vivo dialysis: the kick, the craving or the cognition? , 2003, Neuroscience & Biobehavioral Reviews.

[37]  Samuel M. McClure,et al.  Temporal Prediction Errors in a Passive Learning Task Activate Human Striatum , 2003, Neuron.

[38]  Karl J. Friston,et al.  Temporal Difference Models and Reward-Related Learning in the Human Brain , 2003, Neuron.

[39]  N. Logothetis The Underpinnings of the BOLD Functional Magnetic Resonance Imaging Signal , 2003, The Journal of Neuroscience.

[40]  G. Schoenbaum,et al.  Neural Encoding in Ventral Striatum during Olfactory Discrimination Learning , 2003, Neuron.

[41]  J. Bolam,et al.  Uniform Inhibition of Dopamine Neurons in the Ventral Tegmental Area by Aversive Stimuli , 2004, Science.

[42]  Jonathan D. Cohen,et al.  Computational roles for dopamine in behavioural control , 2004, Nature.

[43]  Karl J. Friston,et al.  Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning , 2004, Science.

[44]  G. Pagnoni,et al.  Human Striatal Responses to Monetary Reward Depend On Saliency , 2004, Neuron.

[45]  Peter Dayan,et al.  Temporal difference models describe higher-order learning in humans , 2004, Nature.