Establishing the dopamine dependency of human striatal signals during reward and punishment reversal learning.

Drugs that alter dopamine transmission have opposite effects on reward and punishment learning. These opposite effects have been suggested to depend on dopamine in the striatum. Here, we establish for the first time the neurochemical specificity of such drug effects, during reward and punishment learning in humans, by adopting a coadministration design. Participants (N = 22) were scanned on 4 occasions using functional magnetic resonance imaging, following intake of placebo, bromocriptine (dopamine-receptor agonist), sulpiride (dopamine-receptor antagonist), or a combination of both drugs. A reversal-learning task was employed, in which both unexpected rewards and punishments signaled reversals. Drug effects were stratified with baseline working memory to take into account individual variations in drug response. Sulpiride induced parallel span-dependent changes on striatal blood oxygen level-dependent (BOLD) signal during unexpected rewards and punishments. These drug effects were found to be partially dopamine-dependent, as they were blocked by coadministration with bromocriptine. In contrast, sulpiride elicited opposite effects on behavioral measures of reward and punishment learning. Moreover, sulpiride-induced increases in striatal BOLD signal during both outcomes were associated with behavioral improvement in reward versus punishment learning. These results provide a strong support for current theories, suggesting that drug effects on reward and punishment learning are mediated via striatal dopamine.

[1]  A. Bond,et al.  The use of analogue scales in rating subjective feelings , 1974 .

[2]  G. Groth-Marnat.  Handbook of Psychological Assessment , 2016 .

[3]  D. C. Howell.  Statistical Methods for Psychology , 1987 .

[4]  Raymond J. Shaw,et al.  Effects of adult age on structural and operational capacities in working memory. , 1991, Psychology and aging.

[5]  Jerrold S. Meyer,et al.  Principles of Neuropsychopharmacology , 1997 .

[6]  M. Farah,et al.  Effects of bromocriptine on human subjects depend on working memory capacity , 1997, Neuroreport.

[7]  P. Redgrave,et al.  Is the short-latency dopamine response too short to signal reward error? , 1999, Trends in Neurosciences.

[8]  J. Mirenowicz,et al.  Dissociation of Pavlovian and instrumental incentive learning under dopamine antagonists. , 2000, Behavioral neuroscience.

[9]  K. Berridge,et al.  Intra-Accumbens Amphetamine Increases the Conditioned Incentive Salience of Sucrose Reward: Enhancement of Reward “Wanting” without Enhanced “Liking” or Response Reinforcement , 2000, The Journal of Neuroscience.

[10]  G. Groth-Marnat,et al.  Specific learning disabilities and difficulties in children and adolescents: The Wechsler intelligence scales , 2001 .

[11]  N. Tzourio-Mazoyer,et al.  Automated Anatomical Labeling of Activations in SPM Using a Macroscopic Anatomical Parcellation of the MNI MRI Single-Subject Brain , 2002, NeuroImage.

[12]  D. Deleu,et al.  Clinical Pharmacokinetic and Pharmacodynamic Properties of Drugs Used in the Treatment of Parkinson’s Disease , 2002, Clinical pharmacokinetics.

[13]  Robin M Heidemann,et al.  Generalized autocalibrating partially parallel acquisitions (GRAPPA) , 2002, Magnetic resonance in medicine.

[14]  Paul M. Grasby,et al.  Systemic sulpiride modulates striatal blood flow: relationships to spatial working memory and planning , 2003, NeuroImage.

[15]  G. Pagnoni,et al.  Human Striatal Response to Salient Nonrewarding Stimuli , 2003, The Journal of Neuroscience.

[16]  T. Robbins,et al.  Impaired set-shifting and dissociable effects on tests of spatial working memory following the dopamine D2 receptor antagonist sulpiride in human volunteers , 2004, Psychopharmacology.

[17]  Michael J. Frank,et al.  By Carrot or by Stick: Cognitive Reinforcement Learning in Parkinsonism , 2004, Science.

[18]  Karl J. Friston,et al.  Opponent appetitive-aversive neural processes underlie predictive learning of pain relief , 2005, Nature Neuroscience.

[19]  Michael J. Frank,et al.  Dynamic Dopamine Modulation in the Basal Ganglia: A Neurocomputational Account of Cognitive Deficits in Medicated and Nonmedicated Parkinsonism , 2005, Journal of Cognitive Neuroscience.

[20]  S. Kapur,et al.  From dopamine to salience to psychosis—linking biology, pharmacology and phenomenology of psychosis , 2005, Schizophrenia Research.

[21]  Brian Knutson,et al.  Linking nucleus accumbens dopamine and blood oxygenation , 2007, Psychopharmacology.

[22]  Michael J. Frank,et al.  A mechanistic account of striatal dopamine function in human cognition: psychopharmacological studies with cabergoline and haloperidol. , 2006, Behavioral neuroscience.

[23]  R. Dolan,et al.  Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans , 2006, Nature.

[24]  M. D’Esposito,et al.  Reversal learning in Parkinson's disease depends on medication status and outcome valence , 2006, Neuropsychologia.

[25]  Michael J. Frank,et al.  Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning , 2007, Proceedings of the National Academy of Sciences.

[26]  S. Kapur,et al.  Separate brain regions code for salience vs. valence during reward prediction in humans , 2007, Human brain mapping.

[27]  T. Robbins,et al.  L-DOPA Disrupts Activity in the Nucleus Accumbens during Reversal Learning in Parkinson's Disease , 2007, Neuropsychopharmacology.

[28]  Michael J. Frank,et al.  Testing Computational Models of Dopamine and Noradrenaline Dysfunction in Attention Deficit/Hyperactivity Disorder , 2007, Neuropsychopharmacology.

[29]  P. Dayan,et al.  Differential Encoding of Losses and Gains in the Human Striatum , 2007, The Journal of Neuroscience.

[30]  Michael J. Frank,et al.  Hold Your Horses: Impulsivity, Deep Brain Stimulation, and Medication in Parkinsonism , 2007, Science.

[31]  P. Grasby,et al.  Dopamine D2 receptor occupancy levels of acute sulpiride challenges that produce working memory and learning impairments in healthy volunteers , 2007, Psychopharmacology.

[32]  M. Delgado,et al.  The role of the striatum in aversive learning and aversive prediction errors , 2008, Philosophical Transactions of the Royal Society B: Biological Sciences.

[33]  H. Heinze,et al.  Mesolimbic Functional Magnetic Resonance Imaging Activations during Reward Anticipation Correlate with Reward-Related Ventral Striatal Dopamine Release , 2008, The Journal of Neuroscience.

[34]  Michael X. Cohen,et al.  A Role for Dopamine in Temporal Decision Making and Reward Maximization in Parkinsonism , 2008, The Journal of Neuroscience.

[35]  M. D’Esposito,et al.  Working Memory Capacity Predicts Dopamine Synthesis Capacity in the Human Striatum , 2008, The Journal of Neuroscience.

[36]  T. Dinan,et al.  Prolactin and dopamine: What is the connection? A Review Article , 2008, Journal of psychopharmacology.

[37]  T. Robbins,et al.  Methylphenidate Has Differential Effects on Blood Oxygenation Level-Dependent Signal Related to Cognitive Subprocesses of Reversal Learning , 2008, The Journal of Neuroscience.

[38]  Ivan Toni,et al.  Perceptuo-Motor Interactions during Prehension Movements , 2008, The Journal of Neuroscience.

[39]  Frank Baumann,et al.  The dopamine D2 receptor antagonist sulpiride modulates striatal BOLD signal during the manipulation of information in working memory , 2009, Psychopharmacology.

[40]  R. Murray,et al.  From real-world events to psychosis: the emerging neuropharmacology of delusions. , 2009, Schizophrenia bulletin.

[41]  Michael J. Frank,et al.  Single dose of a dopamine agonist impairs reinforcement learning in humans: Evidence from event‐related potentials and computational modeling of striatal‐cortical function , 2009, Human brain mapping.

[42]  M. Pessiglione,et al.  Pharmacological modulation of subliminal learning in Parkinson's and Tourette's syndromes , 2009, Proceedings of the National Academy of Sciences.

[43]  Mark Hallett,et al.  Impulsive choice and response in dopamine agonist-related impulse control behaviors , 2009, Psychopharmacology.

[44]  O. Hikosaka,et al.  Two types of dopamine neuron distinctly convey positive and negative motivational signals , 2009, Nature.

[45]  W. Jagust,et al.  Striatal dopamine and working memory. , 2009, Cerebral cortex.

[46]  M. Frank,et al.  Striatal Dopamine Predicts Outcome-Specific Reversal Learning and Its Sensitivity to Dopaminergic Drug Administration , 2009, The Journal of Neuroscience.

[47]  M. Frank,et al.  Genetic contributions to avoidance-based decisions: striatal D2 receptor polymorphisms , 2009, Neuroscience.

[48]  M. Gluck,et al.  Reward-learning and the novelty-seeking personality: a between- and within-subjects study of the effects of dopamine agonists on young Parkinson's patients. , 2009, Brain : a journal of neurology.

[49]  P. Tobler,et al.  Functional imaging of the human dopaminergic midbrain , 2009, Trends in Neurosciences.

[50]  M. Hallett,et al.  Mechanisms Underlying Dopamine-Mediated Reward Bias in Compulsive Behaviors , 2010, Neuron.

[51]  Jean-Luc Anton,et al.  Region of interest analysis using an SPM toolbox , 2010 .

[52]  Roshan Cools,et al.  Dissociable responses to punishment in distinct striatal regions during reversal learning , 2010, NeuroImage.

[53]  J. O'Doherty,et al.  Human Medial Orbitofrontal Cortex Is Recruited during Experience of Imagined and Real Rewards Prescan Training , 2022 .

[54]  M. Frank,et al.  Neurogenetics and Pharmacology of Learning, Motivation, and Cognition , 2011, Neuropsychopharmacology.

[55]  M. D’Esposito,et al.  Inverted-U–Shaped Dopamine Actions on Human Working Memory and Cognitive Control , 2011, Biological Psychiatry.

[56]  E. Crone,et al.  Distinct linear and non-linear trajectories of reward and punishment reversal learning during development: Relevance for dopamine's role in adolescent decision making , 2011, Developmental Cognitive Neuroscience.

[57]  T. Robinson,et al.  A selective role for dopamine in reward learning , 2010, Nature.

[58]  R. Cools.  Dopaminergic control of the striatum for high-level cognition , 2011, Current Opinion in Neurobiology.

[59]  B. Franke,et al.  Human cognitive flexibility depends on dopamine D2 receptor signaling , 2011, Psychopharmacology.

[60]  Jocham Gerhard,et al.  Dopamine-mediated reinforcement learning signals in the striatum and ventromedial prefrontal cortex underlie value-based choices , 2011 .

[61]  Markus Ullsperger,et al.  Continuous theta-burst stimulation (cTBS) over the lateral prefrontal cortex alters reinforcement learning bias , 2011, NeuroImage.

[62]  Deanna M. Barch,et al.  Cognition in schizophrenia: core psychological and neural mechanisms , 2012, Trends in Cognitive Sciences.

[63]  Wolfram Schultz,et al.  BOLD responses in reward regions to hypothetical and imaginary monetary rewards , 2012, NeuroImage.

[64]  The Wechsler Intelligence Scales for Children and Adults , 2013 .