Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning

What are the genetic and neural components that support adaptive learning from positive and negative outcomes? Here, we show with genetic analyses that three independent dopaminergic mechanisms contribute to reward and avoidance learning in humans. A polymorphism in the DARPP-32 gene, associated with striatal dopamine function, predicted relatively better probabilistic reward learning. Conversely, the C957T polymorphism of the DRD2 gene, associated with striatal D2 receptor function, predicted the degree to which participants learned to avoid choices that had been probabilistically associated with negative outcomes. The Val/Met polymorphism of the COMT gene, associated with prefrontal cortical dopamine function, predicted participants' ability to rapidly adapt behavior on a trial-to-trial basis. These findings support a neurocomputational dissociation between striatal and prefrontal dopaminergic mechanisms in reinforcement learning. Computational maximum likelihood analyses reveal independent gene effects on three reinforcement learning parameters that can explain the observed dissociations.

[1]  D. Sibley,et al.  The classification of dopamine receptors: relationship to radioligand binding. , 1983, Annual review of neuroscience.

[2]  Paul Greengard,et al.  A dopamine- and cyclic AMP-regulated phosphoprotein enriched in dopamine-innervated brain regions , 1983, Nature.

[3]  P. Greengard,et al.  DARPP-32, a dopamine- and adenosine 3':5'-monophosphate-regulated phosphoprotein enriched in dopamine-innervated brain regions. III. Immunocytochemical localization , 1984, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[4]  H. Fibiger,et al.  D1 and D2 dopamine receptors differentially regulate c-fos expression in striatonigral and striatopallidal neurons , 1992, Neuroscience.

[5]  D. Signorini,et al.  Neural networks , 1995, The Lancet.

[6]  James L. McClelland,et al.  Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. , 1995, Psychological review.

[7]  D. Pfaff,et al.  Catechol-O-methyltransferase-deficient mice exhibit sexually dimorphic changes in catecholamine levels and behavior. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[8]  C. I. Connolly,et al.  Building neural representations of habits. , 1999, Science.

[9]  G. Chiara,et al.  Reciprocal changes in prefrontal and limbic dopamine responsiveness to aversive and rewarding stimuli after chronic mild stress: implications for the psychobiology of depression , 1999, Biological Psychiatry.

[10]  T. Rebbeck,et al.  Collection of genomic DNA by buccal swabs for polymerase chain reaction-based biomarker assays. , 1999, Environmental health perspectives.

[11]  T. Sejnowski,et al.  Dopamine-mediated stabilization of delay-period activity in a network model of prefrontal cortex. , 2000, Journal of neurophysiology.

[12]  P. Greengard,et al.  Dopamine and cAMP-Regulated Phosphoprotein 32 kDa Controls Both Striatal Long-Term Depression and Long-Term Potentiation, Opposing Forms of Synaptic Plasticity , 2000, The Journal of Neuroscience.

[13]  E. Miller,et al.  An integrative theory of prefrontal cortex function. , 2001, Annual review of neuroscience.

[14]  R. Straub,et al.  Effect of COMT Val108/158 Met genotype on frontal lobe function and risk for schizophrenia , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[15]  D. Wilkin,et al.  Neuron , 2001, Brain Research.

[16]  Sham M. Kakade,et al.  Opponent interactions between serotonin and dopamine , 2002, Neural Networks.

[17]  Jonathan D. Cohen,et al.  Computational perspectives on dopamine function in prefrontal cortex , 2002, Current Opinion in Neurobiology.

[18]  W. Schultz Getting Formal with Dopamine and Reward , 2002, Neuron.

[19]  N. Saitou,et al.  Synonymous mutations in the human dopamine receptor D2 (DRD2) affect mRNA stability and synthesis of the receptor. , 2003, Human molecular genetics.

[20]  Tatsuo K Sato,et al.  Correlated Coding of Motivation and Outcome of Decision by Dopamine Neurons , 2003, The Journal of Neuroscience.

[21]  Paul J. Harrison,et al.  Catechol-O-Methyltransferase Inhibition Improves Set-Shifting Performance and Elevates Stimulated Dopamine Release in the Rat Prefrontal Cortex , 2004, The Journal of Neuroscience.

[22]  M. Jackson,et al.  Stimulus‐specific plasticity of prefrontal cortex dopamine neurotransmission , 2004, Journal of neurochemistry.

[23]  Michael J. Frank,et al.  By Carrot or by Stick: Cognitive Reinforcement Learning in Parkinsonism , 2004, Science.

[24]  B. Balleine,et al.  Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning , 2004, The European journal of neuroscience.

[25]  Stephen Grossberg,et al.  How laminar frontal cortex and basal ganglia circuits interact to control planned and reactive saccades , 2004, Neural Networks.

[26]  K. Någren,et al.  C957T polymorphism of the dopamine D2 receptor (DRD2) gene affects striatal DRD2 availability in vivo , 2004, Molecular Psychiatry.

[27]  P. Calabresi,et al.  Chronic Haloperidol Promotes Corticostriatal Long-Term Potentiation by Targeting Dopamine D2L Receptors , 2004, The Journal of Neuroscience.

[28]  Jonathan D. Cohen,et al.  Prefrontal cortex and flexible cognitive control: rules without symbols. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[29]  Michael J. Frank,et al.  Error-Related Negativity Predicts Reinforcement Learning and Conflict Biases , 2005, Neuron.

[30]  K. Doya,et al.  Representation of Action-Specific Reward Values in the Striatum , 2005, Science.

[31]  E. Miller,et al.  Different time courses of learning-related activity in the prefrontal cortex and striatum , 2005, Nature.

[32]  R. Parasuraman,et al.  Beyond Heritability , 2005, Psychological science.

[33]  Michael X. Cohen,et al.  Individual differences in extraversion and dopamine genetics predict neural reward responses. , 2005, Brain research. Cognitive brain research.

[34]  P. Dayan,et al.  Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control , 2005, Nature Neuroscience.

[35]  Michael J. Frank,et al.  Dynamic Dopamine Modulation in the Basal Ganglia: A Neurocomputational Account of Cognitive Deficits in Medicated and Nonmedicated Parkinsonism , 2005, Journal of Cognitive Neuroscience.

[36]  Jerome R. Busemeyer,et al.  Using Cognitive Models to Map Relations Between Neuropsychological Disorders and Human Decision-Making Deficits , 2005, Psychological science.

[37]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[38]  P. Glimcher,et al.  Midbrain Dopamine Neurons Encode a Quantitative Reward Prediction Error Signal , 2005, Neuron.

[39]  Merja Haaparanta,et al.  The A1 allele of the human D2 dopamine receptor gene is associated with increased activity of striatal L-amino acid decarboxylase in healthy subjects , 2005, Pharmacogenetics and genomics.

[40]  B. Moghaddam,et al.  Rule Learning and Reward Contingency Are Associated with Dissociable Patterns of Dopamine Activation in the Rat Prefrontal Cortex, Nucleus Accumbens, and Dorsal Striatum , 2006, The Journal of Neuroscience.

[41]  P. Dayan,et al.  Cortical substrates for exploratory decisions in humans , 2006, Nature.

[42]  Michael J. Frank,et al.  A mechanistic account of striatal dopamine function in human cognition: psychopharmacological studies with cabergoline and haloperidol. , 2006, Behavioral neuroscience.

[43]  Kae Nakamura,et al.  Role of Dopamine in the Primate Caudate Nucleus in Reward Modulation of Saccades , 2006, The Journal of Neuroscience.

[44]  M. D’Esposito,et al.  Reversal learning in Parkinson's disease depends on medication status and outcome valence , 2006, Neuropsychologia.

[45]  M. Frank,et al.  Anatomy of a decision: striato-orbitofrontal interactions in reinforcement learning, decision making, and reversal. , 2006, Psychological review.

[46]  Robert C. Malenka,et al.  Endocannabinoid-mediated rescue of striatal LTD and motor deficits in Parkinson's disease models , 2007, Nature.

[47]  Andreas Meyer-Lindenberg,et al.  Genetic evidence implicating DARPP-32 in human frontostriatal structure, function, and cognition. , 2007, The Journal of clinical investigation.

[48]  Benjamin M. Robinson,et al.  Selective Reinforcement Learning Deficits in Schizophrenia Support Predictions from Computational Models of Striatal-Cortical Dysfunction , 2007, Biological Psychiatry.

[49]  R. Rosenfeld Nature , 2009, Otolaryngology--head and neck surgery : official journal of American Academy of Otolaryngology-Head and Neck Surgery.