By Carrot or by Stick: Cognitive Reinforcement Learning in Parkinsonism

To what extent do we learn from the positive versus negative outcomes of our decisions? The neuromodulator dopamine plays a key role in these reinforcement learning processes. Patients with Parkinson's disease, who have depleted dopamine in the basal ganglia, are impaired in tasks that require learning from trial and error. Here, we show, using two cognitive procedural learning tasks, that Parkinson's patients off medication are better at learning to avoid choices that lead to negative outcomes than they are at learning from positive outcomes. Dopamine medication reverses this bias, making patients more sensitive to positive than negative outcomes. This pattern was predicted by our biologically based computational model of basal ganglia–dopamine interactions in cognition, which has separate pathways for “Go” and “NoGo” responses that are differentially modulated by positive and negative reinforcement.

[1]  The Journal of Comparative Neurology , 1899, The American Naturalist.

[2]  O. Bagasra,et al.  Proceedings of the National Academy of Sciences , 1914, Science.

[3]  Sarah Mae Sincero Heredity , 1875, Nature.

[4]  W. Brogden Annual Review of Psychology , 1957 .

[5]  G. Thines,et al.  Behavioural processes , 1976, Behavioural Processes.

[6]  J. Winn,et al.  Brain , 1878, The Lancet.

[7]  H. J. Gamble Trends in Neurosciences , 1980 .

[8]  E. G. Jones Cerebral Cortex , 1987, Cerebral Cortex.

[9]  S. Kish,et al.  Uneven pattern of dopamine loss in the striatum of patients with idiopathic Parkinson's disease. Pathophysiologic and clinical implications. , 1988, The New England journal of medicine.

[10]  R. Wise,et al.  Brain dopamine and reward. , 1989, Annual review of psychology.

[11]  G. Vining,et al.  Data Analysis: A Model-Comparison Approach , 1989 .

[12]  J. Deniau,et al.  Disinhibition as a basic process in the expression of striatal functions , 1990, Trends in Neurosciences.

[13]  D. Labie,et al.  Molecular Evolution , 1991, Nature.

[14]  J. Staddon,et al.  Transitive inference formation in pigeons. , 1991 .

[15]  C. Gerfen The neostriatal mosaic: multiple levels of compartmental organization in the basal ganglia. , 1992, Annual review of neuroscience.

[16]  AC Tose Cell , 1993, Cell.

[17]  C. Wynne Reinforcement accounts for transitive inference performance , 1995 .

[18]  Kenneth M. Adams,et al.  The Clinical Neuropsychologist , 1995 .

[19]  W. Cowan,et al.  Annual Review of Neuroscience , 1995 .

[20]  A. Mccarthy Development , 1996, Current Opinion in Neurobiology.

[21]  Jennifer A. Mangels,et al.  A Neostriatal Habit Learning System in Humans , 1996, Science.

[22]  J. Mink THE BASAL GANGLIA: FOCUSED SELECTION AND INHIBITION OF COMPETING MOTOR PROGRAMS , 1996, Progress in Neurobiology.

[23]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[24]  J. Bargas,et al.  D1 Receptor Activation Enhances Evoked Discharge in Neostriatal Medium Spiny Neurons by Modulating an L-Type Ca2+ Conductance , 1997, The Journal of Neuroscience.

[25]  P. Greengard,et al.  Bidirectional Regulation of DARPP-32 Phosphorylation by Dopamine , 1997, The Journal of Neuroscience.

[26]  Gregory Ashby,et al.  A neuropsychological theory of multiple systems in category learning. , 1998, Psychological review.

[27]  Juan D Delius,et al.  Transitive responding in animals and humans: Exaptation rather than adaptation? , 1998, Behavioural Processes.

[28]  J. Ashby References and Notes , 1999 .

[29]  Mark Gluck,et al.  Memory and cognition , 2000, Clinical Neurophysiology.

[30]  J. Bargas,et al.  D2 Dopamine Receptors in Striatal Medium Spiny Neurons Reduce L-Type Ca2+ Currents and Excitability via a Novel PLCβ1–IP3–Calcineurin-Signaling Cascade , 2000, The Journal of Neuroscience.

[31]  B. Bloch,et al.  Phenotypical characterization of the neurons expressing the D1 and D2 dopamine receptors in the monkey striatum , 2000, The Journal of comparative neurology.

[32]  坪田 一男 New England Journal of Medicine を夢見て , 2000 .

[33]  T. Robbins,et al.  Probabilistic learning and reversal deficits in patients with Parkinson’s disease or frontal or temporal lobe lesions: possible adverse effects of dopaminergic medication , 2000, Neuropsychologia.

[34]  Michael J. Frank,et al.  Interactions between frontal cortex and basal ganglia in working memory: A computational model , 2001, Cognitive, affective & behavioral neuroscience.

[35]  P. Calabresi,et al.  Dopaminergic control of synaptic plasticity in the dorsal striatum , 2001, The European journal of neuroscience.

[36]  Juha O. Rinne,et al.  Personality traits and brain dopaminergic function in Parkinson's disease , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[37]  D. Wilkin,et al.  Neuron , 2001, Brain Research.

[38]  T. Robbins,et al.  Enhanced or impaired cognitive function in Parkinson's disease as a function of dopaminergic medication and task demands. , 2001, Cerebral cortex.

[39]  W. Schultz Getting Formal with Dopamine and Reward , 2002, Neuron.

[40]  Clay B. Holroyd,et al.  The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. , 2002, Psychological review.

[41]  S. Grossberg,et al.  Psychological Review , 2003 .

[42]  Michael Van Elzakker,et al.  Transitivity, flexibility, conjunctive representations, and the hippocampus. I. An empirical analysis , 2003, Hippocampus.

[43]  M. Delgado,et al.  Dorsal striatum responses to reward and punishment: Effects of valence and magnitude manipulations , 2003, Cognitive, affective & behavioral neuroscience.

[44]  Tatsuo K Sato,et al.  Correlated Coding of Motivation and Outcome of Decision by Dopamine Neurons , 2003, The Journal of Neuroscience.

[45]  J. Bolam,et al.  Uniform Inhibition of Dopamine Neurons in the Ventral Tegmental Area by Aversive Stimuli , 2004, Science.

[46]  G. Pagnoni,et al.  Human Striatal Responses to Monetary Reward Depend On Saliency , 2004, Neuron.

[47]  M. Gluck,et al.  Cortico-striatal contributions to feedback-based learning: converging data from neuroimaging and neuropsychology. , 2004, Brain : a journal of neurology.

[48]  F. McGlone,et al.  Dopamine Transmission in the Human Striatum during Monetary Reward Tasks , 2004, The Journal of Neuroscience.

[49]  J. Aharon-Peretz,et al.  Novelty seeking and harm avoidance in Parkinson’s disease: effects of asymmetric dopamine deficiency , 2004, Journal of Neurology, Neurosurgery & Psychiatry.

[50]  宁北芳,et al.  疟原虫var基因转换速率变化导致抗原变异[英]/Paul H, Robert P, Christodoulou Z, et al//Proc Natl Acad Sci U S A , 2005 .

[51]  Michael J. Frank,et al.  Making Working Memory Work: A Computational Model of Learning in the Prefrontal Cortex and Basal Ganglia , 2006, Neural Computation.