Midbrain Dopaminergic Neurons and Striatal Cholinergic Interneurons Encode the Difference between Reward and Aversive Events at Different Epochs of Probabilistic Classical Conditioning Trials

Midbrain dopaminergic neurons (DANs) typically increase their discharge rate in response to appetitive predictive cues and outcomes, whereas striatal cholinergic tonically active interneurons (TANs) decrease their rate. This may indicate that the activity of TANs and DANs is negatively correlated and that TANs can broaden the basal ganglia reinforcement teaching signal, for instance by encoding worse than predicted events. We studied the activity of 106 DANs and 180 TANs of two monkeys recorded during the performance of a classical conditioning task with cues predicting the probability of food, neutral, and air puff outcomes. DANs responded to all cues with elevations of discharge rate, whereas TANs depressed their discharge rate. Nevertheless, although dopaminergic responses to appetitive cues were larger than their responses to neutral or aversive cues, the TAN responses were more similar. Both TANs and DANs responded faster to an air puff than to a food outcome; however, DANs responded with a discharge elevation, whereas the TAN responses included major negative and positive deflections. Finally, food versus air puff omission was better encoded by TANs. In terms of the activity of single neurons with distinct responses to the different behavioral events, both DANs and TANs were more strongly modulated by reward than by aversive related events and better reflected the probability of reward than aversive outcome. Thus, TANs and DANs encode the task episodes differentially. The DANs encode mainly the cue and outcome delivery, whereas the TANs mainly encode outcome delivery and omission at termination of the behavioral trial episode.

[1]  A. Barbeau The pathogenesis of Parkinson's disease: a new hypothesis. , 1962, Canadian Medical Association journal.

[2]  A. Tversky,et al.  The framing of decisions and the psychology of choice. , 1981, Science.

[3]  J. Lehmann,et al.  The striatal cholinergic interneuron: Synaptic target of dopaminergic terminals? , 1983, Neuroscience.

[4]  W. Schultz,et al.  The activity of pars compacta neurons of the monkey substantia nigra is depressed by apomorphine , 1984, Neuroscience Letters.

[5]  W. Cowan,et al.  A stereotaxic atlas of the brain of the cynomolgus monkey (Macaca fascicularis) , 1984, The Journal of comparative neurology.

[6]  H. Groenewegen,et al.  Regulation of the activity of striatal cholinergic neurons by dopamine , 1992, Neuroscience.

[7]  W. Schultz,et al.  Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[8]  A M Graybiel,et al.  The basal ganglia and adaptive motor control. , 1994, Science.

[9]  W. Schultz,et al.  Preferential activation of midbrain dopamine neurons by appetitive rather than aversive stimuli , 1996, Nature.

[10]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[11]  H. Nagaraja,et al.  Heart rate variability: origins, methods, and interpretive caveats. , 1997, Psychophysiology.

[12]  F. Guarraci,et al.  An electrophysiological characterization of ventral tegmental area dopaminergic neurons during differential pavlovian fear conditioning in the awake rabbit , 1999, Behavioural Brain Research.

[13]  Sabrina Ravel,et al.  Tonically active neurons in the monkey striatum do not preferentially respond to appetitive stimuli , 1999, Experimental Brain Research.

[14]  Richard F. Martin,et al.  Primate brain maps : structure of the macaque brain , 2000 .

[15]  J. Horvitz Mesolimbocortical and nigrostriatal dopamine responses to salient non-reward events , 2000, Neuroscience.

[16]  P. Calabresi,et al.  Acetylcholine-mediated modulation of striatal function , 2000, Trends in Neurosciences.

[17]  J. Wickens,et al.  A cellular mechanism of reward-related learning , 2001, Nature.

[18]  O. Hikosaka,et al.  Role of Tonically Active Neurons in Primate Caudate in Reward-Oriented Saccadic Eye Movement , 2001, The Journal of Neuroscience.

[19]  Hagai Bergman,et al.  Stepping out of the box: information processing in the neural networks of the basal ganglia , 2001, Current Opinion in Neurobiology.

[20]  A. Graybiel,et al.  Neurons in the thalamic CM-Pf complex supply striatal neurons with information about behaviorally significant sensory events. , 2001, Journal of neurophysiology.

[21]  Peter Dayan,et al.  Dopamine: generalization and bonuses , 2002, Neural Networks.

[22]  Sabrina Ravel,et al.  Responses of Tonically Active Neurons in the Monkey Striatum Discriminate between Motivationally Opposing Stimuli , 2003, The Journal of Neuroscience.

[23]  W. Schultz,et al.  Discrete Coding of Reward Probability and Uncertainty by Dopamine Neurons , 2003, Science.

[24]  J. Bolam,et al.  Uniform Inhibition of Dopamine Neurons in the Ventral Tegmental Area by Aversive Stimuli , 2004, Science.

[25]  Naoyuki Matsumoto,et al.  Tonically Active Neurons in the Primate Caudate Nucleus and Putamen Differentially Encode Instructed Motivational Outcomes of Action , 2004, The Journal of Neuroscience.

[26]  J. Wickens,et al.  Computational models of the basal ganglia: from robots to membranes , 2004, Trends in Neurosciences.

[27]  Michael J. Frank,et al.  By Carrot or by Stick: Cognitive Reinforcement Learning in Parkinsonism , 2004, Science.

[28]  E. Vaadia,et al.  Coincident but Distinct Messages of Midbrain Dopamine and Striatal Tonically Active Neurons , 2004, Neuron.

[29]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[30]  P. Glimcher,et al.  Midbrain Dopamine Neurons Encode a Quantitative Reward Prediction Error Signal , 2005, Neuron.

[31]  P. Redgrave,et al.  Nociceptive responses of midbrain dopaminergic neurones are modulated by the superior colliculus in the rat , 2006, Neuroscience.

[32]  P. Redgrave,et al.  The short-latency dopamine signal: a role in discovering novel actions? , 2006, Nature Reviews Neuroscience.

[33]  Hagai Bergman,et al.  Real‐time refinement of subthalamic nucleus targeting using Bayesian decision‐making on the root mean square measure , 2006, Movement disorders : official journal of the Movement Disorder Society.

[34]  B. Richmond,et al.  Dopamine neuronal responses in monkeys performing visually cued reward schedules , 2006, The European journal of neuroscience.

[35]  Henry H. Yin,et al.  Dopaminergic Control of Corticostriatal Long-Term Synaptic Depression in Medium Spiny Neurons Is Mediated by Cholinergic Interneurons , 2006, Neuron.

[36]  M. Kimura,et al.  History- and current instruction-based coding of forthcoming behavioral outcomes in the striatum. , 2007, Journal of neurophysiology.

[37]  P. Glimcher,et al.  Statistics of midbrain dopamine neuron spike trains in the awake primate. , 2007, Journal of neurophysiology.

[38]  Yasushi Miyashita,et al.  MRI-based localization of electrophysiological recording sites within the cerebral cortex at single-voxel accuracy , 2007, Nature Methods.

[39]  O. Hikosaka,et al.  Lateral habenula as a source of negative reward signals in dopamine neurons , 2007, Nature.

[40]  P. Glimcher,et al.  Action and Outcome Encoding in the Primate Caudate Nucleus , 2007, The Journal of Neuroscience.

[41]  Mati Joshua,et al.  Quantifying the isolation quality of extracellularly recorded action potentials , 2007, Journal of Neuroscience Methods.

[42]  R. Wightman,et al.  Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens , 2007, Nature Neuroscience.

[43]  Boris Gourévitch,et al.  A simple indicator of nonstationarity of firing rate in spike trains , 2007, Journal of Neuroscience Methods.

[44]  J. Wickens,et al.  Space, time and dopamine , 2007, Trends in Neurosciences.

[45]  P. Glimcher,et al.  Value Representations in the Primate Striatum during Matching Behavior , 2008, Neuron.