A cellular mechanism of reward-related learning

Positive reinforcement helps to control the acquisition of learned behaviours. Here we report a cellular mechanism in the brain that may underlie the behavioural effects of positive reinforcement. We used intracranial self-stimulation (ICSS) as a model of reinforcement learning, in which each rat learns to press a lever that applies reinforcing electrical stimulation to its own substantia nigra. The outputs from neurons of the substantia nigra terminate on neurons in the striatum in close proximity to inputs from the cerebral cortex on the same striatal neurons. We measured the effect of substantia nigra stimulation on these inputs from the cortex to striatal neurons and also on how quickly the rats learned to press the lever. We found that stimulation of the substantia nigra (with the optimal parameters for lever-pressing behaviour) induced potentiation of synapses between the cortex and the striatum, which required activation of dopamine receptors. The degree of potentiation within ten minutes of the ICSS trains was correlated with the time taken by the rats to learn ICSS behaviour. We propose that stimulation of the substantia nigra when the lever is pressed induces a similar potentiation of cortical inputs to the striatum, positively reinforcing the learning of the behaviour by the rats.

[1]  James L Olds,et al.  Positive reinforcement produced by electrical stimulation of septal area and other regions of rat brain. , 1954, Journal of comparative and physiological psychology.

[2]  R. W. Reynolds The relationship between stimulation voltage and rate of hypothalamic self-stimulation in the rat. , 1958, Journal of comparative and physiological psychology.

[3]  E. Valenstein,et al.  An evaluation of response rate as a measure of rewarding intracranial stimulation. , 1962, Journal of comparative and physiological psychology.

[4]  P. Milner,et al.  Schedule control of behavior reinforced by electrical stimulation of the brain. , 1977, Science.

[5]  N. White,et al.  Memory facilitation by self-stimulation reinforcement mediated by the nigro-neostriatal bundle , 1978, Physiology & Behavior.

[6]  R. Wise,et al.  Intracranial self-stimulation in relation to the ascending dopaminergic systems of the midbrain: A moveable electrode mapping study , 1980, Brain Research.

[7]  Charles J. Wilson,et al.  Spontaneous firing patterns of identified spiny neurons in the rat neostriatum , 1981, Brain Research.

[8]  Robert Miller Meaning and Purpose in the Intact Brain , 1981 .

[9]  L. Swanson,et al.  The projections of the ventral tegmental area and adjacent regions: A combined fluorescent retrograde tracer and immunofluorescence study in the rat , 1982, Brain Research Bulletin.

[10]  G. Paxinos,et al.  The Rat Brain in Stereotaxic Coordinates , 1983 .

[11]  Charles J. Wilson Postsynaptic potentials evoked in spiny neostriatal projection neurons by stimulation of ipsilateral and contralateral neocortex , 1986, Brain Research.

[12]  G. E. Alexander,et al.  Parallel organization of functionally segregated circuits linking basal ganglia and cortex. , 1986, Annual review of neuroscience.

[13]  P. Shizgal,et al.  Evidence implicating descending fibers in self-stimulation of the medial forebrain bundle , 1986, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[14]  A. Phillips,et al.  The role of dopamine in intracranial self-stimulation of the ventral tegmental area , 1987, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[15]  L. Straus Age of the modern Europeans , 1989, Nature.

[16]  G. E. Alexander,et al.  Preparation for movement: neural representations of intended direction in three motor areas of the monkey. , 1990, Journal of neurophysiology.

[17]  M. Kimura Behaviorally contingent property of movement-related activity of the primate putamen. , 1990, Journal of neurophysiology.

[18]  A. D. Smith,et al.  The neural network of the basal ganglia as revealed by the study of synaptic connections of identified neurones , 1990, Trends in Neurosciences.

[19]  D. Jackson,et al.  Reward summation and the effects of dopamine D1 and D2 agonists and antagonists on fixed-interval responding for brain stimulation , 1994, Pharmacology Biochemistry and Behavior.

[20]  W. Schultz,et al.  Importance of unpredictability for reward responses in primate dopamine neurons. , 1994, Journal of neurophysiology.

[21]  A. Graybiel,et al.  Effect of the nigrostriatal dopamine system on acquired neural responses in the striatum of behaving monkeys. , 1994, Science.

[22]  W. Schultz,et al.  Preferential activation of midbrain dopamine neurons by appetitive rather than aversive stimuli , 1996, Nature.

[23]  P. Dayan,et al.  A framework for mesencephalic dopamine systems based on predictive Hebbian learning , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[24]  Charles J. Wilson,et al.  The origins of two-state spontaneous membrane potential fluctuations of neostriatal spiny neurons , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[25]  R. Wise,et al.  Addictive drugs and brain stimulation reward. , 1996, Annual review of neuroscience.

[26]  S. Charpier,et al.  In vivo activity-dependent plasticity at cortico-striatal connections: evidence for physiological long-term potentiation. , 1997, Proceedings of the National Academy of Sciences of the United States of America.

[27]  Charles J. Wilson,et al.  Membrane potential synchrony of simultaneously recorded striatal spiny neurons in vivo , 1998, Nature.

[28]  W. Schultz,et al.  A neural network model with dopamine-like reinforcement signal that learns a spatial delayed response task , 1999, Neuroscience.

[29]  P. Pettitt,et al.  Direct radiocarbon dates for Vindija G(1) and Velika Pecína late Pleistocene hominid remains. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[30]  J. Smart Meaning and Purpose , 1999 .

[31]  A. Graybiel,et al.  Role of [corrected] nigrostriatal dopamine system in learning to perform sequential motor tasks in a predictive manner. , 1999, Journal of neurophysiology.

[32]  J. Mangerud,et al.  Age and extent of the Barents and Kara ice sheets in Northern Russia , 1999 .

[33]  松本 直幸,et al.  Role of Nigrostriatal Dopamine System in Learning to Perform Sequential Motor Tasks in a Predictive Manner , 2000 .

[34]  J. Bocquet-Appel,et al.  Neanderthal contraction and modern human colonization of Europe , 2000, Antiquity.

[35]  L. Straus,et al.  The Upper Palaeolithic settlement of Iberia: first-generation maps , 2000, Antiquity.

[36]  W. Schultz Multiple reward signals in the brain , 2000, Nature Reviews Neuroscience.

[37]  J. Wickens,et al.  Substantia nigra dopamine regulates synaptic plasticity and membrane potential fluctuations in the rat neostriatum, in vivo , 2000, Neuroscience.