Book Review: Reward Signaling by Dopamine Neurons

Dopamine projections from the midbrain to the striatum and frontal cortex are involved in behavioral reactions controlled by rewards, as inferred from deficits in parkinsonism, schizophrenia, and drug addiction. Recent experiments have shown that dopamine neurons are not directly modulated in relation to movements. Rather, they appear to code the rewarding aspects of environmental stimuli. They show short, phasic increases of activity following primary food and liquid rewards (“unconditioned stimuli”) and conditioned, reward-predicting stimuli of visual, auditory, and somatosensory modalities. They also display smaller activation-depression sequences after stimuli resembling rewards and after novel or particularly intense stimuli. Rewards are only reported as far as they occur differently than predicted. According to learning theories, a “prediction error” message may constitute a powerful teaching signal for behavior and learning. The phasic reward message is different from the more tonic enabling function of dopamine that is deficient in Parkinson’s disease, indicating that dopamine neurons subserve different functions at different time scales. Neurons in other brain structures, such as the striatum, orbitofrontal cortex, and amygdala, code the quality, quantity, and preference of rewards. The dopamine reward prediction error signal may cooperate with these reward perception signals during the learning and performance of behavioral reactions to motivating environmental stimuli.

[1]  K. Tang,et al.  Dopamine-dependent synaptic plasticity in striatum during in vivo development. , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[2]  J. Wickens,et al.  Dopamine D-1/D-5 receptor activation is required for long-term potentiation in the rat neostriatum in vitro. , 2001, Journal of neurophysiology.

[3]  W. Schultz Multiple reward signals in the brain , 2000, Nature Reviews Neuroscience.

[4]  R. Carelli,et al.  Evidence That Separate Neural Circuits in the Nucleus Accumbens Encode Cocaine Versus “Natural” (Water and Food) Reward , 2000, The Journal of Neuroscience.

[5]  A. Dickinson,et al.  Neuronal coding of prediction errors. , 2000, Annual review of neuroscience.

[6]  J. Desce,et al.  Dopamine Receptors and Groups I and II mGluRs Cooperate for Long-Term Depression Induction in Rat Prefrontal Cortex through Converging Postsynaptic Activation of MAP Kinases , 1999, The Journal of Neuroscience.

[7]  W. Schultz,et al.  A neural network model with dopamine-like reinforcement signal that learns a spatial delayed response task , 1999, Neuroscience.

[8]  F. Guarraci,et al.  An electrophysiological characterization of ventral tegmental area dopaminergic neurons during differential pavlovian fear conditioning in the awake rabbit , 1999, Behavioural Brain Research.

[9]  M. West,et al.  Phasic Firing Time Locked to Cocaine Self-Infusion and Locomotion: Dissociable Firing Patterns of Single Nucleus Accumbens Neurons in the Rat , 1998, The Journal of Neuroscience.

[10]  O. Hikosaka,et al.  Expectation of reward modulates cognitive signals in the basal ganglia , 1998, Nature Neuroscience.

[11]  J. Hollerman,et al.  Influence of reward expectation on behavior-related neuronal activity in primate striatum. , 1998, Journal of neurophysiology.

[12]  J. Hollerman,et al.  Dopamine neurons report an error in the temporal prediction of reward during learning , 1998, Nature Neuroscience.

[13]  W. Schultz,et al.  Learning of sequential movements by neural network model with dopamine-like reinforcement signal , 1998, Experimental Brain Research.

[14]  F. Gonon Prolonged and Extrasynaptic Excitatory Action of Dopamine Mediated by D1 Receptors in the Rat Striatum In Vivo , 1997, The Journal of Neuroscience.

[15]  P. Calabresi,et al.  Abnormal Synaptic Plasticity in the Striatum of Mice Lacking Dopamine D2 Receptors , 1997, The Journal of Neuroscience.

[16]  J. Horvitz,et al.  Burst activity of ventral tegmental dopamine neurons is elicited by sensory stimuli in the awake cat , 1997, Brain Research.

[17]  Mark O West,et al.  Operant behavior during sessions of intravenous cocaine infusion is necessary and sufficient for phasic firing of single nucleus accumbens neurons , 1997, Brain Research.

[18]  R. Elliott,et al.  Effects of methylphenidate on spatial working memory and planning in healthy young adults , 1997, Psychopharmacology.

[19]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[20]  M. West,et al.  Low-dose amphetamine elevates movement-related firing of rat striatal neurons , 1997, Brain Research.

[21]  T. Robbins,et al.  Neurobehavioural mechanisms of reward and motivation , 1996, Current Opinion in Neurobiology.

[22]  B. Richmond,et al.  Neural signals in the monkey ventral striatum related to motivation for juice and cocaine rewards. , 1996, Journal of neurophysiology.

[23]  P. Dayan,et al.  A framework for mesencephalic dopamine systems based on predictive Hebbian learning , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[24]  P S Goldman-Rakic,et al.  Increased dopamine turnover in the prefrontal cortex impairs spatial working memory performance in rats and monkeys. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[25]  W. Schultz,et al.  Preferential activation of midbrain dopamine neurons by appetitive rather than aversive stimuli , 1996, Nature.

[26]  J. Wickens,et al.  Dopamine reverses the depression of rat corticostriatal synapses which normally follows high-frequency stimulation of cortex In vitro , 1996, Neuroscience.

[27]  P. Garris,et al.  Efflux of dopamine from the synaptic cleft in the nucleus accumbens of the rat brain , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[28]  A. Parent,et al.  Synaptic relationships between dopaminergic afferents and cortical or thalamic input in the sensorimotor territory of the striatum in monkey , 1994, The Journal of comparative neurology.

[29]  D. Woodward,et al.  Electrophysiological and pharmacological evidence for the role of the nucleus accumbens in cocaine self-administration in freely moving rats , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[30]  B. Balleine,et al.  Motivational control of goal-directed action , 1994 .

[31]  Gerald Tesauro,et al.  TD-Gammon, a Self-Teaching Backgammon Program, Achieves Master-Level Play , 1994, Neural Computation.

[32]  P. Garris,et al.  Different kinetics govern dopaminergic transmission in the amygdala, prefrontal cortex, and striatum: an in vivo voltammetric study , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[33]  Robert E. Hampson,et al.  Firing patterns of nucleus accumbens neurons during cocaine self-administration in rats , 1993, Brain Research.

[34]  K. Berridge,et al.  The neural basis of drug craving: An incentive-sensitization theory of addiction , 1993, Brain Research Reviews.

[35]  P. Goldman-Rakic,et al.  Characterization of the dopaminergic innervation of the primate frontal cortex using a dopamine-specific antibody. , 1993, Cerebral cortex.

[36]  W. Schultz,et al.  Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[37]  P. Garris,et al.  Regulation of transient dopamine concentration gradients in the microenvironment surrounding nerve terminals in the rat striatum , 1992, Neuroscience.

[38]  P. Calabresi,et al.  Long-term synaptic depression in the striatum: physiological and pharmacological characterization , 1992, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[39]  Michael D. Doherty,et al.  High-speed chronoamperometric measurements of mesolimbic and nigrostriatal dopamine release associated with repeated daily stress , 1992, Brain Research.

[40]  J. Gray,et al.  Increased dopamine release in vivo in nucleus accumbens and caudate nucleus of the rat during drinking: A microdialysis study , 1992, Neuroscience.

[41]  R. Wise,et al.  Localization of drug reward mechanisms by intracranial injections , 1992, Synapse.

[42]  W. Schultz,et al.  Responses of monkey dopamine neurons during learning of behavioral reactions. , 1992, Journal of neurophysiology.

[43]  W. Schultz,et al.  Dopamine neurons of the monkey midbrain: contingencies of responses to stimuli eliciting immediate behavioral reactions. , 1990, Journal of neurophysiology.

[44]  W. Schultz,et al.  Dopamine neurons of the monkey midbrain: contingencies of responses to active touch during self-initiated arm movements. , 1990, Journal of neurophysiology.

[45]  E. Abercrombie,et al.  Differential Effect of Stress on In Vivo Dopamine Release in Striatum, Nucleus Accumbens, and Medial Frontal Cortex , 1989, Journal of neurochemistry.

[46]  F. Gonon Nonlinear relationship between impulse flow and dopamine released by rat midbrain dopaminergic neurons as studied by in vivo electrochemistry , 1988, Neuroscience.

[47]  W. Schultz,et al.  Responses of nigrostriatal dopamine neurons to high-intensity somatosensory stimulation in the anesthetized monkey. , 1987, Journal of neurophysiology.

[48]  M. Le Moal,et al.  Differential reactivity of dopaminergic neurons in the nucleus accumbens in response to different behavioral situations. An in vivo voltammetric study in free moving rats , 1986, Brain Research.

[49]  W. Schultz Responses of midbrain dopamine neurons to behavioral trigger stimuli in the monkey. , 1986, Journal of neurophysiology.

[50]  B. Jacobs,et al.  Substantia nigra dopaminergic unit activity in behaving cats: Effect of arousal on spontaneous discharge and sensory evoked activity , 1985, Brain Research.

[51]  T. F. Freund,et al.  Tyrosine hydroxylase-immunoreactive boutons in synaptic contact with identified striatonigral neurons, with particular reference to dendritic spines , 1984, Neuroscience.

[52]  M. Chesselet,et al.  Presynaptic regulation of neurotransmitter release in the brain: Facts and hypothesis , 1984, Neuroscience.

[53]  M. D. Crutcher,et al.  Relations between movement and single cell discharge in the substantia nigra of the behaving monkey , 1983, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[54]  D. Jacobowitz,et al.  A primate model of parkinsonism: selective destruction of dopaminergic neurons in the pars compacta of the substantia nigra by N-methyl-4-phenyl-1,2,3,6-tetrahydropyridine. , 1983, Proceedings of the National Academy of Sciences of the United States of America.

[55]  A G Barto,et al.  Toward a modern theory of adaptive networks: expectation and prediction. , 1981, Psychological review.

[56]  H. Simon,et al.  Dopaminergic A10 neurones are involved in cognitive functions , 1980, Nature.

[57]  L. Chiodo,et al.  Reciprocal influences of activating and immobilizing stimuli on the activity of nigrostriatal dopamine neurons , 1979, Brain Research.

[58]  H. E. Rosvold,et al.  Cognitive deficit caused by regional depletion of dopamine in prefrontal cortex of rhesus monkey. , 1979, Science.

[59]  R. Rescorla,et al.  A theory of Pavlovian conditioning : Variations in the effectiveness of reinforcement and nonreinforcement , 1972 .