Reward-Dependent Modulation of Neuronal Activity in the Primate Dorsal Raphe Nucleus

The dopamine system has been thought to play a central role in guiding behavior based on rewards. Recent pharmacological studies suggest that another monoamine neurotransmitter, serotonin, is also involved in reward processing. To elucidate the functional relationship between serotonin neurons and dopamine neurons, we performed single-unit recording in the dorsal raphe nucleus (DRN), a major source of serotonin, and the substantia nigra pars compacta, a major source of dopamine, while monkeys performed saccade tasks in which the position of the target indicated the size of an upcoming reward. After target onset, but before reward delivery, the activity of many DRN neurons was modulated tonically by the expected reward size with either large- or small-reward preference, whereas putative dopamine neurons had phasic responses and only preferred large rewards. After reward delivery, the activity of DRN neurons was modulated tonically by the received reward size with either large- or small-reward preference, whereas the activity of dopamine neurons was not modulated except after the unexpected reversal of the position–reward contingency. Thus, DRN neurons encode the expected and received rewards, whereas dopamine neurons encode the difference between the expected and received rewards. These results suggest that the DRN, probably including serotonin neurons, signals the reward value associated with the current behavior.

[1]  R. Solomon,et al.  An opponent-process theory of motivation. I. Temporal dynamics of affect. , 1974, Psychological review.

[2]  Ronald M. Harper,et al.  Dorsal raphe neurons: depression of firing during sleep in cats , 1976, Brain Research.

[3]  A. Dray,et al.  Evidence for the existence of a raphe projection to the substantia nigra in rat , 1976, Brain Research.

[4]  M. Jouvet,et al.  Afferent connections of the nucleus raphe dorsalis in the cat as visualized by the horseradish peroxidase technique , 1977, Brain Research.

[5]  G. Aghajanian,et al.  Habenular and other midbrain raphe afferents demonstrated by a modified retrograde tracing technique , 1977, Brain Research.

[6]  G. Aghajanian,et al.  Physiological evidence for habenula as major link between forebrain and midbrain raphe. , 1977, Science.

[7]  G. Aghajanian,et al.  Serotonergic and non-serotonergic neurons of the dorsal raphe: reciprocal changes in firing induced by peripheral nerve stimulation , 1978, Brain Research.

[8]  R. Solomon,et al.  An Opponent-Process Theory of Motivation , 1978 .

[9]  Warren C. Stern,et al.  Effects of electrical stimulation of the lateral habenula on single-unit activity of raphe neurons , 1979, Experimental Neurology.

[10]  Masataka Watanabe,et al.  Prefrontal and cingulate unit activity during timing behavior in the monkey , 1979, Brain Research.

[11]  T. Hattori,et al.  Dorsal raphe cells with collateral projections to the caudate-putamen and substantia nigra: A fluorescent retrograde double labeling study in the rat , 1980, Brain Research.

[12]  L. Wiklund,et al.  Monoamine cell distribution in the cat brain stem. A fluorescence histochemical study with quantification of indolaminergic and locus coeruleus cell groups , 1981, The Journal of comparative neurology.

[13]  A. Beaudet,et al.  The serotonin neurons in nucleus raphe dorsalis of adult rat: A light and electron microscope radioautographic study , 1982, The Journal of comparative neurology.

[14]  L. Wiklund,et al.  Distribution and numbers of indoleamine cell bodies in the cat brainstem determined with Falck-Hillarp fluorescence histochemistry , 1982, Brain Research Bulletin.

[15]  J. A. Hobson,et al.  The time-course of dorsal raphe discharge, PGO waves, and muscle tone averaged across multiple sleep cycles , 1983, Brain Research.

[16]  P. Goldman-Rakic,et al.  Selective prefrontal cortical projections to the region of the locus coeruleus and raphe nuclei in the rhesus monkey , 1984, Brain Research.

[17]  J. Tepper,et al.  Antidromic activation of dorsal raphe neurons from neostriatum: Physiological characterization and effects of terminal autoreceptor activation , 1985, Brain Research.

[18]  S. T. Kitai,et al.  The organization of divergent axonal projections from the midbrain raphe nuclei in the rat , 1986, The Journal of comparative neurology.

[19]  Hidehiko Komatsu,et al.  A grid system and a microsyringe for single cell recording , 1988, Journal of Neuroscience Methods.

[20]  Chang-Tien Tsai Involvement of serotonin in mediation of inhibition of substantia nigra neurons by noxious stimuli , 1989, Brain Research Bulletin.

[21]  T. Insel,et al.  Serotonin in Obsessions, Compulsions, and the Control of Aggressive Impulses , 1990, Annals of the New York Academy of Sciences.

[22]  A. Parent,et al.  Immunohistochemical study of the serotoninergic innervation of the basal ganglia in the squirrel monkey , 1990, The Journal of comparative neurology.

[23]  L. Wiklund,et al.  Afferents to the median raphe nucleus of the rat: Retrograde cholera toxin and wheat germ conjugated horseradish peroxidase tracing, and selectived-[3H]aspartate labelling of possible excitatory amino acid inputs , 1990, Neuroscience.

[24]  G. Curzon Serotonin and Appetite , 1990, Annals of the New York Academy of Sciences.

[25]  G. M. Halliday,et al.  Distribution, morphology and number of monoamine-synthesizing and substance P-containing neurons in the human dorsal raphe nucleus , 1991, Neuroscience.

[26]  B. Jacobs,et al.  Activity of brain serotonergic neurons in the behaving animal. , 1991, Pharmacological reviews.

[27]  J. Deakin Depression and 5HT , 1991, International clinical psychopharmacology.

[28]  R. Vertes A PHA‐L analysis of ascending projections of the dorsal raphe nucleus in the rat , 1991, The Journal of comparative neurology.

[29]  B. Jacobs,et al.  Structure and function of the brain serotonin system. , 1992, Physiological reviews.

[30]  G. Doucet,et al.  Ultrastructure and synaptic targets of the raphe-nigral projection in the rat , 1993, Neuroscience.

[31]  B. Jacobs,et al.  5-HT and motor control: a hypothesis , 1993, Trends in Neurosciences.

[32]  W. Schultz,et al.  Importance of unpredictability for reward responses in primate dopamine neurons. , 1994, Journal of neurophysiology.

[33]  M. Linnoila,et al.  Low CSF 5-HIAA concentrations and severe aggression and impaired impulse control in nonhuman primates. , 1994, The American journal of psychiatry.

[34]  P. Dayan,et al.  A framework for mesencephalic dopamine systems based on predictive Hebbian learning , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[35]  F. Graeff,et al.  Role of 5-HT in stress, anxiety, and depression , 1996, Pharmacology Biochemistry and Behavior.

[36]  S. Kapur,et al.  Serotonin-dopamine interaction and its relevance to schizophrenia. , 1996, The American journal of psychiatry.

[37]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[38]  R. Hen,et al.  Insights into the Neurobiology of Impulsive Behavior from Serotonin Receptor Knockout Mice , 1997, Annals of the New York Academy of Sciences.

[39]  C. Rampon,et al.  Forebrain afferents to the rat dorsal raphe nucleus demonstrated by retrograde and anterograde tracing methods , 1997, Neuroscience.

[40]  Barry J. Everitt,et al.  Central 5-HT depletion enhances impulsive responding without affecting the accuracy of attentional performance: interactions with dopaminergic mechanisms , 1997, Psychopharmacology.

[41]  M. Hajós,et al.  An electrophysiological and neuroanatomical study of the medial prefrontal cortical projection to the midbrain raphe nuclei in the rat , 1998, Neuroscience.

[42]  W. Schultz,et al.  Learning of sequential movements by neural network model with dopamine-like reinforcement signal , 1998, Experimental Brain Research.

[43]  J. Hollerman,et al.  Dopamine neurons report an error in the temporal prediction of reward during learning , 1998, Nature Neuroscience.

[44]  S. Maier,et al.  Escapable and inescapable stress differentially and selectively alter extracellular levels of 5-HT in the ventral hippocampus and dorsal periaqueductal gray of the rat , 1998, Brain Research.

[45]  O. Hikosaka,et al.  Expectation of reward modulates cognitive signals in the basal ganglia , 1998, Nature Neuroscience.

[46]  A. Meneses,et al.  5-HT system and cognition , 1999, Neuroscience & Biobehavioral Reviews.

[47]  T. Robbins,et al.  Tryptophan depletion impairs stimulus-reward learning while methylphenidate disrupts attentional control in healthy young adults: implications for the monoaminergic basis of impulsive behaviour , 1999, Psychopharmacology.

[48]  K. Miczek,et al.  Aggressive Behavior, Increased Accumbal Dopamine, and Decreased Cortical Serotonin in Rats , 2000, The Journal of Neuroscience.

[49]  R. Davidson,et al.  Dysfunction in the neural circuitry of emotion regulation--a possible prelude to violence. , 2000, Science.

[50]  S. Mobini,et al.  Effect of central 5-hydroxytryptamine depletion on inter-temporal choice: a quantitative analysis , 2000, Psychopharmacology.

[51]  S. Mobini,et al.  Effects of central 5-hydroxytryptamine depletion on sensitivity to delayed and probabilistic reinforcement , 2000, Psychopharmacology.

[52]  T. Maeda,et al.  Distribution of dopamine-immunoreactive fibers in the rat brainstem , 2000, Journal of Chemical Neuroanatomy.

[53]  R. Drucker-Colín,et al.  Discharge modulation of rat dorsal raphe neurons during sleep and waking: effects of preoptic/basal forebrain warming , 2000, Brain Research.

[54]  C. Dugovic Role of serotonin in sleep mechanisms. , 2001, Revue neurologique.

[55]  M. Hajós,et al.  Evidence for a role of GABA interneurones in the cortical modulation of midbrain 5-hydroxytryptamine neurones , 2001, Neuroscience.

[56]  S. Haj-Dahmane D2‐like dopamine receptor activation excites rat dorsal raphe 5‐HT neurons in vitro , 2001, The European journal of neuroscience.

[57]  T. Robbins,et al.  Impulsive Choice Induced in Rats by Lesions of the Nucleus Accumbens Core , 2001, Science.

[58]  P. Hof,et al.  Anatomical distribution of serotonin‐containing neurons and axons in the central nervous system of the cat , 2001, The Journal of comparative neurology.

[59]  P. Celada,et al.  Control of serotonergic neurons in the dorsal raphe nucleus by the lateral hypothalamus , 2002, Brain Research.

[60]  O. Hikosaka,et al.  A neural correlate of response bias in monkey caudate nucleus , 2002, Nature.

[61]  Sham M. Kakade,et al.  Opponent interactions between serotonin and dopamine , 2002, Neural Networks.

[62]  M. Yanagisawa,et al.  Sleep, feeding, and neuropeptides: roles of orexins and orexin receptors , 2002, Current Opinion in Neurobiology.

[63]  Okihide Hikosaka,et al.  Role of Primate Substantia Nigra Pars Reticulata in Reward-Oriented Saccadic Eye Movement , 2002, The Journal of Neuroscience.

[64]  Kenji Doya,et al.  Metalearning and neuromodulation , 2002, Neural Networks.

[65]  P. Fletcher,et al.  Serotonin and drug reward: focus on 5-HT2C receptors. , 2003, European journal of pharmacology.

[66]  T. Sharp,et al.  Neurochemical and anatomical identification of fast- and slow-firing neurones in the rat dorsal raphe nucleus using juxtacellular labelling methods in vivo , 2003, Neuroscience.

[67]  B. Kocsis,et al.  Electrophysiological evidence for convergence of inputs from the medial prefrontal cortex and lateral habenula on single neurons in the dorsal raphe nucleus , 2003, The European journal of neuroscience.

[68]  Shigeru Shinomoto,et al.  Differences in Spiking Patterns Among Cortical Neurons , 2003, Neural Computation.

[69]  Tatsuo K Sato,et al.  Correlated Coding of Motivation and Outcome of Decision by Dopamine Neurons , 2003, The Journal of Neuroscience.

[70]  F. Graeff Serotonin, the periaqueductal gray and panic , 2004, Neuroscience & Biobehavioral Reviews.

[71]  Saori C. Tanaka,et al.  Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops , 2004, Nature Neuroscience.

[72]  O. Hikosaka,et al.  Dopamine Neurons Can Represent Context-Dependent Prediction Error , 2004, Neuron.

[73]  Samuel M. McClure,et al.  The Neural Substrates of Reward Processing in Humans: The Modern Role of fMRI , 2004, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[74]  O. Hikosaka,et al.  A possible role of midbrain dopamine neurons in short- and long-term adaptation of saccades to position-reward mapping. , 2004, Journal of neurophysiology.

[75]  J. Tepper,et al.  Dorsal raphé stimulation modifies striatal-evoked antidromic invasion of nigral dopaminergic neurons in vivo , 1991, Experimental Brain Research.

[76]  O. Hikosaka,et al.  Reward-predicting activity of dopamine and caudate neurons--a possible mechanism of motivational control of saccadic eye movement. , 2004, Journal of neurophysiology.

[77]  M. Walton,et al.  Differential involvement of serotonin and dopamine systems in cost-benefit decisions about delay or effort , 2005, Psychopharmacology.

[78]  Trevor W Robbins,et al.  Fractionating Impulsivity: Contrasting Effects of Central 5-HT Depletion on Different Measures of Impulsive Behavior , 2004, Neuropsychopharmacology.

[79]  P. Fletcher,et al.  Conditioned place preference induced by microinjection of 8-OH-DPAT into the dorsal or median raphe nucleus , 2005, Psychopharmacology.

[80]  O. Hikosaka,et al.  Immediate changes in anticipatory activity of caudate neurons associated with reversal of position-reward contingency. , 2005, Journal of neurophysiology.

[81]  K. Fuxe,et al.  Localization of monoamines in the lower brain stem , 1964, Experientia.

[82]  C. M. Bradshaw,et al.  Effect of lesions of the ascending 5-hydroxytryptaminergic pathways on choice between delayed reinforcers , 2005, Psychopharmacology.

[83]  G. Gerstein,et al.  Measurement of time-dependent changes in the irregularity of neural spiking. , 2006, Journal of neurophysiology.

[84]  Viktor Varga,et al.  Serotonergic neuron diversity: identification of raphe neurons with discharges time-locked to the hippocampal theta rhythm. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[85]  T. Robbins,et al.  Double dissociation between serotonergic and dopaminergic modulation of medial prefrontal and orbitofrontal cortex during a test of impulsive choice. , 2006, Cerebral cortex.

[86]  Kae Nakamura,et al.  Basal ganglia orient eyes to reward. , 2006, Journal of neurophysiology.

[87]  G. Aston-Jones,et al.  Arousal and reward: a dichotomy in orexin function , 2006, Trends in Neurosciences.

[88]  R. McCarley,et al.  Differential effect of orexins (hypocretins) on serotonin release in the dorsal and median raphe nuclei of freely behaving rats , 2006, Neuroscience.

[89]  Kae Nakamura,et al.  Role of Dopamine in the Primate Caudate Nucleus in Reward Modulation of Saccades , 2006, The Journal of Neuroscience.

[90]  E. Procyk,et al.  Reward encoding in the monkey anterior cingulate cortex. , 2006, Cerebral cortex.

[91]  Christoph Schmitz,et al.  The dorsal raphe nucleus—From silver stainings to a role in depression , 2007, Brain Research Reviews.

[92]  O. Hikosaka,et al.  Lateral habenula as a source of negative reward signals in dopamine neurons , 2007, Nature.

[93]  Saori C. Tanaka,et al.  Serotonin and the Evaluation of Future Rewards , 2007, Annals of the New York Academy of Sciences.

[94]  S. Ikemoto,et al.  The midbrain raphe nuclei mediate primary reinforcement via GABAA receptors , 2007, The European journal of neuroscience.