Striatal Contributions to Reward and Decision Making

Abstract:  The striatum is the major input nucleus of the basal ganglia. It is thought to play a key role in learning on the basis of positive reinforcement and in action selection. One view of the striatum conceives it as comprising a reiterated matrix of processing units that perform common operations in different striatal regions, namely synaptic plasticity according to a three‐factor rule, and lateral inhibition. These operations are required for reinforcement learning and selection of previously reinforced actions. Analysis of the behavioral effects of circumscribed lesions of the striatum, however, suggests regional specialization of learning and decision‐making operations. We consider how a basic processing unit may be modified by regional variations in neurochemical parameters, for example, by the gradient in density of dopamine terminals from dorsal to ventral striatum. These variations suggest subtle differences between dorsolateral and ventromedial striatal regions in the temporal properties of dopamine signaling, which are superimposed on regional differences in connectivity. We propose that these variations make sense in relation to the temporal structure of activity in striatal inputs from different regions, and the requirements of different learning operations. Dorsolateral striatal (DLS) regions may be subject to brief, precisely timed pulses of dopamine, whereas ventromedial striatal regions integrate dopamine signals over a longer time course. These differences may be important for understanding regional variations in the contribution to reinforcement of habits, versus incentive processes that are sensitive to the value of expected rewards.

[1]  K. Fuxe,et al.  EVIDENCE FOR THE EXISTENCE OF MONOAMINE-CONTAINING NEURONS IN THE CENTRAL NERVOUS SYSTEM. I. DEMONSTRATION OF MONOAMINES IN THE CELL BODIES OF BRAIN STEM NEURONS. , 1964, Acta physiologica Scandinavica. Supplementum.

[2]  H. E. Rosvold,et al.  Behavioral effects of selective ablation of the caudate nucleus. , 1967, Journal of comparative and physiological psychology.

[3]  S. Ochs Integrative Activity of the Brain: An Interdisciplinary Approach , 1968 .

[4]  U. Ungerstedt Stereotaxic mapping of the monoamine pathways in the rat brain. , 1971, Acta physiologica Scandinavica. Supplementum.

[5]  D. Bindra A motivational view of learning, performance, and behavior modification. , 1974, Psychological review.

[6]  H. Nauta,et al.  Afferents to the rat caudoputamen studied with horseradish peroxidase. An evaluation of a retrograde neuroanatomical research method. , 1974, Brain research.

[7]  C. W. Ragsdale,et al.  Histochemically distinct compartments in the striatum of human, monkeys, and cat demonstrated by acetylthiocholinesterase staining. , 1978, Proceedings of the National Academy of Sciences of the United States of America.

[8]  O. Phillipson The cytoarchitecture of the interfascicular nucleus and ventral tegmental area of tsai in the rat , 1979, The Journal of comparative neurology.

[9]  Douglas L. Jones,et al.  From motivation to action: Functional interface between the limbic system and the motor system , 1980, Progress in Neurobiology.

[10]  P. Somogyi,et al.  Monosynaptic cortical input and local axon collaterals of identified striatonigral neurons. A light and electron microscopic study using the golgi‐peroxidase transport‐degeneration procedure , 1981, The Journal of comparative neurology.

[11]  Charles J. Wilson,et al.  Spontaneous firing patterns of identified spiny neurons in the rat neostriatum , 1981, Brain Research.

[12]  G. Arbuthnott,et al.  The pattern of innervation of the corpus striatum by the substantia nigra , 1981, Neuroscience.

[13]  M. Herkenham,et al.  Mosaic distribution of opiate receptors, parafascicular projections and acetylcholinesterase in rat striatum , 1981, Nature.

[14]  Christopher D. Adams,et al.  The Effect of the Instrumental Training Contingency on Susceptibility to Reinforcer Devaluation , 1983 .

[15]  D. German,et al.  Activity of mesencephalic dopamine and non-dopamine neurons across stages of sleep and waking in the rat , 1983, Brain Research.

[16]  P. Lovibond Facilitation of instrumental behavior by a Pavlovian appetitive conditioned stimulus. , 1983, Journal of experimental psychology. Animal behavior processes.

[17]  A. Graybiel Correspondence between the Dopamine islands and striosomes of the mammalian striatum , 1984, Neuroscience.

[18]  C. Gerfen The neostriatal mosaic: compartmentalization of corticostriatal input and striatonigral output systems , 1984, Nature.

[19]  R. Rescorla,et al.  Postconditioning devaluation of a reinforcer affects instrumental responding. , 1985 .

[20]  H. T. Chang,et al.  Projection neurons of the nucleus accumbens: an intracellular labeling study , 1985, Brain Research.

[21]  L. Descarries,et al.  Quantification of the dopamine innervation in adult rat neostriatum , 1986, Neuroscience.

[22]  G. E. Alexander,et al.  Parallel organization of functionally segregated circuits linking basal ganglia and cortex. , 1986, Annual review of neuroscience.

[23]  R. Malach,et al.  Mosaic architecture of the somatic sensory-recipient sector of the cat's striatum , 1986, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[24]  W. Schultz Responses of midbrain dopamine neurons to behavioral trigger stimuli in the monkey. , 1986, Journal of neurophysiology.

[25]  A. Graybiel,et al.  Subdivisions of the dopamine-containing A8-A9-A10 complex identified by their differential mesostriatal innervation of striosomes and extrastriosomal matrix , 1987, Neuroscience.

[26]  C. Gerfen,et al.  The neostriatal mosaic: II. Patch- and matrix-directed mesostriatal dopaminergic and non-dopaminergic systems , 1987, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[27]  R. Rescorla,et al.  Associations between the discriminative stimulus and the reinforcer in instrumental learning. , 1988 .

[28]  A. D. Smith,et al.  Convergence of hippocampal and dopaminergic input onto identified neurons in the nucleus accumbens of the rat. , 1989, Journal of chemical neuroanatomy.

[29]  C. Gerfen The neostriatal mosaic: striatal patch-matrix organization is related to cortical lamination. , 1989, Science.

[30]  A. Mcgeorge,et al.  The organization of the projection from the cerebral cortex to the striatum in the rat , 1989, Neuroscience.

[31]  C. W. Ragsdale,et al.  A simple ordering of neocortical areas established by the compartmental organization of their striatal projections. , 1990, Proceedings of the National Academy of Sciences of the United States of America.

[32]  S. Sesack,et al.  In the rat medial nucleus accumbens, hippocampal and catecholaminergic terminals converge on spiny neurons and are in apposition to each other , 1990, Brain Research.

[33]  W. Schultz,et al.  Responses of monkey midbrain dopamine neurons during delayed alternation performance , 1991, Brain Research.

[34]  P. Levitt,et al.  Heterogeneous distribution of the limbic system-associated membrane protein in the caudate nucleus and substantia nigra of the cat , 1991, Neuroscience.

[35]  S. Sesack,et al.  Prefrontal cortical efferents in the rat synapse on unlabeled neuronal targets of catecholamine terminals in the nucleus accumbens septi and on dopamine neurons in the ventral tegmental area , 1992, The Journal of comparative neurology.

[36]  W. Schultz,et al.  Responses of monkey dopamine neurons during learning of behavioral reactions. , 1992, Journal of neurophysiology.

[37]  H. Groenewegen,et al.  Morphological differences between projection neurons of the core and shell in the nucleus accumbens of the rat , 1992, Neuroscience.

[38]  K. Berridge,et al.  The neural basis of drug craving: An incentive-sensitization theory of addiction , 1993, Brain Research Reviews.

[39]  D. S. Zahm,et al.  The patterns of afferent innervation of the core and shell in the “Accumbens” part of the rat ventral striatum: Immunohistochemical detection of retrogradely transported fluoro‐gold , 1993, The Journal of comparative neurology.

[40]  F. H. Lopes da Silva,et al.  Synaptic Plasticity in an In Vitro Slice Preparation of the Rat Nucleus Accumbens , 1993, The European journal of neuroscience.

[41]  W. Schultz,et al.  Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[42]  A. Grace,et al.  Physiological and morphological properties of accumbens core and shell neurons recorded in vitro , 1993, Synapse.

[43]  C. Y. Yim,et al.  Rhythmic delta-frequency activities in the nucleus accumbens of anesthetized and freely moving rats. , 1993, Canadian journal of physiology and pharmacology.

[44]  A. Parent,et al.  Synaptic relationships between dopaminergic afferents and cortical or thalamic input in the sensorimotor territory of the striatum in monkey , 1994, The Journal of comparative neurology.

[45]  B. Balleine,et al.  Effects of ibotenic acid lesions of the Nucleus Accumbens on instrumental action , 1994, Behavioural Brain Research.

[46]  P. Garris,et al.  Different kinetics govern dopaminergic transmission in the amygdala, prefrontal cortex, and striatum: an in vivo voltammetric study , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[47]  W. Schultz,et al.  Importance of unpredictability for reward responses in primate dopamine neurons. , 1994, Journal of neurophysiology.

[48]  S. Totterdell,et al.  Input from the amygdala to the rat nucleus accumbens: Its relationship with tyrosine hydroxylase immunoreactivity and identified neurons , 1994, Neuroscience.

[49]  Charles J. Wilson,et al.  Surround inhibition among projection neurons is weak or nonexistent in the rat neostriatum. , 1994, Journal of neurophysiology.

[50]  R. Boakes,et al.  Motivational control after extended instrumental training , 1995 .

[51]  P. Garris,et al.  Regional Differences in Dopamine Release, Uptake, and Diffusion Measured by Fast-Scan Cyclic Voltammetry , 1995 .

[52]  K. Chergui,et al.  Uptake of Dopamine Released by Impulse Flow in the Rat Mesolimbic and Striatal Systems In Vivo , 1995, Journal of neurochemistry.

[53]  Joel L. Davis,et al.  Adaptive Critics and the Basal Ganglia , 1995 .

[54]  W. Schultz,et al.  Preferential activation of midbrain dopamine neurons by appetitive rather than aversive stimuli , 1996, Nature.

[55]  Charles J. Wilson,et al.  The origins of two-state spontaneous membrane potential fluctuations of neostriatal spiny neurons , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[56]  A. D. Smith,et al.  Synaptic Connections Between Spiny Neurons of the Direct and Indirect Pathways in the Neostriatum of the Rat: Evidence from Dopamine Receptor and Neuropeptide Immunostaining , 1996, The European journal of neuroscience.

[57]  S. Smith‐Roe,et al.  Response-reinforcement learning is dependent on N-methyl-D-aspartate receptor activation in the nucleus accumbens core. , 1997, Proceedings of the National Academy of Sciences of the United States of America.

[58]  V. Pickel,et al.  The Dopamine Transporter: Comparative Ultrastructure of Dopaminergic Axons in Limbic and Motor Compartments of the Nucleus Accumbens , 1997, The Journal of Neuroscience.

[59]  J. Salamone,et al.  The Role of Accumbens Dopamine in Lever Pressing and Response Allocation: Effects of 6-OHDA Injected into Core and Dorsomedial Shell , 1998, Pharmacology Biochemistry and Behavior.

[60]  J. Hollerman,et al.  Influence of reward expectation on behavior-related neuronal activity in primate striatum. , 1998, Journal of neurophysiology.

[61]  Behavioural topography in the striatum: differential effects of quinpirole and D-amphetamine microinjections. , 1998, European journal of pharmacology.

[62]  Charles J. Wilson,et al.  Membrane potential synchrony of simultaneously recorded striatal spiny neurons in vivo , 1998, Nature.

[63]  K. Berridge,et al.  What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? , 1998, Brain Research Reviews.

[64]  G. Arbuthnott,et al.  Double anterograde tracing of outputs from adjacent “barrel columns” of rat somatosensory cortex. Neostriatal projection patterns and terminal ultrastructure , 1999, Neuroscience.

[65]  D. S. Zahm,et al.  Functional‐anatomical Implications of the Nucleus Accumbens Core and Shell Subterritories , 1999, Annals of the New York Academy of Sciences.

[66]  G E Meredith,et al.  The Synaptic Framework for Chemical Signaling in Nucleus Accumbens , 1999, Annals of the New York Academy of Sciences.

[67]  T. Robbins,et al.  Dissociation in Effects of Lesions of the Nucleus Accumbens Core and Shell on Appetitive Pavlovian Approach Behavior and the Potentiation of Conditioned Reinforcement and Locomotor Activity byd-Amphetamine , 1999, The Journal of Neuroscience.

[68]  J. Mirenowicz,et al.  Dissociation of Pavlovian and instrumental incentive learning under dopamine antagonists. , 2000, Behavioral neuroscience.

[69]  L. Heimer Basal forebrain in the context of schizophrenia , 2000, Brain Research Reviews.

[70]  R. Malenka,et al.  Dopaminergic modulation of neuronal excitability in the striatum and nucleus accumbens. , 2000, Annual review of neuroscience.

[71]  Nikolaus R. McFarland,et al.  Striatonigrostriatal Pathways in Primates Form an Ascending Spiral from the Shell to the Dorsolateral Striatum , 2000, The Journal of Neuroscience.

[72]  T. Robbins,et al.  Disconnection of the anterior cingulate cortex and nucleus accumbens core impairs Pavlovian approach behavior: further evidence for limbic cortical-ventral striatopallidal systems. , 2000, Behavioral neuroscience.

[73]  J. Partridge,et al.  Regional and postnatal heterogeneity of activity-dependent long-term changes in synaptic efficacy in the dorsal striatum. , 2000, Journal of neurophysiology.

[74]  J. Bolam,et al.  Synaptic organisation of the basal ganglia , 2000, Journal of anatomy.

[75]  J. Wickens,et al.  Dopamine D-1/D-5 receptor activation is required for long-term potentiation in the rat neostriatum in vitro. , 2001, Journal of neurophysiology.

[76]  J. Wickens,et al.  A cellular mechanism of reward-related learning , 2001, Nature.

[77]  J. Galen Buckwalter,et al.  Regional differences in the expression of corticostriatal synaptic plasticity , 2001, Neuroscience.

[78]  B. Balleine,et al.  The Role of the Nucleus Accumbens in Instrumental Conditioning: Evidence of a Functional Dissociation between Accumbens Core and Shell , 2001, The Journal of Neuroscience.

[79]  W. Schultz,et al.  Dopamine responses comply with basic assumptions of formal learning theory , 2001, Nature.

[80]  R. Cardinal Neuropsychology of reinforcement processes in the rat , 2001 .

[81]  T. Robbins,et al.  Impulsive Choice Induced in Rats by Lesions of the Nucleus Accumbens Core , 2001, Science.

[82]  S. Floresco,et al.  Modulation of Hippocampal and Amygdalar-Evoked Activity of Nucleus Accumbens Neurons by Dopamine: Cellular Mechanisms of Input Selection , 2001, The Journal of Neuroscience.

[83]  B. Hyland,et al.  Firing modes of midbrain dopamine cells in the freely moving rat , 2002, Neuroscience.

[84]  Dietmar Plenz,et al.  Fast synaptic transmission between striatal spiny projection neurons , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[85]  B. Knowlton,et al.  Learning and memory functions of the Basal Ganglia. , 2002, Annual review of neuroscience.

[86]  B. Everitt,et al.  Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex , 2002, Neuroscience & Biobehavioral Reviews.

[87]  Jeffery R Wickens,et al.  Inhibitory interactions between spiny projection neurons in the rat striatum. , 2002, Journal of neurophysiology.

[88]  John N. J. Reynolds,et al.  Dopamine-dependent plasticity of corticostriatal synapses , 2002, Neural Networks.

[89]  O. Hikosaka,et al.  Reward-dependent spatial selectivity of anticipatory activity in monkey caudate neurons. , 2002, Journal of neurophysiology.

[90]  J. N. P. Rawlins,et al.  Effects of cytotoxic nucleus accumbens lesions on instrumental conditioning in rats , 2002, Experimental Brain Research.

[91]  W. Schultz,et al.  Coding of Predicted Reward Omission by Dopamine Neurons in a Conditioned Inhibition Paradigm , 2003, The Journal of Neuroscience.

[92]  J. Wickens,et al.  Neural mechanisms of reward-related motor learning , 2003, Current Opinion in Neurobiology.

[93]  A. Parent,et al.  Chemical anatomy of the human ventral striatum and adjacent basal forebrain structures , 2003, The Journal of comparative neurology.

[94]  T. Jay Dopamine: a potential substrate for synaptic plasticity and memory mechanisms , 2003, Progress in Neurobiology.

[95]  W. Schultz,et al.  Discrete Coding of Reward Probability and Uncertainty by Dopamine Neurons , 2003, Science.

[96]  L. Heimer A new anatomical framework for neuropsychiatric disorders and drug abuse. , 2003, The American journal of psychiatry.

[97]  J. Hollerman,et al.  Changes in behavior-related neuronal activity in the striatum during learning , 2003, Trends in Neurosciences.

[98]  B. Balleine,et al.  Instrumental and Pavlovian incentive processes have dissociable effects on components of a heterogeneous instrumental chain. , 2003, Journal of experimental psychology. Animal behavior processes.

[99]  Tatsuo K Sato,et al.  Correlated Coding of Motivation and Outcome of Decision by Dopamine Neurons , 2003, The Journal of Neuroscience.

[100]  P. Holland Relations between Pavlovian-instrumental transfer and reinforcer devaluation. , 2004, Journal of experimental psychology. Animal behavior processes.

[101]  Henk J Groenewegen,et al.  Direct physiological evidence for synaptic connectivity between medium-sized spiny neurons in rat nucleus accumbens in situ. , 2004, Journal of neurophysiology.

[102]  B. Knowlton,et al.  Contributions of striatal subregions to place and response learning. , 2004, Learning & memory.

[103]  B. Balleine,et al.  Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning , 2004, The European journal of neuroscience.

[104]  A. Kelley Ventral striatal control of appetitive motivation: role in ingestive behavior and reward-related learning , 2004, Neuroscience & Biobehavioral Reviews.

[105]  Laurent Venance,et al.  Electrical and chemical transmission between striatal GABAergic output neurones in rat brain slices , 2004, The Journal of physiology.

[106]  T. Robbins,et al.  Putting a spin on the dorsal–ventral divide of the striatum , 2004, Trends in Neurosciences.

[107]  M. Delgado,et al.  Modulation of Caudate Activity by Action Contingency , 2004, Neuron.

[108]  T. Robbins,et al.  6-Hydroxydopamine lesions of the nucleus accumbens, but not of the caudate nucleus, attenuate enhanced responding with reward-related stimuli produced by intra-accumbens d-amphetamine , 2004, Psychopharmacology.

[109]  Charles J. Wilson,et al.  Comparison of IPSCs Evoked by Spiny and Fast-Spiking Neurons in the Neostriatum , 2004, The Journal of Neuroscience.

[110]  W. Pan,et al.  Dopamine Cells Respond to Predicted Events during Classical Conditioning: Evidence for Eligibility Traces in the Reward-Learning Network , 2005, The Journal of Neuroscience.

[111]  K. Doya,et al.  Representation of Action-Specific Reward Values in the Striatum , 2005, Science.

[112]  W. Schultz,et al.  Adaptive Coding of Reward Value by Dopamine Neurons , 2005, Science.

[113]  A. Faure,et al.  Lesion to the Nigrostriatal Dopamine System Disrupts Stimulus-Response Habit Formation , 2005, The Journal of Neuroscience.

[114]  J. Wickens,et al.  Striatal dopamine in motor activation and reward-mediated learning: steps towards a unifying model , 2005, Journal of Neural Transmission / General Section JNT.

[115]  B. Everitt,et al.  Involvement of the Dorsal Striatum in Cue-Controlled Cocaine Seeking , 2005, The Journal of Neuroscience.

[116]  B. Balleine,et al.  The role of the dorsomedial striatum in instrumental conditioning , 2005, The European journal of neuroscience.

[117]  W. Pan,et al.  Pedunculopontine Tegmental Nucleus Controls Conditioned Responses of Midbrain Dopamine Neurons in Behaving Rats , 2005, The Journal of Neuroscience.

[118]  B. Balleine,et al.  Blockade of NMDA receptors in the dorsomedial striatum prevents action–outcome learning in instrumental conditioning , 2005, The European journal of neuroscience.

[119]  H. Yin,et al.  The role of the basal ganglia in habit formation , 2006, Nature Reviews Neuroscience.

[120]  Elyssa B. Margolis,et al.  The ventral tegmental area revisited: is there an electrophysiological marker for dopaminergic neurons? , 2006, The Journal of physiology.

[121]  B. Balleine,et al.  Inactivation of dorsolateral striatum enhances sensitivity to changes in the action–outcome contingency in instrumental conditioning , 2006, Behavioural Brain Research.

[122]  A. Phillips,et al.  Dopamine efflux in the nucleus accumbens during within-session extinction, outcome-dependent, and habit-based instrumental responding for food reward , 2006, Psychopharmacology.

[123]  Jeffery R Wickens,et al.  Simulation of GABA function in the basal ganglia: computational models of GABAergic mechanisms in basal ganglia function. , 2007, Progress in brain research.