Choice-selective sequences dominate in cortical relative to thalamic inputs to nucleus accumbens, providing a potential substrate for credit assignment

How are actions linked with subsequent outcomes to guide choices? The nucleus accumbens, which is implicated in this process, receives glutamatergic inputs from the prelimbic cortex and midline regions of the thalamus. However, little is known about what is represented in these input pathways. By comparing these inputs during a reinforcement learning task in mice, we discovered that prelimbic cortical inputs preferentially represent actions and choices, whereas midline thalamic inputs preferentially represent cues. Choice-selective activity in the prelimbic cortical inputs is organized in sequences that persist beyond the outcome. Through computational modeling, we demonstrate that these sequences can support the neural implementation of temporal difference learning, a powerful algorithm to connect actions and outcomes across time. Finally, we test and confirm predictions of our circuit model by direct manipulation of nucleus accumbens input neurons. Thus, we integrate experiment and modeling to suggest a neural solution for credit assignment.

[1]  Jacqueline Gottlieb,et al.  Neural Correlates of Temporal Credit Assignment in the Parietal Lobe , 2014, PloS one.

[2]  Anne E Carpenter,et al.  Neuron-type specific signals for reward and punishment in the ventral tegmental area , 2011, Nature.

[3]  Michael Unser,et al.  A pyramid approach to subpixel registration based on intensity , 1998, IEEE Trans. Image Process..

[4]  W. Pan,et al.  Dopamine Cells Respond to Predicted Events during Classical Conditioning: Evidence for Eligibility Traces in the Reward-Learning Network , 2005, The Journal of Neuroscience.

[5]  P. Glimcher,et al.  Midbrain Dopamine Neurons Encode a Quantitative Reward Prediction Error Signal , 2005, Neuron.

[6]  Mark D Humphries,et al.  An ensemble code in medial prefrontal cortex links prior events to outcomes during learning , 2017, Nature Communications.

[7]  Karl J. Friston,et al.  Temporal Difference Models and Reward-Related Learning in the Human Brain , 2003, Neuron.

[8]  E. Nestler,et al.  Molecular basis of long-term plasticity underlying addiction , 2001, Nature Reviews Neuroscience.

[9]  Jocelyn M. Richard,et al.  Recruitment and disruption of ventral pallidal cue encoding during alcohol seeking , 2019, The European journal of neuroscience.

[10]  Makoto Ito,et al.  Parallel Representation of Value-Based and Finite State-Based Strategies in the Ventral and Dorsal Striatum , 2015, PLoS Comput. Biol..

[11]  W. Schultz,et al.  Responses to reward in monkey dorsal and ventral striatum , 2004, Experimental Brain Research.

[12]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[13]  Liam Paninski,et al.  Efficient and accurate extraction of in vivo calcium signals from microendoscopic video data , 2016, eLife.

[14]  Martin Sarter,et al.  The paraventricular thalamus is a critical mediator of top-down control of cue-motivated behavior in rats , 2019, bioRxiv.

[15]  Bernardo L. Sabatini,et al.  The impact of reporter kinetics on the interpretation of data gathered with fluorescent reporters , 2019 .

[16]  Karl Deisseroth,et al.  An interactive framework for whole-brain maps at cellular resolution , 2017, Nature Neuroscience.

[17]  Ilana B. Witten,et al.  Increased Cocaine Motivation Is Associated with Degraded Spatial and Temporal Representations in IL-NAc Neurons , 2019, Neuron.

[18]  Nicholas A. Steinmetz,et al.  Distributed coding of choice, action, and engagement across the mouse brain , 2019, Nature.

[19]  Hagai Bergman,et al.  Temporal Convergence of Dynamic Cell Assemblies in the Striato-Pallidal Network , 2012, The Journal of Neuroscience.

[20]  Dezhe Z. Jin,et al.  Support for a synaptic chain model of neuronal sequence generation , 2010, Nature.

[21]  S. Lammel,et al.  Nucleus Accumbens Subnuclei Regulate Motivated Behavior via Direct Inhibition and Disinhibition of VTA Dopamine Subpopulations , 2018, Neuron.

[22]  A. Zador,et al.  Selective corticostriatal plasticity during acquisition of an auditory discrimination task , 2014, Nature.

[23]  Il Memming Park,et al.  Encoding and decoding in parietal cortex during sensorimotor decision-making , 2014, Nature Neuroscience.

[24]  Gerald Tesauro,et al.  Practical issues in temporal difference learning , 1992, Machine Learning.

[25]  J. Goldberg,et al.  Songbird Ventral Pallidum Sends Diverse Performance Error Signals to Dopaminergic Midbrain , 2019, Neuron.

[26]  K. Deisseroth,et al.  Phasic Firing in Dopaminergic Neurons Is Sufficient for Behavioral Conditioning , 2009, Science.

[27]  R. Carelli,et al.  The Nucleus Accumbens and Pavlovian Reward Learning , 2007, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[28]  Ilana B. Witten,et al.  Reward prediction error does not explain movement selectivity in DMS-projecting dopamine neurons , 2018, bioRxiv.

[29]  W. Schultz,et al.  Learning of sequential movements by neural network model with dopamine-like reinforcement signal , 1998, Experimental Brain Research.

[30]  L. Peoples,et al.  Firing patterns of accumbal neurons during a pavlovian-conditioned approach task. , 2006, Journal of neurophysiology.

[31]  E. Eskandar,et al.  Prefrontal Neurons Encode a Solution to the Credit-Assignment Problem , 2017, The Journal of Neuroscience.

[32]  Junichi Nakai,et al.  Author response: Two-photon calcium imaging of the medial prefrontal cortex and hippocampus without cortical invasion , 2017 .

[33]  G. Shepherd The Synaptic Organization of the Brain , 1979 .

[34]  Vaughn L. Hetrick,et al.  Mesolimbic Dopamine Signals the Value of Work , 2015, Nature Neuroscience.

[35]  W. Schultz,et al.  A neural network model with dopamine-like reinforcement signal that learns a spatial delayed response task , 1999, Neuroscience.

[36]  Jeremiah Y. Cohen,et al.  Distributed and Mixed Information in Monosynaptic Inputs to Dopamine Neurons , 2016, Neuron.

[37]  Namjung Huh,et al.  Role of dopamine D2 receptors in optimizing choice strategy in a dynamic and uncertain environment , 2014, Front. Behav. Neurosci..

[38]  Sachie K. Ogawa,et al.  Whole-Brain Mapping of Direct Inputs to Midbrain Dopamine Neurons , 2012, Neuron.

[39]  M. Gluck,et al.  Dopaminergic Drugs Modulate Learning Rates and Perseveration in Parkinson's Patients in a Dynamic Foraging Task , 2009, The Journal of Neuroscience.

[40]  J. Sakata,et al.  Social modulation of sequence and syllable variability in adult birdsong. , 2008, Journal of neurophysiology.

[41]  Jung Hoon Sul,et al.  Distinct Roles of Rodent Orbitofrontal and Medial Prefrontal Cortex in Decision Making , 2010, Neuron.

[42]  Samuel Gershman,et al.  Time representation in reinforcement learning models of the basal ganglia , 2014, Front. Comput. Neurosci..

[43]  J. Hollerman,et al.  Dopamine neurons report an error in the temporal prediction of reward during learning , 1998, Nature Neuroscience.

[44]  Asohan Amarasingham,et al.  Internally Generated Cell Assembly Sequences in the Rat Hippocampus , 2008, Science.

[45]  H. Groenewegen,et al.  Subcortical afferents of the nucleus accumbens septi in the cat, studied with retrograde axonal transport of horseradish peroxidase and bisbenzimid , 1980, Neuroscience.

[46]  Martin Sarter,et al.  Author response: The paraventricular thalamus is a critical mediator of top-down control of cue-motivated behavior in rats , 2019 .

[47]  Marcelo G Mattar,et al.  Author response: Reward prediction error does not explain movement selectivity in DMS-projecting dopamine neurons , 2019 .

[48]  Yang Dan,et al.  Cell-Type-Specific Activity in Prefrontal Cortex during Goal-Directed Behavior , 2015, Neuron.

[49]  M. Fee,et al.  A hypothesis for basal ganglia-dependent reinforcement learning in the songbird , 2011, Neuroscience.

[50]  Matthew T. Kaufman,et al.  Single-trial neural dynamics are dominated by richly varied movements , 2019, Nature Neuroscience.

[51]  D. Grandy,et al.  Dopamine D2 receptors mediate two-odor discrimination and reversal learning in C57BL/6 mice , 2004, BMC Neuroscience.

[52]  L. Swanson,et al.  The projections of the ventral tegmental area and adjacent regions: A combined fluorescent retrograde tracer and immunofluorescence study in the rat , 1982, Brain Research Bulletin.

[53]  Ilana B. Witten,et al.  Striatal circuits for reward learning and decision-making , 2019, Nature Reviews Neuroscience.

[54]  Daeyeol Lee,et al.  Prefrontal Coding of Temporally Discounted Values during Intertemporal Choice , 2008, Neuron.

[55]  M. Le Moal,et al.  Alternation behavior, spatial discrimination, and reversal disturbances following 6-hydroxydopamine lesions in the nucleus accumbens of the rat. , 1985, Behavioral and neural biology.

[56]  A. Graybiel,et al.  Neurons in the thalamic CM-Pf complex supply striatal neurons with information about behaviorally significant sensory events. , 2001, Journal of neurophysiology.

[57]  Sam E Benezra,et al.  Supplemental Information Population-level Representation of a Temporal Sequence Underlying Song Production in the Zebra Finch , 2022 .

[58]  T. Robbins,et al.  Involvement of the amygdala in stimulus-reward associations: Interaction with the ventral striatum , 1989, Neuroscience.

[59]  T. Robbins,et al.  The basolateral amygdala-ventral striatal system and conditioned place preference: Further evidence of limbic-striatal interactions underlying reward-related processes , 1991, Neuroscience.

[60]  Yanhua Qiao,et al.  N-methyl-D-aspartate receptor-mediated glutamate transmission in nucleus accumbens plays a more important role than that in dorsal striatum in cognitive flexibility , 2014, Front. Behav. Neurosci..

[61]  John N. J. Reynolds,et al.  Dopamine-dependent plasticity of corticostriatal synapses , 2002, Neural Networks.

[62]  Ralf D. Wimmer,et al.  Thalamic amplification of cortical connectivity sustains attentional control , 2017, Nature.

[63]  D. S. Zahm,et al.  The patterns of afferent innervation of the core and shell in the “Accumbens” part of the rat ventral striatum: Immunohistochemical detection of retrogradely transported fluoro‐gold , 1993, The Journal of comparative neurology.

[64]  M. Roitman,et al.  Nucleus accumbens neurons encode Pavlovian approach behaviors: evidence from an autoshaping paradigm , 2006, The European journal of neuroscience.

[65]  S. Smith‐Roe,et al.  Response-reinforcement learning is dependent on N-methyl-D-aspartate receptor activation in the nucleus accumbens core. , 1997, Proceedings of the National Academy of Sciences of the United States of America.

[66]  Trevor W. Robbins,et al.  Mesoaccumbens dopamine-opiate interactions in the control over behaviour by a conditioned reinforcer , 1994, Psychopharmacology.

[67]  T. Robbins,et al.  Limbic-striatal interactions in reward-related processes , 1989, Neuroscience & Biobehavioral Reviews.

[68]  J. Salamone,et al.  Haloperidol and nucleus accumbens dopamine depletion suppress lever pressing for food but increase free food consumption in a novel food choice procedure , 2005, Psychopharmacology.

[69]  Christopher D. Harvey,et al.  Choice-specific sequences in parietal cortex during a virtual-navigation decision task , 2012, Nature.

[70]  Eytan Ruppin,et al.  Actor-critic models of the basal ganglia: new anatomical and computational perspectives , 2002, Neural Networks.

[71]  Liqun Luo,et al.  Circuit Architecture of VTA Dopamine Neurons Revealed by Systematic Input-Output Mapping , 2015, Cell.

[72]  Karl Deisseroth,et al.  Mapping projections of molecularly defined dopamine neuron subtypes using intersectional genetic approaches , 2018, Nature Neuroscience.

[73]  P. Calabresi,et al.  Dopaminergic control of synaptic plasticity in the dorsal striatum , 2001, The European journal of neuroscience.

[74]  G. Tesauro Practical Issues in Temporal Difference Learning , 1992 .

[75]  Karl Deisseroth,et al.  Molecular and Circuit-Dynamical Identification of Top-Down Neural Mechanisms for Restraint of Reward Seeking , 2017, Cell.

[76]  S. Haber,et al.  The Reward Circuit: Linking Primate Anatomy and Human Imaging , 2010, Neuropsychopharmacology.

[77]  Ann M Graybiel,et al.  Neural representation of time in cortico-basal ganglia circuits , 2009, Proceedings of the National Academy of Sciences.

[78]  Ilana B. Witten,et al.  Dissociated sequential activity and stimulus encoding in the dorsomedial striatum during spatial working memory , 2016, eLife.

[79]  Luis Carrillo-Reid,et al.  Dopaminergic modulation of short-term synaptic plasticity at striatal inhibitory synapses , 2007, Proceedings of the National Academy of Sciences.

[80]  Anne L. Collins,et al.  Nucleus Accumbens Cholinergic Interneurons Oppose Cue-Motivated Behavior , 2019, Biological Psychiatry.

[81]  Yingjie Zhu,et al.  Dynamic salience processing in paraventricular thalamus gates associative learning , 2018, Science.

[82]  George Paxinos,et al.  The Mouse Brain in Stereotaxic Coordinates , 2001 .

[83]  Alison L. Barth,et al.  Rapid Plasticity of Higher-Order Thalamocortical Inputs during Sensory Learning , 2019, Neuron.

[84]  R. O’Reilly,et al.  Separate neural substrates for skill learning and performance in the ventral and dorsal striatum , 2007, Nature Neuroscience.

[85]  Thomas E. Hazy,et al.  Neural mechanisms of acquired phasic dopamine responses in learning , 2010, Neuroscience & Biobehavioral Reviews.

[86]  J. Morrison,et al.  The addicted synapse: mechanisms of synaptic and structural plasticity in nucleus accumbens , 2010, Trends in Neurosciences.

[87]  Jeremiah Y. Cohen,et al.  Stable Representations of Decision Variables for Flexible Behavior , 2019, Neuron.

[88]  R. Malenka,et al.  Dopaminergic modulation of neuronal excitability in the striatum and nucleus accumbens. , 2000, Annual review of neuroscience.

[89]  W. Precht The synaptic organization of the brain G.M. Shepherd, Oxford University Press (1975). 364 pp., £3.80 (paperback) , 1976, Neuroscience.

[90]  O. Phillipson,et al.  The topographic order of inputs to nucleus accumbens in the rat , 1985, Neuroscience.

[91]  T. Robbins,et al.  6-Hydroxydopamine lesions of the nucleus accumbens, but not of the caudate nucleus, attenuate enhanced responding with reward-related stimuli produced by intra-accumbens d-amphetamine , 2004, Psychopharmacology.

[92]  M. Roitman,et al.  Nucleus Accumbens Neurons Are Innately Tuned for Rewarding and Aversive Taste Stimuli, Encode Their Predictors, and Are Linked to Motor Output , 2005, Neuron.

[93]  A. Cooper,et al.  Predictive Reward Signal of Dopamine Neurons , 2011 .

[94]  James M. Otis,et al.  Prefrontal cortex output circuits guide reward seeking through divergent cue encoding , 2017, Nature.

[95]  Mark D. Humphries,et al.  Independent population coding of the past and the present in prefrontal cortex during learning , 2019 .

[96]  H. Groenewegen,et al.  Patterns of convergence and segregation in the medial nucleus accumbens of the rat: Relationships of prefrontal cortical, midline thalamic, and basal amygdaloid afferents , 1995, The Journal of comparative neurology.

[97]  Naoshige Uchida,et al.  Erratum: Arithmetic and local circuitry underlying dopamine prediction errors , 2015, Nature.

[98]  A. G. Carter,et al.  Cocaine exposure reorganizes cell type– and input-specific connectivity in the nucleus accumbens , 2014, Nature Neuroscience.

[99]  R. Cardinal,et al.  Nucleus accumbens core lesions retard instrumental learning and performance with delayed reinforcement in the rat , 2005, BMC Neuroscience.

[100]  Bo Li,et al.  Opposing Contributions of GABAergic and Glutamatergic Ventral Pallidal Neurons to Motivational Behaviors , 2020, Neuron.

[101]  Josiah R. Boivin,et al.  A Causal Link Between Prediction Errors, Dopamine Neurons and Learning , 2013, Nature Neuroscience.

[102]  P. Glimcher,et al.  Phasic Dopamine Release in the Rat Nucleus Accumbens Symmetrically Encodes a Reward Prediction Error Term , 2014, The Journal of Neuroscience.

[103]  P. Greengard,et al.  Dichotomous Dopaminergic Control of Striatal Synaptic Plasticity , 2008, Science.

[104]  Lisa M. Saksida,et al.  Genetic and dopaminergic modulation of reversal learning in a touchscreen-based operant procedure for mice , 2006, Behavioural Brain Research.

[105]  Robert E. Hampson,et al.  Firing patterns of nucleus accumbens neurons during cocaine self-administration in rats , 1993, Brain Research.

[106]  Ilana B. Witten,et al.  Specialized coding of sensory, motor, and cognitive variables in VTA dopamine neurons , 2019, Nature.

[107]  Shigeyoshi Fujisawa,et al.  Temporal and Rate Coding for Discrete Event Sequences in the Hippocampus , 2017, Neuron.

[108]  Tianyi Mao,et al.  A comprehensive excitatory input map of the striatum reveals novel functional organization , 2016, eLife.

[109]  A. Kelley,et al.  Early consolidation of instrumental learning requires protein synthesis in the nucleus accumbens , 2002, Nature Neuroscience.

[110]  Anne G E Collins,et al.  Working Memory Contributions to Reinforcement Learning Impairments in Schizophrenia , 2014, The Journal of Neuroscience.

[111]  Aaron S. Andalman,et al.  Multiple overlapping hypothalamus-brainstem circuits drive rapid threat avoidance , 2019, bioRxiv.

[112]  John N. Tsitsiklis,et al.  Analysis of temporal-difference learning with function approximation , 1996, NIPS 1996.

[113]  栁下 祥 A critical time window for dopamine actions on the structural plasticity of dendritic spines , 2016 .

[114]  Simon D. Fisher,et al.  Reinforcement determines the timing dependence of corticostriatal synaptic plasticity in vivo , 2017, Nature Communications.

[115]  Kenneth D. Harris,et al.  Author response: Decision and navigation in mouse parietal cortex , 2018 .

[116]  P. Dayan,et al.  Reinforcement learning: The Good, The Bad and The Ugly , 2008, Current Opinion in Neurobiology.

[117]  P. Dayan,et al.  A framework for mesencephalic dopamine systems based on predictive Hebbian learning , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[118]  Tianyi Mao,et al.  Author response: A comprehensive excitatory input map of the striatum reveals novel functional organization , 2016 .

[119]  Martin O’Neill,et al.  The effect of striatal dopamine depletion and the adenosine A2A antagonist KW-6002 on reversal learning in rats , 2007, Neurobiology of Learning and Memory.

[120]  Kelly R. Tan,et al.  Cocaine Disinhibits Dopamine Neurons by Potentiation of GABA Transmission in the Ventral Tegmental Area , 2013, Science.

[121]  M. Fee,et al.  Changes in the neural control of a complex motor sequence during learning. , 2011, Journal of neurophysiology.

[122]  Jung Hoon Sul,et al.  Role of Striatum in Updating Values of Chosen Actions , 2009, The Journal of Neuroscience.

[123]  M. Roesch,et al.  Ventral Striatal Neurons Encode the Value of the Chosen Action in Rats Deciding between Differently Delayed or Sized Rewards , 2009, The Journal of Neuroscience.

[124]  A. R. Cools,et al.  Spiraling dopaminergic circuitry from the ventral striatum to dorsal striatum is an effective feed-forward loop , 2013, Neuroscience.

[125]  J. Gläscher,et al.  Determining a role for ventromedial prefrontal cortex in encoding action-based value signals during reward-related decision making. , 2009, Cerebral cortex.

[126]  J. Wickens,et al.  Neural control of dopamine neurotransmission: implications for reinforcement learning , 2012, The European journal of neuroscience.

[127]  C. Gerfen,et al.  Modulation of striatal projection systems by dopamine. , 2011, Annual review of neuroscience.

[128]  P. Kalivas,et al.  GABA and enkephalin projection from the nucleus accumbens and ventral pallidum to the ventral tegmental area , 1993, Neuroscience.

[129]  Kenji Doya,et al.  Metalearning and neuromodulation , 2002, Neural Networks.

[130]  J. Wickens,et al.  A cellular mechanism of reward-related learning , 2001, Nature.

[131]  Ilana B. Witten,et al.  Reward and choice encoding in terminals of midbrain dopamine neurons depends on striatal target , 2016, Nature Neuroscience.

[132]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[133]  Aldo Genovesio,et al.  Representation of Future and Previous Spatial Goals by Separate Neural Populations in Prefrontal Cortex , 2006, The Journal of Neuroscience.

[134]  Hiroshi Yamada,et al.  Roles of the Lateral Habenula and Anterior Cingulate Cortex in Negative Outcome Monitoring and Behavioral Adjustment in Nonhuman Primates , 2015, Neuron.

[135]  Xiang Liao,et al.  The paraventricular thalamus is a critical thalamic area for wakefulness , 2018, Science.

[136]  Richard S. Sutton,et al.  Learning to predict by the methods of temporal differences , 1988, Machine Learning.

[137]  Anthony R. West,et al.  Frequency‐Dependent Corticostriatal Disinhibition Resulting from Chronic Dopamine Depletion: Role of Local Striatal cGMP and GABA‐AR Signaling , 2015, Cerebral cortex.

[138]  Gregory J. Quirk,et al.  Thalamic Regulation of Sucrose Seeking during Unexpected Reward Omission , 2017, Neuron.

[139]  G. Schoenbaum,et al.  Neural Encoding in Ventral Striatum during Olfactory Discrimination Learning , 2003, Neuron.

[140]  Amy J. Tindell,et al.  Ventral Pallidal Representation of Pavlovian Cues and Reward: Population and Rate Codes , 2004, The Journal of Neuroscience.

[141]  B. Everitt,et al.  Differential Involvement of NMDA, AMPA/Kainate, and Dopamine Receptors in the Nucleus Accumbens Core in the Acquisition and Performance of Pavlovian Approach Behavior , 2001, The Journal of Neuroscience.

[142]  O. Hikosaka,et al.  Two types of dopamine neuron distinctly convey positive and negative motivational signals , 2009, Nature.

[143]  G. P. Smith,et al.  Efferent connections and nigral afferents of the nucleus accumbens septi in the rat , 1978, Neuroscience.

[144]  Thomas J. Davidson,et al.  Coordinated Reductions in Excitatory Input to the Nucleus Accumbens Underlie Food Consumption , 2018, Neuron.

[145]  John R. Anderson,et al.  Neural Correlates of Temporal Credit Assignment , 2010 .

[146]  Alice M Stamatakis,et al.  Excitatory transmission from the amygdala to nucleus accumbens facilitates reward seeking. , 2011, Nature.

[147]  Adam Ponzi,et al.  Sequentially Switching Cell Assemblies in Random Inhibitory Networks of Spiking Neurons in the Striatum , 2010, The Journal of Neuroscience.

[148]  Nikolaus R. McFarland,et al.  Striatonigrostriatal Pathways in Primates Form an Ascending Spiral from the Shell to the Dorsolateral Striatum , 2000, The Journal of Neuroscience.

[149]  Ivan Trujillo-Pisanty,et al.  Paraventricular Thalamus Projection Neurons Integrate Cortical and Hypothalamic Signals for Cue-Reward Processing , 2019, Neuron.

[150]  R. Wise,et al.  Synaptic and Behavioral Profile of Multiple Glutamatergic Inputs to the Nucleus Accumbens , 2012, Neuron.

[151]  Anne G E Collins,et al.  How much of reinforcement learning is working memory, not reinforcement learning? A behavioral, computational, and neurogenetic analysis , 2012, The European journal of neuroscience.

[152]  David Badre,et al.  Working Memory Load Strengthens Reward Prediction Errors , 2017, The Journal of Neuroscience.

[153]  Chung-Hay Luk,et al.  Choice Coding in Frontal Cortex during Stimulus-Guided or Action-Guided Decision-Making , 2013, The Journal of Neuroscience.

[154]  Wulfram Gerstner,et al.  Eligibility Traces and Plasticity on Behavioral Time Scales: Experimental Support of NeoHebbian Three-Factor Learning Rules , 2018, Front. Neural Circuits.

[155]  G. Hjelmstad,et al.  Dopamine Excites Nucleus Accumbens Neurons through the Differential Modulation of Glutamate and GABA Release , 2004, The Journal of Neuroscience.

[156]  Xiao-Jing Wang,et al.  Inhibitory Control in the Cortico-Basal Ganglia-Thalamocortical Loop: Complex Regulation and Interplay with Memory and Decision Processes , 2016, Neuron.

[157]  Benjamin T. Saunders,et al.  Dopamine neurons create Pavlovian conditioned stimuli with circuit-defined motivational properties , 2018, Nature Neuroscience.

[158]  Ilana B. Witten,et al.  Recombinase-Driver Rat Lines: Tools, Techniques, and Optogenetic Application to Dopamine-Mediated Reinforcement , 2011, Neuron.

[159]  E. Pnevmatikakis,et al.  NoRMCorre: An online algorithm for piecewise rigid motion correction of calcium imaging data , 2017, Journal of Neuroscience Methods.

[160]  Bin Zhang,et al.  Deep Multilayer Brain Proteomics Identifies Molecular Networks in Alzheimer’s Disease Progression , 2020, Neuron.

[161]  Florentin Wörgötter,et al.  Temporal Sequence Learning, Prediction, and Control: A Review of Different Models and Their Relation to Biological Mechanisms , 2005, Neural Computation.

[162]  Kenneth D Harris,et al.  Decision and navigation in mouse parietal cortex , 2017, bioRxiv.

[163]  T. Robbins,et al.  Dissociation in Effects of Lesions of the Nucleus Accumbens Core and Shell on Appetitive Pavlovian Approach Behavior and the Potentiation of Conditioned Reinforcement and Locomotor Activity byd-Amphetamine , 1999, The Journal of Neuroscience.

[164]  Alex C Kwan,et al.  Enhanced population coding for rewarded choices in the medial frontal cortex of the mouse , 2018, bioRxiv.

[165]  M. Bevan,et al.  Dopaminergic Transmission Rapidly and Persistently Enhances Excitability of D1 Receptor-Expressing Striatal Projection Neurons , 2020, Neuron.

[166]  E. Rolls,et al.  Value, Pleasure and Choice in the Ventral Prefrontal Cortex , 2022 .

[167]  Luis Carrillo-Reid,et al.  Encoding network states by striatal cell assemblies. , 2008, Journal of neurophysiology.

[168]  T. Robbins,et al.  Dopamine D2/D3 receptor agonist quinpirole impairs spatial reversal learning in rats: investigation of D3 receptor involvement in persistent behavior , 2009, Psychopharmacology.

[169]  Trevor W. Robbins,et al.  Cholecystokinin-dopamine interactions within the nucleus accumbens in the control over behaviour by conditioned reinforcement , 1993, Behavioural Brain Research.

[170]  M. Ohkura,et al.  Two-photon calcium imaging of the medial prefrontal cortex and hippocampus without cortical invasion , 2017, eLife.

[171]  Yingjie Zhu,et al.  A thalamic input to the nucleus accumbens mediates opiate dependence , 2016, Nature.

[172]  Charu Bai Reddy,et al.  Dopaminergic and Prefrontal Basis of Learning from Sensory Confidence and Reward Value , 2019, Neuron.

[173]  Jocelyn M. Richard,et al.  Ventral pallidum encodes relative reward value earlier and more robustly than nucleus accumbens , 2018, Nature Communications.