A New Framework for Cortico-Striatal Plasticity: Behavioural Theory Meets In Vitro Data at the Reinforcement-Action Interface

A computational model yields new insights into the bewildering complexity of cortico-striatal plasticity and its rationale for supporting operant learning.

[1]  Douglas L. Jones,et al.  From motivation to action: Functional interface between the limbic system and the motor system , 1980, Progress in Neurobiology.

[2]  E. Bienenstock,et al.  Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex , 1982, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[3]  A. Dickinson.  Actions and habits: the development of behavioural autonomy , 1985 .

[4]  G. E. Alexander,et al.  Microstimulation of the primate neostriatum. II. Somatotopic organization of striatal microexcitable zones and their relation to neuronal response properties. , 1985, Journal of neurophysiology.

[5]  G. E. Alexander,et al.  Parallel organization of functionally segregated circuits linking basal ganglia and cortex. , 1986, Annual review of neuroscience.

[6]  A. Preuss,et al.  Corticostriatal cells in comparison with pyramidal tract neurons: contrasting properties in the behaving monkey , 1989, Brain Research.

[7]  A. Mcgeorge,et al.  The organization of the projection from the cerebral cortex to the striatum in the rat , 1989, Neuroscience.

[8]  G. E. Alexander,et al.  Functional architecture of basal ganglia circuits: neural substrates of parallel processing , 1990, Trends in Neurosciences.

[9]  C. Stevens,et al.  Voltage dependence of NMDA-activated macroscopic conductances predicted by single-channel kinetics , 1990, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[10]  W. T. Thach,et al.  Basal ganglia intrinsic circuits and their role in behavior , 1993, Current Opinion in Neurobiology.

[11]  Joel L. Davis,et al.  A Model of How the Basal Ganglia Generate and Use Neural Signals That Predict Reinforcement , 1994 .

[12]  L. Brown,et al.  Metabolic mapping of rat striatum: somatotopic organization of sensorimotor activity , 1995, Brain Research.

[13]  P. Dayan,et al.  A framework for mesencephalic dopamine systems based on predictive Hebbian learning , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[14]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[15]  P. Redgrave,et al.  The basal ganglia: a vertebrate solution to the selection problem? , 1999, Neuroscience.

[16]  J. Deniau,et al.  Three-dimensional distribution of nigrostriatal neurons in the rat: relation to the topography of striatonigral projections , 1999, Neuroscience.

[17]  Mark C. W. van Rossum,et al.  Stable Hebbian Learning from Spike Timing-Dependent Plasticity , 2000, The Journal of Neuroscience.

[18]  L. Abbott,et al.  Synaptic plasticity: taming the beast , 2000, Nature Neuroscience.

[19]  P. Strick,et al.  Basal ganglia and cerebellar loops: motor and cognitive circuits , 2000, Brain Research Reviews.

[20]  O. Hikosaka,et al.  Role of the basal ganglia in the control of purposive saccadic eye movements. , 2000, Physiological reviews.

[21]  S. Hyman,et al.  Addiction, Dopamine, and the Molecular Mechanisms of Memory , 2000, Neuron.

[22]  K. Urushihara,et al.  Renewal of Extinguished Lever-Press Responses upon Return to the Training Context , 2000 .

[23]  D. Joel,et al.  The connections of the dopaminergic system with the striatum in rats and primates: an analysis with respect to the functional and compartmental organization of the striatum , 2000, Neuroscience.

[24]  Nikolaus R. McFarland,et al.  Striatonigrostriatal Pathways in Primates Form an Ascending Spiral from the Shell to the Dorsolateral Striatum , 2000, The Journal of Neuroscience.

[25]  S. Charpier,et al.  Role of a striatal slowly inactivating potassium current in short-term facilitation of corticostriatal inputs: a computer simulation study. , 2000, Learning & memory.

[26]  P. Calabresi,et al.  Dopaminergic control of synaptic plasticity in the dorsal striatum , 2001, The European journal of neuroscience.

[27]  Peter Redgrave,et al.  A computational model of action selection in the basal ganglia. II. Analysis and simulation of behaviour , 2001, Biological Cybernetics.

[28]  Peter Redgrave,et al.  A computational model of action selection in the basal ganglia. I. A new functional anatomy , 2001, Biological Cybernetics.

[29]  J. Wickens,et al.  A cellular mechanism of reward-related learning , 2001, Nature.

[30]  J. Galen Buckwalter,et al.  Regional differences in the expression of corticostriatal synaptic plasticity , 2001, Neuroscience.

[31]  Y. Shaham,et al.  Renewal of drug seeking by contextual cues after prolonged extinction in rats. , 2002, Behavioral neuroscience.

[32]  John N. J. Reynolds,et al.  Dopamine-dependent plasticity of corticostriatal synapses , 2002, Neural Networks.

[33]  Eugene M. Izhikevich,et al.  Simple model of spiking neurons , 2003, IEEE Trans. Neural Networks.

[34]  Shigeru Shinomoto,et al.  Differences in Spiking Patterns Among Cortical Neurons , 2003, Neural Computation.

[35]  Nathan Intrator,et al.  Theory of Cortical Plasticity: (With Software Package “PLASTICITY”) , 2004 .

[36]  B. Balleine,et al.  Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning , 2004, The European journal of neuroscience.

[37]  S. Cragg,et al.  DAncing past the DAT at a DA synapse , 2004, Trends in Neurosciences.

[38]  M. Bouton.  Context and behavioral processes in extinction. , 2004, Learning & memory.

[39]  Nathan Intrator,et al.  Theory of Cortical Plasticity , 2004 .

[40]  B. Hommel,et al.  Contiguity and contingency in action-effect learning , 2004, Psychological research.

[41]  A. Graybiel,et al.  Activity of striatal neurons reflects dynamic encoding and recoding of procedural memories , 2005, Nature.

[42]  G. Heit,et al.  Somatotopy in the basal ganglia: experimental and clinical evidence for segregated sensorimotor channels , 2005, Brain Research Reviews.

[43]  A. Faure,et al.  Lesion to the Nigrostriatal Dopamine System Disrupts Stimulus-Response Habit Formation , 2005, The Journal of Neuroscience.

[44]  P. Dayan,et al.  Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control , 2005, Nature Neuroscience.

[45]  Florentin Wörgötter,et al.  Temporal Sequence Learning, Prediction, and Control: A Review of Different Models and Their Relation to Biological Mechanisms , 2005, Neural Computation.

[46]  Michael J. Frank,et al.  Dynamic Dopamine Modulation in the Basal Ganglia: A Neurocomputational Account of Cognitive Deficits in Medicated and Nonmedicated Parkinsonism , 2005, Journal of Cognitive Neuroscience.

[47]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[48]  P. Glimcher,et al.  Midbrain Dopamine Neurons Encode a Quantitative Reward Prediction Error Signal , 2005, Neuron.

[49]  B. Balleine,et al.  The role of the dorsomedial striatum in instrumental conditioning , 2005, The European journal of neuroscience.

[50]  J. Glowinski,et al.  Bidirectional Activity-Dependent Plasticity at Corticostriatal Synapses , 2005, The Journal of Neuroscience.

[51]  P. Redgrave,et al.  The short-latency dopamine signal: a role in discovering novel actions? , 2006, Nature Reviews Neuroscience.

[52]  Jeanette Kotaleski,et al.  Transient Calcium and Dopamine Increase PKA Activity and DARPP-32 Phosphorylation , 2006, PLoS Comput. Biol..

[53]  H. Yin,et al.  The role of the basal ganglia in habit formation , 2006, Nature Reviews Neuroscience.

[54]  W. Gerstner,et al.  Triplets of Spikes in a Model of Spike Timing-Dependent Plasticity , 2006, The Journal of Neuroscience.

[55]  K. Gurney,et al.  A Physiologically Plausible Model of Action Selection and Oscillatory Activity in the Basal Ganglia , 2006, The Journal of Neuroscience.

[56]  Henry H. Yin,et al.  Dopaminergic Control of Corticostriatal Long-Term Synaptic Depression in Medium Spiny Neurons Is Mediated by Cholinergic Interneurons , 2006, Neuron.

[57]  P. Glimcher,et al.  Statistics of midbrain dopamine neuron spike trains in the awake primate. , 2007, Journal of neurophysiology.

[58]  D. Surmeier,et al.  D1 and D2 dopamine-receptor modulation of striatal glutamatergic signaling in striatal medium spiny neurons , 2007, Trends in Neurosciences.

[59]  Paolo Calabresi,et al.  Dopamine-mediated regulation of corticostriatal synaptic plasticity , 2007, Trends in Neurosciences.

[60]  J. Wickens,et al.  Striatal contributions to reward and decision making: making sense of regional variations in a reiterated processing matrix. , 2007, Annals of the New York Academy of Sciences.

[61]  Jadin C. Jackson,et al.  Reconciling reinforcement learning models with behavioral extinction and renewal: implications for addiction, relapse, and problem gambling. , 2007, Psychological review.

[62]  E. Izhikevich.  Solving the distal reward problem through linkage of STDP and dopamine signaling , 2007, BMC Neuroscience.

[63]  John A Wolf,et al.  Effects of dopaminergic modulation on the integrative properties of the ventral striatal medium spiny neuron. , 2007, Journal of neurophysiology.

[64]  M. West,et al.  Changes in activity of the striatum during formation of a motor habit , 2007, The European journal of neuroscience.

[65]  J. Wickens,et al.  Striatal Contributions to Reward and Decision Making , 2007 .

[66]  B. Balleine,et al.  Selective reinstatement of instrumental performance depends on the discriminative stimulus properties of the mediating outcome , 2007, Learning & behavior.

[67]  P. Redgrave,et al.  What is reinforced by phasic dopamine signals? , 2008, Brain Research Reviews.

[68]  Daphna Joel,et al.  The orbital cortex in rats topographically projects to central parts of the caudate–putamen complex , 2008, Neuroscience Letters.

[69]  P. Greengard,et al.  Dichotomous Dopaminergic Control of Striatal Synaptic Plasticity , 2008, Science.

[70]  J. Kerr,et al.  Dopamine Receptor Activation Is Required for Corticostriatal Spike-Timing-Dependent Plasticity , 2008, The Journal of Neuroscience.

[71]  Mark Laubach,et al.  The Dorsomedial Striatum Reflects Response Bias during Learning , 2009, The Journal of Neuroscience.

[72]  Mark D. Humphries,et al.  Frontiers in Computational Neuroscience , 2022 .

[73]  D. Lovinger,et al.  Dynamic reorganization of striatal circuits during the acquisition and consolidation of a skill , 2009, Nature Neuroscience.

[74]  O. Hikosaka,et al.  Two types of dopamine neuron distinctly convey positive and negative motivational signals , 2009, Nature.

[75]  M. Laubach,et al.  Neuronal correlates of instrumental learning in the dorsal striatum. , 2009, Journal of neurophysiology.

[76]  K. Deisseroth,et al.  Phasic Firing in Dopaminergic Neurons Is Sufficient for Behavioral Conditioning , 2009, Science.

[77]  M. Roesch,et al.  A new perspective on the role of the orbitofrontal cortex in adaptive behaviour , 2009, Nature Reviews Neuroscience.

[78]  B. Balleine,et al.  Acquisition and Performance of Goal-Directed Instrumental Actions Depends on ERK Signaling in Distinct Regions of Dorsal Striatum in Rats , 2010, Journal of Neuroscience.

[79]  Jean-Marc Fellous,et al.  Computational models of reinforcement learning: the role of dopamine as a reward signal , 2010, Cognitive Neurodynamics.

[80]  Jung Hoon Sul,et al.  Distinct Roles of Rodent Orbitofrontal and Medial Prefrontal Cortex in Decision Making , 2010, Neuron.

[81]  A. Graybiel,et al.  Differential Dynamics of Activity Changes in Dorsolateral and Dorsomedial Striatal Loops during Learning , 2010, Neuron.

[82]  T. Prescott,et al.  The ventral basal ganglia, a selection mechanism at the crossroads of space, strategy, and reward. , 2010, Progress in Neurobiology.

[83]  W. Schultz,et al.  Dopamine signals for reward value and risk: basic and recent data , 2010, Behavioral and Brain Functions.

[84]  Junichiro Yoshimoto,et al.  A Kinetic Model of Dopamine- and Calcium-Dependent Striatal Synaptic Plasticity , 2010, PLoS Comput. Biol..

[85]  Anatol C. Kreitzer,et al.  Regulation of parkinsonian motor behaviours by optogenetic control of basal ganglia circuitry , 2010, Nature.

[86]  N. Daw,et al.  Multiplicity of control in the basal ganglia: computational roles of striatal subregions , 2011, Current Opinion in Neurobiology.

[87]  B. Balleine,et al.  Molecular substrates of action control in cortico-striatal circuits , 2011, Progress in Neurobiology.

[88]  J. Tsien,et al.  NMDA Receptors in Dopaminergic Neurons Are Crucial for Habit Learning , 2011, Neuron.

[89]  G. Mandolesi,et al.  Cognitive and neural determinants of response strategy in the dual-solution plus-maze task. , 2011, Learning & memory.

[90]  Anatol C. Kreitzer,et al.  Investigating striatal function through cell-type-specific manipulations , 2011, Neuroscience.

[91]  M. Bouton,et al.  Renewal after the extinction of free operant behavior , 2011, Learning & behavior.

[92]  David Fan,et al.  Mechanisms of Action Selection and Timing in Substantia Nigra Neurons , 2012, The Journal of Neuroscience.

[93]  M. Bouton,et al.  Contextual control of appetite. Renewal of inhibited food-seeking behavior in sated rats after extinction , 2012, Appetite.

[94]  K. Deisseroth,et al.  Striatal Dopamine Release Is Triggered by Synchronized Activity in Cholinergic Interneurons , 2012, Neuron.

[95]  M. Fee.  Oculomotor learning revisited: a model of reinforcement learning in the basal ganglia incorporating an efference copy of motor actions , 2012, Front. Neural Circuits.

[96]  Anatol C. Kreitzer,et al.  Distinct roles for direct and indirect pathway striatal neurons in reinforcement , 2012, Nature Neuroscience.

[97]  M. Khamassi,et al.  Integrating cortico-limbic-basal ganglia architectures for learning model-based and model-free navigation strategies , 2012, Front. Behav. Neurosci..

[98]  A. Zador,et al.  Corticostriatal neurones in auditory cortex drive decisions during auditory discrimination , 2013, Nature.

[99]  Jeanette Hellgren Kotaleski,et al.  GABAergic Circuits Control Spike-Timing-Dependent Plasticity , 2013, The Journal of Neuroscience.

[100]  Kim T. Blackwell,et al.  Signaling Pathways Involved in Striatal Synaptic Plasticity are Sensitive to Temporal Pattern and Exhibit Spatial Specificity , 2013, PLoS Comput. Biol..

[101]  R. Costa,et al.  Orbitofrontal and striatal circuits dynamically encode the shift between goal-directed and habitual actions , 2013, Nature Communications.

[102]  Steven S. Vogel,et al.  Concurrent Activation of Striatal Direct and Indirect Pathways During Action Initiation , 2013, Nature.

[103]  Anatol C. Kreitzer,et al.  Control of Basal Ganglia Output by Direct and Indirect Pathway Projection Neurons , 2013, The Journal of Neuroscience.

[104]  K. Blackwell,et al.  Dynamic modulation of spike timing-dependent calcium influx during corticostriatal upstates. , 2013, Journal of neurophysiology.

[105]  Kevin Gurney,et al.  Action Discovery and Intrinsic Motivation: A Biologically Constrained Formalisation , 2013, Intrinsically Motivated Learning in Natural and Artificial Systems.

[106]  Jeffery R. Wickens,et al.  Optimal Balance of the Striatal Medium Spiny Neuron Network , 2013, PLoS Comput. Biol..

[107]  J. Dudman,et al.  Neural signals of extinction in the inhibitory microcircuit of the ventral midbrain , 2012, Nature Neuroscience.

[108]  Kevin N. Gurney,et al.  A biologically plausible embodied model of action discovery , 2012, Front. Neurorobot..

[109]  P. Glimcher,et al.  Phasic Dopamine Release in the Rat Nucleus Accumbens Symmetrically Encodes a Reward Prediction Error Term , 2014, The Journal of Neuroscience.

[110]  A. Graybiel,et al.  Differential Entrainment and Learning-Related Dynamics of Spike and Local Field Potential Activity in the Sensorimotor and Associative Striatum , 2014, The Journal of Neuroscience.

[111]  S. Ikemoto,et al.  Similar Roles of Substantia Nigra and Ventral Tegmental Dopamine Neurons in Reward and Aversion , 2014, The Journal of Neuroscience.

[112]  William R. Stauffer,et al.  Dopamine prediction error responses integrate subjective value from different reward dimensions , 2014, Proceedings of the National Academy of Sciences.

[113]  Z. Mainen,et al.  Balanced activity in basal ganglia projection pathways is critical for contraversive movements , 2014, Nature Communications.

[114]  Bernard W Balleine,et al.  The Acquisition of Goal-Directed Actions Generates Opposing Plasticity in Direct and Indirect Pathways in Dorsomedial Striatum , 2014, The Journal of Neuroscience.