A Biologically Inspired Computational Model of Basal Ganglia in Action Selection

The basal ganglia (BG) are a subcortical structure implicated in action selection. The aim of this work is to present a new cognitive neuroscience model of the BG, which aspires to represent a parsimonious balance between simplicity and completeness. The model includes the 3 main pathways operating in the BG circuitry, that is, the direct (Go), indirect (NoGo), and hyperdirect pathways. The main original aspects, compared with previous models, are the use of a two-term Hebb rule to train synapses in the striatum, based exclusively on neuronal activity changes caused by dopamine peaks or dips, and the role of the cholinergic interneurons (affected by dopamine themselves) during learning. Some examples are displayed, concerning a few paradigmatic cases: action selection in basal conditions, action selection in the presence of a strong conflict (where the role of the hyperdirect pathway emerges), synapse changes induced by phasic dopamine, and learning new actions based on a previous history of rewards and punishments. Finally, some simulations show model working in conditions of altered dopamine levels, to illustrate pathological cases (dopamine depletion in parkinsonian subjects or dopamine hypermedication). Due to its parsimonious approach, the model may represent a straightforward tool to analyze BG functionality in behavioral experiments.

[1]  D. Surmeier,et al.  Muscarinic modulation of striatal function and circuitry. , 2012, Handbook of Experimental Pharmacology.

[2]  Thomas V. Wiecki,et al.  Subthalamic nucleus stimulation reverses mediofrontal influence over decision threshold , 2011, Nature Neuroscience.

[3]  C. Lebiere,et al.  Conditional routing of information to the cortex: a model of the basal ganglia's role in cognitive coordination. , 2010, Psychological review.

[4]  John M. Ennis,et al.  A neurobiological theory of automaticity in perceptual categorization. , 2007, Psychological review.

[5]  Charles J. Wilson,et al.  Spontaneous Activity of Neostriatal Cholinergic Interneurons In Vitro , 1999, The Journal of Neuroscience.

[6]  Henning Schroll,et al.  Working memory and response selection: A computational account of interactions among cortico-basalganglio-thalamic loops , 2012, Neural Networks.

[7]  E. Vaadia,et al.  Coincident but Distinct Messages of Midbrain Dopamine and Striatal Tonically Active Neurons , 2004, Neuron.

[8]  Jun B. Ding,et al.  Cholinergic modulation of synaptic integration and dendritic excitability in the striatum , 2011, Current Opinion in Neurobiology.

[9]  Mark A. Gluck,et al.  A Neurocomputational Model of Dopamine and Prefrontal–Striatal Interactions during Multicue Category Learning by Parkinson Patients , 2011, Journal of Cognitive Neuroscience.

[10]  Thomas V. Wiecki,et al.  A computational model of inhibitory control in frontal cortex and basal ganglia. , 2011, Psychological review.

[11]  P. Calabresi,et al.  A convergent model for cognitive dysfunctions in Parkinson's disease: the critical dopamine–acetylcholine synaptic balance , 2006, The Lancet Neurology.

[12]  P. Dayan,et al.  Tonic dopamine: opportunity costs and the control of response vigor , 2007, Psychopharmacology.

[13]  P. Glimcher,et al.  Statistics of midbrain dopamine neuron spike trains in the awake primate. , 2007, Journal of neurophysiology.

[14]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[15]  A. Cooper,et al.  Predictive Reward Signal of Dopamine Neurons , 2011 .

[16]  W. Schultz,et al.  A neural network model with dopamine-like reinforcement signal that learns a spatial delayed response task , 1999, Neuroscience.

[17]  P. Calabresi,et al.  The Distinct Role of Medium Spiny Neurons and Cholinergic Interneurons in the D2/A2A Receptor Interaction in the Striatum: Implications for Parkinson's Disease , 2011, The Journal of Neuroscience.

[18]  R. Bogacz,et al.  Improved conditions for the generation of beta oscillations in the subthalamic nucleus-globus pallidus network , 2012, BMC Neuroscience.

[19]  F. Horak,et al.  Inhibition, executive function, and freezing of gait. , 2014, Journal of Parkinson's disease.

[20]  J. Obeso,et al.  Pathophysiology of the basal ganglia in Parkinson's disease , 2000, Trends in Neurosciences.

[21]  R. Ridley,et al.  The Role of the Central Cholinergic Projections in Cognition: Implications of the Effects of Scopolamine on Discrimination Learning by Monkeys , 1998, Brain Research Bulletin.

[22]  F. Hamker,et al.  Dysfunctional and compensatory synaptic plasticity in Parkinson's disease , 2014, The European journal of neuroscience.

[23]  J. Bargas,et al.  D1 Receptor Activation Enhances Evoked Discharge in Neostriatal Medium Spiny Neurons by Modulating an L-Type Ca2+ Conductance , 1997, The Journal of Neuroscience.

[24]  C. Marsden,et al.  'Frontal' cognitive function in patients with Parkinson's disease 'on' and 'off' levodopa. , 1988, Brain : a journal of neurology.

[25]  Sacha Jennifer van Albada,et al.  Mean-field modeling of the basal ganglia-thalamocortical system. I Firing rates in healthy and parkinsonian states. , 2009, Journal of theoretical biology.

[26]  L. Squire,et al.  Preserved learning in monkeys with medial temporal lesions: sparing of motor and cognitive skills , 1984, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[27]  F. Gregory Ashby,et al.  A Computational Model of How Cholinergic Interneurons Protect Striatal-dependent Learning , 2011, Journal of Cognitive Neuroscience.

[28]  B. Sabatini,et al.  Multiphasic Modulation of Cholinergic Interneurons by Nigrostriatal Afferents , 2014, The Journal of Neuroscience.

[29]  D. James Surmeier,et al.  Re-emergence of striatal cholinergic interneurons in movement disorders , 2007, Trends in Neurosciences.

[30]  Robert Chen,et al.  Differential response of speed, amplitude, and rhythm to dopaminergic medications in Parkinson's disease , 2011, Movement disorders : official journal of the Movement Disorder Society.

[31]  Michael X. Cohen,et al.  Neurocomputational models of basal ganglia function in learning, memory and choice , 2009, Behavioural Brain Research.

[32]  A. Nambu,et al.  Functional significance of the cortico–subthalamo–pallidal ‘hyperdirect’ pathway , 2002, Neuroscience Research.

[33]  M. Mazurek,et al.  Behavioral Characterization of Quinolinate-Induced Lesions of the Medial Striatum: Relevance for Huntington's Disease , 1996, Experimental Neurology.

[34]  Giovanni Pezzulo,et al.  A spiking neuron model of the cortico-basal ganglia circuits for goal-directed and habitual action learning. , 2013, Neural networks : the official journal of the International Neural Network Society.

[35]  C. R. de Mello Rieder,et al.  Wearing-off in Parkinson’s disease: neuropsychological differences between on and off periods , 2015, Neuropsychiatric disease and treatment.

[36]  T. Robbins,et al.  Dissociating executive mechanisms of task control following frontal lobe damage and Parkinson's disease. , 1998, Brain : a journal of neurology.

[37]  F. J. Friedrich,et al.  Cognition and the basal ganglia. Separating mental and motor components of performance in Parkinson's disease. , 1984, Brain : a journal of neurology.

[38]  E. A. Berg,et al.  A simple objective technique for measuring flexibility in thinking. , 1948, The Journal of general psychology.

[39]  I. Shimoyama,et al.  The finger-tapping test. A quantitative analysis. , 1990, Archives of neurology.

[40]  Sébastien Hélie,et al.  Exploring the cognitive and motor functions of the basal ganglia: an integrative review of computational cognitive neuroscience models , 2013, Front. Comput. Neurosci..

[41]  Michael X. Cohen,et al.  A Role for Dopamine in Temporal Decision Making and Reward Maximization in Parkinsonism , 2008, The Journal of Neuroscience.

[42]  F. J. Friedrich,et al.  COGNITION AND THE BASAL GANGLIA , 1984 .

[43]  Michael J. Frank,et al.  Hold your horses: A dynamic computational role for the subthalamic nucleus in decision making , 2006, Neural Networks.

[44]  W. T. Thach,et al.  Basal ganglia intrinsic circuits and their role in behavior , 1993, Current Opinion in Neurobiology.

[45]  T. Robbins,et al.  Enhanced or impaired cognitive function in Parkinson's disease as a function of dopaminergic medication and task demands. , 2001, Cerebral cortex.

[46]  H. Kita,et al.  Excitatory Cortical Inputs to Pallidal Neurons Via the Subthalamic Nucleus in the Monkey , 2000 .

[47]  Peter Brown,et al.  The subthalamic nucleus, oscillations, and conflict , 2015, Movement disorders : official journal of the Movement Disorder Society.

[48]  Franklin A. Graybill,et al.  Introduction to The theory , 1974 .

[49]  J. Gray,et al.  Dopamine release in the nucleus accumbens: The perspective from aberrations of consciousness in schizophrenia , 1995, Neuropsychologia.

[50]  Roland E. Suri,et al.  TD models of reward predictive responses in dopamine neurons , 2002, Neural Networks.

[51]  G. Damsma,et al.  Characterization of dopamine release in the substantia nigra by in vivo microdialysis in freely moving rats , 1991, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[52]  P. Calabresi,et al.  Direct and indirect pathways of basal ganglia: a critical reappraisal , 2014, Nature Neuroscience.

[53]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[54]  Y. Kawaguchi,et al.  Dopamine D1-Like Receptor Activation Excites Rat Striatal Large Aspiny Neurons In Vitro , 1998, The Journal of Neuroscience.

[55]  A. Graybiel,et al.  Effect of the nigrostriatal dopamine system on acquired neural responses in the striatum of behaving monkeys. , 1994, Science.

[56]  Fred H. Hamker,et al.  Computational models of basal-ganglia pathway functions: focus on functional neuroanatomy , 2013, Front. Syst. Neurosci..

[57]  Henry H. Yin,et al.  Dopaminergic Control of Corticostriatal Long-Term Synaptic Depression in Medium Spiny Neurons Is Mediated by Cholinergic Interneurons , 2006, Neuron.

[58]  Michael J. Frank,et al.  Dynamic Dopamine Modulation in the Basal Ganglia: A Neurocomputational Account of Cognitive Deficits in Medicated and Nonmedicated Parkinsonism , 2005, Journal of Cognitive Neuroscience.

[59]  W. Todd Maddox,et al.  Rule-based category learning in patients with Parkinson's disease , 2009, Neuropsychologia.

[60]  J. Mink.  THE BASAL GANGLIA: FOCUSED SELECTION AND INHIBITION OF COMPETING MOTOR PROGRAMS , 1996, Progress in Neurobiology.

[61]  Richard S. Sutton,et al.  Learning to predict by the methods of temporal differences , 1988, Machine Learning.

[62]  C. E. Myersb,et al.  neurocomputational model of tonic and phasic dopamine in action selection : comparison with cognitive deficits in Parkinson ’ s disease , 2009 .