Parallel Cortico-Basal Ganglia Mechanisms for Acquisition and Execution of Visuomotor SequencesA Computational Approach

Experimental studies have suggested that many brain areas, including the basal ganglia (BG), contribute to procedural learning. Focusing on the basal ganglia-thalamo-cortical (BG-TC) system, we propose a computational model to explain how different brain areas work together in procedural learning. The BG-TC system is composed of multiple separate loop circuits. According to our model, two separate BG-TC loops learn a visuomotor sequence concurrently but using different coordinates, one visual, and the other motor. The visual loop includes the dorsolateral prefrontal (DLPF) cortex and the anterior part of the BG, while the motor loop includes the supplementary motor area (SMA) and the posterior BG. The concurrent learning in these loops is based on reinforcement signals carried by dopaminergic (DA) neurons that project divergently to the anterior (visual) and posterior (motor) parts of the striatum. It is expected, however, that the visual loop learns a sequence faster than the motor loop due to their different coordinates. The difference in learning speed may lead to inconsistent outputs from the visual and motor loops, and this problem is solved by a mechanism called a coordinator, which adjusts the contribution of the visual and motor loops to a final motor output. The coordinator is assumed to be in the presupplementary motor area (pre-SMA). We hypothesize that the visual and motor loops, with the help of the coordinator, achieve both the quick acquisition of novel sequences and the robust execution of well-learned sequences. A computational model based on the hypothesis is examined in a series of computer simulations, referring to the results of the 2 5 task experiments that have been used on both monkeys and humans. We found that the dual mechanism with the coordinator was superior to the single (visual or motor) mechanism. The model replicated the following essential features of the experimental results: (1) the time course of learning, (2) the effect of opposite hand use, (3) the effect of sequence reversal, and (4) the effects of localized brain inactivations. Our model may account for a common feature of procedural learning: A spatial sequence of discrete actions (subserved by the visual loop) is gradually replaced by a robust motor skill (subserved by the motor loop).

[1]  N. A. Bernshteĭn.  The co-ordination and regulation of movements , 1967 .

[2]  T. Powell,et al.  The cortico-striate projection in the monkey. , 1970, Brain : a journal of neurology.

[3]  J. Winn,et al.  Brain , 1878, The Lancet.

[4]  C. Marsden,et al.  The enigma of the basal ganglia and movement , 1980, Trends in Neurosciences.

[5]  John R. Anderson.  Acquisition of cognitive skill. , 1982 .

[6]  Shun-ichi Amari,et al.  Differential-geometrical methods in statistics , 1985 .

[7]  P. Goldman-Rakic,et al.  Longitudinal topography and interdigitation of corticostriatal projections in the rhesus monkey , 1985, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[8]  G. E. Alexander,et al.  Parallel organization of functionally segregated circuits linking basal ganglia and cortex. , 1986, Annual review of neuroscience.

[9]  S P Wise,et al.  The somatotopic organization of the supplementary motor area: intracortical microstimulation mapping , 1987, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[10]  P. Goldman-Rakic,et al.  Mnemonic coding of visual space in the monkey's dorsolateral prefrontal cortex. , 1989, Journal of neurophysiology.

[11]  M E Goldberg,et al.  Participation of prefrontal neurons in the preparation of visually guided eye movements in the rhesus monkey. , 1989, Journal of neurophysiology.

[12]  D. Pandya,et al.  Anatomical investigation of projections from thalamus to posterior parietal cortex in the rhesus monkey: A WGA‐HRP and fluorescent tracer study , 1990, The Journal of comparative neurology.

[13]  A. Parent.  Extrinsic connections of the basal ganglia , 1990, Trends in Neurosciences.

[14]  G. E. Alexander,et al.  Functional architecture of basal ganglia circuits: neural substrates of parallel processing , 1990, Trends in Neurosciences.

[15]  Mitsuo Kawato,et al.  Computational schemes and neural network models for formulation and control of multijoint arm trajectory , 1990 .

[16]  A. Graybiel,et al.  Distributed but convergent ordering of corticostriatal projections: analysis of the frontal eye field and the supplementary eye field in the macaque monkey , 1992, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[17]  J. Tanji,et al.  A motor area rostral to the supplementary motor area (presupplementary motor area) in the monkey: neuronal activity during a learned motor task. , 1992, Journal of neurophysiology.

[18]  James L. McClelland,et al.  A parallel distributed processing approach to automaticity. , 1992, The American journal of psychology.

[19]  Peter Ford Dominey,et al.  A cortico-subcortical model for generation of spatially accurate sequential saccades. , 1992, Cerebral cortex.

[20]  P. Strick,et al.  Multiple output channels in the basal ganglia. , 1993, Science.

[21]  J. Tanji,et al.  The role of premotor cortex and the supplementary motor area in the temporal control of movement in man. , 1993, Brain : a journal of neurology.

[22]  P. Goldman-Rakic,et al.  Prefrontal connections of medial motor areas in the rhesus monkey , 1993, The Journal of comparative neurology.

[23]  M. Delong,et al.  Role of the cerebellum and basal ganglia in voluntary movement , 2017 .

[24]  W. Schultz,et al.  Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[25]  R. Passingham.  The frontal lobes and voluntary action , 1993 .

[26]  Tim Curran,et al.  Attentional and Nonattentional Forms of Sequence Learning , 1993 .

[27]  RP Dum,et al.  Topographic organization of corticospinal projections from the frontal lobe: motor areas on the lateral surface of the hemisphere , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[28]  G. Rizzolatti,et al.  Corticocortical connections of area F3 (SMA‐proper) and area F6 (pre‐SMA) in the macaque monkey , 1993, The Journal of comparative neurology.

[29]  S. Wise,et al.  Visuospatial versus visuomotor activity in the premotor and prefrontal cortex of a primate , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[30]  Apostolos P. Georgopoulos,et al.  New concepts in generation of movement , 1994, Neuron.

[31]  Joel L. Davis,et al.  A Model of How the Basal Ganglia Generate and Use Neural Signals That Predict Reinforcement , 1994 .

[32]  J. Tanji.  The supplementary motor area in the cerebral cortex , 1994, Neuroscience Research.

[33]  J. B. Preston,et al.  Interconnections between the prefrontal cortex and the premotor areas in the frontal lobe , 1994, The Journal of comparative neurology.

[34]  P. Strick,et al.  Anatomical evidence for cerebellar and basal ganglia involvement in higher cognitive function. , 1994, Science.

[35]  D. Brooks,et al.  Motor sequence learning: a study with positron emission tomography , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[36]  A M Graybiel,et al.  The basal ganglia and adaptive motor control. , 1994, Science.

[37]  J. Tanji,et al.  Neuronal activity in the primate supplementary, pre-supplementary and premotor cortex during externally and internally instructed sequential movements , 1994, Neuroscience Research.

[38]  M. Hallett,et al.  Modulation of cortical motor output maps during development of implicit and explicit knowledge. , 1994, Science.

[39]  G Rizzolatti,et al.  Corticospinal projections from mesial frontal and cingulate areas in the monkey. , 1994, Neuroreport.

[40]  D. Hoffman,et al.  Differential effects of muscimol microinjection into dorsal and ventral aspects of the premotor cortex of monkeys. , 1994, Journal of neurophysiology.

[41]  Peter L. Strick,et al.  Macro-organization of the circuits connecting the basal ganglia with the cortical motor areas , 1995 .

[42]  S. Keele,et al.  Modularity of Sequence Learning Systems in Humans , 1995 .

[43]  H. Imamizu,et al.  The locus of visual-motor learning at the task or manipulator level: implications from intermanual transfer. , 1995, Journal of experimental psychology. Human perception and performance.

[44]  M Petrides,et al.  Impairments on nonspatial self-ordered and externally ordered working memory tasks after lesions of the mid-dorsal part of the lateral frontal cortex in the monkey , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[45]  A. Barto,et al.  Adaptive Critics and the Basal Ganglia , 1994 .

[46]  Richard S. Sutton,et al.  Computational Schemes and Neural Network Models for Formation and Control of Multijoint Arm Trajectory , 1995 .

[47]  E. Hutchins.  Cognition in the wild , 1995 .

[48]  O. Hikosaka,et al.  Learning of sequential movements in the monkey: process of learning and retention of memory. , 1995, Journal of neurophysiology.

[49]  S P Wise,et al.  Distributed modular architectures linking basal ganglia, cerebellum, and cerebral cortex: their role in planning and controlling action. , 1995, Cerebral cortex.

[50]  A. Dickinson,et al.  Reward-related signals carried by dopamine neurons. , 1995 .

[51]  M. Merzenich,et al.  Use-dependent alterations of movement representations in primary motor cortex of adult squirrel monkeys , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[52]  T. Sejnowski,et al.  How the Basal Ganglia Make Decisions , 1996 .

[53]  P. Dayan,et al.  A framework for mesencephalic dopamine systems based on predictive Hebbian learning , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[54]  A. Berthoz,et al.  Functional Anatomy of a Prelearned Sequence of Horizontal Saccades in Humans , 1996, The Journal of Neuroscience.

[55]  J Tanji,et al.  Changing directions of forthcoming arm movements: neuronal activity in the presupplementary and supplementary motor area of monkey cerebral cortex. , 1996, Journal of neurophysiology.

[56]  O. Hikosaka,et al.  Anticipatory saccades in sequential procedural learning in monkeys. , 1996, Journal of neurophysiology.

[57]  J Tanji,et al.  Role for cells in the presupplementary motor area in updating motor plans. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[58]  E. Bizzi,et al.  Consolidation in human motor memory , 1996, Nature.

[59]  Alan C. Evans,et al.  Functional Anatomy of Visuomotor Skill Learning in Human Subjects Examined with Positron Emission Tomography , 1996, The European journal of neuroscience.

[60]  Masataka Watanabe.  Reward expectancy in primate prefrental neurons , 1996, Nature.

[61]  O. Hikosaka,et al.  Activation of human presupplementary motor area in learning of sequential procedures: a functional MRI study. , 1996, Journal of neurophysiology.

[62]  R. Andersen,et al.  Multimodal representation of space in the posterior parietal cortex and its use in planning movements. , 1997, Annual review of neuroscience.

[63]  1551 Corticostriatal input from the presupplementary motor area: Its spatial segregation from supplementary motor area input , 1997, Neuroscience Research.

[64]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[65]  R. Shadmehr,et al.  Neural correlates of motor memory consolidation. , 1997, Science.

[66]  M. Hallett,et al.  Frontal and parietal networks for conditional motor learning: a positron emission tomography study. , 1997, Journal of neurophysiology.

[67]  Richard S. J. Frackowiak,et al.  Anatomy of motor learning. II. Subcortical structures and learning by trial and error. , 1997, Journal of neurophysiology.

[68]  Paul B. Johnson,et al.  Premotor and parietal cortex: corticocortical connectivity and combinatorial computations. , 1997, Annual review of neuroscience.

[69]  Richard S. J. Frackowiak,et al.  Anatomy of motor learning. I. Frontal cortex and attention to action. , 1997, Journal of neurophysiology.

[70]  S. Rauch,et al.  Striatal recruitment during an implicit sequence learning task as measured by functional magnetic resonance imaging , 1997, Human brain mapping.

[71]  中原 裕之.  Sequential Decision Making in Biological Systems: The Role of Nonlinear Dynamical Phenomena in Working Memory and Reinforcement Learning in Long-Term Memory(生物システムの逐次的意思決定: 作業記憶の非線形ダイナミクスの役割と長期記憶での強化学習の役割) , 1997 .

[72]  Edward E. Smith,et al.  Temporal dynamics of brain activation during a working memory task , 1997, Nature.

[73]  H. Nakahara.  Multiple representations in the basal ganglia loops for acquisition and execution of sequential motor control , 1997 .

[74]  O. Hikosaka,et al.  Differential roles of monkey striatum in learning of sequential hand movement , 1997, Experimental Brain Research.

[75]  G. Rizzolatti,et al.  Parietal cortex: from sight to action , 1997, Current Opinion in Neurobiology.

[76]  A. B. Mayer,et al.  Visuomotor transformations: early cortical mechanisms of reaching , 1998, Current Opinion in Neurobiology.

[77]  M. Botvinick,et al.  Anterior cingulate cortex, error detection, and the online monitoring of performance. , 1998, Science.

[78]  O. Hikosaka,et al.  Role of monkey cerebellar nuclei in skill for sequential movement. , 1998, Journal of neurophysiology.

[79]  W. Schultz,et al.  Learning of sequential movements by neural network model with dopamine-like reinforcement signal , 1998, Experimental Brain Research.

[80]  M. Hallett,et al.  Dynamic cortical involvement in implicit and explicit motor sequence learning. A PET study. , 1998, Brain : a journal of neurology.

[81]  Daniel B. Willingham,et al.  A Neuropsychological Theory of Motor Skill Learning , 2004 .

[82]  O. Hikosaka,et al.  Characteristics of a long-term procedural skill in the monkey , 1998, Experimental Brain Research.

[83]  G. Glover,et al.  Differential activation of dorsal basal ganglia during externally and self paced sequences of arm movements. , 1998 .

[84]  J. Tanji,et al.  Role for cingulate motor area cells in voluntary movement selection based on reward. , 1998, Science.

[85]  Leslie G. Ungerleider,et al.  The acquisition of skilled motor performance: fast and slow experience-driven changes in primary motor cortex. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[86]  Kae Nakamura,et al.  Neuronal activity in medial frontal cortex during learning of sequential procedures. , 1998, Journal of neurophysiology.

[87]  Scott T. Grafton,et al.  Abstract and Effector-Specific Representations of Motor Sequences Identified with PET , 1998, The Journal of Neuroscience.

[88]  O. Hikosaka,et al.  Transition of Brain Activation from Frontal to Parietal Areas in Visuomotor Sequence Learning , 1998, The Journal of Neuroscience.

[89]  Kenji Doya,et al.  Near-Saddle-Node Bifurcation Behavior as Dynamics in Working Memory for Goal-Directed Behavior , 1998, Neural Computation.

[90]  C. Gross,et al.  Spatial maps for the control of movement , 1998, Current Opinion in Neurobiology.

[91]  S. Petersen,et al.  The effects of practice on the functional anatomy of task performance. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[92]  R. Passingham,et al.  The Time Course of Changes during Motor Sequence Learning: A Whole-Brain fMRI Study , 1998, NeuroImage.

[93]  M. Kawato,et al.  Adaptive internal model of intrinsic kinematics involved in learning an aiming task. , 1998, Journal of experimental psychology. Human perception and performance.

[94]  Joshua W. Brown,et al.  How the Basal Ganglia Use Parallel Excitatory and Inhibitory Learning Pathways to Selectively Respond to Unexpected Rewarding Cues , 1999, The Journal of Neuroscience.

[95]  Shun-ichi Amari,et al.  Statistical analysis of learning dynamics , 1999, Signal Process..

[96]  Jonathan D. Cohen,et al.  A Biologically Based Computational Model of Working Memory , 1999 .

[97]  O. Hikosaka,et al.  Presupplementary Motor Area Activation during Sequence Learning Reflects Visuo-Motor Association , 1999, The Journal of Neuroscience.

[98]  David A. Rosenbaum,et al.  Remembered positions: stored locations or stored postures? , 1999, Experimental Brain Research.

[99]  John G. Taylor,et al.  A Hard Wired Model of Coupled Frontal Working memories for various Tasks , 1999, Inf. Sci..

[100]  John W. Krakauer,et al.  Independent learning of internal models for kinematic and dynamic control of reaching , 1999, Nature Neuroscience.

[101]  K. Doya,et al.  Parallel neural networks for learning sequential procedures , 1999, Trends in Neurosciences.

[102]  O Hikosaka,et al.  Effects of local inactivation of monkey medial frontal cortex in learning of sequential procedures. , 1999, Journal of neurophysiology.

[103]  M. Gazzaniga,et al.  The new cognitive neurosciences , 2000 .

[104]  Miya K. Rand,et al.  Characteristics of sequential movements during early learning period in monkeys , 2000, Experimental Brain Research.

[105]  O. Hikosaka,et al.  What and When: Parallel and Convergent Processing in Motor Control , 2000, The Journal of Neuroscience.

[106]  Alexander M. Harner,et al.  Evidence for effector independent and dependent representations and their differential time course of acquisition during motor sequence learning , 2000, Experimental Brain Research.

[107]  E. Procyk,et al.  Anterior cingulate activity during routine and non-routine sequential behaviors in macaques , 2000, Nature Neuroscience.

[108]  David S. Touretzky,et al.  Behavioral considerations suggest an average reward TD model of the dopamine system , 2000, Neurocomputing.