Coupling internal cerebellar models enhances online adaptation and supports offline consolidation in sensorimotor tasks

The cerebellum is thought to mediate sensorimotor adaptation through the acquisition of internal models of the body-environment interaction. These representations can be of two types, identified as forward and inverse models. The first predicts the sensory consequences of actions, while the second provides the correct commands to achieve desired state transitions. In this paper, we propose a composite architecture consisting of multiple cerebellar internal models to account for the adaptation performance of humans during sensorimotor learning. The proposed model takes inspiration from the cerebellar microcomplex circuit, and employs spiking neurons to process information. We investigate the intrinsic properties of the cerebellar circuitry subserving efficient adaptation properties, and we assess the complementary contributions of internal representations by simulating our model in a procedural adaptation task. Our simulation results suggest that the coupling of internal models enhances learning performance significantly (compared with independent forward and inverse models), and it allows for the reproduction of human adaptation capabilities. Furthermore, we provide a computational explanation for the performance improvement observed after one night of sleep in a wide range of sensorimotor tasks. We predict that internal model coupling is a necessary condition for the offline consolidation of procedural memories.

[1]  Daniel M. Wolpert,et al.  Forward Models for Physiological Motor Control , 1996, Neural Networks.

[2]  Wade G. Regehr,et al.  Timing dependence of the induction of cerebellar LTD , 2008, Neuropharmacology.

[3]  R. Huganir,et al.  Reevaluating the Role of LTD in Cerebellar Motor Learning , 2011, Neuron.

[4]  J. Born,et al.  Effects of Early and Late Nocturnal Sleep on Declarative and Procedural Memory , 1997, Journal of Cognitive Neuroscience.

[5]  Bruno A Olshausen,et al.  Sparse coding of sensory inputs , 2004, Current Opinion in Neurobiology.

[6]  D. Watanabe,et al.  Roles of Inhibitory Interneurons in the Cerebellar Cortex , 2002, Annals of the New York Academy of Sciences.

[7]  M. Arbib,et al.  Role of the cerebellum in reaching movements in humans. II. A neural model of the intermediate cerebellum , 1998, The European journal of neuroscience.

[8]  M. Kawato,et al.  Internal forward models in the cerebellum: fMRI study on grip force and load force coupling. , 2003, Progress in brain research.

[9]  伊藤 正男 The cerebellum and neural control , 1984 .

[10]  B. McNaughton,et al.  Reactivation of hippocampal ensemble memories during sleep. , 1994, Science.

[11]  Masao Ito Cerebellar circuitry as a neuronal machine , 2006, Progress in Neurobiology.

[12]  W. T. Thach,et al.  Throwing while looking through prisms. II. Specificity and storage of multiple gaze-throw calibrations. , 1996, Brain : a journal of neurology.

[13]  L. Christensen,et al.  University of Birmingham Disruption of state estimation in the human lateral cerebellum , 2007 .

[14]  T. Ebner,et al.  Force field effects on cerebellar Purkinje cell discharge with implications for internal models , 2006, Nature Neuroscience.

[15]  J. Krakauer,et al.  An Implicit Plan Overrides an Explicit Strategy during Visuomotor Adaptation , 2006, The Journal of Neuroscience.

[16]  Hiroshi Imamizu,et al.  Human cerebellar activity reflecting an acquired internal model of a new tool , 2000, Nature.

[17]  D. Wolpert,et al.  Internal models in the cerebellum , 1998, Trends in Cognitive Sciences.

[18]  Y. Miyashita Neuronal correlate of visual associative long-term memory in the primate temporal cortex , 1988, Nature.

[19]  G. Tononi,et al.  Local sleep and learning , 2004, Nature.

[20]  M. Kawato,et al.  A hierarchical neural-network model for control and learning of voluntary movement , 2004, Biological Cybernetics.

[21]  K. Doya,et al.  Unsupervised learning of granule cell sparse codes enhances cerebellar adaptive control , 2001, Neuroscience.

[22]  R. Stickgold,et al.  Practice with Sleep Makes Perfect Sleep-Dependent Motor Skill Learning , 2002, Neuron.

[23]  Giulio Sandini,et al.  Active learning for sensorimotor coordinations of autonomous robots , 2009, 2009 2nd Conference on Human System Interactions.

[24]  Albert K. Lee,et al.  Memory of Sequential Experience in the Hippocampus during Slow Wave Sleep , 2002, Neuron.

[25]  Professor Dr. John C. Eccles,et al.  The Cerebellum as a Neuronal Machine , 1967, Springer Berlin Heidelberg.

[26]  M Ito,et al.  Neurophysiological aspects of the cerebellar motor control system. , 1970, International journal of neurology.

[27]  Axel Cleeremans,et al.  Experience-dependent changes in cerebral activation during human REM sleep , 2000, Nature Neuroscience.

[28]  T. Flash,et al.  The coordination of arm movements: an experimentally confirmed mathematical model , 1985, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[29]  Mitsuo Kawato,et al.  Neural network control for a closed-loop System using Feedback-error-learning , 1993, Neural Networks.

[30]  Shun-ichi Amari,et al.  A computational study of synaptic mechanisms of partial memory transfer in cerebellar vestibulo-ocular-reflex learning , 2008, Journal of Computational Neuroscience.

[31]  Paolo Dario,et al.  A bio-inspired predictive sensory-motor coordination scheme for robot reaching and preshaping , 2008, Auton. Robots.

[32]  I. Raman,et al.  Potentiation of Mossy Fiber EPSCs in the Cerebellar Nuclei by NMDA Receptor Activation followed by Postinhibitory Rebound Current , 2006, Neuron.

[33]  E. D’Angelo,et al.  Beyond parallel fiber LTD: the diversity of synaptic and non-synaptic plasticity in the cerebellum , 2001, Nature Neuroscience.

[34]  Roger Y Tsien,et al.  A new form of cerebellar long-term potentiation is postsynaptic and depends on nitric oxide but not cAMP , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[35]  Scott T. Grafton,et al.  A lesion of the posterior parietal cortex disrupts on-line adjustments during aiming movements , 2002, Neuropsychologia.

[36]  T. Sejnowski,et al.  Storing covariance with nonlinearly interacting neurons , 1977, Journal of mathematical biology.

[37]  Masao Ito,et al.  Long-lasting depression of parallel fiber-Purkinje cell transmission induced by conjunctive stimulation of parallel fibers and climbing fibers in the cerebellar cortex , 1982, Neuroscience Letters.

[38]  Stefan Schaal,et al.  Incremental Online Learning in High Dimensions , 2005, Neural Computation.

[39]  R C Miall,et al.  The cerebellum, predictive control and motor coordination. , 2007, Novartis Foundation symposium.

[40]  R. Stickgold,et al.  Sleep-Dependent Learning and Memory Consolidation , 2004, Neuron.

[41]  L. Boyd,et al.  Sleep Enhances Off-line Spatial and Temporal Motor Learning After Stroke , 2009, Neurorehabilitation and neural repair.

[42]  R. Stickgold Sleep-dependent memory consolidation , 2005, Nature.

[43]  J. Albus A Theory of Cerebellar Function , 1971 .

[44]  A. Karni,et al.  Daytime sleep condenses the time course of motor memory consolidation , 2007, Nature Neuroscience.

[45]  S. Grossberg,et al.  Metabotropic Glutamate Receptor Activation in Cerebellar Purkinje Cells as Substrate for Adaptive Timing of the Classically Conditioned Eye-Blink Response , 1996, The Journal of Neuroscience.

[46]  M. Häusser,et al.  Integration of quanta in cerebellar granule cells during sensory processing , 2004, Nature.

[47]  Mitsuo Kawato,et al.  Internal models for motor control and trajectory planning , 1999, Current Opinion in Neurobiology.

[48]  J L van Hemmen,et al.  Time window control: a model for cerebellar function based on synchronization, reverberation, and time slicing. , 2000, Progress in brain research.

[49]  D. Marr A theory of cerebellar cortex , 1969, The Journal of physiology.

[50]  C. Hansel,et al.  Purkinje Cell-Specific Knockout of the Protein Phosphatase PP2B Impairs Potentiation and Cerebellar Motor Learning , 2010, Neuron.

[51]  G. Augustine,et al.  Quantification of spread of cerebellar long-term depression with chemical two-photon uncaging of glutamate. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[52]  K. Nakayama,et al.  Sleep-dependent learning: a nap is as good as a night , 2003, Nature Neuroscience.

[53]  J. Csicsvari,et al.  Replay and Time Compression of Recurring Spike Sequences in the Hippocampus , 1999, The Journal of Neuroscience.

[54]  R. Rosenfeld Nature , 2009, Otolaryngology--head and neck surgery : official journal of American Academy of Otolaryngology-Head and Neck Surgery.

[55]  M. Walker,et al.  Daytime Naps, Motor Memory Consolidation and Regionally Specific Sleep Spindles , 2007, PloS one.

[56]  D. Wolpert,et al.  Is the cerebellum a smith predictor? , 1993, Journal of motor behavior.

[57]  E. Fine,et al.  The history of the development of the cerebellar examination. , 2002, Seminars in neurology.

[58]  Michael I. Jordan,et al.  Forward Models: Supervised Learning with a Distal Teacher , 1992, Cogn. Sci..

[59]  P. Dean,et al.  The cerebellar microcircuit as an adaptive filter: experimental and computational evidence , 2010, Nature Reviews Neuroscience.

[60]  Tadashi Yamazaki,et al.  The cerebellum as a liquid state machine , 2007, Neural Networks.

[61]  R. Stickgold,et al.  Sleep-dependent motor memory plasticity in the human brain , 2005, Neuroscience.

[62]  W T Thach,et al.  The cerebellum and the adaptive coordination of movement. , 1992, Annual review of neuroscience.

[63]  J. Szentágothai,et al.  Quantitative histological analysis of the cerebellar nuclei in the cat. I. Numerical data on cells and on synapses , 1977, Experimental Brain Research.

[64]  J. Hobson,et al.  Visual discrimination learning requires sleep after training , 2000, Nature Neuroscience.

[65]  Masao Ito Bases and implications of learning in the cerebellum--adaptive control and internal model mechanism. , 2005, Progress in brain research.

[66]  J. Krakauer,et al.  Error correction, sensory prediction, and adaptation in motor control. , 2010, Annual review of neuroscience.

[67]  C. Bell,et al.  The generation and subtraction of sensory expectations within cerebellum-like structures. , 1997, Brain, behavior and evolution.

[68]  J. Hobson,et al.  Visual Discrimination Task Improvement: A Multi-Step Process Occurring During Sleep , 2000, Journal of Cognitive Neuroscience.

[69]  Eilon Vaadia,et al.  Neural basis of sensorimotor learning: modifying internal models , 2008, Current Opinion in Neurobiology.

[70]  Graham C. Goodwin,et al.  Adaptive filtering prediction and control , 1984 .

[71]  S. Grossberg,et al.  A neural model of cerebellar learning for arm movement control: cortico-spino-cerebellar dynamics. , 1997, Learning & memory.

[72]  J. W. Moore,et al.  Adaptive timing in neural networks: The conditioned response , 1988, Biological Cybernetics.

[73]  G. Laurent,et al.  Adaptive regulation of sparseness by feedforward inhibition , 2007, Nature Neuroscience.

[74]  Penelope A. Lewis,et al.  Keeping time in your sleep: Overnight consolidation of temporal rhythm , 2011, Neuropsychologia.

[75]  Masao Ito,et al.  Historical Review of the Significance of the Cerebellum and the Role of Purkinje Cells in Motor Learning , 2002, Annals of the New York Academy of Sciences.

[76]  D. Tolhurst,et al.  Characterizing the sparseness of neural codes , 2001, Network.

[77]  Zoubin Ghahramani,et al.  Computational principles of movement neuroscience , 2000, Nature Neuroscience.

[78]  Eduardo Ros,et al.  Spike Timing Regulation on the Millisecond Scale by Distributed Synaptic Plasticity at the Cerebellum Input Stage: A Simulation Study , 2013, Front. Comput. Neurosci..

[79]  J. Born,et al.  The memory function of sleep , 2010, Nature Reviews Neuroscience.

[80]  Wei Zhang,et al.  Long-Term Depression at the Mossy Fiber–Deep Cerebellar Nucleus Synapse , 2006, The Journal of Neuroscience.

[81]  T. Oshima,et al.  Control properties induced by the existence of antagonistic pairs of bi-articular muscles-Mechanical engineering model analyses , 1994 .

[82]  W. T. Thach,et al.  Throwing while looking through prisms , 2005 .

[83]  J. Born,et al.  Sleep forms memory for finger skills , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[84]  S G Lisberger,et al.  Changes in the responses of Purkinje cells in the floccular complex of monkeys after motor learning in smooth pursuit eye movements. , 2000, Journal of neurophysiology.

[85]  D M Wolpert,et al.  Multiple paired forward and inverse models for motor control , 1998, Neural Networks.

[86]  B. McNaughton,et al.  Reactivation of Hippocampal Cell Assemblies: Effects of Behavioral State, Experience, and EEG Dynamics , 1999, The Journal of Neuroscience.

[87]  M. Shishuku [Visual discrimination]. , 1974, Kokubyo Gakkai zasshi. The Journal of the Stomatological Society, Japan.

[88]  F. Lacquaniti,et al.  Does the brain model Newton's laws? , 2001, Nature Neuroscience.

[89]  Richard A Andersen,et al.  Forward estimation of movement state in posterior parietal cortex , 2008, Proceedings of the National Academy of Sciences.

[90]  R. Stickgold,et al.  Sleep and the time course of motor skill learning. , 2003, Learning & memory.

[91]  R. Passingham,et al.  Sleep-Related Consolidation of a Visuomotor Skill: Brain Mechanisms as Assessed by Functional Magnetic Resonance Imaging , 2003, The Journal of Neuroscience.

[92]  Olivier J. M. D. Coenen,et al.  Model of granular layer encoding in the cerebellum , 2004, Neurocomputing.

[93]  J. Assad,et al.  Dissociation of visual, motor and predictive signals in parietal cortex during visual guidance , 1999, Nature Neuroscience.

[94]  J. Born,et al.  Learning-Dependent Increases in Sleep Spindle Density , 2002, The Journal of Neuroscience.

[95]  Scott T. Grafton,et al.  Role of the posterior parietal cortex in updating reaching movements to a visual target , 1999, Nature Neuroscience.

[96]  M. Mauk,et al.  Simulations of Cerebellar Motor Learning: Computational Analysis of Plasticity at the Mossy Fiber to Deep Nucleus Synapse , 1999, The Journal of Neuroscience.

[97]  J. Nadal,et al.  Optimal Information Storage and the Distribution of Synaptic Weights Perceptron versus Purkinje Cell , 2004, Neuron.

[98]  J. Hámori,et al.  Quantitative morphology and synaptology of cerebellar glomeruli in the rat , 1988, Anatomy and Embryology.

[99]  J. Krakauer,et al.  A computational neuroanatomy for motor control , 2008, Experimental Brain Research.

[100]  Eduardo Ros,et al.  A real-time spiking cerebellum model for learning robot control , 2008, Biosyst..