Sensorimotor learning under switching dynamics

Sensorimotor learning under switching dynamics James Barry Heald Humans have a remarkable capacity to learn new motor behaviours without forgetting old ones. This capacity relies on the ability to acquire and express multiple motor memories without interference. Here we combine behavioural experiments and computational modelling to investigate how the sensorimotor system uses contextual information to create, update and recall motor memories. We first examine the role of muscle co-contraction in the learning of novel movement dynamics. We show that muscle co-contraction, as measured by surface electromyography, accelerates motor learning. We then explore the role of control points on objects in the formation of motor memories during object manipulation. We show that opposing dynamic perturbations, which interfere when controlling a single location on an object, can be learned when each is associated with a separate control point. To account for these results, we develop a parametric switching state-space model, in which the association between cues (control points) and contexts (dynamics) is learned from experience rather than fixed. We then extend this model to a Bayesian nonparametric switching state-space model, in which the number of contexts and cues are learned online rather than specified in advance. This model can instantiate new memories when novel perturbations are experienced and exhibits spontaneous recovery of a memory that has been ostensibly overwritten. To test the model, we perform an experiment in which we briefly present a previously experienced perturbation after behaviour has returned to baseline. As predicted, we observe a qualitatively distinct and more pronounced form of recovery, which we refer to as evoked recovery. Finally, we investigate Bayesian context estimation using single-trial learning. We show that people are able to learn novel associations between cues and contexts and that they use both contextual cues and state feedback to infer the current context and partition learning between memories. Taken together, these findings further the understanding of the behaviour and computational principles of sensorimotor learning under switching dynamics.

[1]  S. Gandevia,et al.  The proprioceptive senses: their roles in signaling body shape, body position and movement, and muscle force. , 2012, Physiological reviews.

[2]  R. Johansson,et al.  Coordinated isometric muscle commands adequately and erroneously programmed for the weight during lifting task with precision grip , 2004, Experimental Brain Research.

[3]  M. Tanaka,et al.  Coding of modified body schema during tool use by macaque postcentral neurones. , 1996, Neuroreport.

[4]  Sinan Yildirim,et al.  Calibrating the Gaussian multi-target tracking model , 2014, Statistics and Computing.

[5]  Jordan A Taylor,et al.  Explicit and Implicit Processes Constitute the Fast and Slow Processes of Sensorimotor Learning , 2015, The Journal of Neuroscience.

[6]  F. Mussa-Ivaldi,et al.  The motor system does not learn the dynamics of the arm by rote memorization of past experience. , 1997, Journal of neurophysiology.

[7]  Michael I. Jordan,et al.  Nonparametric Bayesian Learning of Switching Linear Dynamical Systems , 2008, NIPS.

[8]  Atsushi Iriki,et al.  Shaping multisensory action–space with tools: evidence from patients with cross-modal extinction , 2005, Neuropsychologia.

[9]  Michael I. Jordan,et al.  Forward Models: Supervised Learning with a Distal Teacher , 1992, Cogn. Sci..

[10]  M. Kawato,et al.  Adaptation to Stable and Unstable Dynamics Achieved By Combined Impedance Control and Inverse Dynamics Model , 2003 .

[11]  J. Krakauer,et al.  Generalization of Motor Learning Depends on the History of Prior Action , 2006, PLoS biology.

[12]  R. Trumbower,et al.  Interactions between limb and environmental mechanics influence stretch reflex sensitivity in the human arm. , 2010, Journal of neurophysiology.

[13]  Robert H. Wurtz,et al.  Influence of the thalamus on spatial visual processing in frontal cortex , 2006, Nature.

[14]  D. Wolpert,et al.  Multiple Grasp-Specific Representations of Tool Dynamics Mediate Skillful Manipulation , 2010, Current Biology.

[15]  Daniel M Wolpert,et al.  Q&A: Robotics as a tool to understand the brain , 2010, BMC Biology.

[16]  P. Berkes,et al.  Statistically Optimal Perception and Learning: from Behavior to Neural Representations , 2022 .

[17]  Daniel M. Wolpert,et al.  Making smooth moves , 2022 .

[18]  Frédéric Crevecoeur,et al.  Rapid Online Selection between Multiple Motor Plans , 2014, The Journal of Neuroscience.

[19]  Justin L. Gardner,et al.  A Switching Observer for Human Perceptual Estimation , 2017, Neuron.

[20]  G. Stelmach,et al.  Adaptation to gradual as compared with sudden visuo-motor distortions , 1997, Experimental Brain Research.

[21]  J. Kalaska,et al.  Context-dependent anticipation of different task dynamics: rapid recall of appropriate motor skills using visual cues. , 2003, Journal of neurophysiology.

[22]  K. Ito,et al.  On State Estimation in Switching Environments , 1970 .

[23]  Karl J. Friston,et al.  Uncertainty in perception and the Hierarchical Gaussian Filter , 2014, Front. Hum. Neurosci..

[24]  R Shadmehr,et al.  Electromyographic Correlates of Learning an Internal Model of Reaching Movements , 1999, The Journal of Neuroscience.

[25]  Mark R Hinder,et al.  Position information but not force information is used in adapting to changes in environmental dynamics. , 2006, Journal of neurophysiology.

[26]  J. F. Stein,et al.  Role of the cerebellum in the visual guidance of movement , 1986, Nature.

[27]  H. Robbins A Stochastic Approximation Method , 1951 .

[28]  R. C. Oldfield The assessment and analysis of handedness: the Edinburgh inventory. , 1971, Neuropsychologia.

[29]  J. E. Gregory,et al.  The responses of muscle spindles to small, slow movements in passive muscle and during fusimotor activity , 1999, Brain Research.

[30]  D. J. Bennett Torques generated at the human elbow joint in response to constant position errors imposed during voluntary movements , 2004, Experimental Brain Research.

[31]  J. Flanagan,et al.  Sensorimotor memory of weight asymmetry in object manipulation , 2007, Experimental Brain Research.

[32]  D. Wolpert,et al.  Attenuation of Self-Generated Tactile Sensations Is Predictive, not Postdictive , 2006, PLoS biology.

[33]  M. Nissen,et al.  Attentional requirements of learning: Evidence from performance measures , 1987, Cognitive Psychology.

[34]  J. Flanagan,et al.  Independence of perceptual and sensorimotor predictions in the size–weight illusion , 2000, Nature Neuroscience.

[35]  D F Stegeman,et al.  Generalization and transfer of contextual cues in motor learning. , 2015, Journal of neurophysiology.

[36]  J. Albus A Theory of Cerebellar Function , 1971 .

[37]  Rieko Osu,et al.  CNS Learns Stable, Accurate, and Efficient Movements Using a Simple Algorithm , 2008, The Journal of Neuroscience.

[38]  D. Ostry,et al.  Muscle cocontraction following dynamics learning , 2008, Experimental Brain Research.

[39]  R. Shadmehr,et al.  Decay of Motor Memories in the Absence of Error , 2013, The Journal of Neuroscience.

[40]  D. McCloskey,et al.  Kinaesthetic signals and muscle contraction , 1992, Trends in Neurosciences.

[41]  R. Shadmehr,et al.  A Shared Resource between Declarative Memory and Motor Memory , 2010, The Journal of Neuroscience.

[42]  K. Lynch,et al.  The Separate Neural Control of Hand Movements and Contact Forces , 2009, The Journal of Neuroscience.

[43]  Daniel M Wolpert,et al.  Kinematics and Dynamics Are Not Represented Independently in Motor Working Memory: Evidence from an Interference Study , 2002, The Journal of Neuroscience.

[44]  Reza Shadmehr,et al.  Motor variability is not noise, but grist for the learning mill , 2014, Nature Neuroscience.

[45]  Michael I. Jordan,et al.  Hierarchical Dirichlet Processes , 2006 .

[46]  K. Akazawa,et al.  Modulation of reflex EMG and stiffness in response to stretch of human finger muscle. , 1983, Journal of neurophysiology.

[47]  D. Nozaki,et al.  Distinct Motor Plans Form and Retrieve Distinct Motor Memories for Physically Identical Movements , 2012, Current Biology.

[48]  Y. Rossetti,et al.  Three timescales in prism adaptation. , 2015, Journal of neurophysiology.

[49]  Daniel A. Braun,et al.  Optimal Control Predicts Human Performance on Objects with Internal Degrees of Freedom , 2009, PLoS Comput. Biol..

[50]  Peter J. Beek,et al.  Can co-activation reduce kinematic variability? A simulation study , 2005, Biological Cybernetics.

[51]  Karl J. Friston,et al.  Bayesian model selection for group studies , 2009, NeuroImage.

[52]  A. Vallbo,et al.  Discharge patterns in human muscle spindle afferents during isometric voluntary contractions. , 1970, Acta physiologica Scandinavica.

[53]  Lippincott Williams Wilkins Right‐Left Discrimination and Finger Localization , 1960, Neurology.

[54]  Jeremy D Wong,et al.  Visual cues signaling object grasp reduce interference in motor learning. , 2009, Journal of neurophysiology.

[55]  Konrad P. Körding,et al.  Uncertainty of Feedback and State Estimation Determines the Speed of Motor Adaptation , 2009, Front. Comput. Neurosci..

[56]  Olivier Capp'e Online EM Algorithm for Hidden Markov Models , 2009, 0908.2359.

[57]  Reza Shadmehr,et al.  Estimating properties of the fast and slow adaptive processes during sensorimotor adaptation. , 2018, Journal of neurophysiology.

[58]  R. J. van Beers,et al.  Integration of proprioceptive and visual position-information: An experimentally supported model. , 1999, Journal of neurophysiology.

[59]  F A Mussa-Ivaldi,et al.  Adaptive representation of dynamics during learning of a motor task , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[60]  Michael I. Jordan,et al.  A Sticky HDP-HMM With Application to Speaker Diarization , 2009, 0905.2592.

[61]  R. Johansson,et al.  Eye–Hand Coordination in Object Manipulation , 2001, The Journal of Neuroscience.

[62]  Karl-Theodor Kalveram,et al.  Motor adaptation to different dynamic environments is facilitated by indicative context stimuli , 2004, Psychological research.

[63]  M. Husain,et al.  Reward Pays the Cost of Noise Reduction in Motor and Cognitive Control , 2015, Current Biology.

[64]  Karl J. Friston,et al.  A Bayesian Foundation for Individual Learning Under Uncertainty , 2011, Front. Hum. Neurosci..

[65]  A. Vallbo,et al.  Human muscle spindle discharge during isometric voluntary contractions. Amplitude relations between spindle frequency and torque. , 1974, Acta physiologica Scandinavica.

[66]  Michael Dimitriou,et al.  Human Muscle Spindle Sensitivity Reflects the Balance of Activity between Antagonistic Muscles , 2014, The Journal of Neuroscience.

[67]  D. Wolpert,et al.  Context-Dependent Decay of Motor Memories during Skill Acquisition , 2013, Current Biology.

[68]  Daniel M Wolpert,et al.  Fast But Fleeting: Adaptive Motor Learning Processes Associated with Aging and Cognitive Decline , 2014, The Journal of Neuroscience.

[69]  D. Wolpert,et al.  Failure to Consolidate the Consolidation Theory of Learning for Sensorimotor Adaptation Tasks , 2004, The Journal of Neuroscience.

[70]  Joel Z. Leibo,et al.  Prefrontal cortex as a meta-reinforcement learning system , 2018, bioRxiv.

[71]  D M Wolpert,et al.  Context estimation for sensorimotor control. , 2000, Journal of neurophysiology.

[72]  D. Wolpert,et al.  Central cancellation of self-produced tickle sensation , 1998, Nature Neuroscience.

[73]  E. Bizzi,et al.  Consolidation in human motor memory , 1996, Nature.

[74]  P. Rack,et al.  Task‐dependent changes in the response of human wrist joints to mechanical disturbance. , 1992, The Journal of physiology.

[75]  Zoubin Ghahramani,et al.  A bayesian view of motor adaptation , 2002 .

[76]  M. Miyazaki,et al.  Testing Bayesian models of human coincidence timing. , 2005, Journal of neurophysiology.

[77]  Emanuel Todorov,et al.  Structured variability of muscle activations supports the minimal intervention principle of motor control. , 2009, Journal of neurophysiology.

[78]  U. Proske,et al.  Detection of movements of the human forearm during and after co‐contractions of muscles acting at the elbow joint , 1998, The Journal of physiology.

[79]  Daniel B. Willingham,et al.  On the development of procedural knowledge. , 1989, Journal of experimental psychology. Learning, memory, and cognition.

[80]  E Bizzi,et al.  Motor learning by field approximation. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[81]  D. Wolpert,et al.  The effect of contextual cues on the encoding of motor memories , 2013, Journal of neurophysiology.

[82]  Matthew T. Kaufman,et al.  A neural network that finds a naturalistic solution for the production of muscle activity , 2015, Nature Neuroscience.

[83]  Matthew T. Kaufman,et al.  Supplementary materials for : Cortical activity in the null space : permitting preparation without movement , 2014 .

[84]  Daniel A. Braun,et al.  Facilitation of learning induced by both random and gradual visuomotor task variation , 2011, Journal of neurophysiology.

[85]  J C Rothwell,et al.  Manual motor performance in a deafferented man. , 1982, Brain : a journal of neurology.

[86]  J. Flanagan,et al.  Learning and recall of incremental kinematic and dynamic sensorimotor transformations , 2005, Experimental Brain Research.

[87]  R A Scheidt,et al.  Persistence of motor adaptation during constrained, multi-joint, arm movements. , 2000, Journal of neurophysiology.

[88]  M. Kawato,et al.  Impedance control balances stability with metabolically costly muscle activation. , 2004, Journal of neurophysiology.

[89]  David M. Huberdeau,et al.  Formation of a long-term memory for visuomotor adaptation following only a few trials of practice. , 2015, Journal of neurophysiology.

[90]  Daniel M Wolpert,et al.  Bayesian integration in force estimation. , 2004, Journal of neurophysiology.

[91]  Helen J. Huang,et al.  Reduction of Metabolic Cost during Motor Learning of Arm Reaching Dynamics , 2012, The Journal of Neuroscience.

[92]  J. Diedrichsen Optimal Task-Dependent Changes of Bimanual Feedback Control and Adaptation , 2007, Current Biology.

[93]  Masaya Hirashima,et al.  Gain Field Encoding of the Kinematics of Both Arms in the Internal Model Enables Flexible Bimanual Action , 2011, The Journal of Neuroscience.

[94]  K. J. Cole,et al.  Memory representations underlying motor commands used during manipulation of common and novel objects. , 1993, Journal of neurophysiology.

[95]  Kevin P. Murphy,et al.  Machine learning - a probabilistic perspective , 2012, Adaptive computation and machine learning series.

[96]  Alec Solway,et al.  Optimal Behavioral Hierarchy , 2014, PLoS Comput. Biol..

[97]  Vincent S. Huang,et al.  Persistence of motor memories reflects statistics of the learning event. , 2009, Journal of neurophysiology.

[98]  T. Ebner,et al.  Position, Direction of Movement, and Speed Tuning of Cerebellar Purkinje Cells during Circular Manual Tracking in Monkey , 2005, The Journal of Neuroscience.

[99]  The effect of muscle contraction on kinaesthesia. , 2002, Advances in experimental medicine and biology.

[100]  Samuel J. Gershman,et al.  A Tutorial on Bayesian Nonparametric Models , 2011, 1106.2697.

[101]  E. Ribot-Ciscar,et al.  Ago-antagonist muscle spindle inputs contribute together to joint movement coding in man , 1998, Brain Research.

[102]  R C Miall,et al.  System Identification Applied to a Visuomotor Task: Near-Optimal Human Performance in a Noisy Changing Task , 2003, The Journal of Neuroscience.

[103]  Ferdinando A. Mussa-Ivaldi,et al.  Learning to push and learning to move: the adaptive control of contact forces , 2015, Front. Comput. Neurosci..

[104]  S. Scott,et al.  Multi-compartment model can explain partial transfer of learning within the same limb between unimanual and bimanual reaching , 2009, Experimental Brain Research.

[105]  Kelvin E. Jones,et al.  Sources of signal-dependent noise during isometric force production. , 2002, Journal of neurophysiology.

[106]  Maurice A. Smith,et al.  Motor Memory Is Encoded as a Gain-Field Combination of Intrinsic and Extrinsic Action Representations , 2012, Journal of Neuroscience.

[107]  M. Ernst,et al.  The statistical determinants of adaptation rate in human reaching. , 2008, Journal of vision.

[108]  D. Wolpert,et al.  Naturalistic approaches to sensorimotor control. , 2011, Progress in brain research.

[109]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[110]  Raymond J. Delnicki,et al.  Overcoming Motor “Forgetting” Through Reinforcement Of Learned Actions , 2012, The Journal of Neuroscience.

[111]  Luigi Acerbi,et al.  Practical Bayesian Optimization for Model Fitting with Bayesian Adaptive Direct Search , 2017, NIPS.

[112]  D. Reinkensmeyer,et al.  Motor adaptation to a small force field superimposed on a large background force , 2007, Experimental Brain Research.

[113]  Jerome Carriot,et al.  Learning to expect the unexpected: rapid updating in primate cerebellum during voluntary self-motion , 2015, Nature Neuroscience.

[114]  David J Reinkensmeyer,et al.  Effect of muscle fatigue on internal model formation and retention during reaching with the arm. , 2006, Journal of applied physiology.

[115]  Arnaud Doucet,et al.  Particle filters for state estimation of jump Markov linear systems , 2001, IEEE Trans. Signal Process..

[116]  Timothy J Ebner,et al.  Representation of limb kinematics in Purkinje cell simple spike discharge is conserved across multiple tasks. , 2011, Journal of neurophysiology.

[117]  M. Kawato,et al.  Acquisition and contextual switching of multiple internal models for different viscous force fields , 2003, Neuroscience Research.

[118]  D. Farina,et al.  The optimal neural strategy for a stable motor task requires a compromise between level of muscle cocontraction and synaptic gain of afferent feedback. , 2015, Journal of neurophysiology.

[119]  D. Ostry,et al.  Relationship between cocontraction, movement kinematics and phasic muscle activity in single-joint arm movement , 2001, Experimental Brain Research.

[120]  I Salimi,et al.  Specificity of internal representations underlying grasping. , 2000, Journal of neurophysiology.

[121]  Robert A. Scheidt,et al.  Visuomotor Learning Enhanced by Augmenting Instantaneous Trajectory Error Feedback during Reaching , 2013, PloS one.

[122]  Michael I. Jordan,et al.  An internal model for sensorimotor integration. , 1995, Science.

[123]  Carl E. Rasmussen,et al.  PILCO: A Model-Based and Data-Efficient Approach to Policy Search , 2011, ICML.

[124]  Paul L Gribble,et al.  Role of cocontraction in arm movement accuracy. , 2003, Journal of neurophysiology.

[125]  Stefan Schaal,et al.  Forward models in visuomotor control. , 2002, Journal of neurophysiology.

[126]  S. Kitazawa,et al.  Sensation at the tips of invisible tools , 2001, Nature Neuroscience.

[127]  Ba-Ngu Vo,et al.  The Gaussian Mixture Probability Hypothesis Density Filter , 2006, IEEE Transactions on Signal Processing.

[128]  J. Krakauer,et al.  Explaining savings for visuomotor adaptation: linear time-invariant state-space models are not sufficient. , 2008, Journal of neurophysiology.

[129]  Amy J Bastian,et al.  Accelerating locomotor savings in learning: compressing four training days to one. , 2018, Journal of neurophysiology.

[130]  Mitsuo Kawato,et al.  Equilibrium-Point Control Hypothesis Examined by Measured Arm Stiffness During Multijoint Movement , 1996, Science.

[131]  D. McCloskey,et al.  Maintenance of constant arm position or force: reflex and volitional components in man. , 1987, The Journal of physiology.

[132]  Philip N. Sabes,et al.  How Each Movement Changes the Next: An Experimental and Theoretical Study of Fast Adaptive Priors in Reaching , 2011, The Journal of Neuroscience.

[133]  J. Sethuraman A CONSTRUCTIVE DEFINITION OF DIRICHLET PRIORS , 1991 .

[134]  Matthew T. Kaufman,et al.  Neural population dynamics during reaching , 2012, Nature.

[135]  Reza Shadmehr,et al.  Learning from Sensory and Reward Prediction Errors during Motor Adaptation , 2011, PLoS Comput. Biol..

[136]  F. A. Mussa-Ivaldi,et al.  Does the motor control system use multiple models and context switching to cope with a variable environment? , 2002, Experimental Brain Research.

[137]  T. Lillicrap,et al.  Preference Distributions of Primary Motor Cortex Neurons Reflect Control Solutions Optimized for Limb Biomechanics , 2013, Neuron.

[138]  Gregor Schöner,et al.  The uncontrolled manifold concept: identifying control variables for a functional task , 1999, Experimental Brain Research.

[139]  Dagmar Sternad,et al.  Exploiting the geometry of the solution space to reduce sensitivity to neuromotor noise , 2018, PLoS Comput. Biol..

[140]  D. Wolpert,et al.  The Value of the Follow-Through Derives from Motor Learning Depending on Future Actions , 2015, Current Biology.

[141]  Rieko Osu,et al.  Endpoint Stiffness of the Arm Is Directionally Tuned to Instability in the Environment , 2007, The Journal of Neuroscience.

[142]  M. Ernst,et al.  Humans integrate visual and haptic information in a statistically optimal fashion , 2002, Nature.

[143]  Rieko Osu,et al.  Different mechanisms involved in adaptation to stable and unstable dynamics. , 2003, Journal of neurophysiology.

[144]  Mitsuo Kawato,et al.  Feedback-Error-Learning Neural Network for Supervised Motor Learning , 1990 .

[145]  D. Sternad,et al.  Decomposition of variability in the execution of goal-oriented tasks: three components of skill improvement. , 2004, Journal of experimental psychology. Human perception and performance.

[146]  Thia Kirubarajan,et al.  Estimation with Applications to Tracking and Navigation: Theory, Algorithms and Software , 2001 .

[147]  Gary C. Sing,et al.  Primitives for Motor Adaptation Reflect Correlated Neural Tuning to Position and Velocity , 2009, Neuron.

[148]  Reza Shadmehr,et al.  The Neural Feedback Response to Error As a Teaching Signal for the Motor Learning System , 2016, The Journal of Neuroscience.

[149]  Alaa A. Ahmed,et al.  Reward feedback accelerates motor learning. , 2015, Journal of neurophysiology.

[150]  D. Wolpert,et al.  Why can't you tickle yourself? , 2000, Neuroreport.

[151]  Maurice A. Smith,et al.  The Decay of Motor Memories Is Independent of Context Change Detection , 2015, PLoS Comput. Biol..

[152]  Nasir H. Bhanpuri,et al.  Predictive Modeling by the Cerebellum Improves Proprioception , 2013, The Journal of Neuroscience.

[153]  F. Mussa-Ivaldi,et al.  Experimentally confirmed mathematical model for human control of a non-rigid object. , 2004, Journal of neurophysiology.

[154]  S. Vijayakumar,et al.  A Computational Model of Limb Impedance Control Based on Principles of Internal Model Uncertainty , 2010, PloS one.

[155]  David W Franklin,et al.  Impedance control and internal model use during the initial stage of adaptation to novel dynamics in humans , 2005, The Journal of physiology.

[156]  Kurt A. Thoroughman,et al.  Rapid Reshaping of Human Motor Generalization , 2005, The Journal of Neuroscience.

[157]  Hermano Igo Krebs,et al.  An Internal Model for Acquisition and Retention of Motor Learning During Arm Reaching , 2009, Neural Computation.

[158]  Philip N. Sabes,et al.  Modeling Sensorimotor Learning with Linear Dynamical Systems , 2006, Neural Computation.

[159]  Denise Y. P. Henriques,et al.  The effects of awareness of the perturbation during motor adaptation on hand localization , 2018 .

[160]  David J Ostry,et al.  Transfer of Motor Learning across Arm Configurations , 2002, The Journal of Neuroscience.

[161]  D. Domkin,et al.  Structure of joint variability in bimanual pointing tasks , 2002, Experimental Brain Research.

[162]  J. Randall Flanagan,et al.  Motor learning of novel dynamics is not represented in a single global coordinate system: evaluation of mixed coordinate representations and local learning , 2013, Journal of neurophysiology.

[163]  John W. Krakauer,et al.  Independent learning of internal models for kinematic and dynamic control of reaching , 1999, Nature Neuroscience.

[164]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[165]  Konrad Paul Kording,et al.  Bayesian integration in sensorimotor learning , 2004, Nature.

[166]  O. Cappé,et al.  On‐line expectation–maximization algorithm for latent data models , 2009 .

[167]  Matthew T. Kaufman,et al.  Cortical Preparatory Activity: Representation of Movement or First Cog in a Dynamical Machine? , 2010, Neuron.

[168]  N. Schweighofer,et al.  Dual Adaptation Supports a Parallel Architecture of Motor Memory , 2009, The Journal of Neuroscience.

[169]  S. Scott Inconvenient Truths about neural processing in primary motor cortex , 2008, The Journal of physiology.

[170]  M. Brainard,et al.  Performance variability enables adaptive plasticity of ‘crystallized’ adult birdsong , 2007, Nature.

[171]  J. Krakauer,et al.  Human sensorimotor learning: adaptation, skill, and beyond , 2011, Current Opinion in Neurobiology.

[172]  J. Krakauer,et al.  Sensory prediction errors drive cerebellum-dependent adaptation of reaching. , 2007, Journal of neurophysiology.

[173]  Iven M. Y. Mareels,et al.  Stability and motor adaptation in human arm movements , 2005, Biological Cybernetics.

[174]  S. Scott,et al.  Feedback control during voluntary motor actions , 2015, Current Opinion in Neurobiology.

[175]  Miles C. Bowman,et al.  Control strategies in object manipulation tasks , 2006, Current Opinion in Neurobiology.

[176]  Reza Shadmehr,et al.  Dissociable effects of the implicit and explicit memory systems on learning control of reaching , 2006, Experimental Brain Research.

[177]  J. Rothwell,et al.  The dissociable effects of punishment and reward on motor learning , 2015, Nature Neuroscience.

[178]  Maurice A. Smith,et al.  Environmental Consistency Determines the Rate of Motor Adaptation , 2014, Current Biology.

[179]  R. Jacobs,et al.  Optimal integration of texture and motion cues to depth , 1999, Vision Research.

[180]  Matthew T. Kaufman,et al.  Perspectives on classical controversies about the motor cortex. , 2017, Journal of neurophysiology.

[181]  Daniel M. Wolpert,et al.  A modular planar robotic manipulandum with end-point torque control , 2009, Journal of Neuroscience Methods.

[182]  Konrad Paul Kording,et al.  Estimating the sources of motor errors for adaptation and generalization , 2008, Nature Neuroscience.

[183]  W. Epstein,et al.  Tool use affects perceived distance, but only when you intend to use it. , 2005, Journal of experimental psychology. Human perception and performance.

[184]  M F Land,et al.  The knowledge base of the oculomotor system. , 1997, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[185]  J. L. Taylor,et al.  Detection of slow movements imposed at the elbow during active flexion in man. , 1992, The Journal of physiology.

[186]  Dagmar Sternad,et al.  Optimal control of a hybrid rhythmic-discrete task: the bouncing ball revisited. , 2010, Journal of neurophysiology.

[187]  J Randall Flanagan,et al.  The role of haptic feedback when manipulating nonrigid objects. , 2012, Journal of neurophysiology.

[188]  H. Akaike A new look at the statistical model identification , 1974 .

[189]  R. Ivry,et al.  The coordination of movement: optimal feedback control and beyond , 2010, Trends in Cognitive Sciences.

[190]  J. Randall Flanagan,et al.  An error-tuned model for sensorimotor learning , 2017, PLoS Comput. Biol..

[191]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[192]  Emanuel Todorov,et al.  Evidence for the Flexible Sensorimotor Strategies Predicted by Optimal Feedback Control , 2007, The Journal of Neuroscience.

[193]  J. Randall Flanagan,et al.  Flexible Representations of Dynamics Are Used in Object Manipulation , 2008, Current Biology.

[194]  Kurt A. Thoroughman,et al.  Trial-by-trial transformation of error into sensorimotor adaptation changes with environmental dynamics. , 2007, Journal of neurophysiology.

[195]  D. Wolpert,et al.  Motor Planning, Not Execution, Separates Motor Memories , 2016, Neuron.

[196]  Christopher D Mah,et al.  Manipulating objects with internal degrees of freedom: evidence for model-based control. , 2002, Journal of neurophysiology.

[197]  Mollie K. Marko,et al.  Sensitivity to prediction error in reach adaptation. , 2012, Journal of neurophysiology.

[198]  Sarah E. Criscimagna-Hemminger,et al.  Cerebellar Contributions to Reach Adaptation and Learning Sensory Consequences of Action , 2012, The Journal of Neuroscience.

[199]  Ashutosh Kumar Singh,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2010 .

[200]  A. Haith,et al.  Unlearning versus savings in visuomotor adaptation: comparing effects of washout, passage of time, and removal of errors on motor memory , 2013, Front. Hum. Neurosci..

[201]  Terence D. Sanger,et al.  Neural network learning control of robot manipulators using gradually increasing task difficulty , 1994, IEEE Trans. Robotics Autom..

[202]  J. Nielsen,et al.  The regulation of presynaptic inhibition during co‐contraction of antagonistic muscles in man. , 1993, The Journal of physiology.

[203]  Paul DiZio,et al.  Motor function in microgravity: movement in weightlessness , 1996, Current Opinion in Neurobiology.

[204]  J. Krakauer,et al.  Adaptation to Visuomotor Transformations: Consolidation, Interference, and Forgetting , 2005, The Journal of Neuroscience.

[205]  D. Wolpert,et al.  Disorders of Body Scheme , 2004 .

[206]  Zoubin Ghahramani,et al.  Computational principles of movement neuroscience , 2000, Nature Neuroscience.

[207]  Y. Bar-Shalom,et al.  The probabilistic data association filter , 2009, IEEE Control Systems.

[208]  C Ghez,et al.  Learning of Visuomotor Transformations for Vectorial Planning of Reaching Trajectories , 2000, The Journal of Neuroscience.

[209]  J. F. Soechting,et al.  Force synergies for multifingered grasping , 2000, Experimental Brain Research.

[210]  Rieko Osu,et al.  The central nervous system stabilizes unstable dynamics by learning optimal impedance , 2001, Nature.

[211]  Olivier White,et al.  Use-Dependent and Error-Based Learning of Motor Behaviors , 2010, The Journal of Neuroscience.

[212]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[213]  Leslie G. Ungerleider,et al.  Functional MRI evidence for adult motor cortex plasticity during motor skill learning , 1995, Nature.

[214]  R. S. Johansson,et al.  Roles of glabrous skin receptors and sensorimotor memory in automatic control of precision grip when lifting rougher or more slippery objects , 2004, Experimental Brain Research.

[215]  D. Wolpert,et al.  Perception of the Consequences of Self-Action Is Temporally Tuned and Event Driven , 2005, Current Biology.

[216]  Reza Shadmehr,et al.  Learning of action through adaptive combination of motor primitives , 2000, Nature.

[217]  L. Selen,et al.  Impedance Control Reduces Instability That Arises from Motor Noise , 2009, The Journal of Neuroscience.

[218]  John W Krakauer,et al.  Motor learning and consolidation: the case of visuomotor rotation. , 2009, Advances in experimental medicine and biology.

[219]  Nicolas Schweighofer,et al.  To Overwrite or to Recall? Individual Differences in Motor Adaptation , 2018, bioRxiv.

[220]  K. Shenoy,et al.  Temporal complexity and heterogeneity of single-neuron activity in premotor and motor cortex. , 2007, Journal of neurophysiology.

[221]  A. Doupe,et al.  Contributions of an avian basal ganglia–forebrain circuit to real-time modulation of song , 2005, Nature.

[222]  J R Flanagan,et al.  The Role of Internal Models in Motion Planning and Control: Evidence from Grip Force Adjustments during Movements of Hand-Held Loads , 1997, The Journal of Neuroscience.

[223]  L. M. M.-T. Theory of Probability , 1929, Nature.

[224]  Geoffrey E. Hinton,et al.  Variational Learning for Switching State-Space Models , 2000, Neural Computation.

[225]  D. Wolpert,et al.  Gone in 0.6 Seconds: The Encoding of Motor Memories Depends on Recent Sensorimotor States , 2012, The Journal of Neuroscience.

[226]  M. Kawato,et al.  Optimal impedance control for task achievement in the presence of signal-dependent noise. , 2004, Journal of neurophysiology.

[227]  R. Johansson,et al.  Factors influencing the force control during precision grip , 2004, Experimental Brain Research.

[228]  Matthew J. Crossley,et al.  Savings upon Re-Aiming in Visuomotor Adaptation , 2015, The Journal of Neuroscience.

[229]  Mitsuo Kawato,et al.  MOSAIC Model for Sensorimotor Learning and Control , 2001, Neural Computation.

[230]  Konrad Paul Kording,et al.  The dynamics of memory as a consequence of optimal adaptation to a changing body , 2007, Nature Neuroscience.

[231]  E. Bizzi,et al.  Neural, mechanical, and geometric factors subserving arm posture in humans , 1985, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[232]  Olivier White,et al.  Flexible Switching of Feedback Control Mechanisms Allows for Learning of Different Task Dynamics , 2013, PloS one.

[233]  Karl J. Friston,et al.  Comparing dynamic causal models , 2004, NeuroImage.

[234]  Karl J. Friston,et al.  Comparing Families of Dynamic Causal Models , 2010, PLoS Comput. Biol..

[235]  Sae Franklin,et al.  Visuomotor feedback gains upregulate during the learning of novel dynamics , 2012, Journal of neurophysiology.

[236]  M. Sommer,et al.  Corollary discharge across the animal kingdom , 2008, Nature Reviews Neuroscience.

[237]  Sumeetpal S. Singh,et al.  Forward Smoothing using Sequential Monte Carlo , 2010, 1012.5390.

[238]  J. Krakauer,et al.  Explicit and Implicit Contributions to Learning in a Sensorimotor Adaptation Task , 2014, The Journal of Neuroscience.

[239]  D. Wolpert,et al.  Temporal and amplitude generalization in motor learning. , 1998, Journal of neurophysiology.

[240]  Tatsuya Kimura,et al.  Cerebellar complex spikes encode both destinations and errors in arm movements , 1998, Nature.

[241]  Vincent S. Huang,et al.  Rethinking Motor Learning and Savings in Adaptation Paradigms: Model-Free Memory for Successful Actions Combines with Internal Models , 2011, Neuron.

[242]  Yohsuke R. Miyamoto,et al.  Temporal structure of motor variability is dynamically regulated and predicts motor learning ability , 2014, Nature Neuroscience.

[243]  Konrad Paul Kording,et al.  Causal Inference in Multisensory Perception , 2007, PloS one.

[244]  Marco Santello,et al.  Retention and interference of learned dexterous manipulation: interaction between multiple sensorimotor processes. , 2015, Journal of neurophysiology.

[245]  R. Shadmehr,et al.  Interacting Adaptive Processes with Different Timescales Underlie Short-Term Motor Learning , 2006, PLoS biology.

[246]  D. Wolpert,et al.  Principles of sensorimotor learning , 2011, Nature Reviews Neuroscience.

[247]  Karl J. Friston,et al.  Bayesian model selection for group studies — Revisited , 2014, NeuroImage.

[248]  Reza Shadmehr,et al.  A memory of errors in sensorimotor learning , 2014, Science.

[249]  Lee A Baugh,et al.  Material evidence: interaction of well-learned priors and sensorimotor memory when lifting objects. , 2012, Journal of neurophysiology.

[250]  T. Brashers-Krug,et al.  Functional Stages in the Formation of Human Long-Term Motor Memory , 1997, The Journal of Neuroscience.

[251]  M. Kawato,et al.  A hierarchical neural-network model for control and learning of voluntary movement , 2004, Biological Cybernetics.

[252]  Luigi Acerbi,et al.  On the Origins of Suboptimality in Human Probabilistic Inference , 2014, PLoS Comput. Biol..

[253]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[254]  Luigi Acerbi,et al.  Internal Representations of Temporal Statistics and Feedback Calibrate Motor-Sensory Interval Timing , 2012, PLoS Comput. Biol..

[255]  J. Randall Flanagan,et al.  Coding and use of tactile signals from the fingertips in object manipulation tasks , 2009, Nature Reviews Neuroscience.

[256]  Masaya Hirashima,et al.  Prospective errors determine motor learning , 2015, Nature Communications.

[257]  J. Lackner,et al.  Rapid adaptation to Coriolis force perturbations of arm trajectory. , 1994, Journal of neurophysiology.

[258]  Konrad Paul Kording,et al.  Relevance of error: what drives motor adaptation? , 2009, Journal of neurophysiology.

[259]  Scott E. Bevans,et al.  Effect of visual error size on saccade adaptation in monkey. , 2003, Journal of neurophysiology.

[260]  Nicholas G. Polson,et al.  Particle Learning and Smoothing , 2010, 1011.1098.

[261]  D. Pélisson,et al.  Sensorimotor adaptation of saccadic eye movements , 2010, Neuroscience & Biobehavioral Reviews.

[262]  M. Kawato,et al.  Formation and control of optimal trajectory in human multijoint arm movement , 1989, Biological Cybernetics.

[263]  Geoffrey E. Hinton,et al.  Adaptive Mixtures of Local Experts , 1991, Neural Computation.

[264]  M. Kawato,et al.  Random presentation enables subjects to adapt to two opposing forces on the hand , 2004, Nature Neuroscience.

[265]  J. Randall Flanagan,et al.  A Single-Rate Context-Dependent Learning Process Underlies Rapid Adaptation to Familiar Object Dynamics , 2011, PLoS Comput. Biol..

[266]  David W. Franklin,et al.  Coordinate Representations for Interference Reduction in Motor Learning , 2015, PloS one.

[267]  Warren B. Powell,et al.  Adaptive stepsizes for recursive estimation with applications in approximate dynamic programming , 2006, Machine Learning.

[268]  S. Scott Optimal feedback control and the neural basis of volitional motor control , 2004, Nature Reviews Neuroscience.

[269]  Marco Santello,et al.  Context-Dependent Learning Interferes with Visuomotor Transformations for Manipulation Planning , 2012, The Journal of Neuroscience.

[270]  R. Shumway,et al.  Dynamic linear models with switching , 1991 .

[271]  G. Schwarz Estimating the Dimension of a Model , 1978 .

[272]  D M Wolpert,et al.  Multiple paired forward and inverse models for motor control , 1998, Neural Networks.

[273]  James R. Lackner,et al.  Adaptation to a novel multi-force environment , 2005, Experimental Brain Research.

[274]  Amy J Bastian,et al.  Two ways to save a newly learned motor pattern. , 2015, Journal of neurophysiology.

[275]  M. Kawato,et al.  Temporal firing patterns of Purkinje cells in the cerebellar ventral paraflocculus during ocular following responses in monkeys II. Complex spikes. , 1998, Journal of neurophysiology.

[276]  Stephen H Scott,et al.  Limited transfer of learning between unimanual and bimanual skills within the same limb , 2006, Nature Neuroscience.

[277]  Sarah E. Pekny,et al.  Protection and Expression of Human Motor Memories , 2011, The Journal of Neuroscience.

[278]  M. Latash,et al.  Motor Control Strategies Revealed in the Structure of Motor Variability , 2002, Exercise and sport sciences reviews.

[279]  R. Shadmehr,et al.  Internal models and contextual cues: encoding serial order and direction of movement. , 2005, Journal of neurophysiology.

[280]  Nicolas Schweighofer,et al.  Between-Trial Forgetting Due to Interference and Time in Motor Adaptation , 2015, PloS one.

[281]  Daniel R Lametti,et al.  Control of movement variability and the regulation of limb impedance. , 2007, Journal of neurophysiology.

[282]  D. Marr A theory of cerebellar cortex , 1969, The Journal of physiology.

[283]  Neville Hogan,et al.  The mechanics of multi-joint posture and movement control , 1985, Biological Cybernetics.

[284]  Michael I. Jordan,et al.  Optimal feedback control as a theory of motor coordination , 2002, Nature Neuroscience.

[285]  Reza Shadmehr,et al.  Quantifying Generalization from Trial-by-Trial Behavior of Adaptive Systems that Learn with Basis Functions: Theory and Experiments in Human Motor Control , 2003, The Journal of Neuroscience.

[286]  Elizabeth T Wilson,et al.  Limb stiffness is modulated with spatial accuracy requirements during movement in the absence of destabilizing forces. , 2009, Journal of neurophysiology.

[287]  Maurice A. Smith,et al.  The Binding of Learning to Action in Motor Adaptation , 2011, PLoS Comput. Biol..

[288]  N. Sawtell,et al.  Cerebellum-like structures and their implications for cerebellar function. , 2008, Annual review of neuroscience.

[289]  T. Flash,et al.  Human arm stiffness characteristics during the maintenance of posture , 1990, Experimental Brain Research.

[290]  Michael S. Brainard,et al.  Covert skill learning in a cortical-basal ganglia circuit , 2012, Nature.

[291]  J. Kalaska,et al.  Colored context cues can facilitate the ability to learn and to switch between multiple dynamical force fields. , 2011, Journal of neurophysiology.

[292]  J. Krakauer,et al.  Error correction, sensory prediction, and adaptation in motor control. , 2010, Annual review of neuroscience.

[293]  L. Pinneo On noise in the nervous system. , 1966, Psychological review.

[294]  Jörn Diedrichsen,et al.  Reach adaptation: what determines whether we learn an internal model of the tool or adapt the model of our arm? , 2008, Journal of neurophysiology.

[295]  Mathias Hegele,et al.  Explicit strategies in force field adaptation , 2019, bioRxiv.

[296]  L. Christensen,et al.  University of Birmingham Disruption of state estimation in the human lateral cerebellum , 2007 .

[297]  D. Wolpert,et al.  Two Eyes for an Eye: The Neuroscience of Force Escalation , 2003, Science.

[298]  T. Flash,et al.  The coordination of arm movements: an experimentally confirmed mathematical model , 1985, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[299]  Sarah E. Criscimagna-Hemminger,et al.  Consolidation Patterns of Human Motor Memory , 2008, The Journal of Neuroscience.

[300]  Stephen H Scott,et al.  Influence of the behavioral goal and environmental obstacles on rapid feedback responses. , 2012, Journal of neurophysiology.

[301]  R. E. Kalman,et al.  New Results in Linear Filtering and Prediction Theory , 1961 .

[302]  Michael I. Jordan,et al.  Bayesian Nonparametric Inference of Switching Dynamic Linear Models , 2010, IEEE Transactions on Signal Processing.

[303]  Michael N. Shadlen,et al.  Temporal context calibrates interval timing , 2010, Nature Neuroscience.

[304]  Reza Shadmehr,et al.  Motor Adaptation as a Process of Reoptimization , 2008, The Journal of Neuroscience.

[305]  M. Athans,et al.  State Estimation for Discrete Systems with Switching Parameters , 1978, IEEE Transactions on Aerospace and Electronic Systems.

[306]  J. A. Pruszynski,et al.  Temporal evolution of "automatic gain-scaling". , 2009, Journal of neurophysiology.

[307]  Dagmar Sternad,et al.  Rhythmic Manipulation of Objects with Complex Dynamics: Predictability over Chaos , 2014, PLoS Comput. Biol..

[308]  Fredrik Lindsten,et al.  Recursive Maximum Likelihood Identification of Jump Markov Nonlinear Systems , 2013, IEEE Transactions on Signal Processing.

[309]  P. Gribble,et al.  Are there distinct neural representations of object and limb dynamics? , 2006, Experimental Brain Research.