Contextual inference underlies the learning of sensorimotor repertoires

Humans spend a lifetime learning, storing and refining a repertoire of motor memories. However, it is unknown what principle underlies the way our continuous stream of sensorimotor experience is segmented into separate memories and how we adapt and use this growing repertoire. Here we develop a principled theory of motor learning based on the key insight that memory creation, updating, and expression are all controlled by a single computation – contextual inference. Unlike dominant theories of single-context learning, our repertoire-learning model accounts for key features of motor learning that had no unified explanation and predicts novel phenomena, which we confirm experimentally. These results suggest that contextual inference is the key principle underlying how a diverse set of experiences is reflected in motor behavior.

[1]  Thomas L. Griffiths,et al.  Formalizing Neurath’s Ship: Approximate Algorithms for Online Causal Learning , 2016, Psychological review.

[2]  D. Wolpert,et al.  Failure to Consolidate the Consolidation Theory of Learning for Sensorimotor Adaptation Tasks , 2004, The Journal of Neuroscience.

[3]  Martin J. McKeown,et al.  Dynamic Bayesian network modeling of fMRI: A comparison of group-analysis methods , 2008, NeuroImage.

[4]  Maurice A. Smith,et al.  Implicit adaptation compensates for erratic explicit strategy in human motor learning , 2020, Nature Neuroscience.

[5]  J. Hodges Memory, Amnesia and the Hippocampal System , 1995 .

[6]  Vincent S. Huang,et al.  Rethinking Motor Learning and Savings in Adaptation Paradigms: Model-Free Memory for Successful Actions Combines with Internal Models , 2011, Neuron.

[7]  R. Shadmehr,et al.  Interacting Adaptive Processes with Different Timescales Underlie Short-Term Motor Learning , 2006, PLoS biology.

[8]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[9]  R. Shadmehr,et al.  A Shared Resource between Declarative Memory and Motor Memory , 2010, The Journal of Neuroscience.

[10]  Ajay Jasra,et al.  Markov Chain Monte Carlo Methods and the Label Switching Problem in Bayesian Mixture Modeling , 2005 .

[11]  Michael I. Jordan,et al.  An HDP-HMM for systems with state persistence , 2008, ICML '08.

[12]  R A Scheidt,et al.  Persistence of motor adaptation during constrained, multi-joint, arm movements. , 2000, Journal of neurophysiology.

[13]  Nicholas G. Polson,et al.  Particle Learning and Smoothing , 2010, 1011.1098.

[14]  A. Haith,et al.  Unlearning versus savings in visuomotor adaptation: comparing effects of washout, passage of time, and removal of errors on motor memory , 2013, Front. Hum. Neurosci..

[15]  Kurt A. Thoroughman,et al.  Trial-by-trial transformation of error into sensorimotor adaptation changes with environmental dynamics. , 2007, Journal of neurophysiology.

[16]  Jordan A Taylor,et al.  Explicit and Implicit Processes Constitute the Fast and Slow Processes of Sensorimotor Learning , 2015, The Journal of Neuroscience.

[17]  Reza Shadmehr,et al.  Learning of action through adaptive combination of motor primitives , 2000, Nature.

[18]  John W Krakauer,et al.  Motor learning and consolidation: the case of visuomotor rotation. , 2009, Advances in experimental medicine and biology.

[19]  L. M. M.-T. Theory of Probability , 1929, Nature.

[20]  Gary C. Sing,et al.  Reduction in Learning Rates Associated with Anterograde Interference Results from Interactions between Different Timescales in Motor Adaptation , 2010, PLoS Comput. Biol..

[21]  Mitsuo Kawato,et al.  MOSAIC Model for Sensorimotor Learning and Control , 2001, Neural Computation.

[22]  Reza Shadmehr,et al.  A memory of errors in sensorimotor learning , 2014, Science.

[23]  Richard B. Ivry,et al.  Taking Aim at the Cognitive Side of Learning in Sensorimotor Adaptation Tasks , 2016, Trends in Cognitive Sciences.

[24]  Samuel Gershman,et al.  Statistical Computations Underlying the Dynamics of Memory Updating , 2014, PLoS Comput. Biol..

[25]  Maurice A. Smith,et al.  Environmental Consistency Determines the Rate of Motor Adaptation , 2014, Current Biology.

[26]  D. Blei,et al.  Context, learning, and extinction. , 2010, Psychological review.

[27]  Tzvi Ganel,et al.  A Double Dissociation Between Action and Perception in the Context of Visual Illusions , 2008, Psychological science.

[28]  David W Franklin,et al.  Impedance control and internal model use during the initial stage of adaptation to novel dynamics in humans , 2005, The Journal of physiology.

[29]  J. Pearce Animal Learning and Cognition: An Introduction , 1997 .

[30]  Daniel M. Wolpert,et al.  A modular planar robotic manipulandum with end-point torque control , 2009, Journal of Neuroscience Methods.

[31]  Konrad Paul Kording,et al.  Estimating the sources of motor errors for adaptation and generalization , 2008, Nature Neuroscience.

[32]  Nicolas Schweighofer,et al.  Minimizing Precision-Weighted Sensory Prediction Errors via Memory Formation and Switching in Motor Adaptation , 2019, The Journal of Neuroscience.

[33]  Matthew A. Wilson,et al.  Hippocampal remapping as hidden state inference , 2019, bioRxiv.

[34]  Charles Kemp,et al.  The discovery of structural form , 2008, Proceedings of the National Academy of Sciences.

[35]  J. Flanagan,et al.  Independence of perceptual and sensorimotor predictions in the size–weight illusion , 2000, Nature Neuroscience.

[36]  J. Krakauer,et al.  Explaining savings for visuomotor adaptation: linear time-invariant state-space models are not sufficient. , 2008, Journal of neurophysiology.

[37]  Reza Shadmehr,et al.  Learning from Sensory and Reward Prediction Errors during Motor Adaptation , 2011, PLoS Comput. Biol..

[38]  J. Randall Flanagan,et al.  Multiple motor memories are learned to control different points on a tool , 2018, Nature Human Behaviour.

[39]  Reza Shadmehr,et al.  Quantifying Generalization from Trial-by-Trial Behavior of Adaptive Systems that Learn with Basis Functions: Theory and Experiments in Human Motor Control , 2003, The Journal of Neuroscience.

[40]  Alan A. Stocker,et al.  A Bayesian Model of Conditioned Perception , 2007, NIPS.

[41]  D M Wolpert,et al.  Multiple paired forward and inverse models for motor control , 1998, Neural Networks.

[42]  Carolyn E. Jones,et al.  Gradual extinction prevents the return of fear: implications for the discovery of state , 2013, Front. Behav. Neurosci..

[43]  R. C. Oldfield The assessment and analysis of handedness: the Edinburgh inventory. , 1971, Neuropsychologia.

[44]  Joshua B. Tenenbaum,et al.  Human-level concept learning through probabilistic program induction , 2015, Science.

[45]  Luigi Acerbi,et al.  Practical Bayesian Optimization for Model Fitting with Bayesian Adaptive Direct Search , 2017, NIPS.

[46]  Michael I. Jordan,et al.  Hierarchical Dirichlet Processes , 2006 .

[47]  Konrad Paul Kording,et al.  The dynamics of memory as a consequence of optimal adaptation to a changing body , 2007, Nature Neuroscience.

[48]  Harold W. Kuhn,et al.  The Hungarian method for the assignment problem , 1955, 50 Years of Integer Programming.

[49]  Paul L Gribble,et al.  Both fast and slow learning processes contribute to savings following sensorimotor adaptation. , 2019, Journal of neurophysiology.

[50]  M. J. Bayarri,et al.  Particle Learning for Sequential Bayesian Computation , 2010 .