Reward-Based Improvements in Motor Control Are Driven by Multiple Error-Reducing Mechanisms

Reward has a remarkable ability to invigorate motor behaviour, enabling individuals to select and execute actions with greater precision and speed. However, if reward is to be exploited in applied settings such as rehabilitation, a thorough understanding of its underlying mechanisms is required. Although reward-driven enhancement of movement execution has been proposed to occur through enhanced feedback control, an untested alternative is that it is driven by increased arm stiffness, an energy-consuming process that increases limb stability. First, we demonstrate that during reaching reward improves selection and execution performance concomitantly without interference. Computational analysis revealed that reward led to both an increase in feedback correction during movement and a reduction in motor noise near the target. We provide novel evidence that this noise reduction is driven by a reward-dependent increase in arm stiffness. Therefore, reward drives multiple error-reduction mechanisms which enable individuals to invigorate motor performance without compromising accuracy.

[1]  R. Hübner,et al.  Monetary reward increases attentional effort in the flanker task , 2010, Psychonomic bulletin & review.

[2]  E. Bézard,et al.  Shaping of Motor Responses by Incentive Values through the Basal Ganglia , 2007, The Journal of Neuroscience.

[3]  J. Blouin,et al.  Increased human stretch reflex dynamic sensitivity with height‐induced postural threat , 2018, The Journal of physiology.

[4]  Matthew I. Becker,et al.  Cerebellar Control of Reach Kinematics for Endpoint Precision , 2019, Neuron.

[5]  Daniel M. Wolpert,et al.  Rapid Visuomotor Responses Reflect Value-Based Decisions , 2019, The Journal of Neuroscience.

[6]  Karolina M. Lempert,et al.  Modulation of Saccade Vigor during Value-Based Decision Making , 2015, The Journal of Neuroscience.

[7]  F. Horak,et al.  Postural feedback responses scale with biomechanical constraints in human standing , 2004, Experimental Brain Research.

[8]  Paul L Gribble,et al.  Role of cocontraction in arm movement accuracy. , 2003, Journal of neurophysiology.

[9]  L. Christensen,et al.  University of Birmingham Disruption of state estimation in the human lateral cerebellum , 2007 .

[10]  J. Andrew Pruszynski,et al.  Primary motor cortex underlies multi-joint integration for fast feedback control , 2011, Nature.

[11]  T. Flash,et al.  The coordination of arm movements: an experimentally confirmed mathematical model , 1985, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[12]  Jeffrey Weiler,et al.  Spinal stretch reflexes support efficient hand control , 2019, Nature Neuroscience.

[13]  P. Fitts The information capacity of the human motor system in controlling the amplitude of movement. 1954. , 1992, Journal of experimental psychology. General.

[14]  Masaya Hirashima,et al.  Asymmetric Transfer of Visuomotor Learning between Discrete and Rhythmic Movements , 2010, The Journal of Neuroscience.

[15]  Masaya Hirashima,et al.  Intermittent Visual Feedback Can Boost Motor Learning of Rhythmic Movements: Evidence for Error Feedback Beyond Cycles , 2012, The Journal of Neuroscience.

[16]  D A Robinson,et al.  The use of control systems analysis in the neurophysiology of eye movements. , 1981, Annual review of neuroscience.

[17]  Ioannis Rigas,et al.  Movement vigor as a traitlike attribute of individuality. , 2018, Journal of neurophysiology.

[18]  E. Bizzi,et al.  Neural, mechanical, and geometric factors subserving arm posture in humans , 1985, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[19]  F Crevecoeur,et al.  Feedback responses rapidly scale with the urgency to correct for external perturbations. , 2013, Journal of neurophysiology.

[20]  Emanuel Todorov,et al.  Stochastic Optimal Control and Estimation Methods Adapted to the Noise Characteristics of the Sensorimotor System , 2005, Neural Computation.

[21]  E. Todorov Optimality principles in sensorimotor control , 2004, Nature Neuroscience.

[22]  J. R.,et al.  Quantitative analysis , 1892, Nature.

[23]  J. Krakauer,et al.  A computational neuroanatomy for motor control , 2008, Experimental Brain Research.

[24]  Alain F. Zuur,et al.  A protocol for data exploration to avoid common statistical problems , 2010 .

[25]  M. Husain,et al.  Reward Pays the Cost of Noise Reduction in Motor and Cognitive Control , 2015, Current Biology.

[26]  N. Vøllestad,et al.  Motor Unit Recruitment , 1987 .

[27]  Kelvin E. Jones,et al.  The scaling of motor noise with muscle strength and motor unit number in humans , 2004, Experimental Brain Research.

[28]  Reza Shadmehr,et al.  Computational nature of human adaptive control during learning of reaching movements in force fields , 1999, Biological Cybernetics.

[29]  Wilsaan M. Joiner,et al.  Adaptive Control of Saccades via Internal Feedback , 2008, The Journal of Neuroscience.

[30]  Ethan R. Buch,et al.  Noninvasive cortical stimulation enhances motor skill acquisition over multiple days through an effect on consolidation , 2009, Proceedings of the National Academy of Sciences.

[31]  Shinichi Nakagawa,et al.  Nested by design: model fitting and interpretation in a mixed model era , 2013 .

[32]  Rieko Osu,et al.  Endpoint Stiffness of the Arm Is Directionally Tuned to Instability in the Environment , 2007, The Journal of Neuroscience.

[33]  R. J. van Beers,et al.  The role of execution noise in movement variability. , 2004, Journal of neurophysiology.

[34]  S. Lisberger,et al.  A tale of two species: Neural integration in zebrafish and monkeys , 2015, Neuroscience.

[35]  L. Selen,et al.  Impedance Control Reduces Instability That Arises from Motor Noise , 2009, The Journal of Neuroscience.

[36]  M. Floeter Structure and function of muscle fibers and motor units , 2010 .

[37]  R. Shadmehr,et al.  The intrinsic value of visual information affects saccade velocities , 2009, Experimental Brain Research.

[38]  P. Fitts The information capacity of the human motor system in controlling the amplitude of movement. , 1954, Journal of experimental psychology.

[39]  Richard Inger,et al.  A brief introduction to mixed effects modelling and multi-model inference in ecology , 2018, PeerJ.

[40]  Y. Kojima,et al.  Selective reward affects the rate of saccade adaptation , 2017, Neuroscience.

[41]  E Burdet,et al.  A method for measuring endpoint stiffness during multi-joint arm movements. , 2000, Journal of biomechanics.

[42]  Thomas E. Nichols,et al.  Nonparametric permutation tests for functional neuroimaging: A primer with examples , 2002, Human brain mapping.

[43]  Tao Liu,et al.  Differential Effect of Reward and Punishment on Procedural Learning , 2009, The Journal of Neuroscience.

[44]  Opher Donchin,et al.  Forward Models and State Estimation in Compensatory Eye Movements , 2009, Front. Cell. Neurosci..

[45]  Yanlong Song,et al.  Probability differently modulating the effects of reward and punishment on visuomotor adaptation , 2017, Experimental Brain Research.

[46]  Eizo Miyashita,et al.  Cocontraction of Pairs of Muscles around Joints May Improve an Accuracy of a Reaching Movement: a Numerical Simulation Study , 2011 .

[47]  F Buchthal,et al.  Motor unit of mammalian muscle. , 1980, Physiological reviews.

[48]  Jörn Diedrichsen,et al.  Learning feedback and feedforward control in a mirror-reversed visual environment. , 2015, Journal of neurophysiology.

[49]  K. Deisseroth,et al.  Orderly recruitment of motor units under optical control in vivo , 2010, Nature Medicine.

[50]  D. Robinson The mechanics of human saccadic eye movement , 1964, The Journal of physiology.

[51]  T. Mima,et al.  Momentary reward induce changes in excitability of primary motor cortex , 2011, Clinical Neurophysiology.

[52]  Adam N. Sanborn,et al.  Model evaluation using grouped or individual data , 2008, Psychonomic bulletin & review.

[53]  Jeremy C. Rietschel,et al.  Increased reward in ankle robotics training enhances motor control and cortical efficiency in stroke. , 2014, Journal of rehabilitation research and development.

[54]  O. Hikosaka,et al.  Modulation of saccadic eye movements by predicted reward outcome , 2001, Experimental Brain Research.

[55]  J. Rothwell,et al.  The dissociable effects of punishment and reward on motor learning , 2015, Nature Neuroscience.

[56]  Krishna V Shenoy,et al.  Simultaneous motor preparation and execution in a last-moment reach correction task , 2019, Nature Communications.

[57]  J. Diedrichsen,et al.  Motor skill learning between selection and execution , 2015, Trends in Cognitive Sciences.

[58]  M. Kawato,et al.  Adaptation to Stable and Unstable Dynamics Achieved By Combined Impedance Control and Inverse Dynamics Model , 2003 .

[59]  John W Krakauer,et al.  The neural correlates of learned motor acuity. , 2014, Journal of neurophysiology.

[60]  Bastien Berret,et al.  Vigour of self-paced reaching movement: cost of time and individual traits , 2018, Scientific Reports.

[61]  Alaa A. Ahmed,et al.  Vigor of reaching movements: reward discounts the cost of effort. , 2018, Journal of neurophysiology.

[62]  J. Rothwell,et al.  Reward and punishment enhance motor adaptation in stroke , 2017, Journal of Neurology, Neurosurgery, and Psychiatry.

[63]  Carsten Bundt,et al.  Reward anticipation modulates primary motor cortex excitability during task preparation , 2016, NeuroImage.

[64]  S. Gielen,et al.  A quantitative analysis of generation of saccadic eye movements by burst neurons. , 1981, Journal of neurophysiology.

[65]  Ashesh K Dhawale,et al.  The Role of Variability in Motor Learning. , 2017, Annual review of neuroscience.

[66]  S. Skalicky The Extraocular Muscles , 2016 .

[67]  Darius E. Parvin,et al.  Mirror Reversal and Visual Rotation Are Learned and Consolidated via Separate Mechanisms: Recalibrating or Learning De Novo? , 2014, The Journal of Neuroscience.

[68]  J. Galea,et al.  The effects of reward and punishment on motor skill learning , 2018, Current Opinion in Behavioral Sciences.

[69]  J. D. Porter,et al.  Extraocular muscles: basic and clinical aspects of structure and function. , 1995, Survey of ophthalmology.

[70]  P. Celnik,et al.  Motor Cortex Excitability Reflects the Subjective Value of Reward and Mediates Its Effects on Incentive-Motivated Performance , 2018, The Journal of Neuroscience.

[71]  J. Krakauer,et al.  Motor skill depends on knowledge of facts , 2013, Front. Hum. Neurosci..

[72]  R. Oostenveld,et al.  Nonparametric statistical testing of EEG- and MEG-data , 2007, Journal of Neuroscience Methods.

[73]  Philippe Lefèvre,et al.  Overlap of movement planning and movement execution reduces reaction time by up to 100ms , 2016, bioRxiv.

[74]  Yuki Ueyama,et al.  Signal-Dependent Noise Induces Muscle Co-Contraction to Achieve Required Movement Accuracy: A Simulation Study with an Optimal Control , 2013 .

[75]  Eric J Perreault,et al.  Voluntary control of static endpoint stiffness during force regulation tasks. , 2002, Journal of neurophysiology.

[76]  Félix-Antoine Savoie,et al.  Added value of money on motor performance feedback: Increased left central beta-band power for rewards and fronto-central theta-band power for punishments , 2018, NeuroImage.

[77]  Aaron Christ,et al.  Mixed Effects Models and Extensions in Ecology with R , 2009 .

[78]  M. Husain,et al.  Motivation dynamically increases noise resistance by internal feedback during movement , 2019, Neuropsychologia.

[79]  Eizo Miyashita,et al.  Optimal Feedback Control for Predicting Dynamic Stiffness During Arm Movement , 2014, IEEE Transactions on Industrial Electronics.

[80]  Dario Farina,et al.  Motor unit recruitment strategies and muscle properties determine the influence of synaptic noise on force steadiness. , 2012, Journal of neurophysiology.