Temporal Evolution of Spatial Computations for Visuomotor Control

Goal-directed reaching movements are guided by visual feedback from both target and hand. The classical view is that the brain extracts information about target and hand positions from a visual scene, calculates a difference vector between them, and uses this estimate to control the movement. Here we show that during fast feedback control, this computation is not immediate, but evolves dynamically over time. Immediately after a change in the visual scene, the motor system generates independent responses to the errors in hand and target location. Only about 200 ms later, the changes in target and hand positions are combined appropriately in the response, slowly converging to the true difference vector. Therefore, our results provide evidence for the temporal evolution of spatial computations in the human visuomotor system, in which the accurate difference vector computation is first estimated by a fast approximation. SIGNIFICANCE STATEMENT The dominant view regarding the neural control of reaching is that the visuomotor system controls movement based on the difference vector—the difference between the positions of the hand and target. We directly test this theory by measuring the responses to visual perturbations over a large range of possible variations in both target and hand displacements. By modeling the nonlinearity of the feedback response, we were able to reveal the temporal evolution of the underlying computations. The visuomotor system first uses an approximation to the difference vector computation, simply combining the nonlinear responses to cursor and target displacements, only arriving at the correct difference vector calculation 200 ms later.

[1]  R. J. van Beers,et al.  Integration of proprioceptive and visual position-information: An experimentally supported model. , 1999, Journal of neurophysiology.

[2]  B. Day,et al.  Voluntary modification of automatic arm movements evoked by motion of a visual target , 1999, Experimental Brain Research.

[3]  Daniel M. Wolpert,et al.  A modular planar robotic manipulandum with end-point torque control , 2009, Journal of Neuroscience Methods.

[4]  Smith-Kettlewell,et al.  BIOLOGICAL IMAGE MOTION PROCESSING : A REVIEW , 2012 .

[5]  R. Zemel,et al.  Inference and computation with population codes. , 2003, Annual review of neuroscience.

[6]  Christopher A. Buneo,et al.  Time-invariant reference frames for parietal reach activity , 2008, Experimental Brain Research.

[7]  S. Scott.  Optimal feedback control and the neural basis of volitional motor control , 2004, Nature Reviews Neuroscience.

[8]  S. Grossberg,et al.  Cortical networks for control of voluntary arm movements under variable force conditions. , 1998, Cerebral cortex.

[9]  Joseph G. Makin,et al.  Learning Multisensory Integration and Coordinate Transformation via Density Estimation , 2013, PLoS Comput. Biol..

[10]  Philip N. Sabes,et al.  Multisensory Integration during Motor Planning , 2003, The Journal of Neuroscience.

[11]  J. Diedrichsen,et al.  A Dedicated Binding Mechanism for the Visual Control of Movement , 2014, Current Biology.

[12]  D. Fitzpatrick,et al.  Spatial coding of position and orientation in primary visual cortex , 2002, Nature Neuroscience.

[13]  J. Saunders,et al.  Humans use continuous visual feedback from the hand to control fast reaching movements , 2003, Experimental Brain Research.

[14]  A. Grinvald,et al.  Functional Organization for Direction of Motion and Its Relationship to Orientation Maps in Cat Area 18 , 1996, The Journal of Neuroscience.

[15]  J. Kalaska,et al.  Parietal area 5 neuronal activity encodes movement kinematics, not movement dynamics , 2004, Experimental Brain Research.

[16]  R. Andersen,et al.  The posterior parietal cortex: Sensorimotor interface for the planning and online control of visually guided movements , 2006, Neuropsychologia.

[17]  J. T. Massey,et al.  Mental rotation of the neuronal population vector. , 1989, Science.

[18]  A. Georgopoulos,et al.  Parietal representation of hand velocity in a copy task. , 2005, Journal of neurophysiology.

[19]  Sae Franklin,et al.  Fractionation of the visuomotor feedback response to directions of movement and perturbation , 2014, Journal of neurophysiology.

[20]  S. Scott,et al.  Cortical control of reaching movements , 1997, Current Opinion in Neurobiology.

[21]  D. Wolpert,et al.  Changing your mind: a computational mechanism of vacillation , 2009, Nature.

[22]  I. Toni,et al.  Integration of target and effector information in the human brain during reach planning. , 2007, Journal of neurophysiology.

[23]  J F Kalaska,et al.  Integration of predictive feedforward and sensory feedback signals for online control of visually guided movement. , 2009, Journal of neurophysiology.

[24]  Richard A. Andersen,et al.  Coding of the Reach Vector in Parietal Area 5d , 2012, Neuron.

[25]  D. Wolpert,et al.  Specificity of Reflex Adaptation for Task-Relevant Variability , 2008, The Journal of Neuroscience.

[26]  P. Brown,et al.  Evidence for subcortical involvement in the visual control of human reaching. , 2001, Brain : a journal of neurology.

[27]  J. Vercher,et al.  Target and hand position information in the online control of goal-directed arm movements , 2003, Experimental Brain Research.

[28]  David W. Franklin,et al.  Computational Mechanisms of Sensorimotor Control , 2011, Neuron.

[29]  N. Swindale,et al.  How many maps are there in visual cortex? , 2000, Cerebral cortex.

[30]  D. Wolpert,et al.  The Temporal Evolution of Feedback Gains Rapidly Update to Task Demands , 2013, The Journal of Neuroscience.

[31]  A. P. Georgopoulos,et al.  Movement parameters and neural activity in motor cortex and area 5. , 1994, Cerebral cortex.

[32]  T. Sejnowski,et al.  Spatial Transformations in the Parietal Cortex Using Basis Functions , 1997, Journal of Cognitive Neuroscience.

[33]  David C Knill,et al.  Visual Feedback Control of Hand Movements , 2004, The Journal of Neuroscience.

[34]  Paul B. Johnson,et al.  Cortical networks for visual reaching: physiological and anatomical organization of frontal and parietal lobe arm regions. , 1996, Cerebral cortex.

[35]  John P Donoghue,et al.  Cue to action processing in motor cortex populations. , 2014, Journal of neurophysiology.

[36]  R. C. Oldfield.  The assessment and analysis of handedness: the Edinburgh inventory. , 1971, Neuropsychologia.

[37]  A. Pouget,et al.  Reading population codes: a neural implementation of ideal observers , 1999, Nature Neuroscience.

[38]  David W Franklin,et al.  Impedance control and internal model use during the initial stage of adaptation to novel dynamics in humans , 2005, The Journal of physiology.

[39]  A. P. Georgopoulos,et al.  Primate motor cortex and free arm movements to visual targets in three- dimensional space. II. Coding of the direction of movement by a neuronal population , 1988, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[40]  Richard A Andersen,et al.  Integration of target and hand position signals in the posterior parietal cortex: effects of workspace and hand vision. , 2012, Journal of neurophysiology.

[41]  R. Andersen,et al.  Dorsal Premotor Neurons Encode the Relative Position of the Hand, Eye, and Goal during Reach Planning , 2006, Neuron.

[42]  J. T. Massey,et al.  Spatial trajectories and reaction times of aimed movements: effects of practice, uncertainty, and change in target location. , 1981, Journal of neurophysiology.

[43]  E. Brenner,et al.  Fast corrections of movements with a computer mouse. , 2003, Spatial vision.

[44]  Sae Franklin,et al.  Visuomotor feedback gains upregulate during the learning of novel dynamics , 2012, Journal of neurophysiology.

[45]  A P Georgopoulos,et al.  On the relations between the direction of two-dimensional arm movements and cell discharge in primate motor cortex , 1982, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[46]  David W. Franklin,et al.  Feedback Modulation: A Window into Cortical Function , 2011, Current Biology.

[47]  Heinrich H Bülthoff,et al.  Seeing the hand while reaching speeds up on‐line responses to a sudden change in target position , 2009, The Journal of physiology.

[48]  Adam Santoro,et al.  Patterns across multiple memories are identified over time , 2014, Nature Neuroscience.

[49]  Jörn Diedrichsen,et al.  Mechanisms of responsibility assignment during redundant reaching movements. , 2013, Journal of neurophysiology.

[50]  R A Scheidt,et al.  Persistence of motor adaptation during constrained, multi-joint, arm movements. , 2000, Journal of neurophysiology.

[51]  J. A. Pruszynski,et al.  Long-Latency Reflexes of the Human Arm Reflect an Internal Model of Limb Dynamics , 2008, Current Biology.

[52]  J. Andrew Pruszynski,et al.  Primary motor cortex underlies multi-joint integration for fast feedback control , 2011, Nature.

[53]  A. P. Georgopoulos,et al.  Neuronal population coding of movement direction. , 1986, Science.

[54]  E. Todorov.  Direct cortical control of muscle activation in voluntary arm movements: a model , 2000, Nature Neuroscience.

[55]  S. Scott,et al.  Random change in cortical load representation suggests distinct control of posture and movement , 2005, Nature Neuroscience.

[56]  F. Bremmer,et al.  Spatial invariance of visual receptive fields in parietal cortex neurons , 1997, Nature.

[57]  Scott T. Grafton,et al.  Forward modeling allows feedback control for fast reaching movements , 2000, Trends in Cognitive Sciences.

[58]  A P Batista,et al.  Reach plans in eye-centered coordinates. , 1999, Science.

[59]  D. Burr,et al.  Motion psychophysics: 1985–2010 , 2011, Vision Research.

[60]  S. Grossberg,et al.  A Cortico-Spinal Model of Reaching and Proprioception under Multiple Task Constraints , 1998, Journal of Cognitive Neuroscience.

[61]  Christopher A. Buneo,et al.  Direct visuomotor transformations for reaching , 2002, Nature.

[62]  G. Rizzolatti,et al.  The organization of the cortical motor system: new concepts. , 1998, Electroencephalography and clinical neurophysiology.