Rapid Visuomotor Responses Reflect Value-Based Decisions

Cognitive decision-making is known to be sensitive to the values of potential options, which are the probability and size of rewards associated with different choices. Here, we examine whether rapid motor responses to perturbations of visual feedback about movement, which mediate low-level and involuntary feedback control loops, reflect computations associated with high-level value-based decision-making. In three experiments involving human participants, we varied the value associated with different potential targets for reaching movements by controlling the distributions of rewards across the targets (Experiment 1), the probability with which each target could be specified (Experiment 2), or both (Experiment 3). We found that the size of rapid and involuntary feedback responses to movement perturbations was strongly influenced by the relative value between targets. A statistical model of relative value that includes a term for risk sensitivity provided the best fit to the visuomotor response data, illustrating that feedback control policies are biased to favor more frequent task success at the expense of the overall extrinsic reward accumulated through movement. Importantly however, the regulation of rapid feedback responses was associated with successful pursuit of high-value task outcomes. This implies that when we move, the brain specifies a set of feedback control gains that enable low-level motor areas not only to generate efficient and accurate movement, but also to rapidly and adaptively respond to evolving sensory information in a manner consistent with value-based decision-making. SIGNIFICANCE STATEMENT Current theories of sensorimotor control suggest that, rather than selecting and planning the details of movements in advance, the role of the brain is to set time-varying feedback gains that continuously transform sensory information into motor commands by feedback control. Here, we examine whether the fastest motor responses to perturbations of movement, which mediate low-level and involuntary feedback control loops (i.e., reflexes), reflect computations associated with high-level, value-based decision-making. We find that rapid feedback responses during reaching reflect the relative probabilities and rewards associated with target options. This suggests that low-order components of the sensorimotor control hierarchy, which generate rapid and automatic responses, can continuously evaluate evolving sensory evidence and initiate responses according to the prospect of reward.

[1]  B. Hayden,et al.  A distributed, hierarchical and recurrent framework for reward-based choice , 2017, Nature Reviews Neuroscience.

[2]  Paul Cisek,et al.  Neural Correlates of Biased Competition in Premotor Cortex , 2011, The Journal of Neuroscience.

[3]  D. Wolpert,et al.  The Temporal Evolution of Feedback Gains Rapidly Update to Task Demands , 2013, The Journal of Neuroscience.

[4]  Brian L. Day,et al.  Direct visuomotor mapping for fast visually-evoked arm movements , 2012, Neuropsychologia.

[5]  Daniel M. Wolpert,et al.  A modular planar robotic manipulandum with end-point torque control , 2009, Journal of Neuroscience Methods.

[6]  Aki Vehtari,et al.  Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC , 2015, Statistics and Computing.

[7]  Stephen H Scott,et al.  Influence of the behavioral goal and environmental obstacles on rapid feedback responses. , 2012, Journal of neurophysiology.

[8]  J. Kruschke Doing Bayesian Data Analysis , 2010 .

[9]  K. Nakayama,et al.  Target selection in visual search as revealed by movement trajectories , 2008, Vision Research.

[10]  Daniel A. Braun,et al.  Risk-Sensitive Optimal Feedback Control Accounts for Sensorimotor Behavior under Uncertainty , 2010, PLoS Comput. Biol..

[11]  Alexander C. Schütz,et al.  Dynamic integration of information about salience and value for saccadic eye movements , 2012, Proceedings of the National Academy of Sciences.

[12]  John Salvatier,et al.  Probabilistic programming in Python using PyMC3 , 2016, PeerJ Comput. Sci..

[13]  Daniel A. Braun,et al.  Risk sensitivity in a motor task with speed-accuracy trade-off. , 2011, Journal of neurophysiology.

[14]  Michael N. Shadlen,et al.  Probabilistic reasoning by neurons , 2007, Nature.

[15]  J. Diedrichsen Optimal Task-Dependent Changes of Bimanual Feedback Control and Adaptation , 2007, Current Biology.

[16]  Timothy Edward John Behrens,et al.  Effort-Based Cost–Benefit Valuation and the Human Brain , 2009, The Journal of Neuroscience.

[17]  J. Andrew Pruszynski,et al.  Goal-Dependent Modulation of Fast Feedback Responses in Primary Motor Cortex , 2014, The Journal of Neuroscience.

[18]  Daniel M Wolpert,et al.  Parallel specification of competing sensorimotor control policies for alternative action options , 2016, Nature Neuroscience.

[19]  Sae Franklin,et al.  Temporal Evolution of Spatial Computations for Visuomotor Control , 2016, The Journal of Neuroscience.

[20]  Sae Franklin,et al.  Fractionation of the visuomotor feedback response to directions of movement and perturbation , 2014, Journal of neurophysiology.

[21]  Daniel A. Braun,et al.  Risk-sensitivity and the mean-variance trade-off: decision making in sensorimotor control , 2011, Proceedings of the Royal Society B: Biological Sciences.

[22]  D. Kahneman Maps of Bounded Rationality: Psychology for Behavioral Economics , 2003 .

[23]  E. Todorov Optimality principles in sensorimotor control , 2004, Nature Neuroscience.

[24]  M. Husain,et al.  Reward Pays the Cost of Noise Reduction in Motor and Cognitive Control , 2015, Current Biology.

[25]  Michael S Landy,et al.  Motor control is decision-making , 2012, Current Opinion in Neurobiology.

[26]  Brian C. McCann,et al.  Learning Stochastic Reward Distributions in a Speeded Pointing Task , 2008, The Journal of Neuroscience.

[27]  Matthew J. Crossley,et al.  Credit assignment in movement-dependent reinforcement learning , 2016, Proceedings of the National Academy of Sciences.

[28]  B. Day,et al.  Voluntary modification of automatic arm movements evoked by motion of a visual target , 1999, Experimental Brain Research.

[29]  L. Selen,et al.  Deliberation in the Motor System: Reflex Gains Track Evolving Evidence Leading to a Decision , 2012, The Journal of Neuroscience.

[30]  Daniel A. Braun,et al.  Risk-Sensitivity in Sensorimotor Control , 2011, Front. Hum. Neurosci..

[31]  Stephen H Scott,et al.  Apparent and Actual Trajectory Control Depend on the Behavioral Context in Upper Limb Motor Tasks , 2015, The Journal of Neuroscience.

[32]  Michael I. Jordan,et al.  Optimal feedback control as a theory of motor coordination , 2002, Nature Neuroscience.

[33]  Xiao-Jing Wang Neural dynamics and circuit mechanisms of decision-making , 2012, Current Opinion in Neurobiology.

[34]  L. Maloney,et al.  Economic decision-making compared with an equivalent motor task , 2009, Proceedings of the National Academy of Sciences.

[35]  M. Pessiglione,et al.  Learning To Minimize Efforts versus Maximizing Rewards: Computational Principles and Neural Correlates , 2014, The Journal of Neuroscience.

[36]  Paul R. Schrater,et al.  Dynamic Integration of Value Information into a Common Probability Currency as a Theory for Flexible Decision Making , 2015, PLoS Comput. Biol..

[37]  Stephen H. Scott,et al.  A Functional Taxonomy of Bottom-Up Sensory Feedback Processing for Motor Actions , 2016, Trends in Neurosciences.

[38]  Emanuel Todorov,et al.  Evidence for the Flexible Sensorimotor Strategies Predicted by Optimal Feedback Control , 2007, The Journal of Neuroscience.

[39]  Hammond Ph,et al.  The influence of prior instruction to the subject on an apparently involuntary neuro-muscular response. , 1956 .

[40]  O. Hikosaka,et al.  A neural correlate of response bias in monkey caudate nucleus , 2002, Nature.

[41]  S. Scott Optimal feedback control and the neural basis of volitional motor control , 2004, Nature Reviews Neuroscience.

[42]  Jeffrey Weiler,et al.  Goal-dependent modulation of the long-latency stretch response at the shoulder, elbow, and wrist. , 2015, Journal of neurophysiology.

[43]  David M Milstein,et al.  The Influence of Expected Value on Saccadic Preparation , 2007, The Journal of Neuroscience.

[44]  W. Pieter Medendorp,et al.  State Estimation for Early Feedback Responses in Reaching: Intramodal or Multimodal? , 2017, Front. Integr. Neurosci..

[45]  J. Randall Flanagan,et al.  A Rapid Tactile-Motor Reflex Automatically Guides Reaching toward Handheld Objects , 2016, Current Biology.

[46]  Michael L. Platt,et al.  Neural correlates of decision variables in parietal cortex , 1999, Nature.

[47]  Karl J. Friston,et al.  Neural Signatures of Value Comparison in Human Cingulate Cortex during Decisions Requiring an Effort-Reward Trade-off , 2016, The Journal of Neuroscience.

[48]  D. Wolpert,et al.  Changing your mind: a computational mechanism of vacillation , 2009, Nature.

[49]  P. Hammond,et al.  The influence of prior instruction to the subject on an apparently involuntary neuro-muscular response. , 1956, The Journal of physiology.

[50]  P. Cisek Making decisions through a distributed consensus , 2012, Current Opinion in Neurobiology.

[51]  M. Landy,et al.  Dynamic Estimation of Task-Relevant Variance in Movement under Risk , 2012, The Journal of Neuroscience.

[52]  D. Munoz,et al.  t Immediate Neural Plasticity Shapes Motor Performance , 2000, The Journal of Neuroscience.

[53]  D. Wolpert,et al.  Specificity of Reflex Adaptation for Task-Relevant Variability , 2008, The Journal of Neuroscience.

[54]  A. de Rugy,et al.  Action history influences subsequent movement via two distinct processes , 2017, eLife.

[55]  Timothy E. J. Behrens,et al.  Hierarchical competitions subserving multi-attribute choice , 2014, Nature Neuroscience.

[56]  Adrian R. Willoughby,et al.  Effects of Working Memory Load on Visual Selective Attention: Behavioral and Electrophysiological Evidence , 2011, Front. Hum. Neurosci..

[57]  Helen J. Huang,et al.  A Representation of Effort in Decision-Making and Motor Control , 2016, Current Biology.

[58]  Frédéric Crevecoeur,et al.  Rapid Online Selection between Multiple Motor Plans , 2014, The Journal of Neuroscience.

[59]  M. Landy,et al.  Decision making, movement planning and statistical decision theory , 2008, Trends in Cognitive Sciences.