Risk-Aware Control

Human movement differs from robot control because of its flexibility in unknown environments, robustness to perturbation, and tolerance of unknown parameters and unpredictable variability. We propose a new theory, risk-aware control, in which movement is governed by estimates of risk based on uncertainty about the current state and knowledge of the cost of errors. We demonstrate the existence of a feedback control law that implements risk-aware control and show that this control law can be directly implemented by populations of spiking neurons. Simulated examples of risk-aware control for time-varying cost functions as well as learning of unknown dynamics in a stochastic risky environment are provided.

[1]  R Shadmehr,et al.  Electromyographic Correlates of Learning an Internal Model of Reaching Movements , 1999, The Journal of Neuroscience.

[2]  Rieko Osu,et al.  Short- and long-term changes in joint co-contraction associated with motor learning as revealed from surface EMG. , 2002, Journal of neurophysiology.

[3]  G. Chow.  Analysis and control of dynamic economic systems , 1975 .

[4]  Karl J. Friston,et al.  Reflections on agranular architecture: predictive coding in the motor cortex , 2013, Trends in Neurosciences.

[5]  J. Massion.  Movement, posture and equilibrium: Interaction and coordination , 1992, Progress in Neurobiology.

[6]  D. Wolpert,et al.  Controlling the statistics of action: obstacle avoidance. , 2002, Journal of neurophysiology.

[7]  Terence D. Sanger,et al.  Neuro-mechanical control using differential stochastic operators , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[8]  Wei Ji Ma,et al.  Bayesian inference with probabilistic population codes , 2006, Nature Neuroscience.

[9]  R. Hartley.  Stochastic Modelling and Control , 1985 .

[10]  T. Vincent,et al.  Nonlinear and Optimal Control Systems , 1997 .

[11]  E. Bizzi,et al.  Posture control and trajectory formation during arm movement , 1984, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[12]  Michael I. Jordan,et al.  Optimal feedback control as a theory of motor coordination , 2002, Nature Neuroscience.

[13]  Simon F. Giszter,et al.  Spinal primitives and intra-spinal micro-stimulation (ISMS) based prostheses: a neurobiological perspective on the “known unknowns” in ISMS and future prospects , 2015, Front. Neurosci..

[14]  Terence D. Sanger,et al.  Distributed Control of Uncertain Systems Using Superpositions of Linear Operators , 2011, Neural Computation.

[15]  H. Nagai.  Bellman equations of risk sensitive control , 1995, Proceedings of 1995 34th IEEE Conference on Decision and Control.

[16]  J. Mink.  THE BASAL GANGLIA: FOCUSED SELECTION AND INHIBITION OF COMPETING MOTOR PROGRAMS , 1996, Progress in Neurobiology.

[17]  P. Whittle.  Risk-Sensitive Optimal Control , 1990 .

[18]  R. Bellman.  Dynamic programming. , 1957, Science.

[19]  Terence D Sanger.  Controlling Variability , 2010, Journal of motor behavior.

[20]  Terence D. Sanger,et al.  Probability Density Methods for Smooth Function Approximation and Learning in Populations of Tuned Spiking Neurons , 1998, Neural Computation.

[21]  D. Wolpert,et al.  Computations underlying sensorimotor learning , 2016, Current Opinion in Neurobiology.

[22]  Bernard Widrow,et al.  Adaptive switching circuits , 1988 .

[23]  Karl J. Friston,et al.  Free Energy, Value, and Attractors , 2011, Comput. Math. Methods Medicine.

[24]  S. Bouisset,et al.  Posture, dynamic stability, and voluntary movement , 2008, Neurophysiologie Clinique/Clinical Neurophysiology.

[25]  Karl J. Friston.  What Is Optimal about Motor Control? , 2011, Neuron.