From task parameters to motor synergies: A hierarchical framework for approximately optimal control of redundant manipulators

We present a hierarchical framework for approximately-optimal control of redundant manipulators. The plant is augmented with a low-level feedback controller, designed to yield input-output behavior that captures the task-relevant aspects of plant dynamics but has reduced dimensionality. This makes it possible to reformulate the optimal control problem in terms of the augmented dynamics, and optimize a high-level feedback controller without running into the curse of dimensionality. The resulting control hierarchy compares favorably to existing methods in robotics. Furthermore we demonstrate a number of similarities to (non-hierarchical) optimal feedback control. Besides its engineering applications, the new framework addresses a key unresolved problem in the neural control of movement. It has long been hypothesized that coordination involves selective control of task parameters via muscle synergies, but the link between these parameters and the synergies capable of controlling them has remained elusive. Our framework provides this missing link.

[1]  W. Smith.  The Integrative Action of the Nervous System , 1907, Nature.

[2]  G. Sutton,et al.  The variation of hand tremor with force in healthy subjects , 1967, The Journal of physiology.

[3]  N. A. Bernshteĭn.  The co-ordination and regulation of movements , 1967 .

[4]  J. Meditch,et al.  Applied optimal control , 1972, IEEE Transactions on Automatic Control.

[5]  David Q. Mayne,et al.  Differential dynamic programming , 1972, The Mathematical Gazette.

[6]  H. Zelaznik,et al.  Motor-output variability: a theory for the accuracy of rapid motor acts. , 1979, Psychological review.

[7]  Geoffrey E. Hinton,et al.  Parallel computations for controlling an arm. , 1984, Journal of motor behavior.

[8]  A. Pellionisz,et al.  Coordination: a vector-matrix description of transformations of overcomplete CNS coordinates and a tensorial solution using the Moore-Penrose generalized inverse. , 1984, Journal of theoretical biology.

[9]  Rodney A. Brooks,et al.  A Robust Layered Control Syste For A Mobile Robot , 2022 .

[10]  John Baillieul,et al.  Avoiding obstacles and resolving kinematic redundancy , 1986, Proceedings. 1986 IEEE International Conference on Robotics and Automation.

[11]  Oussama Khatib,et al.  A unified approach for motion and force control of robot manipulators: The operational space formulation , 1987, IEEE J. Robotics Autom..

[12]  R A Abrams,et al.  Optimality in human motor performance: ideal control of rapid aimed movements. , 1988, Psychological review.

[13]  S. Grossberg,et al.  Neural dynamics of planned arm movements: emergent invariants and speed-accuracy properties during trajectory formation. , 1988, Psychological review.

[14]  G E Loeb,et al.  Understanding sensorimotor feedback through optimal control. , 1990, Cold Spring Harbor symposia on quantitative biology.

[15]  Neville Hogan,et al.  Integrable Solutions of Kinematic Redundancy via Impedance Control , 1991, Int. J. Robotics Res..

[16]  Michael A. Arbib,et al.  A computational description of the organization of human reaching and prehension , 1992 .

[17]  M. A. Arbib,et al.  Models of Trajectory Formation and Temporal Interaction of Reach and Grasp. , 1993, Journal of motor behavior.

[18]  A.D. Kuo,et al.  An optimal control model for analyzing human postural balance , 1995, IEEE Transactions on Biomedical Engineering.

[19]  George J. Pappas,et al.  Hybrid control in air traffic management systems , 1995, Proceedings of 1995 34th IEEE Conference on Decision and Control.

[20]  N. A. Bernstein.  Dexterity and Its Development , 1996 .

[21]  Peter I. Corke,et al.  A robotics toolbox for MATLAB , 1996, IEEE Robotics Autom. Mag..

[22]  Doina Precup,et al.  Multi-time Models for Temporally Abstract Planning , 1997, NIPS.

[23]  Michael I. Jordan,et al.  The Role of Inertial Sensitivity in Motor Planning , 1998, The Journal of Neuroscience.

[24]  J F Kalaska,et al.  Cortical control of whole-arm motor tasks. , 1998, Novartis Foundation symposium.

[25]  Jerry E. Pratt,et al.  Intuitive control of a planar bipedal walking robot , 1998, Proceedings. 1998 IEEE International Conference on Robotics and Automation (Cat. No.98CH36146).

[26]  V. Borkar,et al.  A unified framework for hybrid control: model and optimal control theory , 1998, IEEE Trans. Autom. Control..

[27]  A. Rantzer,et al.  Optimal control of hybrid systems , 1999, Proceedings of the 38th IEEE Conference on Decision and Control (Cat. No.99CH36304).

[28]  Gregor Schöner,et al.  The uncontrolled manifold concept: identifying control variables for a functional task , 1999, Experimental Brain Research.

[29]  G. E. Loeb,et al.  A hierarchical foundation for models of sensorimotor control , 1999, Experimental Brain Research.

[30]  Thomas G. Dietterich.  Hierarchical Reinforcement Learning with the MAXQ Value Function Decomposition , 1999, J. Artif. Intell. Res..

[31]  Anthony A. Maciejewski,et al.  On the implementation of velocity control for kinematically redundant manipulators , 2000, IEEE Trans. Syst. Man Cybern. Part A.

[32]  Oussama Khatib,et al.  Gauss' principle and the dynamics of redundant and constrained manipulators , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[33]  Chee-Meng Chew,et al.  Virtual Model Control: An Intuitive Approach for Bipedal Locomotion , 2001, Int. J. Robotics Res..

[34]  M. Pandy,et al.  Dynamic optimization of human walking. , 2001, Journal of biomechanical engineering.

[35]  Michael I. Jordan,et al.  Optimal feedback control as a theory of motor coordination , 2002, Nature Neuroscience.

[36]  Michael I. Jordan,et al.  A Minimal Intervention Principle for Coordinated Movement , 2002, NIPS.

[37]  David Zipser,et al.  Reaching to grasp with a multi-jointed arm. I. Computational model. , 2002, Journal of neurophysiology.

[38]  Emanuel Todorov,et al.  Cosine Tuning Minimizes Motor Errors , 2002, Neural Computation.

[39]  Jiping He,et al.  A novel model of motor learning capable of developing an optimal movement control law online from scratch , 2004, Biological Cybernetics.

[40]  Emanuel Todorov,et al.  Iterative Linear Quadratic Regulator Design for Nonlinear Biological Movement Systems , 2004, ICINCO.

[41]  E. Todorov.  Optimality principles in sensorimotor control , 2004, Nature Neuroscience.

[42]  E. J. Cheng,et al.  Measured and modeled properties of mammalian skeletal muscle. II. The effectsof stimulus frequency on force-length and force-velocity relationships , 1999, Journal of Muscle Research & Cell Motility.

[43]  Weiwei Li,et al.  Hierarchical optimal control of redundant biomechanical systems , 2004, The 26th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[44]  E. Todorov,et al.  A generalized iterative LQG method for locally-optimal feedback control of constrained nonlinear stochastic systems , 2005, Proceedings of the 2005, American Control Conference, 2005..

[45]  Russ Tedrake,et al.  Efficient Bipedal Robots Based on Passive-Dynamic Walkers , 2005, Science.