An Approximate Inference Approach to Temporal Optimization for Robotics

Algorithms based on iterative local approximations present a practical approach to optimal control in robotic systems. However, they generally require the temporal parameters (for e.g. the movement duration or the time point of reaching an intermediate goal) to be specified a priori. Here, we present a methodology that is capable of jointly optimizing the temporal parameters in addition to the control command profiles. The presented approach is based on a Bayesian formulation of the optimal control problem, which includes the time course of the movement as a random variable. An approximate EM algorithm is derived that efficiently optimizes both the time course of the movement and the control commands offering, for the first time, a practical approach to tackling generic via point problems in a systematic way under the optimal control framework. The proposed approach, which is applicable to plants with non-linear dynamics as well as arbitrary state dependent and quadratic control costs, is evaluated on realistic simulations of a redundant robotic plant and on a simulated KUKA robotic arm.

[1]  Kevin M. Lynch,et al.  Optimal motion planning for a class of hybrid dynamical systems with impacts , 2011, 2011 IEEE International Conference on Robotics and Automation.

[2]  Michael I. Jordan,et al.  Optimal feedback control as a theory of motor coordination , 2002, Nature Neuroscience.

[3]  Robert F. Stengel,et al.  Optimal Control and Estimation , 1994 .

[4]  Sethu Vijayakumar,et al.  Optimal variable stiffness control: formulation and application to explosive movement tasks , 2012, Auton. Robots.

[5]  Emanuel Todorov,et al.  First-exit model predictive control of fast discontinuous dynamics: Application to ball bouncing , 2011, 2011 IEEE International Conference on Robotics and Automation.

[6]  Nicholas Roy,et al.  Hierarchical Motion Planning in Topological Representations , 2012, Robotics: Science and Systems.

[7]  Marc Toussaint,et al.  Probabilistic inference for solving discrete and continuous state Markov Decision Processes , 2006, ICML.

[8]  K. Rawlik.  On probabilistic inference approaches to stochastic optimal control , 2013 .

[9]  E. Todorov,et al.  A generalized iterative LQG method for locally-optimal feedback control of constrained nonlinear stochastic systems , 2005, Proceedings of the 2005, American Control Conference, 2005..

[10]  D. Barber,et al.  Solving deterministic policy ( PO ) MDPs using Expectation-Maximisation and Antifreeze , 2009 .

[11]  Yuval Tassa,et al.  Stochastic Differential Dynamic Programming , 2010, Proceedings of the 2010 American Control Conference.

[12]  N. Roy,et al.  On Stochastic Optimal Control and Reinforcement Learning by Approximate Inference , 2013 .

[13]  Marc Toussaint,et al.  Robot trajectory optimization using approximate inference , 2009, ICML '09.

[14]  Taku Komura,et al.  Topology-based representations for motion planning and generalization in dynamic environments with interactions , 2013, Int. J. Robotics Res..

[15]  Nicholas Roy,et al.  Exploiting Passive Dynamics with Variable Stiffness Actuation in Robot Brachiation , 2013 .

[16]  Shinichiro Nakaoka,et al.  BLUE: A bipedal robot with variable stiffness and damping , 2012, 2012 12th IEEE-RAS International Conference on Humanoid Robots (Humanoids 2012).