A topology-guided path integral approach for stochastic optimal control

This work presents an efficient method to solve a class of continuous-time, continuous-space stochastic optimal control problems of robot motion in a cluttered environment. The method builds upon a path integral representation of the stochastic optimal control problem that allows computation of the optimal solution through sampling and estimation process. As this sampling process often leads to a local minimum especially when the state space is highly non-convex due to the obstacle field, we present an efficient method to alleviate this issue by devising a proposed topological motion planning algorithm. Combined with a receding-horizon scheme in execution of the optimal control solution, the proposed method can generate a dynamically feasible and collision-free trajectory while reducing concern about local optima. Illustrative numerical examples are presented to demonstrate the applicability and validity of the proposed approach.

[1]  Emilio Frazzoli,et al.  Optimal kinodynamic motion planning using incremental sampling-based methods , 2010, 49th IEEE Conference on Decision and Control (CDC).

[2]  H. Kappen,et al.  Path integral control and state-dependent feedback. , 2014, Physical review. E, Statistical, nonlinear, and soft matter physics.

[3]  H. Kappen.  Path integrals and symmetry breaking for optimal control theory , 2005, physics/0505066.

[4]  Vijay Kumar,et al.  Topological constraints in search-based robot path planning , 2012, Auton. Robots.

[5]  Evangelos Theodorou,et al.  Nonlinear Stochastic Control and Information Theoretic Dualities: Connections, Interdependencies and Thermodynamic Interpretations , 2015, Entropy.

[6]  Emilio Frazzoli,et al.  Optimal sampling-based Feedback Motion Trees among obstacles for controllable linear systems with linear constraints , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[7]  Emanuel Todorov,et al.  Efficient computation of optimal actions , 2009, Proceedings of the National Academy of Sciences.

[8]  Emilio Frazzoli,et al.  A martingale approach and time-consistent sampling-based algorithms for risk management in stochastic optimal control , 2013, 53rd IEEE Conference on Decision and Control.

[9]  Jur P. van den Berg,et al.  Kinodynamic RRT*: Asymptotically optimal motion planning for robots with linear dynamics , 2013, 2013 IEEE International Conference on Robotics and Automation.

[10]  Evangelos Theodorou,et al.  Information-theoretic stochastic optimal control via incremental sampling-based algorithms , 2014, 2014 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL).

[11]  C. Gardiner.  Handbook of Stochastic Methods , 1983 .

[12]  Nicholas Roy,et al.  Rapidly-exploring Random Belief Trees for motion planning under uncertainty , 2011, 2011 IEEE International Conference on Robotics and Automation.

[13]  Stefan Schaal,et al.  A Generalized Path Integral Control Approach to Reinforcement Learning , 2010, J. Mach. Learn. Res..

[14]  Vijay Kumar,et al.  Invariants for homology classes with application to optimal search and planning problem in robotics , 2012, Annals of Mathematics and Artificial Intelligence.

[15]  Danica Kragic,et al.  High-dimensional Winding-Augmented Motion Planning with 2D topological task projections and persistent homology , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[16]  Hilbert J. Kappen,et al.  Adaptive Importance Sampling for Control and Inference , 2015, ArXiv.

[17]  Emilio Frazzoli,et al.  Sampling-based algorithms for optimal motion planning , 2011, Int. J. Robotics Res..

[18]  Stefan Schaal,et al.  Reinforcement learning of motor skills in high dimensions: A path integral approach , 2010, 2010 IEEE International Conference on Robotics and Automation.

[19]  Han-Lim Choi,et al.  A successive approximation-based approach for optimal kinodynamic motion planning with nonlinear differential constraints , 2013, 52nd IEEE Conference on Decision and Control.

[20]  Xiongzhi Chen.  Brownian Motion and Stochastic Calculus , 2008 .