An Efficient Path Integral Approach for Stochastic Optimal Control with a Topology-Embedded Sampling-Based Planner

This work presents an efficient method to solve a class of continuous-time, continuous-space stochastic optimal control problems in the context of motion planning in a cluttered environment. The method builds upon a path integral representation of the stochastic optimal control problem that allows for computation of the optimal solution through some type of sampling process. As this sampling process often leads to a local minimum especially when the state space is highly nonconvex due to the obstacle field, we present an efficient method to alleviate this issue of local optima by devising a topologyembedded sampling-based planning algorithm. Combined with a receding-horizon scheme in execution of the optimal control solution, the proposed method can generate a globally optimal, dynamically feasible and collision-free trajectory. An illustrative numerical example is presented to demonstrate the applicability and validity of the proposed approach.

[1]  Jur P. van den Berg,et al.  Kinodynamic RRT*: Asymptotically optimal motion planning for robots with linear dynamics , 2013, 2013 IEEE International Conference on Robotics and Automation.

[2]  N. Roy,et al.  The Belief Roadmap: Efficient Planning in Belief Space by Factoring the Covariance , 2009, Int. J. Robotics Res..

[3]  H. Kappen,et al.  Path integral control and state-dependent feedback. , 2014, Physical review. E, Statistical, nonlinear, and soft matter physics.

[4]  Stefan Schaal,et al.  A Generalized Path Integral Control Approach to Reinforcement Learning , 2010, J. Mach. Learn. Res..

[5]  Emanuel Todorov,et al.  Efficient computation of optimal actions , 2009, Proceedings of the National Academy of Sciences.

[6]  Emilio Frazzoli,et al.  A martingale approach and time-consistent sampling-based algorithms for risk management in stochastic optimal control , 2013, 53rd IEEE Conference on Decision and Control.

[7]  Evangelos Theodorou,et al.  Nonlinear Stochastic Control and Information Theoretic Dualities: Connections, Interdependencies and Thermodynamic Interpretations , 2015, Entropy.

[8]  Evangelos Theodorou,et al.  Information-theoretic stochastic optimal control via incremental sampling-based algorithms , 2014, 2014 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL).

[9]  H. Kappen.  Path integrals and symmetry breaking for optimal control theory , 2005, physics/0505066.

[10]  Vijay Kumar,et al.  Topological constraints in search-based robot path planning , 2012, Auton. Robots.

[11]  Hilbert J. Kappen,et al.  Adaptive Importance Sampling for Control and Inference , 2015, ArXiv.

[12]  Emilio Frazzoli,et al.  Sampling-based algorithms for optimal motion planning , 2011, Int. J. Robotics Res..

[13]  Han-Lim Choi,et al.  A successive approximation-based approach for optimal kinodynamic motion planning with nonlinear differential constraints , 2013, 52nd IEEE Conference on Decision and Control.

[14]  Xiongzhi Chen.  Brownian Motion and Stochastic Calculus , 2008 .

[15]  Emilio Frazzoli,et al.  Optimal sampling-based Feedback Motion Trees among obstacles for controllable linear systems with linear constraints , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[16]  Nicholas Roy,et al.  Rapidly-exploring Random Belief Trees for motion planning under uncertainty , 2011, 2011 IEEE International Conference on Robotics and Automation.

[17]  Stefan Schaal,et al.  Reinforcement learning of motor skills in high dimensions: A path integral approach , 2010, 2010 IEEE International Conference on Robotics and Automation.

[18]  Vijay Kumar,et al.  Invariants for homology classes with application to optimal search and planning problem in robotics , 2012, Annals of Mathematics and Artificial Intelligence.

[19]  C. Gardiner.  Handbook of Stochastic Methods , 1983 .

[20]  Emilio Frazzoli,et al.  Optimal kinodynamic motion planning using incremental sampling-based methods , 2010, 49th IEEE Conference on Decision and Control (CDC).