Improving Kinodynamic Planners for Vehicular Navigation with Learned Goal-Reaching Controllers

This paper aims to improve the path quality and computational efficiency of sampling-based kinodynamic planners for vehicular navigation. It proposes a learning framework for identifying promising controls during the expansion process of sampling-based planners. Given a dynamics model, a reinforcement learning process is trained offline to return a low-cost control that reaches a local goal state (i.e., a waypoint) in the absence of obstacles. By focusing on the system’s dynamics and not knowing the environment, this process is data-efficient and takes place once for a robotic system. In this way, it can be reused in different environments. The planner generates online local goal states for the learned controller in an informed manner to bias towards the goal and consecutively in an exploratory, random manner. For the informed expansion, local goal states are generated either via (a) medial axis information in environments with obstacles, or (b) wavefront information for setups with traversability costs. The learning process and the resulting planning framework are evaluated for a first and second-order differential drive system, as well as a physically simulated Segway robot. The results show that the proposed integration of learning and planning can produce higher quality paths than sampling-based kinodynamic planning with random controls in fewer iterations and computation time.

[1]  Thierry Siméon,et al.  Transition-based RRT for path planning in continuous cost spaces , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[2]  Nathan R. Sturtevant,et al.  Benchmarks for Grid-Based Pathfinding , 2012, IEEE Transactions on Computational Intelligence and AI in Games.

[3]  Sergey Levine,et al.  Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor , 2018, ICML.

[4]  Michael C. Yip,et al.  Dynamically Constrained Motion Planning Networks for Non-Holonomic Robots , 2020, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[5]  Wojciech Zaremba,et al.  OpenAI Gym , 2016, ArXiv.

[6]  Steven M. LaValle,et al.  Randomized Kinodynamic Planning , 1999, Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C).

[7]  Michael C. Yip,et al.  Motion Planning Networks: Bridging the Gap Between Learning-Based and Classical Motion Planners , 2019, IEEE Transactions on Robotics.

[8]  Jiankun Wang,et al.  Neural RRT*: Learning-Based Optimal Path Planning , 2020, IEEE Transactions on Automation Science and Engineering.

[9]  Lydia Tapia,et al.  RL-RRT: Kinodynamic Motion Planning via Learning Reachability Estimators From RL Policies , 2019, IEEE Robotics and Automation Letters.

[10]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[11]  Marcin Andrychowicz,et al.  Hindsight Experience Replay , 2017, NIPS.

[12]  Kostas E. Bekris,et al.  Efficient and Asymptotically Optimal Kinodynamic Motion Planning via Dominance-Informed Regions , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[13]  Marco Pavone,et al.  Learning Sampling Distributions for Robot Motion Planning , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[14]  Lydia Tapia,et al.  PRM-RL: Long-range Robotic Navigation Tasks by Combining Reinforcement Learning and Sampling-Based Planning , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[15]  Herke van Hoof,et al.  Addressing Function Approximation Error in Actor-Critic Methods , 2018, ICML.

[16]  Aviv Tamar,et al.  Harnessing Reinforcement Learning for Neural Motion Planning , 2019, Robotics: Science and Systems.

[17]  Zhijun Li,et al.  Neural Network Approximation Based Near-Optimal Motion Planning With Kinodynamic Constraints Using RRT , 2018, IEEE Transactions on Industrial Electronics.

[18]  Kostas E. Bekris,et al.  Asymptotically optimal sampling-based kinodynamic planning , 2014, Int. J. Robotics Res..

[19]  Siddhartha S. Srinivasa,et al.  LEGO: Leveraging Experience in Roadmap Generation for Sampling-Based Planning , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[20]  Kris Hauser,et al.  Asymptotically Optimal Planning by Feasible Kinodynamic Planning in a State–Cost Space , 2015, IEEE Transactions on Robotics.

[21]  Edward Schmerling,et al.  Learned Critical Probabilistic Roadmaps for Robotic Motion Planning , 2019 .