An Improved Algorithm of Robot Path Planning in Complex Environment Based on Double DQN

Deep Q Network (DQN) has several limitations when applied in planning a path in environment with a number of dilemmas according to our experiment. The reward function may be hard to model, and successful experience transitions are difficult to find in experience replay. In this context, this paper proposes an improved Double DQN (DDQN) to solve the problem by reference to A* and Rapidly-Exploring Random Tree (RRT). In order to achieve the rich experiments in experience replay, the initialization of robot in each training round is redefined based on RRT strategy. In addition, reward for the free positions is specially designed to accelerate the learning process according to the definition of position cost in A*. The simulation experimental results validate the efficiency of the improved DDQN, and robot could successfully learn the ability of obstacle avoidance and optimal path planning in which DQN or DDQN has no effect.

[1]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[2]  David Silver,et al.  Deep Reinforcement Learning with Double Q-Learning , 2015, AAAI.

[3]  Steven M. LaValle,et al.  Randomized Kinodynamic Planning , 1999, Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C).

[4]  Robert J. Szczerba,et al.  Robust algorithm for real-time route planning , 2000, IEEE Trans. Aerosp. Electron. Syst..

[5]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[6]  Ming Liu,et al.  Virtual-to-real deep reinforcement learning: Continuous control of mobile robots for mapless navigation , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[7]  P Supriya,et al.  Comparison of Temporal Difference Learning Algorithm and Dijkstra's Algorithm for Robotic Path Planning , 2018, 2018 Second International Conference on Intelligent Computing and Control Systems (ICICCS).

[8]  Jifeng Guo,et al.  A Deep Q-network (DQN) Based Path Planning Method for Mobile Robots , 2018, 2018 IEEE International Conference on Information and Automation (ICIA).

[9]  Derui Ding,et al.  Path Planning via an Improved DQN-Based Learning Policy , 2019, IEEE Access.

[10]  Xuesong Qiu,et al.  3D Aerial Base Station Position Planning based on Deep Q-Network for Capacity Enhancement , 2019, 2019 IFIP/IEEE Symposium on Integrated Network and Service Management (IM).

[11]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.