The Path Planning of Mobile Robot by Neural Networks and Hierarchical Reinforcement Learning

Existing mobile robots cannot complete some functions. To solve these problems, which include autonomous learning in path planning, the slow convergence of path planning, and planned paths that are not smooth, it is possible to utilize neural networks to enable to the robot to perceive the environment and perform feature extraction, which enables them to have a fitness of environment to state action function. By mapping the current state of these actions through Hierarchical Reinforcement Learning (HRL), the needs of mobile robots are met. It is possible to construct a path planning model for mobile robots based on neural networks and HRL. In this article, the proposed algorithm is compared with different algorithms in path planning. It underwent a performance evaluation to obtain an optimal learning algorithm system. The optimal algorithm system was tested in different environments and scenarios to obtain optimal learning conditions, thereby verifying the effectiveness of the proposed algorithm. Deep Deterministic Policy Gradient (DDPG), a path planning algorithm for mobile robots based on neural networks and hierarchical reinforcement learning, performed better in all aspects than other algorithms. Specifically, when compared with Double Deep Q-Learning (DDQN), DDPG has a shorter path planning time and a reduced number of path steps. When introducing an influence value, this algorithm shortens the convergence time by 91% compared with the Q-learning algorithm and improves the smoothness of the planned path by 79%. The algorithm has a good generalization effect in different scenarios. These results have significance for research on guiding, the precise positioning, and path planning of mobile robots.

[1]  Chao Wang,et al.  Design of Traffic Emergency Response System Based on Internet of Things and Data Mining in Emergencies , 2019, IEEE Access.

[2]  Stefano Stramigioli,et al.  On Reward Shaping for Mobile Robot Navigation: A Reinforcement Learning and SLAM Based Approach , 2020, ArXiv.

[3]  Dan Zhang,et al.  Path planning for active SLAM based on deep reinforcement learning under unknown environments , 2020, Intell. Serv. Robotics.

[4]  Chien-wen Shen,et al.  Behavioural intentions of using virtual reality in learning: perspectives of acceptance of information technology and learning style , 2018, Virtual Reality.

[5]  Brahim Bouzouia,et al.  Optimal path planning and execution for mobile robots using genetic algorithm and adaptive fuzzy-logic control , 2017, Robotics Auton. Syst..

[6]  Chaitali Chakrabarti,et al.  A Deep Q-Learning Approach for Dynamic Management of Heterogeneous Processors , 2019, IEEE Computer Architecture Letters.

[7]  Zhe Liu,et al.  Mobile Robot Path Planning in Dynamic Environments Through Globally Guided Reinforcement Learning , 2020, IEEE Robotics and Automation Letters.

[8]  Qiong Liu,et al.  Effects of environmental education on environmental ethics and literacy based on virtual reality technology , 2019, Electron. Libr..

[9]  Min Chen,et al.  Analyzing the trend of O2O commerce by bilingual text mining on social media , 2019, Comput. Hum. Behav..

[10]  Yonghui Song,et al.  A New Deep-Q-Learning-Based Transmission Scheduling Mechanism for the Cognitive Internet of Things , 2018, IEEE Internet of Things Journal.

[11]  Hans-Peter Seidel,et al.  VNect , 2017, ACM Trans. Graph..

[12]  Peter Nielsen,et al.  On the training of a neural network for online path planning with offline path planning algorithms , 2020, Int. J. Inf. Manag..

[13]  Yan Xu,et al.  Data-Driven Load Frequency Control for Stochastic Power Systems: A Deep Reinforcement Learning Method With Continuous Action Search , 2019, IEEE Transactions on Power Systems.

[14]  Jingda Wu,et al.  Energy management based on reinforcement learning with double deep Q-learning for a hybrid electric tracked vehicle , 2019, Applied Energy.

[15]  Min Chen,et al.  An adaptive deep Q-learning strategy for handwritten digit recognition , 2018, Neural Networks.

[16]  Wusheng Chou,et al.  Path planning for mobile robot using self-adaptive learning particle swarm optimization , 2016, Science China Information Sciences.

[17]  Yang Liu,et al.  Survey on computational-intelligence-based UAV path planning , 2018, Knowl. Based Syst..

[18]  Laurence T. Yang,et al.  A Double Deep Q-Learning Model for Energy-Efficient Edge Scheduling , 2019, IEEE Transactions on Services Computing.

[19]  Min Chen,et al.  The research of human individual's conformity behavior in emergency situations , 2018, Libr. Hi Tech.

[20]  Zhiyu Qu,et al.  Radar Signal Intra-Pulse Modulation Recognition Based on Convolutional Neural Network and Deep Q-Learning Network , 2020, IEEE Access.

[21]  Wang Peng,et al.  Research on Dynamic Path Planning of Wheeled Robot Based on Deep Reinforcement Learning on the Slope Ground , 2020 .

[22]  Frank L. Lewis,et al.  Discrete-Time Deterministic $Q$ -Learning: A Novel Convergence Analysis , 2017, IEEE Transactions on Cybernetics.

[23]  Oscar Montiel,et al.  Mobile robot path planning using membrane evolutionary artificial potential field , 2019, Appl. Soft Comput..

[24]  Chuanhuan Yin,et al.  Exponential Moving Averaged Q-Network for DDPG , 2019, PRCV.

[25]  Shin Ishii,et al.  Constrained Deep Q-Learning Gradually Approaching Ordinary Q-Learning , 2019, Front. Neurorobot..

[26]  Peng Wang,et al.  Research on Dynamic Path Planning of Wheeled Robot Based on Deep Reinforcement Learning on the Slope Ground , 2020, J. Robotics.

[27]  Pauline Ong,et al.  Solving the optimal path planning of a mobile robot using improved Q-learning , 2019, Robotics Auton. Syst..

[28]  Dushyant Rao,et al.  Large-scale cost function learning for path planning using deep inverse reinforcement learning , 2017, Int. J. Robotics Res..

[29]  Yingying Zheng,et al.  Bibliometric analysis for talent identification by the subject-author-citation three-dimensional evaluation model in the discipline of physical education , 2020, Libr. Hi Tech.

[30]  Yoonho Seo,et al.  Mobile robot path planning with surrounding point set and path improvement , 2017, Appl. Soft Comput..

[31]  Henry Zhu,et al.  Soft Actor-Critic Algorithms and Applications , 2018, ArXiv.

[32]  Marc Peter Deisenroth,et al.  Deep Reinforcement Learning: A Brief Survey , 2017, IEEE Signal Processing Magazine.

[33]  B. B. V. L. Deepak,et al.  Optimal Path Planning of Mobile Robot using Hybrid Cuckoo Search-Bat Algorithm , 2018 .

[34]  Zhian Zhang,et al.  Dynamic Path Planning of Unknown Environment Based on Deep Reinforcement Learning , 2018, J. Robotics.

[35]  Dayal R. Parhi,et al.  Analysis of FPA and BA meta-heuristic controllers for optimal path planning of mobile robot in cluttered environment , 2017 .