A novel reinforcement learning based grey wolf optimizer algorithm for unmanned aerial vehicles (UAVs) path planning

Abstract Unmanned aerial vehicles (UAVs) have been used in wide range of areas, and a high-quality path planning method is needed for UAVs to satisfy their applications. However, many algorithms reported in the literature may not feasible or efficient, especially in the face of three-dimensional complex flight environment. In this paper, a novel reinforcement learning based grey wolf optimizer algorithm called RLGWO has been presented for solving this problem. In the proposed algorithm, the reinforcement learning is inserted that the individual is controlled to switch operations adaptively according to the accumulated performance. Considering that the proposed algorithm is designed to serve for UAVs path planning, four operations have been introduced for each individual: exploration, exploitation, geometric adjustment, and optimal adjustment. In addition, the cubic B-spline curve is used to smooth the generated flight route and make the planning path be suitable for the UAVs. The simulation experimental results show that the RLGWO algorithm can acquire a feasible and effective route successfully in complicated environment.

[1]  Jan Peters,et al.  Reinforcement learning in robotics: A survey , 2013, Int. J. Robotics Res..

[2]  Seyedali Mirjalili,et al.  Three-dimensional path planning for UCAV using an improved bat algorithm , 2016 .

[3]  Richard S. Sutton,et al.  Learning to predict by the methods of temporal differences , 1988, Machine Learning.

[4]  Bin Luo,et al.  Lifetime Enhancement in Wireless Sensor Networks Using Fuzzy Approach and A-Star Algorithm , 2012 .

[5]  Zong Woo Geem,et al.  A survey on applications of the harmony search algorithm , 2013, Eng. Appl. Artif. Intell..

[6]  Wen Yu,et al.  Path planning of multi-agent systems in unknown environment with neural kernel smoothing and reinforcement learning , 2017, Neurocomputing.

[7]  Meng-Han Tsai,et al.  UAV path planning method for digital terrain model reconstruction – A debris fan example , 2018, Automation in Construction.

[8]  Wei Liu,et al.  Bi-level programming based real-time path planning for unmanned aerial vehicles , 2013, Knowl. Based Syst..

[9]  Dipayan Guha,et al.  Load frequency control of interconnected power system using grey wolf optimization , 2016, Swarm Evol. Comput..

[10]  Songmin Jia,et al.  A novel phase angle-encoded fruit fly optimization algorithm with mutation adaptation mechanism applied to UAV path planning , 2018, Appl. Soft Comput..

[11]  Haibin Duan,et al.  Social-class pigeon-inspired optimization and time stamp segmentation for multi-UAV cooperative path planning , 2018, Neurocomputing.

[12]  Qidi Wu,et al.  A survey of biogeography-based optimization , 2017, Neural Computing and Applications.

[13]  P. B. Sujit,et al.  Unmanned Aerial Vehicle Path Following: A Survey and Analysis of Algorithms for Fixed-Wing Unmanned Aerial Vehicless , 2014, IEEE Control Systems.

[14]  Cheng Yongbo,et al.  Three-dimensional unmanned aerial vehicle path planning using modified wolf pack search algorithm , 2017 .

[15]  Frank L. Lewis,et al.  Optimal and Autonomous Control Using Reinforcement Learning: A Survey , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[16]  Aboul Ella Hassanien,et al.  Binary grey wolf optimization approaches for feature selection , 2016, Neurocomputing.

[17]  Ian Postlethwaite,et al.  A Probabilistically Robust Path Planning Algorithm for UAVs Using Rapidly-Exploring Random Trees , 2013, J. Intell. Robotic Syst..

[18]  Yu Huang,et al.  Path planning for solar-powered UAV in urban environment , 2018, Neurocomputing.

[19]  Song Wang,et al.  UGV Navigation Optimization Aided by Reinforcement Learning-Based Path Tracking , 2018, IEEE Access.

[20]  Robert Babuska,et al.  A Survey of Actor-Critic Reinforcement Learning: Standard and Natural Policy Gradients , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[21]  Jianqiao Yu,et al.  UAV path planning using artificial potential field method updated by optimal control theory , 2016, Int. J. Syst. Sci..

[22]  J. Anitha,et al.  Optimum laplacian wavelet mask based medical image using hybrid cuckoo search - grey wolf optimization algorithm , 2017, Knowl. Based Syst..

[23]  Dinesh Kumar,et al.  An astrophysics-inspired Grey wolf algorithm for numerical optimization and its application to engineering design problems , 2017, Adv. Eng. Softw..

[24]  Sen Wang,et al.  Adaptive low-level control of autonomous underwater vehicles using deep reinforcement learning , 2018, Robotics Auton. Syst..

[25]  Jianjun Jiao,et al.  Inspired grey wolf optimizer for solving large-scale function optimization problems , 2018, Applied Mathematical Modelling.

[26]  Mohammadreza Radmanesh,et al.  Flight formation of UAVs in presence of moving obstacles using fast-dynamic mixed integer linear programming , 2016 .

[27]  Mohammadreza Radmanesh,et al.  Grey wolf optimization based sense and avoid algorithm in a Bayesian framework for multiple UAV path planning in an uncertain environment , 2018, Aerospace Science and Technology.

[28]  Jun Zhang,et al.  Adaptive Particle Swarm Optimization , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[29]  Jianjun Jiao,et al.  An exploration-enhanced grey wolf optimizer to solve high-dimensional numerical optimization , 2018, Eng. Appl. Artif. Intell..

[30]  Amit Konar,et al.  A Deterministic Improved Q-Learning for Path Planning of a Mobile Robot , 2013, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[31]  Shengxiang Yang,et al.  Hybrid meta-heuristic algorithms for independent job scheduling in grid computing , 2018, Appl. Soft Comput..

[32]  Jianqiao Yu,et al.  Modified central force optimization (MCFO) algorithm for 3D UAV path planning , 2016, Neurocomputing.

[33]  Tran Hiep Dinh,et al.  Enhanced discrete particle swarm optimization path planning for UAV vision-based surface inspection , 2017, ArXiv.

[34]  Andrew Lewis,et al.  Grey Wolf Optimizer , 2014, Adv. Eng. Softw..

[35]  Yang Liu,et al.  Survey on computational-intelligence-based UAV path planning , 2018, Knowl. Based Syst..

[36]  Srikanth Saripalli,et al.  Sampling-Based Path Planning for UAV Collision Avoidance , 2017, IEEE Transactions on Intelligent Transportation Systems.