Joint Optimization of Multi-UAV Target Assignment and Path Planning Based on Multi-Agent Reinforcement Learning

One of the major research topics in unmanned aerial vehicle (UAV) collaborative control systems is the problem of multi-UAV target assignment and path planning (MUTAPP). It is a complicated optimization problem in which target assignment and path planning are solved separately. However, recalculation of the optimal results is too slow for real-time operations in dynamic environments because of the large number of calculations required. In this paper, we propose an artificial intelligence method named simultaneous target assignment and path planning (STAPP) based on a multi-agent deep deterministic policy gradient (MADDPG) algorithm, which is a type of multi-agent reinforcement learning algorithm. In STAPP, the MUTAPP problem is first constructed as a multi-agent system. Then, the MADDPG framework is used to train the system to solve target assignment and path planning simultaneously according to a corresponding reward structure. The proposed system can deal with dynamic environments effectively as its execution only requires the locations of the UAVs, targets, and threat areas. Real-time performance can be guaranteed as the neural network used in the system is simple. In addition, we develop a technique to improve the training effect and use experiments to demonstrate the effectiveness of our method.

[1]  David C. Hyland,et al.  Control law design using repeated trials , 2001, Proceedings of the 2001 American Control Conference. (Cat. No.01CH37148).

[2]  Panos M. Pardalos,et al.  A Greedy Randomized Adaptive Search Procedure for the Quadratic Assignment Problem , 1993, Quadratic Assignment and Related Problems.

[3]  Martin Grötschel,et al.  Frequency assignment in cellular phone networks , 1998, Ann. Oper. Res..

[4]  Yi Wu,et al.  Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments , 2017, NIPS.

[5]  Timothy W. McLain,et al.  Cooperative control of UAV rendezvous , 2001, Proceedings of the 2001 American Control Conference. (Cat. No.01CH37148).

[6]  Mehdi Alighanbari,et al.  Task assignment algorithms for teams of UAVs in dynamic environments , 2004 .

[7]  Steven R. Rasmussen,et al.  Task Allocation for Wide Area Search Munitions via Iterative Network Flow , 2002 .

[8]  Wynn C. Stirling,et al.  Conditional preferences for social systems , 2001, 2001 IEEE International Conference on Systems, Man and Cybernetics. e-Systems and e-Man for Cybernetics in Cyberspace (Cat.No.01CH37236).

[9]  Jonathan P. How,et al.  Receding horizon control of autonomous aerial vehicles , 2002, Proceedings of the 2002 American Control Conference (IEEE Cat. No.CH37301).

[10]  Q H Wang,et al.  A Modeling Method of Multiple Targets Assignment under Multiple UAVs’ Cooperation , 2017 .

[11]  Luitpold Babel,et al.  Coordinated Target Assignment and UAV Path Planning with Timing Constraints , 2018, J. Intell. Robotic Syst..

[12]  Timothy W. McLain,et al.  Coordinated target assignment and intercept for unmanned air vehicles , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[13]  T. Shima,et al.  UAV cooperative multiple task assignments using genetic algorithms , 2005, Proceedings of the 2005, American Control Conference, 2005..

[14]  Yuval Tassa,et al.  Continuous control with deep reinforcement learning , 2015, ICLR.

[15]  Derui Ding,et al.  Path Planning via an Improved DQN-Based Learning Policy , 2019, IEEE Access.

[16]  L. Singh,et al.  Trajectory generation for a UAV in urban terrain, using nonlinear MPC , 2001, Proceedings of the 2001 American Control Conference. (Cat. No.01CH37148).

[17]  Xiaohong Xu,et al.  Targets Assignment for Cooperative Multi-UAVs Based on Chaos Optimization Algorithm , 2008, 2008 The 9th International Conference for Young Computer Scientists.

[18]  Zhi-guang Shi,et al.  A New Method of UAVs Multi-target Task Assignment , 2018, DEStech Transactions on Engineering and Technology Research.

[19]  Jonathan P. How,et al.  Multi-Task Allocation and Path Planning for Cooperating UAVs , 2003 .

[20]  Pu Miao,et al.  Low-Complexity Path Planning Algorithm for Unmanned Aerial Vehicles in Complicated Scenarios , 2018, IEEE Access.

[21]  Jonathan P. How,et al.  COORDINATION AND CONTROL OF MULTIPLE UAVs , 2002 .

[22]  Wright-Patterson Afb,et al.  Hierarchical Control for Autonomous Teams , 2001 .

[23]  Jorge Tierno,et al.  Distributed autonomous control of concurrent combat tasks , 2001 .