Path Planning for UAV-Mounted Mobile Edge Computing With Deep Reinforcement Learning

In this letter, we study an unmanned aerial vehicle (UAV)-mounted mobile edge computing network, where the UAV executes computational tasks offloaded from mobile terminal users (TUs) and the motion of each TU follows a Gauss-Markov random model. To ensure the quality-of-service (QoS) of each TU, the UAV with limited energy dynamically plans its trajectory according to the locations of mobile TUs. Towards this end, we formulate the problem as a Markov decision process, wherein the UAV trajectory and UAV-TU association are modeled as the parameters to be optimized. To maximize the system reward and meet the QoS constraint, we develop a QoS-based action selection policy in the proposed algorithm based on double deep Q-network. Simulations show that the proposed algorithm converges more quickly and achieves a higher sum throughput than conventional algorithms.

[1]  Qingqing Wu,et al.  Joint Trajectory and Communication Design for Multi-UAV Enabled Wireless Networks , 2017, IEEE Transactions on Wireless Communications.

[2]  David Silver,et al.  Deep Reinforcement Learning with Double Q-Learning , 2015, AAAI.

[3]  Feng Lyu,et al.  Space/Aerial-Assisted Computing Offloading for IoT Applications: A Learning-Based Approach , 2019, IEEE Journal on Selected Areas in Communications.

[4]  Fumiyuki Adachi,et al.  Deep Reinforcement Learning for UAV Navigation Through Massive MIMO Technique , 2019, IEEE Transactions on Vehicular Technology.

[5]  Jian Ma,et al.  Learning-Based Energy-Efficient Data Collection by Unmanned Vehicles in Smart Cities , 2018, IEEE Transactions on Industrial Informatics.

[6]  Suvadip Batabyal,et al.  Mobility Models, Traces and Impact of Mobility on Opportunistic Routing Algorithms: A Survey , 2015, IEEE Communications Surveys & Tutorials.

[7]  Hui Zhao,et al.  Energy-Aware Dynamic Resource Allocation in UAV Assisted Mobile Edge Computing Over Social Internet of Vehicles , 2018, IEEE Access.

[8]  Ryu Miura,et al.  AC-POCA: Anticoordination Game Based Partially Overlapping Channels Assignment in Combined UAV and D2D-Based Networks , 2017, IEEE Transactions on Vehicular Technology.

[9]  Yan Zhang,et al.  Mobile Edge Computing: A Survey , 2018, IEEE Internet of Things Journal.

[10]  Ryu Miura,et al.  On A Novel Adaptive UAV-Mounted Cloudlet-Aided Recommendation System for LBSNs , 2019, IEEE Transactions on Emerging Topics in Computing.

[11]  Yuanwei Liu,et al.  Multi-UAV Dynamic Wireless Networking With Deep Reinforcement Learning , 2019, IEEE Communications Letters.

[12]  Gao Xiang,et al.  Fuzzy Q learning algorithm for dual-aircraft path planning to cooperatively detect targets by passive radars , 2013 .

[13]  F. Richard Yu,et al.  Intelligent Trajectory Design in UAV-Aided Communications With Reinforcement Learning , 2019, IEEE Transactions on Vehicular Technology.

[14]  Dusit Niyato,et al.  Hierarchical Game-Theoretic and Reinforcement Learning Framework for Computational Offloading in UAV-Enabled Mobile Edge Computing Networks With Multiple Service Providers , 2019, IEEE Internet of Things Journal.

[15]  Feng Shu,et al.  User Association and Path Planning for UAV-Aided Mobile Edge Computing With Energy Restriction , 2019, IEEE Wireless Communications Letters.

[16]  Jun Xu,et al.  Internet of Things Applications: Animal Monitoring with Unmanned Aerial Vehicle , 2016, ArXiv.

[17]  Xiao Liu,et al.  Trajectory Design and Power Control for Multi-UAV Assisted Wireless Networks: A Machine Learning Approach , 2018, IEEE Transactions on Vehicular Technology.

[18]  Walid Saad,et al.  Mobile Unmanned Aerial Vehicles (UAVs) for Energy-Efficient Internet of Things Communications , 2017, IEEE Transactions on Wireless Communications.

[19]  Rose Qingyang Hu,et al.  Computation Rate Maximization in UAV-Enabled Wireless-Powered Mobile-Edge Computing Systems , 2018, IEEE Journal on Selected Areas in Communications.

[20]  Ness B. Shroff,et al.  Non-convex optimization and rate control for multi-class services in the Internet , 2005, IEEE/ACM Transactions on Networking.