Energy and Service-Priority aware Trajectory Design for UAV-BSs using Double Q-Learning

Next-generation mobile networks have proposed the integration of Unmanned Aerial Vehicles (UAVs) as aerial base stations (UAV-BS) to serve ground nodes. Despite having advantages of using UAV-BSs, their dependence on the on-board, limited-capacity battery hinders their service continuity. Shorter trajectories can save flying energy, however, UAV-BSs must also serve nodes based on their service priority since nodes' service requirements are not always the same. In this paper, we present an energy-efficient trajectory optimization for a UAV assisted IoT system in which the UAV-BS considers the IoT nodes' service priorities in making its movement decisions. We solve the trajectory optimization problem using Double Q-Learning algorithm. Simulation results reveal that the Q-Learning based optimized trajectory outperforms a benchmark algorithm, namely Greedily-served algorithm, in terms of reducing the average energy consumption of the UAV-BS as well as the service delay for high priority nodes.

[1]  Nahum Shimkin,et al.  Averaged-DQN: Variance Reduction and Stabilization for Deep Reinforcement Learning , 2016, ICML.

[2]  Mahbub Hassan,et al.  Survey on UAV Cellular Communications: Practical Aspects, Standardization Advancements, Regulation, and Security Challenges , 2018, IEEE Communications Surveys & Tutorials.

[3]  Giorgio C. Buttazzo,et al.  Energy-Aware Coverage Path Planning of UAVs , 2015, 2015 IEEE International Conference on Autonomous Robot Systems and Competitions.

[4]  S. Kanhere,et al.  Trajectory Optimization of Flying Energy Sources using Q-Learning to Recharge Hotspot UAVs , 2020, IEEE INFOCOM 2020 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS).

[5]  Shigeru Shimamoto,et al.  Priority-Based Data Gathering Framework in UAV-Assisted Wireless Sensor Networks , 2016, IEEE Sensors Journal.

[6]  Dimitrios Zorbas,et al.  Energy Efficient Mobile Target Tracking Using Flying Drones , 2013, ANT/SEIT.

[7]  Jie Xu,et al.  Energy Minimization for Wireless Communication With Rotary-Wing UAV , 2018, IEEE Transactions on Wireless Communications.

[8]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[9]  Yuanwei Liu,et al.  Priority-Oriented Trajectory Planning for UAV-Aided Time-Sensitive IoT Networks , 2020, 2020 IEEE International Conference on Communications Workshops (ICC Workshops).

[10]  Nicolo Michelusi,et al.  Power-Constrained Trajectory optimization for Wireless UAV Relays with Random Requests , 2020, ICC 2020 - 2020 IEEE International Conference on Communications (ICC).

[11]  Hado van Hasselt,et al.  Double Q-learning , 2010, NIPS.

[12]  Salil S. Kanhere,et al.  AETD: An Application-Aware, Energy-Efficient Trajectory Design for Flying Base Stations , 2019, 2019 IEEE 14th Malaysia International Conference on Communication (MICC).

[13]  Wei Hu,et al.  Priority based data reporting algorithm in wireless sensor networks , 2017 .

[14]  Rui Zhang,et al.  Energy-Efficient UAV Communication With Trajectory Optimization , 2016, IEEE Transactions on Wireless Communications.

[15]  Salil S. Kanhere,et al.  Recharging of Flying Base Stations using Airborne RF Energy Sources , 2019, 2019 IEEE Wireless Communications and Networking Conference Workshop (WCNCW).