Energy and Quality Aware Multi-UAV Flight Path Design Through Q-Learning Algorithms

We address the problem of devising an optimized energy aware flight plan for multiple Unmanned Aerial Vehicles (UAVs) mounted Base Stations (BS) within heterogeneous networks. The chosen approach makes use of Q-learning algorithms, through the definition of a reward related to relevant quality and battery consumption metrics, providing also service overlapping avoidance between UAVs, that is two or more UAVs serving the same cluster area. Numerical simulations and different training show the effectiveness of the devised flight paths in improving the general quality of the heterogeneous network users.

[1]  Harpreet S. Dhillon,et al.  Poisson Cluster Process Based Analysis of HetNets With Correlated User and Base Station Locations , 2016, IEEE Transactions on Wireless Communications.

[2]  Francesca Cuomo,et al.  Qoe-Aware UAV Flight Path Design for Mobile Video Streaming in HetNet , 2018, 2018 IEEE 10th Sensor Array and Multichannel Signal Processing Workshop (SAM).

[3]  Rui Zhang,et al.  UAV-Aided Offloading for Cellular Hotspot , 2017, IEEE Transactions on Wireless Communications.

[4]  Izhak Rubin,et al.  CLEVER: A Cooperative and Cross-Layer Approach to Video Streaming in HetNets , 2018, IEEE Transactions on Mobile Computing.

[5]  Walid Saad,et al.  A Tutorial on UAVs for Wireless Networks: Applications, Challenges, and Open Problems , 2018, IEEE Communications Surveys & Tutorials.

[6]  Ben J. A. Kröse,et al.  Learning from delayed rewards , 1995, Robotics Auton. Syst..

[7]  Walid Saad,et al.  Machine Learning for Predictive On-Demand Deployment of Uavs for Wireless Communications , 2018, 2018 IEEE Global Communications Conference (GLOBECOM).

[8]  Bernhard Rinner,et al.  Persistent multi-UAV surveillance with energy and communication constraints , 2016, 2016 IEEE International Conference on Automation Science and Engineering (CASE).

[9]  David Hyunchul Shim,et al.  Persistent UAV Service: An Improved Scheduling Formulation and Prototypes of System Components , 2013, 2013 International Conference on Unmanned Aircraft Systems (ICUAS).

[10]  Rui Zhang,et al.  Wireless communications with unmanned aerial vehicles: opportunities and challenges , 2016, IEEE Communications Magazine.

[11]  Luciano Bononi,et al.  Joint Coverage, Connectivity, and Charging Strategies for Distributed UAV Networks , 2018, IEEE Transactions on Robotics.

[12]  Rui Zhang,et al.  Cellular-Connected UAV: Potential, Challenges, and Promising Technologies , 2018, IEEE Wireless Communications.

[13]  Paolo Dell'Olmo,et al.  Optimal Throughput Management in UAV-based Networks during Disasters , 2019, IEEE INFOCOM 2019 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS).

[14]  Raimund Schatz,et al.  Quantifying the impact of network bandwidth fluctuations and outages on Web QoE , 2015, 2015 Seventh International Workshop on Quality of Multimedia Experience (QoMEX).