UAV Access Point Placement for Connectivity to a User with Unknown Location Using Deep RL

In recent years, unmanned aerial vehicles (UAVs) have been considered for telecommunications purposes as relays, caches, or IoT data collectors. In addition to being easy to deploy, their maneuverability allows them to adjust their location to optimize the capacity of the link to the user equipment on the ground or of the link to the basestation. The majority of the previous work that analyzes the optimal placement of such a UAV makes at least one of two assumptions: the channel can be predicted using a simple model or the locations of the users on the ground are known. In this paper, we use deep reinforcement learning (deep RL) to optimally place a UAV serving a ground user in an urban environment, without the previous knowledge of the channel or user location. Our algorithm relies on signal-to-interference-plus- noise ratio (SINR) measurements and a 3D map of the topology to account for blockage and scatterers. Furthermore, it is designed to operate in any urban environment. Results in conditions simulated by a ray tracing software show that with the constraint on the maximum number of iterations our algorithm has a 90% success rate in converging to a target SINR.

[1]  Walid Saad,et al.  Efficient Deployment of Multiple Unmanned Aerial Vehicles for Optimal Wireless Coverage , 2016, IEEE Communications Letters.

[2]  Lin Cai,et al.  UAV-Assisted Dynamic Coverage in a Heterogeneous Cellular System , 2017, IEEE Network.

[3]  Rui Zhang,et al.  Throughput Maximization for UAV-Enabled Mobile Relaying Systems , 2016, IEEE Transactions on Communications.

[4]  Yasamin Mostofi,et al.  Path Planning for a Connectivity Seeking Robot , 2017, 2017 IEEE Globecom Workshops (GC Wkshps).

[5]  David Gesbert,et al.  Optimal positioning of flying relays for wireless networks: A LOS map approach , 2017, 2017 IEEE International Conference on Communications (ICC).

[6]  David Gesbert,et al.  Learning to Communicate in UAV-Aided Wireless Networks: Map-Based Approaches , 2018, IEEE Internet of Things Journal.

[7]  Hyondong Oh,et al.  Optimal positioning of communication relay unmanned aerial vehicles in urban environments , 2016, 2016 International Conference on Unmanned Aircraft Systems (ICUAS).

[8]  Jie Xu,et al.  Capacity Characterization of UAV-Enabled Two-User Broadcast Channel , 2018, IEEE Journal on Selected Areas in Communications.

[9]  Fatih Erden,et al.  RSS-Based Q-Learning for Indoor UAV Navigation , 2019, MILCOM 2019 - 2019 IEEE Military Communications Conference (MILCOM).

[10]  David Silver,et al.  Deep Reinforcement Learning with Double Q-Learning , 2015, AAAI.

[11]  Jin Chen,et al.  Unmanned Aerial Vehicle-Aided Communications: Joint Transmit Power and Trajectory Optimization , 2018, IEEE Wireless Communications Letters.

[12]  Xiao Liu,et al.  Deployment and Movement for Multiple Aerial Base Stations by Reinforcement Learning , 2018, 2018 IEEE Globecom Workshops (GC Wkshps).

[13]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[14]  Tom Schaul,et al.  Dueling Network Architectures for Deep Reinforcement Learning , 2015, ICML.

[15]  Yimin Zhang,et al.  Joint optimization of relay position and power allocation in cooperative broadcast wireless networks , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[16]  David Gesbert,et al.  UAV-relay Placement with Unknown User Locations and Channel Parameters , 2018, 2018 52nd Asilomar Conference on Signals, Systems, and Computers.