A Two-Step Environment-Learning-Based Method for Optimal UAV Deployment

Unmanned aerial vehicles (UAVs) can be used as low-altitude flight base stations to satisfy the coverage requirements of wireless users in various scenarios. In practical applications, since the transmitted power and energy resources of the UAVs are limited and the propagation environments are complicated and time-variant, it is challenging to control a group of UAVs to ensure coverage performance while preserving the connectivity and safety of the UAV networks. To this end, a two-step environment-learning-based method is proposed for the intelligent deployment of the UAVs. First, a machine learning algorithm is used to establish an accurate prediction model of the link qualities from the UAVs to the users under a specific scenario for the next step. Then, a modified deep deterministic policy gradient (DDPG) algorithm is employed to control the movements of the UAVs according to the predicted link qualities and to maximize the proportion of covered users. The prioritized experience replay mechanism is introduced to the standard DDPG algorithm to accelerate the deployment procedure. The coverage performance is analyzed in both the interference-free situation and the situation with co-channel interference. Simulation results have shown that the proposed method has a higher convergence speed than the standard DDPG method. Additionally, the proposed deployment method can achieve higher coverage performance and better adaptability to the dynamic environment than three commonly used methods, the random method, the K-means-based method, and the statistical-channel-model-based method.