Continuous Maneuver Control and Data Capture Scheduling of Autonomous Drone in Wireless Sensor Networks

Thanks to flexible deployment and excellent maneuverability, autonomous drones are regarded as an effective means to enable aerial data capture in large-scale wireless sensor networks with limited to no cellular infrastructure, e.g., smart farming in a remote area. A key challenge in drone-assisted sensor networks is that the autonomous drone's maneuvering can give rise to buffer overflows at the ground sensors and unsuccessful data collection due to lossy airborne channels. In this paper, we propose a new Deep Deterministic Policy Gradient based Maneuver Control (DDPG-MC) scheme which minimizes the overall data packet loss through online training instantaneous headings and patrol velocities of the drone, and the selection of the ground sensors for data collection in a continuous action space. Moreover, the maneuver control of the drone and communication schedule is formulated as an absorbing Markov chain, where network states consist of battery energy levels, data queue backlogs, timestamps of the data collection, and channel conditions between the ground sensors and the drone. An experience replay memory is utilized onboard at the drone to store the training experiences of the maneuver control and communication schedule at each time step.