Learning and Uncertainty-Exploited Directional Antenna Control for Robust Long-Distance and Broad-Band Aerial Communication

Aerial communication using directional antennas (ACDA) is a promising solution to enable long-distance and broad-band unmanned aerial vehicle (UAV)-to-UAV networking. The automatic alignment of directional antennas allows the transmission energy to focus in certain direction and significantly extends the communication range and rejects interference. Robust automatic alignment of directional antennas is not easy to achieve, considering practical issues such as the limited on-board sensing devices due to the physical constraints of UAV payload and power supplies, uncertain and varying UAV movement patterns, and unstable GPS and unknown communication environments. In this paper, we develop reinforcement learning (RL)-based online antenna control solutions for the ACDA system to conquer these challenges. The control solution adopts an uncertain UAV mobility modeling and intention estimation framework to capture and predict the uncertain intentions of UAV maneuvers and hence permit robust tracking. To account for an unstable GPS environment, the control solution features a learning of communication channel models to provide additional measurement signals in GPS-denied settings. A novel stochastic optimal control solution for nonlinear random switching dynamics is developed that integrates RL, an effective uncertainty evaluation method called multivariate probabilistic collocation method (MPCM), and unscented Kalman Filter (UKF). Simulation studies are conducted to illustrate and validate the proposed solutions.

[1]  Donald F. Towsley,et al.  Topology control protocol using sectorized antennas in dense 802.11 wireless networks , 2009, 2009 17th IEEE International Conference on Network Protocols.

[2]  Christian Bettstetter,et al.  Achieving air-ground communications in 802.11 networks with three-dimensional aerial mobility , 2013, 2013 Proceedings IEEE INFOCOM.

[3]  Yan Wan,et al.  UAV-carried Long-distance Wi-Fi Communication Infrastructure , 2016 .

[4]  Sandip Roy,et al.  Multivariate Probabilistic Collocation Method for Effective Uncertainty Evaluation With Application to Air Traffic Flow Management , 2014, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[5]  Mahbub Hassan,et al.  Survey on UAV Cellular Communications: Practical Aspects, Standardization Advancements, Regulation, and Security Challenges , 2018, IEEE Communications Surveys & Tutorials.

[6]  Frank L. Lewis,et al.  Analysis of the Random Direction Mobility Model with a Sense-and-Avoid Protocol , 2017, 2017 IEEE Globecom Workshops (GC Wkshps).

[7]  Yan Wan,et al.  Analysis of Random Mobility Model with Sense and Avoid Protocols for UAV Traffic Management , 2018 .

[8]  Wan-Young Chung,et al.  Enhanced RSSI-Based Real-Time User Location Tracking System for Indoor and Outdoor Environments , 2007, 2007 International Conference on Convergence Information Technology (ICCIT 2007).

[9]  Yan Wan,et al.  A Comprehensive 3-Dimensional Random Mobility Modeling Framework for Airborne Networks , 2018, IEEE Access.

[10]  Pascal Lorenz,et al.  UAV-Assisted Supporting Services Connectivity in Urban VANETs , 2019, IEEE Transactions on Vehicular Technology.

[11]  Gilbert L. Peterson,et al.  Estimation and Validation of the 3D Smooth-Turn Mobility Model for Airborne Networks , 2013, MILCOM 2013 - 2013 IEEE Military Communications Conference.

[12]  Jeffrey K. Uhlmann,et al.  Unscented filtering and nonlinear estimation , 2004, Proceedings of the IEEE.

[13]  L. Josefsson,et al.  Conformal array antenna theory and design , 2006 .

[14]  Sherali Zeadally,et al.  Privacy and Civilian Drone Use: The Need for Further Regulation , 2018, IEEE Security & Privacy.

[15]  Mehdi Bennis,et al.  Drone Small Cells in the Clouds: Design, Deployment and Performance Analysis , 2014, GLOBECOM 2014.

[16]  Yi Zhou,et al.  A Smooth-Turn Mobility Model for Airborne Networks , 2013, IEEE Trans. Veh. Technol..

[17]  Sha Hu,et al.  On Ergodic Capacity and Optimal Number of Tiers in UAV-Assisted Communication Systems , 2018, IEEE Transactions on Vehicular Technology.

[18]  Frank L. Lewis,et al.  Learning and Uncertainty-Exploited Directional Antenna Control for Robust Aerial Networking , 2019, 2019 IEEE 90th Vehicular Technology Conference (VTC2019-Fall).

[19]  Evsen Yanmaz,et al.  Survey on Unmanned Aerial Vehicle Networks for Civil Applications: A Communications Viewpoint , 2016, IEEE Communications Surveys & Tutorials.

[20]  Yan Wan,et al.  Long-Range and Broadband Aerial Communication Using Directional Antennas (ACDA): Design and Implementation , 2017, IEEE Transactions on Vehicular Technology.

[21]  Jian Huang,et al.  Human-Walking-Intention-Based Motion Control of an Omnidirectional-Type Cane Robot , 2013, IEEE/ASME Transactions on Mechatronics.

[22]  Liang Dong,et al.  Performance Analysis of Mobile Ad Hoc Unmanned Aerial Vehicle Communication Networks with Directional Antennas , 2010 .

[23]  Frank L. Lewis,et al.  Strategic air traffic flow management under uncertainties using scalable sampling-based dynamic programming and Q-learning approaches , 2017, 2017 11th Asian Control Conference (ASCC).

[24]  Edward W. Knightly,et al.  Mobility resilience and overhead constrained adaptation in directional 60 GHz WLANs: protocol design and system implementation , 2016, MobiHoc.

[25]  Theodore S. Rappaport,et al.  Wireless communications - principles and practice , 1996 .

[26]  Luigi Fortuna,et al.  Reinforcement Learning and Adaptive Dynamic Programming for Feedback Control , 2009 .

[27]  Rudolph van der Merwe,et al.  The Unscented Kalman Filter , 2002 .

[28]  Yan Wan,et al.  A Survey and Analysis of Mobility Models for Airborne Networks , 2014, IEEE Communications Surveys & Tutorials.

[29]  D. Rajan Probability, Random Variables, and Stochastic Processes , 2017 .

[30]  Jeffery W. Weston,et al.  Experiment and field demonstration of a 802.11-based ground-UAV mobile ad-hoc network , 2009, MILCOM 2009 - 2009 IEEE Military Communications Conference.

[31]  James J. Filliben,et al.  A Scalable Sampling Method to High-Dimensional Uncertainties for Optimal and Reinforcement Learning-Based Controls , 2017, IEEE Control Systems Letters.

[32]  Christian Laugier,et al.  Exploiting map information for driver intention estimation at road intersections , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[33]  方震,et al.  RSSI Variability Characterization and Calibration Method in Wireless Sensor Network , 2011 .

[34]  Shuzhi Sam Ge,et al.  Human–Robot Collaboration Based on Motion Intention Estimation , 2014, IEEE/ASME Transactions on Mechatronics.

[35]  Rui Zhang,et al.  Wireless communications with unmanned aerial vehicles: opportunities and challenges , 2016, IEEE Communications Magazine.

[36]  Frank L. Lewis,et al.  Optimized Assistive Human–Robot Interaction Using Reinforcement Learning , 2016, IEEE Transactions on Cybernetics.

[37]  Eduardo Tovar,et al.  Energy Efficient Legitimate Wireless Surveillance of UAV Communications , 2019, IEEE Transactions on Vehicular Technology.

[38]  Frank L. Lewis,et al.  Adaptive Optimal Decision in Multi-Agent Random Switching Systems , 2020, IEEE Control Systems Letters.

[39]  Kazuhiro Kosuge,et al.  Dance Step Estimation Method Based on HMM for Dance Partner Robot , 2007, IEEE Transactions on Industrial Electronics.

[40]  F. Lewis,et al.  Reinforcement Learning and Feedback Control: Using Natural Decision Methods to Design Optimal Adaptive Controllers , 2012, IEEE Control Systems.

[41]  Luiz A. DaSilva,et al.  Coverage Analysis for Low-Altitude UAV Networks in Urban Environments , 2017, GLOBECOM 2017 - 2017 IEEE Global Communications Conference.

[42]  Yan Wan,et al.  Design and implementation of aerial communication using directional antennas: learning control in unknown communication environments , 2019, IET Control Theory & Applications.

[43]  James Wheeler,et al.  Characterizing Routing with Radio-to-Router Information in a Heterogeneous Airborne Network , 2013, IEEE Transactions on Wireless Communications.

[44]  Mi Zhou,et al.  Airborne WiFi networks through directional antennae: An experimental study , 2015, 2015 IEEE Wireless Communications and Networking Conference (WCNC).

[45]  Frank L. Lewis,et al.  Estimation of Random Mobility Models using the Expectation-Maximization Method , 2018, 2018 IEEE 14th International Conference on Control and Automation (ICCA).