Integration of Software Defined Radios and Software Defined Networking Towards Reinforcement Learning Enabled Unmanned Aerial Vehicle Networks

Advances in unmanned aerial vehicles (UAVs) enablethe development of many fields. The UAV networking capabilityis vital to the performance of the UAV fleet. However, thefluid characteristics make the UAV networking hard to meetthe requirement of quick response when some UAVs are out ofpower. In this paper, we propose an approach that integrates thesoftware defined radios (SDR) and software defined networking(SDN) to realize the update of UAV networking. We leverage theSDR to update link layer and SDN to manage the networkinglayer, which is driven by reinforcement learning. In our eval-uation, the results show that reinforcement learning can meetthe requirement of network update when some UAVs are out ofpower.

[1]  Yisheng Zhong,et al.  Time-varying formation tracking for UAV swarm systems with switching interaction topologies , 2016, 2016 35th Chinese Control Conference (CCC).

[2]  Abolfazl Razi,et al.  Distributed Cooperative Spectrum Sharing in UAV Networks Using Multi-Agent Reinforcement Learning , 2018, 2019 16th IEEE Annual Consumer Communications & Networking Conference (CCNC).

[3]  Walid Saad,et al.  Interference Management for Cellular-Connected UAVs: A Deep Reinforcement Learning Approach , 2018, IEEE Transactions on Wireless Communications.

[4]  Liu Kai,et al.  Research on UAV communication network topology based on small world network model , 2017, 2017 IEEE International Conference on Unmanned Systems (ICUS).

[5]  Chi Harold Liu,et al.  Distributed Energy-Efficient Multi-UAV Navigation for Long-Term Communication Coverage by Deep Reinforcement Learning , 2020, IEEE Transactions on Mobile Computing.

[6]  Ziyang Meng,et al.  Distributed Formation Control for Multiple Vertical Takeoff and Landing UAVs With Switching Topologies , 2018, IEEE/ASME Transactions on Mechatronics.

[7]  Jean Berger,et al.  A hybrid approach based on multi-agent geosimulation and reinforcement learning to solve a UAV patrolling problem , 2008, 2008 Winter Simulation Conference.

[8]  Antonio Franchi,et al.  Bilateral teleoperation of a group of UAVs with communication delays and switching topology , 2012, 2012 IEEE International Conference on Robotics and Automation.

[9]  Abolfazl Razi,et al.  Optimal measurement policy for predicting UAV network topology , 2017, 2017 51st Asilomar Conference on Signals, Systems, and Computers.

[10]  Jian Wang,et al.  Software Defined Radio and Wireless Acoustic Networking for Amateur Drone Surveillance , 2018, IEEE Communications Magazine.

[11]  I-Ming Chen,et al.  Autonomous navigation of UAV by using real-time model-based reinforcement learning , 2016, 2016 14th International Conference on Control, Automation, Robotics and Vision (ICARCV).

[12]  Xiaoyu Cui,et al.  A Delay-based Reliability Assessment for MultiUAV with Time-varying Topologies , 2018 .

[13]  Walid Saad,et al.  Deep Reinforcement Learning for Interference-Aware Path Planning of Cellular-Connected UAVs , 2018, 2018 IEEE International Conference on Communications (ICC).

[14]  Li Li,et al.  A new rechargeable WSNs based Multi-UAVs Network and topology control algorithm , 2015, 2015 IEEE 10th Conference on Industrial Electronics and Applications (ICIEA).

[15]  Lav Gupta,et al.  Survey of Important Issues in UAV Communication Networks , 2016, IEEE Communications Surveys & Tutorials.

[16]  Wei Zhang,et al.  Coarse-to-Fine UAV Target Tracking With Deep Reinforcement Learning , 2019, IEEE Transactions on Automation Science and Engineering.

[17]  Lingyang Song,et al.  Reinforcement Learning for Decentralized Trajectory Design in Cellular UAV Networks With Sense-and-Send Protocol , 2018, IEEE Internet of Things Journal.

[18]  Guoqiang Hu,et al.  Time-Varying Formation Tracking for UAV Swarm Systems With Switching Directed Topologies , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[19]  Tor Arne Johansen,et al.  Cluster-based communication topology selection and UAV path planning in wireless sensor networks , 2013, 2013 International Conference on Unmanned Aircraft Systems (ICUAS).

[20]  Jianqiang Yi,et al.  Consensus Based Formation Control for Multi-UAV Systems with Time-varying Delays and Jointly Connected Topologies , 2018, 2018 IEEE 14th International Conference on Automation Science and Engineering (CASE).

[21]  Ying-Chang Liang,et al.  Deep Reinforcement Learning for User Access Control in UAV Networks , 2018, 2018 IEEE International Conference on Communication Systems (ICCS).

[22]  Jun Li,et al.  Task Offloading for UAV-based Mobile Edge Computing via Deep Reinforcement Learning , 2018, 2018 IEEE/CIC International Conference on Communications in China (ICCC).

[23]  A I Lysenko,et al.  Method of optimal UAV network topology control , 2010, 2010 20th International Crimean Conference "Microwave & Telecommunication Technology".

[24]  Cairo L. Nascimento,et al.  An experimental validation of reinforcement learning applied to the position control of UAVs , 2012, 2012 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[25]  Shen Lincheng,et al.  Vision-based behavior for UAV reactive avoidance by using a reinforcement learning method , 2016, 2016 12th World Congress on Intelligent Control and Automation (WCICA).

[26]  Yuan Shen,et al.  Autonomous Navigation of UAVs in Large-Scale Complex Environments: A Deep Reinforcement Learning Approach , 2019, IEEE Transactions on Vehicular Technology.

[27]  Weihua Zhuang,et al.  UAV Relay in VANETs Against Smart Jamming With Reinforcement Learning , 2018, IEEE Transactions on Vehicular Technology.

[28]  Chuan Ma,et al.  A Reinforcement Learning Based User Association Algorithm for UAV Networks , 2018, 2018 28th International Telecommunication Networks and Applications Conference (ITNAC).

[29]  Chi Harold Liu,et al.  Energy-Efficient UAV Control for Effective and Fair Communication Coverage: A Deep Reinforcement Learning Approach , 2018, IEEE Journal on Selected Areas in Communications.

[30]  Chao Wang,et al.  A DEEP REINFORCEMENT LEARNING APPROACH TO FLOCKING AND NAVIGATION OF UAVS IN LARGE-SCALE COMPLEX ENVIRONMENTS , 2018, 2018 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[31]  Fang Bin,et al.  Research on Cooperative Collision Avoidance Problem of Multiple UAV Based on Reinforcement Learning , 2017, 2017 10th International Conference on Intelligent Computation Technology and Automation (ICICTA).

[32]  Mehran Mesbahi,et al.  UAV flocking with wind gusts: Adaptive topology and model reduction , 2011, Proceedings of the 2011 American Control Conference.

[33]  Lincheng Shen,et al.  Digraph-based anti-communication-destroying topology design for multi-UAV formation , 2017, 2017 36th Chinese Control Conference (CCC).

[34]  Xiao Zhang,et al.  Autonomous navigation of UAV in large-scale unknown complex environment with deep reinforcement learning , 2017, 2017 IEEE Global Conference on Signal and Information Processing (GlobalSIP).