Interference-Aware Cooperative Anti-Jamming Distributed Channel Selection in UAV Communication Networks

This paper investigates the cooperative anti-jamming distributed channel selection problem in UAV communication networks. Considering the existence of malicious jamming and co-channel interference, we design an interference-aware cooperative anti-jamming scheme for the purpose of maximizing users’ utilities. Moreover, the channel switching cost and cooperation cost are introduced, which have a great impact on users’ utilities. Users in the UAV group sense the co-channel interference signal energy to judge whether they are influenced by co-channel interference. When the received co-channel interference signal energy is lower than the co-channel interference threshold, users conduct channel selection strategies independently. Otherwise, users cooperate with each other and take joint actions with a cooperative anti-jamming pattern under the impact of co-channel interference. Aiming at the independent anti-jamming channel selection problem under no co-channel interference, a Markov decision process framework is introduced, whereas for the cooperative anti-jamming channel selection case under the influence of co-channel mutual interference, a Markov game framework is employed. Furthermore, motivated by Q-learning with a “cooperation-decision-feedback-adjustment” idea, we design an interference-aware cooperative anti-jamming distributed channel selection algorithm (ICADCSA) to obtain the optimal anti-jamming channel strategies for users in a distributed way. In addition, a discussion on the quick decision for UAVs is conducted. Finally, simulation results show that the proposed algorithm converges to a stable solution with which the UAV group can avoid malicious jamming, as well as co-channel interference effectively and can realize a quick decision in high mobility UAV communication networks.

[1]  Alagan Anpalagan,et al.  Distributed Channel Selection in Time-Varying Radio Environment: Interference Mitigation Game With Uncoupled Stochastic Learning , 2013, IEEE Transactions on Vehicular Technology.

[2]  Feten Slimeni,et al.  Cooperative Q-learning based channel selection for cognitive radio networks , 2019, Wirel. Networks.

[3]  Lav Gupta,et al.  Survey of Important Issues in UAV Communication Networks , 2016, IEEE Communications Surveys & Tutorials.

[4]  Alagan Anpalagan,et al.  Anti-Jamming Communications Using Spectrum Waterfall: A Deep Reinforcement Learning Approach , 2017, IEEE Communications Letters.

[5]  Weihua Zhuang,et al.  UAV Relay in VANETs Against Smart Jamming With Reinforcement Learning , 2018, IEEE Transactions on Vehicular Technology.

[6]  Haitao Zheng,et al.  Distributed Rule-Regulated Spectrum Sharing , 2008, IEEE Journal on Selected Areas in Communications.

[7]  Ian F. Akyildiz,et al.  NeXt generation/dynamic spectrum access/cognitive radio wireless networks: A survey , 2006, Comput. Networks.

[8]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[9]  Yuhua Xu,et al.  A hierarchical learning approach to anti-jamming channel selection strategies , 2019, Wirel. Networks.

[10]  Sudharman K. Jayaweera,et al.  A cognitive anti-jamming and interference-avoidance stochastic game , 2017, 2017 IEEE 16th International Conference on Cognitive Informatics & Cognitive Computing (ICCI*CC).

[11]  Bart De Schutter,et al.  A Comprehensive Survey of Multiagent Reinforcement Learning , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[12]  Alagan Anpalagan,et al.  Stackelberg Game Approaches for Anti-Jamming Defence in Wireless Networks , 2018, IEEE Wireless Communications.

[13]  Mengyuan Li,et al.  You Can Jam But You Cannot Hide: Defending Against Jamming Attacks for Geo-Location Database Driven Spectrum Sharing , 2016, IEEE Journal on Selected Areas in Communications.

[14]  Alagan Anpalagan,et al.  A game-theoretic perspective on self-organizing optimization for cognitive small cells , 2015, IEEE Communications Magazine.

[15]  Sudharman K. Jayaweera,et al.  A novel cognitive anti-jamming stochastic game , 2017, 2017 Cognitive Communications for Aerospace Applications Workshop (CCAA).

[16]  Liang Xiao,et al.  Anti-Jamming Transmission Stackelberg Game With Observation Errors , 2015, IEEE Communications Letters.

[17]  L. Buşoniu,et al.  A comprehensive survey of multi-agent reinforcement learning , 2011 .

[18]  Liang Xiao,et al.  Power control Stackelberg game in cooperative anti-jamming communications , 2014, The 2014 5th International Conference on Game Theory for Networks.

[19]  Lajos Hanzo,et al.  A Survey on Wireless Security: Technical Challenges, Recent Advances, and Future Trends , 2015, Proceedings of the IEEE.

[20]  Sudharman K. Jayaweera,et al.  Multi-Agent Reinforcement Learning Based Cognitive Anti-Jamming , 2017, 2017 IEEE Wireless Communications and Networking Conference (WCNC).

[21]  Jinlong Wang,et al.  Energy-Aware Joint User Scheduling and Power Control for Two-Tier Femtocell Networks: A Hierarchical Game Approach , 2018, IEEE Systems Journal.

[22]  Luliang Jia,et al.  A Collaborative Multi-Agent Reinforcement Learning Anti-Jamming Algorithm in Wireless Networks , 2018, IEEE Wireless Communications Letters.

[23]  Jin Chen,et al.  A One-Leader Multi-Follower Bayesian-Stackelberg Game for Anti-Jamming Transmission in UAV Communication Networks , 2018, IEEE Access.

[24]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[25]  Alagan Anpalagan,et al.  Opportunistic Spectrum Access in Unknown Dynamic Environment: A Game-Theoretic Stochastic Learning Solution , 2012, IEEE Transactions on Wireless Communications.

[26]  Nikos Vlassis,et al.  A Concise Introduction to Multiagent Systems and Distributed Artificial Intelligence I Mobk077-fm Synthesis Lectures on Artificial Intelligence and Machine Learning a Concise Introduction to Multiagent Systems and Distributed Artificial Intelligence a Concise Introduction to Multiagent Systems and D , 2007 .

[27]  Alagan Anpalagan,et al.  Opportunistic Spectrum Access with Spatial Reuse: Graphical Game and Uncoupled Learning Solutions , 2013, IEEE Transactions on Wireless Communications.

[28]  Stephanie Thalberg Markov Decision Processes With Their Applications , 2016 .

[29]  Alagan Anpalagan,et al.  Dynamic Spectrum Access in Time-Varying Environment: Distributed Learning Beyond Expectation Optimization , 2015, IEEE Transactions on Communications.

[30]  Tommaso Melodia,et al.  United Against the Enemy: Anti-Jamming Based on Cross-Layer Cooperation in Wireless Networks , 2016, IEEE Transactions on Wireless Communications.

[31]  Alagan Anpalagan,et al.  Self-Organizing Relay Selection in UAV Communication Networks: A Matching Game Perspective , 2018, IEEE Wireless Communications.

[32]  Walid Saad,et al.  Game Theory in Wireless and Communication Networks: Preface , 2011 .

[33]  Ximing Wang,et al.  A Reinforcement Learning Approach for Dynamic Spectrum Anti-jamming in Fading Environment , 2018, 2018 IEEE 18th International Conference on Communication Technology (ICCT).

[34]  Chunsheng Xin,et al.  A game-theoretical anti-jamming scheme for cognitive radio networks , 2013, IEEE Network.

[35]  Xi Fang,et al.  Coping with a Smart Jammer in Wireless Networks: A Stackelberg Game Approach , 2013, IEEE Transactions on Wireless Communications.