A Collaborative Multi-Agent Reinforcement Learning Anti-Jamming Algorithm in Wireless Networks

In this letter, we investigate the anti-jamming defense problem in multi-user scenarios, where the coordination among users is taken into consideration. The Markov game framework is employed to model and analyze the anti-jamming defense problem, and a collaborative multi-agent anti-jamming algorithm (CMAA) is proposed to obtain the optimal anti-jamming strategy. In sweep jamming scenarios, on the one hand, the proposed CMAA can tackle the external malicious jamming. On the other hand, it can effectively cope with the mutual interference among users. Moreover, we consider the impact of sensing errors due to miss detection and false alarm. Simulation results show that the proposed CMAA is superior to both sensing-based method and independent ${Q}$ -learning method, and has the highest normalized rate.

[1]  Alvin S. Lim,et al.  Jamming and anti-jamming techniques in wireless networks: a survey , 2014, Int. J. Ad Hoc Ubiquitous Comput..

[2]  Feten Slimeni,et al.  Cooperative Q-learning based channel selection for cognitive radio networks , 2019, Wirel. Networks.

[3]  Alagan Anpalagan,et al.  Stackelberg Game Approaches for Anti-Jamming Defence in Wireless Networks , 2018, IEEE Wireless Communications.

[4]  Jin Chen,et al.  Interference-Aware Cooperative Anti-Jamming Distributed Channel Selection in UAV Communication Networks , 2018, Applied Sciences.

[5]  Alagan Anpalagan,et al.  A Game-Theoretic Learning Approach for Anti-Jamming Dynamic Spectrum Access in Dense Wireless Networks , 2019, IEEE Transactions on Vehicular Technology.

[6]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[7]  Q. Hu,et al.  Markov decision processes with their applications , 2007 .

[8]  Nikos Vlassis,et al.  A Concise Introduction to Multiagent Systems and Distributed Artificial Intelligence I Mobk077-fm Synthesis Lectures on Artificial Intelligence and Machine Learning a Concise Introduction to Multiagent Systems and Distributed Artificial Intelligence a Concise Introduction to Multiagent Systems and D , 2007 .

[9]  Liang Xiao,et al.  Anti-Jamming Transmission Stackelberg Game With Observation Errors , 2015, IEEE Communications Letters.

[10]  Sudharman K. Jayaweera,et al.  A novel cognitive anti-jamming stochastic game , 2017, 2017 Cognitive Communications for Aerospace Applications Workshop (CCAA).

[11]  Alagan Anpalagan,et al.  Opportunistic Spectrum Access in Cognitive Radio Networks: Global Optimization Using Local Interaction Games , 2012, IEEE Journal of Selected Topics in Signal Processing.

[12]  Sudharman K. Jayaweera,et al.  Multi-Agent Reinforcement Learning Based Cognitive Anti-Jamming , 2017, 2017 IEEE Wireless Communications and Networking Conference (WCNC).

[13]  Yuhua Xu,et al.  A hierarchical learning approach to anti-jamming channel selection strategies , 2019, Wirel. Networks.

[14]  Feten Slimeni,et al.  Jamming mitigation in cognitive radio networks using a modified Q-learning algorithm , 2015, 2015 International Conference on Military Communications and Information Systems (ICMCIS).

[15]  Sudharman K. Jayaweera,et al.  Reinforcement learning based anti-jamming with wideband autonomous cognitive radios , 2016, 2016 IEEE/CIC International Conference on Communications in China (ICCC).

[16]  Xi Fang,et al.  Coping with a Smart Jammer in Wireless Networks: A Stackelberg Game Approach , 2013, IEEE Transactions on Wireless Communications.

[17]  Alagan Anpalagan,et al.  Opportunistic Spectrum Access with Spatial Reuse: Graphical Game and Uncoupled Learning Solutions , 2013, IEEE Transactions on Wireless Communications.

[18]  Alagan Anpalagan,et al.  A Hierarchical Learning Solution for Anti-Jamming Stackelberg Game With Discrete Power Strategies , 2017, IEEE Wireless Communications Letters.

[19]  Sudharman K. Jayaweera,et al.  A cognitive anti-jamming and interference-avoidance stochastic game , 2017, 2017 IEEE 16th International Conference on Cognitive Informatics & Cognitive Computing (ICCI*CC).

[20]  Bart De Schutter,et al.  A Comprehensive Survey of Multiagent Reinforcement Learning , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).