On the Performance of Deep Reinforcement Learning-Based Anti-Jamming Method Confronting Intelligent Jammer

With the development of access technologies and artificial intelligence, a deep reinforcement learning (DRL) algorithm is proposed into channel accessing and anti-jamming. Assuming the jamming modes are sweeping, comb, dynamic and statistic, the DRL-based method through training can almost perfectly avoid jamming signal and communicate successfully. Instead, in this paper, from the perspective of jammers, we investigate the performance of a DRL-based anti-jamming method. First of all, we design an intelligent jamming method based on reinforcement learning to combat the DRL-based user. Then, we theoretically analyze the condition when the DRL-based anti-jamming algorithm cannot converge, and provide the proof. Finally, in order to investigate the performance of DRL-based method, various scenarios where users with different communicating modes combat jammers with different jamming modes are compared. As the simulation results show, the theoretical analysis is verified, and the proposed RL-based jamming can effectively restrict the performance of DRL-based anti-jamming method.

[1]  Xin Liu,et al.  A heterogeneous information fusion deep reinforcement learning for intelligent frequency selection of HF communication , 2018, China Communications.

[2]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[3]  Philip Bachman,et al.  Deep Reinforcement Learning that Matters , 2017, AAAI.

[4]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[5]  Alagan Anpalagan,et al.  Dynamic Spectrum Access in Time-Varying Environment: Distributed Learning Beyond Expectation Optimization , 2015, IEEE Transactions on Communications.

[6]  Mihaela van der Schaar,et al.  A systematic learning method for optimal jamming , 2015, 2015 IEEE International Conference on Communications (ICC).

[7]  Luliang Jia,et al.  Anti-jamming power control game for data packets transmission , 2017, 2017 IEEE 17th International Conference on Communication Technology (ICCT).

[8]  Bharat Bhushan,et al.  Security vulnerabilities and countermeasures against jamming attacks in Wireless Sensor Networks: A survey , 2017, 2017 International Conference on Computer, Communications and Electronics (Comptelix).

[9]  Furqan Jameel,et al.  A Comprehensive Survey on Cooperative Relaying and Jamming Strategies for Physical Layer Security , 2019, IEEE Communications Surveys & Tutorials.

[10]  Zhu Han,et al.  Dogfight in Spectrum: Jamming and Anti-Jamming in Multichannel Cognitive Radio Systems , 2009, GLOBECOM 2009 - 2009 IEEE Global Telecommunications Conference.

[11]  Liang Xiao,et al.  UAV-Aided 5G Communications with Deep Reinforcement Learning Against Jamming , 2018 .

[12]  Shlomo Shamai,et al.  Worst-case power-constrained noise for binary-input channels , 1992, IEEE Trans. Inf. Theory.

[13]  Alagan Anpalagan,et al.  Anti-Jamming Communications Using Spectrum Waterfall: A Deep Reinforcement Learning Approach , 2017, IEEE Communications Letters.

[14]  Sudharman K. Jayaweera,et al.  A novel cognitive anti-jamming stochastic game , 2017, 2017 Cognitive Communications for Aerospace Applications Workshop (CCAA).

[15]  Wei Feng,et al.  An Anti-Jamming Game in VANET Platoon with Reinforcement Learning , 2018, 2018 IEEE International Conference on Consumer Electronics-Taiwan (ICCE-TW).

[16]  Lajos Hanzo,et al.  A Survey on Wireless Security: Technical Challenges, Recent Advances, and Future Trends , 2015, Proceedings of the IEEE.

[17]  Sinan Gezici,et al.  Optimum Power Allocation for Average Power Constrained Jammers in the Presence of Non-Gaussian Noise , 2012, IEEE Communications Letters.

[18]  Yong Gao,et al.  Frequency-Space Domain Anti-Jamming Algorithm Assisted with Probability Statistics , 2013, 2013 International Conference on Information Technology and Applications.

[19]  R. Lippmann,et al.  An introduction to computing with neural nets , 1987, IEEE ASSP Magazine.

[20]  John N. Tsitsiklis,et al.  Analysis of Temporal-Diffference Learning with Function Approximation , 1996, NIPS.

[21]  P. Dayan.  The Convergence of TD(λ) for General λ , 1992, Machine Learning.

[22]  Sudharman K. Jayaweera,et al.  Reinforcement learning based anti-jamming with wideband autonomous cognitive radios , 2016, 2016 IEEE/CIC International Conference on Communications in China (ICCC).

[23]  Srdjan Capkun,et al.  Anti-jamming broadcast communication using uncoordinated spread spectrum techniques , 2010, IEEE Journal on Selected Areas in Communications.

[24]  Aditya Trivedi,et al.  Anti-jamming in cognitive radio networks using reinforcement learning algorithms , 2012, 2012 Ninth International Conference on Wireless and Optical Communications Networks (WOCN).

[25]  Jian Sun,et al.  Convolutional neural networks at constrained time cost , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Peng Ning,et al.  Randomized Differential DSSS: Jamming-Resistant Wireless Broadcast Communication , 2010, 2010 Proceedings IEEE INFOCOM.

[27]  H. T. Kung,et al.  Competing Mobile Network Game: Embracing antijamming and jamming strategies with reinforcement learning , 2013, 2013 IEEE Conference on Communications and Network Security (CNS).

[28]  Narendra Kumar,et al.  Ethical aspects and future of artificial intelligence , 2016, 2016 International Conference on Innovation and Challenges in Cyber Security (ICICCS-INBUSH).

[29]  Yuhua Xu,et al.  A hierarchical learning approach to anti-jamming channel selection strategies , 2019, Wirel. Networks.

[30]  Frank L. Lewis,et al.  Optimal and Autonomous Control Using Reinforcement Learning: A Survey , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[31]  Shesh Vardhan,et al.  Information jamming in Electronic warfare: Operational requirements and techniques , 2014, 2014 International Conference on Electronics, Communication and Computational Engineering (ICECCE).

[32]  Alagan Anpalagan,et al.  A Hierarchical Learning Solution for Anti-Jamming Stackelberg Game With Discrete Power Strategies , 2017, IEEE Wireless Communications Letters.

[33]  Liang Xiao,et al.  Anti-Jamming Underwater Transmission With Mobility and Learning , 2018, IEEE Communications Letters.

[34]  Anthony Ephremides,et al.  Jamming games in wireless networks with incomplete information , 2011, IEEE Communications Magazine.

[35]  Wayne E. Stark,et al.  An information theoretic study of communication in the presence of jamming , 1981 .

[36]  Charalampos Konstantopoulos,et al.  A survey on jamming attacks and countermeasures in WSNs , 2009, IEEE Communications Surveys & Tutorials.

[37]  Georges Kaddoum,et al.  A Survey on Intelligent MAC Layer Jamming Attacks and Countermeasures in WSNs , 2016, 2016 IEEE 84th Vehicular Technology Conference (VTC-Fall).

[38]  Muhammad Mahtab Alam,et al.  Jamming of UAV remote control systems using software defined radio , 2018, 2018 International Conference on Military Communications and Information Systems (ICMCIS).

[39]  Hui Liu,et al.  A novel jamming strategy-greedy bandit , 2017, 2017 IEEE 9th International Conference on Communication Software and Networks (ICCSN).

[40]  Mihaela van der Schaar,et al.  Jamming Bandits—A Novel Learning Method for Optimal Jamming , 2016, IEEE Transactions on Wireless Communications.

[41]  Wang Qiang,et al.  Reinforcement learning model, algorithms and its application , 2011, 2011 International Conference on Mechatronic Science, Electric Engineering and Computer (MEC).

[42]  Alagan Anpalagan,et al.  A Multi-Domain Anti-Jamming Defense Scheme in Heterogeneous Wireless Networks , 2018, IEEE Access.