Joint Time-frequency Anti-jamming Communications: A Reinforcement Learning Approach

This paper investigates the channel selection and transmission duration scheduling problem in jamming environment. Although stable communication frequency and long transmission time can reduce switching overhead and achieve higher throughput, it is more likely to be disturbed by jammer. Our goal is to find the optimal transmission channel and duration strategy in jamming environment to maximize the long-term cumulative throughput of the system. We formulate the decision-making problem as a Markov decision process (MDP). Then, we propose a reinforcement learning (Q-learning) based channel selection and transmission duration scheduling algorithm. The simulation results show that compared with the reinforcement learning based fixed transmission duration algorithm, the system utility is significantly improved.

[1]  Luliang Jia,et al.  A Collaborative Multi-Agent Reinforcement Learning Anti-Jamming Algorithm in Wireless Networks , 2018, IEEE Wireless Communications Letters.

[2]  Ximing Wang,et al.  Joint Power and Channel Selection for Anti-jamming Communications: A Reinforcement Learning Approach , 2019, ICML 2019.

[3]  Liu Hongwei,et al.  Reinforcement Learning based Anti-jamming Frequency Hopping Strategies Design for Cognitive Radar , 2018, 2018 IEEE International Conference on Signal Processing, Communications and Computing (ICSPCC).

[4]  Dong In Kim,et al.  Performance Analysis of Wireless Energy Harvesting Cognitive Radio Networks Under Smart Jamming Attacks , 2015, IEEE Transactions on Cognitive Communications and Networking.

[5]  Ki-Hyung Kim,et al.  Survey of Security Requirements, Attacks and Network Integration in Wireless Mesh Networks , 2008, FCST.

[6]  Bruno Clerckx,et al.  Does Frequent Low Resolution Feedback Outperform Infrequent High Resolution Feedback for Multiple Antenna Beamforming Systems? , 2011, IEEE Transactions on Signal Processing.

[7]  Mehul Motani,et al.  Buffer and channel adaptive transmission over fading channels with imperfect channel state information , 2004, 2004 IEEE Wireless Communications and Networking Conference (IEEE Cat. No.04TH8733).

[8]  Ying-Chang Liang,et al.  Average Throughput Analysis and Optimization in Cooperative IoT Networks With Short Packet Communication , 2018, IEEE Transactions on Vehicular Technology.

[9]  Alagan Anpalagan,et al.  Anti-Jamming Communications Using Spectrum Waterfall: A Deep Reinforcement Learning Approach , 2017, IEEE Communications Letters.

[10]  Sudharman K. Jayaweera,et al.  Reinforcement learning based anti-jamming with wideband autonomous cognitive radios , 2016, 2016 IEEE/CIC International Conference on Communications in China (ICCC).

[11]  Mohsen Guizani,et al.  Survey of Security Issues in Cognitive Radio Networks , 2011 .

[12]  Feten Slimeni,et al.  Jamming mitigation in cognitive radio networks using a modified Q-learning algorithm , 2015, 2015 International Conference on Military Communications and Information Systems (ICMCIS).

[13]  Guey-Yun Chang,et al.  A Jamming-Resistant Channel Hopping Scheme for Cognitive Radio Networks , 2017, IEEE Transactions on Wireless Communications.

[14]  Ximing Wang,et al.  A Reinforcement Learning Approach for Dynamic Spectrum Anti-jamming in Fading Environment , 2018, 2018 IEEE 18th International Conference on Communication Technology (ICCT).

[15]  Meng Wang,et al.  Power Control in Relay-Assisted Anti-Jamming Systems: A Bayesian Three-Layer Stackelberg Game Approach , 2019, IEEE Access.

[16]  Yangyang Li,et al.  A Q-Learning-Based Channel Selection and Data Scheduling Approach for High-Frequency Communications in Jamming Environment , 2019, MLICOM.

[17]  Abraham O. Fapojuwo,et al.  Stackelberg Equilibria of an Anti-Jamming Game in Cooperative Cognitive Radio Networks , 2018, IEEE Transactions on Cognitive Communications and Networking.

[18]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[19]  Michael C. Anderson,et al.  An Overview of Wireless Network Security , 2017, 2017 IEEE 4th International Conference on Cyber Security and Cloud Computing (CSCloud).

[20]  Alagan Anpalagan,et al.  A Hierarchical Learning Solution for Anti-Jamming Stackelberg Game With Discrete Power Strategies , 2017, IEEE Wireless Communications Letters.

[21]  Yang Yang,et al.  A Multi-Leader One-Follower Stackelberg Game Approach for Cooperative Anti-Jamming: No Pains, No Gains , 2018, IEEE Communications Letters.

[22]  Georgios B. Giannakis,et al.  Queuing with adaptive modulation and coding over wireless links: cross-Layer analysis and design , 2005, IEEE Transactions on Wireless Communications.