Generalization of Deep Reinforcement Learning for Jammer-Resilient Frequency and Power Allocation

We tackle the problem of joint frequency and power allocation while emphasizing the generalization capability of a deep reinforcement learning model. Most of the existing methods solve reinforcement learning-based wireless problems for a specific pre-determined wireless network scenario. The performance of a trained agent tends to be very specific to the network and deteriorates when used in a different network operating scenario (e.g., different in size, neighborhood, and mobility, among others). We demonstrate our approach to enhance training to enable a higher generalization capability during inference of the deployed model in a distributed multi-agent setting in a hostile jamming environment. With all these, we show the improved training and inference performance of the proposed methods when tested on previously unseen simulated wireless networks of different sizes and architectures. More importantly, to prove practical impact, the end-to-end solution was implemented on the embedded software-defined radio and validated using over-the-air evaluation.

[1]  Anu Jagannath,et al.  Marconi-Rosenblatt Framework for Intelligent Networks (MR-iNet Gym): For Rapid Design and Implementation of Distributed Multi-agent Reinforcement Learning Solutions for Wireless Networks , 2022, Comput. Networks.

[2]  Keyvan Ramezanpour,et al.  MR-iNet Gym: Framework for Edge Deployment of Deep Reinforcement Learning on Embedded Software Defined Radio , 2022, WiseML@WiSec.

[3]  Qihao Zhou,et al.  Federated Reinforcement Learning: Techniques, Applications, and Open Challenges , 2021, Intelligence & Robotics.

[4]  Ying-Chang Liang,et al.  Deep Reinforcement Learning for Joint Channel Selection and Power Control in D2D Networks , 2021, IEEE Transactions on Wireless Communications.

[5]  Dongning Guo,et al.  Deep Reinforcement Learning for Joint Spectrum and Power Allocation in Cellular Networks , 2020, 2021 IEEE Globecom Workshops (GC Wkshps).

[6]  Tommaso Melodia,et al.  Redefining Wireless Communication for 6G: Signal Processing Meets Deep Learning With Deep Unfolding , 2020, IEEE Transactions on Artificial Intelligence.

[7]  Dongning Guo,et al.  Deep Actor-Critic Learning for Distributed Power Control in Wireless Mobile Networks , 2020, 2020 54th Asilomar Conference on Signals, Systems, and Computers.

[8]  Anatolij Zubow,et al.  ns-3 meets OpenAI Gym: The Playground for Machine Learning in Networking Research , 2019, MSWiM.

[9]  T. Başar,et al.  Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms , 2019, Handbook of Reinforcement Learning and Control.

[10]  Christophe Moy,et al.  Transfer restless multi-armed bandit policy for energy-efficient heterogeneous cellular network , 2019, EURASIP J. Adv. Signal Process..

[11]  Ekram Hossain,et al.  A Deep Q-Learning Method for Downlink Power Allocation in Multi-Cell Networks , 2019, ArXiv.

[12]  Tommaso Melodia,et al.  Machine Learning for Wireless Communications in the Internet of Things: A Comprehensive Survey , 2019, Ad Hoc Networks.

[13]  Ying-Chang Liang,et al.  Applications of Deep Reinforcement Learning in Communications and Networking: A Survey , 2018, IEEE Communications Surveys & Tutorials.

[14]  Dongning Guo,et al.  Multi-Agent Deep Reinforcement Learning for Dynamic Power Allocation in Wireless Networks , 2018, IEEE Journal on Selected Areas in Communications.

[15]  Kobi Cohen,et al.  Deep Multi-User Reinforcement Learning for Distributed Dynamic Spectrum Access , 2017, IEEE Transactions on Wireless Communications.

[16]  Li Wang,et al.  Learning Radio Resource Management in RANs: Framework, Opportunities, and Challenges , 2018, IEEE Communications Magazine.

[17]  Ranjan K. Mallik,et al.  A Machine Learning Approach for Power Allocation in HetNets Considering QoS , 2018, 2018 IEEE International Conference on Communications (ICC).

[18]  Zhi Chen,et al.  Intelligent Power Control for Spectrum Sharing in Cognitive Radios: A Deep Reinforcement Learning Approach , 2017, IEEE Access.

[19]  Soung Chang Liew,et al.  Deep-Reinforcement Learning Multiple Access for Heterogeneous Wireless Networks , 2017, 2018 IEEE International Conference on Communications (ICC).

[20]  Maryline Hélard,et al.  Energy Minimization in HARQ-I Relay-Assisted Networks With Delay-Limited Users , 2017, IEEE Transactions on Vehicular Technology.

[21]  Ananthram Swami,et al.  Distributed Algorithms for Learning and Cognitive Medium Access with Logarithmic Regret , 2010, IEEE Journal on Selected Areas in Communications.