Coordination Graph-Based Deep Reinforcement Learning for Cooperative Spectrum Sensing Under Correlated Fading

In this letter, cooperative spectrum sensing (CSS) under correlated fading in cognitive radio networks is modeled, and the distributed deep reinforcement learning method is adopted to learn the optimal CSS strategy. To reduce the dimension of the solution space in large networks, this letter takes advantage of the coordination graph to decompose the problem into a max-plus problem and employs the message passing to obtain the optimal strategy sequentially. The simulation results verify the superior performance of the proposed algorithm by comparison with the conventional reinforcement learning implementations.

[1]  K. B. Letaief,et al.  Optimization of cooperative spectrum sensing with energy detection in cognitive radio networks , 2009, IEEE Transactions on Wireless Communications.

[2]  Bhaskar Krishnamachari,et al.  Deep Reinforcement Learning for Dynamic Multichannel Access in Wireless Networks , 2018, IEEE Transactions on Cognitive Communications and Networking.

[3]  Sudharman K. Jayaweera,et al.  Learning-Aided Sub-Band Selection Algorithms for Spectrum Sensing in Wide-Band Cognitive Radios , 2014, IEEE Transactions on Wireless Communications.

[4]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[5]  Jie Li,et al.  Joint Spectrum Sensing and Resource Allocation Scheme in Cognitive Radio Networks with Spectrum Sensing Data Falsification Attack , 2016, IEEE Transactions on Vehicular Technology.

[6]  Yawen Fan,et al.  Message Passing Based Distributed Learning for Joint Resource Allocation in Millimeter Wave Heterogeneous Networks , 2019, IEEE Transactions on Wireless Communications.

[7]  Ian F. Akyildiz,et al.  Reinforcement learning for cooperative sensing gain in cognitive radio ad hoc networks , 2013, Wirel. Networks.

[8]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[9]  Cong Shen,et al.  Adaptive spectrum sensing , 2010, 2010 Information Theory and Applications Workshop (ITA).

[10]  Sheetal Kalyani,et al.  Performance Analysis of Cooperative Spectrum Sensing Over $\kappa{-}\mu$ Shadowed Fading , 2015, IEEE Wireless Communications Letters.

[11]  H. Vincent Poor,et al.  Multiagent Reinforcement Learning Based Spectrum Sensing Policies for Cognitive Radio Networks , 2013, IEEE Journal of Selected Topics in Signal Processing.

[12]  Joseph Mitola,et al.  Cognitive radio: making software radios more personal , 1999, IEEE Wirel. Commun..

[13]  Carlos Guestrin,et al.  Multiagent Planning with Factored MDPs , 2001, NIPS.

[14]  Changyong Pan,et al.  Multi-Agent Deep Reinforcement Learning-Based Cooperative Spectrum Sensing With Upper Confidence Bound Exploration , 2019, IEEE Access.

[15]  Janne J. Lehtomäki,et al.  Efficient Decision Fusion for Cooperative Spectrum Sensing Based on OR-rule , 2012, IEEE Transactions on Wireless Communications.

[16]  Alexandru Martian,et al.  Energy Detection Algorithm for Spectrum Sensing Using Three Consecutive Sensing Events , 2016, IEEE Wireless Communications Letters.