Reinforcement learning based distributed multiagent sensing policy for cognitive radio networks

In this paper a distributed multiagent, multiband reinforcement learning based sensing policy for cognitive radio ad hoc networks is proposed. The proposed sensing policy employs secondary user (SU) collaboration through local interactions. The goal is to maximize the amount of available spectrum found for secondary use given a desired diversity order, i.e. a desired number of SUs sensing simultaneously each frequency band. The SUs in the cognitive radio network make local decisions based on their own and their neighbors' local test statistics or decisions to identify unused spectrum locally. Thus, the network builds a locally available map of spectrum occupancy of its geographical area. Simulation results show that the proposed sensing policy provides a significant increase in the amount of available spectrum found for secondary use compared to a random sensing policy.

[1]  Visa Koivunen,et al.  Reinforcement learning method for energy efficient cooperative multiband spectrum sensing , 2010, 2010 IEEE International Workshop on Machine Learning for Signal Processing.

[2]  Kaushik R. Chowdhury,et al.  A survey on MAC protocols for cognitive radio networks , 2009, Ad Hoc Networks.

[3]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[4]  Husheng Li.  Multiagent Q-Learning for Aloha-Like Spectrum Access in Cognitive Radio Systems , 2010, EURASIP J. Wirel. Commun. Netw..

[5]  Luciano Bononi,et al.  To Sense or to Transmit: A Learning-Based Spectrum Management Scheme for Cognitive Radiomesh Networks , 2010, 2010 Fifth IEEE Workshop on Wireless Mesh Networks.

[6]  Fangwen Fu,et al.  Detection of Spectral Resources in Cognitive Radios Using Reinforcement Learning , 2008, 2008 3rd IEEE Symposium on New Frontiers in Dynamic Spectrum Access Networks.

[7]  Mohamed-Slim Alouini,et al.  On the Energy Detection of Unknown Signals Over Fading Channels , 2007, IEEE Transactions on Communications.

[8]  Nikos A. Vlassis,et al.  Collaborative Multiagent Reinforcement Learning by Payoff Propagation , 2006, J. Mach. Learn. Res..

[9]  Ian F. Akyildiz,et al.  CRAHNs: Cognitive radio ad hoc networks , 2009, Ad Hoc Networks.

[10]  Husheng Li,et al.  Learning the Spectrum via Collaborative Filtering in Cognitive Radio Networks , 2010, 2010 IEEE Symposium on New Frontiers in Dynamic Spectrum (DySPAN).

[11]  J. I. Mararm,et al.  Energy Detection of Unknown Deterministic Signals , 2022 .

[12]  Bart De Schutter,et al.  A Comprehensive Survey of Multiagent Reinforcement Learning , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[13]  Kok-Lim Alvin Yau,et al.  Achieving Efficient and Optimal Joint Action in Distributed Cognitive Radio Networks Using Payoff Propagation , 2010, 2010 IEEE International Conference on Communications.