Improving reinforcement learning algorithms for dynamic spectrum allocation in cognitive sensor networks

Cognitive Radio Networks enable a higher number of users to access the spectrum of frequency simultaneously. This access is possible due to the implementation of dynamic spectrum allocation algorithms. In this context, one of the main algorithms found in the literature is the reinforcement learning based approach called Q-Learning. Although been widely applied, this algorithm does not take into account accurate information about the behavior of users neither the channel propagation conditions. In this sense, we propose three improvements to the dynamic spectrum allocation algorithms based on reinforcement learning for cognitive sensor networks. Simulation results show that all the proposed algorithms allow allocating channels with up to 6dB better quality and 4% higher efficiency than Q-Learning.

[1]  Joseph Mitola,et al.  Cognitive radio: making software radios more personal , 1999, IEEE Wirel. Commun..

[2]  Dusit Niyato,et al.  A cognitive radio system for e-health applications in a hospital environment , 2010, IEEE Wireless Communications.

[3]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[4]  Cyril Leung,et al.  Resource allocation in an OFDM-based cognitive radio system , 2009, IEEE Transactions on Communications.

[5]  Geoffrey Ye Li,et al.  Probabilistic Resource Allocation for Opportunistic Spectrum Access , 2010, IEEE Transactions on Wireless Communications.

[6]  Carlos Eduardo Pereira,et al.  WirelessHART field devices , 2011, IEEE Instrumentation & Measurement Magazine.

[7]  Shuguang Cui,et al.  Dynamic Resource Allocation in Cognitive Radio Networks , 2010, IEEE Signal Processing Magazine.

[8]  Jianfeng Wang,et al.  Emerging cognitive radio applications: A survey , 2011, IEEE Communications Magazine.

[9]  Chang-Joo Kim,et al.  Channel management in IEEE 802.22 WRAN systems , 2010, IEEE Communications Magazine.

[10]  Theodore S. Rappaport,et al.  Wireless Communications: Principles and Practice (2nd Edition) by , 2012 .

[11]  Zhen Peng,et al.  Cognitive radio spectrum allocation using evolutionary algorithms , 2009, IEEE Transactions on Wireless Communications.

[12]  Theodore S. Rappaport,et al.  Wireless communications - principles and practice , 1996 .

[13]  Dharma P. Agrawal,et al.  A framework for statistical wireless spectrum occupancy modeling , 2010, IEEE Transactions on Wireless Communications.

[14]  David Grace,et al.  Efficient exploration in reinforcement learning-based cognitive radio spectrum sharing , 2011, IET Commun..

[15]  Ian F. Akyildiz,et al.  A survey on spectrum management in cognitive radio networks , 2008, IEEE Communications Magazine.