A New Q-Learning Based Spectrum Access Strategy

To optimize the opportunistic spectrum access strategy, a new Q-Learning based spectrum access strategy is proposed. The strategy can lead secondary user to select channels with maximum cumulative reward, and maximize secondary user throughput. From the simulation results, compared with random selection algorithm, the algorithm does not require prior knowledge or prediction models of the channel environment, yet can still select the optimal channel adaptively, improve the secondary user capability and attain to the convergence in short time.

[1]  Brian M. Sadler,et al.  A Survey of Dynamic Spectrum Access , 2007, IEEE Signal Processing Magazine.

[2]  Mingyan Liu,et al.  Optimality of Myopic Sensing in Multi-Channel Opportunistic Access , 2008, 2008 IEEE International Conference on Communications.

[3]  Q. Zhao,et al.  Decentralized cognitive mac for dynamic spectrum access , 2005, First IEEE International Symposium on New Frontiers in Dynamic Spectrum Access Networks, 2005. DySPAN 2005..

[4]  Ananthram Swami,et al.  Joint Design and Separation Principle for Opportunistic Spectrum Access in the Presence of Sensing Errors , 2007, IEEE Transactions on Information Theory.

[5]  Ananthram Swami,et al.  Distributed Spectrum Sensing and Access in Cognitive Radio Networks With Energy Constraint , 2009, IEEE Transactions on Signal Processing.

[6]  Abhijit Gosavi,et al.  Reinforcement Learning: A Tutorial Survey and Recent Advances , 2009, INFORMS J. Comput..

[7]  K. J. Ray Liu,et al.  Primary-prioritized Markov approach for dynamic spectrum allocation , 2009, IEEE Transactions on Wireless Communications.

[8]  Simon Haykin,et al.  Cognitive radio: brain-empowered wireless communications , 2005, IEEE Journal on Selected Areas in Communications.

[9]  Alagan Anpalagan,et al.  Opportunistic Spectrum Access in Unknown Dynamic Environment: A Game-Theoretic Stochastic Learning Solution , 2012, IEEE Transactions on Wireless Communications.

[10]  Alex Weissensteiner,et al.  A $Q$ -Learning Approach to Derive Optimal Consumption and Investment Strategies , 2008, IEEE Transactions on Neural Networks.