An Application of Reinforcement Learning for Efficient Spectrum Usage in Next-Generation Mobile Cellular Networks

This paper proposes reinforcement learning as a foundational stone of a framework for efficient spectrum usage in the context of next-generation mobile cellular networks. The objective of the framework is to efficiently use the spectrum in a cellular orthogonal frequency-division multiple access network while unnecessary spectrum is released for secondary spectrum usage within a private commons spectrum access model. Numerical results show that the proposed framework obtains the best performance compared with other approaches for spectrum assignment. Moreover, the framework is relatively simple to implement in terms of computational requirements and signaling overhead.

[1]  Oriol Sallent,et al.  A Novel Framework for Dynamic Spectrum Management in MultiCell OFDMA Networks Based on Reinforcement Learning , 2009, 2009 IEEE Wireless Communications and Networking Conference.

[2]  Oriol Sallent,et al.  A Self-Organized Spectrum Assignment Strategy in Next Generation OFDMA Networks Providing Secondary Spectrum Access , 2009, 2009 IEEE International Conference on Communications.

[3]  Zhaocheng Wang,et al.  Frequency reuse scheme for cellular OFDM systems , 2002 .

[4]  Jie Zhang,et al.  Dynamic Frequency Planning Versus Frequency Reuse Schemes in OFDMA Networks , 2009, VTC Spring 2009 - IEEE 69th Vehicular Technology Conference.

[5]  M.M. Buddhikot,et al.  Understanding Dynamic Spectrum Access: Models,Taxonomy and Challenges , 2007, 2007 2nd IEEE International Symposium on New Frontiers in Dynamic Spectrum Access Networks.

[6]  Dimitri P. Bertsekas,et al.  Reinforcement Learning for Dynamic Channel Allocation in Cellular Telephone Systems , 1996, NIPS.

[7]  Bernhard Walke,et al.  MAC Performance of a 3GPP-LTE Multihop Cellular Network , 2008, 2008 IEEE International Conference on Communications.

[8]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[9]  S. Haykin,et al.  A Q-learning-based dynamic channel assignment technique for mobile communication systems , 1999 .

[10]  Theodore S. Rappaport,et al.  Wireless communications - principles and practice , 1996 .

[11]  Oriol Sallent,et al.  Dynamic spectrum assignment in multicell OFDMA networks enabling a secondary spectrum usage , 2009, Wirel. Commun. Mob. Comput..

[12]  Saewoong Bahk,et al.  Cell-Throughput Analysis of the Proportional Fair Scheduler in the Single-Cell Environment , 2007, IEEE Transactions on Vehicular Technology.

[13]  Oriol Sallent,et al.  Temporal and Spatial Spectrum Assignment in Next Generation OFDMA Networks through Reinforcement Learning , 2009, VTC Spring 2009 - IEEE 69th Vehicular Technology Conference.

[14]  Chen-Khong Tham,et al.  Adaptive provisioning of differentiated services networks based on reinforcement learning , 2003, IEEE Trans. Syst. Man Cybern. Part C.

[15]  Mandayam A. L. Thathachar,et al.  Local and Global Optimization Algorithms for Generalized Learning Automata , 1995, Neural Computation.

[16]  Alberto Leon-Garcia,et al.  Market-Based Resource Management for Cognitive Radios Using Machine Learning , 2007, IEEE GLOBECOM 2007 - IEEE Global Telecommunications Conference.

[17]  Oriol Sallent,et al.  Fuzzy Neural Control for Economic-Driven Radio Resource Management in Beyond 3G Networks , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[18]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[19]  Witold Pedrycz,et al.  Optimizing QoS routing in hierarchical ATM networks using computational intelligence techniques , 2003, IEEE Trans. Syst. Man Cybern. Part C.

[20]  T. Ottosson,et al.  Attaining both coverage and high spectral efficiency with adaptive OFDM downlinks , 2003, 2003 IEEE 58th Vehicular Technology Conference. VTC 2003-Fall (IEEE Cat. No.03CH37484).

[21]  Chung-Ju Chang,et al.  Q-learning-based multirate transmission control scheme for RRM in multimedia WCDMA systems , 2004, IEEE Transactions on Vehicular Technology.

[22]  Fangwen Fu,et al.  Detection of Spectral Resources in Cognitive Radios Using Reinforcement Learning , 2008, 2008 3rd IEEE Symposium on New Frontiers in Dynamic Spectrum Access Networks.

[23]  Oriol Sallent,et al.  Dynamic spectrum assignment in multicell OFDMA networks enabling a secondary spectrum usage , 2009 .

[24]  Alexander Golitschek Edler von Elbwart,et al.  Fairness and throughput analysis for generalized proportional fair frequency scheduling in OFDMA , 2005, 2005 IEEE 61st Vehicular Technology Conference.