A Distributed Q-Learning Approach for Adaptive Sleep Modes in 5G Networks

In 5G networks, specific requirements are defined on the periodicity of Synchronization Signaling (SS) bursts. This imposes a constraint on the maximum period a Base Station (BS) can be deactivated. On the other hand, BS densification is expected in 5G architecture. This will lead to an energy crunch if kept ignored. In this paper, we propose a distributed algorithm based on Reinforcement Learning (RL) that controls the states of the BSs while respecting the requirements of 5G. By considering different levels of Sleep Modes (SMs), the algorithm chooses how deep a BS can sleep according to the best switch-off SM level policy that maximizes the trade-off between energy savings and system delay. The latter is calculated based on the wake-up time required by the different SM levels. Results show that our algorithm performs better than the case of using only one type of SM. Furthermore, our simulations show a gain in energy savings up to 90% when the users are delay tolerant while respecting the periodicity of the SS bursts in 5G.

[1]  Zhisheng Niu,et al.  Delay-Constrained Energy-Optimal Base Station Sleeping Control , 2016, IEEE Journal on Selected Areas in Communications.

[2]  David Grace,et al.  Transfer learning for QoS aware topology management in energy efficient 5G cognitive radio networks , 2014, 1st International Conference on 5G for Ubiquitous Connectivity.

[3]  Balasubramaniam Natarajan,et al.  Small Cell Base Station Sleep Strategies for Energy Efficiency , 2016, IEEE Transactions on Vehicular Technology.

[4]  Erol Gelenbe,et al.  The impact of information technology on energy consumption and carbon emissions , 2015, UBIQ.

[5]  Moshe Zukerman,et al.  Energy-Efficient Base-Stations Sleep-Mode Techniques in Green Cellular Networks: A Survey , 2015, IEEE Communications Surveys & Tutorials.

[6]  Loutfi Nuaymi,et al.  An overview and classification of research approaches in green wireless networks , 2012, EURASIP J. Wirel. Commun. Netw..

[7]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[8]  Björn Debaillie,et al.  A Flexible and Future-Proof Power Model for Cellular Base Stations , 2015, 2015 IEEE 81st Vehicular Technology Conference (VTC Spring).

[9]  Eitan Altman,et al.  Reinforcement Learning Approach for Advanced Sleep Modes Management in 5G Networks , 2018, 2018 IEEE 88th Vehicular Technology Conference (VTC-Fall).

[10]  Tiejun Lv,et al.  Deep Q-Learning Based Dynamic Resource Allocation for Self-Powered Ultra-Dense Networks , 2018, 2018 IEEE International Conference on Communications Workshops (ICC Workshops).

[11]  Peilin Hong,et al.  Stochastic Analysis of Optimal Base Station Energy Saving in Cellular Networks with Sleep Mode , 2014, IEEE Communications Letters.

[12]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Rui Zhang,et al.  Dynamic Base Station Operation in Large-Scale Green Cellular Networks , 2015, IEEE Journal on Selected Areas in Communications.

[14]  Zhu Han,et al.  Machine Learning Paradigms for Next-Generation Wireless Networks , 2017, IEEE Wireless Communications.

[15]  Panu Lähdekorpi,et al.  Energy efficiency of 5G mobile networks with base station sleep modes , 2017, 2017 IEEE Conference on Standards for Communications and Networking (CSCN).

[16]  Tao Jiang,et al.  BOOST: Base station ON-OFF switching strategy for energy efficient massive MIMO HetNets , 2016, IEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications.

[17]  Marco Miozzo,et al.  Distributed Q-learning for energy harvesting Heterogeneous Networks , 2015, 2015 IEEE International Conference on Communication Workshop (ICCW).

[18]  Tao Jiang,et al.  Base Station ON-OFF Switching in 5G Wireless Networks: Approaches and Challenges , 2017, IEEE Wireless Communications.

[19]  Zwi Altman,et al.  Advanced Sleep Modes and Their Impact on Flow-Level Performance of 5G Networks , 2017, 2017 IEEE 86th Vehicular Technology Conference (VTC-Fall).