Reinforcement Learning for Delay-Constrained Energy-Aware Small Cells with Multi-Sleeping Control

In 5G networks, specific requirements are defined on the periodicity of Synchronization Signaling (SS) bursts. This imposes a constraint on the maximum period a Base Station (BS) can be deactivated. On the other hand, BS densification is expected in 5G architecture. This will cause a drastic increase in the network energy consumption followed by a complex interference management. In this paper, we study the Energy-Delay-Tradeoff (EDT) problem in a Heterogeneous Network (HetNet) where small cells can switch to different sleep mode levels to save energy while maintaining a good Quality of Service (QoS). We propose a distributed Q-learning algorithm controller for small cells that adapts the cell activity while taking into account the co-channel interference between the cells. Our numerical results show that multi-level sleep scheme outperforms binary sleep scheme with an energy saving up to 80% in the case when the users are delay tolerant, and while respecting the periodicity of the SS bursts in 5G.

[1]  Leandros Tassiulas,et al.  Dynamic Resource Provisioning for Energy Efficiency in Wireless Access Networks: A Survey and an Outlook , 2014, IEEE Communications Surveys & Tutorials.

[2]  Eduardo F. Morales,et al.  An Introduction to Reinforcement Learning , 2011 .

[3]  Björn Debaillie,et al.  A Flexible and Future-Proof Power Model for Cellular Base Stations , 2015, 2015 IEEE 81st Vehicular Technology Conference (VTC Spring).

[4]  Loutfi Nuaymi,et al.  Location-Aware Sleep Strategy for Energy-Delay Tradeoffs in 5G with Reinforcement Learning , 2019, 2019 IEEE 30th Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC).

[5]  Moshe Zukerman,et al.  Energy-Efficient Base-Stations Sleep-Mode Techniques in Green Cellular Networks: A Survey , 2015, IEEE Communications Surveys & Tutorials.

[6]  Loutfi Nuaymi,et al.  A Distributed Q-Learning Approach for Adaptive Sleep Modes in 5G Networks , 2019, 2019 IEEE Wireless Communications and Networking Conference (WCNC).

[7]  Panu Lähdekorpi,et al.  Energy efficiency of 5G mobile networks with base station sleep modes , 2017, 2017 IEEE Conference on Standards for Communications and Networking (CSCN).

[8]  Jinsong Wu,et al.  Survey of Strategies for Switching Off Base Stations in Heterogeneous Networks for Greener 5G Systems , 2016, IEEE Access.

[9]  Balasubramaniam Natarajan,et al.  Small Cell Base Station Sleep Strategies for Energy Efficiency , 2016, IEEE Transactions on Vehicular Technology.

[10]  Tao Jiang,et al.  Base Station ON-OFF Switching in 5G Wireless Networks: Approaches and Challenges , 2017, IEEE Wireless Communications.

[11]  Loutfi Nuaymi,et al.  Green Mobile Networks for 5G and Beyond , 2019, IEEE Access.

[12]  Dimitri Ktenas,et al.  Reinforcement learning for interference-aware cell DTX in heterogeneous networks , 2018, 2018 IEEE Wireless Communications and Networking Conference (WCNC).

[13]  Eitan Altman,et al.  Reinforcement Learning Approach for Advanced Sleep Modes Management in 5G Networks , 2018, 2018 IEEE 88th Vehicular Technology Conference (VTC-Fall).