Reinforcement Learning for Traffic-Adaptive Sleep Mode Management in 5G Networks

In mobile networks, base stations (BSs) have the largest share in energy consumption. To reduce BS energy consumption, BS components with similar (de)activation times can be grouped and put into sleep during their times of inactivity. The deeper and the more energy saving a sleep mode (SM) is, the longer (de)activation time it takes to wake up, which incurs a proportional service interruption. Therefore, it is challenging to timely decide on the best SM, bearing in mind the daily traffic fluctuation and imposed service level constraints on delay/dropping. In this study, we leverage an online reinforcement learning technique, i.e., SARSA, and propose an algorithm to decide which SM to choose given time and BS load. We use real mobile traffic obtained from a BS in Stockholm to evaluate the performance of the proposed algorithm. Simulation results show that considerable energy saving can be achieved at the cost of acceptable delay, i.e., wake-up time until we serve users, compared to two lower/upper baselines, namely, fixed (non-adaptive) SMs and optimal non-causal solution.

[1]  A GreenTouch White,et al.  GreenTouch Final Results from Green Meter Research Study Reducing the Net Energy Consumption in Communications Networks by up to 98 % by 2020 , 2015 .

[2]  Tijani Chahed,et al.  Optimal control for base station sleep mode in energy efficient radio access networks , 2011, 2011 Proceedings IEEE INFOCOM.

[3]  Kiyomichi Araki,et al.  Practical evaluation of on-demand smallcell ON/OFF based on traffic model for 5G cellular networks , 2016, WCNC.

[4]  Björn Debaillie,et al.  A Flexible and Future-Proof Power Model for Cellular Base Stations , 2015, 2015 IEEE 81st Vehicular Technology Conference (VTC Spring).

[5]  Zwi Altman,et al.  Traffic-aware Advanced Sleep Modes management in 5G networks , 2019, 2019 IEEE Wireless Communications and Networking Conference (WCNC).

[6]  Cicek Cavdar,et al.  Energy efficient heterogeneous network deployment with cell DTX , 2016, 2016 IEEE International Conference on Communications (ICC).

[7]  Rahim Tafazolli,et al.  Energy-Efficient and Load-Proportional eNodeB for 5G User-Centric Networks: A Multilevel Sleep Strategy Mechanism , 2018, IEEE Vehicular Technology Magazine.

[8]  Cicek Cavdar,et al.  Optimal Processing Allocation to Minimize Energy and Bandwidth Consumption in Hybrid CRAN , 2018, IEEE Transactions on Green Communications and Networking.

[9]  Cicek Cavdar,et al.  5GrEEn: Towards Green 5G mobile networks , 2013, 2013 IEEE 9th International Conference on Wireless and Mobile Computing, Networking and Communications (WiMob).

[10]  Christopher Cox,et al.  An Introduction to LTE: LTE, LTE-Advanced, SAE and 4G Mobile Communications , 2012 .

[11]  Cicek Cavdar,et al.  Joint Functional Splitting and Content Placement for Green Hybrid CRAN , 2019, 2019 IEEE 30th Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC).

[12]  Enrico Del Re,et al.  Energy efficient adaptive cellular network configuration with QoS guarantee , 2015, 2015 IEEE International Conference on Communications (ICC).

[13]  Loutfi Nuaymi,et al.  Location-Aware Sleep Strategy for Energy-Delay Tradeoffs in 5G with Reinforcement Learning , 2019, 2019 IEEE 30th Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC).

[14]  Kiyomichi Araki,et al.  Practical evaluation of on-demand smallcell ON/OFF based on traffic model for 5G cellular networks , 2016, 2016 IEEE Wireless Communications and Networking Conference.

[15]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[16]  Zwi Altman,et al.  Advanced Sleep Modes and Their Impact on Flow-Level Performance of 5G Networks , 2017, 2017 IEEE 86th Vehicular Technology Conference (VTC-Fall).

[17]  Eitan Altman,et al.  Reinforcement Learning Approach for Advanced Sleep Modes Management in 5G Networks , 2018, 2018 IEEE 88th Vehicular Technology Conference (VTC-Fall).

[18]  Loutfi Nuaymi,et al.  Green Mobile Networks for 5G and Beyond , 2019, IEEE Access.