On using reinforcement learning for network slice admission control in 5G: Offline vs. online

Achieving a fair usage of network resources is of vital importance in Slice-ready 5G network. The dilemma of which network slice to accept or to reject is very challenging for the Infrastructure Provider (InfProv). On one hand, InfProv aims to maximize the network resources usage by accepting as many network slices as possible; on the other hand, the network resources are limited, and the network slice requirements regarding Quality of Service (QoS) need to be fulfilled. In this paper, we devise three admission control mechanisms based on Reinforcement Learning, namely Q-Learning, Deep Q-Learning, and Regret Matching, which allow deriving admission control decisions (policy) to be applied by InfProv to admit or reject network slice requests. We evaluated the three algorithms using computer simulation, showing results on each mechanism’s performance in terms of maximizing the InfProv revenue and their ability to learn offline or online.

[1]  Gustavo de Veciana,et al.  Joint Scheduling of URLLC and eMBB Traffic in 5G Wireless Networks , 2017, IEEE INFOCOM 2018 - IEEE Conference on Computer Communications.

[2]  Marco Gramaglia,et al.  Optimising 5G infrastructure markets: The business of network slicing , 2017, IEEE INFOCOM 2017 - IEEE Conference on Computer Communications.

[3]  Pantelis A. Frangoudis,et al.  Dynamic Slicing of RAN Resources for Heterogeneous Coexisting 5G Services , 2019, 2019 IEEE Global Communications Conference (GLOBECOM).

[4]  Gang Feng,et al.  Intelligent Resource Scheduling for 5G Radio Access Network Slicing , 2019, IEEE Transactions on Vehicular Technology.

[5]  Navid Nikaein,et al.  Providing Low Latency Guarantees for Slicing-Ready 5G Systems via Two-Level MAC Scheduling , 2018, IEEE Network.

[6]  Andres Garcia-Saavedra,et al.  Overbooking network slices through yield-driven end-to-end orchestration , 2018, CoNEXT.

[7]  Navrati Saxena,et al.  Next Generation 5G Wireless Networks: A Comprehensive Survey , 2016, IEEE Communications Surveys & Tutorials.

[8]  Jiming Chen,et al.  Regret Matching Based Channel Assignment for Wireless Sensor Networks , 2010, 2010 IEEE International Conference on Communications.

[9]  Yasin Yilmaz,et al.  Reinforcement Learning for Adaptive Resource Allocation in Fog RAN for IoT With Heterogeneous Latency Requirements , 2019, IEEE Access.

[10]  Ying Chen,et al.  Real-Time Resource Slicing for 5G RAN via Deep Reinforcement Learning , 2019, 2019 IEEE 25th International Conference on Parallel and Distributed Systems (ICPADS).

[11]  Navid Nikaein,et al.  Towards enforcing Network Slicing on RAN: Flexibility and Resources abstraction , 2017 .

[12]  Wang Qiang,et al.  Reinforcement learning model, algorithms and its application , 2011, 2011 International Conference on Mechatronic Science, Electric Engineering and Computer (MEC).

[13]  Cheng-Chew Lim,et al.  Online Versus Offline Reinforcement Learning for False Target Control Against Known Threat , 2018, ICIRA.

[14]  Abhishek Verma,et al.  Comparison of Deep Reinforcement Learning Approaches for Intelligent Game Playing , 2019, 2019 IEEE 9th Annual Computing and Communication Workshop and Conference (CCWC).

[15]  K. J. Ray Liu,et al.  Game theoretic Markov decision processes for optimal decision making in social systems , 2014, 2014 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[16]  Punit Pandey,et al.  Approximate Q-Learning: An Introduction , 2010, 2010 Second International Conference on Machine Learning and Computing.

[17]  Lena Wosinska,et al.  Reinforcement Learning for Slicing in a 5G Flexible RAN , 2019, Journal of Lightwave Technology.

[18]  Elena Lopez-Aguilera,et al.  Maximizing Infrastructure Providers’ Revenue Through Network Slicing in 5G , 2019, IEEE Access.