RAN Resource Slicing in 5G Using Multi-Agent Correlated Q-Learning

5G is regarded as a revolutionary mobile network, which is expected to satisfy a vast number of novel services, ranging from remote health care to smart cities. However, heterogeneous Quality of Service (QoS) requirements of different services and limited spectrum make the radio resource allocation a challenging problem in 5G. In this paper, we propose a multi-agent reinforcement learning (MARL) method for radio resource slicing in 5G. We model each slice as an intelligent agent that competes for limited radio resources, and the correlated Q-learning is applied for inter-slice resource block (RB) allocation. The proposed correlated Q-learning based inter-slice RB allocation (COQRA) scheme is compared with Nash Q-learning (NQL), Latency-Reliability-Throughput Q-learning (LRTQ) methods, and the priority proportional fairness (PPF) algorithm. Our simulation results show that the proposed CO-QRA achieves 32.4% lower latency and 6.3% higher throughput when compared with LRTQ, and 5.8% lower latency and 5.9% higher throughput than NQL. Significantly higher throughput and lower packet drop rate (PDR) is observed in comparison to PPF.

[1]  Hao Zhou,et al.  Correlated Deep Q-learning based Microgrid Energy Management , 2020, 2020 IEEE 25th International Workshop on Computer Aided Modeling and Design of Communication Links and Networks (CAMAD).

[2]  Yi Shi,et al.  Reinforcement Learning for Dynamic Resource Optimization in 5G Radio Access Network Slicing , 2020, 2020 IEEE 25th International Workshop on Computer Aided Modeling and Design of Communication Links and Networks (CAMAD).

[3]  Christos Verikoukis,et al.  Offline SLA-Constrained Deep Learning for 5G Networks Reliable and Dynamic End-to-End Slicing , 2020, IEEE Journal on Selected Areas in Communications.

[4]  Klaus I. Pedersen,et al.  Multi-User Preemptive Scheduling For Critical Low Latency Communications in 5G Networks , 2018, 2018 IEEE Symposium on Computers and Communications (ISCC).

[5]  Melike Erol-Kantarci,et al.  AI-Enabled Future Wireless Networks: Challenges, Opportunities, and Open Issues , 2019, IEEE Vehicular Technology Magazine.

[6]  Luis Alonso,et al.  Continuous Multi-objective Zero-touch Network Slicing via Twin Delayed DDPG and OpenAI Gym , 2020, GLOBECOM 2020 - 2020 IEEE Global Communications Conference.

[7]  Olabisi Emmanuel Falowo,et al.  Latency-Aware Dynamic Resource Allocation Scheme for Multi-Tier 5G Network: A Network Slicing-Multitenancy Scenario , 2020, IEEE Access.

[8]  Melike Erol-Kantarci,et al.  AI-Enabled Radio Resource Allocation in 5G for URLLC and eMBB Users , 2019, 2019 IEEE 2nd 5G World Forum (5GWF).

[9]  Yan Huang,et al.  A Deep-Reinforcement-Learning-Based Approach to Dynamic eMBB/URLLC Multiplexing in 5G NR , 2020, IEEE Internet of Things Journal.

[10]  Navid Nikaein,et al.  Slice Scheduling with QoS-Guarantee Towards 5G , 2019, 2019 IEEE Global Communications Conference (GLOBECOM).

[11]  Choong Seon Hong,et al.  A Downlink Resource Scheduling Strategy for URLLC Traffic , 2019, 2019 IEEE International Conference on Big Data and Smart Computing (BigComp).

[12]  Klaus I. Pedersen,et al.  Joint Link Adaptation and Scheduling for 5G Ultra-Reliable Low-Latency Communications , 2018, IEEE Access.

[13]  Tarik Taleb,et al.  A Novel QoS Framework for Network Slicing in 5G and Beyond Networks Based on SDN and NFV , 2020, IEEE Network.

[14]  Sijing Zhang,et al.  Towards 5G: A Reinforcement Learning-Based Scheduling Solution for Data Traffic Management , 2018, IEEE Transactions on Network and Service Management.

[15]  Fredrik Tufvesson,et al.  5G: A Tutorial Overview of Standards, Trials, Challenges, Deployment, and Practice , 2017, IEEE Journal on Selected Areas in Communications.

[16]  Mehdi Bennis,et al.  eMBB-URLLC Resource Slicing: A Risk-Sensitive Approach , 2019, IEEE Communications Letters.

[17]  Xiaofeng Tao,et al.  Machine Learning Based Flexible Transmission Time Interval Scheduling for eMBB and uRLLC Coexistence Scenario , 2019, IEEE Access.

[18]  Hao Zhou,et al.  Decentralized Microgrid Energy Management: A Multi-agent Correlated Q-learning Approach , 2020, 2020 IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids (SmartGridComm).

[19]  Xiaorong Zhu,et al.  An End-to-End Network Slicing Algorithm Based on Deep Q-Learning for 5G Network , 2020, IEEE Access.

[20]  Prodromos-Vasileios Mekikis,et al.  Dynamic partitioning of radio resources based on 5G RAN Slicing , 2020, GLOBECOM 2020 - 2020 IEEE Global Communications Conference.