5MART: A 5G SMART Scheduling Framework for Optimizing QoS Through Reinforcement Learning

The massive growth in mobile data traffic and the heterogeneity and stringency of Quality of Service (QoS) requirements of various applications have put significant pressure on the underlying network infrastructure and represent an important challenge even for the very anticipated 5G networks. In this context, the solution is to employ smart Radio Resource Management (RRM) in general and innovative packet scheduling in particular in order to offer high flexibility and cope with both current and upcoming QoS challenges. Given the increasing demand for bandwidth-hungry applications, conventional scheduling strategies face significant problems in meeting the heterogeneous QoS requirements of various application classes under dynamic network conditions. This paper proposes 5MART, a 5G smart scheduling framework that manages the QoS provisioning for heterogeneous traffic. Reinforcement learning and neural networks are jointly used to find the most suitable scheduling decisions based on current networking conditions. Simulation results show that the proposed 5MART framework can achieve up to 50% improvement in terms of time fraction (in sub-frames) when the heterogeneous QoS constraints are met with respect to other state-of-the-art scheduling solutions.

[1]  Ioan-Sorin Comsa.  Sustainable scheduling policies for radio access networks based on LTE technology , 2014 .

[2]  Ashwin Sampath,et al.  Downlink Scheduling for Multiclass Traffic in LTE , 2009, EURASIP J. Wirel. Commun. Netw..

[3]  Geoffrey Ye Li,et al.  Utility-based resource allocation and scheduling in OFDM-based wireless broadband networks , 2005, IEEE Commun. Mag..

[4]  Sijing Zhang,et al.  Towards 5G: A Reinforcement Learning-Based Scheduling Solution for Data Traffic Management , 2018, IEEE Transactions on Network and Service Management.

[5]  Giuseppe Piro,et al.  Two-Level Downlink Scheduling for Real-Time Multimedia Services in LTE Networks , 2011, IEEE Transactions on Multimedia.

[6]  Giuseppe Piro,et al.  Simulating LTE Cellular Systems: An Open-Source Framework , 2011, IEEE Transactions on Vehicular Technology.

[7]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[8]  Jalel Ben-Othman,et al.  An enhanced two level scheduler to increase multimedia services performance in LTE networks , 2014, 2014 IEEE International Conference on Communications (ICC).

[9]  Jeroen Wigard,et al.  Dynamic Packet Scheduling for Traffic Mixes of Best Effort and VoIP Users in E-UTRAN Downlink , 2010, 2010 IEEE 71st Vehicular Technology Conference.

[10]  Frida Eng,et al.  Streaming applications over HSDPA in mixed service scenarios , 2004, IEEE 60th Vehicular Technology Conference, 2004. VTC2004-Fall. 2004.

[11]  Chung-Ju Chang,et al.  An Intelligent Priority Resource Allocation Scheme for LTE-A Downlink Systems , 2012, IEEE Wireless Communications Letters.

[12]  Andrea Zanella,et al.  Scheduling Policies in Time and Frequency Domains for LTE Downlink Channel: A Performance Comparison , 2017, IEEE Transactions on Vehicular Technology.

[13]  Dimitri Ktenas,et al.  QoS-Driven Scheduling in 5G Radio Access Networks - A Reinforcement Learning Approach , 2017, GLOBECOM 2017 - 2017 IEEE Global Communications Conference.

[14]  Ramona Trestian,et al.  Seamless Multimedia Delivery Within a Heterogeneous Wireless Networks Environment: Are We There Yet? , 2018, IEEE Communications Surveys & Tutorials.

[15]  Jeffrey G. Andrews,et al.  What Will 5G Be? , 2014, IEEE Journal on Selected Areas in Communications.

[16]  Gunther Auer,et al.  Opportunistic packet loss fair scheduling for delay-sensitive applications over LTE systems , 2012, 2012 IEEE Wireless Communications and Networking Conference (WCNC).

[17]  Marco Wiering,et al.  Using continuous action spaces to solve discrete problems , 2009, 2009 International Joint Conference on Neural Networks.

[18]  Jianping Chen,et al.  Adaptive proportional fair parameterization based LTE scheduling using continuous actor-critic reinforcement learning , 2014, 2014 IEEE Global Communications Conference.

[19]  A. Jalali,et al.  Data throughput of CDMA-HDR a high efficiency-high data rate personal communication wireless system , 2000, VTC2000-Spring. 2000 IEEE 51st Vehicular Technology Conference Proceedings (Cat. No.00CH37026).

[20]  Gabriel-Miro Muntean,et al.  A Traffic Type-Based Differentiated Reputation Algorithm for Radio Resource Allocation During Multi-Service Content Delivery in 5G Heterogeneous Scenarios , 2019, IEEE Access.

[21]  Hado Philip van Hasselt,et al.  Insights in reinforcement rearning : formal analysis and empirical evaluation of temporal-difference learning algorithms , 2011 .

[22]  Hong Ji,et al.  Heterogeneous traffic scheduling in downlink high speed railway LTE systems , 2013, 2013 IEEE Global Communications Conference (GLOBECOM).

[23]  王家志.  Technical Specification Group Services and System Aspects ; 3 G Security ; Specification of the MILENAGE Algorithm Set : An example algorithm set for the 3 GPP authentication and key generation functions , 2001 .

[24]  Nicola Baldo,et al.  A new channel and QoS aware scheduler to enhance the capacity of voice over LTE systems , 2014, 2014 IEEE 11th International Multi-Conference on Systems, Signals & Devices (SSD14).