Data-driven dynamic resource scheduling for network slicing: A Deep reinforcement learning approach

Abstract Network slicing is designed to support a variety of emerging applications with diverse performance and flexibility requirements, by dividing the physical network into multiple logical networks. These applications along with a massive number of mobile phones produce large amounts of data, bringing tremendous challenges for network slicing performance. From another perspective, this huge amount of data also offers a new opportunity for the management of network slicing resources. Leveraging the knowledge and insights retrieved from the data, we develop a novel Machine Learning-based scheme for dynamic resource scheduling for networks slicing, aiming to achieve automatic and efficient resource optimisation and End-to-End (E2E) service reliability. However, it is difficult to obtain the user-related data, which is crucial to understand the user behaviour and requests, due to the privacy issue. Therefore, Deep Reinforcement Learning (DRL) is leveraged to extract knowledge from experience by interacting with the network and enable dynamic adjustment of the resources allocated to various slices in order to maximise the resource utilisation while guaranteeing the Quality-of-Service (QoS). The experiment results demonstrate that the proposed resource scheduling scheme can dynamically allocate resources for multiple slices and meet the corresponding QoS requirements.

[1]  Yuval Tassa,et al.  Continuous control with deep reinforcement learning , 2015, ICLR.

[2]  Toktam Mahmoodi,et al.  Network slicing management & prioritization in 5G mobile systems , 2016 .

[3]  Albert Y. Zomaya,et al.  Big Data and Computational Intelligence in Networking , 2017 .

[4]  Matthias Rost,et al.  Service Chain and Virtual Network Embeddings: Approximations using Randomized Rounding , 2016, ArXiv.

[5]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[6]  Marc Peter Deisenroth,et al.  Deep Reinforcement Learning: A Brief Survey , 2017, IEEE Signal Processing Magazine.

[7]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[8]  Zhixiang Liu,et al.  Service Function Chaining Resource Allocation: A Survey , 2016, ArXiv.

[9]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[10]  Keke Gai,et al.  Smart Resource Allocation Using Reinforcement Learning in Content-Centric Cyber-Physical Systems , 2017, SmartCom.

[11]  Gustavo de Veciana,et al.  Network slicing games: Enabling customization in multi-tenant networks , 2016, IEEE INFOCOM 2017 - IEEE Conference on Computer Communications.

[12]  Mahesh K. Marina,et al.  Network Slicing in 5G: Survey and Challenges , 2017, IEEE Communications Magazine.

[13]  Geyong Min,et al.  Data-Driven Information Plane in Software-Defined Networking , 2017, IEEE Communications Magazine.

[14]  John Salvatier,et al.  Theano: A Python framework for fast computation of mathematical expressions , 2016, ArXiv.

[15]  Tarik Taleb,et al.  Content delivery network slicing: QoE and cost awareness , 2017, 2017 IEEE International Conference on Communications (ICC).

[16]  Filip De Turck,et al.  Design and evaluation of learning algorithms for dynamic resource management in virtual networks , 2014, 2014 IEEE Network Operations and Management Symposium (NOMS).

[17]  Bin Han,et al.  Network Slicing to Enable Scalability and Flexibility in 5G Mobile Networks , 2017, IEEE Communications Magazine.

[18]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[19]  Roberto Riggio,et al.  Virtual network functions orchestration in enterprise WLANs , 2015, 2015 IFIP/IEEE International Symposium on Integrated Network Management (IM).

[20]  Albert Y. Zomaya,et al.  Network Function Virtualization in Dynamic Networks: A Stochastic Perspective , 2018, IEEE Journal on Selected Areas in Communications.

[21]  Hussein A. Abbass,et al.  Hierarchical Deep Reinforcement Learning for Continuous Action Control , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[22]  Sebastian Ruder,et al.  An overview of gradient descent optimization algorithms , 2016, Vestnik komp'iuternykh i informatsionnykh tekhnologii.

[23]  Kao-Shing Hwang,et al.  An adaptive decision-making method with fuzzy Bayesian reinforcement learning for robot soccer , 2018, Inf. Sci..

[24]  Meikang Qiu,et al.  Reinforcement Learning for Cyber-Physical Systems , 2019, 2019 IEEE International Conference on Industrial Internet (ICII).

[25]  Lazaros Gkatzikis,et al.  The Algorithmic Aspects of Network Slicing , 2017, IEEE Communications Magazine.

[26]  Tarik Taleb,et al.  Network Slicing and Softwarization: A Survey on Principles, Enabling Technologies, and Solutions , 2018, IEEE Communications Surveys & Tutorials.

[27]  Yuan Zuo,et al.  Learning-based network path planning for traffic engineering , 2019, Future Gener. Comput. Syst..

[28]  Keke Gai,et al.  Optimal resource allocation using reinforcement learning for IoT content-centric services , 2018, Appl. Soft Comput..

[29]  Meikang Qiu,et al.  Reinforcement Learning-based Content-Centric Services in Mobile Sensing , 2018, IEEE Network.

[30]  Nadra Guizani,et al.  Recent Advances and Challenges in Mobile Big Data , 2018, IEEE Communications Magazine.

[31]  Tarik Taleb,et al.  End-to-end Network Slicing for 5G Mobile Networks , 2017, J. Inf. Process..

[32]  Jose Ordonez-Lucena,et al.  Network Slicing for 5G with SDN/NFV: Concepts, Architectures, and Challenges , 2017, IEEE Communications Magazine.

[33]  Lin Wang,et al.  Machine learning based mobile malware detection using highly imbalanced network traffic , 2017, Inf. Sci..

[34]  Matias Richart,et al.  Resource Slicing in Virtual Wireless Networks: A Survey , 2016, IEEE Transactions on Network and Service Management.

[35]  Geyong Min,et al.  Stochastic Performance Analysis of Network Function Virtualization in Future Internet , 2019, IEEE Journal on Selected Areas in Communications.

[36]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.