Policy Controlled Multi-domain cloud-network Slice Orchestration Strategy based on Reinforcement Learning

The concept of network slicing plays a thriving role as 5G rolls out business models vouched by different stakeholders. The dynamic and variable characterization of end-to-end cloud-network slices encompasses the composition of different slice parts laying at different administrative domains. Following a profit-maximizing Slice-as-a-Service (SaaS) model, such a multi-domain facet offers promising business opportunities in support of diverse vertical industries, rendering to network slicing marketplace members the roles of Infrastructure Provider, Slice Provider, and Tenants. The effective realization of SaaS approaches introduces a dynamic resource allocation problem, manifested as challenging run-time decisions upon on-demand slice part requests. The Orchestrator is hence responsible to perform an optimized decision on-the-fly on which elasticity requests to address based on an orchestration policy defined within the context of Network Slice architecture for the followed revenue model. This paper presents a slice management strategy for such an orchestrator can follow, based on reinforcement learning, able to efficiently orchestrate slice elasticity requests to comprehend the maximum revenue for the stakeholders of end-to-end network slice lifecycle. The proposed strategy orients a Slice Orchestrator to learn which slice requests to address as per availability of the required resources at the different participating Infrastructure Providers. The experimental results show the Reinforcement Learning based Orchestrator outperforms several benchmark heuristics focused on revenue maximization.

[1]  Eduardo F. Morales,et al.  An Introduction to Reinforcement Learning , 2011 .

[2]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[3]  Min Huang,et al.  Deep learning–based network application classification for SDN , 2018, Trans. Emerg. Telecommun. Technol..

[4]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[5]  Yishay Mansour,et al.  Policy Gradient Methods for Reinforcement Learning with Function Approximation , 1999, NIPS.

[6]  Yoshua Bengio,et al.  How transferable are features in deep neural networks? , 2014, NIPS.

[7]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[8]  Tom Schaul,et al.  Dueling Network Architectures for Deep Reinforcement Learning , 2015, ICML.

[9]  Luis Contreras,et al.  NECOS Project: Towards Lightweight Slicing of Cloud Federated Infrastructures , 2018, 2018 4th IEEE Conference on Network Softwarization and Workshops (NetSoft).

[10]  Javier Baliosian,et al.  End-to-end elasticity control of cloud-network slices , 2019, Internet Technol. Lett..

[11]  Srikanth Kandula,et al.  Resource Management with Deep Reinforcement Learning , 2016, HotNets.

[12]  Muntasir Raihan Rahman,et al.  Deep Reinforcement Learning for Network Slicing with Heterogeneous Resource Requirements and Time Varying Traffic Dynamics , 2019, 2019 15th International Conference on Network and Service Management (CNSM).

[13]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[14]  Clayman Stuart,et al.  Slicing 5G Networks: An Architectural Survey , 2020 .

[15]  Konstantinos Poularakis,et al.  MACS: Deep Reinforcement Learning based SDN Controller Synchronization Policy Design , 2019, 2019 IEEE 27th International Conference on Network Protocols (ICNP).

[16]  Alex Graves,et al.  Asynchronous Methods for Deep Reinforcement Learning , 2016, ICML.

[17]  Masahiko Jinno,et al.  Introducing elasticity and adaptation into the optical domain toward more efficient and scalable optical transport networks , 2010, 2010 ITU-T Kaleidoscope: Beyond the Internet? - Innovations for Future Networks and Services.