Using Reinforcement Learning to Allocate and Manage SFC in Cellular Networks

In this paper, we propose the use of reinforcement learning to deploy a service function chain (SFC) of cellular network service and manage the VNFs operation. We consider that the SFC is deployed by the reinforcement learning agent considering a scenario with distributed data centers, where the virtual network functions (VNFs) are deployed in virtual machines in commodity servers. The VNF management is related to create, delete, and restart the VNFs. The main purpose is to reduce the number of lost packets taking into account the energy consumption of the servers. We use the Proximal Policy Optimization (PPO2) algorithm to implement the agent and preliminary results show that the agent is able to allocate the SFC and manage the VNFs, reducing the number of lost packets.

[1]  Preeti Arora,et al.  Analysis of K-Means and K-Medoids Algorithm For Big Data , 2016 .

[2]  Judith Kelner,et al.  Minimizing and Managing Cloud Failures , 2017, Computer.

[3]  Sergey Levine,et al.  Trust Region Policy Optimization , 2015, ICML.

[4]  Chris Metz,et al.  COLAP: A predictive framework for service function chain placement in a multi-cloud environment , 2017, 2017 IEEE 7th Annual Computing and Communication Workshop and Conference (CCWC).

[5]  Alec Radford,et al.  Proximal Policy Optimization Algorithms , 2017, ArXiv.

[6]  Marco De Nadai,et al.  A multi-source dataset of urban life in the city of Milan and the Province of Trentino , 2015, Scientific Data.

[7]  B D Satoto,et al.  Integration K-Means Clustering Method and Elbow Method For Identification of The Best Customer Profile Cluster , 2018, IOP Conference Series: Materials Science and Engineering.

[8]  Alex Graves,et al.  Asynchronous Methods for Deep Reinforcement Learning , 2016, ICML.

[9]  Juan Felipe Botero,et al.  Resource Allocation in NFV: A Comprehensive Survey , 2016, IEEE Transactions on Network and Service Management.

[10]  Mohammed Samaka,et al.  A survey on service function chaining , 2016, J. Netw. Comput. Appl..

[11]  Gustavo Rau de Almeida Callou,et al.  Availability modeling and analysis of a disaster-recovery-as-a-service solution , 2017, Computing.

[12]  Jiaxing Zhang,et al.  NFVdeep: Adaptive Online Service Function Chain Deployment with Deep Reinforcement Learning , 2019, 2019 IEEE/ACM 27th International Symposium on Quality of Service (IWQoS).

[13]  K. K. Ramakrishnan,et al.  CleanG: A Clean-Slate EPC Architecture and ControlPlane Protocol for Next Generation Cellular Networks , 2016, CAN@CoNEXT.

[14]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.