Scaling Geo-Distributed Network Function Chains: A Prediction and Learning Framework

Geo-distributed virtual network function (VNF) chaining has been useful, such as in network slicing in 5G networks and for network traffic processing in the WAN. Agile scaling of the VNF chains according to real-time traffic rates is the key in network function virtualization. Designing efficient scaling algorithms is challenging, especially for geo-distributed chains, where bandwidth costs and latencies incurred by the WAN traffic are important but difficult to handle in making scaling decisions. Existing studies have largely resorted to optimization algorithms in scaling design. Aiming at better decisions empowered by in-depth learning from experiences, this paper proposes a deep learning-based framework for scaling of the geo-distributed VNF chains, exploring inherent pattern of traffic variation and good deployment strategies over time. We novelly combine a recurrent neural network as the traffic model for predicting upcoming flow rates and a deep reinforcement learning (DRL) agent for making chain placement decisions. We adopt the experience replay technique based on the actor–critic DRL algorithm to optimize the learning results. Trace-driven simulation shows that with limited offline training, our learning framework adapts quickly to traffic dynamics online and achieves lower system costs, compared to the existing representative algorithms.

[1]  Jürgen Schmidhuber,et al.  Recurrent World Models Facilitate Policy Evolution , 2018, NeurIPS.

[2]  Franck Le,et al.  Online Scaling of NFV Service Chains Across Geo-Distributed Datacenters , 2016, IEEE/ACM Transactions on Networking.

[3]  Zongpeng Li,et al.  An efficient auction mechanism for service chains in the NFV market , 2016, IEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications.

[4]  Razvan Pascanu,et al.  Imagination-Augmented Agents for Deep Reinforcement Learning , 2017, NIPS.

[5]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[6]  Demis Hassabis,et al.  Mastering the game of Go without human knowledge , 2017, Nature.

[7]  Byrav Ramamurthy,et al.  Optimized Service Chain Mapping and reduced flow processing with Application-Awareness , 2018, 2018 4th IEEE Conference on Network Softwarization and Workshops (NetSoft).

[8]  Jia Zhang,et al.  MDP and Machine Learning-Based Cost-Optimization of Dynamic Resource Allocation for Network Function Virtualization , 2015, 2015 IEEE International Conference on Services Computing.

[9]  Antoine Cully,et al.  Robots that can adapt like animals , 2014, Nature.

[10]  Quoc V. Le,et al.  A Hierarchical Model for Device Placement , 2018, ICLR.

[11]  Peilin Hong,et al.  Efficiently Embedding Service Function Chains with Dynamic Virtual Network Function Placement in Geo-Distributed Cloud System , 2019, IEEE Transactions on Parallel and Distributed Systems.

[12]  Alex Graves,et al.  Asynchronous Methods for Deep Reinforcement Learning , 2016, ICML.

[13]  Raouf Boutaba,et al.  Service Function Chaining Simplified , 2016, ArXiv.

[14]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[15]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[16]  Zongpeng Li,et al.  Virtualized Network Coding Functions on the Internet , 2017, 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS).

[17]  Joseph Naor,et al.  Near optimal placement of virtual network functions , 2015, 2015 IEEE Conference on Computer Communications (INFOCOM).

[18]  Raouf Boutaba,et al.  On orchestrating virtual network functions , 2015, 2015 11th International Conference on Network and Service Management (CNSM).

[19]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[20]  Gonzalo Camarillo,et al.  The 3G IP Multimedia Subsystem : Merging the Internet and the Cellular Worlds , 2004 .

[21]  Chi Harold Liu,et al.  Experience-driven Networking: A Deep Reinforcement Learning based Approach , 2018, IEEE INFOCOM 2018 - IEEE Conference on Computer Communications.

[22]  Mingquan Wu,et al.  On Wide Area Network Optimization , 2012, IEEE Communications Surveys & Tutorials.

[23]  Xiaojiang Du,et al.  Provably efficient algorithms for joint placement and allocation of virtual network functions , 2017, IEEE INFOCOM 2017 - IEEE Conference on Computer Communications.

[24]  Seungjoon Lee,et al.  Network function virtualization: Challenges and opportunities for innovations , 2015, IEEE Communications Magazine.

[25]  Ying Jun Zhang,et al.  DRAG: Deep Reinforcement Learning Based Base Station Activation in Heterogeneous Networks , 2018, IEEE Transactions on Mobile Computing.

[26]  Hongzi Mao,et al.  Neural Adaptive Video Streaming with Pensieve , 2017, SIGCOMM.

[27]  Srikanth Kandula,et al.  Resource Management with Deep Reinforcement Learning , 2016, HotNets.

[28]  Marco Pavone,et al.  Cellular Network Traffic Scheduling With Deep Reinforcement Learning , 2018, AAAI.

[29]  Jose Ordonez-Lucena,et al.  Network Slicing for 5G with SDN/NFV: Concepts, Architectures, and Challenges , 2017, IEEE Communications Magazine.

[30]  Richard Evans,et al.  Deep Reinforcement Learning in Large Discrete Action Spaces , 2015, 1512.07679.