Traffic engineering based on deep reinforcement learning in hybrid IP/SR network

Segment Routing (SR) is a new routing paradigm based on source routing and provide traffic engineering (TE) capabilities in IP network. By extending interior gateway protocol(IGP), SR can be easily applied to IP network. However, upgrading current IP network to a full SR one can be costly and difficult. Hybrid IP/SR network will last for some time. Aiming at the low flexibility problem of static TE policies in the current SR networks, this paper proposes a Deep Reinforcement Learning (DRL) based TE scheme. The proposed scheme employs multi-path transmission and use DRL to dynamically adjust the traffic splitting ratio among different paths based on the network traffic distribution. As a result, the network congestion can be mitigated and the performance of the network is improved. Simulation results show that our proposed scheme can improve the throughput of the network by up to 9% than existing schemes.

[1]  Kwan L. Yeung,et al.  Traffic Engineering in Segment Routing Networks Using MILP , 2020, IEEE Transactions on Network and Service Management.

[2]  Xingang Shi,et al.  Traffic Engineering in Partially Deployed Segment Routing Over IPv6 Network With Deep Reinforcement Learning , 2020, IEEE/ACM Transactions on Networking.

[3]  Hong Xu,et al.  Node-Constrained Traffic Engineering: Theory and Applications , 2019, IEEE/ACM Transactions on Networking.

[4]  Stefano Salsano,et al.  Segment Routing: A Comprehensive Survey of Research Activities, Standardization Efforts, and Implementation Results , 2019, IEEE Communications Surveys & Tutorials.

[5]  Pei Zhang,et al.  Throughput Maximization Routing in the Hybrid Segment Routing Network , 2018, ICTCE 2018.

[6]  Chi Harold Liu,et al.  Experience-driven Networking: A Deep Reinforcement Learning based Approach , 2018, IEEE INFOCOM 2018 - IEEE Conference on Computer Communications.

[7]  Sergey Levine,et al.  Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor , 2018, ICML.

[8]  Dafna Shahaf,et al.  Learning to Route , 2017, HotNets.

[9]  Marco Listanti,et al.  Incremental Deployment of Segment Routing Into an ISP Network: a Traffic Engineering Perspective , 2017, IEEE/ACM Transactions on Networking.

[10]  Markus Chimani,et al.  Traffic engineering using segment routing and considering requirements of a carrier IP network , 2017, 2017 IFIP Networking Conference (IFIP Networking) and Workshops.

[11]  Hong Xu,et al.  Centrality-based Middlepoint Selection for Traffic Engineering with Segment Routing , 2017 .

[12]  Filippo Cugini,et al.  Traffic engineering in segment routing networks , 2017, Comput. Networks.

[13]  Alex Graves,et al.  Asynchronous Methods for Deep Reinforcement Learning , 2016, ICML.

[14]  Yuval Tassa,et al.  Continuous control with deep reinforcement learning , 2015, ICLR.

[15]  T. V. Lakshman,et al.  Optimized network traffic engineering using segment routing , 2015, 2015 IEEE Conference on Computer Communications (INFOCOM).

[16]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[17]  Clarence Filsfils,et al.  The Segment Routing Architecture , 2014, 2015 IEEE Global Communications Conference (GLOBECOM).

[18]  Konstantina Papagiannaki,et al.  Structural analysis of network traffic flows , 2004, SIGMETRICS '04/Performance '04.

[19]  Mikkel Thorup,et al.  Traffic engineering with traditional IP routing protocols , 2002, IEEE Commun. Mag..

[20]  Vijay Srinivasan,et al.  RSVP-TE: Extensions to RSVP for LSP Tunnels , 2001, RFC.

[21]  Lionel M. Ni,et al.  Traffic engineering with MPLS in the Internet , 2000, IEEE Netw..

[22]  Daniel O. Awduche,et al.  Requirements for Traffic Engineering Over MPLS , 1999, RFC.