A Deep-reinforcement Learning Approach for SDN Routing Optimization

In order to realize a real-time and customizable routing optimization for software-defined network (SDN), the deep reinforcement learning method is integrated to the routing process of SDN. In this paper, we design a novel routing optimization mechanism based on deep reinforcement learning. This mechanism is capable of reducing the network maintenance and operational costs via improving network performance (e.g. delay, throughput), and it can also realize black-box optimization in continuous time. In addition, we conducted a series of experiments to evaluate the proposed routing optimization mechanism. The experimental results demonstrated that our proposed routing optimization mechanism has good effectiveness and convergence, which can not only achieve more stable performance than the traditional routing mechanism, but also provide better routing schemes.

[1]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[2]  Yuval Tassa,et al.  Continuous control with deep reinforcement learning , 2015, ICLR.

[3]  Raouf Boutaba,et al.  A comprehensive survey on machine learning for networking: evolution, applications and research opportunities , 2018, Journal of Internet Services and Applications.

[4]  Rui Sun,et al.  Q-FDBA: improving QoE fairness for video streaming , 2018, Multimedia Tools and Applications.

[5]  András Varga,et al.  An overview of the OMNeT++ simulation environment , 2008, SimuTools.

[6]  Jaime Lloret,et al.  Including artificial intelligence in a routing protocol using Software Defined Networks , 2017, 2017 IEEE International Conference on Communications Workshops (ICC Workshops).

[7]  Matthew Roughan,et al.  Simplifying the synthesis of internet traffic matrices , 2005, CCRV.

[8]  Ian F. Akyildiz,et al.  QoS-Aware Adaptive Routing in Multi-layer Hierarchical Software Defined Networks: A Reinforcement Learning Approach , 2016, 2016 IEEE International Conference on Services Computing (SCC).

[9]  Feng Tian,et al.  Dynamic routing and spectrum assignment based on multilayer virtual topology and ant colony optimization in elastic software-defined optical networks , 2017 .

[10]  Guy Lever,et al.  Deterministic Policy Gradient Algorithms , 2014, ICML.

[11]  Wei Li,et al.  A fast traffic classification method based on SDN network , 2015 .

[12]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[13]  Nei Kato,et al.  State-of-the-Art Deep Learning: Evolving Machine Intelligence Toward Tomorrow’s Intelligent Network Traffic Control Systems , 2017, IEEE Communications Surveys & Tutorials.

[14]  Mohammad Reza Parsaei,et al.  A new adaptive traffic engineering method for telesurgery using ACO algorithm over Software Defined Networks , 2017 .

[15]  Sándor Molnár,et al.  Proceedings of the 1st international conference on Simulation tools and techniques for communications, networks and systems & workshops , 2008, Simutools 2008.