QoS-Aware Adaptive Routing in Multi-layer Hierarchical Software Defined Networks: A Reinforcement Learning Approach

Software-defined networks (SDNs) have been recognized as the next-generation networking paradigm that decouples the data forwarding from the centralized control. To realize the merits of dedicated QoS provisioning and fast route (re-)configuration services over the decoupled SDNs, various QoS requirements in packet delay, loss, and throughput should be supported by an efficient transportation with respect to each specific application. In this paper, a QoS-aware adaptive routing (QAR) is proposed in the designed multi-layer hierarchical SDNs. Specifically, the distributed hierarchical control plane architecture is employed to minimize signaling delay in large SDNs via three-levels design of controllers, i.e., the super, domain (or master), and slave controllers. Furthermore, QAR algorithm is proposed with the aid of reinforcement learning and QoS-aware reward function, achieving a time-efficient, adaptive, QoS-provisioning packet forwarding. Simulation results confirm that QAR outperforms the existing learning solution and provides fast convergence with QoS provisioning, facilitating the practical implementations in large-scale software service-defined networks.

[1]  Ian F. Akyildiz,et al.  A roadmap for traffic engineering in SDN-OpenFlow networks , 2014, Comput. Networks.

[2]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[3]  Yashar Ganjali,et al.  Kandoo: a framework for efficient and scalable offloading of control applications , 2012, HotSDN '12.

[4]  Jim Kurose,et al.  Computer Networking: A Top-Down Approach , 1999 .

[5]  Ian F. Akyildiz,et al.  Wireless software-defined networks (W-SDNs) and network function virtualization (NFV) for 5G cellular systems: An overview and qualitative evaluation , 2015, Comput. Networks.

[6]  Min Luo,et al.  Control traffic balancing in software defined networks , 2016, Comput. Networks.

[7]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[8]  Srikanth Kandula,et al.  Achieving high utilization with software-driven WAN , 2013, SIGCOMM.

[9]  Ian F. Akyildiz,et al.  Reinforcement learning for cooperative sensing gain in cognitive radio ad hoc networks , 2013, Wirel. Networks.

[10]  Wu Chou,et al.  An adaptive multi-path computation framework for centrally controlled networks , 2015, Comput. Networks.

[11]  Martín Casado,et al.  Onix: A Distributed Control Platform for Large-scale Production Networks , 2010, OSDI.

[12]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[13]  Nick McKeown,et al.  OpenFlow: enabling innovation in campus networks , 2008, CCRV.

[14]  Michael N. Katehakis,et al.  The Multi-Armed Bandit Problem: Decomposition and Computation , 1987, Math. Oper. Res..

[15]  Ian F. Akyildiz,et al.  SoftAir: A software defined networking architecture for 5G wireless systems , 2015, Comput. Networks.

[16]  Min Zhu,et al.  B4: experience with a globally-deployed software defined wan , 2013, SIGCOMM.

[17]  Mahesan Niranjan,et al.  On-line Q-learning using connectionist systems , 1994 .