Load Balancing Optimization in Software-Defined Wide Area Networking (SD-WAN) using Deep Reinforcement Learning

Software-Defined Wide Area Network (SD-WAN) holds tremendous potential to provide multi-cloud multi-network interconnection and prevent channel congestion. However, traffic among Customer Premises Edge (CPE) and controllers continuously increases, requiring pre-emptive load balancing in the control plane. In this paper, we study the flow migration problem in SD-WANs when the controller presents a limited processing capacity. Specifically, the data plane may include one or more CPE deployed at a site where service traffic is forwarded. To address this narrow, we propose a new approach based on a Deep Reinforcement Learning (DRL) strategy to optimize the balancing process under a latency constraint. As far as we can tell, we have not observed any pertinent research published in this context. The obtained simulation results revealed that our proposed approach decreases the load balancing and outperforms other baseline methods.

[1]  Wei Guo,et al.  Bandwidth constrained client selection and scheduling for federated learning over SD-WAN , 2022, IET Commun..

[2]  Guido Maier,et al.  On Deep Reinforcement Learning for Traffic Engineering in SD-WAN , 2021, IEEE Journal on Selected Areas in Communications.

[3]  Yi Wang,et al.  HybridFlow: Achieving Load Balancing in Software-Defined WANs With Scalable Routing , 2021, IEEE Transactions on Communications.

[4]  Xiangnan Zhong,et al.  Semicentralized Deep Deterministic Policy Gradient in Cooperative StarCraft Games , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[5]  Zoubeir Mlika,et al.  Preemptive SDN Load Balancing With Machine Learning for Delay Sensitive Applications , 2020, IEEE Transactions on Vehicular Technology.

[6]  Piotr Wydrych,et al.  Dynamic Traffic Management for SD-WAN Inter-Cloud Communication , 2020, IEEE Journal on Selected Areas in Communications.

[7]  Amir H. Gandomi,et al.  Improving End-Users Utility in Software-Defined Wide Area Network Systems , 2020, IEEE Transactions on Network and Service Management.

[8]  Miao Pan,et al.  Optimal VNF Placement via Deep Reinforcement Learning in SDN/NFV-Enabled Networks , 2020, IEEE Journal on Selected Areas in Communications.

[9]  Xiuguo Zhang,et al.  An Autonomous Path Planning Model for Unmanned Ships Based on Deep Reinforcement Learning , 2020, Sensors.

[10]  Ken Chen,et al.  Feedback Deep Deterministic Policy Gradient With Fuzzy Reward for Robotic Multiple Peg-in-Hole Assembly Tasks , 2019, IEEE Transactions on Industrial Informatics.

[11]  Lu Liu,et al.  A Load-Balancing Mechanism for Distributed SDN Control Plane Using Response Time , 2018, IEEE Transactions on Network and Service Management.

[12]  Jianping Pan,et al.  Capacity-Aware and Delay-Guaranteed Resilient Controller Placement for Software-Defined WANs , 2018, IEEE Transactions on Network and Service Management.

[13]  Tanir Ozcelebi,et al.  Deep Reinforcement Learning for Load-Balancing Aware Network Control in IoT Edge Systems , 2021, IEEE Transactions on Parallel and Distributed Systems.

[14]  Daoxing Guo,et al.  Multi-Controller Placement for Load Balancing in SDWAN , 2019, IEEE Access.