Congestion Control: A Renaissance with Machine Learning

In the past several decades, it has been well known that the Transmission Control Protocol (TCP), even with its modern variants such as CUBIC, may not perform optimally when available bottleneck bandwidth needs to be fully utilized, yet without unnecessarily increasing the end-to-end latency. These observations have led to a recent resurgence of interest in the topic of redesigning congestion control protocols and replacing modern TCP variants using machine learning. In this article, we examine and compare some of the most prominent recent research results on the use of machine learning to redesign congestion control protocols, with an editorial commentary on potential research directions in the near-term future.

[1]  Hari Balakrishnan,et al.  TCP ex machina: computer-generated congestion control , 2013, SIGCOMM.

[2]  Alexander H. Miller,et al.  MVFST-RL: An Asynchronous RL Framework for Congestion Control with Delayed Actions , 2019, ArXiv.

[3]  Brighten Godfrey,et al.  A Deep Reinforcement Learning Perspective on Internet Congestion Control , 2019, ICML.

[4]  Mo Dong,et al.  PCC Vivace: Online-Learning Congestion Control , 2018, NSDI.

[5]  H. Jonathan Chao,et al.  Classic Meets Modern: a Pragmatic Learning-Based Congestion Control for the Internet , 2020, SIGCOMM.

[6]  Philip Levis,et al.  Pantheon: the training ground for Internet congestion-control research , 2018, USENIX Annual Technical Conference.

[7]  Zhiyuan Xu,et al.  Experience-Driven Congestion Control: When Multi-Path TCP Meets Deep Reinforcement Learning , 2019, IEEE Journal on Selected Areas in Communications.

[8]  Hari Balakrishnan,et al.  Copa: Practical Delay-Based Congestion Control for the Internet , 2018, ANRW.

[9]  Jeffrey M. Jaffe,et al.  Flow Control Power is Nondecentralizable , 1981, IEEE Trans. Commun..

[10]  Van Jacobson,et al.  BBR: Congestion-Based Congestion Control , 2016, ACM Queue.

[11]  Mo Dong,et al.  PCC: Re-architecting Congestion Control for Consistent High Performance , 2014, NSDI.

[12]  Mark Handley,et al.  Congestion control for high bandwidth-delay product networks , 2002, SIGCOMM '02.

[13]  Johannes Gehrke,et al.  Reinforcement learning for bandwidth estimation and congestion control in real-time communications , 2019, ArXiv.

[14]  Yanjiao Chen,et al.  Eagle: Refining Congestion Control by Learning from the Experts , 2020, IEEE INFOCOM 2020 - IEEE Conference on Computer Communications.

[15]  Injong Rhee,et al.  CUBIC: a new TCP-friendly high-speed TCP variant , 2008, OPSR.