Potential performance bottleneck in Linux TCP

TCP is the most widely used transport protocol on the Internet today. Over the years, especially recently, due to requirements of high bandwidth transmission, various approaches have been proposed to improve TCP performance. The Linux 2.6 kernel is now preemptible. It can be interrupted mid-task, making the system more responsive and interactive. However, we have noticed that Linux kernel preemption can interact badly with the performance of the networking subsystem. In this paper we investigate the performance bottleneck in Linux TCP. We systematically describe the trip of a TCP packet from its ingress into a Linux network end system to its final delivery to the application; we study the performance bottleneck in Linux TCP through mathematical modeling and practical experiments; finally we propose and test one possible solution to resolve this performance bottleneck in Linux TCP.

[1]  Van Jacobson,et al.  TCP Extension for High-Speed Paths , 1990, RFC.

[2]  Matthew Wilcox,et al.  Interactive Kernel Performance Kernel Performance in Desktop and Real-time Applications , 2003 .

[3]  Randy H. Katz,et al.  The Eifel algorithm: making TCP robust against spurious retransmissions , 2000, CCRV.

[4]  Injong Rhee,et al.  Delay-based congestion avoidance for TCP , 2003, TNET.

[5]  Sally Floyd,et al.  Simulation-based comparisons of Tahoe, Reno and SACK TCP , 1996, CCRV.

[6]  Wu-chun Feng,et al.  User-space auto-tuning for TCP flow control in computational grids , 2004, Comput. Commun..

[7]  Martin Mauve,et al.  A survey on TCP-friendly congestion control , 2001, IEEE Netw..

[8]  Y. Raghu Reddy,et al.  Web100: extended TCP instrumentation for research, education and diagnosis , 2003, CCRV.

[9]  Greg Kroah-Hartman,et al.  Linux device drivers, third edition , 2005 .

[10]  Injong Rhee,et al.  Binary increase congestion control (BIC) for fast long-distance networks , 2004, IEEE INFOCOM 2004.

[11]  Erich M. Nahum,et al.  Server Network Scalability and TCP Offload , 2005, USENIX Annual Technical Conference, General Track.

[12]  Larry L. Peterson,et al.  TCP Vegas: End to End Congestion Avoidance on a Global Internet , 1995, IEEE J. Sel. Areas Commun..

[13]  Van Jacobson,et al.  TCP extensions for long-delay paths , 1988, RFC.

[14]  Gordon Fischer,et al.  The Linux Kernel Primer: A Top-Down Approach for x86 and PowerPC Architectures , 2005 .

[15]  Brian Tierney,et al.  A TCP Tuning Daemon , 2002, ACM/IEEE SC 2002 Conference (SC'02).

[16]  D. Leith,et al.  H-TCP : A framework for congestion control in high-speed and long-distance networks , 2005 .

[17]  David D. Clark,et al.  An analysis of TCP processing overhead , 1988, IEEE Communications Magazine.

[18]  Francine Berman,et al.  Overview of the Book: Grid Computing – Making the Global Infrastructure a Reality , 2003 .

[19]  Ren Wang,et al.  TCP Westwood: congestion window control using bandwidth estimation , 2001, GLOBECOM'01. IEEE Global Telecommunications Conference (Cat. No.01CH37270).

[20]  Marco Cesati,et al.  Understanding the Linux Kernel, Third Edition , 2005 .

[21]  Greg Kroah-Hartman,et al.  Linux Device Drivers , 1998 .

[22]  Klaus Wehrle,et al.  The Linux networking architecture : design and implementation of network protocols in the Linux kernel , 2005 .

[23]  Fernando Paganini,et al.  FAST TCP: from theory to experiments , 2005, IEEE Netw..

[24]  V. Jacobson,et al.  Congestion avoidance and control , 1988, CCRV.

[25]  Sally Floyd,et al.  HighSpeed TCP for Large Congestion Windows , 2003, RFC.

[26]  Ren Wang,et al.  TCP westwood: Bandwidth estimation for enhanced transport over wireless links , 2001, MobiCom '01.

[27]  Kimmo E. E. Raatikainen,et al.  F-RTO: an enhanced recovery algorithm for TCP retransmission timeouts , 2003, CCRV.

[28]  K. K. Ramakrishnan,et al.  Eliminating receive livelock in an interrupt-driven kernel , 1996, TOCS.

[29]  William E. Allcock,et al.  The Globus Striped GridFTP Framework and Server , 2005, ACM/IEEE SC 2005 Conference (SC'05).

[30]  Wenji Wu,et al.  The performance analysis of linux networking - Packet receiving , 2007, Comput. Commun..

[31]  Ami Marowka,et al.  The GRID: Blueprint for a New Computing Infrastructure , 2000, Parallel Distributed Comput. Pract..

[32]  Ian Foster,et al.  The Grid 2 - Blueprint for a New Computing Infrastructure, Second Edition , 1998, The Grid 2, 2nd Edition.

[33]  Steven McCanne,et al.  On improving the fairness of TCP congestion avoidance , 1998, IEEE GLOBECOM 1998 (Cat. NO. 98CH36250).

[34]  Robert Love,et al.  Linux Kernel Development (2nd Edition) (Novell Press) , 2005 .

[35]  Matthew Mathis,et al.  The macroscopic behavior of the TCP congestion avoidance algorithm , 1997, CCRV.

[36]  Pasi Sarolahti,et al.  Congestion Control in Linux TCP , 2002, USENIX Annual Technical Conference, FREENIX Track.

[37]  Matthew Mathis,et al.  Forward acknowledgement: refining TCP congestion control , 1996, SIGCOMM '96.

[38]  Vern Paxson,et al.  Computing TCP's Retransmission Timer , 2000, RFC.