Potential performance bottleneck in Linux TCP

Transmission control protocol (TCP) is the most widely used transport protocol on the Internet today. Over the years, especially recently, due to requirements of high bandwidth transmission, various approaches have been proposed to improve TCP performance. The Linux 2.6 kernel is now preemptible. It can be interrupted mid-task, making the system more responsive and interactive. However, we have noticed that Linux kernel preemption can interact badly with the performance of the networking subsystem. In this paper, we investigate the performance bottleneck in Linux TCP. We systematically describe the trip of a TCP packet from its ingress into a Linux network end system to its final delivery to the application; we study the performance bottleneck in Linux TCP through mathematical modelling and practical experiments; finally, we propose and test one possible solution to resolve this performance bottleneck in Linux TCP. Copyright © 2007 John Wiley & Sons, Ltd.

[1]  Ian Foster,et al.  The Grid 2 - Blueprint for a New Computing Infrastructure, Second Edition , 1998, The Grid 2, 2nd Edition.

[2]  Sally Floyd,et al.  HighSpeed TCP for Large Congestion Windows , 2003, RFC.

[3]  Ren Wang,et al.  TCP westwood: Bandwidth estimation for enhanced transport over wireless links , 2001, MobiCom '01.

[4]  Kimmo E. E. Raatikainen,et al.  F-RTO: an enhanced recovery algorithm for TCP retransmission timeouts , 2003, CCRV.

[5]  Andy Oram,et al.  Understanding the Linux Kernel, Second Edition , 2002 .

[6]  Pasi Sarolahti,et al.  Congestion Control in Linux TCP , 2002, USENIX Annual Technical Conference, FREENIX Track.

[7]  Wenji Wu,et al.  The performance analysis of linux networking - Packet receiving , 2007, Comput. Commun..

[8]  Martin Mauve,et al.  A survey on TCP-friendly congestion control , 2001, IEEE Netw..

[9]  David Clark,et al.  An analysis of TCP processing overhead , 1989 .

[10]  Klaus Wehrle,et al.  The Linux networking architecture : design and implementation of network protocols in the Linux kernel , 2005 .

[11]  V. Jacobson,et al.  Congestion avoidance and control , 1988, CCRV.

[12]  Fernando Paganini,et al.  FAST TCP: from theory to experiments , 2005, IEEE Network.

[13]  Daniel Pierre Bovet,et al.  Understanding the Linux Kernel , 2000 .

[14]  Brian Tierney,et al.  A TCP Tuning Daemon , 2002, ACM/IEEE SC 2002 Conference (SC'02).

[15]  Robert Love,et al.  Linux Kernel Development (2nd Edition) (Novell Press) , 2005 .

[16]  Matthew Mathis,et al.  The macroscopic behavior of the TCP congestion avoidance algorithm , 1997, CCRV.

[17]  Abraham Silberschatz,et al.  Operating System Concepts , 1983 .

[18]  Injong Rhee,et al.  Delay-based congestion avoidance for TCP , 2003, TNET.

[19]  H KatzRandy,et al.  The Eifel algorithm , 2000 .

[20]  Y. Raghu Reddy,et al.  Web100: extended TCP instrumentation for research, education and diagnosis , 2003, CCRV.

[21]  Robert Love,et al.  Linux Kernel Development , 2003 .

[22]  Injong Rhee,et al.  Binary increase congestion control (BIC) for fast long-distance networks , 2004, IEEE INFOCOM 2004.

[23]  Steven McCanne,et al.  On improving the fairness of TCP congestion avoidance , 1998, IEEE GLOBECOM 1998 (Cat. NO. 98CH36250).

[24]  Erich M. Nahum,et al.  Server Network Scalability and TCP Offload , 2005, USENIX Annual Technical Conference, General Track.

[25]  K. K. Ramakrishnan,et al.  Eliminating receive livelock in an interrupt-driven kernel , 1996, TOCS.

[26]  William E. Allcock,et al.  The Globus Striped GridFTP Framework and Server , 2005, ACM/IEEE SC 2005 Conference (SC'05).

[27]  Randy H. Katz,et al.  The Eifel algorithm: making TCP robust against spurious retransmissions , 2000, CCRV.

[28]  Ren Wang,et al.  TCP Westwood: congestion window control using bandwidth estimation , 2001, GLOBECOM'01. IEEE Global Telecommunications Conference (Cat. No.01CH37270).

[29]  Sally Floyd,et al.  Simulation-based comparisons of Tahoe, Reno and SACK TCP , 1996, CCRV.

[30]  Wu-chun Feng,et al.  User-space auto-tuning for TCP flow control in computational grids , 2004, Comput. Commun..

[31]  Greg Kroah-Hartman,et al.  Linux Device Drivers, 3rd Edition , 2005 .

[32]  Vern Paxson,et al.  Computing TCP's Retransmission Timer , 2000, RFC.

[33]  Gordon Fischer,et al.  The Linux Kernel Primer: A Top-Down Approach for x86 and PowerPC Architectures , 2005 .

[34]  Larry L. Peterson,et al.  TCP Vegas: End to End Congestion Avoidance on a Global Internet , 1995, IEEE J. Sel. Areas Commun..

[35]  Van Jacobson,et al.  TCP extensions for long-delay paths , 1988, RFC.

[36]  Greg Kroah-Hartman,et al.  Linux Device Drivers , 1998 .

[37]  Francine Berman,et al.  Grid Computing: Making the Global Infrastructure a Reality , 2003 .