Renovate high performance user-level stacks' innovation utilizing commodity network adaptors

Today's data center servers are equipped with high speed and complex network adaptors, featuring an array of functions, e.g. hardware TX/RX queues, packet filters, rate limiters, etc. Recent work like IX, Arrakis, MultiStack has made us rekindle the user-level network stacks' innovation utilizing these commodity network adaptors. In this paper, we revisit the idea to move stacks' design from in-kernel shared space into user-level application-specific dedicated one, for high performance and ease of development and deployment. We provide an unified control plane TAPM to exploit and manage the hardware adaptors' resources, and a dedicated data plane hwTAP to support different user-level stacks. TAPM and hwTAP highlight the utilization of hardware features from commodity network adaptors, to support the innovation of different user-level stacks. Experiments show that the hardware switching module can keep the input rate without any overheads and costs. TAPM could configure the hwTAP dynamically. Our run-to-completion user-level stack also achieves high throughput and low latency.

[1]  Christoforos E. Kozyrakis,et al.  IX: A Protected Dataplane Operating System for High Throughput and Low Latency , 2014, OSDI.

[2]  Eunyoung Jeong,et al.  mTCP: a Highly Scalable User-level TCP Stack for Multicore Systems , 2014, NSDI.

[3]  David A. Maltz,et al.  Data center TCP (DCTCP) , 2010, SIGCOMM 2010.

[4]  Erich M. Nahum,et al.  A measurement-based study of MultiPath TCP performance over wireless networks , 2013, Internet Measurement Conference.

[5]  Armin R. Mikler,et al.  NetPIPE: A Network Protocol Independent Performance Evaluator , 1996 .

[6]  Timothy Roscoe,et al.  Arrakis , 2014, OSDI.

[7]  Mahidhar Tatineni,et al.  SR-IOV: Performance Benefits for Virtualized Interconnects , 2014, XSEDE '14.

[8]  Luigi Rizzo,et al.  netmap: A Novel Framework for Fast Packet I/O , 2012, USENIX ATC.

[9]  Van Jacobson,et al.  BBR: Congestion-Based Congestion Control , 2016, ACM Queue.

[10]  Costin Raiciu,et al.  Rekindling network protocol innovation with user-level stacks , 2014, CCRV.

[11]  Thomas E. Anderson,et al.  Ingress Pipeline Queues Packet Buffer DMA PipelineDMA Egress Pipeline , 2015 .

[12]  Dawson R. Engler,et al.  Fast and flexible application-level networking on exokernel systems , 2002, TOCS.

[13]  Timothy Roscoe,et al.  We Need to Talk About NICs , 2013, HotOS.

[14]  Steven McCanne,et al.  The BSD Packet Filter: A New Architecture for User-level Packet Capture , 1993, USENIX Winter.

[15]  Krishna P. Gummadi,et al.  Proceedings of the 2013 conference on Internet measurement conference , 2013, IMC 2013.

[16]  Mark Handley,et al.  Network stack specialization for performance , 2015, SIGCOMM 2015.

[17]  Arvind Krishnamurthy,et al.  Proceedings of the 2014 ACM conference on SIGCOMM , 2014, SIGCOMM 2014.

[18]  Daniel Raumer,et al.  MoonGen: A Scriptable High-Speed Packet Generator , 2014, Internet Measurement Conference.