netmap: A Novel Framework for Fast Packet I/O

Many applications (routers, traffic monitors, firewalls, etc.) need to send and receive packets at line rate even on very fast links. In this paper we present netmap, a novel framework that enables commodity operating systems to handle the millions of packets per seconds traversing 1..10 Gbit/s links, without requiring custom hardware or changes to applications. In building netmap, we identified and successfully reduced or removed three main packet processing costs: per-packet dynamic memory allocations, removed by preallocating resources; system call overheads, amortized over large batches; and memory copies, eliminated by sharing buffers and metadata between kernel and userspace, while still protecting access to device registers and other kernel memory areas. Separately, some of these techniques have been used in the past. The novelty in our proposal is not only that we exceed the performance of most of previouswork, but also that we provide an architecture that is tightly integrated with existing operating system primitives, not tied to specific hardware, and easy to use and maintain. Netmap has been implemented in FreeBSD and Linux for several 1 and 10 Gbit/s network adapters. In our prototype, a single core running at 900 MHz can send or receive 14.88 Mpps (the peak packet rate on 10 Gbit/s links). This is more than 20 times faster than conventional APIs. Large speedups (5× and more) are also achieved on user-space Click and other packet forwarding applications using a libpcap emulation library running on top of netmap.

[1]  Andrew J. T. Colin,et al.  The Implementation , 1972, Softw. Pract. Exp..

[2]  Steven McCanne,et al.  The BSD Packet Filter: A New Architecture for User-level Packet Capture , 1993, USENIX Winter.

[3]  W. Richard Stevens,et al.  TCP/IP Illustrated, Volume 2: The Implementation , 1995 .

[4]  K. K. Ramakrishnan,et al.  Eliminating receive livelock in an interrupt-driven kernel , 1996, TOCS.

[5]  Eddie Kohler,et al.  The Click modular router , 1999, SOSP.

[6]  L. Deri Improving Passive Packet Capture : Beyond Device Polling , 2003 .

[7]  Mark Handley,et al.  XORP: an open platform for network research , 2003, CCRV.

[8]  Luca Deri nCap: wire-speed packet capture and transmission , 2005, Workshop on End-to-End Monitoring Techniques and Services, 2005..

[9]  Glen Gibb,et al.  NetFPGA--An Open Platform for Gigabit-Rate Network Switching and Routing , 2007, 2007 IEEE International Conference on Microelectronic Systems Education (MSE'07).

[10]  Nick McKeown,et al.  OpenFlow: enabling innovation in campus networks , 2008, CCRV.

[11]  Amiel A. Heyde,et al.  Using the Endace DAG 3.7GF card with FreeBSD 7.0 , 2008 .

[12]  Katerina J. Argyraki,et al.  RouteBricks: exploiting parallelism to scale software routers , 2009, SOSP '09.

[13]  Sangjin Han,et al.  PacketShader: a GPU-accelerated software router , 2010, SIGCOMM '10.

[14]  Hakim Weatherspoon,et al.  Operating Systems Abstractions for Software Packet Processing in Datacenters , 2011 .

[15]  Luigi Rizzo,et al.  Transparent acceleration of software packet forwarding using netmap , 2012, 2012 Proceedings IEEE INFOCOM.