Bullet trains: a study of NIC burst behavior at microsecond timescales

While numerous studies have examined the macro-level behavior of traffic in data center networks---overall flow sizes, destination variability, and TCP burstiness---little information is available on the behavior of data center traffic at packet-level timescales. Whereas one might assume that flows from different applications fairly share available link bandwidth, and that packets within a single flow are uniformly paced, the reality is more complex. To meet increasingly high link rates of 10 Gbps and beyond, batching is typically introduced across the network stack---at the application, middleware, OS, transport, and NIC layers. This batching results in short-term packet bursts, which have implications for the design and performance requirements of packet processing devices along the path, including middleboxes, SDN-enabled switches, and virtual machine hypervisors. In this paper, we study the burst behavior of traffic emanating from a 10-Gbps end host across a variety of data center applications. We find that at 10--100 microsecond timescales, the traffic exhibits large bursts (i.e., 10s of packets in length). We further find that this level of burstiness is largely outside of application control, and independent of the high-level behavior of applications.