Adaptive bandwidth control for efficient aggregate QoS provisioning

Internet links operate at high speeds, and past trends predict that these speeds will continue to increase rapidly. Routers and Intrusion Detection Systems that operate at up to OC-768 speeds (40 Gigabits/second) are currently being developed. In this paper we address a basic function common to several security and measurement applications running at line speeds: counting the number of distinct header patterns (flows) seen on a high speed link in a specified time period. For example one can detect port scans by counting the number of connections opened by the suspected port scanner and one can ascertain that a denial of service attack is in progress by counting the number of distinct (fake) source addresses that send packets to the suspected victim. We provide algorithms solving the flow counting problem using extremely small amounts of memory. This can translate into savings of scarce fast memory (SRAM) for hardware implementations. It can also help systems that use cheaper DRAM to allow them to scale to larger instances of the problem. The reduction in memory is particularly important for network security applications such as detecting port scans and DoS attacks that need to run a separate instance of the algorithm for each suspected attacker or victim. Our algorithms can be implemented in hardware at wire speeds (8 nsec per packet for OC-768) using simple CRC based hash functions, multiplexers and SRAM. They access at most one or two memory locations per packet. We expose the behavior of simple building blocks and provide a family of customizable counting algorithms that can be adapted to the requirements of various applications. This allows our algorithms to use less memory than the best known counting algorithm, probabilistic counting, to provide the same accuracy. • Virtual bitmap is well suited for triggers (which need only be accurate around a threshold value) such as detecting DoS attacks, and uses 292 bytes to achieve an error of 2.619% compared to 5,200 bytes for probabilistic counting. • Our “no assumptions” counting algorithm, multiresolution bitmap algorithms uses only 2143 bytes to count up to 100 million flows with an average error of 3%. Its accuracy is slightly better when the number of flows is small, while the accuracy of probabilistic counting using the same memory is much worse. • Our adaptive bitmap that exploits stationarity in the number of flows can count the number of distinct flows on a link that contains anywhere from 0 to 100 million flows with an average error of less than 1 % using only 2 Kbytes of memory. Probabilistic counting needs eight times more memory to achieve the same accuracy. • We used our triggered bitmap to replace the port scan detection component of the popular intrusion detection system Snort. Doing so reduced the memory usage from 89 Mbytes to 6.4 Mbytes (with an average error of 13.5%) as measured on a 10 minute trace. Probabilistic counting uses 23 Mbytes. This work was made possible by a grant from NIST for the Sensilla Project. King: Estimating Latency between Arbitrary Internet End Hosts ∗ Krishna P. Gummadi, Stefan Saroiu, Steven D. Gribble Department of Computer Science & Engineering University of Washington Seattle, WA, USA, 98195-2350 {tzoompy,gummadi,gribble}@cs.washington.edu The ability to estimate network latencies accurately between arbitrary Internet end hosts would enable wide-area measurement studies and applications, such as investigating routing path inefficiencies on a wide-scale [1] or constructing topologically sensitive overlay networks. In this poster we present King [2], a tool that accurately, quickly and cheaply estimates the latency between arbitrary end hosts by using existing DNS infrastructure in a novel way. Our approach is inspired by tools like Sting [3] and TBIT [4], which show that it is possible to use existing protocols in unanticipated ways to obtain results that were previously intractable. Our technique relies on two observations. First, given a pair of end hosts to be measured, in most cases it is possible for King to find their authoritative DNS name servers that are topologically close to the end hosts. Second, given a pair of DNS name servers, King can accurately estimate the latency between them using recursive DNS queries. Thus, King is able to use the measured latency between the name servers as an estimate of the latency between the end hosts. Although this estimate will inevitably suffer from inaccuracies, our extensive evaluation demonstrates that this estimation error is small (less than 20% error in over three-quarters of generated estimates). Compared to previous approaches like IDMaps [5] and GNP [6], King has two primary advantages (1) deployability and (2) accuracy. 1. Deployability: Unlike IDMaps, King does not require the deployment of additional infrastructure, and unlike GNP, King does not require end hosts to agree upon a single set of landmarks or share the coordinates with other end hosts. Because King uses existing DNS infrastructure, it scales naturally in terms of the number hosts that could be measured. Our evaluation indicates that King can be used to estimate latencies between arbitrary IP hosts more than 75% of time in the Internet today. 2. Accuracy: Unlike IDMaps and GNP the accuracy of King is not affected by the placement of measurement points (tracers or landmarks) in the Internet. King’s estimates are based on direct online measurements rather than extrapolation from latencies This work was supported by National Science Foundation under ITR grant No. CCR-0121341. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation. of a subset of paths measured offline. Our evaluation shows that the error in King latency estimates between end hosts is significantly smaller than that of IDMaps, while the error in its estimates between name servers is negligible. Further, we show that King can identify those of its own estimates that are likely to be inaccurate. These characteristics make King an ideal tool for a number of wide-area measurement studies that involve measuring latencies accurately between a large number of geographically distant end hosts. King is also fast and lightweight requiring the generation of only a few packets to produce an estimate. Our evaluation also demonstrates that (a) King preserves order among its latency estimates; a feature useful in identifying a close server from a group of server replicas and (b) authoritative name servers are located topologically close to their end hosts in a majority of scenarios (i.e., latencies as measured from a far away host to both the end host and its name server are very similar); this explains the high accuracy of King. Finally, we test the feasibility of using King for measurement studies by performing Detour study [1] on a larger scale. The results we obtained are consistent with the earlier Detour study. We were able to obtain our results with far greater ease as, unlike the previous study, we were not restricted to the publicly available traceroute servers and their usage policies. We also believe that the simplicity of our King tool will enable researchers to use it in myriads of unanticipated ways like the more popular Ping tool.