Dynamic Sketch: Efficient and Adjustable Heavy Hitter Detection for Software Packet Processing

Heavy hitter detection is a key task for networking traffic profiling, which can be used for various purposes such as Denial of Service (DoS) attack detection, Quality of Service (QoS) scheduling, load balancing, and flow size based routing, etc. Over the years, many efforts have been made on designing data structures and algorithms to achieve fast and memory-efficient inline profiling in cloud networks. Traditional heavy hitter detection methods, however, yield an innate and nonadjustable profiling accuracy (i.e., false positive or false negative) once the data structure is initialized. Users have no runtime feedback information nor control on the profiling accuracy, which could be an important factor for their usages. In this paper, we propose and evaluate a novel dynamic and memory-efficient heavy hitter detection algorithm, called Dynamic sketch. Dynamic sketch performs runtime accuracy monitoring and provides feedback to users via a sampling based method. It also self-adjusts the accuracy at runtime to satisfy the target given by the user. We implemented Dynamic sketch and our evaluations show that Dynamic sketch is able to report profiling accuracy with only a minimal 2% performance overhead. In addition, Dynamic sketch is 2.35 × faster than the state-of-the-art hash table based heavy hitter detector and achieves more than 2× memory efficiency than the state-of-the-art sketch based implementation.