High-performance algorithms and data structures to catch elephant flows

In high-speed networks, it is important to detect the presence of large flows-also known as elephant flows-because of their adverse effects on delay-sensitive flows. If detected on a timely fashion, network operators can apply active policies such as flow redirection or traffic shaping to ensure the overall quality of service of the network is preserved. Towards this objective, we develop a high-performance data structure and algorithm to address the problem of detecting large flows at very high-speed rates. Our solution leverages the concept of optimal sampling rate under partial information to avoid the need for processing every single packet on the network. With this strategy, we present a prototype of a high-performance network sensor capable of processing traffic rates at 100Gbps and detect the largest flows with high accuracy.

[1]  Kun-Chan Lan,et al.  A measurement study of correlations of Internet flow characteristics , 2006, Comput. Networks.

[2]  Malathi Veeraraghavan,et al.  On How to Provision Quality of Service (QoS) for Large Dataset Transfers , 2013 .

[3]  Konstantinos Psounis,et al.  SIFT : A simple algorithm for tracking elephant flows , and taking advantage of power laws , 2005 .

[4]  Shigeki Goto,et al.  Identifying elephant flows through periodically sampled packets , 2004, IMC '04.

[5]  Richard A. Lethin,et al.  High-performance many-core networking: design and implementation , 2015, INDIS '15.

[6]  Yongzheng Zhang,et al.  Identifying high-rate flows based on Bayesian single sampling , 2010, 2010 2nd International Conference on Computer Engineering and Technology.

[7]  Yi Lu,et al.  ElephantTrap: A low cost device for identifying large flows , 2007, 15th Annual IEEE Symposium on High-Performance Interconnects (HOTI 2007).

[8]  Chita R. Das,et al.  Design of a Dynamic Priority-Based Fast Path Architecture for On-Chip Interconnects , 2007 .

[9]  Richard G. Baraniuk,et al.  Connection-level analysis and modeling of network traffic , 2001, IMW '01.

[10]  Xin-She Yang,et al.  Introduction to Algorithms , 2021, Nature-Inspired Optimization Algorithms.

[11]  Aiko Pras,et al.  A Statistical Analysis of Network Parameters for the Self-management of Lambda-Connections , 2009, AIMS.

[12]  Jordi Ros-Giralt,et al.  A Mathematical Framework for the Detection of Elephant Flows , 2017, ArXiv.

[13]  Walter Willinger,et al.  Self-similarity through high-variability: statistical analysis of Ethernet LAN traffic at the source level , 1997, TNET.