SCREAM: sketch resource allocation for software-defined measurement

Software-defined networks can enable a variety of concurrent, dynamically instantiated, measurement tasks, that provide fine-grain visibility into network traffic. Recently, there have been many proposals for using sketches for network measurement. However, sketches in hardware switches use constrained resources such as SRAM memory, and the accuracy of measurement tasks is a function of the resources devoted to them on each switch. This paper presents SCREAM, a system for allocating resources to sketch-based measurement tasks that ensures a user-specified minimum accuracy. SCREAM estimates the instantaneous accuracy of tasks so as to dynamically adapt the allocated resources for each task. Thus, by finding the right amount of resources for each task on each switch and correctly merging sketches at the controller, SCREAM can multiplex resources among network-wide measurement tasks. Simulations with three measurement tasks (heavy hitter, hierarchical heavy hitter, and super source/destination detection) show that SCREAM can support more measurement tasks with higher accuracy than existing approaches.

[1]  H. Chernoff.  A Measure of Asymptotic Efficiency for Tests of a Hypothesis Based on the sum of Observations , 1952 .

[2]  Philippe Flajolet,et al.  Probabilistic Counting Algorithms for Data Base Applications , 1985, J. Comput. Syst. Sci..

[3]  Anja Feldmann,et al.  Deriving traffic demands for operational IP networks: methodology and experience , 2000, SIGCOMM.

[4]  Cristian Estan,et al.  New directions in traffic measurement and accounting , 2001, IMW '01.

[5]  Anja Feldmann,et al.  Deriving traffic demands for operational IP networks: methodology and experience , 2001, TNET.

[6]  Moses Charikar,et al.  Finding frequent items in data streams , 2002, Theor. Comput. Sci..

[7]  Balachander Krishnamurthy,et al.  Sketch-based change detection: methods, evaluation, and applications , 2003, IMC '03.

[8]  Divesh Srivastava,et al.  Finding Hierarchical Heavy Hitters in Data Streams , 2003, VLDB.

[9]  Abhishek Kumar,et al.  Data streaming algorithms for efficient and accurate estimation of flow size distribution , 2004, SIGMETRICS '04/Performance '04.

[10]  Yan Chen,et al.  Reversible sketches for efficient and accurate change detection over network data streams , 2004, IMC '04.

[11]  Marios Hadjieleftheriou,et al.  Robust Sketching and Aggregation of Distributed Data Streams , 2005 .

[12]  Graham Cormode,et al.  Space efficient mining of multigraph streams , 2005, PODS.

[13]  Dawn Xiaodong Song,et al.  New Streaming Algorithms for Fast Detection of Superspreaders , 2005, NDSS.

[14]  Graham Cormode,et al.  Summarizing and Mining Skewed Data Streams , 2005, SDM.

[15]  Graham Cormode,et al.  An improved data stream summary: the count-min sketch and its applications , 2004, J. Algorithms.

[16]  Yin Zhang,et al.  Improving sketch reconstruction accuracy using linear least squares method , 2005, IMC '05.

[17]  S. Muthukrishnan,et al.  Modeling skew in data streams , 2006, SIGMOD Conference.

[18]  Vyas Sekar,et al.  LADS: Large-scale Automated DDoS Detection System , 2006, USENIX Annual Technical Conference, General Track.

[19]  P. Flajolet,et al.  HyperLogLog: the analysis of a near-optimal cardinality estimation algorithm , 2007 .

[20]  Andrea Montanari,et al.  Counter braids: a novel counter architecture for per-flow measurement , 2008, SIGMETRICS '08.

[21]  Walter Willinger,et al.  cSamp: A System for Network-Wide Flow Monitoring , 2008, NSDI.

[22]  Junda Liu,et al.  Multi-enterprise networking , 2000 .

[23]  Jing Cao,et al.  Identifying High Cardinality Internet Hosts , 2009, IEEE INFOCOM 2009.

[24]  Amin Vahdat,et al.  Hedera: Dynamic Flow Scheduling for Data Center Networks , 2010, NSDI.

[25]  Jin Cao,et al.  Sequential hashing: A flexible approach for unveiling significant patterns in high speed networks , 2010, Comput. Networks.

[26]  Graham Cormode,et al.  Sketch Techniques for Approximate Query Processing , 2010 .

[27]  Sujata Banerjee,et al.  DevoFlow: scaling flow management for high-performance networks , 2011, SIGCOMM.

[28]  Ping Li,et al.  A New Algorithm for Compressed Counting with Applications in Shannon Entropy Estimation in Dynamic Data , 2011, COLT.

[29]  Graham Cormode,et al.  Mergeable summaries , 2012, PODS '12.

[30]  Ying Zhang,et al.  An adaptive flow counting method for anomaly detection in SDN , 2013, CoNEXT.

[31]  Shicong Meng,et al.  Volley: Violation Likelihood Based State Monitoring for Datacenters , 2013, 2013 IEEE 33rd International Conference on Distributed Computing Systems.

[32]  Minlan Yu,et al.  Software Defined Traffic Measurement with OpenSketch , 2013, NSDI.

[33]  Ramesh Govindan,et al.  Resource/accuracy tradeoffs in software-defined measurement , 2013, HotSDN '13.

[34]  Alexander Hall,et al.  HyperLogLog in practice: algorithmic engineering of a state of the art cardinality estimation algorithm , 2013, EDBT '13.

[35]  Raouf Boutaba,et al.  PayLess: A low cost network monitoring framework for Software Defined Networks , 2014, 2014 IEEE Network Operations and Management Symposium (NOMS).

[36]  Amin Vahdat,et al.  DREAM: dynamic resource allocation for software-defined measurement , 2015, SIGCOMM.

[37]  Soheil Ghiasi,et al.  Streaming Solutions for Fine-Grained Network Traffic Measurements and Analysis , 2011, IEEE/ACM Transactions on Networking.

[38]  Ankit Singla,et al.  OSA: An Optical Switching Architecture for Data Center Networks With Unprecedented Flexibility , 2012, IEEE/ACM Transactions on Networking.