Per flow packet sampling for high-speed network monitoring

We present a per-flow packet sampling method that enables the real-time classification of high-speed network traffic. Our method, based upon the partial sampling of each flow (i.e., performing sampling at only early stages in each flow's lifetime), provides a sufficient reduction in total traffic (e.g., a factor of five in packets, a factor of ten in bytes) as to allow practical implementations at one Gigabit/s, and, using limited hardware assistance, ten Gigabit/s.

[1]  Maurizio Dusi,et al.  Traffic classification through simple statistical fingerprinting , 2007, CCRV.

[2]  Michael Mitzenmacher,et al.  Less Hashing, Same Performance: Building a Better Bloom Filter , 2006, ESA.

[3]  Cristian Estan,et al.  New directions in traffic measurement and accounting , 2001, IMW '01.

[4]  Sebastian Zander,et al.  A preliminary performance comparison of five machine learning algorithms for practical IP traffic flow classification , 2006, CCRV.

[5]  Li Fan,et al.  Summary cache: a scalable wide-area web cache sharing protocol , 2000, TNET.

[6]  Vern Paxson,et al.  The shunt: an FPGA-based accelerator for network intrusion prevention , 2007, FPGA '07.

[7]  Anja Feldmann,et al.  Building a time machine for efficient recording and retrieval of high-volume network traffic , 2005, IMC '05.

[8]  Jonathan D. Cryer,et al.  Time Series Analysis , 1986 .

[9]  Michalis Faloutsos,et al.  Is P2P dying or just hiding? [P2P traffic measurement] , 2004, IEEE Global Telecommunications Conference, 2004. GLOBECOM '04..

[10]  Abhishek Kumar,et al.  Space-code bloom filter for efficient per-flow traffic measurement , 2004, IEEE INFOCOM 2004.

[11]  Renata Teixeira,et al.  Early application identification , 2006, CoNEXT '06.

[12]  Andrew W. Moore,et al.  A Machine Learning Approach for Efficient Traffic Classification , 2007, 2007 15th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems.

[13]  Haoyu Song,et al.  Fast packet classification using bloom filters , 2006, 2006 Symposium on Architecture For Networking And Communications Systems.

[14]  Michael Mitzenmacher,et al.  Less hashing, same performance: Building a better Bloom filter , 2006, Random Struct. Algorithms.

[15]  Larry Carter,et al.  Universal classes of hash functions (Extended Abstract) , 1977, STOC '77.

[16]  Panagiotis Manolios,et al.  Bloom Filters in Probabilistic Verification , 2004, FMCAD.

[17]  R. Bolla,et al.  Characterizing the network behavior of P2P traffic , 2008, 2008 4th International Telecommunication Networking Workshop on QoS in Multiservice IP Networks.

[18]  Andrei Broder,et al.  Network Applications of Bloom Filters: A Survey , 2004, Internet Math..

[19]  Balachander Krishnamurthy,et al.  Sketch-based change detection: methods, evaluation, and applications , 2003, IMC '03.

[20]  Martin Roesch,et al.  Snort - Lightweight Intrusion Detection for Networks , 1999 .

[21]  A. Kumar,et al.  Space-code bloom filter for efficient per-flow traffic measurement , 2004, IEEE INFOCOM 2004.

[22]  Nick Duffield,et al.  Sampling for Passive Internet Measurement: A Review , 2004 .

[23]  George Varghese,et al.  New directions in traffic measurement and accounting , 2002, CCRV.