Handling High Speed Traffic Measurement Using Network Processors

Traffic measurement management, accounting, worm detection, etc. It becomes more and more difficult today and cannot keep pace with the increasing speed of today's Internet. Many new algorithms and hardware have been proposed to solve this problem. Network Processors, known for its nice tradeoff between performance and programming flexibility, are chosen as the platform. The goal of this paper is to find a suitable algorithm based on NPs to handle high speed network traffic measurement. Four algorithms, Raw measurement, Sampling, Multi-stage Filter and Multi-Resolution Space Code Bloom Filter (MRSCBF) are implemented and evaluated on Intel's IXP 2400 network processor. The results reveal that Sampling and Multi-stage Filters can fully exploit the parallel and heterogeneous architecture of network processor, so are suitable for high speed network traffic measurement on the network processor platform. MRSCBF, on the other hand, is not so efficient on network processors because of its complex process and frequent memory access.

[1]  George Varghese,et al.  New directions in traffic measurement and accounting: Focusing on the elephants, ignoring the mice , 2003, TOCS.

[2]  George Varghese,et al.  Building a better NetFlow , 2004, SIGCOMM 2004.

[3]  Edward W. Knightly,et al.  Measurement-based admission control with aggregate traffic envelopes , 2001, TNET.

[4]  Zhi-Li Zhang,et al.  Adaptive random sampling for load change detection , 2002, SIGMETRICS '02.

[5]  Richard M. Karp,et al.  A simple algorithm for finding frequent elements in streams and bags , 2003, TODS.

[6]  Carsten Lund,et al.  Estimating flow distributions from sampled flow statistics , 2003, SIGCOMM '03.

[7]  Haoyu Song,et al.  Fast hash table lookup using extended bloom filter: an aid to network processing , 2005, SIGCOMM '05.

[8]  George Varghese,et al.  Efficient implementation of a statistics counter architecture , 2003, SIGMETRICS '03.

[9]  George Varghese,et al.  Bitmap algorithms for counting active flows on high speed links , 2003, IMC '03.

[10]  George Bakos,et al.  Early detection of Internet worm activity by metering ICMP destination unreachable messages , 2002, SPIE Defense + Commercial Sensing.

[11]  Abhishek Kumar,et al.  Space-code bloom filter for efficient per-flow traffic measurement , 2004, IEEE INFOCOM 2004.

[12]  Burton H. Bloom,et al.  Space/time trade-offs in hash coding with allowable errors , 1970, CACM.

[13]  Thomas M. Chen,et al.  Worm epidemics in high-speed networks , 2004, Computer.

[14]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[15]  Larry Peterson,et al.  Inter-AS traffic patterns and their implications , 1999, Seamless Interconnection for Universal Services. Global Telecommunications Conference. GLOBECOM'99. (Cat. No.99CH37042).

[16]  David Moore,et al.  Internet quarantine: requirements for containing self-propagating code , 2003, IEEE INFOCOM 2003. Twenty-second Annual Joint Conference of the IEEE Computer and Communications Societies (IEEE Cat. No.03CH37428).