Spectral Models for Bitrate Measurement from Packet Sampled Traffic

In network measurement systems, packet sampling techniques are usually adopted to reduce the overall amount of data to collect and process. Being based on a subset of packets, they introduce estimation errors that have to be properly counteracted by using a fine tuning of the sampling strategy and sophisticated inversion methods. This problem has been deeply investigated in the literature with particular attention to the statistical properties of packet sampling and to the recovery of the original network measurements. Herein, we propose a novel approach to predict the energy of the sampling error in the real time estimation of traffic bitrate, based on spectral analysis in the frequency domain. We start by demonstrating that the error introduced by packet sampling can be modeled as an aliasing effect in the frequency domain. Then, we derive closed-form expressions for the Signal-to-Noise Ratio (SNR) to predict the distortion of traffic bitrate estimates over time. The accuracy of the proposed SNR metric is validated by means of real packet traces. Furthermore, a comparison with respect to an analogous SNR expression derived using classic stochastic tools is proposed, showing that the frequency domain approach grants for a higher accuracy when traffic rate measurements are carried out at fine time granularity.

[1]  kc claffy,et al.  Application of sampling methodologies to network traffic characterization , 1993, SIGCOMM 1993.

[2]  Abhishek Kumar,et al.  Sketch Guided Sampling - Using On-Line Estimates of Flow Size for Adaptive Data Collection , 2006, Proceedings IEEE INFOCOM 2006. 25TH IEEE International Conference on Computer Communications.

[3]  Ramesh Govindan,et al.  ASTUTE: detecting a different class of traffic anomalies , 2010, SIGCOMM '10.

[4]  Fang Hao,et al.  Fast, Memory Efficient Flow Rate Estimation Using Runs , 2007, IEEE/ACM Transactions on Networking.

[5]  Chadi Barakat,et al.  A Frequency Domain Model to Predict the Estimation Accuracy of Packet Sampling , 2010, 2010 Proceedings IEEE INFOCOM.

[6]  Zhi-Li Zhang,et al.  Adaptive packet sampling for accurate and scalable flow measurement , 2004, IEEE Global Telecommunications Conference, 2004. GLOBECOM '04..

[7]  Paul Barford,et al.  A signal analysis of network traffic anomalies , 2002, IMW '02.

[8]  Nick G. Duffield,et al.  Trajectory Sampling With Unreliable Reporting , 2004, IEEE/ACM Transactions on Networking.

[9]  Eddie Kohler,et al.  MultiQ: automated detection of multiple bottleneck capacities along a path , 2004, IMC '04.

[10]  Nick Feamster,et al.  Fast monitoring of traffic subpopulations , 2008, IMC '08.

[11]  Chadi Barakat,et al.  Reformulating the Monitor Placement Problem: Optimal Network-Wide Sampling , 2006, 2006 40th Annual Conference on Information Sciences and Systems.

[12]  Donald F. Towsley,et al.  Fisher information of sampled packets: an application to flow size estimation , 2006, IMC '06.

[13]  Martin May,et al.  Impact of packet sampling on anomaly detection metrics , 2006, IMC '06.

[14]  George Varghese,et al.  New directions in traffic measurement and accounting: Focusing on the elephants, ignoring the mice , 2003, TOCS.

[15]  George Varghese,et al.  Building a better NetFlow , 2004, SIGCOMM 2004.

[16]  Nicolas Hohn,et al.  Inverting sampled traffic , 2003, IMC '03.

[17]  F. Harris.  On the use of windows for harmonic analysis with the discrete Fourier transform , 1978, Proceedings of the IEEE.

[18]  Mostafa H. Ammar,et al.  Spectral probing, crosstalk and frequency multiplexing in internet paths , 2008, IMC '08.

[19]  Chadi Barakat,et al.  An analysis of packet sampling in the frequency domain , 2009, IMC '09.

[20]  Carsten Lund,et al.  Estimating flow distributions from sampled flow statistics , 2003, SIGCOMM '03.

[21]  Stéphane Girard,et al.  Maximum likelihood estimation of the flow size distribution tail index from sampled packet data , 2009, SIGMETRICS '09.

[22]  Carsten Lund,et al.  Learn more, sample less: control of volume and variance in network measurement , 2005, IEEE Transactions on Information Theory.

[23]  Nick Duffield.  Status of this Memo A Framework for Packet Selection and Reporting , 2003 .

[24]  John G. Proakis,et al.  Digital signal processing (3rd ed.): principles, algorithms, and applications , 1996 .

[25]  T. V. Lakshman,et al.  Fast, memory efficient flow rate estimation using runs , 2007, TNET.

[26]  Shigeki Goto,et al.  Identifying elephant flows through periodically sampled packets , 2004, IMC '04.