Characterisation of the Effect of a Measurement Library on the Performance of Instrumented Tools

The OMF Measurement Library (OML) is an instrumentation system which enables an experimenter to process any type of measurements from distributed applications and collect them in a unified way. We present a comprehensive study of the performance of this library. The analysis focuses on the behaviour, accuracy and precision of instrumented applications, as well as the background footprint such as CPU load and memory usage. To this end, we have modified typical network measurement tools (Iperf) and libraries (libtrace, libsigar) to use OML as their reporting channel. Following extensive experiments, we find little or no negative impact of OML when comparing the OML-enhanced tools to their original versions. Moreover, in the case of Iperf, when we find significant differences, they are positive, with improvements in the accuracy of both the network probing and jitter measurements. We discuss the implications of using OML in the context of experiment-based networking research and give recommendations on its use and the analysis of the produced results. Mehani et al. Characterisation of the Effect of a Measurement Library. 3

[1]  G. Glass,et al.  Consequences of Failure to Meet Assumptions Underlying the Fixed Effects Analyses of Variance and Covariance , 1972 .

[2]  S. Morley,et al.  Some simple statistical tests for exploring single-case time-series data. , 1989, The British journal of clinical psychology.

[3]  Michael R. Harwell,et al.  Summarizing Monte Carlo Results in Methodological Research: The One- and Two-Factor Fixed Effects ANOVA Cases , 1992 .

[4]  Henning Schulzrinne,et al.  RTP: A Transport Protocol for Real-Time Applications , 1996, RFC.

[5]  Marti J. Anderson,et al.  A new method for non-parametric multivariate analysis of variance in ecology , 2001 .

[6]  Pascale Vicat-Blanc Primet,et al.  Experiments of Network Throughput Measurement and Forecasting Using the Network Weather , 2002, 2nd IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGRID'02).

[7]  Philip F. Chimento,et al.  IP Packet Delay Variation Metric for IP Performance Metrics (IPPM) , 2002, RFC.

[8]  Yevgeni Koucheryavy,et al.  Performance Evaluation of Live Video Streaming Service in 802.11b WLAN Environment under Different Load Conditions , 2003, MIPS.

[9]  Les Cottrell.  Measuring End-To-End Bandwidth with Iperf Using Web100 , 2003 .

[10]  Prasant Mohapatra,et al.  Experimental characterization of multi-hop communications in vehicular ad hoc network , 2005, VANET '05.

[11]  L. Rizzo,et al.  Como: An open infrastructure for network monitoring-research agenda , 2005 .

[12]  KyoungSoo Park,et al.  CoMon: a mostly-scalable monitoring system for PlanetLab , 2006, OPSR.

[13]  Larry L. Peterson,et al.  PlanetFlow: maintaining accountability for network services , 2006, OPSR.

[14]  Anja Feldmann,et al.  Packet Capture in 10-Gigabit Ethernet Environments Using Contemporary Commodity Hardware , 2007, PAM.

[15]  Shaneel Narayan,et al.  Network Performance Analysis of VPN Protocols: An Empirical Comparison on Different Operating Systems , 2009, 2009 International Conference on Networks Security, Wireless Communications and Trusted Computing.

[16]  Jean-Yves Le Boudec.  Performance Evaluation of Computer and Communication Systems , 2010, Computer and communication sciences.

[17]  Maximilian Ott,et al.  OMF: a control and management framework for networking testbeds , 2010, OPSR.

[18]  José Santa,et al.  Design and Experimental Evaluation of a Vehicular Network Based on NEMO and MANET , 2010, EURASIP J. Adv. Signal Process..

[19]  Maximilian Ott,et al.  Measurement Architectures for Network Experiments with Disconnected Mobile Nodes , 2010, TRIDENTCOM.

[20]  Maximilian Ott,et al.  A Portal to Support Rigorous Experimental Methodology in Networking Research , 2011, TRIDENTCOM.

[21]  Samad S. Kolahi,et al.  Performance Monitoring of Various Network Traffic Generators , 2011, 2011 UkSim 13th International Conference on Computer Modelling and Simulation.