The quest for bandwidth estimation techniques for large-scale distributed systems

In recent years the research community has developed many techniques to estimate the end-to-end available bandwidth of an Internet path. This important metric can be potentially exploited to optimize the performance of several distributed systems and, even, to improve the effectiveness of the congestion control mechanism of TCP. Thus, it has been suggested that some existing estimation techniques could be used for this purpose. However, existing tools were not designed for large-scale deployments and were mostly validated in controlled settings, considering only one measurement running at a time. In this paper, we argue that current tools, while offering good estimates when used alone, might not work in large-scale systems where several estimations severely interfere with each other. We analyze the properties of the measurement paradigms employed today and discuss their functioning, study their overhead and analyze their interference. Our testbed results show that current techniques are insufficient as they are. Finally, we will discuss and propose some principles that should be taken into account for including available bandwidth measurements in large-scale distributed systems.

[1]  M. Frans Kaashoek,et al.  A measurement study of available bandwidth estimation tools , 2003, IMC '03.

[2]  Brian Neil Levine,et al.  Evaluation of a novel two-step server selection metric , 2001, Proceedings Ninth International Conference on Network Protocols. ICNP 2001.

[3]  Jasleen Kaur,et al.  RAPID: Shrinking the Congestion-Control Timescale , 2009, IEEE INFOCOM 2009.

[4]  Manish Jain,et al.  Path Selection Using Available Bandwidth Estimation in Overlay-Based Video Streaming , 2007, Networking.

[5]  Richard G. Baraniuk,et al.  pathChirp: Efficient available bandwidth estimation for network paths , 2003 .

[6]  Peter Steenkiste,et al.  Evaluation and characterization of available bandwidth probing techniques , 2003, IEEE J. Sel. Areas Commun..

[7]  Manish Jain,et al.  End-to-end available bandwidth: measurement methodology, dynamics, and relation with TCP throughput , 2002, SIGCOMM 2002.

[8]  Scott Shenker,et al.  Endpoint admission control: architectural issues and performance , 2000, SIGCOMM.

[9]  Zhao Wen-tao,et al.  Efficient available bandwidth estimation for network paths , 2008 .

[10]  Guillaume Urvoy-Keller,et al.  Operational comparison of available bandwidth estimation tools , 2008, CCRV.

[11]  M. Y. Sanadidi,et al.  The probe gap model can underestimate the available bandwidth of multihop paths , 2006, CCRV.

[12]  Anees Shaikh,et al.  An empirical evaluation of wide-area internet bottlenecks , 2003, IMC '03.

[13]  Alok Shriram,et al.  Empirical Evaluation of Techniques for Measuring Available Bandwidth , 2007, IEEE INFOCOM 2007 - 26th IEEE International Conference on Computer Communications.

[14]  Dmitri Loguinov,et al.  A stochastic foundation of available bandwidth estimation: multi-hop analysis , 2008, IEEE/ACM Trans. Netw..

[15]  Parameswaran Ramanathan,et al.  What do packet dispersion techniques measure? , 2001, Proceedings IEEE INFOCOM 2001. Conference on Computer Communications. Twentieth Annual Joint Conference of the IEEE Computer and Communications Society (Cat. No.01CH37213).

[16]  Manish Jain,et al.  Ten fallacies and pitfalls on end-to-end available bandwidth estimation , 2004, IMC '04.

[17]  Michael Hicks,et al.  Passive aggressive measurement with MGRP , 2009, SIGCOMM '09.

[18]  Abraham Silberschatz,et al.  P4p: provider portal for applications , 2008, SIGCOMM '08.

[19]  Kimberly C. Claffy,et al.  Comparison of Public End-to-End Bandwidth Estimation Tools on High-Speed Links , 2005, PAM.

[20]  Manish Jain,et al.  End-to-end available bandwidth: measurement methodology, dynamics, and relation with TCP throughput , 2003, TNET.

[21]  Guillaume Urvoy-Keller,et al.  Fast Available Bandwidth Sampling for ADSL Links: Rethinking the Estimation for Larger-Scale Measurements , 2009, PAM.