Bandwidth Measurements within the Cloud

The search for availability, reliability, and quality of service has led cloud infrastructure customers to disseminate their services, contents, and data over multiple cloud data centers, often involving several Cloud service providers (CSPs). The consequence of this is that a large amount of data must be transmitted across the public Cloud. However, little is known about the bandwidth dynamics involved. To address this, we have conducted a measurement campaign for bandwidth between 18 data centers of four major CSPs. This extensive campaign allowed us to characterize the resulting time series of bandwidth as the addition of a stationary component and some infrequent excursions (typically downtimes). While the former provides a description of the bandwidth users can expect in the Cloud, the latter is closely related to the robustness of the Cloud (i.e., the occurrence of downtimes is correlated). Both components have been studied further by applying factor analysis, specifically analysis of variance, as a mechanism to formally compare data centers’ behaviors and extract generalities. The results show that the stationary process is closely related to the data center locations and CSPs involved in transfers that, fortunately, make the Cloud more predictable and allow the set of reported measurements to be extrapolated. On the other hand, although correlation in the Cloud is low, that is, only 10% of the measured pair of paths showed some correlation, we found evidence that such correlation depends on the particular relationships between pairs of data centers with little connection to more general factors. Positively, this implies that data centers either in the same area or within the same CSP do not show qualitatively more correlation than other data centers, which eases the deployment of robust infrastructures. On the downside, this metric is scarcely generalizable and, consequently, calls for exhaustive monitoring.

[1]  Philipp Leitner,et al.  Patterns in the Chaos—A Study of Performance Variation and Predictability in Public IaaS Clouds , 2014, ACM Trans. Internet Techn..

[2]  Sally Floyd,et al.  Difficulties in simulating the internet , 2001, TNET.

[3]  Antonio Pescapè,et al.  A First Look at Public-Cloud Inter-Datacenter Network Performance , 2016, 2016 IEEE Global Communications Conference (GLOBECOM).

[4]  Sujata Banerjee,et al.  Measuring Bandwidth Between PlanetLab Nodes , 2005, PAM.

[5]  Ajay Mahimkar,et al.  Bandwidth on demand for inter-data center communication , 2011, HotNets-X.

[6]  Israel Cidon,et al.  The power of prediction: cloud bandwidth and cost reduction , 2011, SIGCOMM.

[7]  Antonio Pescapè,et al.  Cloud monitoring: A survey , 2013, Comput. Networks.

[8]  Marco Mellia,et al.  Anomaly detection in diurnal data , 2014, Comput. Networks.

[9]  Ethan Katz-Bassett,et al.  SPANStore: cost-effective geo-replicated storage spanning multiple cloud services , 2013, SOSP.

[10]  T. S. Eugene Ng,et al.  The Impact of Virtualization on Network Performance of Amazon EC2 Data Center , 2010, 2010 Proceedings IEEE INFOCOM.

[11]  Antonio Pescapè,et al.  On Network Throughput Variability in Microsoft Azure Cloud , 2014, 2015 IEEE Global Communications Conference (GLOBECOM).

[12]  Lucas Chaufournier,et al.  CloudNet: Dynamic Pooling of Cloud Resources by Live WAN Migration of Virtual Machines , 2011, IEEE/ACM Transactions on Networking.

[13]  T. S. Eugene Ng,et al.  Application-specific configuration selection in the cloud: Impact of provider policy and potential of systematic testing , 2015, 2015 IEEE Conference on Computer Communications (INFOCOM).

[14]  M. L. Plume,et al.  SPSS (Statistical Package for the Social Sciences) , 2002, Encyclopedia of Information Systems.

[15]  G. Glass,et al.  Consequences of Failure to Meet Assumptions Underlying the Fixed Effects Analyses of Variance and Covariance , 1972 .

[16]  Kimberly Claffy,et al.  Internet traffic characterization , 1994 .

[17]  Christopher Frost,et al.  Spanner: Google's Globally-Distributed Database , 2012, OSDI.

[18]  Marco Mellia,et al.  Exploring the cloud from passive measurements: The Amazon AWS case , 2013, 2013 Proceedings IEEE INFOCOM.

[19]  Michael Sirivianos,et al.  Inter-datacenter bulk transfers with netstitcher , 2011, SIGCOMM.

[20]  Edward Walker,et al.  Benchmarking Amazon EC2 for High-Performance Scientific Computing , 2008, login Usenix Mag..

[21]  Second Edition,et al.  Statistical Package for the Social Sciences , 1970 .

[22]  Aiko Pras,et al.  Gaussian traffic everywhere? , 2006, 2006 IEEE International Conference on Communications.

[23]  Virginia A. Clark,et al.  Applied Statistics: Analysis of Variance and Regression. , 1975 .

[24]  Raid Amin,et al.  Applied Statistics: Analysis of Variance and Regression , 2004, Technometrics.

[25]  Schahram Dustdar,et al.  JCloudScale: Closing the Gap Between IaaS and PaaS , 2014, TOIT.

[26]  A. Karmouch,et al.  Performance analysis of modern TCP variants: A comparison of Cubic, Compound and New Reno , 2010, 2010 25th Biennial Symposium on Communications.

[27]  Harald C. Gall,et al.  Cloud WorkBench: Benchmarking IaaS Providers based on Infrastructure-as-Code , 2015, WWW.

[28]  Prashant Malik,et al.  Cassandra: a decentralized structured storage system , 2010, OPSR.

[29]  Alec Wolman,et al.  Volley: Automated Data Placement for Geo-Distributed Cloud Services , 2010, NSDI.

[30]  Antonio Pescapè,et al.  On the performance of the wide-area networks interconnecting public-cloud datacenters around the globe , 2017, Comput. Networks.

[31]  Zhe Wu,et al.  Understanding the latency benefits of multi-cloud webservice deployments , 2013, CCRV.

[32]  Steve Uhlig,et al.  IP geolocation databases: unreliable? , 2011, CCRV.

[33]  Manish Jain,et al.  End-to-end available bandwidth: measurement methodology, dynamics, and relation with TCP throughput , 2003, TNET.

[34]  Javier Aracil,et al.  Characterization of the busy-hour traffic of IP networks based on their intrinsic features , 2011, Comput. Networks.

[35]  Ray Jain,et al.  The art of computer systems performance analysis - techniques for experimental design, measurement, simulation, and modeling , 1991, Wiley professional computing.

[36]  Hari Balakrishnan,et al.  Resilient overlay networks , 2001, SOSP.

[37]  Xiaowei Yang,et al.  CloudCmp: comparing public cloud providers , 2010, IMC '10.

[38]  Mohit Tawarmalani,et al.  Performance Sensitive Replication in Geo-distributed Cloud Datastores , 2014, 2014 44th Annual IEEE/IFIP International Conference on Dependable Systems and Networks.

[39]  A. Bowman,et al.  Applied smoothing techniques for data analysis : the kernel approach with S-plus illustrations , 1999 .

[40]  Richard D. Deveaux,et al.  Applied Smoothing Techniques for Data Analysis , 1999, Technometrics.

[41]  Rajkumar Buyya,et al.  CloudSim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms , 2011, Softw. Pract. Exp..

[42]  Bo Li,et al.  Jetway: minimizing costs on inter-datacenter video traffic , 2012, ACM Multimedia.

[43]  Balachander Krishnamurthy,et al.  Internet Measurement - Infrastructure, Traffic, and Applications , 2006 .

[44]  Thyaga Nandagopal,et al.  Lowering Inter-datacenter Bandwidth Costs via Bulk Data Scheduling , 2012, 2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (ccgrid 2012).

[45]  Antonio Pescapè,et al.  Measuring network throughput in the cloud: The case of Amazon EC2 , 2015, Comput. Networks.

[46]  José Luis García-Dorado,et al.  Cost-aware Multi Data-Center Bulk Transfers in the Cloud from a Customer-Side Perspective , 2019, IEEE Transactions on Cloud Computing.

[47]  James Algina,et al.  Parametric ANCOVA and the Rank Transform ANCOVA When the Data are Conditionally Non-Normal and Heteroscedastic , 1984 .