Cloud WorkBench: Benchmarking IaaS Providers based on Infrastructure-as-Code

Optimizing the deployment of applications in Infrastructure-as-a-Service clouds requires to evaluate the costs and performance of different combinations of cloud configurations which is unfortunately a cumbersome and error-prone process. In this paper, we present Cloud WorkBench (CWB), a concrete implementation of a cloud benchmarking Web service, which fosters the definition of reusable and representative benchmarks. We demonstrate the complete cycle of benchmarking an IaaS service with the sample benchmark SysBench. In distinction to existing work, our system is based on the notion of Infrastructure-as-Code, which is a state of the art concept to define IT infrastructure in a reproducible, well-defined, and testable way.

[1]  Nabor das Chagas Mendonça,et al.  A Declarative Environment for Automatic Performance Evaluation in IaaS Clouds , 2013, 2013 IEEE Sixth International Conference on Cloud Computing.

[2]  Michael Httermann,et al.  DevOps for Developers , 2012 .

[3]  Michael Hüttermann.  DevOps for Developers , 2012, Apress.

[4]  Bin Li,et al.  Fair Benchmarking for Cloud Computing systems , 2012, Journal of Cloud Computing: Advances, Systems and Applications.

[5]  Benjamin Farley,et al.  More for your money: exploiting performance heterogeneity in public clouds , 2012, SoCC '12.

[6]  Philipp Leitner,et al.  Patterns in the Chaos—A Study of Performance Variation and Predictability in Public IaaS Clouds , 2014, ACM Trans. Internet Techn..

[7]  Calton Pu,et al.  An automated approach to create, store, and analyze large-scale experimental data in clouds , 2013, 2013 IEEE 14th International Conference on Information Reuse & Integration (IRI).

[8]  Dilma Da Silva,et al.  CloudBench: Experiment Automation for Cloud Environments , 2013, 2013 IEEE International Conference on Cloud Engineering (IC2E).

[9]  P. Mell,et al.  The NIST Definition of Cloud Computing , 2011 .

[10]  J Jardine.  More for your money , 1971 .

[11]  Harald C. Gall,et al.  Cloud Work Bench -- Infrastructure-as-Code Based Cloud Benchmarking , 2014, 2014 IEEE 6th International Conference on Cloud Computing Technology and Science.