Performance evaluation of cloud systems: A behavioural approach

The concept of Cloud Engineering (CE) as a superset of performance engineering emerges in an extensive number of industrial architectural approaches and implementation flavours. To effectively manage cloud-based systems, it is crucial to monitor, to meter and then to allocate their structural behaviour and performance. In the context of this work, considerable number of experimental measurements are conducted in a distributed virtualised infrastructure and statistically analysed. The paper aims to evaluate cloud systems using multiple open-source benchmarking tools. Also, it addresses and discusses operational deviations of a commercial cloud system in respect to the service level objects (SLOs) per se.

[1]  Arshdeep Bahga,et al.  Performance Evaluation Approach for Multi-Tier Cloud Applications , 2013 .

[2]  Ryszard Kowalczyk,et al.  Smart CloudBench -- Automated Performance Benchmarking of the Cloud , 2013, 2013 IEEE Sixth International Conference on Cloud Computing.

[3]  Bran Selic,et al.  Cost-oriented proactive fault tolerance approach to high performance computing (HPC) in the cloud , 2014, Int. J. Parallel Emergent Distributed Syst..

[4]  Michela Meo,et al.  Probabilistic Consolidation of Virtual Machines in Self-Organizing Cloud Data Centers , 2013, IEEE Transactions on Cloud Computing.

[5]  Diwakar Krishnamurthy,et al.  Performance Testing Web Applications on the Cloud , 2014, 2014 IEEE Seventh International Conference on Software Testing, Verification and Validation Workshops.

[6]  Theo Lynn,et al.  A survey of Cloud monitoring tools: Taxonomy, capabilities and objectives , 2014, J. Parallel Distributed Comput..

[7]  K. Howell.  An Introduction to the Philosophy of Methodology , 2012 .

[8]  Antonio Pescapè,et al.  Cloud monitoring: A survey , 2013, Comput. Networks.

[9]  Yuan-Hsin Tung,et al.  Test as a Service: A Framework for Web Security TaaS Service in Cloud Environment , 2014, 2014 IEEE 8th International Symposium on Service Oriented System Engineering.

[10]  Schahram Dustdar,et al.  Composable cost estimation and monitoring for computational applications in cloud computing environments , 2010, ICCS.

[11]  Jörg Schneider,et al.  Do You Get What You Pay For? Using Proof-of-Work Functions to Verify Performance Assertions in the Cloud , 2010, 2010 IEEE Second International Conference on Cloud Computing Technology and Science.

[12]  S. Seljan.  Digital Agenda for Europe: A Europe 2020 Initiative , 2014 .

[13]  David C. Chou,et al.  Cloud computing: A value creation model , 2015, Comput. Stand. Interfaces.

[14]  Hailong Sun,et al.  Delivering Web service load testing as a service with a global cloud , 2015, Concurr. Comput. Pract. Exp..

[15]  V. Suma,et al.  A Study on Cloud Computing Testing Tools , 2014 .

[16]  Promise Mvelase,et al.  A comparative analysis of pricing models for enterprise cloud platforms , 2013, 2013 Africon.

[17]  Mark Harman,et al.  Cloud engineering is Search Based Software Engineering too , 2013, J. Syst. Softw..

[18]  Wang Yong,et al.  A Configuration Algorithm under Cloud Computing Environment , 2014, 2014 Sixth International Conference on Measuring Technology and Mechatronics Automation.

[19]  John Murphy,et al.  A DSL for Deployment and Testing in the Cloud , 2014, 2014 IEEE Seventh International Conference on Software Testing, Verification and Validation Workshops.