Kaa: Evaluating Elasticity of Cloud-Hosted DBMS

Auto-scaling is able to change the scale of an application at runtime. Understanding the application characteristics, scaling impact as well as the workload, an auto-scaler aligns the acquired resources to match the current workload. For distributed Database Management Systems (DBMS) forming the backend of many large-scale cloud applications, it is currently an open question to what extent they support scaling at run-time. In particular, elasticity properties of existing distributed DBMS are widely unknown and difficult to evaluate and compare. This paper presents a comprehensive methodology for the evaluation of the elasticity of distributed DBMS. On the basis of this methodology, we introduce a framework that automates the full evaluation process. We validate the framework by defining significant elasticity scenarios for a case study that comprises two DBMS for write-heavy and read-heavy workloads of different intensities. The results show that scalable distributed DBMS are not necessarily elastic and that adding more instances to a cluster at run-time may even decrease the experienced performance.

[1]  Sherif Sakr,et al.  Cloud-hosted databases: technologies, challenges and opportunities , 2014, Cluster Computing.

[2]  Alan Fekete,et al.  A Versatile Framework for Painless Benchmarking of Database Management Systems , 2019, ADC.

[3]  Michael I. Jordan,et al.  Characterizing, modeling, and generating workload spikes for stateful services , 2010, SoCC '10.

[4]  Daniel Seybold Towards a framework for orchestrated distributed database evaluation in the cloud , 2017, Middleware 2017.

[5]  Jörn Kuhlenkamp,et al.  Benchmarking Scalability and Elasticity of Distributed Database Systems , 2014, Proc. VLDB Endow..

[6]  Jörg Domaschka,et al.  Is elasticity of scalable databases a Myth? , 2016, 2016 IEEE International Conference on Big Data (Big Data).

[7]  João Paulo,et al.  MeT: workload aware elasticity for NoSQL , 2013, EuroSys '13.

[8]  Samuel Kounev,et al.  Chameleon: A Hybrid, Proactive Auto-Scaling Mechanism on a Level-Playing Field , 2019, IEEE Transactions on Parallel and Distributed Systems.

[9]  Cristina L. Abad,et al.  Quantifying Cloud Performance and Dependability , 2018, ACM Trans. Model. Perform. Evaluation Comput. Syst..

[10]  Ioannis Konstantinou,et al.  On the elasticity of NoSQL databases over cloud management platforms , 2011, CIKM '11.

[11]  David Bermbach,et al.  A Runtime Quality Measurement Framework for Cloud Database Service Systems , 2012, 2012 Eighth International Conference on the Quality of Information and Communications Technology.

[12]  Rajkumar Buyya,et al.  Auto-Scaling Web Applications in Clouds , 2018, ACM Comput. Surv..

[13]  Divyakant Agrawal,et al.  Database Scalability, Elasticity, and Autonomy in the Cloud - (Extended Abstract) , 2011, DASFAA.

[14]  Jörg Domaschka,et al.  A Provider-Agnostic Approach to Multi-cloud Orchestration Using a Constraint Language , 2018, 2018 18th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID).

[15]  Liam O'Brien,et al.  On a Catalogue of Metrics for Evaluating Commercial Cloud Services , 2012, 2012 ACM/IEEE 13th International Conference on Grid Computing.

[16]  Philippe Merle,et al.  Elasticity in Cloud Computing: State of the Art and Research Challenges , 2018, IEEE Transactions on Services Computing.

[17]  Carsten Binnig,et al.  How is the weather tomorrow?: towards a benchmark for the cloud , 2009, DBTest '09.

[18]  Adam Silberstein,et al.  Benchmarking cloud serving systems with YCSB , 2010, SoCC '10.

[19]  Flávio R. C. Sousa,et al.  On defining metrics for elasticity of cloud databases , 2013, SBBD.

[20]  Liu Chen,et al.  A Survey on NoSQL Stores , 2018, ACM Comput. Surv..

[21]  Giannis Verginadis,et al.  A survey on data storage and placement methodologies for Cloud-Big Data ecosystem , 2019, Journal of Big Data.

[22]  Cristina L. Abad,et al.  Methodological Principles for Reproducible Performance Evaluation in Cloud Computing SPEC RG Cloud Working Group , 2019 .

[23]  Ioannis Konstantinou,et al.  Automated, Elastic Resource Provisioning for NoSQL Clusters Using TIRAMOLA , 2013, 2013 13th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing.

[24]  Jim Gray,et al.  Benchmark Handbook: For Database and Transaction Processing Systems , 1992 .

[25]  Jörg Domaschka,et al.  Mowgli: Finding Your Way in the DBMS Jungle , 2019, ICPE.

[26]  Samuel Kounev,et al.  BUNGEE: An Elasticity Benchmark for Self-Adaptive IaaS Cloud Environments , 2015, 2015 IEEE/ACM 10th International Symposium on Software Engineering for Adaptive and Self-Managing Systems.

[27]  Jörg Domaschka,et al.  Beyond IaaS and PaaS: An Extended Cloud Taxonomy for Computation, Storage and Networking , 2013, 2013 IEEE/ACM 6th International Conference on Utility and Cloud Computing.

[28]  Peter Van Roy,et al.  Measuring Elasticity for Cloud Databases , 2011, CLOUD 2011.

[29]  Jörg Domaschka,et al.  Is Distributed Database Evaluation Cloud-Ready? , 2017, ADBIS.