Performance Isolation in Multi-Tenant Applications

The thesis presents methods to isolate different tenants, sharing one application instance, with regards to he performance they observe. Therefore, a request based admission control is introduced. Furthermore, the publication presents methods and novel metrics to evaluate the degree of isolation a system achieves. These insights are used to evaluate the developed isolation methods, resulting in recommendations of methods for various scenarios.

[1]  Samuel Kounev,et al.  Automated simulation-based capacity planning for enterprise data fabrics , 2011, SimuTools.

[2]  Slinger Jansen,et al.  Defining multi-tenancy: A systematic mapping study on the academic and the industrial perspective , 2015, J. Syst. Softw..

[3]  Srikumar Venugopal,et al.  Modeling Performance of Elasticity Rules for Cloud-Based Applications , 2013, 2013 17th IEEE International Enterprise Distributed Object Computing Conference.

[4]  Christof Momm,et al.  Using software performance curves for dependable and cost-efficient service hosting , 2010, QUASOSS '10.

[5]  Chenyang Lu,et al.  Introduction to Control Theory And Its Application to Computing Systems , 2008 .

[6]  Randy H. Katz,et al.  Above the Clouds: A Berkeley View of Cloud Computing , 2009 .

[7]  Ajay Gulati.  VMware distributed resource Management : design , Implementation , and lessons learned , 2022 .

[8]  Frank Leymann,et al.  A Framework for Optimized Distribution of Tenants in Cloud Applications , 2010, 2010 IEEE 3rd International Conference on Cloud Computing.

[9]  Thomas Sandholm,et al.  What's inside the Cloud? An architectural map of the Cloud landscape , 2009, 2009 ICSE Workshop on Software Engineering Challenges of Cloud Computing.

[10]  Flávio R. C. Sousa,et al.  Towards Elastic Multi-Tenant Database Replication with Quality of Service , 2012, 2012 IEEE Fifth International Conference on Utility and Cloud Computing.

[11]  Rouven Krebs,et al.  Metrics and techniques for quantifying performance isolation in cloud environments , 2012, QoSA '12.

[12]  Samuel Kounev,et al.  Experimental Evaluation of the Performance-Influencing Factors of Virtualized Storage Systems , 2012, EPEW/UKPEW.

[13]  Krishna Kant,et al.  Overload Control Mechanisms for Web Servers , 2001 .

[14]  Bo Gao,et al.  A Study and Performance Evaluation of the Multi-Tenant Data Tier Design Patterns for Service Oriented Computing , 2008, 2008 IEEE International Conference on e-Business Engineering.

[15]  Rajarshi Das,et al.  Utility functions in autonomic systems , 2004, International Conference on Autonomic Computing, 2004. Proceedings..

[16]  Renee Koen,et al.  ASPECTS OF MCDA CLASSIFICATION AND SORTING METHODS , 2008 .

[17]  Cor-Paul Bezemer,et al.  Multi-tenant SaaS applications: maintenance dream or nightmare? , 2010, IWPSE-EVOL '10.

[18]  Rouven Krebs,et al.  Multi-tenancy Performance Benchmark for Web Application Platforms , 2013, ICWE.

[19]  Samuel Kounev,et al.  Evaluating and Modeling Virtualization Performance Overhead for Cloud Environments , 2011, CLOSER.

[20]  P. Mell,et al.  The NIST Definition of Cloud Computing , 2011 .

[21]  C. Martin.  2015 , 2015, Les 25 ans de l’OMC: Une rétrospective en photos.

[22]  Alexandru Iosup,et al.  On the Performance Variability of Production Cloud Services , 2011, 2011 11th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing.

[23]  Virgílio A. F. Almeida,et al.  Capacity Planning and Performance Modeling: From Mainframes to Client-Server Systems , 1994 .

[24]  Conceptual Approach for Performance Isolation in Multi-tenant Systems , 2013, CLOSER.

[25]  Jorge-Arnulfo Quiané-Ruiz,et al.  Runtime measurements in the cloud , 2010, Proc. VLDB Endow..

[26]  Dean Jacobs,et al.  Ruminations on Multi-Tenant Databases , 2007, BTW.

[27]  Connie U. Smith,et al.  Software Performance Engineering: A Case Study Including Performance Comparison with Design Alternatives , 1993, IEEE Trans. Software Eng..

[28]  Qian Wang,et al.  Feedback performance control for computer systems: an LPV approach , 2005, Proceedings of the 2005, American Control Conference, 2005..

[29]  Frank Leymann,et al.  Defining Composite Configurable SaaS Application Packages Using SCA, Variability Descriptors and Multi-tenancy Patterns , 2008, 2008 Third International Conference on Internet and Web Applications and Services.

[30]  Xiaoyun Zhu,et al.  Triage: Performance differentiation for storage systems using adaptive control , 2005, TOS.

[31]  Jerome A. Rolia,et al.  Parameter estimation for performance models of distributed application systems , 1995, CASCON.

[32]  Amin Vahdat,et al.  Dynamic Scheduling of Virtual Machines Running HPC Workloads in Scientific Grids , 2007, 2009 3rd International Conference on New Technologies, Mobility and Security.

[33]  Craig D. Weissman,et al.  The design of the force.com multitenant internet application development platform , 2009, SIGMOD Conference.

[34]  Peter Van Roy,et al.  Measuring Elasticity for Cloud Databases , 2011, CLOUD 2011.

[35]  Bo Gao,et al.  An Effective Heuristic for On-line Tenant Placement Problem in SaaS , 2010, 2010 IEEE International Conference on Web Services.

[36]  Uwe Aßmann,et al.  Towards modeling a variable architecture for multi-tenant SaaS-applications , 2012, VaMoS.

[37]  Gregory J. Barlow,et al.  Article in Press Robotics and Autonomous Systems ( ) – Robotics and Autonomous Systems Fitness Functions in Evolutionary Robotics: a Survey and Analysis , 2022 .

[38]  Asser N. Tantawi,et al.  CPU demand for web serving: Measurement analysis and dynamic estimation , 2008, Perform. Evaluation.

[39]  Yixin Diao,et al.  Feedback Control of Computing Systems , 2004 .

[40]  Kevin Lee,et al.  How a consumer can measure elasticity for cloud platforms , 2012, ICPE '12.

[41]  Rini van Solingen,et al.  Goal Question Metric (GQM) Approach , 2002 .

[42]  Yixin Diao,et al.  A first-principles approach to constructing transfer functions for admission control in computing systems , 2002, Proceedings of the 41st IEEE Conference on Decision and Control, 2002..

[43]  Richard Talaber,et al.  USING VIRTUALIZATION TO IMPROVE DATA CENTER EFFICIENCY , 2009 .

[44]  Frank Leymann,et al.  Combining Different Multi-tenancy Patterns in Service-Oriented Applications , 2009, 2009 IEEE International Enterprise Distributed Object Computing Conference.

[45]  M. Pors,et al.  A Decision Support Model for Multi-Tenant Architectures , 2013 .

[46]  Mikko H. Lipasti,et al.  An architectural evaluation of Java TPC-W , 2001, Proceedings HPCA Seventh International Symposium on High-Performance Computer Architecture.

[47]  Manuel Loesch,et al.  Locations for Performance Ensuring Admission Control in Load Balanced Multi-tenant Systems , 2013, CLOSER.

[48]  Stephen Dawson,et al.  Resource demand modeling for multi-tier services , 2010, WOSP/SIPEW '10.

[49]  Samuel Kounev,et al.  Analysis of the Performance-Influencing Factors of Virtualization Platforms , 2010, OTM Conferences.

[50]  Marin Litoiu,et al.  Performance Model Estimation and Tracking Using Optimal Filters , 2008, IEEE Transactions on Software Engineering.

[51]  Myron Hlynka,et al.  Queueing Networks and Markov Chains (Modeling and Performance Evaluation With Computer Science Applications) , 2007, Technometrics.

[52]  Diwakar Krishnamurthy,et al.  RPO: Runtime web server optimization under simultaneous multithreading , 2013, 2013 IFIP/IEEE International Symposium on Integrated Network Management (IM 2013).

[53]  Samuel Kounev,et al.  Automated extraction of palladio component models from running enterprise Java applications , 2009, VALUETOOLS.

[54]  Rouven Krebs,et al.  Architectural Concerns in Multi-tenant SaaS Applications , 2012, CLOSER.

[55]  Pierre L'Ecuyer,et al.  Simulation in Java with SSJ , 2005, Proceedings of the Winter Simulation Conference, 2005..

[56]  Peter J. Denning,et al.  The Operational Analysis of Queueing Network Models , 1978, CSUR.

[57]  Madhu Goyal,et al.  Multi-tenant Database Access Control , 2013, 2013 IEEE 16th International Conference on Computational Science and Engineering.

[58]  Samuel Kounev,et al.  Elasticity in Cloud Computing: What It Is, and What It Is Not , 2013, ICAC.

[59]  Adam Silberstein,et al.  Benchmarking cloud serving systems with YCSB , 2010, SoCC '10.

[60]  David E. Culler,et al.  USENIX Association Proceedings of USITS ’ 03 : 4 th USENIX Symposium on Internet Technologies and Systems , 2003 .

[61]  Roberto Turrin,et al.  Robust Workload Estimation in Queueing Network Performance Models , 2008, 16th Euromicro Conference on Parallel, Distributed and Network-Based Processing (PDP 2008).

[62]  Sang Hyuk Son,et al.  Feedback Control Architecture and Design Methodology for Service Delay Guarantees in Web Servers , 2006, IEEE Transactions on Parallel and Distributed Systems.

[63]  Moreno Marzolla,et al.  Dynamic resource provisioning for cloud-based gaming infrastructures , 2012, CIE.

[64]  Bo Gao,et al.  A Framework for Native Multi-Tenancy Application Development and Management , 2007, The 9th IEEE International Conference on E-Commerce Technology and The 4th IEEE International Conference on Enterprise Computing, E-Commerce and E-Services (CEC-EEE 2007).

[65]  Samuel Kounev,et al.  Automated extraction of architecture-level performance models of distributed component-based systems , 2011, 2011 26th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011).

[66]  Stephen Dawson,et al.  Estimating service resource consumption from response time measurements , 2009, VALUETOOLS.

[67]  Carey L. Williamson,et al.  A workload characterization study of Internet Web servers , 1995, SIAP.

[68]  Jun Wei,et al.  Application-Level CPU Consumption Estimation: Towards Performance Isolation of Multi-tenancy Web Applications , 2012, 2012 IEEE Fifth International Conference on Cloud Computing.

[69]  Ramez Elmasri,et al.  Fundamentals of Database Systems , 1989 .

[70]  Rouven Krebs,et al.  Resource Usage Control in Multi-tenant Applications , 2014, 2014 14th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing.

[71]  Qi Zhang,et al.  A Regression-Based Analytic Model for Dynamic Resource Provisioning of Multi-Tier Applications , 2007, Fourth International Conference on Autonomic Computing (ICAC'07).

[72]  Paula Smith,et al.  VMmark: A Scalable Benchmark for Virtualized Systems , 2006 .

[73]  Jeanna Neefe Matthews,et al.  Quantifying the performance isolation properties of virtualization systems , 2007, ExpCS '07.

[74]  Bo Gao,et al.  A Non-intrusive Multi-tenant Database Software for Large Scale SaaS Application , 2011, 2011 IEEE 8th International Conference on e-Business Engineering.

[75]  C. Murray Woodside,et al.  Using regression splines for software performance analysis , 2000, WOSP '00.

[76]  Ying Chen,et al.  SPIN: Service Performance Isolation Infrastructure in Multi-tenancy Environment , 2008, ICSOC.

[77]  Rouven Krebs,et al.  Optimization Method for Request Admission Control to Guarantee Performance Isolation , 2014, HotTopiCS '14.

[78]  Wouter Joosen,et al.  Towards performance isolation in multi-tenant SaaS applications , 2012, MW4NG '12.

[79]  Carsten Binnig,et al.  How is the weather tomorrow?: towards a benchmark for the cloud , 2009, DBTest '09.

[80]  Anees Shaikh,et al.  Performance Isolation and Fairness for Multi-Tenant Cloud Storage , 2012, OSDI.

[82]  Amin Vahdat,et al.  Enforcing Performance Isolation Across Virtual Machines in Xen , 2006, Middleware.

[83]  Sheldon M. Ross,et al.  Introduction to Probability Models, Eighth Edition , 1972 .

[84]  Jóakim von Kistowski,et al.  How to Build a Benchmark , 2015, ICPE.

[85]  Egon Berghout,et al.  Integrating goal-oriented measurement in industrial software engineering: industrial experiences with and additions to the Goal/Question/Metric method (GQM) , 2001, Proceedings Seventh International Software Metrics Symposium.

[86]  Samuel Kounev,et al.  QPME: a performance modeling tool based on queueing Petri Nets , 2009, PERV.

[87]  Jignesh M. Patel,et al.  Towards Multi-Tenant Performance SLOs , 2012, IEEE Transactions on Knowledge and Data Engineering.

[88]  Dan Feng,et al.  An Improved Xen Credit Scheduler for I/O Latency-Sensitive Applications on Multicores , 2013, 2013 International Conference on Cloud Computing and Big Data.

[89]  Jock D. Mackinlay,et al.  The information visualizer, an information workspace , 1991, CHI.

[90]  Alan M. Davis,et al.  Software Requirements: Objects, Functions and States , 1993 .

[91]  Marko Becker.  Performance By Design Computer Capacity Planning By Example , 2016 .

[92]  Asser N. Tantawi,et al.  Real-time performance modeling for adaptive software systems , 2009, VALUETOOLS.

[93]  Johan Tordsson,et al.  An adaptive hybrid elasticity controller for cloud infrastructures , 2012, 2012 IEEE Network Operations and Management Symposium.

[94]  Peter J. Varman,et al.  mClock: Handling Throughput Variability for Hypervisor IO Scheduling , 2010, OSDI.

[95]  Samuel Kounev,et al.  BUNGEE: An Elasticity Benchmark for Self-Adaptive IaaS Cloud Environments , 2015, 2015 IEEE/ACM 10th International Symposium on Software Engineering for Adaptive and Self-Managing Systems.

[96]  Alexandru Iosup,et al.  Performance Analysis of Cloud Computing Services for Many-Tasks Scientific Computing , 2011, IEEE Transactions on Parallel and Distributed Systems.

[97]  Kang G. Shin,et al.  What does control theory bring to systems research? , 2009, OPSR.

[98]  Guilherme Galante,et al.  A Survey on Cloud Computing Elasticity , 2012, 2012 IEEE Fifth International Conference on Utility and Cloud Computing.

[99]  Venkataraman Ramesh,et al.  Research in software engineering: an analysis of the literature , 2002, Inf. Softw. Technol..

[100]  Barry W. Boehm,et al.  Quantitative evaluation of software quality , 1976, ICSE '76.

[101]  Rouven Krebs,et al.  A Qualitative Discussion of Different Approaches for Implementing Multi-Tenant SaaS Offerings , 2011, Software Engineering.

[102]  Jens Happe,et al.  The Performance Cockpit Approach: A Framework For Systematic Performance Evaluations , 2010, 2010 36th EUROMICRO Conference on Software Engineering and Advanced Applications.

[103]  Manuel Loesch,et al.  Comparison of Request Admission Based Performance Isolation Approaches in Multi-tenant SaaS Applications , 2014, CLOSER.

[104]  Kang G. Shin,et al.  Automated control of multiple virtualized resources , 2009, EuroSys '09.

[105]  T. Bittman,et al.  Magic Quadrant for x 86 Server Virtualization Infrastructure , 2010 .

[106]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[107]  Xue Liu,et al.  Optimal multivariate control for differentiated services on a shared hosting platform , 2007, 2007 46th IEEE Conference on Decision and Control.

[108]  Jeffrey S. Chase,et al.  Cutting Corners: Workbench Automation for Server Benchmarking , 2008, USENIX Annual Technical Conference.

[109]  Frank Leymann,et al.  Cloud Computing Patterns: Fundamentals to Design, Build, and Manage Cloud Applications , 2014 .

[110]  Manuel Loesch,et al.  Platform-as-a-Service Architecture for Performance Isolated Multi-tenant Applications , 2014, 2014 IEEE 7th International Conference on Cloud Computing.

[111]  Austen Rainer,et al.  Case Study Research in Software Engineering - Guidelines and Examples , 2012 .

[112]  Yanbo Han,et al.  Feedback-Control-Based Performance Regulation for Multi-Tenant Applications , 2009, 2009 15th International Conference on Parallel and Distributed Systems.

[113]  Steffen Becker,et al.  Quantitative Evaluation of Model-Driven Performance Analysis and Simulation of Component-Based Architectures , 2015, IEEE Transactions on Software Engineering.

[114]  Torsten Grust,et al.  Multi-tenant databases for software as a service: schema-mapping techniques , 2008, SIGMOD Conference.

[115]  Jóakim von Kistowski,et al.  Defining and Quantifying Elasticity of Resources in Cloud Computing and Scalable Platforms , 2011 .

[116]  Divyakant Agrawal,et al.  Albatross: Lightweight Elasticity in Shared Storage Databases for the Cloud using Live Data Migration , 2011, Proc. VLDB Endow..

[117]  Daniel A. Menascé,et al.  Scaling for E-Business: Technologies, Models, Performance, and Capacity Planning , 2000 .

[118]  Chenyang Lu,et al.  RT-Xen: Towards real-time hypervisor scheduling in Xen , 2011, 2011 Proceedings of the Ninth ACM International Conference on Embedded Software (EMSOFT).

[119]  Yixin Diao,et al.  Optimizing Quality of Service Using Fuzzy Control , 2002, DSOM.

[120]  Rouven Krebs,et al.  A Feedback Controlled Scheduler for Performance Isolation in Multi-Tenant Applications , 2013, 2013 International Conference on Cloud and Green Computing.