Adaptive Performance Isolation Middleware for Multi-tenant SaaS

SaaS providers typically adopt a multi-tenant architecture to leverage economies of scale. By maximizing the sharing of resources among multiple customer organizations, called tenants, operational costs are reduced. However, this high degree of resource sharing complicates performance isolation between tenants: different tenants have different requirements regarding performance and ensuring compliance with these co-existing and competing performance constraints within a shared application remains a challenge. This paper presents an adaptive middleware that enables SaaS providers to efficiently enforce different and competing performance constraints in multi-tenant SaaS applications. It can manage a combination of performance constraints in terms of latency, throughput and deadlines at a fine-grained level, and enables rapid response on changing circumstances, while preserving the resource usage efficiency of application-level multi-tenancy. We focus on service-oriented applications that asynchronously process small units of work, as supported by many cloud platforms via queues or simple workflows (e.g. Google App Engine, Amazon AWS). We have implemented a prototype on top of a private cloud platform, based on OpenStack and JBoss. The evaluation shows the effectiveness and flexibility of our solution in the context of an industry-relevant SaaS application, with minimal performance overhead.