Profit-Based Experimental Analysis of IaaS Cloud Performance: Impact of Software Resource Allocation

High resource utilization is an important goal in achieving high return on investment in cloud environments. Guaranteed quality of service (QoS) is an important goal for web-facing applications such as e-commerce. Achieving both high utilization and high QoS simultaneously is a significant challenge, since high utilization often implies more QoS failures such as long response times. In this paper, we adopt a profit model based on response time (i.e., decreasing or negative revenues for increasing query answer response time) to represent the QoS requirements. Our data shows that such a profit model often yields different analytical results compared to traditional performance metrics such as average throughput. Using extensive experimental measurements (on the same hardware platform and software stack) of the standard RUBBoS n-tier benchmark, we study the impact of different allocations of software resources such as the size of thread pools in various servers in an n-tier system. First, the profit model emphasizes the importance of appropriate allocations, showing a difference of up to 48.6% when system utilization is high (over 80%). Second, our experiments show that over-allocation of thread pool may lead to unnecessary consumption of critical resources (e.g., CPU) that reduce profits by up to 84.8%. Third, we found that under-allocation of thread pool in one server may lead to under-utilization of several servers downstream in an n-tier system, also reducing profits by up to 52.8%. Our data shows that the best allocation depends on several system parameters, including resource availability. We designed an adaptive algorithm to find the best allocations and show its effectiveness through our experiments and analyses.