Should infrastructure clouds be priced entirely on performance? An EC2 case study

The increasing number of public clouds, the large and varied range of VMs they offer, and the provider specific terminology used for describing performance characteristics, makes price/performance comparisons difficult. Large performance variation of identically priced instances can lead to clouds being described as ‘unreliable’ and ‘unpredictable’. In this paper, we suggest that instances might be considered mispriced with respect to their deliverable performance – even when provider supplied performance ratings are taken into account. We demonstrate how CPU model determines instance performance, show associations between instance classes and sets of CPU models, and determine class-to-model performance characteristics. We show that pricing based on CPU models may significantly reduce, but not eliminate, price/performance variation. We further show that CPU model distribution differs across different AZs and so it may be possible to obtain better price/performance in some AZs by determining proportions of models found per AZ. However, the resources obtained in an AZ are account dependent, displays random variation and is subject to abrupt change.