Characterizing spot price dynamics in public cloud environments

The surge in demand for utilizing public Cloud resources has introduced many trade-offs between price, performance and recently reliability. Amazon's Spot Instances (SIs) create a competitive bidding option for public Cloud users at lower prices without providing reliability on services. It is generally believed that SIs reduce monetary cost to the Cloud users, however it appears from the literature that their characteristics have not been explored and reported. We believe that characterization of SIs is fundamental in the design of stochastic scheduling algorithms and fault tolerant mechanisms in public Cloud environments for the spot market. In this paper, we have done a comprehensive analysis of SIs based on one year price history in four data centers of Amazon's EC2. For this purpose, we have analyzed all different types of SIs in terms of spot price and the inter-price time (time between price changes) and determined the time dynamics for spot price in hour-in-day and day-of-week. Moreover, we have proposed a statistical model that fits well these two data series. The results reveal that we are able to model spot price dynamics as well as the inter-price time of each SI by a mixture of Gaussians distribution with three or four components. The proposed model is validated through extensive simulations, which demonstrate that our model exhibits a good degree of accuracy under realistic working conditions.

[1]  Hui Li Realistic Workload Modeling and Its Performance Impacts in Large-Scale eScience Grids , 2010, IEEE Transactions on Parallel and Distributed Systems.

[2]  Quanyan Zhu,et al.  Dynamic Resource Allocation for Spot Markets in Cloud Computing Environments , 2011, 2011 Fourth IEEE International Conference on Utility and Cloud Computing.

[3]  Asser N. Tantawi,et al.  See Spot Run: Using Spot Instances for MapReduce Workflows , 2010, HotCloud.

[4]  Rajkumar Buyya,et al.  Managing Peak Loads by Leasing Cloud Infrastructure Services from a Spot Market , 2010, 2010 IEEE 12th International Conference on High Performance Computing and Communications (HPCC).

[5]  Artur Andrzejak,et al.  Reducing Costs of Spot Instances via Checkpointing in the Amazon Elastic Compute Cloud , 2010, 2010 IEEE 3rd International Conference on Cloud Computing.

[6]  Muli Ben-Yehuda,et al.  Deconstructing Amazon EC2 Spot Instance Pricing , 2011, 2011 IEEE Third International Conference on Cloud Computing Technology and Science.

[7]  Sewook Wee,et al.  Debunking Real-Time Pricing in Cloud Computing , 2011, 2011 11th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing.

[8]  Artur Andrzejak,et al.  Monetary Cost-Aware Checkpointing and Migration on Amazon Cloud Spot Instances , 2012, IEEE Transactions on Services Computing.

[9]  Alexandru Iosup,et al.  The Failure Trace Archive: Enabling Comparative Analysis of Failures in Diverse Distributed Systems , 2010, 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing.

[10]  Dror G. Feitelson,et al.  Workload Modeling for Computer Systems Performance Evaluation , 2015 .

[11]  Jean-Marc Vincent,et al.  Mining for statistical models of availability in large-scale distributed systems: An empirical study of SETI@home , 2009, 2009 IEEE International Symposium on Modeling, Analysis & Simulation of Computer and Telecommunication Systems.

[12]  Jinesh Varia,et al.  Best Practices in Architecting Cloud Applications in the AWS Cloud , 2011 .

[13]  Rajkumar Buyya,et al.  Future Generation Computer Systems Deadline-driven Provisioning of Resources for Scientific Applications in Hybrid Clouds with Aneka , 2022 .

[14]  Alexandru Iosup,et al.  The Grid Workloads Archive , 2008, Future Gener. Comput. Syst..

[15]  Muntasir Raihan Rahman Risk Aware Resource Allocation for Clouds , 2011 .

[16]  Jan Broeckhove,et al.  Combining Futures and Spot Markets: A Hybrid Market Approach to Economic Grid Resource Management , 2011, Journal of Grid Computing.

[17]  R. Buyya,et al.  Comprehensive Statistical Analysis and Modeling of Spot Instances in Public Cloud Environments , 2011 .

[18]  F. Ortuño,et al.  A Stochastic Calculus Model for the Spot Price of Computing Power , 2010 .

[19]  Bu-Sung Lee,et al.  Cost Minimization for Provisioning Virtual Servers in Amazon Elastic Compute Cloud , 2011, 2011 IEEE 19th Annual International Symposium on Modelling, Analysis, and Simulation of Computer and Telecommunication Systems.

[20]  Adrian E. Raftery,et al.  Model-Based Clustering, Discriminant Analysis, and Density Estimation , 2002 .

[21]  Rajkumar Buyya,et al.  CloudSim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms , 2011, Softw. Pract. Exp..

[22]  Raouf Boutaba,et al.  Dynamic Resource Allocation for Spot Markets in Clouds , 2011, Hot-ICE.

[23]  David P. Anderson,et al.  On correlated availability in Internet-distributed systems , 2008, 2008 9th IEEE/ACM International Conference on Grid Computing.

[24]  Rajkumar Buyya,et al.  Provisioning Spot Market Cloud Resources to Create Cost-Effective Virtual Clusters , 2011, ICA3PP.

[25]  Michele Mazzucco,et al.  Achieving Performance and Availability Guarantees with Spot Instances , 2011, 2011 IEEE International Conference on High Performance Computing and Communications.

[26]  Rajkumar Buyya,et al.  Cloud Computing Principles and Paradigms , 2011 .

[27]  Artur Andrzejak,et al.  Decision Model for Cloud Computing under SLA Constraints , 2010, 2010 IEEE International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems.

[28]  Michael Muskulus,et al.  Modeling correlated workloads by combining model based clustering and a localized sampling algorithm , 2007, ICS '07.