Price forecasting for spot instances in Cloud computing

Abstract Big data applications usually need to rent a large number of virtual machines from Cloud computing providers. As a result of the policies employed by Cloud providers, the prices of spot virtual machine instances behavior stochastically. Spot prices (prices of spot instances) fluctuate greatly or have multiple regimes. Choosing virtual machines according to trends in prices is helpful in decreasing the resource rental cost. Existing price prediction methods are unable to accurately predict prices in these environments. As a result, a dynamic-ARIMA and two markov regime-switching autoregressive model based forecasting methods have been developed in this paper. Experimental results show that the proposals are better than the existing MonthAR in most scenarios.

[1]  Thilo Kielmann,et al.  Fast (re-)configuration of mixed on-demand and spot instance pools for high-throughput computing , 2013, ORMaCloud '13.

[2]  Artur Andrzejak,et al.  Monetary Cost-Aware Checkpointing and Migration on Amazon Cloud Spot Instances , 2012, IEEE Transactions on Services Computing.

[3]  Yang Wang,et al.  Budget-Driven Scheduling Algorithms for Batches of MapReduce Jobs in Heterogeneous Clouds , 2014, IEEE Transactions on Cloud Computing.

[4]  Quanyan Zhu,et al.  Dynamic Resource Allocation for Spot Markets in Cloud Computing Environments , 2011, 2011 Fourth IEEE International Conference on Utility and Cloud Computing.

[5]  Asser N. Tantawi,et al.  See Spot Run: Using Spot Instances for MapReduce Workflows , 2010, HotCloud.

[6]  Tim Kraska,et al.  Putting Analytics on the Spot: Or How to Lower the Cost for Analytics , 2014, IEEE Internet Computing.

[7]  Marco Abundo,et al.  QoS-aware bidding strategies for VM spot instances: A reinforcement learning approach applied to periodic long running jobs , 2015, 2015 IFIP/IEEE International Symposium on Integrated Network Management (IM).

[8]  Yang Song,et al.  Optimal Bids for Spot VMs in a Cloud for Deadline Constrained Jobs , 2012, 2012 IEEE Fifth International Conference on Cloud Computing.

[9]  Dick H. J. Epema,et al.  Deadline-constrained workflow scheduling algorithms for Infrastructure as a Service Clouds , 2013, Future Gener. Comput. Syst..

[10]  Nian-Feng Tzeng,et al.  Effective Cost Reduction for Elastic Clouds under Spot Instance Pricing Through Adaptive Checkpointing , 2015, IEEE Transactions on Computers.

[11]  Chunming Yuan,et al.  Forecasting exchange rates: The multi-state Markov-switching model with smoothing , 2011 .

[12]  Yang Song,et al.  Optimal bidding in spot instance market , 2012, 2012 Proceedings IEEE INFOCOM.

[13]  Kaushik Dutta,et al.  Dynamic Price Prediction for Amazon Spot Instances , 2015, 2015 48th Hawaii International Conference on System Sciences.

[14]  Rajkumar Buyya,et al.  A reliable and cost-efficient auto-scaling system for web applications using heterogeneous spot instances , 2015, J. Netw. Comput. Appl..

[15]  H. Tong,et al.  Threshold Autoregression, Limit Cycles and Cyclical Data , 1980 .

[16]  Daeyong Jung,et al.  A Workflow Scheduling Technique to Consider Task Processing Rate in Spot Instance-Based Cloud , 2014, FCC.

[17]  Naidila Sadashiv,et al.  Hybrid spot instance based resource provisioning strategy in dynamic cloud environment , 2014, 2014 International Conference on High Performance Computing and Applications (ICHPCA).

[18]  Jin-Soo Kim,et al.  Cost optimized provisioning of elastic resources for application workflows , 2011, Future Gener. Comput. Syst..

[19]  C. Holt.  Author's retrospective on ‘Forecasting seasonals and trends by exponentially weighted moving averages’ , 2004 .

[20]  Franco Zambonelli,et al.  Engineering Pervasive Service Ecosystems: The SAPERE Approach , 2015, TAAS.

[21]  Jan Broeckhove,et al.  Combining Futures and Spot Markets: A Hybrid Market Approach to Economic Grid Resource Management , 2011, Journal of Grid Computing.

[22]  He Huang,et al.  CAP3: A Cloud Auto-Provisioning Framework for Parallel Processing Using On-Demand and Spot Instances , 2013, 2013 IEEE Sixth International Conference on Cloud Computing.

[23]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[24]  Rajkumar Buyya,et al.  Fault-tolerant Workflow Scheduling using Spot Instances on Clouds , 2014, ICCS.

[25]  Jan Broeckhove,et al.  Online cost-efficient scheduling of deadline-constrained workloads on hybrid clouds , 2013, Future Gener. Comput. Syst..

[26]  Artur Andrzejak,et al.  Decision Model for Cloud Computing under SLA Constraints , 2010, 2010 IEEE International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems.

[27]  James D. Hamilton.  A New Approach to the Economic Analysis of Nonstationary Time Series and the Business Cycle , 1989 .

[28]  Xiang Cheng,et al.  Cost-efficient coordinated scheduling for leasing cloud resources on hybrid workloads , 2015, Parallel Comput..

[29]  Rubén Ruiz,et al.  A delay-based dynamic scheduling algorithm for bag-of-task workflows with stochastic task execution times in clouds , 2017, Future Gener. Comput. Syst..

[30]  Wei Tan,et al.  Self-Adaptive Learning PSO-Based Deadline Constrained Task Scheduling for Hybrid IaaS Cloud , 2014, IEEE Transactions on Automation Science and Engineering.

[31]  Muli Ben-Yehuda,et al.  Deconstructing Amazon EC2 Spot Instance Pricing , 2011, CloudCom.

[32]  Carlo Grillenzoni,et al.  Evaluation of on-line trading systems: Markov-switching vs time-varying parameter models , 2017, Decis. Support Syst..

[33]  Shaojie Tang,et al.  Towards Optimal Bidding Strategy for Amazon EC2 Cloud Spot Instance , 2012, 2012 IEEE Fifth International Conference on Cloud Computing.

[34]  Rajkumar Buyya,et al.  Enhancing Reliability of Workflow Execution Using Task Replication and Spot Instances , 2016, ACM Trans. Auton. Adapt. Syst..

[35]  Artur Andrzejak,et al.  Reducing Costs of Spot Instances via Checkpointing in the Amazon Elastic Compute Cloud , 2010, 2010 IEEE 3rd International Conference on Cloud Computing.

[36]  Rajkumar Buyya,et al.  Statistical Modeling of Spot Instance Prices in Public Cloud Environments , 2011, 2011 Fourth IEEE International Conference on Utility and Cloud Computing.

[37]  Jarek Nabrzyski,et al.  Cost- and deadline-constrained provisioning for scientific workflow ensembles in IaaS clouds , 2012, 2012 International Conference for High Performance Computing, Networking, Storage and Analysis.

[38]  Rob J Hyndman,et al.  Another look at measures of forecast accuracy , 2006 .

[39]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[40]  Fred L. Collopy,et al.  Error Measures for Generalizing About Forecasting Methods: Empirical Comparisons , 1992 .

[41]  Liang Zheng,et al.  How to Bid the Cloud , 2015, Comput. Commun. Rev..