Hidden Markov Model Based Spot Price Prediction for Cloud Computing

Cloud computing accelerates the processing of big data applications by providing elastic virtual machines. Spot instances are virtual machines sold by auctions which offer lower prices (called Spot prices) than the same virtual machines with fixed prices. Spot prices change stochastically according to the real-time user demands and supplies. Accurate Spot price prediction helps users choose appropriate resources and set optimal bids to reduce resource rental cost. However, great fluctuations of recent Spot prices decrease the prediction accuracy of existing autoregression based prediction methods. Therefore, in this paper, a Hidden Markov Model and Expectation based prediction method (HMM-E) is proposed which uses Markov processes to describe the different demand markets of Cloud computing (hidden states) and the fluctuation degrees of Spot prices (observations) separately. Experimental results demonstrate that HMM-E predicts Spot prices more accurately than the existing autoregression based forecasting methods when the forecast period is shorter than about five hours.

[1]  Abdul Talib Bon,et al.  Hidden Markov Model and Forward-Backward Algorithm in Crude Oil Price Forecasting , 2016 .

[2]  Jiahua Chen,et al.  English Sentence Recognition Based on HMM and Clustering , 2013 .

[3]  Hongfei Lin,et al.  Granular transfer learning using type-2 fuzzy HMM for text sequence recognition , 2016, Neurocomputing.

[4]  Fred L. Collopy,et al.  Error Measures for Generalizing About Forecasting Methods: Empirical Comparisons , 1992 .

[5]  Rajkumar Buyya,et al.  Statistical Modeling of Spot Instance Prices in Public Cloud Environments , 2011, 2011 Fourth IEEE International Conference on Utility and Cloud Computing.

[6]  Leonard J. Paas,et al.  A dynamic analysis of stock markets using a hidden Markov model , 2013 .

[7]  Thomas Bartz-Beielstein,et al.  Experimental Methods for the Analysis of Optimization Algorithms , 2010 .

[8]  Marco Abundo,et al.  QoS-aware bidding strategies for VM spot instances: A reinforcement learning approach applied to periodic long running jobs , 2015, 2015 IFIP/IEEE International Symposium on Integrated Network Management (IM).

[9]  Yang Song,et al.  Optimal bidding in spot instance market , 2012, 2012 Proceedings IEEE INFOCOM.

[10]  Yang Song,et al.  Optimal Bids for Spot VMs in a Cloud for Deadline Constrained Jobs , 2012, 2012 IEEE Fifth International Conference on Cloud Computing.

[11]  Kaushik Dutta,et al.  Dynamic Price Prediction for Amazon Spot Instances , 2015, 2015 48th Hawaii International Conference on System Sciences.

[12]  Lu Hou,et al.  A nonhomogeneous hidden Markov model of response dynamics and mailing optimization in direct marketing , 2016, Eur. J. Oper. Res..

[13]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[14]  Zhitao Guan,et al.  Electricity Price Forecast Using Combined Models with Adaptive Weights Selected and Errors Calibrated by Hidden Markov Model , 2013 .

[15]  Shaojie Tang,et al.  Towards Optimal Bidding Strategy for Amazon EC2 Cloud Spot Instance , 2012, 2012 IEEE Fifth International Conference on Cloud Computing.

[16]  Geoffrey J. McLachlan,et al.  The EM Algorithm , 2012 .

[17]  H. Kobayashi,et al.  An efficient forward-backward algorithm for an explicit-duration hidden Markov model , 2003, IEEE Signal Processing Letters.

[18]  Xueying Zhang,et al.  A Hybrid Speech Recognition Training Method for HMM Based on Genetic Algorithm and Baum Welch Algorithm , 2007, Second International Conference on Innovative Computing, Informatio and Control (ICICIC 2007).

[19]  Asser N. Tantawi,et al.  See Spot Run: Using Spot Instances for MapReduce Workflows , 2010, HotCloud.

[20]  Shunzheng Yu,et al.  A spatial HMM approach for green networking , 2016, Comput. Commun..