A Cloud Bidding Framework for Deadline Constrained Jobs

Reliable completion of the computing jobs through Amazon spot instances (SIs) with proper bargaining is challenging. Therefore, an SI bidding system is developed for deadline constrained jobs considering both the conditions of the market and the condition of the user. The system tries to bargain with the provider by bidding low when the task is not urgent. After that, the system increases the price or the price distribution gradually when the progress is lower than required. To calculate the bid distribution, we compute the probability density of the price after five minutes. Then, we apply our developed equations to compute bid-prices from the probability density function. Equations are easily interpretable to both humans and machines. We also consider long-term probability distributions of the prices for the reliable completion of the job. Tasks with several days deadline are prescribed to bid considering the daily price-curve. According to the evaluation of Amazon SI price, the proposed system effectively saves 79%-87% for jobs with several hours deadline and saves 82%-100% for jobs with several days deadline compared to the on-demand instances. Moreover, our algorithm helps all bidders by keeping the price low.

[1]  Prateek Sharma,et al.  SpotOn: a batch computing service for the spot market , 2015, SoCC.

[2]  Rajkumar Buyya,et al.  Fault-tolerant Workflow Scheduling using Spot Instances on Clouds , 2014, ICCS.

[3]  Shaojie Tang,et al.  Towards Optimal Bidding Strategy for Amazon EC2 Cloud Spot Instance , 2012, 2012 IEEE Fifth International Conference on Cloud Computing.

[4]  Markus Lumpe,et al.  On Estimating Minimum Bids for Amazon EC2 Spot Instances , 2017, 2017 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID).

[5]  R. Buyya,et al.  An Auction Mechanism for a Cloud Spot Market , 2014 .

[6]  M. G. De Giorgi,et al.  Improvements in the predictions for the photovoltaic system performance of the Mediterranean regions , 2016 .

[7]  Saeid Nahavandi,et al.  Partial Adversarial Training for Prediction Interval , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).

[8]  Liang Zheng,et al.  How to Bid the Cloud , 2015, Comput. Commun. Rev..

[9]  John P. Parmigiani,et al.  An experimental assessment of methods to predict crack deflection at an interface , 2017 .

[10]  Maria Kihl,et al.  Using a Predator-Prey Model to Explain Variations of Cloud Spot Price , 2016, CLOSER.

[11]  Qian Zhu,et al.  Resource Provisioning with Budget Constraints for Adaptive Applications in Cloud Environments , 2010, IEEE Transactions on Services Computing.

[12]  David E. Irwin,et al.  Towards Index-based Global Trading in Cloud Spot Markets , 2017, HotCloud.

[13]  Rajkumar Buyya,et al.  Revenue Maximization with Optimal Capacity Control in Infrastructure as a Service Cloud Markets , 2015, IEEE Transactions on Cloud Computing.

[14]  Nian-Feng Tzeng,et al.  Effective Cost Reduction for Elastic Clouds under Spot Instance Pricing Through Adaptive Checkpointing , 2015, IEEE Transactions on Computers.

[15]  Rajkumar Buyya,et al.  A reliable and cost-efficient auto-scaling system for web applications using heterogeneous spot instances , 2015, Journal of Network and Computer Applications.

[16]  Rajkumar Buyya,et al.  Characterizing spot price dynamics in public cloud environments , 2013, Future Gener. Comput. Syst..

[17]  David E. Irwin,et al.  Cloud Index Tracking: Enabling Predictable Costs in Cloud Spot Markets , 2018, SoCC.

[18]  Kyungyong Lee,et al.  Time-Series Analysis for Price Prediction of Opportunistic Cloud Computing Resources , 2018 .

[19]  Yang Song,et al.  Optimal bidding in spot instance market , 2012, 2012 Proceedings IEEE INFOCOM.

[20]  Daniel Grosu,et al.  Physical Machine Resource Management in Clouds: A Mechanism Design Approach , 2015, IEEE Transactions on Cloud Computing.

[21]  Saeid Nahavandi,et al.  Prediction interval with examples of similar pattern and prediction strength , 2017, 2017 IEEE 30th Canadian Conference on Electrical and Computer Engineering (CCECE).

[22]  Rajkumar Buyya,et al.  An Auction Mechanism for Cloud Spot Markets , 2016, TAAS.

[23]  Maria Grazia De Giorgi,et al.  Forecasting of PV Power Generation using weather input data‐preprocessing techniques , 2017 .

[24]  Saeid Nahavandi,et al.  Neural Network-Based Uncertainty Quantification: A Survey of Methodologies and Applications , 2018, IEEE Access.

[25]  Shaolei Ren,et al.  Colocation Demand Response: Joint Online Mechanisms for Individual Utility and Social Welfare Maximization , 2016, IEEE Journal on Selected Areas in Communications.

[26]  Cho-Li Wang,et al.  Error-Tolerant Resource Allocation and Payment Minimization for Cloud System , 2013, IEEE Transactions on Parallel and Distributed Systems.

[27]  Zahid Raza,et al.  A Survey on Spot Pricing in Cloud Computing , 2017, Journal of Network and Systems Management.

[28]  Mansun Chan,et al.  Coil-Shaped Electrodes to Reduce the Current Variation of Drop-Casted OTFTs , 2017, IEEE Electron Device Letters.

[29]  Hussain Mohammed Dipu Kabir,et al.  A Resilient Auction Framework for Deadline-Aware Jobs in Cloud Spot Market , 2017, 2017 IEEE 36th Symposium on Reliable Distributed Systems (SRDS).

[30]  George Kesidis,et al.  Exploiting Spot and Burstable Instances for Improving the Cost-efficacy of In-Memory Caches on the Public Cloud , 2017, EuroSys.

[31]  Abbas Khosravi,et al.  A review on artificial intelligence based load demand forecasting techniques for smart grid and buildings , 2015 .

[32]  Daeyong Jung,et al.  An Efficient Checkpointing Scheme Using Price History of Spot Instances in Cloud Computing Environment , 2011, NPC.

[33]  Martin Schulz,et al.  Exploiting Redundancy and Application Scalability for Cost-Effective, Time-Constrained Execution of HPC Applications on Amazon EC2 , 2016, IEEE Transactions on Parallel and Distributed Systems.

[34]  Bu-Sung Lee,et al.  Optimization of Resource Provisioning Cost in Cloud Computing , 2012, IEEE Transactions on Services Computing.

[35]  Anand Sivasubramaniam,et al.  Cloudy with a Chance of Cost Savings , 2013, IEEE Transactions on Parallel and Distributed Systems.

[36]  Zongpeng Li,et al.  Dynamic resource provisioning in cloud computing: A randomized auction approach , 2014, IEEE INFOCOM 2014 - IEEE Conference on Computer Communications.