Dynamic bidding in spot market for profit maximization in the public cloud

In public cloud domain, some cloud providers sell surplus resources in the form of spot instances to improve their profits. The average spot price is much cheaper than the on-demand options, which drives more and more cloud users to use spot instances to accelerate their services. However, the cloud user faces a challenge in determining the instance rental and management policy due to the unconventional pricing structure in the spot market and fluctuations in workload and spot price. In this work, we propose a dynamic bidding and resource management algorithm for the cloud user to cut down the instance rental bill in the spot market. The proposed algorithm operates in two timescales, i.e., it determines the bidding, instance rental, and job dispatching policy hourly and makes instances allocation decision in a finer granularity. The advantages of our approach are that it has a simple structure and needs no a priori statistical information of spot price and workload. We prove the optimality of the proposed algorithm and use extensive simulations to study its performance. It is shown that D-bid can save up to almost 70% of the rental cost compared to a baseline nonwork-conserving strategy.

[1]  Rajkumar Buyya,et al.  Pricing Cloud Compute Commodities: A Novel Financial Economic Model , 2012, 2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (ccgrid 2012).

[2]  Ran Zhang,et al.  Reactive Pricing: An Adaptive Pricing Policy for Cloud Providers to Maximize Profit , 2016, IEEE Transactions on Network and Service Management.

[3]  Muli Ben-Yehuda,et al.  Deconstructing Amazon EC2 Spot Instance Pricing , 2011, CloudCom.

[4]  Kai Song,et al.  Exploring the profit-reliability trade-off in Amazon's spot instance market: A better pricing mechanism , 2013, 2013 IEEE/ACM 21st International Symposium on Quality of Service (IWQoS).

[5]  Pramod Bhatotia,et al.  Orchestrating the Deployment of Computations in the Cloud with Conductor , 2012, NSDI.

[6]  Ying Chen,et al.  Energy Efficient Dynamic Service Selection for Large-Scale Web Service Systems , 2014, 2014 IEEE International Conference on Web Services.

[7]  Artur Andrzejak,et al.  Decision Model for Cloud Computing under SLA Constraints , 2010, 2010 IEEE International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems.

[8]  Shaojie Tang,et al.  Towards Optimal Bidding Strategy for Amazon EC2 Cloud Spot Instance , 2012, 2012 IEEE Fifth International Conference on Cloud Computing.

[9]  Hai Jin,et al.  Carbon-Aware Load Balancing for Geo-distributed Cloud Services , 2013, 2013 IEEE 21st International Symposium on Modelling, Analysis and Simulation of Computer and Telecommunication Systems.

[10]  Daniel Grosu,et al.  A Combinatorial Auction-Based Mechanism for Dynamic VM Provisioning and Allocation in Clouds , 2013, IEEE Transactions on Cloud Computing.

[11]  Shaolei Ren,et al.  Provably-Efficient Job Scheduling for Energy and Fairness in Geographically Distributed Data Centers , 2012, 2012 IEEE 32nd International Conference on Distributed Computing Systems.

[12]  Kai Song,et al.  Improving the Revenue, Efficiency and Reliability in Data Center Spot Market: A Truthful Mechanism , 2013, 2013 IEEE 21st International Symposium on Modelling, Analysis and Simulation of Computer and Telecommunication Systems.

[13]  Baochun Li,et al.  Revenue maximization with dynamic auctions in IaaS cloud markets , 2013, 2013 IEEE/ACM 21st International Symposium on Quality of Service (IWQoS).

[14]  Artur Andrzejak,et al.  Monetary Cost-Aware Checkpointing and Migration on Amazon Cloud Spot Instances , 2012, IEEE Transactions on Services Computing.

[15]  Yang Song,et al.  Optimal bidding in spot instance market , 2012, 2012 Proceedings IEEE INFOCOM.

[16]  Quanyan Zhu,et al.  Dynamic Resource Allocation for Spot Markets in Cloud Computing Environments , 2011, 2011 Fourth IEEE International Conference on Utility and Cloud Computing.

[17]  Asser N. Tantawi,et al.  See Spot Run: Using Spot Instances for MapReduce Workflows , 2010, HotCloud.

[18]  Tim Kraska,et al.  Putting Analytics on the Spot: Or How to Lower the Cost for Analytics , 2014, IEEE Internet Computing.

[19]  Baochun Li,et al.  A study of pricing for cloud resources , 2013, PERV.

[20]  Longbo Huang,et al.  Power Cost Reduction in Distributed Data Centers: A Two-Time-Scale Approach for Delay Tolerant Workloads , 2015, IEEE Transactions on Parallel and Distributed Systems.

[21]  Ke Xiao,et al.  Pricing reserved and On-Demand Schemes of cloud computing based on option pricing model , 2013, 2013 15th Asia-Pacific Network Operations and Management Symposium (APNOMS).

[22]  Martin Schulz,et al.  Exploiting redundancy for cost-effective, time-constrained execution of HPC applications on amazon EC2 , 2014, HPDC '14.

[23]  Artur Andrzejak,et al.  Reducing Costs of Spot Instances via Checkpointing in the Amazon Elastic Compute Cloud , 2010, 2010 IEEE 3rd International Conference on Cloud Computing.

[24]  Huan Liu,et al.  Cutting MapReduce Cost with Spot Market , 2011, HotCloud.

[25]  Bo Li,et al.  On arbitrating the power-performance tradeoff in SaaS clouds , 2013, 2013 Proceedings IEEE INFOCOM.

[26]  Baochun Li,et al.  Dynamic Cloud Pricing for Revenue Maximization , 2013, IEEE Transactions on Cloud Computing.

[27]  Bu-Sung Lee,et al.  Cost Minimization for Provisioning Virtual Servers in Amazon Elastic Compute Cloud , 2011, 2011 IEEE 19th Annual International Symposium on Modelling, Analysis, and Simulation of Computer and Telecommunication Systems.

[28]  Scott Shenker,et al.  Delay scheduling: a simple technique for achieving locality and fairness in cluster scheduling , 2010, EuroSys '10.

[29]  Albert Y. Zomaya,et al.  Tradeoffs Between Profit and Customer Satisfaction for Service Provisioning in the Cloud , 2011, HPDC '11.

[30]  Miao Pan,et al.  Optimal Resource Rental Planning for Elastic Applications in Cloud Market , 2012, 2012 IEEE 26th International Parallel and Distributed Processing Symposium.

[31]  George Kesidis,et al.  Pricing of service in clouds: optimal response and strategic interactions , 2014, PERV.

[32]  Rajkumar Buyya,et al.  Reliable Provisioning of Spot Instances for Compute-intensive Applications , 2011, 2012 IEEE 26th International Conference on Advanced Information Networking and Applications.

[33]  Zongpeng Li,et al.  Dynamic pricing and profit maximization for the cloud with geo-distributed data centers , 2014, IEEE INFOCOM 2014 - IEEE Conference on Computer Communications.

[34]  Xin Wang,et al.  Delay-Aware Cost Optimization for Dynamic Resource Provisioning in Hybrid Clouds , 2014, 2014 IEEE International Conference on Web Services.