Self-managed cost-efficient virtual elastic clusters on hybrid Cloud infrastructures

In this study, we describe the ?further development of Elastic Cloud Computing Cluster (EC3), a tool ?for creating self-managed cost-efficient virtual hybrid elastic clusters on top of Infrastructure as a Service (IaaS) clouds. By using spot ?instances and checkpointing techniques, EC3 can significantly reduce the total ?execution cost as well as facilitating automatic fault tolerance. Moreover, EC3 can deploy and manage hybrid clusters across on-premises and public ?cloud resources, thereby introducing ?cloud bursting capabilities. ?We present the results of a case study that we conducted to assess the effectiveness of the tool ?based on the structural dynamic analysis of buildings. In addition, we evaluated the checkpointing algorithms in a real ?cloud environment with existing workloads to study their effectiveness. The results ?demonstrate the feasibility and benefits of this type of ?cluster for computationally intensive applications. Cost-efficient hybrid elastic virtual clusters are deployed across clouds.Spot instances and checkpointing reduce the costs of execution.Hybrid clusters reduce the total execution time by employing cloud bursting.Computationally intensive applications are executed easily with EC3.

[1]  J. Duell.  The design and implementation of Berkeley Lab's linux checkpoint/restart , 2005 .

[2]  Nandini Mukherjee,et al.  A Novel Checkpointing Scheme for Amazon EC2 Spot Instances , 2013, 2013 13th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing.

[3]  Yang Song,et al.  Optimal bidding in spot instance market , 2012, 2012 Proceedings IEEE INFOCOM.

[4]  Daeyong Jung,et al.  An Efficient Checkpointing Scheme Using Price History of Spot Instances in Cloud Computing Environment , 2011, NPC.

[5]  Ignacio Blanquer,et al.  Dynamic Management of Virtual Infrastructures , 2015, Journal of Grid Computing.

[6]  Rubén S. Montero,et al.  Multicloud Deployment of Computing Clusters for Loosely Coupled MTC Applications , 2011, IEEE Transactions on Parallel and Distributed Systems.

[7]  Rajkumar Buyya,et al.  The Aneka platform and QoS-driven resource provisioning for elastic applications on hybrid Clouds , 2012, Future Gener. Comput. Syst..

[8]  Paul Marshall,et al.  Rebalancing in a multi-cloud environment , 2013, Science Cloud '13.

[9]  Paul Marshall,et al.  Architecting a Large-scale Elastic Environment - Recontextualization and Adaptive Cloud Services for Scientific Computing , 2012, ICSOFT.

[10]  Rajkumar Buyya,et al.  Characterizing spot price dynamics in public cloud environments , 2013, Future Gener. Comput. Syst..

[11]  Paul Marshall,et al.  Elastic Site: Using Clouds to Elastically Extend Site Resources , 2010, 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing.

[12]  Borja Sotomayor,et al.  Capacity Leasing in Cloud Systems using the OpenNebula Engine , 2008 .

[13]  P. de la Fuente,et al.  Architrave: Advanced Analysis of Building Structures Integrated in Computer-Aided Design , 2014 .

[14]  Abdallah Khreishah,et al.  SpotMPI: A Framework for Auction-Based HPC Computing Using Amazon Spot Instances , 2011, ICA3PP.

[15]  Michele Mazzucco,et al.  Achieving Performance and Availability Guarantees with Spot Instances , 2011, 2011 IEEE International Conference on High Performance Computing and Communications.

[16]  Rafael Moreno-Vozmediano,et al.  Elastic management of cluster-based services in the cloud , 2009, ACDC '09.

[17]  Carlos de Alfonso,et al.  EC3: Elastic Cloud Computing Cluster , 2013, J. Comput. Syst. Sci..

[18]  Rubén S. Montero,et al.  An elasticity model for High Throughput Computing clusters , 2011, J. Parallel Distributed Comput..

[19]  Rajkumar Buyya,et al.  Managing Peak Loads by Leasing Cloud Infrastructure Services from a Spot Market , 2010, 2010 IEEE 12th International Conference on High Performance Computing and Communications (HPCC).

[20]  Vicente Hernández,et al.  Infrastructure Deployment Over the Cloud , 2011, 2011 IEEE Third International Conference on Cloud Computing Technology and Science.

[21]  Artur Andrzejak,et al.  Reducing Costs of Spot Instances via Checkpointing in the Amazon Elastic Compute Cloud , 2010, 2010 IEEE 3rd International Conference on Cloud Computing.

[22]  P. Mell,et al.  The NIST Definition of Cloud Computing , 2011 .

[23]  Shaojie Tang,et al.  Towards Optimal Bidding Strategy for Amazon EC2 Cloud Spot Instance , 2012, 2012 IEEE Fifth International Conference on Cloud Computing.

[24]  Muli Ben-Yehuda,et al.  Deconstructing Amazon EC2 Spot Instance Pricing , 2011, CloudCom.

[25]  Vicente Hernández,et al.  An Energy Manager for High Performance Computer Clusters , 2012, 2012 IEEE 10th International Symposium on Parallel and Distributed Processing with Applications.

[26]  Alexandru Iosup,et al.  The Grid Workloads Archive , 2008, Future Gener. Comput. Syst..

[27]  Carlos de Alfonso,et al.  An economic and energy-aware analysis of the viability of outsourcing cluster computing to a cloud , 2013, Future Gener. Comput. Syst..

[28]  Rajkumar Buyya,et al.  Provisioning Spot Market Cloud Resources to Create Cost-Effective Virtual Clusters , 2011, ICA3PP.

[29]  Rajkumar Buyya,et al.  Evaluating the cost-benefit of using cloud computing to extend the capacity of clusters , 2009, HPDC '09.

[30]  Andy B. Yoo,et al.  Approved for Public Release; Further Dissemination Unlimited X-ray Pulse Compression Using Strained Crystals X-ray Pulse Compression Using Strained Crystals , 2002 .

[31]  Jason Duell,et al.  The design and implementation of Berkeley Lab's linuxcheckpoint/restart , 2005 .

[32]  Anthony Sulistio,et al.  ViteraaS: Virtual Cluster as a Service , 2011, 2011 IEEE Third International Conference on Cloud Computing Technology and Science.

[33]  Miguel Caballer,et al.  A Generic Catalog and Repository Service for Virtual Machine Images , 2010 .