Joint Elastic Cloud and Virtual Network Framework for Application Performance-cost Optimization

Cloud computing infrastructures are providing resources on demand for tackling the needs of large-scale distributed applications. To adapt to the diversity of cloud infrastructures and usage, new operation tools and models are needed. Estimating the amount of resources consumed by each application in particular is a difficult problem, both for end users who aim at minimizing their costs and infrastructure providers who aim at controlling their resources allocation. Furthermore, network provision is generally not controlled on clouds. This paper describes a framework automating cloud resources allocation, deployment and application execution control. It is based on a cost estimation model taking into account both virtual network and nodes managed by the cloud. The flexible provisioning of network resources permits the optimization of applications performance and infrastructure cost reduction. Four resource allocation strategies relying on the expertise that can be captured in workflow-based applications are considered. Results of these strategies are confined virtual infrastructure descriptions that are interpreted by the HIPerNet engine responsible for allocating, reserving and configuring physical resources. The evaluation of this framework was carried out on the Aladdin/Grid’5000 testbed using a real application from the area of medical image analysis.

[1]  Johan Montagnat,et al.  Flexible and Efficient Workflow Deployment of Data-Intensive Applications On Grids With MOTEUR , 2008, Int. J. High Perform. Comput. Appl..

[2]  Guilherme Piegas Koslovski,et al.  Specifying and provisioning virtual infrastructures with HIPerNET , 2010, Int. J. Netw. Manag..

[3]  Salim Hariri,et al.  Performance-Effective and Low-Complexity Task Scheduling for Heterogeneous Computing , 2002, IEEE Trans. Parallel Distributed Syst..

[4]  Guilherme Piegas Koslovski,et al.  Executing Distributed Applications on Virtualized Infrastructures Specified with the VXDL Language and Managed by the HIPerNET Framework , 2009, CloudComp.

[5]  Lavanya Ramakrishnan,et al.  VGrADS: enabling e-Science workflows on grids and clouds with fault tolerance , 2009, Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis.

[6]  Nick Feamster,et al.  In VINI veritas: realistic and controlled network experimentation , 2006, SIGCOMM.

[7]  Jörn Altmann,et al.  Resource allocation algorithm for light communication grid-based workflows within an SLA context , 2009, Int. J. Parallel Emergent Distributed Syst..

[8]  Adam Arbree,et al.  Mapping Abstract Complex Workflows onto Grid Environments , 2003, Journal of Grid Computing.

[9]  Ying Li,et al.  DaVinci: dynamically adaptive virtual networks for a customized internet , 2008, CoNEXT '08.

[10]  D. Frank Hsu,et al.  Mapping Heavy Communication Grid-Based Workflows Onto Grid Resources Within an SLA Context Using Metaheuristics , 2008, Int. J. High Perform. Comput. Appl..

[11]  Rizos Sakellariou,et al.  Scheduling multiple DAGs onto heterogeneous systems , 2006, Proceedings 20th IEEE International Parallel & Distributed Processing Symposium.

[12]  Johan Montagnat,et al.  Performance Evaluation of Grid-Enabled Registration Algorithms Using Bronze-Standards , 2006, MICCAI.

[13]  Rajkumar Buyya,et al.  A Taxonomy of Workflow Management Systems for Grid Computing , 2005, Proceedings of the 38th Annual Hawaii International Conference on System Sciences.

[14]  Marios D. Dikaiakos,et al.  Scheduling Workflows with Budget Constraints , 2007, Grid 2007.

[15]  Ismail Hakki Toroslu,et al.  An architecture for workflow scheduling under resource allocation constraints , 2005, Inf. Syst..

[16]  Andrew A. Chien,et al.  Using virtual grids to simplify application scheduling , 2006, Proceedings 20th IEEE International Parallel & Distributed Processing Symposium.

[17]  HuiYou Chang,et al.  Optimization of Workflow Resources Allocation with Cost Constraint , 2006, CSCWD.

[18]  Zsolt Németh,et al.  Characterizing Grids: Attributes, Definitions, and Formalisms , 2003, Journal of Grid Computing.

[19]  Guilherme Piegas Koslovski,et al.  A Scalable Security Model for Enabling Dynamic Virtual Private Execution Infrastructures on the Internet , 2009, 2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid.

[20]  Luiz Fernando Bittencourt,et al.  Towards the Scheduling of Multiple Workflows on Computational Grids , 2010, Journal of Grid Computing.

[21]  Ladislau Bölöni,et al.  A Comparison of Eleven Static Heuristics for Mapping a Class of Independent Tasks onto Heterogeneous Distributed Computing Systems , 2001, J. Parallel Distributed Comput..

[22]  Ronald G. Addie,et al.  Netml: a language and website for collaborative work on networks and their algorithms , 2006 .

[23]  Xuxian Jiang,et al.  vBET: a VM-based emulation testbed , 2003, MoMeTools '03.

[24]  Yaohui Jin,et al.  Resource Allocation Strategies for Data-Intensive Workflow-Based Applications in Optical Grids , 2006, 2006 10th IEEE Singapore International Conference on Communication Systems.

[25]  Cees T. A. M. de Laat,et al.  Using the Network Description Language in Optical Networks , 2007, 2007 10th IFIP/IEEE International Symposium on Integrated Network Management.

[26]  R. F. Freund,et al.  Dynamic matching and scheduling of a class of independent tasks onto heterogeneous computing systems , 1999, Proceedings. Eighth Heterogeneous Computing Workshop (HCW'99).

[27]  Johan Montagnat,et al.  A Service-Oriented Architecture enabling dynamic service grouping for optimizing distributed workflow execution , 2008, Future Gener. Comput. Syst..

[28]  Guilherme Piegas Koslovski,et al.  VXDL: Virtual Resources and Interconnection Networks Description Language , 2008, GridNets.

[29]  Luís Veiga,et al.  Heuristic for resources allocation on utility computing infrastructures , 2008, MGC '08.

[30]  Eliza Varney.  Distributed Management Task Force, Inc , 2010 .

[31]  Deborah Estrin,et al.  GENI Design Principles , 2006, Computer.

[32]  Rajkumar Buyya,et al.  Cost-based scheduling of scientific workflow applications on utility grids , 2005, First International Conference on e-Science and Grid Computing (e-Science'05).

[33]  Guilherme Piegas Koslovski,et al.  Exploring the virtual infrastructure service concept in Grid'5000 , 2009 .

[34]  Lixin Gao,et al.  How to lease the internet in your spare time , 2007, CCRV.

[35]  Ken Kennedy,et al.  Scheduling strategies for mapping application workflows onto the grid , 2005, HPDC-14. Proceedings. 14th IEEE International Symposium on High Performance Distributed Computing, 2005..