Multi-Objective Game Theoretic Schedulingof Bag-of-Tasks Workflows on Hybrid Clouds

Scheduling multiple large-scale parallel workflow applications on heterogeneous computing systems like hybrid clouds is a fundamental NP-complete problem that is critical to meeting various types of QoS (Quality of Service) requirements. This paper addresses the scheduling problem of large-scale applications inspired from real-world, characterized by a huge number of homogeneous and concurrent bags-of-tasks that are the main sources of bottlenecks but open great potential for optimization. The scheduling problem is formulated as a new sequential cooperative game and propose a communication and storage-aware multi-objective algorithm that optimizes two user objectives (execution time and economic cost) while fulfilling two constraints (network bandwidth and storage requirements). We present comprehensive experiments using both simulation and real-world applications that demonstrate the efficiency and effectiveness of our approach in terms of algorithm complexity, makespan, cost, system-level efficiency, fairness, and other aspects compared with other related algorithms.

[1]  Jarek Nabrzyski,et al.  Cost- and deadline-constrained provisioning for scientific workflow ensembles in IaaS clouds , 2012, 2012 International Conference for High Performance Computing, Networking, Storage and Analysis.

[2]  Debra A. Hensgen,et al.  The relative performance of various mapping algorithms is independent of sizable variances in run-time predictions , 1998, Proceedings Seventh Heterogeneous Computing Workshop (HCW'98).

[3]  L. Evans.  The Large Hadron Collider , 2012, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[4]  Raj Jain,et al.  A Quantitative Measure Of Fairness And Discrimination For Resource Allocation In Shared Computer Systems , 1998, ArXiv.

[5]  Rajkumar Buyya,et al.  A Particle Swarm Optimization-Based Heuristic for Scheduling Workflow Applications in Cloud Computing Environments , 2010, 2010 24th IEEE International Conference on Advanced Information Networking and Applications.

[6]  Dennis Gannon,et al.  Workflows for e-Science, Scientific Workflows for Grids , 2014 .

[7]  Oscar H. Ibarra,et al.  Heuristic Algorithms for Scheduling Independent Tasks on Nonidentical Processors , 1977, JACM.

[8]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[9]  R. Buyya,et al.  A budget constrained scheduling of workflow applications on utility Grids using genetic algorithms , 2006, 2006 Workshop on Workflows in Support of Large-Scale Science.

[10]  Radu Prodan,et al.  Performance and cost optimization for multiple large-scale grid workflow applications , 2007, Proceedings of the 2007 ACM/IEEE Conference on Supercomputing (SC '07).

[11]  Rajkumar Buyya,et al.  InterCloud: Utility-Oriented Federation of Cloud Computing Environments for Scaling of Application Services , 2010, ICA3PP.

[12]  Shanshan Song,et al.  Selfish grid computing: game-theoretic modeling and NAS performance results , 2005, CCGrid 2005. IEEE International Symposium on Cluster Computing and the Grid, 2005..

[13]  Salim Hariri,et al.  Performance-Effective and Low-Complexity Task Scheduling for Heterogeneous Computing , 2002, IEEE Trans. Parallel Distributed Syst..

[14]  E. van Kampen,et al.  Metal enrichment and energetics of galactic winds in galaxy clusters , 2004 .

[15]  R. F. Freund,et al.  Dynamic Mapping of a Class of Independent Tasks onto Heterogeneous Computing Systems , 1999, J. Parallel Distributed Comput..

[16]  Radu Prodan,et al.  Performance analysis of Grid applications in the ASKALON environment , 2009, 2009 10th IEEE/ACM International Conference on Grid Computing.

[17]  Mourad Hakem,et al.  Reliability and Scheduling on Systems Subject to Failures , 2007, 2007 International Conference on Parallel Processing (ICPP 2007).

[18]  Joaquim Sousa Pinto,et al.  Sky computing , 2011, 6th Iberian Conference on Information Systems and Technologies (CISTI 2011).

[19]  Wei Guo,et al.  Dynamic multi DAG scheduling algorithm for optical grid environment , 2007, SPIE/OSA/IEEE Asia Communications and Photonics.

[20]  Hisao Kameda,et al.  An algorithm for optimal static load balancing in distributed computer systems , 1992 .

[21]  Roger B. Myerson,et al.  Game theory - Analysis of Conflict , 1991 .

[22]  Rajkumar Buyya,et al.  Scheduling Parallel Applications on Utility Grids: Time and Cost Trade-off Management , 2009, ACSC.

[23]  Andrei Tchernykh,et al.  Multiple Workflow Scheduling Strategies with User Run Time Estimates on a Grid , 2012, Journal of Grid Computing.

[24]  Ronald L. Graham,et al.  Bounds for certain multiprocessing anomalies , 1966 .

[25]  Sajal K. Das,et al.  A Game Theory-Based Pricing Strategy to Support Single/Multiclass Job Allocation Schemes for Bandwidth-Constrained Distributed Computing Systems , 2007, IEEE Transactions on Parallel and Distributed Systems.

[26]  Djamal Zeghlache,et al.  Cloud Service Delivery across Multiple Cloud Platforms , 2011, 2011 IEEE International Conference on Services Computing.

[27]  Luiz Fernando Bittencourt,et al.  Towards the Scheduling of Multiple Workflows on Computational Grids , 2010, Journal of Grid Computing.

[28]  Francine Berman,et al.  Heuristics for scheduling parameter sweep applications in grid environments , 2000, Proceedings 9th Heterogeneous Computing Workshop (HCW 2000) (Cat. No.PR00556).

[29]  Nelson Luis Saldanha da Fonseca,et al.  Scheduling in hybrid clouds , 2012, IEEE Communications Magazine.

[30]  R. F. Freund,et al.  Dynamic matching and scheduling of a class of independent tasks onto heterogeneous computing systems , 1999, Proceedings. Eighth Heterogeneous Computing Workshop (HCW'99).

[31]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[32]  Tamer Basar,et al.  A game-theoretic formulation of multi-agent resource allocation , 2000, AGENTS '00.

[33]  Anthony T. Chronopoulos,et al.  Cooperative load balancing for a network of heterogeneous computers , 2006, Proceedings 20th IEEE International Parallel & Distributed Processing Symposium.

[34]  Ying Wang,et al.  Enabling Data and Compute Intensive Workflows in Bioinformatics , 2011, Euro-Par Workshops.

[35]  John Darlington,et al.  Scheduling Architecture and Algorithms within the ICENI Grid Middleware , 2003 .

[36]  Ladislau Bölöni,et al.  A Comparison of Eleven Static Heuristics for Mapping a Class of Independent Tasks onto Heterogeneous Distributed Computing Systems , 2001, J. Parallel Distributed Comput..

[37]  Radu Prodan,et al.  A Multi-objective Approach for Workflow Scheduling in Heterogeneous Environments , 2012, 2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (ccgrid 2012).

[38]  Atakan Dogan,et al.  Biobjective Scheduling Algorithms for Execution Time?Reliability Trade-off in Heterogeneous Computing Systems , 2005, Comput. J..

[39]  Johan Tordsson,et al.  Towards Secure Cloud Bursting, Brokerage and Aggregation , 2010, 2010 Eighth IEEE European Conference on Web Services.

[40]  Alain Girault,et al.  A bi-criteria scheduling heuristic for distributed embedded systems under reliability and real-time constraints , 2004, International Conference on Dependable Systems and Networks, 2004.

[41]  K. Schwarz,et al.  Electronic structure calculations of solids using the WIEN2k package for material sciences , 2002 .

[42]  Rajkumar Buyya,et al.  A Deadline and Budget Constrained Scheduling Algorithm for eScience Applications on Data Grids , 2005, ICA3PP.

[43]  Rizos Sakellariou,et al.  Scheduling multiple DAGs onto heterogeneous systems , 2006, Proceedings 20th IEEE International Parallel & Distributed Processing Symposium.