Genetic Programming for Dynamic Workflow Scheduling in Fog Computing

Dynamic Workflow Scheduling in Fog Computing (DWSFC) is an important optimisation problem with many real-world applications. The current workflow scheduling problems only consider cloud servers but ignore the roles of mobile devices and edge servers. Some applications need to consider the mobile devices, edge, and cloud servers simultaneously, making them work together to generate an effective schedule. In this article, a new problem model for DWSFC is considered and a new simulator is designed for the new DWSFC problem model. The designed simulator takes the mobile devices, edge, and cloud servers as a whole system, where they all can execute tasks. In the designed simulator, two kinds of decision points are considered, which are the routing decision points and the sequencing decision points. To solve this problem, a new Multi-Tree Genetic Programming (MTGP) method is developed to automatically evolve scheduling heuristics that can make effective real-time decisions on these decision points. The proposed MTGP method with a multi-tree representation can handle the routing decision points and sequencing decision points simultaneously. The experimental results show that the proposed MTGP can achieve significantly better test performance (reduce the makespan by up to 50%) on all the tested scenarios than existing state-of-the-art methods.

[1]  Jing Mei,et al.  Cost-Efficient Workflow Scheduling Algorithm for Applications With Deadline Constraint on Heterogeneous Clouds , 2022, IEEE Transactions on Parallel and Distributed Systems.

[2]  Mengjie Zhang,et al.  Genetic Programming With Knowledge Transfer and Guided Search for Uncertain Capacitated Arc Routing Problem , 2022, IEEE Transactions on Evolutionary Computation.

[3]  Mark Goh,et al.  Genetic programming-based hyper-heuristic approach for solving dynamic job shop scheduling problem with extended technical precedence constraints , 2021, Comput. Oper. Res..

[4]  Hengyu Tian,et al.  Analysis of Overall Assignment and Sorting of Tasks in Heterogeneous Computing Systems Based on Mathematical Programming Algorithms , 2021, Wireless Personal Communications.

[5]  Gang Chen,et al.  Budget and SLA Aware Dynamic Workflow Scheduling in Cloud Computing with Heterogeneous Resources , 2021, 2021 IEEE Congress on Evolutionary Computation (CEC).

[6]  Mengjie Zhang,et al.  Two-stage multi-objective genetic programming with archive for uncertain capacitated arc routing problem , 2021, GECCO.

[7]  Mengjie Zhang,et al.  Multitask Genetic Programming-Based Generative Hyperheuristics: A Case Study in Dynamic Scheduling , 2021, IEEE Transactions on Cybernetics.

[8]  Ravi Shankar Singh,et al.  Energy Efficient and Reliability Aware Workflow Task Scheduling in Cloud Environment , 2021, Wireless Personal Communications.

[9]  Tian Xiang,et al.  Dynamic DNN Decomposition for Lossless Synergistic Inference , 2021, 2021 IEEE 41st International Conference on Distributed Computing Systems Workshops (ICDCSW).

[10]  Yi Mei,et al.  Evolving Scheduling Heuristics via Genetic Programming With Feature Selection in Dynamic Flexible Job-Shop Scheduling , 2020, IEEE Transactions on Cybernetics.

[11]  Gang Chen,et al.  Genetic Programming Based Hyper Heuristic Approach for Dynamic Workflow Scheduling in the Cloud , 2020, DEXA.

[12]  Bryan Ng,et al.  Dynamic multi-workflow scheduling: A deadline and cost-aware approach for commercial clouds , 2019, Future Gener. Comput. Syst..

[13]  Yi Mei,et al.  A Hybrid Genetic Programming Algorithm for Automated Design of Dispatching Rules , 2019, Evolutionary Computation.

[14]  Jun Zhang,et al.  Multiobjective Cloud Workflow Scheduling: A Multiple Populations Ant Colony System Approach , 2019, IEEE Transactions on Cybernetics.

[15]  Hui Ma,et al.  Achieving Flexible Scheduling of Heterogeneous Workflows in Cloud through a Genetic Programming Based Approach , 2019, 2019 IEEE Congress on Evolutionary Computation (CEC).

[16]  Latif Pourkarimi,et al.  Integer linear programming-based multi-objective scheduling for scientific workflows in multi-cloud environments , 2019, Journal of Supercomputing.

[17]  Hossein Pedram,et al.  Integer linear programming-based multi-objective scheduling for scientific workflows in multi-cloud environments , 2019, The Journal of Supercomputing.

[18]  Ju Ren,et al.  Online Multi-Workflow Scheduling under Uncertain Task Execution Time in IaaS Clouds , 2019, IEEE Transactions on Cloud Computing.

[19]  Fangfang Zhang,et al.  Genetic Programming with Multi-tree Representation for Dynamic Flexible Job Shop Scheduling , 2018, Australasian Conference on Artificial Intelligence.

[20]  Tournament selection , 2018, Evolutionary Computation 1.

[21]  Hui Li,et al.  A Genetic Algorithm Based Data Replica Placement Strategy for Scientific Applications in Clouds , 2018, IEEE Transactions on Services Computing.

[22]  Yi Mei,et al.  Constrained Dimensionally Aware Genetic Programming for Evolving Interpretable Dispatching Rules in Dynamic Job Shop Scheduling , 2017, SEAL.

[23]  Eui-nam Huh,et al.  A cost- and performance-effective approach for task scheduling based on collaboration between cloud and fog computing , 2017, Int. J. Distributed Sens. Networks.

[24]  Rajkumar Buyya,et al.  iFogSim: A toolkit for modeling and simulation of resource management techniques in the Internet of Things, Edge and Fog computing environments , 2016, Softw. Pract. Exp..

[25]  Mengjie Zhang,et al.  Automated Design of Production Scheduling Heuristics: A Review , 2016, IEEE Transactions on Evolutionary Computation.

[26]  Q. Wu,et al.  Workflow scheduling in cloud: a survey , 2015, The Journal of Supercomputing.

[27]  Jun Zhang,et al.  Deadline constrained cloud computing resources scheduling for cost optimization based on dynamic objective genetic algorithm , 2015, 2015 IEEE Congress on Evolutionary Computation (CEC).

[28]  Michel Gendreau,et al.  Hyper-heuristics: a survey of the state of the art , 2013, J. Oper. Res. Soc..

[29]  Emma Hart,et al.  Generating single and multiple cooperative heuristics for the one dimensional bin packing problem using a single node genetic programming island model , 2013, GECCO '13.

[30]  Pinal Salot,et al.  A SURVEY OF VARIOUS SCHEDULING ALGORITHM IN CLOUD COMPUTING ENVIRONMENT , 2013 .

[31]  Ewa Deelman,et al.  WorkflowSim: A toolkit for simulating scientific workflows in distributed environments , 2012, 2012 IEEE 8th International Conference on E-Science.

[32]  Ke Tang,et al.  A developmental solution to (dynamic) capacitated arc routing problems using genetic programming , 2012, GECCO '12.

[33]  Graham Kendall,et al.  Automating the Packing Heuristic Design Process with Genetic Programming , 2012, Evolutionary Computation.

[34]  Rajkumar Buyya,et al.  CloudSim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms , 2011, Softw. Pract. Exp..

[35]  Rajkumar Buyya,et al.  A Particle Swarm Optimization-Based Heuristic for Scheduling Workflow Applications in Cloud Computing Environments , 2010, 2010 24th IEEE International Conference on Advanced Information Networking and Applications.

[36]  Erich Schikuta,et al.  A Parallel Branch and Bound Algorithm for Workflow QoS Optimization , 2009, 2009 International Conference on Parallel Processing.

[37]  Rajkumar Buyya,et al.  Modeling and simulation of scalable Cloud computing environments and the CloudSim toolkit: Challenges and opportunities , 2009, 2009 International Conference on High Performance Computing & Simulation.

[38]  Mei-Hui Su,et al.  Characterization of scientific workflows , 2008, 2008 Third Workshop on Workflows in Support of Large-Scale Science.

[39]  Kobra Etminani,et al.  A Min-Min Max-Min Selective Algorithm for Grid Task Scheduling , 2007, 2007 3rd IEEE/IFIP International Conference in Central Asia on Internet.

[40]  Rajkumar Buyya,et al.  Extending GridSim with an architecture for failure detection , 2007, 2007 International Conference on Parallel and Distributed Systems.

[41]  Graham Kendall,et al.  Evolving Bin Packing Heuristics with Genetic Programming , 2006, PPSN.

[42]  Ian J. Taylor,et al.  Distributed P2P computing within Triana: a galaxy visualization test case , 2003, Proceedings International Parallel and Distributed Processing Symposium.

[43]  R. Buyya,et al.  GridSim: a toolkit for the modeling and simulation of distributed resource management and scheduling for Grid computing , 2002, Concurr. Comput. Pract. Exp..

[44]  Salim Hariri,et al.  Performance-Effective and Low-Complexity Task Scheduling for Heterogeneous Computing , 2002, IEEE Trans. Parallel Distributed Syst..

[45]  Sean Luke,et al.  A survey and comparison of tree generation algorithms , 2001 .

[46]  Ladislau Bölöni,et al.  A Comparison of Eleven Static Heuristics for Mapping a Class of Independent Tasks onto Heterogeneous Distributed Computing Systems , 2001, J. Parallel Distributed Comput..

[47]  John R. Koza,et al.  Genetic programming as a means for programming computers by natural selection , 1994 .

[48]  W. Zhao,et al.  Performance analysis of FCFS and improved FCFS scheduling algorithms for dynamic real-time computer systems , 1989, [1989] Proceedings. Real-Time Systems Symposium.

[49]  N. Fujii,et al.  Evolving Dispatching Rules Using Genetic Programming for Multi-objective Dynamic Job Shop Scheduling with Machine Breakdowns , 2021, Procedia CIRP.

[50]  Helen D. Karatza,et al.  A Scheduling Algorithm for a Fog Computing System with Bag-of-Tasks Jobs: Simulation and Performance Evaluation , 2020, Simul. Model. Pract. Theory.

[51]  G. Vijayakumari,et al.  Exploring the Efficacy of Branch and Bound Strategy for Scheduling Workflows on Heterogeneous Computing Systems , 2016 .

[52]  Xiaorong Li,et al.  SABA: A security-aware and budget-aware workflow scheduling strategy in clouds , 2015, J. Parallel Distributed Comput..

[53]  Chengfeng Jian,et al.  A particle swarm optimisation algorithm for cloud-oriented workflow scheduling based on reliability , 2014, Int. J. Comput. Appl. Technol..

[54]  ABDUL RAUF BAIG,et al.  Review of Classification Using Genetic Programming , 2010 .

[55]  Saeed Parsa,et al.  RASA-A New Grid Task Scheduling Algorithm , 2009, J. Digit. Content Technol. its Appl..

[56]  Jun Zhang,et al.  An Ant Colony Optimization Approach to a Grid Workflow Scheduling Problem With Various QoS Requirements , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[57]  Junwei Cao,et al.  A Case Study on the Use of Workflow Technologies for Scientific Analysis: Gravitational Wave Data Analysis , 2007, Workflows for e-Science, Scientific Workflows for Grids.

[58]  Ken Kennedy,et al.  TaskScheduling Strategies forWorkflow-based Applications inGrids , 2005 .

[59]  Vidroha Debroy,et al.  Genetic Programming , 1998, Lecture Notes in Computer Science.

[60]  Ray Jain,et al.  The art of computer systems performance analysis - techniques for experimental design, measurement, simulation, and modeling , 1991, Wiley professional computing.